Compare commits

..

6 Commits

141 changed files with 9313 additions and 565 deletions

22
API.md
View File

@@ -6,6 +6,8 @@
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
v0.14 的 [完整开发指南](DEVELOPMENT.md) 说明响应根类型、`@request`、multipart 字段、注释/格式化、Go 错误目录导出与共享契约。
## 与 go-zero 的对应关系
调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式:统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写,不引入 go-zero 或 Gin 运行时依赖。
@@ -61,7 +63,7 @@ go-omni new --api user.api --module example.com/userService \
初次选项记录在 `.goOmni/apiManifest.json`,以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错,因为初始化代码属于业务方;可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。new 与 api new 统一生成契约项目,不再使用旧式 CRUD 目录。
MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
MySQL 使用官方 `database/mysql` 结构化适配器,密码从 `GO_OMNI_DB_PASSWORD` 覆盖,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
HTTP 请求类型不能推导数据库表结构;迁移和 Repository 由业务自己维护。当前目录的完整示例见 [managedService](examples/managedService/README.md)。MySQL、事务审计和鉴权实现可参考 [旧目录用户管理示例](examples/userService/README.md),接入时按 [项目指南](PROJECT.md) 装配到 config/svc/logic。
@@ -99,21 +101,21 @@ service user-api {
- 可省略的 `syntax = "v1"`;`info` 中的 title、version 和描述性元数据。
- 单独的 `type X { ... }` 与 `type (...)`;基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和未授权的目录越界。跨服务共享须在 `omni.imports.json` 声明 roots,见 [共享契约](DEVELOPMENT.md#两个服务共享契约)。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
- 同一 service 名称的多个块;`@server` 的 group、prefix、middleware、auth、errors、response,group 可以写 `admin/user`。
- 每个路由必须有 `@handler`;可选单行/多行 `@doc`、`@auth`、`@errors`、`@response`;GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。
- 可省略请求或响应;有响应默认 200,无响应默认 204。Omni 扩展 `@status 201`、`@status 202`;204 不允许响应类型,HEAD 不允许响应体,GET/HEAD 不允许 JSON 请求体。
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query,不解析 multipart 或表单 body。query 切片用重复键;header/path 使用标量。
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`,或声明式上传的 `multipart`、`multipartJSON`、`file`。`form` 始终表示 URL query;query 切片用重复键,header/path 使用标量。上传示例与限额见 [声明式 multipart](DEVELOPMENT.md#声明式-multipart)。
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、未通过 scalar 显式接入的自定义编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
业务错误、具名鉴权、多行文档及下述开发命令从 CLI `v0.7.0-preview.1` 提供。公共响应模板从 `v0.8.0-preview` 提供,CLI 与业务项目中的框架依赖需一起升级。
## 业务错误:声明、代码和文档共用一份定义
**错误目录也可以只在 Go 中维护。** 从 `v0.11.1-preview` 起,生成路由没有声明错误列表时,继承 `omni.WithErrors(catalog)` 注册的 App 目录;`.api` 继续定义接口和返回格式,无需重复登记错误。全局中间件提前返回同样适用。显式引用 `.api` 错误时才使用路由独立目录,规则见 [目录继承](errors/README.md)。CLI 不执行运行时 Go 配置,因此静态文档不会自动枚举只在 Go 中定义的具体错误。
**错误目录也可以只在 Go 中维护。** 从 `v0.11.1-preview` 起,生成路由没有声明错误列表时,继承 `omni.WithErrors(catalog)` 注册的 App 目录;`.api` 继续定义接口和返回格式,无需重复登记错误。全局中间件提前返回同样适用。显式引用 `.api` 错误时才使用路由独立目录,规则见 [目录继承](errors/README.md)。CLI 不执行项目 Go 配置;可显式用 `errors.ExportCatalog` 导出 `omni.errors.json`,供文档合并同一目录,见 [导出步骤](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
从 `v0.10.0-preview` 起,推荐用 `errorCatalog` 同时定义逻辑标识、对外业务码、提示和 HTTP 状态:
@@ -258,11 +260,11 @@ go test ./...
- 动态 JSON 字段 `any` / `interface{}`,包含切片和 map;外围契约仍严格。
- `scalar Name (...)` 显式引用项目类型,声明 JSON 输入/输出类型及文本绑定;支持已有 JSON/Text 转换方法。
- `@body (mode: json/raw/multipart; maxBytes: ...)`:按路由限制请求体。
- `@binding custom` 生成保留的绑定 hook;`@handlerMode manual` 生成保留的整个 Handler,仅用于自定义协议。raw/multipart 默认生成自动 Handler,将受限流交给 Logic。
- `@binding custom` 生成保留的绑定 hook;`@handlerMode manual` 生成保留的整个 Handler,仅用于自定义协议。raw 与未声明上传字段的 multipart 将受限流交给 Logic;声明了上传字段时则交给 Logic 已校验的 DTO。
- `@produces "text/plain"` 等声明无 DTO 成功输出;自动模式的 Logic 返回 `*httpx.Stream`,Handler 负责流式发送和关闭。默认状态 200。manual 模式保留项目自己的输出实现。
- 自动 multipart Handler 检查未读部分和结束边界;损坏/截断默认返回 400,超限为 413。公共状态、码和文案仍可用项目错误目录配置,见 [流式接入与错误处理](PROJECT.md#自动上传回调与下载)。
迁移扩展起始于 v0.9.0,原有手写扩展说明见 [迁移扩展指南](MIGRATION.md)。当前根目录入口和自动流式 Handler 要求 CLI 与框架依赖一起使用 v0.13.0-preview;版本说明和新目录维护规则见 [项目指南](PROJECT.md)。不兼容全部 goctl 语法。
迁移扩展起始于 v0.9.0,原有手写扩展说明见 [迁移扩展指南](MIGRATION.md)。当前根目录入口和自动流式 Handler 要求 CLI 与框架依赖一起使用 v0.14.0-preview;版本说明和新目录维护规则见 [项目指南](PROJECT.md)。不兼容全部 goctl 语法。
## 参数语义
@@ -270,9 +272,9 @@ go test ./...
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;普通非指针字段拒绝 null(动态 `any` 允许);`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
普通 JSON 请求体必须是一个 object,Content-Type 为 application/json;响应根类型另支持数组、nullable 对象和 null。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;非 nullable 集合请返回空集合而不是 nil。
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;嵌套的非 nullable 集合请返回空集合而不是 nil,根 `returns ([]Item)` 会把 nil slice 规范为 `[]`。
## 生成目录和业务边界
@@ -282,7 +284,7 @@ main.go # 启动、日志、优雅退出
etc/config.yaml
internal/config/
internal/svc/ # 依赖初始化和释放
internal/types/typesGen.go
internal/types/*Gen.go
internal/apierrors/errorsGen.go # 声明错误后生成
internal/handler/routesGen.go
internal/handler/openapiGen.go

26
BUSINESS_DEVELOPMENT.md Normal file
View File

@@ -0,0 +1,26 @@
# 通用业务开发能力完善计划
本轮目标:业务维护契约、配置、依赖装配与业务逻辑,框架持续维护 HTTP 接入、契约校验、通用运行期与文档。按实际生成项目和可删除的业务适配点验收,不写入 eTax 业务规则。
## 实施与验收清单
- [x] 运行期:资源登记、逆序释放、幂等关闭、失败回滚、故障原因保留归框架;生成入口与项目依赖初始化边界明确。多 App 日志隔离、标准 log.Logger 适配、可信请求元数据与依赖中间件示例。
- [x] 配置:项目完整结构体环境覆盖;默认值→文件→环境;嵌套、duration、slice、显式零值、空值策略、必填与脱敏;日志无副作用完整校验。
- [x] 数据库:官方 MySQL 结构化配置/TLS/驱动日志适配,继续复用 OpenConnector 和唯一连接池;独立启动检查、失败清理、生成器接入;不内置版本/表结构/业务错误策略。
- [x] 响应契约:数组、可空对象、成功 null、独立 HTTP 204;Logic/Handler/OpenAPI/漂移检查一致。
- [x] 请求契约:未知 query、JSON UTF-8/charset、无 Body/必填空对象/可选 Body,App 默认与路由覆盖;scalar/any/数组/大小限制一致,普通 JSON 只绑定一次。
- [x] 上传:声明文件/文本/JSON 字段,重复/未知/必填/单字段/单文件/总量限制;有界暂存与清理,调用 Logic 前完成普通表单校验;保留流式扩展与 Complete 约定。
- [x] 文档工具:字段注释、example/枚举描述进入 DTO 与 OpenAPI;保注释、幂等格式化及 check;显式 Go 错误目录元数据导出与文档合并,不自动执行项目代码。
- [x] 共享契约:受控 roots/摘要、循环/深度/文件数/路径保护;分源文件 DTO、多个独立 module 的工作区检查;版本提示与基础模板升级预览,保留业务文件。
- [x] 集成:实际生成项目验证新能力、连续生成、业务文件保留、失败清理、nethttp/native 请求与退出;现有错误继承/参数/上传下载回归。
- [x] 最终回归:全量 race(含生成器集成)、vet、build、格式化/链接检查;使用说明、迁移步骤及验收报告完整,发布后远程无 replace 验收。
已发布 `v0.14.0-preview`。逐项验收、可替换的业务适配点与远程报告见 [验收记录](reports/businessDevelopmentAcceptance.md);接入和升级见 [开发指南](DEVELOPMENT.md)。
## 核心约定
响应与错误码由项目配置。字段与请求策略由契约表达;基础设施采用可选适配器,保留标准 Connector。所有业务依赖在请求排空后关闭;框架不重试写操作、不自动迁移业务表、不关闭借用资源。
普通 multipart 为保证进入 Logic 前完成校验,以有界复制暂存到临时文件;框架清理自己创建的资源。纯流式路径明确由调用方在提交持久副作用前完成输入检查。
不修改 eTax/ePay 业务,不连接既有业务服务器。真实基础设施如需验证,只使用本轮隔离资源。验收报告明确区分单元测试、生成消费者测试与真实服务测试。

261
DEVELOPMENT.md Normal file
View File

@@ -0,0 +1,261 @@
# 通用业务开发指南(v0.15.0-preview)
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
## 返回数组、可空对象和 null
```text
@handler list
get /items returns ([]Item)
@handler find
get /item returns (*Item)
@handler save
post /item returns (null)
@handler remove
delete /item
```
| 契约 | Logic 返回 | 成功数据 |
| --- | --- | --- |
| `returns (Item)` | `(*types.Item, error)` | 对象;nil 被视为实现错误 |
| `returns ([]Item)` | `([]types.Item, error)` | 数组;nil 统一输出 `[]` |
| `returns (*Item)` | `(*types.Item, error)` | 对象或 `null` |
| `returns (null)` | `error` | HTTP 200 和空业务数据 |
| 不写 returns | `error` | HTTP 204,无响应体 |
统一响应中的空数据是否显示,仍遵循项目模板的 `dataMode`;需要 `data:null` 就设置 `dataMode: always`。`@response raw` 输出业务数据本身,包括 `[]`、对象或 `null`。可用 `@status 201/202`;204 不能声明响应类型。根类型当前支持以上四种形式,嵌套字段继续支持既有 slice/map/pointer/scalar/any。
## 契约驱动的请求策略
入口文件可以声明默认策略,单个接口按需覆盖:
```text
@app (
defaultResponse: CommonResponse
request: {"unknownQuery":"reject", "utf8":"reject", "charset":"utf8"}
)
@handler create
@request (body: required)
post /items (CreateReq) returns (Item)
@handler touch
@request (body: emptyObject)
post /touch returns (null)
```
- `unknownQuery: reject|ignore`:是否拒绝未声明的 URL 查询字段;重复标量仍拒绝,数组用重复的同名 query。
- `utf8: reject|replace`:拒绝原始非法 UTF-8,或采用 JSON 的替换字符行为。不是只检查解码后的字符串。
- `charset: utf8|any`:`utf8` 仅接受未声明 charset 或 `utf-8`;不会对其他编码做转码。
- `body: auto|required|optional|none|emptyObject`:auto 按 JSON 字段推断;required 要求一个对象;optional 允许完全没有 body,但提供 body 后仍完整校验;none 禁止任何 body 字节;emptyObject 必须是 `{}`。
未设置的项继承 App 的 `config.Config.Request`,最后采用既有默认行为。`.api` 的 `@app` 显式字段优先于项目配置;路由 `@request` 优先于 App。raw/multipart 使用各自解析流程,不能套 JSON body 策略。普通 JSON 只解析一次;重复/未知 JSON 字段、约束、scalar 和 `any` 仍使用同一编译器。自定义 binder/manual Handler 的输入逻辑由项目显式负责。
没有 body 与提供 `{}` 不等价。optional 无 body 时 JSON 字段保留零值;需要业务区分是否出现某字段时使用 optional 指针字段。raw/multipart 不继承 App 的 JSON body 策略;GET/HEAD 如遇 App 的 required/optional/emptyObject 默认值,须明确覆盖为 auto 或 none。参数校验不会替代企业权限、记账规则或数据库约束。
## 声明式 multipart
```text
type UploadReq {
Title string `multipart:"title" binding:"required,max=80"`
Metadata AttachmentMeta `multipartJSON:"metadata,maxBytes=16384"`
Attachment File `file:"attachment,maxBytes=10485760"`
Extras []File `file:"extras,optional,maxFiles=3,maxBytes=1048576"`
}
type AttachmentMeta {
Category string `json:"category"`
}
@handler upload
@body (mode: multipart; maxBytes: 16777216; maxParts: 8; maxFieldBytes: 65536)
post /attachments (UploadReq) returns (AttachmentResp)
```
Logic 直接得到 `*types.UploadReq`,无需维护 Handler。`form` 仍表示 URL query;`multipart` 表示文本表单字段,可用基础类型、文本 scalar 和重复 scalar 数组;`multipartJSON` 是一个严格解码的 JSON 表单字段,可包含对象、数组和已声明的 scalar。
`File` 提供 `Name`、`ContentType`、`Size`、`Present()` 和 `Open() (io.ReadCloser, error)`。文件名和媒体类型属于客户端元数据,项目继续决定内容检测、存储权限和目标对象名。不要把客户端文件名直接用作存储路径。
```go
reader, err := req.Attachment.Open()
if err != nil { return nil, err }
defer reader.Close()
// 将 reader 传给项目选定的流式存储接口;不要先 io.ReadAll 整个大文件。
```
所有声明字段、重复/未知字段、文件数量和各级限额都会在进入 Logic 前校验。缺少必填项、重复标量/JSON 字段、文件与文本类型混用、畸形 JSON 均失败。JSON 的类型、未知字段和重复 key 检查与普通 JSON 一致。
默认总量 1 MiB、最多 128 个 part、单文本/JSON 字段 64 KiB;单文件默认受总量约束,文件数组默认最多 16 个。`@body` 可设置 `maxBytes/maxParts/maxFieldBytes/maxFileBytes`;字段上的 `maxBytes/maxFiles` 再收紧对应限额。总量包括 MIME 头和结尾数据;传输编码的 part 被拒绝,避免字节含义不明确。
文件使用有界复制写入权限为 0600 的临时文件,文本/JSON 字段在各自限额内缓冲。生成 Handler 返回或 panic 时关闭打开的文件 reader 并删除临时文件;不能把请求的 File 留到异步任务使用。失败不会交付半个 DTO,也不会进入 Logic。临时磁盘容量、并发请求上限与 HTTP 超时由部署配置共同约束。
不声明 multipart 字段时,保留原来的 `body *httpx.Body` 流式入口,不创建临时文件。该方式适合特殊协议;项目应在不可撤销副作用之前完成校验并调用 `body.Complete()`。下载和原始回调继续使用 `@produces` / `@body(mode: raw)`。
## 完整业务配置的环境覆盖
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML 文件 → 全结构体环境覆盖 → 既有框架环境别名 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
```go
type ClientConfig struct {
Endpoint string `json:"endpoint" env:"CLIENT_ENDPOINT,required"`
Token string `json:"-" env:"CLIENT_TOKEN,required,secret,empty=reject"`
Timeout time.Duration `json:"timeout" envDefault:"3s"`
Retries []int `json:"retries"`
Enabled bool `json:"enabled"`
}
```
把字段加入项目 `Config` 后,默认环境名取 `GO_OMNI_` 加 JSON 路径的 UPPER_SNAKE_CASE,如 `GO_OMNI_CLIENT_TIMEOUT`。显式 env 名是完整名称,可以使用自己的业务前缀。`env:"-"` 排除整个子树;`json:"-"` 字段只有显式 env 标签才参与。
duration 使用 `250ms`、`3s`;slice/map 使用 JSON,如 `[1,3,5]`、`{"retry":"2s"}`。支持嵌套、零/false、空数组、自定义 TextUnmarshaler。空环境值默认允许;可选择 `empty=ignore` 或 `empty=reject`。required 要求字符串非空、指针/集合非 nil;0 和 false 可以是合法业务值,范围由项目 Validate 决定。
手写项目可使用 `config.LoadProject(path, defaults, config.EnvOptions{Prefix:"MY_APP_"})`,或分别调用 `ApplyDefaults/DecodeFile/OverlayEnv`。读取失败不修改输入默认值;诊断不会打印输入值。不要主动打印含密钥的整个配置,或把底层解析错误展开到公开响应。Validate 不打开日志文件、数据库或存储连接;证书的可读性和握手信任在依赖初始化阶段验证。
## 依赖、日志与中间件
`internal/svc/runtimeGen.go` 持续由生成器维护。项目的 ServiceContext 嵌入 `*lifecycle.Runtime`,只在 `Init(ctx)` 中装配额外依赖:
```go
func (c *ServiceContext) Init(ctx context.Context) error {
client, err := NewClient(c.Config.Client)
if err != nil { return err }
c.Client = client
c.OnClose(client.Close)
c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(client)
return nil
}
```
每个资源初始化成功就登记清理;后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求,再执行后台组件的 Stop/Drain 阶段,确认收尾后逆序关闭依赖,最后关闭日志。关闭幂等,单个清理函数 panic/返回错误不会跳过其他清理;初始化 panic 清理后继续传播。后台组件按 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭) 登记;无法确认收尾完成时保留共享依赖并异常退出,不能把取消等同于任务已经结束。
中间件用业务所需的小接口接收依赖,避免反向导入 svc:
```go
// internal/middleware/accessFactory.go
package middleware
import (
"context"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
type AccessChecker interface {
CheckAccess(context.Context, string) error
}
func NewRequireAccess(checker AccessChecker) httpx.Middleware {
return func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
return err
}
return next(c)
}
}
}
```
`.api` 声明对应 middleware 后,保留生成的初始 hook,并另建上述工厂;在 Init 用工厂替换 registry 的同名项即可。该包只依赖接口和 HTTP 抽象,不导入 svc,也不把数据库藏到 context 里。
生成入口为每个 App 注入独立 Logger,不修改进程默认 Logger。业务可用 `c.Logger.InfoContext(...)`,或请求链中的 `logx.InfoContext(ctx,...)`。标准库/第三方需要 `*log.Logger` 时使用 `c.Logger.Standard(slog.LevelWarn)`;适配器借用原 Logger,不负责关闭。手写多 App 用 `omni.WithLogInstance(logger)` 显式注入。
`httpx.Metadata(ctx)` 在业务的标准 context 中提供可信代理规则解析后的 ClientIP、HTTP Method 和路由模板;不放入凭据、完整 URL、查询或 body。业务 userId/tenantId 等仍由项目选择,框架不预设身份模型。
`middleware.ContextErrors("REQUEST_CANCELED", "REQUEST_TIMEOUT")` 可将原始取消/超时错误映射到项目错误目录;显式 APIError 优先,内部 cause 链保留。状态、业务码与提示仍在项目 Catalog 配置。
## 字段文档、格式化、Go 错误目录导出
```text
type Item {
// 企业名称
Name string `json:"name" example:"示例企业"`
State int `json:"state,options=0|1" example:"1" enumDescriptions:"[\"禁用\",\"启用\"]"` // 状态
}
```
字段前/同行注释同步到 Go 注释和 OpenAPI description。显式 `description` 标签优先;example 的基础字段值也检查类型和约束,枚举说明按声明值顺序填写。字符串 example 直接写内容,其他类型写 JSON。自定义 scalar 的示例不能替代其项目解码器验证;输入输出类型不同时,不把输入示例冒充输出示例。
```sh
go-omni api format --api api/service.api
go-omni api format --api api/service.api --check
go-omni api generate
go-omni api generate --check
```
格式化保留注释、token 和多行字面量,规范四空格缩进、空行、行首尾空白及相邻单行字段的类型/标签/行尾注释对齐;结果幂等。先校验完整契约,check 不写文件;导入的共享目录不被某个服务的格式化命令修改。DTO 按源 `.api` 输出 `dto<Source><摘要>Gen.go`,都属于 `types` 包,跨文件引用无需改包名。
Go Catalog 是错误的唯一来源时,显式导出文档元数据:
```go
// 在项目自己的导出命令中;appErrors.Catalog 与 AppOptions 注册同一函数。
if err := omnierrors.ExportCatalog(os.Stdout, appErrors.Catalog()); err != nil {
log.Fatal(err)
}
```
```sh
go run ./cmd/exportErrors > api/omni.errors.json
go-omni api generate
```
文件是派生产物,应纳入版本控制并由业务 CI 检查导出漂移。CLI 自动读取入口目录的 `omni.errors.json`,校验错误目录、合并状态/模板参数/具体示例到 OpenAPI;不会运行项目 Go,不创建另一份运行期注册器。带独立错误目录的路由继续使用自己的目录。没有路由级精确错误列表时,文档描述 App 目录的公共错误集合;特殊自定义 Encoder 的文档由项目显式扩展。
## 两个服务共享契约
```text
workspace/
├── contracts/common/response.api
├── services/api/api/service.api
├── services/api/api/omni.imports.json
├── services/id/api/service.api
├── services/id/api/omni.imports.json
└── omni.workspace.json
```
两个服务的 `omni.imports.json` 可以分别声明:
```json
{"roots":{"common":{"path":"../../../contracts/common","version":"1.0.0"}}}
```
```text
import "@common/response.api"
```
只读取入口目录和显式声明的根。保留真实路径检查、symlink 越界拒绝、循环检测、32 层、64 文件、单文件 1 MiB/总量 8 MiB 限额;最多 16 个共享根。不会下载依赖或复制共享定义进服务。移动目录时显式更新 roots。
`version` 是项目管理的版本标签。生成文档记录每个文件 SHA-256,以及每个共享根所导入文件集合的摘要;可把文档中的 `x-omni-contract-sources.shared.common.sha256` 填入 root 的 `sha256`,固定输入集合,内容变化后必须显式更新。摘要算法是对“该根内相对路径→文件 SHA-256”的键排序 JSON 对象再做 SHA-256,未导入的文件不参与。
工作区清单:
```json
{"services":[{"dir":"services/api"},{"dir":"services/id"}]}
```
```sh
go-omni workspace format --file omni.workspace.json
go-omni workspace generate --file omni.workspace.json
go-omni workspace check --file omni.workspace.json
```
各服务保持独立 module;可显式填写各服务的 `api`(相对服务目录)和 `module`。所有服务预检通过后再写;每个服务独立提交文件事务,不承诺跨项目的崩溃原子性。check 汇总格式与生成漂移;业务编译/测试仍在各 module 执行。
## v0.13 项目升级
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.15.0-preview(或显式引用当前源码)。
```sh
go-omni api upgrade --dir . --dry-run
go-omni api upgrade --dir . --preview .goOmni/upgradePreview
# 只有标为 safe 的原始模板才会自动替换;review 阻止全部写入。
go-omni api upgrade --dir . --apply
go mod tidy
go-omni api generate --check
go test ./...
go build .
```
自动迁移只替换与 v0.13 原始模板一致的配置/svc 基础文件、移除原始 close.go,并更新最低依赖及生成产物;保留 Logic、Repository、AppOptions、独立命令。迁移失败回滚已改的项目文件。存在业务修改时,仅输出这些基础文件的候选版本和清单,不生成整棵业务代码让人重新拼接。
定制项目按预览定点迁移:保留依赖字段,把初始化移到 Init,嵌入 Runtime,清理改为 OnClose 登记;移除旧 New/Close/OnClose;合并业务配置和结构化 MySQL 配置,更新 go.mod,再正常 generate。生成器会拒绝遗留构造/关闭实现,避免重复函数或悄悄绕过清理。v0.12 及更早的定制结构不能被精确自动合并,使用同一预览逐项迁移并编译验证。

View File

@@ -1,6 +1,6 @@
# 生成项目与文件维护边界
v0.13.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
v0.15.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
## 创建和日常开发
@@ -47,7 +47,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
| `internal/config/config.go` | 自定义配置字段、加载、默认值与校验 |
| `internal/svc/serviceContext.go` | 依赖字段和初始化,持有加载后的 Config |
| `internal/svc/options.go` | AppOptions、GlobalMiddleware;注册 Go 错误目录和全局扩展 |
| `internal/svc/close.go` | 资源关闭;OnClose 为自定义资源提供标准清理入口 |
| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 |
| `internal/svc/storage.go` | 选用存储时生成,之后保留 |
| `internal/logic/<group>/*.go` | 业务方法,使用标准 context.Context |
| `internal/repository/<group>/*.go` | 数据访问,按需实现 |
@@ -58,7 +58,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。
生成器持续管理:`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。
@@ -66,16 +66,16 @@ Logic 实例在注册时创建并由请求并发复用;请求数据用方法
## 依赖与启动扩展
`svc.New(ctx, logger, cfg)` 初始化依赖,可返回错误;Logger 由入口创建并借用。新增客户端时只需要扩展配置和 svc:
`svc.New(ctx, logger, cfg)` 初始化依赖,可返回错误;Logger 由入口创建并借用。New 本身由生成器维护;新增客户端时扩展配置和 svc 的 `Init(ctx) error`:
```go
client, err := newClient(cfg.Client)
if err != nil { return nil, err }
client, err := newClient(c.Config.Client)
if err != nil { return err }
c.Client = client
c.OnClose(client.Close) // 创建成功后立即登记,后续失败也能释放
```
`New` 的失败清理骨架已生成。OnClose 按登记的相反顺序执行;Close 幂等。项目不要重复关闭同一资源,借用的 Logger 也不应由 svc 关闭。构造器在返回错误前若创建了尚未交付的资源,该构造器自己负责释放。
ServiceContext 嵌入 `*lifecycle.Runtime`;框架维护锁、清理栈和失败回滚,项目不再复制 Close/OnClose 实现。OnClose 按登记的相反顺序执行;Close 幂等。项目不要重复关闭同一资源,借用的 Logger 也不应由 svc 关闭。构造器在返回错误前若创建了尚未交付的资源,该构造器自己负责释放。
错误注册写在 `internal/svc/options.go`,错误表可以来自任意项目共用包:
@@ -85,7 +85,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
}
```
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。静态 OpenAPI 不执行 Go,不能自动枚举仅存在于 Go 中的错误目录。
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。静态 OpenAPI 不执行 Go;可从同一 Catalog 显式导出 `api/omni.errors.json`,详见 [元数据导出](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
`GlobalMiddleware()` 提供全局中间件;`Middlewares` 映射提供 `.api` 声明的具名中间件。全部显式调用,不依赖项目 init 注册,也不要求 svc 导入 handler。框架请求链按原有次序运行。
@@ -93,7 +93,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
最终配置校验包含自定义响应编码器与错误目录的组合。码类型/映射冲突、编码器已绑定另一份目录等问题在创建 App 前返回错误,进入入口的清理流程,不等到构造 App 时才 panic。
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后按框架规则强制关闭连接。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,在 OnClose 中取消。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后按框架规则强制关闭连接。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,使用 WorkContext 并登记成对的 OnStop/OnDrain,详见 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭)。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
## 依赖主动申请退出
@@ -106,7 +106,7 @@ c.RequestShutdown(err)
这只是通知,立即返回,不等待 HTTP 排空,也不直接关闭数据库。生成入口停止接收新请求,等待在途请求完成;超过 `shutdownTimeout` 后取消请求并关闭连接,再执行 svc.Close,最后关闭日志。非空故障原因保留在返回错误链中,进程退出码为 1;`RequestShutdown(nil)` 或正常 SIGINT/SIGTERM 在没有其他故障时退出码为 0。
多个依赖可并发通知,关闭流程仅执行一次,首个非空故障原因保留。正常信号已开始排空时收到的故障也不会被清理动作覆盖。初始化期间的通知同样有效,不依赖 HTTP 监听已经启动。业务应在 Close/OnClose 中停止并等待自己启动的监控任务,再结束资源清理;关闭完成以后才报告的故障无法参与已经结束的进程结果。
多个依赖可并发通知,关闭流程仅执行一次,首个非空故障原因保留。正常信号已开始排空时收到的故障也不会被清理动作覆盖。初始化期间的通知同样有效,不依赖 HTTP 监听已经启动。后台监控任务应通过 OnStop/OnDrain 停止并等待,再进入资源清理;关闭完成以后才报告的故障无法参与已经结束的进程结果。
`Lifecycle *lifecycle.Controller` 是 svc 持有的通用控制器,初始化时创建,生成入口使用它的 Context。该 Context 表示服务正在停止,不代表数据库已经关闭。不能把它当作在途请求的 Context,也不要在它取消时立刻关闭请求仍依赖的资源。发号器失去租约后拒绝发号等业务安全规则仍由项目实现。
@@ -140,16 +140,11 @@ server:
关闭或改名后,旧路径不再由框架保留,项目可以在 `.api` 自己声明。启用的内置 GET 路由与业务路由相同或被业务参数/通配路径覆盖时,在创建 App、开始监听之前报错;不会覆盖业务路由。内置接口仍经过全局中间件;项目在鉴权中间件中明确决定这些路径的访问规则,框架不自动绕过鉴权。
## 从 v0.12 生成项目升级
<a id="从-v012-生成项目升级"></a>
manifest v2 和现有目录保留。升级同版 CLI 与框架依赖后执行 `api generate`,生成器自动更新 main 和 Handler;`svc`、配置与业务文件仍按字节保留,所以需要在项目拥有的 svc 文件补入以下扩展:
## 从 v0.13 或更早版本升级
1. `serviceContext.go` 导入 `git.yuntuo.hk/k2safe/go-omni/lifecycle`,给 ServiceContext 增加 `Lifecycle *lifecycle.Controller`;创建 c 时用 `lifecycle.New(ctx)` 初始化,并在后续依赖初始化前设置 `ctx = c.Lifecycle.Context()`。
2. 增加 `func (c *ServiceContext) RequestShutdown(err error) { c.Lifecycle.RequestShutdown(err) }`。
3. 构造失败的 defer 在 `c.Close()` 后保留 `c.Lifecycle.Err()`;Close 的 `closeOnce.Do` 开头调用 `c.Lifecycle.RequestShutdown(nil)`,重复关闭仍由已有 once 保护。
4. 配置默认值若来自 `config.Default()`,自动具有默认端点配置;如需禁用或改路径,在 YAML 或项目 Config 中显式设置。手动拼 Config 的项目需要自行填写要启用的端点。
精确实现可对照 [示例 svc](examples/managedService/internal/svc/serviceContext.go) 和 [关闭逻辑](examples/managedService/internal/svc/close.go),也可在临时目录用新版 CLI 生成参考。不要覆盖原有数据库/客户端初始化;不需要手改 main、Handler 或引入另一套启动器。最后执行 `api generate --check`、`go test ./...`、`go build .`,并验证故障退出。
请按 [v0.14 定点迁移说明](DEVELOPMENT.md#v013-项目升级) 使用 `api upgrade`。原始模板可以自动升级;含业务修改的基础文件必须参考局部预览合并。旧 New/Close/OnClose 移到框架 Runtime,项目依赖初始化保留在 Init。不要继续向项目里复制生命周期锁和清理栈。
## App 默认响应格式
@@ -193,7 +188,7 @@ post /callback
get /download (DownloadReq)
```
- raw/multipart 的 Logic 增加 `body *httpx.Body` 参数。`body.Raw` 是受路由上限约束的原始字节流;`body.Header` 是请求头副本。
- 以下为未声明 multipart 字段的流式用法;声明式绑定见 [上传字段](DEVELOPMENT.md#声明式-multipart)。raw/流式 multipart 的 Logic 增加 `body *httpx.Body` 参数。`body.Raw` 是受路由上限约束的原始字节流;`body.Header` 是请求头副本。
- multipart 用 `body.Parts.NextPart()` 顺序读取,关闭每个 Part;不调用 ParseMultipartForm,不产生上传临时文件,也不把整个文件放入内存。每个字段/文件的业务要求由 Logic 校验。请求 DTO 可定义 path/form/header;`form` 仍是 URL 查询参数,不能用 json DTO 重复读取上传体。
- Parts 是 `*httpx.MultipartReader`,返回的 `*httpx.MultipartPart` 提供 Read、Close、Header、FormName、FileName;NextRawPart 可读取未做 quoted-printable 解码的部分。NextPart 自动关闭上一个未读完的 Part;Close 会报告读取错误。不要混用 Parts 与 Raw 读取同一个 multipart 请求。
- multipart 缺少结束边界、头部/内容损坏默认返回 `400 / INVALID_MULTIPART`;超过路由限额仍为 413,媒体类型/缺少 boundary 为 415。框架只提供通用标识和默认提示,项目可以在 Go 错误目录中配置自己的状态、业务码和文案。内部解析原因保留在错误链,不进入公开响应;Logic 自己返回的数据库等错误不会被统一改成 400。
@@ -213,7 +208,7 @@ get /download (DownloadReq)
## 已有服务接入步骤
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.13.0-preview:
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.15.0-preview:
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。

View File

@@ -2,12 +2,18 @@
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
**从这份 README 开始接入。** 当前版本 **`v0.13.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**从这份 README 开始接入。** 当前版本 **`v0.15.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。
新增通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
后台组件可在 svc 登记 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭),先停止接收工作并等待收尾,再关闭共享资源;无需绑定队列或 ORM。
使用 GORM 的项目可以选择 [gormx 适配器](database/gormx/README.md),复用数据库初始化和连接池,并将 SQL 日志接入 logx。
## 选择接入方式
| 你的项目 | 从哪里开始 |
@@ -28,7 +34,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
```sh
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.13.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.15.0-preview
export PATH="$(go env GOPATH)/bin:$PATH"
go-omni --help
```
@@ -72,7 +78,7 @@ hello/
├── internal/
│ ├── config/ # 配置读取
│ ├── svc/ # 数据库、日志、外部客户端等依赖
│ ├── types/typesGen.go # 生成的请求与业务返回类型
│ ├── types/*Gen.go # 按 .api 文件生成 DTO
│ ├── handler/ # 生成的绑定、响应策略、路由及文档
│ ├── logic/greet/ # 在这里实现业务逻辑
│ ├── repository/greet/ # 在这里实现数据访问
@@ -175,7 +181,7 @@ go-omni new --api catalog.api \
--log-output file --log-format json ./catalogFull
```
该命令在 `catalog.api` 所在目录执行,目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护,API 模板不自动建表;MySQL DSN 通过 `GO_OMNI_DB_DSN` 提供。存储还可选 `s3`、`r2`、`oss`,按生成项目的配置说明提供桶、端点和凭据。完整生成项目示例见 [managedService](examples/managedService/README.md)。
该命令在 `catalog.api` 所在目录执行,目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护,API 模板不自动建表;MySQL 使用结构化配置,密码通过 `GO_OMNI_DB_PASSWORD` 提供。存储还可选 `s3`、`r2`、`oss`,按生成项目的配置说明提供桶、端点和凭据。完整生成项目示例见 [managedService](examples/managedService/README.md)。
`go-omni new` 与 `go-omni api new` 统一生成契约项目。需要手写项目时,可直接引用框架包;生成器不再提供另一套 CRUD 启动目录。
@@ -184,7 +190,7 @@ go-omni new --api catalog.api \
在业务项目的 **go.mod 所在目录**执行:
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.13.0-preview
go get git.yuntuo.hk/k2safe/go-omni@v0.15.0-preview
```
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
@@ -382,7 +388,7 @@ return apierrors.TextTooLong("企业名称", 100)
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
已有项目使用 `driver.Connector` 时,直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 等驱动的独立 TLS、Logger、时区与拨号参数由项目构造 Connector;框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN,也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。
已有项目使用 `driver.Connector` 时,直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 可直接用 [database/mysql](database/mysql/README.md) 配置独立 TLS、Logger、时区与拨号参数;任意驱动也可由项目构造 Connector。框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN,也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。
| 能力 | 包与接入说明 |
| --- | --- |
@@ -409,7 +415,7 @@ API 项目的数据库迁移需要单独维护并显式执行;启动不会自
| `GO_OMNI_SHUTDOWN_TIMEOUT` | 优雅退出等待时间,默认 `10s` |
| `GO_OMNI_ENABLE_LOGGER` / `GO_OMNI_LOG_LEVEL` | 访问日志开关、日志级别 |
| `GO_OMNI_TRUSTED_PROXIES` | 可解析转发地址头的可信代理 IP/CIDR,逗号分隔 |
| `GO_OMNI_DB_DSN` | 选择 MySQL 初始化时的数据库连接配置 |
| `GO_OMNI_DB_HOST/USER/PASSWORD/DATABASE` | 官方 MySQL 适配器的结构化连接配置 |
例如在生成项目中切换 native:
@@ -454,7 +460,7 @@ go test ./...
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
CLI 与业务 module 固定使用 `v0.13.0-preview`。新布局不兼容旧 manifest;旧试验项目需在新目录生成并搬入业务文件,不能直接覆盖手写入口。新布局内部升级再执行上述生成与测试步骤,手写框架项目只需依赖升级及业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
CLI 与业务 module 固定使用 `v0.15.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
## 按需深入

View File

@@ -1,4 +1,77 @@
# v0.13.0-preview
# v0.15.0-preview
增加通用后台分阶段关闭:生成入口先完成 HTTP 排空,再运行全部 OnStop、全部 OnDrain,确认组件收尾后逆序释放 OnClose 资源,最后关闭日志。适用于任意后台组件;不引入 Orbit,不改变业务事务或任务一致性策略。
- svc 登记成对、同名的 OnStop/OnDrain,Context 用于早期退出通知,WorkContext 供已接受工作在收尾期间使用。同一阶段并发执行,存在内部先后依赖的组件组合登记。
- server.lifecycle 配置 stopTimeout、drainTimeout、cancelTimeout。阶段超时取消工作并给未返回钩子收尾窗口;错误与阶段/组件名称可通过错误链追踪。
- 不能确认后台已经结束时,返回 ErrResourcesRetained,保留共享资源和借用日志,生成进程非零退出。未返回的 Stop 不会与自己的 Drain 重叠;错误、panic 不阻止其他组件尝试收尾。
- task.Runner 增加 StopAccepting/Wait,原 Shutdown/Close 继续可用。核心 lifecycle 不依赖 task 或其他适配器。
- 初始化和监听失败复用同一流程;重复关闭只执行一次。生成器继续保留配置、svc 和业务代码。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.15.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.15.0-preview
go-omni api generate
go-omni api generate --check
go test ./...
```
v0.14.x 无需重写 svc/config;生成器更新 main/runtimeGen,未配置时使用默认时限。也可 api upgrade --dry-run/--apply 更新依赖与生成文件,保留已有项目扩展。需要阶段保证的旧后台 OnClose 登记,应显式迁到 Stop/Drain;v0.13 及更早版本仍按原迁移指南处理。
全量 race(含实际生成项目)、vet、build 和隔离 MySQL/GORM 验收已通过。详见 [生命周期接入](lifecycle/README.md#后台组件分阶段关闭) 与 [验收记录](reports/phasedLifecycleAcceptance.md)。Go 无法强制结束任意 goroutine;OnClose 仍须能返回,HTTP 处理器仍须响应取消。此版不会将未知的第三方组件自动改造成优雅关闭组件。
---
# v0.14.1-preview(历史版本)
增加可选 `database/gormx`:GORM 复用 go-omni 已初始化的标准池、TLS、限额、健康检查和关闭流程,业务继续使用原生 GORM 模型与事务。核心 App/数据库包的编译依赖链不引入 GORM。
在 svc.Init 中调用 `gormx.MySQL(ctx, c.DB, gormx.WithLogger(c.Logger))`;通用 Open 工厂保留其他方言入口。适配器保护借用池不被初始化失败路径关闭,并为初始化查询设置时限。查询使用 WithContext,SQL 日志接入 logx,请求标识、慢查询和错误分类贯通;默认不输出绑定参数与驱动完整错误文本。
v0.14.0 项目更新 module 与 CLI 即可,不改生成目录或 main。模板最低运行时保持 v0.14.0;修正新版 CLI 把最低兼容版本误当自身版本、导致同版消费者出现版本提示的问题。其他数据库方言和管理额外池的插件未纳入本次实库验收,原生 PrepareStmt 的生命周期限制见 [接入说明](database/gormx/README.md)。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
```
全量 race(含生成消费者)、vet、build,以及隔离 MySQL 的事务/取消/日志/生成项目接入通过。发布后的空缓存远程 MySQL/GORM 和生成项目验收均通过,无本地 replace,CLI 与框架使用同版。标签固定在 `b8ab6b35219d53d6de392c920fe7ecbe9a76dade`,详见 [GORM 验收](reports/gormAcceptance.md)。
---
# v0.14.0-preview(历史版本)
本版补齐契约驱动的业务开发与通用基础设施。项目继续维护接口、配置、依赖装配、Logic、SQL 和业务规则;生成器维护入口、请求绑定、路由、响应与文档。
## 安装与升级
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.0-preview
go-omni new --module example.com/billing ./billing
```
新生成器与运行时一起升级。已有 v0.13 项目先运行 `go-omni api upgrade --dry-run`,再按 [定点迁移步骤](DEVELOPMENT.md#v013-项目升级) 处理;不会静默覆盖自定义 svc/config。运行期的新构造器由生成器维护,项目初始化迁入 `svc.Init`,使用 `OnClose` 登记资源。
## 本版能力
- `.api` 支持根数组、可空对象、成功 null;真正 204 独立保留。生成 Logic、Handler、OpenAPI 和漂移检查使用相同语义。
- 请求契约控制未知 query、原始 UTF-8、charset、必须/可选/禁止 body 和严格空对象。路由覆盖 App 策略,普通 JSON 只解析一次。
- multipart 声明文件、文本和 JSON 表单字段。Logic 收到校验后的 DTO;文件有界暂存、流式读取、请求结束清理,保留底层 raw/multipart/下载扩展。
- 完整项目配置支持环境覆盖、显式零值、空值策略、必填、嵌套及复杂类型。独立校验先于日志和连接创建。
- `lifecycle.Runtime` 统一逆序清理、幂等关闭、失败回滚及故障保留;App Logger 独立,支持标准 `log.Logger` 与可信请求元数据。提供依赖中间件装配示例和可选 context 错误映射。
- 官方 `database/mysql` 适配器提供结构化配置、每连接池独立 TLS/mTLS、驱动日志、启动检查与失败清理。继续返回同一 `database.DB`,不自动迁移表或重试业务事务。
- 字段注释、示例和枚举说明进入 DTO/OpenAPI;`api format` 幂等格式化。Go 错误目录可显式导出供文档使用,无需复制错误定义。
- 显式共享契约 roots、来源摘要、分文件 DTO、多服务 workspace 命令、基础模板升级预览;保留目录边界及业务文件所有权。
详见 [业务开发指南](DEVELOPMENT.md)、[数据库接入](database/mysql/README.md) 和 [验收记录](reports/businessDevelopmentAcceptance.md)。
全量 race(含生成项目)、vet、build 与本地进程验收通过。发布后的空缓存远程消费者和隔离 MySQL 验收也均通过;CLI/框架同版且无本地 replace。标签固定在 `f5478cdc7662704b164637649305361e52f7c598`,报告单独提交。
仍需业务维护:契约、配置、svc 依赖创建/探针/清理登记、错误目录、Logic、Repository 和权限中间件。特殊协议可使用保留的扩展点。声明式上传的暂存磁盘和并发量需配置容量;共享契约版本标签由项目管理,框架校验摘要但不下载依赖。此版仍为 preview,不代表 eTax/ePay 已完成业务迁移。
---
# v0.13.0-preview(历史版本)
本版补齐项目依赖主动退出与内置接口配置。ID 租约、数据库或后台任务等故障由项目判断,框架通过统一生命周期完成退出;不内置业务规则。
@@ -33,7 +106,7 @@ python3 scripts/verifyManagedProject.py --report performance/lifecycleEndpointsL
python3 scripts/verifyManagedRelease.py --version v0.13.0-preview --report performance/lifecycleEndpointsRemote.json
```
远程验收在发布后执行,报告单独提交,不改写发布标签。此版仍为 preview;具体业务的租约失效保护、鉴权与事务回归由接入项目验证。资源清理顺序不代表能强行结束忽略 context 取消的业务 goroutine;项目 Close 必须停止并等待自己启动的后台任务。
发布后的远程空缓存验收已通过,耗时 26.55 秒;[机器报告](performance/lifecycleEndpointsRemote.json) 确认 CLI 和生成项目使用同版远程 module,无本地 replace。报告单独提交,发布标签保持在代码提交 `f0991d3ff248e4cb32812c368f0f8ba28740a6c7`。此版仍为 preview;具体业务的租约失效保护、鉴权与事务回归由接入项目验证。资源清理顺序不代表能强行结束忽略 context 取消的业务 goroutine;项目 Close 必须停止并等待自己启动的后台任务。
---

View File

@@ -1,5 +1,7 @@
# GoOmni 成熟度计划
当前进展:`v0.14.0-preview` 已完成契约驱动的业务开发能力、通用 Runtime/配置和官方 MySQL 适配器,并通过全量 race、实际生成服务及空缓存远程验收。新项目/升级以 [README](README.md) 和 [开发指南](DEVELOPMENT.md) 为准;下方为历史阶段记录,完整本轮验收见 [报告](reports/businessDevelopmentAcceptance.md)。具体 eTax/ePay 业务迁移与业务回归仍由接入项目完成。
## 1. 新项目启动与发布流程(已完成,v0.1.1)
- 修正脚手架固定版本引用,保留显式本地开发选项。

58
app/logIsolation_test.go Normal file
View File

@@ -0,0 +1,58 @@
package app_test
import (
"bytes"
"context"
"net/http/httptest"
"strings"
"testing"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
func TestAppLogAndRequestMetadataIsolation(t *testing.T) {
var a, b bytes.Buffer
la, _ := logx.New(logx.Config{Writer: &a})
lb, _ := logx.New(logx.Config{Writer: &b})
apps := []*omni.App{
omni.New(omni.WithLogger(true), omni.WithLogInstance(la), omni.WithTrustedProxies("127.0.0.1/32")),
omni.New(omni.WithLogger(true), omni.WithLogInstance(lb)),
}
for i, app := range apps {
app.GET("/probe/:id", func(c *httpx.Context) error {
ctx := c.Context()
meta := httpx.Metadata(ctx)
expected := "127.0.0.1"
if i == 0 {
expected = "203.0.113.7"
}
if meta.ClientIP != expected || meta.Route != "/probe/:id" || meta.Method != "GET" {
t.Error(meta)
}
logx.InfoContext(ctx, "business", "app", i)
clone := c.Clone(context.WithValue(ctx, struct{}{}, true), httptest.NewRecorder())
if httpx.Metadata(clone.Context()) != meta {
t.Error("metadata lost")
}
return c.OK(nil)
})
}
for _, app := range apps {
r := httptest.NewRequest("GET", "/probe/42", nil)
r.RemoteAddr = "127.0.0.1:1234"
r.Header.Set("X-Forwarded-For", "203.0.113.7")
app.ServeHTTP(httptest.NewRecorder(), r)
}
for i, out := range []string{a.String(), b.String()} {
if !strings.Contains(out, "http.request") || !strings.Contains(out, "business") || strings.Contains(out, `"app":`+string(rune('0'+(1-i)))) {
t.Fatal(out)
}
}
_ = apps[0].Close()
lb.Info("still open")
if !strings.Contains(b.String(), "still open") {
t.Fatal("borrowed logger closed")
}
}

View File

@@ -3,7 +3,9 @@ package app
import (
"net/http"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
)
@@ -26,10 +28,14 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid
}
}
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
request = request.WithContext(logx.WithLogger(contract.WithRequestPolicy(request.Context(), a.config.Request), a.config.LogInstance))
if len(a.config.TrustedProxies) > 0 {
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
}
requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context()))
requestContext.WithContext(httpx.WithRequestMetadata(request.Context(), httpx.RequestMetadata{
ClientIP: requestContext.ClientIP(), Method: request.Method, Route: httpx.RoutePattern(request.Context()),
}))
if provider, ok := a.responseEncoder.(response.ErrorRegistryProvider); ok {
requestContext.SetErrorRegistry(provider.ErrorRegistry())
}

View File

@@ -1,12 +1,14 @@
# go-omni CLI
v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目录>|--apply]`,以及 `workspace format|generate|check --file omni.workspace.json`。详见 [通用开发指南](../../DEVELOPMENT.md)。
新项目采用根目录 `main.go` 与 `internal/config、svc、logic、handler、types`。完整职责、启动扩展、流式接口和生成文件边界见 [项目指南](../../PROJECT.md)。
## 创建项目
```sh
# 仓库读取权限与 PATH 配置见根目录 README
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.13.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.15.0-preview
go-omni new --module example.com/hello ./hello
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
@@ -41,14 +43,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
## 版本与迁移
CLI 与框架依赖固定使用 v0.13.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
CLI 与框架依赖固定使用 v0.15.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
```sh
# 在 go-omni 源码仓库中执行
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
```
新模板需要 v0.13.0-preview 的依赖退出与端点配置能力,继续使用 manifest v2。v0.12 项目需按 [升级步骤](../../PROJECT.md#从-v012-生成项目升级) 补入 svc 生命周期字段和方法,生成器保留这些项目文件。不兼容旧目录,旧生成项目请在新目录重新生成后移入业务代码,不覆盖手写 main。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
模板最低需要 v0.15.0-preview 的分阶段 Runtime 能力,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
## 数据库迁移命令

View File

@@ -15,16 +15,19 @@ func runAPI(args []string) error {
return nil
}
command := args[0]
if command == "upgrade" {
return runUpgrade(args[1:])
}
if command == "go" {
command = "generate"
}
if command == "check" {
command = "validate"
}
if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" {
if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" && command != "format" {
return fmt.Errorf("unknown api command %q", command)
}
readOnly := command == "validate" || command == "inspect" || command == "export"
readOnly := command == "validate" || command == "inspect" || command == "export" || command == "format"
o := apigen.Options{}
for i := 1; i < len(args); i++ {
arg := args[i]
@@ -33,7 +36,7 @@ func runAPI(args []string) error {
return nil
}
if arg == "--check" || arg == "--dry-run" {
if command != "generate" {
if command != "generate" && !(command == "format" && arg == "--check") {
return fmt.Errorf("%s belongs to api generate", arg)
}
if arg == "--check" {
@@ -108,6 +111,14 @@ func runAPI(args []string) error {
return err
}
}
if command == "format" {
changes, err := apigen.FormatProject(o.API, o.Check)
if err != nil {
return err
}
fmt.Printf("API format: %d files updated\n", len(changes))
return nil
}
if readOnly {
spec, err := apigen.Parse(o.API)
if err != nil {

View File

@@ -0,0 +1,91 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestBusinessContract(t *testing.T) {
spec, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/business.api"))
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
var doc map[string]any
if err = json.Unmarshal(c.Document, &doc); err != nil {
t.Fatal(err)
}
paths := doc["paths"].(map[string]any)
body := paths["/optional"].(map[string]any)["post"].(map[string]any)["requestBody"].(map[string]any)
if body["required"] != false {
t.Fatal("optional body documented as required")
}
for _, path := range []string{"/empty", "/absent"} {
if paths[path].(map[string]any)["post"].(map[string]any)["x-omni-request-policy"] == nil {
t.Fatal("missing policy metadata")
}
}
}
func TestBusinessGeneratedProject(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "business")
o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/business.api"), Module: "example.com/business", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
for _, name := range []string{"upload", "nullable", "nothing", "rawNull", "rawArray", "empty", "absent", "optional", "echo", "tolerant", "noContent"} {
writeFile(t, filepath.Join(dir, "internal/logic/demo/"+name+".go"), "package demo\n")
}
writeFile(t, filepath.Join(dir, "internal/logic/demo/list.go"), readFile(t, "testdata/business/operations.go"))
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/business/consumer_test.go"))
writeFile(t, filepath.Join(dir, "internal/middleware/accessFactory.go"), readFile(t, "testdata/business/accessFactory.go"))
writeFile(t, filepath.Join(dir, "internal/svc/client.go"), readFile(t, "testdata/business/client.go"))
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n Client *accessClient", 1)
source = strings.Replace(source, "return nil", "return c.initAccess(ctx)", 1)
writeFile(t, svcFile, source)
o.API = filepath.Join(dir, "api/business.api")
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-race", "-mod=mod", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("%v\n%s", err, out)
}
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
raw, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(raw))
}
var module struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(raw, &module); err != nil {
t.Fatal(err)
}
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
t.Fatal("not a fixed remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
}
}

View File

@@ -0,0 +1,69 @@
package apigen
import (
"errors"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/response"
"io"
"os"
"path/filepath"
)
const catalogDocumentFile = "omni.errors.json"
func readDocumentCatalog(root string, spec *Spec) error {
path := filepath.Join(root, catalogDocumentFile)
stat, err := os.Lstat(path)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return err
}
if !stat.Mode().IsRegular() {
return errors.New("api: error metadata must be a regular file")
}
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
raw, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
if err != nil {
return err
}
if len(raw) > 1<<20 {
return errors.New("api: error metadata exceeds 1 MiB")
}
var catalog apierrors.Catalog
if err = apierrors.DecodeCatalogJSON(raw, &catalog); err != nil {
return err
}
registry, err := apierrors.NewRegistry(catalog)
if err != nil {
return err
}
normalized := registry.Catalog()
spec.DocumentCatalog = &normalized
spec.Sources[catalogDocumentFile] = raw
spec.SourcePaths[catalogDocumentFile] = path
return nil
}
func validateDocumentCatalog(spec *Spec) error {
if spec.DocumentCatalog == nil {
return nil
}
for _, r := range spec.Routes {
if len(r.Errors) > 0 {
continue
}
policy, err := response.New(responseConfig(spec, r))
if err != nil {
return err
}
if err = policy.ValidateErrors(*spec.DocumentCatalog); err != nil {
return err
}
}
return nil
}

View File

@@ -40,8 +40,17 @@ func Compile(spec *Spec) (*Compiled, error) {
if prepareErr != nil {
return nil, prepareErr
}
if err := validateDocumentCatalog(spec); err != nil {
return nil, err
}
if err := spec.RequestPolicy.Validate(); err != nil {
return nil, err
}
definitions := map[string]Type{}
for _, t := range spec.Types {
if t.Name == "File" {
return nil, errors.New("File is reserved for multipart uploads")
}
if t.Name == "ContractScalars" && len(spec.Scalars) > 0 {
return nil, errors.New("ContractScalars is reserved when declaring scalars")
}
@@ -61,7 +70,7 @@ func Compile(spec *Spec) (*Compiled, error) {
resolving := map[string]bool{}
var resolve func(string, int) (reflect.Type, error)
var expression func(ast.Expr, int) (reflect.Type, error)
primitives := map[string]reflect.Type{"any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
primitives := map[string]reflect.Type{"File": reflect.TypeOf(contract.File{}), "any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
if depth > 32 {
return nil, errors.New("type nesting exceeds 32")
@@ -138,7 +147,7 @@ func Compile(spec *Spec) (*Compiled, error) {
if err != nil {
return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err)
}
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)})
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(fieldTag(f))})
}
t := reflect.StructOf(fields)
resolved[name] = t
@@ -162,7 +171,7 @@ func Compile(spec *Spec) (*Compiled, error) {
}
handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{}
middlewareSymbols := map[string]string{}
for _, r := range spec.Routes {
for routeIndex, r := range spec.Routes {
if !goName.MatchString(r.Handler) {
return nil, fmt.Errorf("%s: invalid handler name", r.Position)
}
@@ -233,14 +242,26 @@ func Compile(spec *Spec) (*Compiled, error) {
if err := validateRouteExtensions(r); err != nil {
return nil, fmt.Errorf("%s: %w", r.Position, err)
}
effectivePolicy := contract.MergePolicy(spec.RequestPolicy, r.RequestPolicy)
if (r.Method == "get" || r.Method == "head") && effectivePolicy.Body != "" && effectivePolicy.Body != "auto" && effectivePolicy.Body != "none" {
return nil, errors.New("GET/HEAD cannot inherit a JSON body policy; override body with auto or none")
}
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
if r.Request != "" {
b, ok := c.Bindings[r.Request]
if !ok {
return nil, fmt.Errorf("unknown request %s", r.Request)
}
r.BoundMultipart = b.HasMultipart() && !r.CustomBinding && !r.Manual
spec.Routes[routeIndex].BoundMultipart = r.BoundMultipart
if b.HasMultipart() && r.BodyMode != "multipart" {
return nil, errors.New("multipart fields require @body (mode: multipart)")
}
op.Parameters = b.Parameters()
op.Request = b.BodySchema()
if op.Request != nil && (effectivePolicy.Body == "none" || effectivePolicy.Body == "emptyObject") {
return nil, errors.New("request policy conflicts with JSON fields")
}
if op.Request != nil && (r.BodyMode == "raw" || r.BodyMode == "multipart") {
return nil, errors.New("raw/multipart request DTOs may declare only path/form/header fields; use the generated body stream for content")
}
@@ -250,11 +271,7 @@ func Compile(spec *Spec) (*Compiled, error) {
}
var response *openapi.Schema
if r.Response != "" {
b, ok := c.Bindings[r.Response]
if !ok {
return nil, fmt.Errorf("unknown response %s", r.Response)
}
s, err := b.ResponseSchema()
s, err := responseDataSchema(r.Response, c.Bindings)
if err != nil {
return nil, err
}
@@ -295,7 +312,7 @@ func Compile(spec *Spec) (*Compiled, error) {
if err = decoder.Decode(&doc); err != nil {
return nil, err
}
decorateDocument(doc, spec)
decorateDocument(doc, spec, c.Bindings)
c.Document, err = json.MarshalIndent(doc, "", " ")
return c, err
}

View File

@@ -90,7 +90,8 @@ func snapshotContract(spec *Spec) *contractSnapshot {
}
}
visit(r.Request)
visit(r.Response)
_, responseName := responseShape(r.Response)
visit(responseName)
op := routeContract{
Method: strings.ToUpper(r.Method), Path: r.Path, Group: r.Group, Handler: r.Handler,
Request: r.Request, Response: r.Response, BodyMode: r.BodyMode, Produces: r.Produces,
@@ -223,14 +224,14 @@ func expectedLogicSignature(r Route) string {
if r.Request != "" {
params = append(params, "*types."+r.Request)
}
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
if streamsBody(r) {
params = append(params, "*httpx.Body")
}
results := []string{"error"}
if r.Produces != "" {
results = []string{"*httpx.Stream", "error"}
} else if r.Response != "" {
results = []string{"*types." + r.Response, "error"}
} else if result := responseGoType(r.Response); result != "" {
results = []string{result, "error"}
}
return displaySignature(upper(r.Handler), params, results)
}

View File

@@ -3,6 +3,7 @@ package apigen
import (
"encoding/json"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"net/http"
"regexp"
"sort"
@@ -119,7 +120,8 @@ func prepareMetadata(original *Spec) (*Spec, error) {
return &spec, nil
}
func decorateDocument(doc map[string]any, spec *Spec) {
func decorateDocument(doc map[string]any, spec *Spec, bindings map[string]*contract.Binder) {
doc["x-omni-contract-sources"] = sourceMetadata(spec)
if spec.DefaultResponse != "" {
doc["x-omni-default-response"] = spec.DefaultResponse
}
@@ -144,7 +146,12 @@ func decorateDocument(doc map[string]any, spec *Spec) {
}
doc["components"].(map[string]any)["securitySchemes"] = schemes
definitions := map[string]ErrorDefinition{}
registry := apierrors.MustRegistry(errorCatalog(spec, nil))
declaredRegistry := apierrors.MustRegistry(errorCatalog(spec, nil))
var documentRegistry *apierrors.Registry
if spec.DocumentCatalog != nil {
documentRegistry = apierrors.MustRegistry(*spec.DocumentCatalog)
doc["x-omni-app-error-catalog"] = documentRegistry.Catalog()
}
for _, e := range spec.Errors {
definitions[e.Name] = e
}
@@ -152,6 +159,7 @@ func decorateDocument(doc map[string]any, spec *Spec) {
doc["x-omni-errors"] = spec.Errors
}
for _, r := range spec.Routes {
registry := declaredRegistry
policy := response.MustNew(responseConfig(spec, r))
cfg := policy.Config()
parts := strings.Split(r.Path, "/")
@@ -161,7 +169,16 @@ func decorateDocument(doc map[string]any, spec *Spec) {
}
}
op := doc["paths"].(map[string]any)[strings.Join(parts, "/")].(map[string]any)[r.Method].(map[string]any)
decorateRequestOptions(op, r)
requestRoute := r
requestRoute.RequestPolicy = contract.MergePolicy(spec.RequestPolicy, r.RequestPolicy)
decorateRequestOptions(op, requestRoute)
if b := bindings[r.Request]; b != nil && b.HasMultipart() {
encoding := map[string]any{}
for _, name := range b.MultipartJSONFields() {
encoding[name] = map[string]any{"contentType": "application/json"}
}
op["requestBody"] = map[string]any{"required": true, "content": map[string]any{"multipart/form-data": map[string]any{"schema": b.MultipartSchema(), "encoding": encoding}}}
}
if r.ResponseFormat != "" {
op["x-omni-response"] = r.ResponseFormat
}
@@ -193,6 +210,13 @@ func decorateDocument(doc map[string]any, spec *Spec) {
e := definitions[name]
grouped[e.Status] = append(grouped[e.Status], e)
}
if len(r.Errors) == 0 && documentRegistry != nil {
registry = documentRegistry
op["x-omni-error-catalog-scope"] = "app"
for key, definition := range registry.Catalog().Definitions {
grouped[definition.Status] = append(grouped[definition.Status], ErrorDefinition{Name: key, Code: key, Status: definition.Status, Message: definition.Message, BusinessCode: definition.Code, Params: definition.Params})
}
}
if len(r.Errors) > 0 {
op["x-omni-errors"] = r.Errors
}

View File

@@ -0,0 +1,127 @@
package apigen
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
)
func TestFieldDocumentationAndFormat(t *testing.T) {
source := "// 项目说明\n type Req{\n// 状态\n State int `json:\"state,options=0|1\" example:\"1\" enumDescriptions:\"[\\\"禁用\\\",\\\"启用\\\"]\"` // 状态值\n /* 第一行\n 第二行 */\nName string `json:\"name\" example:\"企业名称\"`\n}\nservice demo{\n @handler echo\npost /echo (Req) returns (Req)\n}\n"
formatted, err := Format([]byte(source))
if err != nil {
t.Fatal(err)
}
again, err := Format(formatted)
if err != nil || !bytes.Equal(formatted, again) {
t.Fatal("format is not idempotent", err, string(again))
}
if !bytes.Contains(formatted, []byte(" // 状态\n State")) {
t.Fatal(string(formatted))
}
spec, err := parseText(t, string(formatted))
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
if spec.Types[0].Fields[0].Description != "状态\n状态值" || spec.Types[0].Fields[1].Description != "第一行\n第二行" {
t.Fatalf("%+v", spec.Types[0].Fields)
}
schema, err := c.Bindings["Req"].ResponseSchema()
if err != nil {
t.Fatal(err)
}
state := schema.Properties["state"]
if state.Description != "状态\n状态值" || len(state.EnumDescriptions) != 2 || state.Example != json.Number("1") {
t.Fatalf("%+v", state)
}
dir := t.TempDir()
entry := filepath.Join(dir, "demo.api")
writeFile(t, entry, source)
if _, err = FormatProject(entry, true); err == nil {
t.Fatal("check accepted drift")
}
if readFile(t, entry) != source {
t.Fatal("check wrote source")
}
if _, err = FormatProject(entry, false); err != nil {
t.Fatal(err)
}
if _, err = FormatProject(entry, true); err != nil {
t.Fatal(err)
}
project := filepath.Join(t.TempDir(), "demo")
if _, err = Generate(Options{API: entry, Dir: project, Module: "example.com/docs", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
dto := readFile(t, filepath.Join(project, "internal/types/"+dtoFile("demo.api:1:1")))
if !strings.Contains(dto, "// 状态值") || !strings.Contains(dto, `description:"状态\n状态值"`) {
t.Fatal(dto)
}
// Invalid examples fail before project output is created.
bad := strings.Replace(source, "example:\"1\"", "example:\"9\"", 1)
writeFile(t, entry, bad)
if _, err = FormatProject(entry, false); err == nil {
t.Fatal("invalid enum example accepted")
}
if data, _ := os.ReadFile(entry); string(data) != bad {
t.Fatal("invalid source changed")
}
}
func TestFormatAlignsFieldsAndPreservesTokens(t *testing.T) {
source := []byte("type Req {\n Name string `json:\"name\"` // 名称\n Sequence int64 `json:\"sequence\"` // 编号\n}\n")
out, err := Format(source)
if err != nil {
t.Fatal(err)
}
again, err := Format(out)
if err != nil || !bytes.Equal(out, again) {
t.Fatal("alignment not idempotent", err)
}
lines := strings.Split(string(out), "\n")
if strings.Index(lines[1], "`") != strings.Index(lines[2], "`") || strings.Index(lines[1], "//") != strings.Index(lines[2], "//") {
t.Fatal("fields not aligned", string(out))
}
for _, literal := range []string{"`json:\"name\"`", "`json:\"sequence\"`", "// 名称", "// 编号"} {
if !bytes.Contains(out, []byte(literal)) {
t.Fatal("literal changed", literal)
}
}
}
func TestRequestPolicyCannotCreateGETBody(t *testing.T) {
for _, source := range []string{
"service demo { @handler get\n @request(body: required)\n get /x }",
"@app(request: {\"body\":\"optional\"})\n service demo { @handler get\n get /x }",
} {
spec, err := parseText(t, source)
if err == nil {
_, err = Compile(spec)
}
if err == nil {
t.Fatal("GET body policy accepted")
}
}
}
func TestFormatLiteralPreservation(t *testing.T) {
source := []byte("type Demo {\n Body string `json:\"body\"\ndescription:\"literal space\"`\n}\n")
out, err := Format(source)
if err != nil {
t.Fatal(err)
}
if !bytes.Contains(out, []byte("`json:\"body\"\ndescription:\"literal space\"`")) {
t.Fatal(string(out))
}
for _, src := range []string{"type X {", "}", "info (title:\"unterminated)"} {
if _, err := Format([]byte(src)); err == nil {
t.Fatal("invalid input accepted")
}
}
}

View File

@@ -0,0 +1,207 @@
package apigen
import (
"bytes"
"errors"
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"text/scanner"
"unicode/utf8"
)
// Format normalizes indentation and blank lines without rewriting tokens,
// comments, quoted metadata, struct tags or multiline literal contents.
func Format(source []byte) ([]byte, error) {
if len(source) > 1<<20 {
return nil, errors.New("api: source size limit exceeded")
}
var scan scanner.Scanner
scan.Init(bytes.NewReader(source))
scan.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments
var scanErr error
scan.Error = func(s *scanner.Scanner, msg string) {
if scanErr == nil {
scanErr = fmt.Errorf("%s: %s", s.Position, msg)
}
}
depths := map[int]int{}
protected := map[int]bool{}
depth := 0
for k := scan.Scan(); k != scanner.EOF; k = scan.Scan() {
line := scan.Position.Line
value := scan.TokenText()
if k == '}' || k == ')' {
depth--
if depth < 0 {
return nil, errors.New("api: unmatched closing delimiter")
}
}
if _, ok := depths[line]; !ok {
depths[line] = depth
}
// Preserve interior lines exactly, including the closing line of a literal.
count := strings.Count(value, "\n")
for n := 1; n <= count; n++ {
protected[line+n] = true
}
if k == '{' || k == '(' {
depth++
}
}
if scanErr != nil {
return nil, scanErr
}
if depth != 0 {
return nil, errors.New("api: unclosed delimiter")
}
lines := strings.Split(string(source), "\n")
out := make([]string, 0, len(lines))
outProtected := map[int]bool{}
blank := true
lastDepth := 0
for i, line := range lines {
if protected[i+1] {
out = append(out, line)
outProtected[len(out)] = true
blank = false
continue
}
trimmed := strings.TrimSpace(line)
if trimmed == "" {
if !blank {
out = append(out, "")
blank = true
}
continue
}
d, ok := depths[i+1]
if ok {
lastDepth = d
}
out = append(out, strings.Repeat(" ", lastDepth)+trimmed)
blank = false
}
for len(out) > 0 && out[len(out)-1] == "" {
out = out[:len(out)-1]
}
alignFieldLines(out, outProtected)
return []byte(strings.Join(out, "\n") + "\n"), nil
}
// Only complete, single-line fields are aligned. Literal bytes and multiline
// comment contents are never candidates, even when they resemble a field.
var singleLineField = regexp.MustCompile("^([ \\t]*)([A-Z][A-Za-z0-9_]*)[ \\t]+([^ \\t`]+)[ \\t]+(`[^`]*`)[ \\t]*(//.*)?$")
func alignFieldLines(lines []string, protected map[int]bool) {
for start := 0; start < len(lines); {
var fields [][]string
widths := [3]int{}
end := start
for end < len(lines) && !protected[end+1] {
parts := singleLineField.FindStringSubmatch(lines[end])
if parts == nil || (len(fields) > 0 && parts[1] != fields[0][1]) {
break
}
for col := range widths {
if n := utf8.RuneCountInString(parts[col+2]); n > widths[col] {
widths[col] = n
}
}
fields = append(fields, parts)
end++
}
for i, parts := range fields {
line := parts[1]
for col := range widths {
line += parts[col+2]
if col < 2 || parts[5] != "" {
line += strings.Repeat(" ", widths[col]-utf8.RuneCountInString(parts[col+2])+1)
}
}
lines[start+i] = line + parts[5]
}
if end == start {
end++
}
start = end
}
}
// FormatProject validates the full contract before writing any local source.
// Check mode is read-only and reports every file needing formatting.
func FormatProject(entry string, check bool) ([]string, error) {
spec, err := Parse(entry)
if err != nil {
return nil, err
}
if _, err = Compile(spec); err != nil {
return nil, err
}
real, err := filepath.EvalSymlinks(entry)
if err != nil {
return nil, err
}
root := filepath.Dir(real)
names := make([]string, 0, len(spec.Sources))
for name := range spec.Sources {
if strings.HasPrefix(name, "@") || filepath.Ext(name) != ".api" {
continue
}
names = append(names, name)
}
sort.Strings(names)
next := map[string][]byte{}
var changed []string
for _, name := range names {
data, err := Format(spec.Sources[name])
if err != nil {
return nil, fmt.Errorf("%s: %w", name, err)
}
if !bytes.Equal(data, spec.Sources[name]) {
next[name] = data
changed = append(changed, name)
}
}
if check {
if len(changed) > 0 {
return changed, fmt.Errorf("API format drift: %s", strings.Join(changed, ", "))
}
return changed, nil
}
for _, name := range changed {
path := filepath.Join(root, filepath.FromSlash(name))
old, err := os.ReadFile(path)
if err != nil {
return changed, err
}
if !bytes.Equal(old, spec.Sources[name]) {
return changed, fmt.Errorf("API source changed during formatting: %s", name)
}
stat, err := os.Stat(path)
if err != nil {
return changed, err
}
f, err := os.CreateTemp(filepath.Dir(path), ".omni-format-*")
if err != nil {
return changed, err
}
tmp := f.Name()
err = f.Chmod(stat.Mode().Perm())
if err == nil {
_, err = f.Write(next[name])
}
err = errors.Join(err, f.Close())
if err == nil {
err = os.Rename(tmp, path)
}
_ = os.Remove(tmp)
if err != nil {
return changed, err
}
}
return changed, nil
}

View File

@@ -19,10 +19,11 @@ import (
)
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
const minimumVersion = "v0.13.0-preview"
const minimumVersion = "v0.15.0-preview"
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
type Options struct {
lockHeld bool
Database, Storage string
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
Check bool
@@ -95,6 +96,9 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
return Output{}, err
}
o.Dir = directory
if err = checkRuntimeExtension(directory); err != nil {
return Output{}, err
}
for _, r := range c.Spec.Routes {
if !r.Manual {
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
@@ -109,6 +113,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
return Output{}, err
}
seeds := map[string][]byte{}
requiredVersion := ""
moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod"))
if readErr == nil {
mf, err := modfile.Parse("go.mod", moduleData, nil)
@@ -120,7 +125,6 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
}
o.Module = mf.Module.Mod.Path
found := false
requiredVersion := ""
for _, r := range mf.Require {
if r.Mod.Path == frameworkModule {
found = true
@@ -141,13 +145,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
if !filepath.IsAbs(path) {
path = filepath.Join(directory, path)
}
if _, err := os.Stat(filepath.Join(path, "lifecycle", "controller.go")); err == nil {
if _, err := os.Stat(filepath.Join(path, "lifecycle", "runtime.go")); err == nil {
localCurrent = true
}
}
}
if !localCurrent {
return Output{}, fmt.Errorf("existing framework dependency predates dependency shutdown and configurable endpoint support; use %s or a current local replacement", minimumVersion)
return Output{}, fmt.Errorf("existing framework dependency predates the generated phased lifecycle support; use %s or a current local replacement", minimumVersion)
}
}
if o.FrameworkDir != "" {
@@ -191,7 +195,16 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
// from its own api/ tree, including all imports.
entry := "api/" + c.Spec.Entry
for name, data := range c.Spec.Sources {
seeds["api/"+name] = data
if !strings.HasPrefix(name, "@") {
seeds["api/"+name] = data
}
}
if len(c.Spec.SharedRoots) > 0 {
raw, e := sharedSeeds(c.Spec, directory)
if e != nil {
return Output{}, e
}
seeds["api/"+importsFile] = raw
}
if o.API != "" {
source, _ := filepath.Abs(o.API)
@@ -200,6 +213,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
rel, e := filepath.Rel(targetReal, sourceReal)
if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
entry = filepath.ToSlash(rel)
delete(seeds, "api/"+importsFile)
for name := range c.Spec.Sources {
delete(seeds, "api/"+name)
}
@@ -208,7 +222,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
// Source collisions must never leave a copied contract different from the
// generated code. On updates, use the developer-owned API in the project.
for name, data := range seeds {
if strings.HasSuffix(name, ".api") {
if strings.HasSuffix(name, ".api") || filepath.Base(name) == catalogDocumentFile || filepath.Base(name) == importsFile {
current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name)))
if e == nil && string(current) != string(data) {
return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name)
@@ -227,7 +241,16 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
}
}
}
return apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec))
out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld)
expectedVersion := minimumVersion
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) {
expectedVersion = info.Main.Version
}
if requiredVersion != "" && requiredVersion != expectedVersion {
out.Diagnostics = append(out.Diagnostics, Diagnostic{Severity: "review", Code: "frameworkVersion", Path: "go.mod", Before: requiredVersion, After: expectedVersion, Message: "project dependency differs from this CLI version; use matching CLI/framework versions, or verify the explicit local replacement"})
sortDiagnostics(out.Diagnostics)
}
return out, err
}
func moduleFile(o Options) ([]byte, error) {
@@ -336,6 +359,13 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
types := "package types\n\n"
needsUpload := false
for _, b := range c.Bindings {
needsUpload = needsUpload || b.HasMultipart()
}
if needsUpload {
types += fmt.Sprintf("import %q\n", frameworkModule+"/contract")
}
if len(c.Spec.Scalars) > 0 {
types += "import("
for i, d := range c.Spec.Scalars {
@@ -350,12 +380,28 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
registry += "}}\n"
put(generated, "internal/types/scalarsGen.go", generatedHeader+registry)
}
if needsUpload {
types += "type File = contract.File\n"
}
bySource := map[string]string{}
for _, t := range c.Spec.Types {
types += "type " + t.Name + " struct {\n"
for _, f := range t.Fields {
types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag)
name := dtoFile(t.Position)
if bySource[name] == "" {
bySource[name] = "package types\n\n"
}
types += "}\n"
text := "type " + t.Name + " struct {\n"
for _, f := range t.Fields {
if f.Description != "" {
for _, line := range strings.Split(f.Description, "\n") {
text += "// " + line + "\n"
}
}
text += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, fieldTag(f))
}
bySource[name] += text + "}\n"
}
for name, text := range bySource {
put(generated, "internal/types/"+name, generatedHeader+text)
}
put(generated, "internal/types/typesGen.go", generatedHeader+types)
put(generated, "docs/openapi.json", string(c.Document)+"\n")
@@ -408,8 +454,8 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
continue
}
hasAutomatic = true
needsTypes = needsTypes || r.Request != "" || r.Response != ""
needsBinder = needsBinder || (r.Request != "" && !r.CustomBinding)
needsTypes = needsTypes || r.Request != "" || responseGoType(r.Response) != ""
needsBinder = needsBinder || !r.CustomBinding
}
text += fmt.Sprintf("%q;%q;", frameworkModule+"/httpx", o.Module+"/internal/svc")
if needsTypes {
@@ -442,12 +488,18 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Manual.go", fmt.Sprintf("package %s\nimport(%q;apierrors %q)\n// %s owns raw/multipart binding and response output. h.deps holds project dependencies.\n// This file is preserved on regeneration.\nfunc(h *Handler)%s(c *httpx.Context)error{return apierrors.New(501,\"NOT_IMPLEMENTED\",\"manual handler is not implemented\")}\n", packageName, frameworkModule+"/httpx", frameworkModule+"/errors", name, name))
continue
}
if r.Request != "" && !r.CustomBinding {
if !r.CustomBinding {
scalarArgs := ""
if len(c.Spec.Scalars) > 0 {
scalarArgs = "types.ContractScalars()..."
}
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s](%s)\n", lower(r.Handler), r.Request, scalarArgs)
requestType := "struct{}"
if r.Request != "" {
requestType = "types." + r.Request
} else {
scalarArgs = ""
}
text += fmt.Sprintf("var %sRequest = contract.MustNew[%s](%s)\n", lower(r.Handler), requestType, scalarArgs)
}
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
call := "c.Context()"
@@ -455,16 +507,32 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
if r.CustomBinding {
text += fmt.Sprintf("var req types.%s\nif err:=h.bind%s(c,&req);err!=nil{return err}\n", r.Request, name)
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Bind.go", fmt.Sprintf("package %s\nimport(%q;%q;apierrors %q)\n// bind%s is project-owned; repeated generation preserves it.\nfunc(h *Handler)bind%s(c *httpx.Context,req *types.%s)error{return apierrors.New(501,\"BINDING_NOT_IMPLEMENTED\",\"custom binding is not implemented\")}\n", packageName, frameworkModule+"/httpx", o.Module+"/internal/types", frameworkModule+"/errors", name, name, r.Request))
} else if r.BoundMultipart {
text += fmt.Sprintf("var req types.%s\ncleanup,err:=%sRequest.BindMultipart(c.Request(),httpx.RouteParams(c.Context()),&req,contract.MultipartOptions{MaxBodyBytes:%d,MaxParts:%d,MaxFieldBytes:%d,MaxFileBytes:%d,Policy:%s});if err!=nil{return err};defer cleanup()\n", r.Request, lower(r.Handler), r.MaxBodyBytes, r.MaxParts, r.MaxFieldBytes, r.MaxFileBytes, policyLiteral(r.RequestPolicy))
} else {
limit := ""
if r.MaxBodyBytes > 0 {
limit = fmt.Sprintf(",%d", r.MaxBodyBytes)
}
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req%s);err!=nil{return err}\n", r.Request, lower(r.Handler), limit)
policy := r.RequestPolicy
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
policy.Body = "auto"
}
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.BindWithPolicy(c.Request(),httpx.RouteParams(c.Context()),&req,%s%s);err!=nil{return err}\n", r.Request, lower(r.Handler), policyLiteral(policy), limit)
}
call += ", &req"
} else {
limit := ""
if r.MaxBodyBytes > 0 {
limit = fmt.Sprintf(",%d", r.MaxBodyBytes)
}
policy := r.RequestPolicy
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
policy.Body = "auto"
}
text += fmt.Sprintf("if err:=%sRequest.BindWithPolicy(c.Request(),httpx.RouteParams(c.Context()),&struct{}{},%s%s);err!=nil{return err}\n", lower(r.Handler), policyLiteral(policy), limit)
}
streamingBody := r.BodyMode == "raw" || r.BodyMode == "multipart"
streamingBody := streamsBody(r)
if streamingBody {
text += fmt.Sprintf("body,err:=httpx.NewBody(c.Request(),%q);if err!=nil{return err}\n", r.BodyMode)
call += ",body"
@@ -475,8 +543,15 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
text += "if err:=body.Complete();err!=nil{if resp!=nil{_ =resp.Close()};return err}\n"
}
text += fmt.Sprintf("return c.WriteStream(%d,%q,resp)\n}\n", r.Status, r.Produces)
} else if r.Response != "" {
text += fmt.Sprintf("resp,err:=h.logic.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name)
} else if responseGoType(r.Response) != "" {
text += fmt.Sprintf("resp,err:=h.logic.%s(%s);if err!=nil{return err};\n", name, call)
kind, _ := responseShape(r.Response)
if kind == "object" {
text += fmt.Sprintf("if resp==nil{return fmt.Errorf(%q)}\n", name+" returned a nil response")
}
if kind == "array" {
text += "if resp==nil{resp=make(" + responseGoType(r.Response) + ",0)}\n"
}
if streamingBody {
text += "if err:=body.Complete();err!=nil{return err}\n"
}
@@ -487,10 +562,15 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
if streamingBody {
text += "if err:=body.Complete();err!=nil{return err};"
}
text += "return c.NoContent()\n}\n"
if r.Response == "null" {
method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status]
text += "return c." + method + "(nil)\n}\n"
} else {
text += "return c.NoContent()\n}\n"
}
}
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
if r.Request != "" || r.Response != "" {
if r.Request != "" || responseGoType(r.Response) != "" {
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
}
if streamingBody || r.Produces != "" {
@@ -503,7 +583,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}"
} else {
service += "// TODO: implement business behavior and use s.deps for dependencies.\n"
if r.Response != "" || r.Produces != "" {
if responseGoType(r.Response) != "" || r.Produces != "" {
service += "return nil, "
} else {
service += "return "
@@ -549,7 +629,11 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
return nil, nil, err
}
for name, data := range infra {
seeds[name] = data
if strings.HasSuffix(name, "Gen.go") {
generated[name] = append([]byte(generatedHeader), data...)
} else {
seeds[name] = data
}
}
if o.Storage == "local" {
for _, r := range c.Spec.Routes {
@@ -564,7 +648,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
func hasResponse(routes []Route) bool {
for _, r := range routes {
if r.Response != "" && !r.Manual && r.Produces == "" {
if kind, _ := responseShape(r.Response); r.Response != "" && kind == "object" && !r.Manual && r.Produces == "" {
return true
}
}
@@ -575,17 +659,41 @@ func signature(r Route) string {
if r.Request != "" {
s += ",req *types." + r.Request
}
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
if streamsBody(r) {
s += ",body *httpx.Body"
}
s += ") "
if r.Produces != "" {
return s + "(*httpx.Stream,error)"
}
if r.Response != "" {
s += "(*types." + r.Response + ",error)"
if result := responseGoType(r.Response); result != "" {
s += "(" + result + ",error)"
} else {
s += "error"
}
return s
}
func streamsBody(r Route) bool {
return r.BodyMode == "raw" || (r.BodyMode == "multipart" && !r.BoundMultipart)
}
func dtoFile(position string) string {
source := position
for i := 0; i < 2; i++ {
if idx := strings.LastIndex(source, ":"); idx >= 0 {
source = source[:idx]
}
}
stem := strings.TrimSuffix(filepath.Base(source), ".api")
var b strings.Builder
for _, r := range stem {
if r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' {
b.WriteRune(r)
}
}
if b.Len() == 0 {
b.WriteString("contract")
}
return "dto" + upper(b.String()) + digest([]byte(source))[:12] + "Gen.go"
}

View File

@@ -42,7 +42,7 @@ func checkLogic(root, module string, contract *contractSnapshot, seeds map[strin
typeExtensions := false
if entries, err := os.ReadDir(filepath.Join(root, "internal/types")); err == nil {
for _, e := range entries {
if sourceFile(e.Name()) && e.Name() != "typesGen.go" && e.Name() != "scalarsGen.go" {
if sourceFile(e.Name()) && !ownedPath("internal/types/"+e.Name()) {
typeExtensions = true
}
}

View File

@@ -71,6 +71,7 @@ func TestManagedGeneratedProject(t *testing.T) {
files := map[string]string{
"consumer_test.go": "consumer_test.go", "internal/svc/options.go": "options.go", "internal/svc/client.go": "client.go", "internal/logic/demo/check.go": "operations.go",
"shutdown_test.go": "shutdown_test.go",
"phases_test.go": "phases_test.go", "internal/svc/background.go": "background.go",
}
for path, fixture := range files {
writeFile(t, filepath.Join(dir, path), readFile(t, filepath.Join("testdata/managed", fixture)))
@@ -79,10 +80,10 @@ func TestManagedGeneratedProject(t *testing.T) {
writeFile(t, filepath.Join(dir, "internal/logic/demo", name+".go"), "package demo\n")
}
configPath := filepath.Join(dir, "internal/config/config.go")
writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1))
writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n BackgroundMode string `json:\"backgroundMode\"`\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1))
svcPath := filepath.Join(dir, "internal/svc/serviceContext.go")
source := strings.Replace(readFile(t, svcPath), "type ServiceContext struct {", "type ServiceContext struct {\n Client *Client", 1)
source = strings.Replace(source, "return c, nil", "if err=initClient(ctx,c);err!=nil{return nil,err}\n return c,nil", 1)
source = strings.Replace(source, "return nil", "return initClient(ctx,c)", 1)
writeFile(t, svcPath, source)
// Independent tooling stays outside generator ownership.
writeFile(t, filepath.Join(dir, "cmd/migrate/main.go"), "package main\nfunc main(){}\n")

View File

@@ -5,6 +5,8 @@ package apigen
import (
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"io"
"os"
"path/filepath"
@@ -14,6 +16,10 @@ import (
)
type Spec struct {
DocumentCatalog *apierrors.Catalog
SharedRoots map[string]SharedRoot
SourcePaths map[string]string
RequestPolicy contract.RequestPolicy
DefaultResponse string
appDeclared bool
Title, Version, Service, Description string
@@ -32,8 +38,12 @@ type Type struct {
Fields []Field
Position string
}
type Field struct{ Name, Type, Tag string }
type Field struct{ Name, Type, Tag, Description string }
type Route struct {
BoundMultipart bool
MaxParts int
MaxFieldBytes, MaxFileBytes int64
RequestPolicy contract.RequestPolicy
Manual, CustomBinding bool
BodyMode, Produces string
MaxBodyBytes int64
@@ -51,6 +61,7 @@ type Route struct {
}
type token struct {
text, position string
comment string
kind rune
space bool
}
@@ -71,11 +82,15 @@ func Parse(path string) (*Spec, error) {
return nil, err
}
root := filepath.Dir(entry)
spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
roots, err := readSharedRoots(root)
if err != nil {
return nil, err
}
spec := &Spec{Version: "1.0.0", SharedRoots: roots, SourcePaths: map[string]string{}, Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
visiting, visited := map[string]bool{}, map[string]bool{}
total := 0
var visit func(string, int) error
visit = func(name string, depth int) error {
var visit func(string, string, int) error
visit = func(name, alias string, depth int) error {
if depth > 32 || len(visited)+len(visiting) >= 64 {
return errors.New("api: import limit exceeded")
}
@@ -83,9 +98,16 @@ func Parse(path string) (*Spec, error) {
if err != nil {
return err
}
rel, err := filepath.Rel(root, real)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return errors.New("api: imports must stay within the entry API directory")
sourceRoot := root
if alias != "" {
sourceRoot = roots[alias].Path
}
rel, err := sourceWithin(sourceRoot, real)
if err != nil {
return err
}
if alias != "" {
rel = "@" + alias + "/" + rel
}
if filepath.Ext(real) != ".api" {
return errors.New("api: expected .api file")
@@ -120,18 +142,37 @@ func Parse(path string) (*Spec, error) {
return err
}
spec.Sources[filepath.ToSlash(rel)] = data
spec.SourcePaths[filepath.ToSlash(rel)] = real
for _, imp := range p.imports {
if filepath.IsAbs(imp) || strings.Contains(imp, "\\") {
return errors.New("api: imports must use relative slash paths")
}
if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil {
nextAlias := alias
next := filepath.Join(filepath.Dir(real), filepath.FromSlash(imp))
if strings.HasPrefix(imp, "@") {
var sub string
var ok bool
nextAlias, sub, ok = strings.Cut(strings.TrimPrefix(imp, "@"), "/")
declared, found := roots[nextAlias]
if !ok || !found || sub == "" || strings.HasPrefix(sub, "/") || strings.HasPrefix(sub, "../") || filepath.ToSlash(filepath.Clean(sub)) != sub {
return errors.New("api: import needs a declared @root and a clean relative path")
}
next = filepath.Join(declared.Path, filepath.FromSlash(sub))
}
if err = visit(next, nextAlias, depth+1); err != nil {
return err
}
}
visited[real] = true
return nil
}
if err = visit(entry, 0); err != nil {
if err = visit(entry, "", 0); err != nil {
return nil, err
}
if err = readDocumentCatalog(root, spec); err != nil {
return nil, err
}
if err = validateSharedPins(spec); err != nil {
return nil, err
}
if spec.Service == "" || len(spec.Routes) == 0 {
@@ -146,7 +187,7 @@ func lex(name, source string) ([]token, error) {
var s scanner.Scanner
s.Init(strings.NewReader(source))
s.Filename = name
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments
s.Whitespace ^= 1 << '\n'
var scanErr error
s.Error = func(s *scanner.Scanner, message string) {
@@ -156,9 +197,29 @@ func lex(name, source string) ([]token, error) {
}
var out []token
end := 0
pending := ""
lastLine := 0
lastToken := -1
for {
kind := s.Scan()
out = append(out, token{s.TokenText(), s.Position.String(), kind, s.Position.Offset > end})
if kind == scanner.Comment {
comment := cleanComment(s.TokenText())
if lastToken >= 0 && s.Position.Line == lastLine {
out[lastToken].comment = joinComment(out[lastToken].comment, comment)
} else {
pending = joinComment(pending, comment)
}
end = s.Position.Offset + len(s.TokenText())
continue
}
current := token{text: s.TokenText(), position: s.Position.String(), kind: kind, space: s.Position.Offset > end}
if kind != '\n' {
current.comment = pending
pending = ""
lastLine = s.Position.Line
lastToken = len(out)
}
out = append(out, current)
end = s.Position.Offset + len(s.TokenText())
if kind == scanner.EOF {
break
@@ -315,8 +376,20 @@ func (p *parser) parse() error {
if err != nil {
return err
}
if len(m) != 1 || m["defaultResponse"] == "" {
return p.err("@app requires only defaultResponse: TemplateName")
if len(m) == 0 {
return p.err("@app requires defaultResponse or request")
}
for key, value := range m {
switch key {
case "defaultResponse":
case "request":
p.spec.RequestPolicy, err = decodeRequestPolicy(value)
if err != nil {
return err
}
default:
return p.err("unsupported @app property " + key)
}
}
p.spec.DefaultResponse = m["defaultResponse"]
p.spec.appDeclared = true
@@ -422,6 +495,7 @@ func (p *parser) parseType() error {
}
typ := Type{Name: name, Position: pos}
for p.skip(); p.peek() != "}"; p.skip() {
description := p.tokens[p.at].comment
name, err := p.ident()
if err != nil {
return err
@@ -444,11 +518,13 @@ func (p *parser) parseType() error {
if p.tokens[p.at].kind != scanner.RawString {
return p.err("field source tag required")
}
tag, err := strconv.Unquote(p.next().text)
tagToken := p.next()
description = joinComment(description, tagToken.comment)
tag, err := strconv.Unquote(tagToken.text)
if err != nil {
return err
}
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag})
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag, Description: description})
}
p.next()
p.spec.Types = append(p.spec.Types, typ)
@@ -496,6 +572,12 @@ func (p *parser) parseService(server map[string]string) error {
}
seen[annotation] = true
switch annotation {
case "request":
var props map[string]string
props, e = p.metadata()
if e == nil {
route.RequestPolicy, e = parseRequestPolicy(props)
}
case "handlerMode":
mode, err := p.ident()
if err != nil {
@@ -589,9 +671,48 @@ func (p *parser) routeType() (string, error) {
if err := p.expect("("); err != nil {
return "", err
}
prefix := ""
if p.peek() == "[" {
p.next()
if err := p.expect("]"); err != nil {
return "", err
}
prefix = "[]"
} else if p.peek() == "*" {
p.next()
prefix = "*"
}
name, err := p.ident()
if err != nil {
return "", err
}
return name, p.expect(")")
return prefix + name, p.expect(")")
}
func joinComment(a, b string) string {
if a == "" {
return b
}
if b == "" {
return a
}
return a + "\n" + b
}
func cleanComment(s string) string {
if strings.HasPrefix(s, "//") {
return strings.TrimSpace(strings.TrimPrefix(s, "//"))
}
s = strings.TrimSuffix(strings.TrimPrefix(s, "/*"), "*/")
lines := strings.Split(s, "\n")
for i, line := range lines {
lines[i] = strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "*"))
}
return strings.TrimSpace(strings.Join(lines, "\n"))
}
func fieldTag(f Field) string {
tags, err := contract.ParseTags(f.Tag)
if err != nil || f.Description == "" || tags["description"] != "" {
return f.Tag
}
return f.Tag + " description:" + strconv.Quote(f.Description)
}

View File

@@ -1,9 +1,11 @@
package apigen
import (
"encoding/json"
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"mime"
"strconv"
"strings"
@@ -16,6 +18,21 @@ func (p *parser) parseBody(r *Route) error {
}
for key, value := range props {
switch key {
case "maxParts":
r.MaxParts, err = strconv.Atoi(value)
if err != nil || r.MaxParts < 1 || r.MaxParts > 10000 {
return p.err("maxParts must be 1..10000")
}
case "maxFieldBytes", "maxFileBytes":
limit, e := strconv.ParseInt(value, 10, 64)
if e != nil || limit < 1 || limit > 1<<30 {
return p.err("part limits must be 1..1073741824 bytes")
}
if key == "maxFieldBytes" {
r.MaxFieldBytes = limit
} else {
r.MaxFileBytes = limit
}
case "mode":
r.BodyMode = value
case "maxBytes":
@@ -33,6 +50,16 @@ func (p *parser) parseBody(r *Route) error {
return nil
}
func validateRouteExtensions(r Route) error {
if (r.MaxParts > 0 || r.MaxFieldBytes > 0 || r.MaxFileBytes > 0) && r.BodyMode != "multipart" {
return errors.New("part limits require multipart mode")
}
if err := r.RequestPolicy.Validate(); err != nil {
return err
}
if (r.BodyMode == "raw" || r.BodyMode == "multipart") && r.RequestPolicy.Body != "" && r.RequestPolicy.Body != "auto" {
return errors.New("raw/multipart cannot declare a JSON body policy")
}
if r.BodyMode != "" && r.BodyMode != "json" && r.BodyMode != "raw" && r.BodyMode != "multipart" {
return errors.New("body mode must be json, raw or multipart")
}
@@ -42,6 +69,9 @@ func validateRouteExtensions(r Route) error {
if r.MaxBodyBytes > 0 && (r.Method == "get" || r.Method == "head") {
return errors.New("GET/HEAD cannot declare a body limit")
}
if (r.Method == "get" || r.Method == "head") && r.RequestPolicy.Body != "" && r.RequestPolicy.Body != "auto" && r.RequestPolicy.Body != "none" {
return errors.New("GET/HEAD cannot declare a JSON body policy")
}
if r.Produces != "" {
media, params, err := mime.ParseMediaType(r.Produces)
if err != nil || len(params) != 0 || media != r.Produces || strings.Contains(media, "*") {
@@ -58,6 +88,23 @@ func validateRouteExtensions(r Route) error {
}
func decorateRequestOptions(op map[string]any, r Route) {
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
r.RequestPolicy.Body = "auto"
}
op["x-omni-request-policy"] = r.RequestPolicy
if r.BodyMode == "multipart" {
limits, _ := (contract.MultipartOptions{MaxBodyBytes: r.MaxBodyBytes, MaxParts: r.MaxParts, MaxFieldBytes: r.MaxFieldBytes, MaxFileBytes: r.MaxFileBytes}).Normalized()
op["x-omni-multipart-limits"] = map[string]any{"maxBytes": limits.MaxBodyBytes, "maxParts": limits.MaxParts, "maxFieldBytes": limits.MaxFieldBytes, "maxFileBytes": limits.MaxFileBytes}
}
if r.BodyMode != "raw" && r.BodyMode != "multipart" {
if body, ok := op["requestBody"].(map[string]any); ok && r.RequestPolicy.Body == "optional" {
body["required"] = false
}
if _, ok := op["requestBody"]; !ok && (r.RequestPolicy.Body == "emptyObject" || r.RequestPolicy.Body == "required" || r.RequestPolicy.Body == "optional") {
op["requestBody"] = map[string]any{"required": r.RequestPolicy.Body != "optional", "content": map[string]any{"application/json": map[string]any{"schema": map[string]any{"type": "object", "additionalProperties": false}}}}
}
}
if r.Manual {
op["x-omni-handler-mode"] = "manual"
}
@@ -87,3 +134,21 @@ func decorateRequestOptions(op map[string]any, r Route) {
op["responses"].(map[string]any)[strconv.Itoa(r.Status)] = map[string]any{"description": "Success", "content": map[string]any{r.Produces: map[string]any{"schema": schema}}}
}
}
func parseRequestPolicy(props map[string]string) (contract.RequestPolicy, error) {
data, err := json.Marshal(props)
if err != nil {
return contract.RequestPolicy{}, err
}
return decodeRequestPolicy(string(data))
}
func decodeRequestPolicy(raw string) (contract.RequestPolicy, error) {
var p contract.RequestPolicy
if err := apierrors.DecodeCatalogJSON([]byte(raw), &p); err != nil {
return p, fmt.Errorf("request policy: %w", err)
}
return p, p.Validate()
}
func policyLiteral(p contract.RequestPolicy) string {
return fmt.Sprintf("contract.RequestPolicy{UnknownQuery:%q,UTF8:%q,Charset:%q,Body:%q}", p.UnknownQuery, p.UTF8, p.Charset, p.Body)
}

View File

@@ -0,0 +1,54 @@
package apigen
import (
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/openapi"
"strings"
)
// Response roots deliberately allow named DTOs, []DTO, *DTO and null. Keeping
// these four shapes explicit makes nil/empty behavior uniform across encoders.
func responseShape(value string) (kind, name string) {
if value == "null" {
return "null", ""
}
if strings.HasPrefix(value, "[]") {
return "array", strings.TrimPrefix(value, "[]")
}
if strings.HasPrefix(value, "*") {
return "nullable", strings.TrimPrefix(value, "*")
}
return "object", value
}
func responseGoType(value string) string {
kind, name := responseShape(value)
if kind == "null" || value == "" {
return ""
}
if kind == "array" {
return "[]types." + name
}
return "*types." + name
}
func responseDataSchema(value string, bindings map[string]*contract.Binder) (*openapi.Schema, error) {
kind, name := responseShape(value)
if kind == "null" {
return &openapi.Schema{Type: "object", Nullable: true, Enum: []any{nil}}, nil
}
b, ok := bindings[name]
if !ok {
return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, *Name or null", name)
}
s, err := b.ResponseSchema()
if err != nil {
return nil, err
}
switch kind {
case "array":
return &openapi.Schema{Type: "array", Items: s}, nil
case "nullable":
s.Nullable = true
}
return s, nil
}

View File

@@ -0,0 +1,150 @@
package apigen
import (
"encoding/json"
"errors"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"io"
"os"
"path/filepath"
"sort"
"strings"
)
const importsFile = "omni.imports.json"
// SharedRoot explicitly authorizes reads from one canonical local directory.
// Version is a project-owned label; SHA256 optionally pins the imported source
// set, not a remote tag. The CLI never downloads or executes shared contracts.
type SharedRoot struct {
Path string `json:"path"`
Version string `json:"version,omitempty"`
SHA256 string `json:"sha256,omitempty"`
}
type sharedConfig struct {
Roots map[string]SharedRoot `json:"roots"`
}
func readSharedRoots(root string) (map[string]SharedRoot, error) {
stat, err := os.Lstat(filepath.Join(root, importsFile))
if errors.Is(err, os.ErrNotExist) {
return nil, nil
}
if err != nil {
return nil, err
}
if !stat.Mode().IsRegular() {
return nil, errors.New("api: shared root configuration must be a regular file")
}
f, err := os.Open(filepath.Join(root, importsFile))
if errors.Is(err, os.ErrNotExist) {
return nil, nil
}
if err != nil {
return nil, err
}
defer f.Close()
raw, err := io.ReadAll(io.LimitReader(f, 65537))
if err != nil {
return nil, err
}
if len(raw) > 65536 {
return nil, errors.New("api: shared root config too large")
}
var cfg sharedConfig
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
return nil, err
}
if len(cfg.Roots) > 16 {
return nil, errors.New("api: at most 16 shared roots")
}
for name, r := range cfg.Roots {
if !lowerName.MatchString(name) || r.Path == "" || len(r.Version) > 128 {
return nil, errors.New("api: invalid shared root")
}
if r.SHA256 != "" && (len(r.SHA256) != 64 || strings.Trim(r.SHA256, "0123456789abcdef") != "") {
return nil, errors.New("api: shared sha256 must be 64 lowercase hex characters")
}
path := r.Path
if !filepath.IsAbs(path) {
path = filepath.Join(root, path)
}
r.Path, err = filepath.EvalSymlinks(path)
if err != nil {
return nil, err
}
r.Path, err = filepath.Abs(r.Path)
if err != nil {
return nil, err
}
stat, err := os.Stat(r.Path)
if err != nil {
return nil, err
}
if !stat.IsDir() {
return nil, errors.New("api: shared root must be a directory")
}
cfg.Roots[name] = r
}
return cfg.Roots, nil
}
func sourceWithin(root, name string) (string, error) {
real, err := filepath.EvalSymlinks(name)
if err != nil {
return "", err
}
rel, err := filepath.Rel(root, real)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return "", errors.New("api: imports must stay within their declared contract root")
}
return filepath.ToSlash(rel), nil
}
func sharedHash(sources map[string][]byte, alias string) string {
values := map[string]string{}
for name, data := range sources {
if strings.HasPrefix(name, "@"+alias+"/") {
values[strings.TrimPrefix(name, "@"+alias+"/")] = digest(data)
}
}
raw, _ := json.Marshal(values)
return digest(raw)
}
func sourceMetadata(spec *Spec) map[string]any {
hashes := map[string]string{}
for name, data := range spec.Sources {
hashes[name] = digest(data)
}
shared := map[string]any{}
for alias, r := range spec.SharedRoots {
shared[alias] = map[string]string{"version": r.Version, "sha256": sharedHash(spec.Sources, alias)}
}
return map[string]any{"files": hashes, "shared": shared}
}
func sharedSeeds(spec *Spec, directory string) ([]byte, error) {
cfg := sharedConfig{Roots: map[string]SharedRoot{}}
for name, r := range spec.SharedRoots {
rel, err := filepath.Rel(filepath.Join(directory, "api"), r.Path)
if err != nil {
return nil, err
}
r.Path = filepath.ToSlash(rel)
cfg.Roots[name] = r
}
data, err := json.MarshalIndent(cfg, "", " ")
return append(data, '\n'), err
}
func validateSharedPins(spec *Spec) error {
names := make([]string, 0, len(spec.SharedRoots))
for name := range spec.SharedRoots {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
r := spec.SharedRoots[name]
if r.SHA256 != "" && r.SHA256 != sharedHash(spec.Sources, name) {
return fmt.Errorf("api: shared contract %s checksum differs (got %s)", name, sharedHash(spec.Sources, name))
}
}
return nil
}

View File

@@ -0,0 +1,96 @@
package apigen
import (
"bytes"
"encoding/json"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"os"
"path/filepath"
"strings"
"testing"
)
func TestControlledSharedContracts(t *testing.T) {
root := t.TempDir()
shared := filepath.Join(root, "shared")
writeFile(t, filepath.Join(shared, "response.api"), "response Common (codeType: int32; successCode: 0; errorCode: 9999; dataMode: always)\ntype Item { Name string `json:\"name\"` }\n")
for _, service := range []string{"api", "id"} {
dir := filepath.Join(root, service)
entry := filepath.Join(dir, "api/service.api")
writeFile(t, entry, "import \"@common/response.api\"\n@app(defaultResponse: Common)\nservice demo {\n@handler get\nget /get returns (Item)\n}\n")
config := `{"roots":{"common":{"path":"../../shared","version":"1.0.0"}}}`
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), config)
spec, err := Parse(entry)
if err != nil {
t.Fatal(err)
}
if len(spec.Sources) != 2 || len(spec.SourcePaths) != 2 {
t.Fatalf("%+v", spec)
}
pin := sharedHash(spec.Sources, "common")
config = strings.Replace(config, `"version":"1.0.0"`, `"version":"1.0.0","sha256":"`+pin+`"`, 1)
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), config)
o := Options{Dir: dir, API: entry, Module: "example.com/" + service, FrameworkDir: frameworkRoot(t)}
if _, err = Generate(o); err != nil {
t.Fatal(err)
}
o.Check = true
if _, err = Generate(o); err != nil {
t.Fatal(err)
}
if _, err = os.Stat(filepath.Join(dir, "api/@common")); !os.IsNotExist(err) {
t.Fatal("shared contract copied into service")
}
document := readFile(t, filepath.Join(dir, "docs/openapi.json"))
if !strings.Contains(document, pin) || !strings.Contains(document, "@common/response.api") {
t.Fatal("missing provenance")
}
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), strings.Replace(config, pin, strings.Repeat("0", 64), 1))
if _, err = Parse(entry); err == nil {
t.Fatal("checksum mismatch accepted")
}
}
// An undeclared alias, traversal, symlink escape and cycle are all rejected.
for _, imp := range []string{"@missing/response.api", "@common/../outside.api", "../shared/response.api", "@common/escape.api", "@common/cycle.api"} {
local := filepath.Join(root, "consumer")
writeFile(t, filepath.Join(local, "omni.imports.json"), `{"roots":{"common":{"path":"../shared"}}}`)
writeFile(t, filepath.Join(root, "outside.api"), "type Outside {}\n")
_ = os.Symlink(filepath.Join(root, "outside.api"), filepath.Join(shared, "escape.api"))
writeFile(t, filepath.Join(shared, "cycle.api"), "import \"cycle.api\"\n")
entry := filepath.Join(local, "service.api")
writeFile(t, entry, "import "+strconvQuote(imp)+"\nservice demo{\n@handler get\nget /get\n}\n")
if _, err := Parse(entry); err == nil {
t.Fatal("unsafe import accepted", imp)
}
}
}
func strconvQuote(value string) string { data, _ := json.Marshal(value); return string(data) }
func TestCatalogDocumentExportDoesNotGenerateRuntimeRegistry(t *testing.T) {
dir := t.TempDir()
entry := filepath.Join(dir, "service.api")
writeFile(t, entry, "@app(defaultResponse: Common)\nresponse Common(codeType: int32; successCode: 0; errorCode: 9999; messageField: msg; dataMode: always)\nservice demo{\n@handler get\nget /get returns (null)\n}\n")
var exported bytes.Buffer
err := apierrors.ExportCatalog(&exported, apierrors.Catalog{DefaultStatus: 200, Definitions: map[string]apierrors.Definition{"TOO_LONG": {Code: 400002, Message: "{label}不能超过{max}个字符", Params: []apierrors.Parameter{{Name: "label", Type: "string", Example: "企业名称"}, {Name: "max", Type: "int", Example: 100}}}}})
if err != nil {
t.Fatal(err)
}
writeFile(t, filepath.Join(dir, "omni.errors.json"), exported.String())
spec, err := Parse(entry)
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
if len(c.Spec.Errors) != 0 || !strings.Contains(string(c.Document), "企业名称不能超过100个字符") {
t.Fatal(string(c.Document))
}
target := filepath.Join(t.TempDir(), "catalog")
if _, err = Generate(Options{Dir: target, API: entry, Module: "example.com/catalog", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
if _, err = os.Stat(filepath.Join(target, "internal/apierrors/errorsGen.go")); !os.IsNotExist(err) {
t.Fatal("document metadata became a runtime error registry")
}
}

View File

@@ -20,8 +20,11 @@ import (
func NewApp(deps *svc.ServiceContext)(*omni.App,error){
if deps==nil || deps.Lifecycle==nil{return nil,fmt.Errorf("service context and lifecycle controller required; see PROJECT.md upgrade instructions")}
cfg:=deps.Config.Server
cfg.LogInstance=deps.Logger
for _,option:=range deps.AppOptions(){if option!=nil{option(&cfg)}}
`, frameworkModule, frameworkModule+"/httpx", frameworkModule+"/config", frameworkModule+"/health", o.Module+"/internal/svc")
source += fmt.Sprintf("cfg.Request=contract.MergePolicy(cfg.Request,%#v)\n", spec.RequestPolicy)
source = strings.Replace(source, "\"fmt\"", "\"fmt\";"+fmt.Sprintf("%q", frameworkModule+"/contract"), 1)
if spec.DefaultResponse != "" {
for _, r := range spec.Responses {
if r.Name == spec.DefaultResponse {

View File

@@ -13,6 +13,7 @@ import (
"os/signal"
"syscall"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"MODULE/internal/config"
"MODULE/internal/handler"
"MODULE/internal/svc"
@@ -27,15 +28,13 @@ func main(){
func run(ctx context.Context,path string)(runErr error){
cfg,err:=config.Load(path);if err!=nil{return err}
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,logger.Close())}()
logx.SetDefault(logger)
// Unjoined background work may still log. The owning process exits on this
// error; do not close its borrowed logger before that work has finished.
defer func(){if !errors.Is(runErr,lifecycle.ErrResourcesRetained){runErr=errors.Join(runErr,logger.Close())}}()
deps,err:=svc.New(ctx,logger,cfg);if err!=nil{return err}
// RunContext drains HTTP requests before returning. This scope owns dependencies
// and closes them exactly once, before closing the borrowed logger.
defer func(){
runErr=errors.Join(runErr,deps.Close())
if cause:=deps.Lifecycle.Err();cause!=nil && !errors.Is(runErr,cause){runErr=errors.Join(runErr,cause)}
}()
defer func(){runErr=deps.Finish(runErr)}()
app,err:=handler.NewApp(deps);if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,app.Close())}()
logger.InfoContext(ctx,"service.starting","address",app.Config().Address)
@@ -111,6 +110,16 @@ and produce REVIEW notices. This source scan does not execute project code or
replace go test ./... . Snapshot-based REVIEW notices describe changes since the
last successful generation; they do not persist as an acknowledgement checklist.
## Contract tools
api format [--check] preserves comments and normalizes indentation. api upgrade
previews changed infrastructure; --apply only replaces recognized prior seeds.
Use @request policy overrides and @app(request: {...}) for query/JSON strictness.
Explicitly exported api/omni.errors.json documents the Go-owned error catalog;
the CLI never executes project Go. omni.imports.json permits named shared roots;
workspace format/generate/check processes explicitly listed independent services.
Full reference: https://git.yuntuo.hk/k2safe/go-omni/src/branch/main/DEVELOPMENT.md
## Layout
`+"```text"+`
@@ -119,7 +128,8 @@ main.go # GENERATED startup, lifecycle and route registrati
etc/config.yaml # lowerCamelCase configuration
internal/config/ # defaults -> file -> environment
internal/svc/ # dependency initialization and cleanup
internal/types/typesGen.go # generated request/response types
internal/types/*Gen.go # DTOs split by source API, one types package
internal/svc/runtimeGen.go # GENERATED lifecycle and infrastructure startup
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
internal/handler/<group>/ # generated binding and Logic adapters
internal/logic/<group>/ # developer-owned business operations
@@ -138,7 +148,7 @@ use the same contract. 404/405 and ordinary App routes use the declared App defa
No template is selected by declaration/import order. Without @app, existing framework
response defaults apply. An explicit API App default takes precedence over response
options in svc, keeping generated routes and OpenAPI consistent.
Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default.
Object responses reject nil; []DTO maps nil to []; *DTO permits null; returns(null) is a successful null value. No returns means 204. Body DTOs use application/json and have a 1 MiB limit by default.
Use @body(maxBytes: ...) for route limits. @body(mode: raw|multipart; maxBytes: ...)
adds a *httpx.Body argument to Logic, without handwritten handlers. Raw is the exact
limited request stream; Parts supports sequential multipart reading with no temp files.
@@ -146,11 +156,10 @@ limited request stream; Parts supports sequential multipart reading with no temp
Logic result of *httpx.Stream; the handler closes Content and streams it unwrapped.
Signature verification and storage policy belong to Logic/middleware. A consumed
request stream cannot be reread automatically; use one reader owner or explicitly
bounded buffering. Stream reads respect configured server timeouts. Multipart parts
are not DTO form fields: form tags remain URL query parameters. The optional manual
bounded buffering. Stream reads respect configured server timeouts. Declare multipart/multipartJSON/file DTO fields for validated uploads; without those fields the existing streaming Body is passed. form tags remain URL query parameters. The optional manual
handler/custom binder hooks are preserved, and are needed only for custom protocols.
Fields require exactly one source tag: json, path, form (URL query only), header.
Fields require one source tag: json, path, form (URL query), header, multipart, multipartJSON or file. Multipart fields select full automatic binding before Logic; File/[]File are request-scoped disk-backed uploads. Use File.Open to stream them; generated cleanup owns their lifetime.
All are required by presence unless optional, omitempty or default is specified.
Zero and false are valid present values; binding:"required" rejects scalar zero.
Use *T with optional when business logic needs to distinguish missing from zero.
@@ -171,12 +180,21 @@ The selected auth runs first; @auth none removes inherited declarative auth.
Validation policy remains in the middleware implementation. Public messages must
not contain secrets; error causes may be included in server logs.
svc.New receives the loaded Config and borrowed Logger. Add dependencies there,
Generated svc.New receives Config and a borrowed Logger. Add dependencies in svc.Init,
then immediately register each cleanup with OnClose. A later initialization failure
runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware
provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated
once before route registration; logging is configured earlier through Config.
The generated main runs HTTP draining before closing svc resources, then the logger.
The generated main runs HTTP draining, all OnStop hooks, all OnDrain hooks, resource
OnClose callbacks, then logger close. Stop/drain hooks within a phase run concurrently;
combine components with ordering dependencies into one pair. Pair OnStop/OnDrain
with the same name. OnDrain returns nil only after final work and acknowledgements
have finished. Use WorkContext for accepted background work; Context is an early
shutdown notification. server.lifecycle configures stopTimeout (5s), drainTimeout
(10s), and cancelTimeout (5s grace per timed-out phase). Zero uses defaults.
Timeout/failure cancels WorkContext. If hooks cannot prove completion, resources
and the borrowed logger are retained and the generated process exits unsuccessfully.
Errors retain their causes. No automatic retry of close or force killing of goroutines.
Do not close borrowed resources or attach duplicate cleanup hooks to App. The startup
context cancels on signals; long-lived clients/workers that requests depend on need
their own lifetime, canceled by OnClose after HTTP draining.
@@ -208,7 +226,7 @@ func infrastructureREADME(o Options) string {
Selected database: %s; storage: %s; logging: %s/%s.
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
Use structured database/mysql configuration and GO_OMNI_DB_PASSWORD. Other connection/pool/TLS settings live in etc/config.yaml and GO_OMNI_DB_*.
No tables are invented or migrated automatically from request/response DTOs.
The none/memory selection opens no database; implement repositories per business.

View File

@@ -0,0 +1,22 @@
package middleware
import (
"context"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
// This package deliberately imports no svc; the dependency is supplied explicitly.
type AccessChecker interface {
CheckAccess(context.Context, string) error
}
func NewRequireAccess(checker AccessChecker) httpx.Middleware {
return func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
return err
}
return next(c)
}
}
}

View File

@@ -0,0 +1,32 @@
package svc
import (
"context"
"example.com/business/internal/middleware"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"sync/atomic"
)
// A local fixture checks wiring; production authentication is project-owned.
type accessClient struct{ closed atomic.Bool }
func (c *accessClient) Close() error { c.closed.Store(true); return nil }
func (c *accessClient) CheckAccess(ctx context.Context, token string) error {
if c.closed.Load() {
return apierrors.New(503, "CLOSED", "closed")
}
if httpx.Metadata(ctx).Route == "" {
return apierrors.New(500, "NO_METADATA", "missing route")
}
if token == "reject" {
return apierrors.New(403, "DENIED", "denied")
}
return nil
}
func (c *ServiceContext) initAccess(ctx context.Context) error {
c.Client = &accessClient{}
c.OnClose(c.Client.Close)
c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(c.Client)
return nil
}

View File

@@ -0,0 +1,119 @@
package main
import (
"bytes"
"context"
"encoding/json"
"example.com/business/internal/config"
"example.com/business/internal/handler"
"example.com/business/internal/svc"
"git.yuntuo.hk/k2safe/go-omni/logx"
"io"
"mime/multipart"
"net/http/httptest"
"strings"
"testing"
)
func TestShapesAndPolicy(t *testing.T) {
cfg := config.Default()
cfg.Server.Log.Writer = io.Discard
logger, err := logx.New(cfg.Server.Log)
if err != nil {
t.Fatal(err)
}
defer logger.Close()
deps, err := svc.New(context.Background(), logger, cfg)
if err != nil {
t.Fatal(err)
}
defer deps.Close()
app, err := handler.NewApp(deps)
if err != nil {
t.Fatal(err)
}
defer app.Close()
for _, tt := range []struct {
method, path, body, media string
status int
data string
raw bool
}{
{"GET", "/list", "", "", 200, `[]`, false},
{"GET", "/list?mode=full&tags=a&tags=b", "", "", 200, `[{"name":"one"}]`, false},
{"GET", "/nullable", "", "", 200, `null`, false},
{"GET", "/nullable?mode=full", "", "", 200, `{"name":"one"}`, false},
{"POST", "/nothing", "", "", 200, `null`, false},
{"GET", "/rawNull", "", "", 200, `null`, true},
{"GET", "/rawArray", "", "", 200, `[]`, true},
{"DELETE", "/noContent", "", "", 204, "", true},
{"POST", "/empty", `{}`, "application/json", 200, `null`, false},
{"POST", "/empty", ``, "application/json", 400, "", false},
{"POST", "/empty", `{"unknown":1}`, "application/json", 400, "", false},
{"POST", "/absent", ``, "", 200, `null`, false},
{"POST", "/absent", `{}`, "application/json", 400, "", false},
{"POST", "/optional", ``, "", 200, `{"name":""}`, false},
{"POST", "/optional", `{}`, "application/json", 400, "", false},
{"POST", "/echo", `{"name":"ok"}`, "application/json", 200, `{"name":"ok"}`, false},
{"POST", "/echo", `{"name":"ok"}`, "application/json; charset=latin1", 415, "", false},
{"POST", "/echo", "{\"name\":\"\xff\"}", "application/json", 400, "", false},
{"GET", "/list?unknown=1", "", "", 400, "", false},
{"GET", "/rawNull?unknown=1", "", "", 400, "", false},
{"POST", "/tolerant?unknown=1", "{\"name\":\"\xff\"}", "application/json; charset=latin1", 200, `{"name":"<22>"}`, false},
} {
t.Run(tt.method+tt.path+tt.body+tt.media, func(t *testing.T) {
r := httptest.NewRequest(tt.method, tt.path, strings.NewReader(tt.body))
if tt.media != "" {
r.Header.Set("Content-Type", tt.media)
}
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
if w.Code != tt.status {
t.Fatalf("%d %s", w.Code, w.Body)
}
if tt.status >= 400 {
return
}
data := strings.TrimSpace(w.Body.String())
if !tt.raw {
var obj map[string]json.RawMessage
if err := json.Unmarshal(w.Body.Bytes(), &obj); err != nil {
t.Fatal(err)
}
v, ok := obj["data"]
if !ok {
t.Fatal("missing data")
}
data = string(v)
}
if data != tt.data {
t.Fatalf("data %s expected %s", data, tt.data)
}
})
}
var body bytes.Buffer
form := multipart.NewWriter(&body)
_ = form.WriteField("title", "title")
_ = form.WriteField("metadata", `{"name":"meta"}`)
f, err := form.CreateFormFile("attachment", "a.txt")
if err != nil {
t.Fatal(err)
}
_, _ = io.WriteString(f, "content")
_ = form.Close()
r := httptest.NewRequest("POST", "/upload", &body)
r.Header.Set("Content-Type", form.FormDataContentType())
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
if w.Code != 200 || !strings.Contains(w.Body.String(), "title:meta:content") {
t.Fatal(w.Code, w.Body.String())
}
denied := httptest.NewRequest("GET", "/list", nil)
denied.Header.Set("Authorization", "reject")
result := httptest.NewRecorder()
app.ServeHTTP(result, denied)
if result.Code != 403 {
t.Fatal("injected middleware did not run", result.Code, result.Body.String())
}
}

View File

@@ -0,0 +1,42 @@
package demo
import (
"context"
"example.com/business/internal/types"
"io"
)
func (s *Logic) List(ctx context.Context, req *types.Query) ([]types.Item, error) {
if req.Mode == "full" {
return []types.Item{{Name: "one"}}, nil
}
return nil, nil
}
func (s *Logic) Nullable(ctx context.Context, req *types.Query) (*types.Item, error) {
if req.Mode == "full" {
return &types.Item{Name: "one"}, nil
}
return nil, nil
}
func (s *Logic) Nothing(ctx context.Context) error { return nil }
func (s *Logic) RawNull(ctx context.Context) error { return nil }
func (s *Logic) RawArray(ctx context.Context) ([]types.Item, error) { return nil, nil }
func (s *Logic) Empty(ctx context.Context) error { return nil }
func (s *Logic) Absent(ctx context.Context) error { return nil }
func (s *Logic) Optional(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil }
func (s *Logic) Echo(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil }
func (s *Logic) Tolerant(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil }
func (s *Logic) NoContent(ctx context.Context) error { return nil }
func (s *Logic) Upload(ctx context.Context, req *types.Upload) (*types.Item, error) {
f, err := req.Attachment.Open()
if err != nil {
return nil, err
}
defer f.Close()
data, err := io.ReadAll(f)
if err != nil {
return nil, err
}
return &types.Item{Name: req.Title + ":" + req.Metadata.Name + ":" + string(data)}, nil
}

View File

@@ -0,0 +1,49 @@
package svc
import (
"context"
"errors"
"git.yuntuo.hk/k2safe/go-omni/task"
"sync/atomic"
"time"
)
type BackgroundState struct {
Stop, Drain, Release, Started chan struct{}
Wrote atomic.Bool
Future *task.Future
}
var Background atomic.Pointer[BackgroundState]
func initBackground(c *ServiceContext) error {
state := &BackgroundState{Stop: make(chan struct{}), Drain: make(chan struct{}), Release: make(chan struct{}), Started: make(chan struct{})}
baseline := Closed.Load()
runner, err := task.New(task.Config{Workers: 1, QueueSize: 1, TaskTimeout: time.Minute, OnError: func(string, error) {}})
if err != nil {
return err
}
c.OnStop("jobs", func(ctx context.Context) error { close(state.Stop); return runner.StopAccepting(ctx) })
c.OnDrain("jobs", func(ctx context.Context) error { close(state.Drain); return runner.Wait(ctx) })
state.Future, err = runner.Submit(c.WorkContext(), "finish", func(ctx context.Context) error {
close(state.Started)
if c.Config.BackgroundMode == "auto" {
<-state.Stop
} else {
<-state.Release
}
if ctx.Err() != nil {
return ctx.Err()
}
if Closed.Load() != baseline {
return errors.New("shared client closed before background write")
}
state.Wrote.Store(true)
return nil
})
if err == nil {
<-state.Started
}
Background.Store(state)
return err
}

View File

@@ -20,6 +20,11 @@ func initClient(ctx context.Context, c *ServiceContext) error {
Opened.Add(1)
c.Client = &Client{}
c.OnClose(func() error { Closed.Add(1); c.Logger.Info("test.client.closed"); return nil })
if c.Config.BackgroundMode != "" {
if err := initBackground(c); err != nil {
return err
}
}
if c.Config.FailInit {
return errors.New("client initialization failed")
}

View File

@@ -0,0 +1,155 @@
package main
import (
"context"
"encoding/json"
"errors"
"example.com/managed/internal/svc"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"io"
"net"
"net/http"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func phasedConfig(t *testing.T, address, mode string, fail bool) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.json")
drainTimeout := "1s"
if mode == "stuck" {
drainTimeout = "50ms"
}
data, _ := json.Marshal(map[string]any{"backgroundMode": mode, "failInit": fail, "server": map[string]any{"address": address, "lifecycle": map[string]string{"stopTimeout": "1s", "drainTimeout": drainTimeout, "cancelTimeout": "50ms"}}})
if err := os.WriteFile(path, data, 0600); err != nil {
t.Fatal(err)
}
return path
}
func await(t *testing.T, ch <-chan struct{}) {
t.Helper()
select {
case <-ch:
case <-time.After(3 * time.Second):
t.Fatal("phase did not arrive")
}
}
func TestGeneratedBackgroundPhases(t *testing.T) {
for _, transport := range []string{"nethttp", "native"} {
t.Run(transport, func(t *testing.T) {
t.Setenv("GO_OMNI_TRANSPORT", transport)
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
addr := listener.Addr().String()
_ = listener.Close()
svc.Started = make(chan struct{}, 1)
svc.Release = make(chan struct{})
before := svc.Closed.Load()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
done := make(chan error, 1)
path := phasedConfig(t, addr, "hold", false)
go func() { done <- run(ctx, path) }()
waitReady(t, "http://"+addr, done)
state := svc.Background.Load()
defer func() {
select {
case <-state.Release:
default:
close(state.Release)
}
}()
requestDone := make(chan error, 1)
go func() {
client := http.Client{Timeout: 3 * time.Second}
resp, e := client.Get("http://" + addr + "/slow")
if e == nil {
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
if resp.StatusCode != 200 {
e = errors.New("HTTP drain failed")
}
}
requestDone <- e
}()
await(t, svc.Started)
deps := svc.Current.Load()
deps.RequestShutdown(svc.FatalDependency)
if deps.WorkContext().Err() != nil {
t.Fatal("accepted work canceled at signal")
}
select {
case <-state.Stop:
t.Fatal("stopped background before HTTP finished")
default:
}
close(svc.Release)
if err := <-requestDone; err != nil {
t.Fatal(err)
}
await(t, state.Drain)
if svc.Closed.Load() != before {
t.Fatal("client closed during background drain")
}
close(state.Release)
if err := <-done; !errors.Is(err, svc.FatalDependency) || errors.Is(err, lifecycle.ErrResourcesRetained) {
t.Fatal(err)
}
if !state.Wrote.Load() || svc.Closed.Load() != before+1 {
t.Fatal("missing final write or cleanup")
}
if err := state.Future.Wait(context.Background()); err != nil {
t.Fatal(err)
}
})
}
}
func TestGeneratedBackgroundFailureCleanup(t *testing.T) {
for _, failInit := range []bool{false, true} {
t.Run(map[bool]string{true: "initialization", false: "listen"}[failInit], func(t *testing.T) {
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer l.Close()
before := svc.Closed.Load()
err = run(context.Background(), phasedConfig(t, l.Addr().String(), "auto", failInit))
if err == nil || errors.Is(err, lifecycle.ErrResourcesRetained) {
t.Fatal(err)
}
state := svc.Background.Load()
if !state.Wrote.Load() || svc.Closed.Load() != before+1 {
t.Fatal("failure cleanup did not join background first")
}
})
}
}
func TestGeneratedBackgroundTimeoutProcess(t *testing.T) {
if path := os.Getenv("OMNI_PHASE_TIMEOUT_CONFIG"); path != "" {
os.Args = []string{os.Args[0], "-config", path}
main()
return
}
path := phasedConfig(t, "127.0.0.1:0", "stuck", true)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestGeneratedBackgroundTimeoutProcess$")
cmd.Env = append(os.Environ(), "OMNI_PHASE_TIMEOUT_CONFIG="+path)
output, err := cmd.CombinedOutput()
var exit *exec.ExitError
if !errors.As(err, &exit) || exit.ExitCode() != 1 || ctx.Err() != nil {
t.Fatal(err, string(output))
}
if strings.Contains(string(output), "test.client.closed") || !strings.Contains(string(output), "shared resources retained") || !strings.Contains(string(output), "client initialization failed") {
t.Fatal(string(output))
}
}

View File

@@ -0,0 +1,302 @@
package apigen
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
"go/ast"
"go/format"
goparser "go/parser"
gotoken "go/token"
"golang.org/x/mod/modfile"
"golang.org/x/mod/semver"
"os"
"path/filepath"
"sort"
"strings"
)
type UpgradeChange struct {
Path, Action string
Manual bool
before, after []byte
mode os.FileMode
exists bool
}
type UpgradePlan struct {
Changes []UpgradeChange
Module, Entry string
}
// PlanUpgrade compares infrastructure against the exact previous release seeds.
// A custom file is never classified as safe to replace. Logic, middleware,
// options and repositories outside the changed templates are left untouched.
func PlanUpgrade(dir string) (UpgradePlan, error) {
var plan UpgradePlan
root := canonical(dir)
if err := safePath(root, manifestPath); err != nil {
return plan, err
}
raw, err := os.ReadFile(filepath.Join(root, manifestPath))
if err != nil {
return plan, err
}
var m manifest
if err = json.Unmarshal(raw, &m); err != nil {
return plan, err
}
if m.Version != 2 || m.Profile == nil {
return plan, errors.New("upgrade requires a managed v0.13+ project")
}
if filepath.Ext(m.Entry) != ".api" {
return plan, errors.New("upgrade requires a valid API entry")
}
if err = safePath(root, m.Entry); err != nil {
return plan, err
}
plan.Module, plan.Entry = m.Module, m.Entry
if semver.IsValid(m.TemplateVersion) && semver.Compare(m.TemplateVersion, minimumVersion) >= 0 {
return plan, nil
}
p := m.Profile
legacy, err := scaffold.LegacyAPIInfrastructure(m.Module, p.Database, p.Storage, p.LogOutput, p.LogFormat)
if err != nil {
return plan, err
}
current, err := scaffold.APIInfrastructure(m.Module, p.Database, p.Storage, p.LogOutput, p.LogFormat)
if err != nil {
return plan, err
}
names := map[string]bool{}
for name := range legacy {
names[name] = true
}
for name := range current {
if !strings.HasSuffix(name, "Gen.go") {
names[name] = true
}
}
// v0.14 already owns config/svc extensions. Only bump the module here;
// Generate updates generator-owned main/runtime files without replaying the
// v0.13 seed migration over developer configuration and dependency code.
if semver.IsValid(m.TemplateVersion) && semver.Compare(m.TemplateVersion, "v0.14.0-preview") >= 0 {
names = map[string]bool{}
}
sorted := make([]string, 0, len(names))
for name := range names {
sorted = append(sorted, name)
}
sort.Strings(sorted)
for _, name := range sorted {
// options.go is unchanged by this release. Its application customizations remain untouched.
previous, next := legacy[name], current[name]
if strings.HasSuffix(name, ".go") {
if previous != nil {
previous, err = format.Source(previous)
if err != nil {
return plan, err
}
}
if next != nil {
next, err = format.Source(next)
if err != nil {
return plan, err
}
}
}
if bytes.Equal(previous, next) {
continue
}
if err = safePath(root, name); err != nil {
return plan, err
}
existing, readErr := os.ReadFile(filepath.Join(root, name))
if readErr != nil && !errors.Is(readErr, os.ErrNotExist) {
return plan, readErr
}
exists := readErr == nil
if bytes.Equal(existing, next) {
continue
}
mode := os.FileMode(0644)
if exists {
stat, e := os.Stat(filepath.Join(root, name))
if e != nil {
return plan, e
}
mode = stat.Mode().Perm()
}
action := "update"
if next == nil {
action = "remove"
} else if !exists {
action = "create"
}
plan.Changes = append(plan.Changes, UpgradeChange{Path: name, Action: action, Manual: exists && !bytes.Equal(existing, previous), before: existing, after: next, mode: mode, exists: exists})
}
if err = safePath(root, "go.mod"); err != nil {
return plan, err
}
raw, err = os.ReadFile(filepath.Join(root, "go.mod"))
if err != nil {
return plan, err
}
mf, err := modfile.Parse("go.mod", raw, nil)
if err != nil {
return plan, err
}
for _, r := range mf.Require {
if r.Mod.Path == frameworkModule && semver.Compare(r.Mod.Version, minimumVersion) < 0 {
if err = mf.AddRequire(frameworkModule, minimumVersion); err != nil {
return plan, err
}
next, e := mf.Format()
if e != nil {
return plan, e
}
stat, e := os.Stat(filepath.Join(root, "go.mod"))
if e != nil {
return plan, e
}
plan.Changes = append(plan.Changes, UpgradeChange{Path: "go.mod", Action: "update", before: raw, after: next, exists: true, mode: stat.Mode().Perm()})
break
}
}
return plan, nil
}
// PreviewUpgrade writes only proposed infrastructure files into an empty directory.
// It never mutates the service or emits a replacement business tree.
func PreviewUpgrade(dir, destination string) (UpgradePlan, error) {
plan, err := PlanUpgrade(dir)
if err != nil {
return plan, err
}
if entries, e := os.ReadDir(destination); e == nil && len(entries) > 0 {
return plan, errors.New("upgrade preview directory must be empty")
} else if e != nil && !errors.Is(e, os.ErrNotExist) {
return plan, e
}
for _, c := range plan.Changes {
if c.after == nil {
continue
}
if err = safePath(destination, c.Path); err != nil {
return plan, err
}
if err = atomicWrite(destination, c.Path, c.after, 0644); err != nil {
return plan, err
}
}
note := "# Infrastructure upgrade preview\n\nOnly untouched v0.13 seed files can be replaced automatically. Custom files require a targeted edit; this preview is never copied over the project automatically.\n\nMove dependency initialization into ServiceContext.Init; embed *lifecycle.Runtime; register cleanup with OnClose. Remove old New/Close/OnClose implementations after preserving their project work in Init or cleanup hooks. Use structured database/mysql.Config and DatabaseOptions. Preserve business configuration fields, AppOptions, middleware and Logic.\n\n"
for _, c := range plan.Changes {
note += fmt.Sprintf("- %s %s (manual review: %t)\n", c.Action, c.Path, c.Manual)
}
if err = safePath(destination, "UPGRADE.md"); err != nil {
return plan, err
}
return plan, atomicWrite(destination, "UPGRADE.md", []byte(note), 0644)
}
// ApplyUpgrade applies only recognized seed changes under the same generation
// lock. A generation failure restores every changed developer file. Any manual
// review item blocks all writes; use PreviewUpgrade and migrate that file first.
func ApplyUpgrade(dir string) (plan UpgradePlan, err error) {
root := canonical(dir)
if err = safePath(root, ".goOmni/api.lock"); err != nil {
return plan, err
}
lock, err := os.OpenFile(filepath.Join(root, ".goOmni/api.lock"), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
if err != nil {
return plan, err
}
_ = lock.Close()
defer os.Remove(filepath.Join(root, ".goOmni/api.lock"))
plan, err = PlanUpgrade(root)
if err != nil {
return plan, err
}
for _, c := range plan.Changes {
if c.Manual {
return plan, fmt.Errorf("upgrade: %s contains project changes; inspect an upgrade preview and migrate it explicitly", c.Path)
}
}
var changed []UpgradeChange
defer func() {
if err != nil {
for i := len(changed) - 1; i >= 0; i-- {
c := changed[i]
if c.exists {
err = errors.Join(err, atomicWrite(root, c.Path, c.before, c.mode))
} else {
e := os.Remove(filepath.Join(root, c.Path))
if !errors.Is(e, os.ErrNotExist) {
err = errors.Join(err, e)
}
}
}
}
}()
for _, c := range plan.Changes {
if c.after == nil {
err = os.Remove(filepath.Join(root, c.Path))
} else {
err = atomicWrite(root, c.Path, c.after, c.mode)
}
if err != nil {
return plan, err
}
changed = append(changed, c)
}
preview, e := Generate(Options{Dir: root, API: filepath.Join(root, plan.Entry), DryRun: true, lockHeld: true})
if e != nil {
return plan, e
}
if diagnosticErrors(preview.Diagnostics) > 0 {
return plan, errors.New("upgrade: Logic checks failed; resolve diagnostics before applying")
}
_, err = Generate(Options{Dir: root, API: filepath.Join(root, plan.Entry), lockHeld: true})
return plan, err
}
func checkRuntimeExtension(root string) error {
directory := filepath.Join(root, "internal/svc")
files, err := os.ReadDir(directory)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return err
}
for _, entry := range files {
if !sourceFile(entry.Name()) || strings.HasSuffix(entry.Name(), "Gen.go") {
continue
}
path := filepath.Join(directory, entry.Name())
file, err := goparser.ParseFile(gotoken.NewFileSet(), path, nil, goparser.SkipObjectResolution)
if err != nil {
return err
}
for _, decl := range file.Decls {
fn, ok := decl.(*ast.FuncDecl)
if !ok {
continue
}
if fn.Recv == nil && fn.Name.Name == "New" {
return fmt.Errorf("%s owns New; run go-omni api upgrade --dir %s and move project initialization into ServiceContext.Init", path, root)
}
if fn.Recv != nil && (fn.Name.Name == "Close" || fn.Name.Name == "OnClose" || fn.Name.Name == "Finish") {
receiver := fn.Recv.List[0].Type
if p, ok := receiver.(*ast.StarExpr); ok {
receiver = p.X
}
if id, ok := receiver.(*ast.Ident); ok && id.Name == "ServiceContext" {
return fmt.Errorf("%s overrides framework resource lifecycle; inspect go-omni api upgrade and register cleanup in Init", path)
}
}
}
}
return nil
}

View File

@@ -0,0 +1,167 @@
package apigen
import (
"encoding/json"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
"go/format"
"os"
"path/filepath"
"strings"
"testing"
)
func oldInfrastructureProject(t *testing.T) string {
t.Helper()
dir := filepath.Join(t.TempDir(), "app")
if _, err := New(Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
previous, err := scaffold.LegacyAPIInfrastructure("example.com/upgrade", "none", "none", "stderr", "json")
if err != nil {
t.Fatal(err)
}
for name, content := range previous {
if strings.HasSuffix(name, ".go") {
content, err = format.Source(content)
if err != nil {
t.Fatal(err)
}
}
writeFile(t, filepath.Join(dir, name), string(content))
}
var m manifest
if err = json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
t.Fatal(err)
}
delete(m.Generated, "internal/svc/runtimeGen.go")
m.TemplateVersion = ""
_ = os.Remove(filepath.Join(dir, "internal/svc/runtimeGen.go"))
raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
path := filepath.Join(dir, "go.mod")
writeFile(t, path, strings.ReplaceAll(readFile(t, path), minimumVersion, "v0.13.0-preview"))
return dir
}
func TestUpgradeRejectsEscapingEntry(t *testing.T) {
dir := oldInfrastructureProject(t)
var m manifest
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
t.Fatal(err)
}
m.Entry = "../outside.api"
raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
before := snapshot(t, dir)
if _, err := ApplyUpgrade(dir); err == nil {
t.Fatal("escaping API entry accepted")
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("invalid upgrade wrote project")
}
}
func TestUpgradeV014PreservesExtensions(t *testing.T) {
dir := filepath.Join(t.TempDir(), "service")
if _, err := New(Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
var m manifest
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
t.Fatal(err)
}
m.TemplateVersion = "v0.14.0-preview"
raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
mod := filepath.Join(dir, "go.mod")
writeFile(t, mod, strings.ReplaceAll(readFile(t, mod), minimumVersion, "v0.14.1-preview"))
owned := map[string]string{}
for _, name := range []string{"internal/svc/serviceContext.go", "internal/config/config.go", "etc/config.yaml"} {
path := filepath.Join(dir, name)
value := readFile(t, path)
if strings.HasSuffix(name, ".go") {
value += "\n// project extension kept\n"
} else {
value += "\n# project configuration kept\n"
}
writeFile(t, path, value)
owned[path] = value
}
plan, err := PlanUpgrade(dir)
if err != nil || len(plan.Changes) != 1 || plan.Changes[0].Path != "go.mod" {
t.Fatal(plan, err)
}
if _, err := ApplyUpgrade(dir); err != nil {
t.Fatal(err)
}
for path, want := range owned {
if readFile(t, path) != want {
t.Fatal("developer file modified", path)
}
}
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
t.Fatal(err)
}
}
func TestSafeInfrastructureUpgrade(t *testing.T) {
dir := oldInfrastructureProject(t)
before := snapshot(t, dir)
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}); err == nil {
t.Fatal("legacy constructor accepted")
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("failed generate wrote files")
}
plan, err := PlanUpgrade(dir)
if err != nil || len(plan.Changes) == 0 {
t.Fatal(plan, err)
}
for _, c := range plan.Changes {
if c.Manual {
t.Fatal(c.Path)
}
}
preview := filepath.Join(t.TempDir(), "preview")
if _, err = PreviewUpgrade(dir, preview); err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("preview changed project")
}
logic := readFile(t, filepath.Join(dir, "internal/logic/greet/greet.go"))
if _, err = ApplyUpgrade(dir); err != nil {
t.Fatal(err)
}
if _, err = os.Stat(filepath.Join(dir, "internal/svc/close.go")); !os.IsNotExist(err) {
t.Fatal("old lifecycle remains")
}
if readFile(t, filepath.Join(dir, "internal/logic/greet/greet.go")) != logic {
t.Fatal("business changed")
}
if _, err = Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
t.Fatal(err)
}
plan, err = PlanUpgrade(dir)
if err != nil || len(plan.Changes) != 0 {
t.Fatal("upgrade not idempotent", plan, err)
}
}
func TestUpgradePreservesCustomFilesAndRollsBack(t *testing.T) {
for _, custom := range []bool{true, false} {
dir := oldInfrastructureProject(t)
if custom {
path := filepath.Join(dir, "internal/svc/serviceContext.go")
writeFile(t, path, readFile(t, path)+"\n// custom dependency\n")
} else {
path := filepath.Join(dir, "internal/logic/greet/greet.go")
writeFile(t, path, "package greet;func(s *Logic)Greet(){}")
}
before := snapshot(t, dir)
if _, err := ApplyUpgrade(dir); err == nil {
t.Fatal("unsafe upgrade accepted")
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("failed upgrade changed project")
}
}
}

View File

@@ -0,0 +1,126 @@
package apigen
import (
"errors"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"io"
"os"
"path/filepath"
"strings"
)
type WorkspaceService struct {
Dir string `json:"dir"`
API string `json:"api,omitempty"`
Module string `json:"module,omitempty"`
}
type WorkspaceConfig struct {
Services []WorkspaceService `json:"services"`
}
// Workspace operates independent Go modules listed in an explicit local file.
// All projects are preflighted before generate/format writes. Each generator
// transaction is atomic within its project; no cross-module transaction is promised.
func Workspace(path, command string, w io.Writer) error {
if command != "generate" && command != "check" && command != "format" {
return errors.New("workspace command must be generate, check or format")
}
if w == nil {
w = io.Discard
}
real, err := filepath.Abs(path)
if err != nil {
return err
}
stat, err := os.Lstat(real)
if err != nil {
return err
}
if !stat.Mode().IsRegular() || stat.Size() > 65536 {
return errors.New("workspace configuration must be a regular file of at most 64 KiB")
}
raw, err := os.ReadFile(real)
if err != nil {
return err
}
var cfg WorkspaceConfig
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
return err
}
if len(cfg.Services) == 0 || len(cfg.Services) > 64 {
return errors.New("workspace must list 1..64 services")
}
root := canonical(filepath.Dir(real))
var options []Options
seen := map[string]bool{}
for _, service := range cfg.Services {
if service.Dir == "" || service.Dir == "." {
return errors.New("workspace requires distinct relative service directories")
}
if err = safePath(root, service.Dir+"/.goOmniWorkspaceCheck"); err != nil {
return err
}
dir := filepath.Join(root, filepath.FromSlash(service.Dir))
key := strings.ToLower(dir)
for prior := range seen {
if prior == key || strings.HasPrefix(key, prior+string(filepath.Separator)) || strings.HasPrefix(prior, key+string(filepath.Separator)) {
return errors.New("workspace service directories overlap")
}
}
seen[key] = true
entry := service.API
if entry == "" {
entry, err = ProjectEntry(dir)
} else {
if err = safePath(dir, entry); err != nil {
return err
}
entry = filepath.Join(dir, entry)
}
if err != nil {
return err
}
options = append(options, Options{API: entry, Dir: dir, Module: service.Module, DryRun: true})
}
var failures error
for _, o := range options {
if command == "format" {
spec, e := Parse(o.API)
if e == nil {
_, e = Compile(spec)
}
err = e
} else {
_, err = Generate(o)
}
if err != nil {
failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err))
}
}
if failures != nil {
return failures
}
for _, o := range options {
switch command {
case "format":
_, err = FormatProject(o.API, false)
case "generate":
o.DryRun = false
_, err = Generate(o)
case "check":
_, fmtErr := FormatProject(o.API, true)
o.DryRun = false
o.Check = true
_, genErr := Generate(o)
err = errors.Join(fmtErr, genErr)
}
if err != nil {
failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err))
fmt.Fprintf(w, "FAIL %s\n", o.Dir)
} else {
fmt.Fprintf(w, "OK %s\n", o.Dir)
}
}
return failures
}

View File

@@ -0,0 +1,43 @@
package apigen
import (
"io"
"path/filepath"
"strings"
"testing"
)
func TestWorkspaceIndependentModules(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"api", "id"} {
dir := filepath.Join(root, "services", name)
if _, err := New(Options{Dir: dir, Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
}
config := filepath.Join(root, "omni.workspace.json")
writeFile(t, config, `{"services":[{"dir":"services/api"},{"dir":"services/id"}]}`)
for _, op := range []string{"format", "generate", "check", "generate", "check"} {
if err := Workspace(config, op, io.Discard); err != nil {
t.Fatal(op, err)
}
}
// A bad second service must not leave the first service partially regenerated.
first := filepath.Join(root, "services/api/api/service.api")
writeFile(t, first, strings.Replace(readFile(t, first), "Greet a visitor", "Greet", 1))
second := filepath.Join(root, "services/id/api/service.api")
writeFile(t, second, "invalid source")
before := snapshot(t, root)
if err := Workspace(config, "generate", io.Discard); err == nil {
t.Fatal("invalid workspace accepted")
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("preflight failure changed project")
}
for _, bad := range []string{`{"services":[{"dir":"../outside"}]}`, `{"services":[{"dir":"services/api"},{"dir":"services/api/nested"}]}`, `{"services":[{"dir":"services/api","dir":"services/id"}]}`} {
writeFile(t, config, bad)
if err := Workspace(config, "check", io.Discard); err == nil {
t.Fatal("invalid workspace accepted")
}
}
}

View File

@@ -16,12 +16,13 @@ import (
const manifestPath = ".goOmni/apiManifest.json"
type manifest struct {
Profile *projectProfile `json:"profile,omitempty"`
Version int `json:"version"`
Module string `json:"module"`
Entry string `json:"entry"`
Generated map[string]string `json:"generated"`
Contract *contractSnapshot `json:"contract,omitempty"`
TemplateVersion string `json:"templateVersion,omitempty"`
Profile *projectProfile `json:"profile,omitempty"`
Version int `json:"version"`
Module string `json:"module"`
Entry string `json:"entry"`
Generated map[string]string `json:"generated"`
Contract *contractSnapshot `json:"contract,omitempty"`
}
type change struct {
name string
@@ -30,7 +31,7 @@ type change struct {
mode os.FileMode
}
func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile, contract *contractSnapshot) (Output, error) {
func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile, contract *contractSnapshot, lockHeld ...bool) (Output, error) {
var result Output
if info, err := os.Lstat(root); err == nil {
if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() {
@@ -61,7 +62,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
if err := safePath(root, manifestPath); err != nil {
return result, err
}
if !check && !dryRun {
if !check && !dryRun && !(len(lockHeld) > 0 && lockHeld[0]) {
if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
return result, err
}
@@ -108,7 +109,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
return result, err
}
}
next := manifest{Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
next := manifest{TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
for name, data := range generated {
next.Generated[name] = digest(data)
}
@@ -271,7 +272,10 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
}
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
func ownedPath(name string) bool {
if name == "main.go" || name == "internal/handler/appGen.go" || name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" {
if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) {
return true
}
if name == "main.go" || name == "internal/svc/runtimeGen.go" || name == "internal/handler/appGen.go" || name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" {
return true
}
if name == "internal/apierrors/errorsGen.go" || name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {

View File

@@ -18,6 +18,11 @@ func main() {
}
switch os.Args[1] {
case "workspace":
if err := runWorkspace(os.Args[2:]); err != nil {
fmt.Fprintln(os.Stderr, "go-omni workspace:", err)
os.Exit(1)
}
case "api":
if err := runAPI(os.Args[2:]); err != nil {
fmt.Fprintln(os.Stderr, "go-omni api:", err)
@@ -170,6 +175,9 @@ func usage() {
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
fmt.Fprintln(os.Stderr, " go-omni api new [--module <module>] <directory> Create a runnable API project")
fmt.Fprintln(os.Stderr, " go-omni api generate [--api <file.api>] [--dir <project>] [--check|--dry-run]")
fmt.Fprintln(os.Stderr, " go-omni api format [--api <file.api>] [--check]")
fmt.Fprintln(os.Stderr, " go-omni api upgrade [--dir <project>] [--dry-run|--preview <directory>|--apply]")
fmt.Fprintln(os.Stderr, " go-omni workspace <format|generate|check> [--file omni.workspace.json]")
fmt.Fprintln(os.Stderr, " go-omni api inspect [--api <file.api>] Show routes, auth and business errors")
fmt.Fprintln(os.Stderr, " go-omni api export [--api <file.api>] Write OpenAPI JSON to stdout")
fmt.Fprintln(os.Stderr, " go-omni api validate --api <file.api> Validate definitions and OpenAPI")

View File

@@ -8,7 +8,7 @@ import (
)
func TestNewWithRemoteFrameworkVersion(t *testing.T) {
for _, flag := range [][]string{{"--framework-version", "v0.13.0-preview"}, {"--framework-version=v0.13.0-preview"}} {
for _, flag := range [][]string{{"--framework-version", "v0.15.0-preview"}, {"--framework-version=v0.15.0-preview"}} {
project := filepath.Join(t.TempDir(), "billing")
args := append([]string{"--module", "example.com/billing"}, flag...)
if err := runNew(append(args, project)); err != nil {
@@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(content), "go-omni v0.13.0-preview") || strings.Contains(string(content), "replace ") {
if !strings.Contains(string(content), "go-omni v0.15.0-preview") || strings.Contains(string(content), "replace ") {
t.Fatalf("CLI did not pin remote dependency: %s", content)
}
}

View File

@@ -8,17 +8,37 @@ import (
// APIInfrastructure renders shared infrastructure without inventing business
// tables or routes. The API generator applies these as developer-owned seeds.
func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
return apiInfrastructure(module, database, storage, logOutput, logFormat, false)
}
// LegacyAPIInfrastructure renders only the known v0.13 seed baseline for a safe upgrade.
func LegacyAPIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
return apiInfrastructure(module, database, storage, logOutput, logFormat, true)
}
func apiInfrastructure(module, database, storage, logOutput, logFormat string, legacy bool) (map[string][]byte, error) {
data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true}
files := map[string]string{
"internal/svc/close.go": "templates/api-close.go.tmpl",
"internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl",
"internal/svc/options.go": "templates/api-options.go.tmpl",
"internal/config/config.go": "templates/modules-settings.go.tmpl",
"internal/config/config.go": "templates/api-config.go.tmpl",
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
"internal/svc/serviceContext.go": "templates/api-context.go.tmpl",
}
if database == "mysql" {
files["internal/svc/databaseOptions.go"] = "templates/api-databaseOptions.go.tmpl"
}
if storage != "none" {
files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl"
}
if legacy {
delete(files, "internal/svc/runtimeGen.go")
delete(files, "internal/svc/databaseOptions.go")
files["internal/svc/close.go"] = "templates/api-close.go.tmpl"
files["internal/config/config.go"] = "templates/modules-settings.go.tmpl"
for name, source := range files {
files[name] = strings.Replace(source, "templates/", "templates/legacyV013/", 1)
}
}
result := make(map[string][]byte, len(files))
for name, source := range files {
content, err := render(source, data)

View File

@@ -0,0 +1,104 @@
package settings
import (
"os"
"strings"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
{{if ne .Storage "none"}}"errors"
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
)
type Config struct {
Server config.Config `json:"server"`
{{if eq .Database "mysql"}}Database mysqladapter.Config `json:"database" env:"GO_OMNI_DB"`{{end}}
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage" env:"GO_OMNI_STORAGE"`{{end}}
}
func Default() Config {
server:=config.Default()
server.Address="127.0.0.1:8080"
server.EnableLogger=true
server.SlowRequestThreshold=500*time.Millisecond
server.Log.Service={{printf "%q" .Module}}
server.Log.Format={{printf "%q" .LogFormat}}
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:mysqladapter.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
{{if eq .Storage "r2"}}Region:"auto",{{end}}
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
}{{end}}}
}
// Load applies defaults -> file -> typed environment -> validation.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string)(Config,error){
cfg:=Default();var err error
cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err}
if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}}
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_"});if err!=nil{return Config{},err}
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
cfg.Server,err=config.ApplyEnv(cfg.Server,"");if err!=nil{return Config{},err}
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
return cfg,cfg.Validate()
}
func(c Config)Validate()error{
if err:=c.Server.Validate();err!=nil{return err}
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return err}{{end}}
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return err}{{end}}
return nil
}
{{if ne .Storage "none"}}
type StorageConfig struct {
Provider string `json:"provider"`
Root string `json:"root"`
PublicURL string `json:"publicUrl"`
Endpoint string `json:"endpoint"`
Region string `json:"region"`
Bucket string `json:"bucket"`
AccountID string `json:"accountId"`
Secure bool `json:"secure"`
MaxBytes int64 `json:"maxBytes"`
ContentTypes []string `json:"contentTypes"`
LinkTTL time.Duration `json:"linkTtl"`
RequestTimeout time.Duration `json:"requestTimeout"`
// Secrets are environment-only and cannot be populated by config files.
FilesToken string `json:"-" env:"GO_OMNI_FILES_TOKEN,secret"`
SigningKey string `json:"-" env:"GO_OMNI_STORAGE_SIGNING_KEY,secret"`
AccessKey string `json:"-" env:"GO_OMNI_STORAGE_ACCESS_KEY,secret"`
SecretKey string `json:"-" env:"GO_OMNI_STORAGE_SECRET_KEY,secret"`
SessionToken string `json:"-" env:"GO_OMNI_STORAGE_SESSION_TOKEN,secret"`
}
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
func (s StorageConfig) Validate()error {
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
if err:=s.Policy().Validate();err!=nil{return err}
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
{{if eq .Storage "local"}}
if s.Root=="" {return errors.New("storage.root is required")}
u,err:=url.Parse(s.PublicURL)
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
{{else}}
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
{{if eq .Storage "r2"}}
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
{{else if eq .Storage "s3"}}
u,err:=url.Parse("https://"+s.Endpoint)
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
{{else if eq .Storage "oss"}}
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
if !s.Secure{return errors.New("OSS requires secure=true")}
{{end}}
{{end}}
return nil
}
{{end}}

View File

@@ -1,26 +1,20 @@
package servicecontext
import (
"context"
"errors"
"sync"
{{if ne .Storage "none"}}"net/http"
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
{{if eq .Database "mysql"}}"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"github.com/go-sql-driver/mysql"{{end}}
{{if eq .Database "mysql"}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
"git.yuntuo.hk/k2safe/go-omni/health"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"{{.Module}}/internal/settings"
"{{.Module}}/internal/middleware"
)
// Context owns infrastructure, while Logger is borrowed from main.
// Add business repositories here; HTTP contracts do not define database tables.
// Context holds project dependencies. Runtime owns cleanup and shutdown.
// Logger is borrowed; the generated entry closes it after dependencies.
type Context struct {
Lifecycle *lifecycle.Controller
*lifecycle.Runtime
Config settings.Config
cleanups []func()error
Logger *logx.Logger
Health *health.Checker
Middlewares map[string]httpx.Middleware
@@ -31,38 +25,11 @@ type Context struct {
Downloads storage.Presigner
UploadLinks storage.UploadPresigner
FileHTTP http.Handler
storageClose func()error
{{end}}
closeMu sync.Mutex
closed bool
closeOnce sync.Once
closeErr error
}
// Pass the settings loaded by main. The optional form supports lightweight
// tests without infrastructure; selected adapters still require valid settings.
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
if ctx==nil{ctx=context.Background()}
if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))}
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
_=cfg
checker,err:=health.New();if err!=nil{return nil,err}
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Lifecycle:lifecycle.New(ctx)}
ctx=c.Lifecycle.Context()
defer func(){if err!=nil{err=errors.Join(err,c.Close());if cause:=c.Lifecycle.Err();cause!=nil && !errors.Is(err,cause){err=errors.Join(err,cause)}}}()
{{if eq .Database "mysql"}}
if err=cfg.Validate();err!=nil{return nil,err}
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
dsn.ParseTime=true;dsn.Loc=time.UTC
cfg.Database.DSN=dsn.FormatDSN()
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
{{end}}
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
return c,nil
}
// RequestShutdown reports a fatal dependency failure, or a normal stop when
// err is nil. It returns immediately; generated main drains and cleans up.
func(c *Context)RequestShutdown(err error){c.Lifecycle.RequestShutdown(err)}
// Init is project-owned. Add dependencies here and immediately register each
// cleanup with c.OnClose. Background components use paired OnStop/OnDrain hooks
// and WorkContext. Failed initialization stops/drains before releasing resources;
// unjoined background work returns ErrResourcesRetained instead of closing its dependencies.
// Generated infrastructure is ready here. Never create another App or logger.
func(c *Context)Init(ctx context.Context)error {return nil}

View File

@@ -0,0 +1,5 @@
package servicecontext
import mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"
// DatabaseOptions supplies explicit driver settings or bounded startup checks.
// Keep schema/version requirements in this project; do not open another pool.
func(c *Context)DatabaseOptions()[]mysqladapter.Option{return nil}

View File

@@ -0,0 +1,34 @@
package servicecontext
import (
"context"
"errors"
"git.yuntuo.hk/k2safe/go-omni/health"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
"{{.Module}}/internal/settings"
"{{.Module}}/internal/middleware"
)
// New is generator-owned. Customize dependencies in ServiceContext.Init.
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
if ctx==nil{ctx=context.Background()}
if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))}
if logger==nil{return nil,errors.New("service context requires a logger")}
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
if err=cfg.Validate();err!=nil{return nil,err}
checker,err:=health.New();if err!=nil{return nil,err}
runtime,err:=lifecycle.NewRuntimeWithConfig(logx.WithLogger(ctx,logger),cfg.Server.Lifecycle);if err!=nil{return nil,err}
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Runtime:runtime}
ctx=logx.WithLogger(c.Context(),logger)
defer func(){if p:=recover();p!=nil{_ = c.Close();panic(p)};if err!=nil{err=c.Finish(err)}}()
{{if eq .Database "mysql"}}
options:=append([]mysqladapter.Option{mysqladapter.WithLogger(logger)},c.DatabaseOptions()...)
c.DB,err=mysqladapter.Open(ctx,cfg.Database,options...);if err!=nil{return nil,err}
c.OnClose(c.DB.Close)
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.Pool.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
{{end}}
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
if err=c.Init(ctx);err!=nil{return nil,err}
return c,nil
}

View File

@@ -0,0 +1,23 @@
package servicecontext
import "errors"
// Close is idempotent, including startup failure and graceful shutdown.
func(c *Context)Close()error{
if c==nil{return nil}
c.closeOnce.Do(func(){
if c.Lifecycle!=nil{c.Lifecycle.RequestShutdown(nil)}
c.closeMu.Lock();c.closed=true;cleanups:=c.cleanups;c.cleanups=nil;c.closeMu.Unlock()
for i:=len(cleanups)-1;i>=0;i--{c.closeErr=errors.Join(c.closeErr,cleanups[i]())}
{{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}}
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
})
return c.closeErr
}
// OnClose transfers a resource cleanup to this context immediately after creation.
// Hooks run once in reverse order, also when later initialization fails.
func(c *Context)OnClose(cleanup func()error) {
if cleanup==nil{return}
c.closeMu.Lock();defer c.closeMu.Unlock()
if c.closed{panic("svc: register cleanup after Close")}
c.cleanups=append(c.cleanups,cleanup)
}

View File

@@ -0,0 +1,68 @@
package servicecontext
import (
"context"
"errors"
"sync"
{{if ne .Storage "none"}}"net/http"
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
{{if eq .Database "mysql"}}"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"github.com/go-sql-driver/mysql"{{end}}
"git.yuntuo.hk/k2safe/go-omni/health"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"{{.Module}}/internal/settings"
"{{.Module}}/internal/middleware"
)
// Context owns infrastructure, while Logger is borrowed from main.
// Add business repositories here; HTTP contracts do not define database tables.
type Context struct {
Lifecycle *lifecycle.Controller
Config settings.Config
cleanups []func()error
Logger *logx.Logger
Health *health.Checker
Middlewares map[string]httpx.Middleware
{{if eq .Database "mysql"}}DB *database.DB{{end}}
{{if ne .Storage "none"}}
Files storage.Store
Uploads *storage.Uploader
Downloads storage.Presigner
UploadLinks storage.UploadPresigner
FileHTTP http.Handler
storageClose func()error
{{end}}
closeMu sync.Mutex
closed bool
closeOnce sync.Once
closeErr error
}
// Pass the settings loaded by main. The optional form supports lightweight
// tests without infrastructure; selected adapters still require valid settings.
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
if ctx==nil{ctx=context.Background()}
if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))}
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
_=cfg
checker,err:=health.New();if err!=nil{return nil,err}
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Lifecycle:lifecycle.New(ctx)}
ctx=c.Lifecycle.Context()
defer func(){if err!=nil{err=errors.Join(err,c.Close());if cause:=c.Lifecycle.Err();cause!=nil && !errors.Is(err,cause){err=errors.Join(err,cause)}}}()
{{if eq .Database "mysql"}}
if err=cfg.Validate();err!=nil{return nil,err}
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
dsn.ParseTime=true;dsn.Loc=time.UTC
cfg.Database.DSN=dsn.FormatDSN()
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
{{end}}
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
return c,nil
}
// RequestShutdown reports a fatal dependency failure, or a normal stop when
// err is nil. It returns immediately; generated main drains and cleans up.
func(c *Context)RequestShutdown(err error){c.Lifecycle.RequestShutdown(err)}

View File

@@ -0,0 +1,11 @@
package servicecontext
import (
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
// AppOptions runs before routes are registered. Register project error catalogs
// here with omni.WithErrors. It must not create resources or another App.
func(c *Context)AppOptions()[]omni.Option{return nil}
// GlobalMiddleware runs for all endpoints, including App 404/405 fallbacks.
// Named route middleware lives in Middlewares and is selected by the API contract.
func(c *Context)GlobalMiddleware()[]httpx.Middleware{return nil}

View File

@@ -0,0 +1,59 @@
# Non-secret settings only. Environment variables override these values.
server:
address: '127.0.0.1:8080'
enableLogger: true
slowRequestThreshold: 500ms
shutdownTimeout: 10s
{{if .API}} endpoints:
liveness:
enabled: true
path: /health/live
responseMode: app
readiness:
enabled: true
path: /health/ready
responseMode: app
openapi:
enabled: true
path: /openapi.json
{{end}}
log:
service: {{printf "%q" .Module}}
level: info
format: {{.LogFormat}}
{{if eq .LogOutput "file"}} file: './var/log/server.log'
rotation:
enabled: true
maxSizeMb: 100
maxBackups: 10
maxAgeDays: 7
compress: true
{{else}} output: {{.LogOutput}}
{{end}} redactKeys: []
{{if ne .Storage "none"}}storage:
provider: {{.Storage}}
maxBytes: 33554432
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
linkTtl: 5m
requestTimeout: 30s
{{if eq .Storage "local"}} root: './var/files'
publicUrl: 'http://127.0.0.1:8080/storage/object'
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
secure: true
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
region: auto
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
region: us-east-1
{{else}} region: cn-hangzhou
endpoint: '' # empty uses the official regional endpoint
{{end}}{{end}}{{end}}
{{if and .API (eq .Database "mysql")}}database:
driver: mysql
maxOpenConns: 10
maxIdleConns: 5
connMaxLifetime: 30m
connMaxIdleTime: 5m
pingTimeout: 5s
# GO_OMNI_DB_DSN supplies the secret connection string.
{{end}}

View File

@@ -0,0 +1,140 @@
// Package settings owns this project's configuration; it adds no business identity model.
package settings
import (
"errors"
"os"
"strings"
"time"
{{if ne .Storage "none"}}"fmt"
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
"strconv"
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
type Config struct {
Server config.Config `json:"server"`
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
}
func Default() Config {
server:=config.Default()
server.Address="127.0.0.1:8080"
server.EnableLogger=true
server.SlowRequestThreshold=500*time.Millisecond
server.Log.Service={{printf "%q" .Module}}
server.Log.Format={{printf "%q" .LogFormat}}
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
{{if eq .Storage "r2"}}Region:"auto",{{end}}
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
}{{end}}}
}
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
// Pass an empty path to use only defaults and environment.
func Load(path string)(Config,error) {
cfg:=Default()
var err error
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
if err!=nil{return Config{},err}
// A stream explicitly selected by environment overrides a file profile.
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
}
{{if ne .Storage "none"}}
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
{{end}}
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
if err=cfg.Validate();err!=nil{return Config{},err}
return cfg,nil
}
func (c Config) Validate()error {
{{if and .API (eq .Database "mysql")}}
if err:=c.Database.Validate();err!=nil{return err}
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
{{end}}
if err:=c.Server.Validate();err!=nil{return err}
if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")}
switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")}
{{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}}
}
{{if ne .Storage "none"}}
type StorageConfig struct {
Provider string `json:"provider"`
Root string `json:"root"`
PublicURL string `json:"publicUrl"`
Endpoint string `json:"endpoint"`
Region string `json:"region"`
Bucket string `json:"bucket"`
AccountID string `json:"accountId"`
Secure bool `json:"secure"`
MaxBytes int64 `json:"maxBytes"`
ContentTypes []string `json:"contentTypes"`
LinkTTL time.Duration `json:"linkTtl"`
RequestTimeout time.Duration `json:"requestTimeout"`
// Secrets are environment-only and cannot be populated by config files.
FilesToken string `json:"-"`
SigningKey string `json:"-"`
AccessKey string `json:"-"`
SecretKey string `json:"-"`
SessionToken string `json:"-"`
}
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
func (s *StorageConfig) applyEnv()error {
values:=map[string]*string{
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
}
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
}
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
return nil
}
func (s StorageConfig) Validate()error {
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
if err:=s.Policy().Validate();err!=nil{return err}
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
{{if eq .Storage "local"}}
if s.Root=="" {return errors.New("storage.root is required")}
u,err:=url.Parse(s.PublicURL)
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
{{else}}
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
{{if eq .Storage "r2"}}
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
{{else if eq .Storage "s3"}}
u,err:=url.Parse("https://"+s.Endpoint)
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
{{else if eq .Storage "oss"}}
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
if !s.Secure{return errors.New("OSS requires secure=true")}
{{end}}
{{end}}
return nil
}
{{end}}

View File

@@ -0,0 +1,79 @@
package servicecontext
import (
"context"
"errors"
{{if eq .Storage "local"}}
"git.yuntuo.hk/k2safe/go-omni/storage/local"
{{else}}
"net"
"net/http"
"time"
{{if eq .Storage "oss"}}
ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss"
ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
{{else}}
"git.yuntuo.hk/k2safe/go-omni/storage/s3"
"github.com/minio/minio-go/v7/pkg/credentials"
{{end}}{{end}}
"git.yuntuo.hk/k2safe/go-omni/storage"
"{{.Module}}/internal/settings"
)
// InitStorage is called once, before serving requests. The composition root
// owns provider clients; business services receive only capability interfaces.
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
c.closeMu.Lock();defer c.closeMu.Unlock()
if c.closed{return errors.New("service dependencies are closed")}
if err:=ctx.Err();err!=nil{return err}
if c.Files!=nil{return errors.New("storage already initialized")}
if err:=cfg.Validate();err!=nil{return err}
{{if eq .Storage "local"}}
files,err:=local.New(cfg.Root,cfg.MaxBytes)
if err!=nil{return err}
signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy())
if err!=nil{return err}
uploader,err:=storage.NewUploader(files,cfg.Policy())
if err!=nil{return err}
c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer
{{else}}
// Dedicated bounded pool, never the process-wide default transport.
transport:=&http.Transport{
Proxy:http.ProxyFromEnvironment,
DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext,
ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32,
IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second,
ResponseHeaderTimeout:cfg.RequestTimeout,
}
initialized:=false
defer func(){if !initialized{transport.CloseIdleConnections()}}()
{{if eq .Storage "oss"}}
sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region).
WithSignatureVersion(ossSDK.SignatureVersionV4).
WithAdditionalHeaders([]string{"content-length"}).
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)).
WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout})
if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)}
files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes)
{{else}}
creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)
{{if eq .Storage "r2"}}
options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds)
if err!=nil{return err}
options.MaxBytes,options.Transport=cfg.MaxBytes,transport
{{else}}
options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport}
{{end}}
files,err:=s3.Open(options)
{{end}}
if err!=nil{return err}
uploader,err:=storage.NewUploader(files,cfg.Policy())
if err!=nil{return err}
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
initialized=true
{{end}}
return nil
}

View File

@@ -4,7 +4,11 @@ server:
enableLogger: true
slowRequestThreshold: 500ms
shutdownTimeout: 10s
{{if .API}} endpoints:
{{if .API}} lifecycle:
stopTimeout: 5s
drainTimeout: 10s
cancelTimeout: 5s
endpoints:
liveness:
enabled: true
path: /health/live
@@ -49,11 +53,16 @@ server:
{{end}}{{end}}{{end}}
{{if and .API (eq .Database "mysql")}}database:
driver: mysql
maxOpenConns: 10
maxIdleConns: 5
connMaxLifetime: 30m
connMaxIdleTime: 5m
pingTimeout: 5s
# GO_OMNI_DB_DSN supplies the secret connection string.
host: 127.0.0.1
port: 3306
database: '' # GO_OMNI_DB_DATABASE
user: '' # GO_OMNI_DB_USER; password via GO_OMNI_DB_PASSWORD
tls:
mode: required
pool:
maxOpenConns: 10
maxIdleConns: 5
connMaxLifetime: 30m
connMaxIdleTime: 5m
pingTimeout: 5s
{{end}}

View File

@@ -25,8 +25,8 @@ import (
// owns provider clients; business services receive only capability interfaces.
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
c.closeMu.Lock();defer c.closeMu.Unlock()
if c.closed{return errors.New("service dependencies are closed")}
{{if .API}}if c.Runtime==nil || c.Context().Err()!=nil{return errors.New("service dependencies are closed")}{{else}}c.closeMu.Lock();defer c.closeMu.Unlock()
if c.closed{return errors.New("service dependencies are closed")}{{end}}
if err:=ctx.Err();err!=nil{return err}
if c.Files!=nil{return errors.New("storage already initialized")}
if err:=cfg.Validate();err!=nil{return err}
@@ -72,7 +72,7 @@ func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)err
uploader,err:=storage.NewUploader(files,cfg.Policy())
if err!=nil{return err}
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
{{if .API}}c.OnClose(func()error{transport.CloseIdleConnections();return nil}){{else}}c.storageClose=func()error{transport.CloseIdleConnections();return nil}{{end}}
initialized=true
{{end}}
return nil

View File

@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Source builds can carry an unpublished VCS version (including +dirty).
// Keep this fallback in sync with the next release; installed CLIs use their
// downloaded module version, which also has a module checksum.
const defaultFrameworkVersion = "v0.13.0-preview"
const defaultFrameworkVersion = "v0.15.0-preview"
// This target is not a claim that the release already exists. Source users
// select a local framework checkout until the module templates are published.

44
cmd/go-omni/upgrade.go Normal file
View File

@@ -0,0 +1,44 @@
package main
import (
"errors"
"flag"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
"io"
)
func runUpgrade(args []string) error {
flags := flag.NewFlagSet("upgrade", flag.ContinueOnError)
flags.SetOutput(io.Discard)
dir := flags.String("dir", ".", "service directory")
apply := flags.Bool("apply", false, "apply only recognized baseline changes")
preview := flags.String("preview", "", "write proposed infrastructure into an empty directory")
dry := flags.Bool("dry-run", false, "show the upgrade plan")
if err := flags.Parse(args); err != nil {
return err
}
if flags.NArg() != 0 {
return errors.New("unexpected upgrade argument")
}
if *apply && (*preview != "" || *dry) {
return errors.New("--apply cannot be combined with preview/dry-run")
}
var plan apigen.UpgradePlan
var err error
if *apply {
plan, err = apigen.ApplyUpgrade(*dir)
} else if *preview != "" {
plan, err = apigen.PreviewUpgrade(*dir, *preview)
} else {
plan, err = apigen.PlanUpgrade(*dir)
}
for _, c := range plan.Changes {
kind := "safe"
if c.Manual {
kind = "review"
}
fmt.Printf("%-6s %-6s %s\n", kind, c.Action, c.Path)
}
return err
}

25
cmd/go-omni/workspace.go Normal file
View File

@@ -0,0 +1,25 @@
package main
import (
"errors"
"flag"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
"io"
"os"
)
func runWorkspace(args []string) error {
if len(args) < 1 {
return errors.New("usage: go-omni workspace <format|generate|check> --file omni.workspace.json")
}
flags := flag.NewFlagSet("workspace", flag.ContinueOnError)
flags.SetOutput(io.Discard)
path := flags.String("file", "omni.workspace.json", "workspace configuration")
if err := flags.Parse(args[1:]); err != nil {
return err
}
if flags.NArg() != 0 {
return errors.New("unexpected workspace argument")
}
return apigen.Workspace(*path, args[0], os.Stdout)
}

81
config/clone.go Normal file
View File

@@ -0,0 +1,81 @@
package config
import (
"errors"
"reflect"
"strings"
)
// Clone only configuration data. json:"-" capabilities remain borrowed and
// are never traversed (log writers, clients and mutex-bearing runtime objects).
// Unexported immutable implementation details such as time.Time are copied.
func cloneConfiguration(v reflect.Value, depth int) (reflect.Value, error) {
if depth > 64 {
return reflect.Value{}, errors.New("config: defaults contain cycles or excessive nesting")
}
result := reflect.New(v.Type()).Elem()
result.Set(v)
switch v.Kind() {
case reflect.Struct:
for i := 0; i < v.NumField(); i++ {
field := v.Type().Field(i)
if field.PkgPath != "" || strings.Split(field.Tag.Get("json"), ",")[0] == "-" {
continue
}
child, err := cloneConfiguration(v.Field(i), depth+1)
if err != nil {
return reflect.Value{}, err
}
result.Field(i).Set(child)
}
case reflect.Pointer:
if !v.IsNil() {
child, err := cloneConfiguration(v.Elem(), depth+1)
if err != nil {
return reflect.Value{}, err
}
result.Set(reflect.New(v.Type().Elem()))
result.Elem().Set(child)
}
case reflect.Interface:
if !v.IsNil() {
child, err := cloneConfiguration(v.Elem(), depth+1)
if err != nil {
return reflect.Value{}, err
}
result.Set(child)
}
case reflect.Slice:
if !v.IsNil() {
result = reflect.MakeSlice(v.Type(), v.Len(), v.Len())
for i := 0; i < v.Len(); i++ {
child, err := cloneConfiguration(v.Index(i), depth+1)
if err != nil {
return reflect.Value{}, err
}
result.Index(i).Set(child)
}
}
case reflect.Array:
for i := 0; i < v.Len(); i++ {
child, err := cloneConfiguration(v.Index(i), depth+1)
if err != nil {
return reflect.Value{}, err
}
result.Index(i).Set(child)
}
case reflect.Map:
if !v.IsNil() {
result = reflect.MakeMapWithSize(v.Type(), v.Len())
iter := v.MapRange()
for iter.Next() {
child, err := cloneConfiguration(iter.Value(), depth+1)
if err != nil {
return reflect.Value{}, err
}
result.SetMapIndex(iter.Key(), child)
}
}
}
return result, nil
}

View File

@@ -4,8 +4,10 @@ import (
"strings"
"time"
"git.yuntuo.hk/k2safe/go-omni/contract"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
)
@@ -26,20 +28,23 @@ type TLSConfig struct {
// Config controls the default runtime behavior of an App.
type Config struct {
Address string `json:"address"`
Transport string `json:"transport"`
TLS TLSConfig `json:"tls"`
HTTP HTTPConfig `json:"http"`
Endpoints EndpointsConfig `json:"endpoints"`
RequestIDHeader string `json:"requestIdHeader"`
EnableLogger bool `json:"enableLogger"`
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
TrustedProxies []string `json:"trustedProxies"`
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
Log logx.Config `json:"log"`
Errors apierrors.Catalog `json:"errors"`
Response response.Config `json:"response"`
ResponseEncoder response.Encoder `json:"-"`
Request contract.RequestPolicy `json:"request"`
Address string `json:"address"`
Transport string `json:"transport"`
TLS TLSConfig `json:"tls"`
HTTP HTTPConfig `json:"http"`
Endpoints EndpointsConfig `json:"endpoints"`
RequestIDHeader string `json:"requestIdHeader"`
EnableLogger bool `json:"enableLogger"`
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
TrustedProxies []string `json:"trustedProxies"`
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
Lifecycle lifecycle.ShutdownConfig `json:"lifecycle"`
Log logx.Config `json:"log"`
Errors apierrors.Catalog `json:"errors"`
Response response.Config `json:"response"`
ResponseEncoder response.Encoder `json:"-"`
LogInstance *logx.Logger `json:"-"`
}
func Default() Config {
@@ -48,6 +53,7 @@ func Default() Config {
Transport: TransportNetHTTP,
RequestIDHeader: defaultRequestIDHeader,
ShutdownTimeout: 10 * time.Second,
Lifecycle: lifecycle.DefaultShutdownConfig(),
Endpoints: DefaultEndpoints(),
Log: logx.Default(),
}
@@ -55,6 +61,15 @@ func Default() Config {
// Validate checks configuration values before an application starts.
func (c Config) Validate() error {
if err := c.Lifecycle.Validate(); err != nil {
return err
}
if err := c.Request.Validate(); err != nil {
return err
}
if err := c.Log.Validate(); err != nil {
return err
}
if err := c.Endpoints.Validate(); err != nil {
return err
}
@@ -221,3 +236,10 @@ func WithErrors(catalog apierrors.Catalog) Option {
catalog = catalog.Clone()
return func(cfg *Config) { cfg.Errors = catalog.Clone() }
}
// WithLogInstance borrows a logger for this App without changing process logging.
func WithLogInstance(logger *logx.Logger) Option { return func(c *Config) { c.LogInstance = logger } }
func WithRequestPolicy(policy contract.RequestPolicy) Option {
return func(c *Config) { c.Request = policy }
}

View File

@@ -77,6 +77,9 @@ func applyEnv(cfg Config, prefix string) (Config, error) {
{"HTTP_READ_TIMEOUT", &cfg.HTTP.ReadTimeout},
{"HTTP_WRITE_TIMEOUT", &cfg.HTTP.WriteTimeout},
{"HTTP_IDLE_TIMEOUT", &cfg.HTTP.IdleTimeout},
{"LIFECYCLE_STOP_TIMEOUT", &cfg.Lifecycle.StopTimeout},
{"LIFECYCLE_DRAIN_TIMEOUT", &cfg.Lifecycle.DrainTimeout},
{"LIFECYCLE_CANCEL_TIMEOUT", &cfg.Lifecycle.CancelTimeout},
} {
if value := strings.TrimSpace(os.Getenv(prefix + field.name)); value != "" {
*field.value, err = time.ParseDuration(value)

View File

@@ -22,6 +22,23 @@ import (
// field names for both formats, rejects unknown fields and accepts durations
// such as "3s". Application-specific validation remains with the caller.
func DecodeFile[T any](path string, defaults T) (T, error) {
result, err := decodeFile(path, defaults)
if err != nil {
var zero T
return zero, &DecodeError{cause: err}
}
return result, nil
}
// DecodeError keeps input values out of public diagnostics while preserving
// errors.Is/As for explicitly handled underlying I/O and syntax errors.
type DecodeError struct{ cause error }
func (e *DecodeError) Error() string {
return "config: cannot decode file; check its path, format, field names and value types"
}
func (e *DecodeError) Unwrap() error { return e.cause }
func decodeFile[T any](path string, defaults T) (T, error) {
var zero T
f, err := os.Open(path)
if err != nil {
@@ -64,15 +81,13 @@ func DecodeFile[T any](path string, defaults T) (T, error) {
if err != nil {
return zero, err
}
// Deep copy defaults so errors cannot partially mutate caller-owned maps.
original, err := json.Marshal(defaults)
// Clone configuration without serializing borrowed json:"-" capabilities.
var result T
original, err := cloneConfiguration(reflect.ValueOf(&defaults).Elem(), 0)
if err != nil {
return zero, err
}
var result T
if err = json.Unmarshal(original, &result); err != nil {
return zero, err
}
reflect.ValueOf(&result).Elem().Set(original)
dec := json.NewDecoder(bytes.NewReader(encoded))
dec.DisallowUnknownFields()
if err = dec.Decode(&result); err != nil {
@@ -115,6 +130,17 @@ func durations(v any, t reflect.Type) (any, error) {
}
}
}
if t.Kind() == reflect.Map && t.Key().Kind() == reflect.String {
if values, ok := v.(map[string]any); ok {
for key, value := range values {
converted, err := durations(value, t.Elem())
if err != nil {
return nil, err
}
values[key] = converted
}
}
}
if t.Kind() == reflect.Slice {
if items, ok := v.([]any); ok {
for n, item := range items {

37
config/lifecycle_test.go Normal file
View File

@@ -0,0 +1,37 @@
package config_test
import (
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"os"
"path/filepath"
"testing"
"time"
)
func TestLifecycleConfiguration(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.yaml")
if err := os.WriteFile(path, []byte("lifecycle:\n stopTimeout: 2s\n drainTimeout: 3s\n cancelTimeout: 4s\n"), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.LoadFile(path, "PHASE_TEST_")
if err != nil {
t.Fatal(err)
}
if cfg.Lifecycle != (lifecycle.ShutdownConfig{StopTimeout: 2 * time.Second, DrainTimeout: 3 * time.Second, CancelTimeout: 4 * time.Second}) {
t.Fatal(cfg.Lifecycle)
}
for _, key := range []string{"STOP_TIMEOUT", "DRAIN_TIMEOUT", "CANCEL_TIMEOUT"} {
t.Setenv("PHASE_TEST_LIFECYCLE_"+key, "6s")
}
cfg, err = config.ApplyEnv(cfg, "PHASE_TEST_")
if err != nil || cfg.Lifecycle.StopTimeout != 6*time.Second || cfg.Lifecycle.DrainTimeout != 6*time.Second || cfg.Lifecycle.CancelTimeout != 6*time.Second {
t.Fatal(cfg.Lifecycle, err)
}
for _, v := range []string{"bad", "-1s"} {
t.Setenv("PHASE_TEST_LIFECYCLE_DRAIN_TIMEOUT", v)
if _, err := config.ApplyEnv(cfg, "PHASE_TEST_"); err == nil {
t.Fatal("invalid duration accepted")
}
}
}

290
config/project.go Normal file
View File

@@ -0,0 +1,290 @@
package config
import (
"bytes"
"encoding"
"encoding/json"
"errors"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"math"
"os"
"reflect"
"strconv"
"strings"
"time"
"unicode"
)
// EnvOptions controls typed project configuration overlays. Explicit env tags
// are absolute names; otherwise JSON names become PREFIX_NESTED_FIELD names.
// Empty is allow (default), ignore or reject. No input values appear in errors.
type EnvOptions struct {
Prefix string
Empty string
Lookup func(string) (string, bool)
}
// ApplyDefaults applies envDefault tags to zero fields before file overlays.
func ApplyDefaults[T any](base T) (T, error) { return projectOverlay(base, EnvOptions{}, true) }
// LoadProject applies code/tag defaults, a strict optional file, environment,
// then the project's Validate method, before any logger or dependency is opened.
// envDefault is only applied to zero-valued defaults before reading the file.
func LoadProject[T any](path string, defaults T, options EnvOptions) (T, error) {
var zero T
cfg, err := projectOverlay(defaults, options, true)
if err != nil {
return zero, err
}
if path != "" {
cfg, err = DecodeFile(path, cfg)
if err != nil {
return zero, errors.New("config: cannot decode file; check its path, format, field names and value types")
}
}
cfg, err = OverlayEnv(cfg, options)
if err != nil {
return zero, err
}
if valid, ok := any(&cfg).(interface{ Validate() error }); ok {
if err = valid.Validate(); err != nil {
return zero, err
}
}
return cfg, nil
}
// OverlayEnv overlays the entire project struct without mutating the input.
// Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a
// subtree. required means nonempty string/non-nil pointer or collection; false
// and zero are valid scalars. Use project Validate for semantic constraints.
// Slices/maps use JSON syntax, avoiding ambiguous comma and empty semantics.
func OverlayEnv[T any](base T, options EnvOptions) (T, error) {
return projectOverlay(base, options, false)
}
var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
var durationType = reflect.TypeOf(time.Duration(0))
func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error) {
var zero T
if options.Lookup == nil {
options.Lookup = os.LookupEnv
}
if options.Empty == "" {
options.Empty = "allow"
}
if options.Empty != "allow" && options.Empty != "ignore" && options.Empty != "reject" {
return zero, errors.New("config: empty policy must be allow, ignore or reject")
}
v := reflect.ValueOf(&base).Elem()
if v.Kind() != reflect.Struct {
return zero, errors.New("config: project configuration must be a struct")
}
result := reflect.New(v.Type()).Elem()
result.Set(v)
seen := map[string]bool{}
var walk func(reflect.Value, string, int) error
walk = func(value reflect.Value, prefix string, depth int) error {
if depth > 32 {
return errors.New("config: nesting exceeds 32")
}
for i := 0; i < value.NumField(); i++ {
field := value.Type().Field(i)
dst := value.Field(i)
if !dst.CanSet() || field.PkgPath != "" {
continue
}
tag := strings.Split(field.Tag.Get("env"), ",")
if tag[0] == "-" {
continue
}
jsonName := strings.Split(field.Tag.Get("json"), ",")[0]
if jsonName == "-" && tag[0] == "" {
continue
}
if jsonName == "" {
jsonName = field.Name
}
key := prefix + envName(jsonName)
if tag[0] != "" {
key = tag[0]
}
empty, required := options.Empty, false
for _, opt := range tag[1:] {
switch opt {
case "required":
required = true
case "secret", "":
case "empty=allow":
empty = "allow"
case "empty=ignore":
empty = "ignore"
case "empty=reject":
empty = "reject"
default:
return fmt.Errorf("config: invalid env tag for %s", key)
}
}
if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) {
if err := walk(dst, key+"_", depth+1); err != nil {
return err
}
continue
}
if dst.Kind() == reflect.Pointer && dst.Type().Elem().Kind() == reflect.Struct && !dst.Type().Implements(textType) {
// Clone before traversing so a failed overlay never edits defaults.
child := reflect.New(dst.Type().Elem())
if !dst.IsNil() {
child.Elem().Set(dst.Elem())
}
if err := walk(child.Elem(), key+"_", depth+1); err != nil {
return err
}
if !dst.IsNil() || !child.Elem().IsZero() {
dst.Set(child)
}
if required && !defaults && dst.IsNil() {
return fmt.Errorf("config: %s is required", key)
}
continue
}
// Unexported runtime capabilities are skipped unless explicitly tagged.
if dst.Kind() == reflect.Func || dst.Kind() == reflect.Chan || dst.Kind() == reflect.Interface {
if tag[0] != "" {
return fmt.Errorf("config: %s has unsupported environment type", key)
}
continue
}
if seen[key] {
return fmt.Errorf("config: duplicate environment key %s", key)
}
seen[key] = true
raw, present := options.Lookup(key)
if defaults {
raw, present = field.Tag.Lookup("envDefault")
present = present && dst.IsZero()
}
if present && raw == "" && !defaults {
if empty == "reject" {
return fmt.Errorf("config: %s must not be empty", key)
}
if empty == "ignore" {
present = false
}
}
if present {
parsed := reflect.New(dst.Type()).Elem()
if err := parseEnvValue(parsed, raw); err != nil {
return fmt.Errorf("config: %s must be a valid %s", key, dst.Type())
}
dst.Set(parsed)
}
if required && !defaults && emptyRequired(dst) {
return fmt.Errorf("config: %s is required", key)
}
}
return nil
}
if err := walk(result, options.Prefix, 0); err != nil {
return zero, err
}
return result.Interface().(T), nil
}
func emptyRequired(v reflect.Value) bool {
switch v.Kind() {
case reflect.String:
return v.Len() == 0
case reflect.Pointer, reflect.Map, reflect.Slice:
return v.IsNil()
}
return false
}
func parseEnvValue(v reflect.Value, raw string) (err error) {
defer func() {
if recover() != nil {
err = errors.New("invalid custom value")
}
}()
if v.Kind() == reflect.Pointer {
v.Set(reflect.New(v.Type().Elem()))
return parseEnvValue(v.Elem(), raw)
}
if v.CanAddr() && v.Addr().Type().Implements(textType) {
return v.Addr().Interface().(encoding.TextUnmarshaler).UnmarshalText([]byte(raw))
}
if v.Type() == durationType {
d, err := time.ParseDuration(raw)
if err == nil {
v.SetInt(int64(d))
}
return err
}
switch v.Kind() {
case reflect.String:
v.SetString(raw)
return nil
case reflect.Bool:
x, err := strconv.ParseBool(raw)
if err == nil {
v.SetBool(x)
}
return err
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
x, err := strconv.ParseInt(raw, 10, v.Type().Bits())
if err == nil {
v.SetInt(x)
}
return err
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
x, err := strconv.ParseUint(raw, 10, v.Type().Bits())
if err == nil {
v.SetUint(x)
}
return err
case reflect.Float32, reflect.Float64:
x, err := strconv.ParseFloat(raw, v.Type().Bits())
if err == nil && (math.IsNaN(x) || math.IsInf(x, 0)) {
return errors.New("nonfinite value")
}
if err == nil {
v.SetFloat(x)
}
return err
case reflect.Slice, reflect.Map:
var value any
if err := apierrors.DecodeCatalogJSON([]byte(raw), &value); err != nil {
return err
}
value, err := durations(value, v.Type())
if err != nil {
return err
}
encoded, err := json.Marshal(value)
if err != nil {
return err
}
decoder := json.NewDecoder(bytes.NewReader(encoded))
decoder.DisallowUnknownFields()
return decoder.Decode(v.Addr().Interface())
}
return errors.New("unsupported environment value")
}
func envName(s string) string {
r := []rune(s)
var out strings.Builder
for i, c := range r {
if i > 0 && unicode.IsUpper(c) && (unicode.IsLower(r[i-1]) || unicode.IsDigit(r[i-1]) || (i+1 < len(r) && unicode.IsLower(r[i+1]))) {
out.WriteByte('_')
}
if c == '-' {
c = '_'
}
out.WriteRune(unicode.ToUpper(c))
}
return out.String()
}

View File

@@ -0,0 +1,61 @@
package config
import (
"bytes"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func TestConfigFilePreservesDefaultsWithoutSharingMutableData(t *testing.T) {
type cfg struct {
Writer io.Writer `json:"-"`
Secret string `json:"-"`
Values map[string]int64 `json:"values"`
Timers map[string]time.Duration `json:"timers"`
ID int64 `json:"id"`
}
var writer bytes.Buffer
base := cfg{Writer: &writer, Secret: "borrowedSecret", Values: map[string]int64{"old": 7}, ID: 9007199254740993}
file := filepath.Join(t.TempDir(), "config.json")
if err := os.WriteFile(file, []byte(`{"values":{"new":2},"timers":{"retry":"2s"}}`), 0600); err != nil {
t.Fatal(err)
}
next, err := DecodeFile(file, base)
if err != nil {
t.Fatal(err)
}
if next.Writer != &writer || next.Secret != base.Secret || next.ID != base.ID || next.Timers["retry"] != 2*time.Second || len(base.Values) != 1 {
t.Fatal("defaults lost or mutated")
}
next.Values["old"] = 9
if base.Values["old"] != 7 {
t.Fatal("defaults shared map")
}
if err = os.WriteFile(file, []byte(`{"values":{"secretValue":"credential-super-secret"}}`), 0600); err != nil {
t.Fatal(err)
}
if _, err = DecodeFile(file, base); err == nil || strings.Contains(err.Error(), "credential-super-secret") {
t.Fatal("decode error leaked contents", err)
}
}
func TestEnvCollectionPrecisionAndDurationMaps(t *testing.T) {
type cfg struct {
IDs []int64 `json:"ids"`
Timers map[string]time.Duration `json:"timers"`
}
lookup := func(key string) (string, bool) {
v, ok := map[string]string{"IDS": "[9007199254740993]", "TIMERS": `{"backoff":"250ms"}`}[key]
return v, ok
}
c, err := OverlayEnv(cfg{}, EnvOptions{Lookup: lookup})
if err != nil || c.IDs[0] != 9007199254740993 || c.Timers["backoff"] != 250*time.Millisecond {
t.Fatal(c, err)
}
if _, err = OverlayEnv(cfg{}, EnvOptions{Lookup: func(key string) (string, bool) { return `{"x":"1s","x":"2s"}`, key == "TIMERS" }}); err == nil {
t.Fatal("duplicate map key accepted")
}
}

124
config/project_test.go Normal file
View File

@@ -0,0 +1,124 @@
package config
import (
"errors"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
type projectNested struct {
ReadTimeout time.Duration `json:"readTimeout"`
Names []string `json:"names"`
}
type projectConfig struct {
Nested *projectNested `json:"nested"`
Enabled bool `json:"enabled"`
Count int `json:"count" envDefault:"12"`
Password string `json:"-" env:"ETAX_PASSWORD,secret,required,empty=reject"`
Label string `json:"label" env:",empty=ignore"`
}
func (c projectConfig) Validate() error {
if c.Count < 0 {
return errors.New("count must be nonnegative")
}
return nil
}
func TestProjectConfigurationOrderAndValues(t *testing.T) {
f := filepath.Join(t.TempDir(), "config.yaml")
if err := os.WriteFile(f, []byte("count: 3\nenabled: true\nnested:\n readTimeout: 2s\n names: [file]\nlabel: keep\n"), 0600); err != nil {
t.Fatal(err)
}
env := map[string]string{"ETAX_PASSWORD": " password ", "APP_ENABLED": "false", "APP_COUNT": "0", "APP_NESTED_READ_TIMEOUT": "7s", "APP_NESTED_NAMES": "[]", "APP_LABEL": ""}
options := EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
cfg, err := LoadProject(f, projectConfig{}, options)
if err != nil {
t.Fatal(err)
}
if cfg.Enabled || cfg.Count != 0 || cfg.Password != " password " || cfg.Nested.ReadTimeout != 7*time.Second || cfg.Nested.Names == nil || len(cfg.Nested.Names) != 0 || cfg.Label != "keep" {
t.Fatal("overlay mismatch")
}
delete(env, "APP_COUNT")
cfg, err = LoadProject("", projectConfig{}, options)
if err != nil || cfg.Count != 12 {
t.Fatal(cfg.Count, err)
}
env["APP_COUNT"] = "-1"
if _, err = LoadProject(f, projectConfig{}, options); err == nil {
t.Fatal("project Validate not called")
}
}
func TestProjectOverlayIsTransactionalAndRedacted(t *testing.T) {
base := projectConfig{Nested: &projectNested{ReadTimeout: time.Second, Names: []string{"old"}}, Password: "original"}
env := map[string]string{"APP_NESTED_READ_TIMEOUT": "2s", "APP_NESTED_NAMES": "[\"new\"]", "APP_COUNT": "secret-value"}
_, err := OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }})
if err == nil || strings.Contains(err.Error(), "secret-value") {
t.Fatal(err)
}
if base.Nested.ReadTimeout != time.Second || !reflect.DeepEqual(base.Nested.Names, []string{"old"}) {
t.Fatal("default pointer modified")
}
delete(env, "APP_COUNT")
env["ETAX_PASSWORD"] = ""
if _, err = OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}); err == nil {
t.Fatal("empty secret accepted")
}
type required struct {
Token string `env:"TOKEN,required"`
}
if _, err = OverlayEnv(required{}, EnvOptions{Lookup: func(string) (string, bool) { return "", false }}); err == nil {
t.Fatal("required absent")
}
type duplicate struct {
A string `env:"SAME"`
B string `env:"SAME"`
}
if _, err = OverlayEnv(duplicate{}, EnvOptions{}); err == nil {
t.Fatal("duplicate env binding accepted")
}
}
func TestEnvironmentCollectionAndEmptySemantics(t *testing.T) {
type sample struct {
Values []time.Duration
Flag bool
Text string
Optional *int
}
cases := []struct {
raw string
valid bool
}{{`["1s","2s"]`, true}, {`["invalid-secret"]`, false}}
for _, tc := range cases {
cfg, err := OverlayEnv(sample{}, EnvOptions{Lookup: func(k string) (string, bool) {
if k == "VALUES" {
return tc.raw, true
}
return "", false
}})
if (err == nil) != tc.valid {
t.Fatal(err)
}
if tc.valid && !reflect.DeepEqual(cfg.Values, []time.Duration{time.Second, 2 * time.Second}) {
t.Fatal(cfg)
}
if err != nil && strings.Contains(err.Error(), "secret") {
t.Fatal(err)
}
}
for _, mode := range []string{"allow", "ignore", "reject"} {
cfg, err := OverlayEnv(sample{Text: "old"}, EnvOptions{Empty: mode, Lookup: func(k string) (string, bool) { return "", k == "TEXT" }})
if mode == "reject" {
if err == nil {
t.Fatal("empty not rejected")
}
} else if err != nil || (mode == "allow" && cfg.Text != "") || (mode == "ignore" && cfg.Text != "old") {
t.Fatal(cfg, err)
}
}
}

View File

@@ -1,6 +1,7 @@
package contract
import (
"bufio"
"encoding/json"
"errors"
"fmt"
@@ -11,6 +12,7 @@ import (
"net/url"
"reflect"
"strconv"
"strings"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
)
@@ -19,9 +21,17 @@ import (
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
// parameters never overwrite path/body/header values. form means URL query only.
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
return b.BindWithPolicy(r, params, target, RequestPolicy{}, maxBytes...)
}
// BindWithPolicy merges route overrides over the App request policy.
func (b *Binder) BindWithPolicy(r *http.Request, params map[string]string, target any, policy RequestPolicy, maxBytes ...int64) error {
if b == nil || r == nil || b.schemaOnly {
return apierrors.Internal(errors.New("contract: nil binder or request"))
}
if b.HasMultipart() {
return apierrors.Internal(errors.New("multipart fields require BindMultipart"))
}
v := reflect.ValueOf(target)
if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ {
return apierrors.Internal(errors.New("contract: target type mismatch"))
@@ -33,54 +43,9 @@ func (b *Binder) Bind(r *http.Request, params map[string]string, target any, max
if len(maxBytes) == 1 {
limit = maxBytes[0]
}
fresh := reflect.New(b.typ).Elem()
body := map[string]any{}
if len(b.root.schema.Properties) > 0 {
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
if err != nil || ct != "application/json" {
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json")
}
if r.Body == nil {
return invalid(errors.New("body required"))
}
if r.ContentLength > limit {
return tooLarge()
}
reader := &io.LimitedReader{R: r.Body, N: limit + 1}
dec := json.NewDecoder(reader)
dec.UseNumber()
value, err := decodeValue(dec, 0)
if reader.N == 0 {
return tooLarge()
}
if err != nil {
var maxErr *http.MaxBytesError
if errors.As(err, &maxErr) {
return tooLarge()
}
return invalid(err)
}
_, err = dec.Token()
if reader.N == 0 {
return tooLarge()
}
if err != io.EOF {
var maxErr *http.MaxBytesError
if errors.As(err, &maxErr) {
return tooLarge()
}
return invalid(errors.New("exactly one JSON value required"))
}
var ok bool
body, ok = value.(map[string]any)
if !ok {
return invalid(errors.New("body must be an object"))
}
for key := range body {
if _, ok = b.root.schema.Properties[key]; !ok {
return invalid(fmt.Errorf("unknown body field %s", key))
}
}
policy = MergePolicy(RequestPolicyFromContext(r.Context()), policy)
if err := policy.Validate(); err != nil {
return apierrors.Internal(err)
}
query := url.Values{}
if r.URL != nil {
@@ -90,10 +55,121 @@ func (b *Binder) Bind(r *http.Request, params map[string]string, target any, max
return invalid(errors.New("malformed query"))
}
}
if policy.UnknownQuery == "reject" {
allowed := map[string]bool{}
for _, f := range b.root.fields {
if f.source == "form" {
allowed[f.name] = true
}
}
for key := range query {
if !allowed[key] {
return invalid(fmt.Errorf("unknown query field %s", key))
}
}
}
fresh := reflect.New(b.typ).Elem()
body := map[string]any{}
bodyAbsent := false
bodyMode := policy.Body
if bodyMode == "" || bodyMode == "auto" {
if len(b.root.schema.Properties) > 0 {
bodyMode = "required"
} else {
bodyMode = "auto"
}
}
if (bodyMode == "none" || bodyMode == "emptyObject") && len(b.root.schema.Properties) > 0 {
return apierrors.Internal(errors.New("request body policy conflicts with JSON fields"))
}
if bodyMode == "none" {
if r.Body != nil {
var one [1]byte
n, err := io.ReadFull(r.Body, one[:])
if n > 0 {
return invalid(errors.New("request body is forbidden"))
}
if err != io.EOF {
return invalid(err)
}
}
}
if bodyMode == "required" || bodyMode == "optional" || bodyMode == "emptyObject" {
var bodyReader io.Reader = r.Body
if bodyReader == nil {
bodyReader = strings.NewReader("")
}
if r.ContentLength > limit {
return tooLarge()
}
reader := &io.LimitedReader{R: bodyReader, N: limit + 1}
var checked io.Reader = reader
if policy.UTF8 == "reject" {
checked = &utf8Reader{r: checked}
}
buffered := bufio.NewReader(checked)
_, peekErr := buffered.Peek(1)
absent := peekErr == io.EOF
bodyAbsent = absent
if absent && bodyMode != "optional" {
return invalid(errors.New("body required"))
}
if peekErr != nil && !absent {
var maxErr *http.MaxBytesError
if errors.As(peekErr, &maxErr) {
return tooLarge()
}
return invalid(peekErr)
}
if !absent {
ct, mediaParams, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
if err != nil || ct != "application/json" || (policy.Charset == "utf8" && mediaParams["charset"] != "" && !strings.EqualFold(mediaParams["charset"], "utf-8")) {
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json with an allowed charset")
}
dec := json.NewDecoder(buffered)
dec.UseNumber()
value, err := decodeValue(dec, 0)
if reader.N == 0 {
return tooLarge()
}
if err != nil {
var maxErr *http.MaxBytesError
if errors.As(err, &maxErr) {
return tooLarge()
}
return invalid(err)
}
_, err = dec.Token()
if reader.N == 0 {
return tooLarge()
}
if err != io.EOF {
var maxErr *http.MaxBytesError
if errors.As(err, &maxErr) {
return tooLarge()
}
return invalid(errors.New("exactly one JSON value required"))
}
var ok bool
body, ok = value.(map[string]any)
if !ok {
return invalid(errors.New("body must be an object"))
}
for key := range body {
if _, ok = b.root.schema.Properties[key]; !ok {
return invalid(fmt.Errorf("unknown body field %s", key))
}
}
}
}
for _, f := range b.root.fields {
var value any
var present bool
if f.source == "json" {
if bodyAbsent && bodyMode == "optional" {
continue
}
value, present = body[f.name]
} else {
var raw []string

View File

@@ -36,6 +36,8 @@ type node struct {
}
type field struct {
index int
maxBytes int64
maxFiles int
name, source string
value *node
optional bool
@@ -113,6 +115,13 @@ func (c compiler) compile(t reflect.Type, root bool, stack map[reflect.Type]bool
if t.Kind() != reflect.Pointer && (t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler)) {
return nil, errors.New("contract: custom JSON marshalers require explicit Scalar registration")
}
if t == uploadType {
if root {
return nil, errors.New("upload File cannot be a root DTO")
}
n.schema = &openapi.Schema{Type: "string", Format: "binary"}
return n, nil
}
switch t.Kind() {
case reflect.Interface:
if t.NumMethod() != 0 {
@@ -223,7 +232,7 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
var sourceTag string
for key, value := range tags {
switch key {
case "json", "path", "form", "header":
case "json", "path", "form", "header", "file", "multipart", "multipartJSON":
if value == "-" {
return f, errors.New("ignored fields are not part of an API contract")
}
@@ -232,6 +241,8 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
}
f.source = key
sourceTag = value
case "description", "example", "enumDescriptions":
// Documentation is validated after the field constraints.
case "binding":
f.binding = value
default:
@@ -254,6 +265,45 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
return f, err
}
f.value = n
if f.source == "file" {
if sf.Type != uploadType && (sf.Type.Kind() != reflect.Slice || sf.Type.Elem() != uploadType) {
return f, errors.New("file field requires File or []File")
}
seen := map[string]bool{}
for _, option := range parts[1:] {
key, v, _ := strings.Cut(option, "=")
if seen[key] {
return f, errors.New("duplicate file option")
}
seen[key] = true
switch key {
case "optional":
if option != key {
return f, errors.New("optional takes no value")
}
f.optional = true
case "maxBytes":
f.maxBytes, err = fieldLimit(v)
if err != nil {
return f, err
}
case "maxFiles":
f.maxFiles, err = strconv.Atoi(v)
if err != nil || f.maxFiles < 1 || f.maxFiles > 10000 || sf.Type.Kind() != reflect.Slice {
return f, errors.New("maxFiles requires []File and a limit 1..10000")
}
default:
return f, fmt.Errorf("unsupported file option %q", key)
}
}
if f.binding != "" {
return f, errors.New("file constraints use optional/maxBytes/maxFiles")
}
return f, applyDocumentation(&f, tags)
}
if containsUpload(n) {
return f, errors.New("File requires file source")
}
opaque := base(n).scalar
isList := base(n).typ.Kind() == reflect.Slice && opaque == nil
if isList {
@@ -263,25 +313,35 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
if isList && (f.source == "path" || f.source == "header") {
return f, errors.New("path/header values must be scalars")
}
if f.source != "json" && !opaque.Text {
if !jsonSource(f.source) && !opaque.Text {
return f, errors.New("custom scalar needs text: true and UnmarshalText for path/query/header")
}
if f.binding != "" {
return f, errors.New("custom scalar constraints belong in its decoder; use optional/pointer for presence and null")
}
for _, option := range parts[1:] {
if strings.HasPrefix(option, "maxBytes=") && multipartSource(f.source) {
f.maxBytes, err = fieldLimit(strings.TrimPrefix(option, "maxBytes="))
if err != nil {
return f, err
}
continue
}
if option != "optional" && option != "omitempty" {
return f, errors.New("custom scalar supports only optional/omitempty field options")
return f, errors.New("custom scalar supports optional/omitempty and multipart maxBytes")
}
f.optional = true
}
if f.source == "path" && (f.optional || sf.Type.Kind() == reflect.Pointer) {
return f, errors.New("path fields must be required non-pointer values")
}
if err := applyDocumentation(&f, tags); err != nil {
return f, err
}
return f, nil
}
scalar := base(n).typ.Kind()
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
if !jsonSource(f.source) && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
return f, errors.New("path/query/header values must be scalars or scalar slices")
}
if f.source == "path" && !isScalar(scalar) {
@@ -303,6 +363,14 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
return f, errors.New("optional takes no value")
}
f.optional = true
case "maxBytes":
if !multipartSource(f.source) {
return f, errors.New("maxBytes field option is only for multipart")
}
f.maxBytes, err = fieldLimit(value)
if err != nil {
return f, err
}
case "default":
if !isScalar(scalar) {
return f, errors.New("default requires a scalar")
@@ -354,6 +422,9 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
return f, fmt.Errorf("default violates validation: %w", err)
}
}
if err := applyDocumentation(&f, tags); err != nil {
return f, err
}
return f, nil
}
@@ -432,7 +503,7 @@ func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
func (b *Binder) Parameters() []openapi.Parameter {
var out []openapi.Parameter
for _, f := range b.root.fields {
if f.source == "json" {
if f.source == "json" || multipartSource(f.source) {
continue
}
in := f.source
@@ -445,7 +516,7 @@ func (b *Binder) Parameters() []openapi.Parameter {
} else if base(f.value).typ.Kind() == reflect.Slice && base(base(f.value).elem).scalar != nil {
schema.Items = &openapi.Schema{Type: "string"}
}
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema})
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema, Description: f.value.schema.Description})
}
return out
}
@@ -455,3 +526,10 @@ func copySchema(s *openapi.Schema) *openapi.Schema {
_ = json.Unmarshal(data, &out)
return &out
}
func containsUpload(n *node) bool {
if n.typ == uploadType {
return true
}
return n.elem != nil && containsUpload(n.elem)
}

174
contract/documentation.go Normal file
View File

@@ -0,0 +1,174 @@
package contract
import (
"encoding/json"
"errors"
"fmt"
"io"
"reflect"
"strings"
)
// Documentation is part of the contract compiler, so CLI and runtime agree.
func applyDocumentation(f *field, tags map[string]string) error {
s := f.value.schema
if f.maxBytes > 0 {
v := f.maxBytes
s.MaxBytes = &v
}
if f.source == "file" && f.value.typ.Kind() == reflect.Slice {
v := f.maxFiles
if v == 0 {
v = 16
}
s.MaxItems = &v
}
s.Description = tags["description"]
if raw, ok := tags["enumDescriptions"]; ok {
if err := json.Unmarshal([]byte(raw), &s.EnumDescriptions); err != nil {
return errors.New("enumDescriptions must be a JSON string array")
}
count := len(s.Enum)
if s.Nullable && count > 0 && s.Enum[count-1] == nil {
count--
}
if count == 0 || len(s.EnumDescriptions) != count {
return errors.New("enumDescriptions must describe every declared enum value")
}
for _, v := range s.EnumDescriptions {
if strings.TrimSpace(v) == "" {
return errors.New("enum description cannot be empty")
}
}
}
if raw, ok := tags["example"]; ok {
var value any
if base(f.value).typ.Kind() == reflect.String && base(f.value).scalar == nil {
value = raw
} else {
decoder := json.NewDecoder(strings.NewReader(raw))
decoder.UseNumber()
var err error
value, err = decodeValue(decoder, 0)
if err != nil {
return errors.New("invalid example JSON")
}
if _, err = decoder.Token(); err != io.EOF {
return errors.New("example requires one value")
}
}
// A schema-only custom scalar never executes project decoders.
if !documentationHasScalar(f.value) {
v := reflect.New(f.value.typ).Elem()
if err := setValue(f.value, v, value); err != nil {
return fmt.Errorf("example: %w", err)
}
if err := checkField(*f, v); err != nil {
return fmt.Errorf("example: %w", err)
}
} else if err := checkDocumentationScalarTree(f.value, value); err != nil {
return err
}
s.Example = value
}
return nil
}
func documentationHasScalar(n *node) bool {
if n.scalar != nil {
return true
}
if n.elem != nil && documentationHasScalar(n.elem) {
return true
}
for _, f := range n.fields {
if documentationHasScalar(f.value) {
return true
}
}
return false
}
// Metadata never invokes application decoders, including nested scalar types.
func checkDocumentationScalarTree(n *node, value any) error {
if value == nil {
if n.typ.Kind() == reflect.Pointer || n.typ.Kind() == reflect.Interface {
return nil
}
return errors.New("example null requires a nullable field")
}
if n.typ.Kind() == reflect.Pointer {
return checkDocumentationScalarTree(n.elem, value)
}
if n.scalar != nil {
for _, kind := range strings.Split(n.scalar.JSONInput, "|") {
if scalarAccepts(strings.TrimSpace(kind), value) {
return nil
}
}
return errors.New("example does not match scalar input")
}
if !documentationHasScalar(n) {
v := reflect.New(n.typ).Elem()
return setValue(n, v, value)
}
switch n.typ.Kind() {
case reflect.Slice:
values, ok := value.([]any)
if !ok {
return errors.New("example requires an array")
}
if n.schema.MinItems != nil && len(values) < *n.schema.MinItems || n.schema.MaxItems != nil && len(values) > *n.schema.MaxItems {
return errors.New("example violates item limits")
}
for _, v := range values {
if err := checkDocumentationScalarTree(n.elem, v); err != nil {
return err
}
}
case reflect.Map:
values, ok := value.(map[string]any)
if !ok {
return errors.New("example requires an object")
}
for _, v := range values {
if err := checkDocumentationScalarTree(n.elem, v); err != nil {
return err
}
}
case reflect.Struct:
values, ok := value.(map[string]any)
if !ok {
return errors.New("example requires an object")
}
declared := map[string]bool{}
for _, f := range n.fields {
declared[f.name] = true
v, ok := values[f.name]
if !ok {
if !f.optional && !f.hasDefault {
return errors.New("example missing field")
}
continue
}
if err := checkDocumentationScalarTree(f.value, v); err != nil {
return err
}
if !documentationHasScalar(f.value) {
typed := reflect.New(f.value.typ).Elem()
if err := setValue(f.value, typed, v); err != nil {
return err
}
if err := checkField(f, typed); err != nil {
return err
}
}
}
for name := range values {
if !declared[name] {
return errors.New("unknown example field")
}
}
}
return nil
}

View File

@@ -0,0 +1,55 @@
package contract
import (
"encoding/json"
"reflect"
"testing"
)
type documentationScalar string
var documentationDecodeCalls int
func (v documentationScalar) MarshalJSON() ([]byte, error) { return json.Marshal(string(v)) }
func (v *documentationScalar) UnmarshalJSON(raw []byte) error {
documentationDecodeCalls++
var s string
err := json.Unmarshal(raw, &s)
*v = documentationScalar(s)
return err
}
func TestDocumentationNeverRunsNestedScalarDecoders(t *testing.T) {
type nested struct {
ID documentationScalar `json:"id"`
}
type input struct {
Value nested `json:"value" example:"{\"id\":\"123\"}"`
}
documentationDecodeCalls = 0
scalar := ScalarOf[documentationScalar]("string", "string", false)
if _, err := New[input](scalar); err != nil {
t.Fatal(err)
}
if _, err := Describe(reflect.TypeOf(input{}), scalar); err != nil {
t.Fatal(err)
}
if documentationDecodeCalls != 0 {
t.Fatal("metadata invoked business decoder")
}
type bad struct {
Value nested `json:"value" example:"{\"id\":123}"`
}
if _, err := Describe(reflect.TypeOf(bad{}), scalar); err == nil {
t.Fatal("invalid scalar wire example accepted")
}
type constrained struct {
ID documentationScalar `json:"id"`
Count int `json:"count,range=[1:3]"`
}
type invalidConstraint struct {
Value constrained `json:"value" example:"{\"id\":\"123\",\"count\":9}"`
}
if _, err := Describe(reflect.TypeOf(invalidConstraint{}), scalar); err == nil {
t.Fatal("nested scalar bypassed another field's example constraints")
}
}

80
contract/file.go Normal file
View File

@@ -0,0 +1,80 @@
package contract
import (
"errors"
"io"
"os"
"sync"
)
// File is a request-scoped upload, staged on disk using a bounded copy buffer.
// Name and ContentType are untrusted client metadata. Open streams content;
// readers and the temporary file are closed when the generated handler returns.
// Persist wanted content before returning. Retaining File does not extend its lifetime.
type File struct {
Name string
ContentType string
Size int64
state *uploadState
}
type uploadState struct {
mu sync.Mutex
path string
closed bool
readers map[*uploadReader]bool
}
type uploadReader struct {
file *os.File
once sync.Once
err error
state *uploadState
}
// Present reports whether the optional upload was supplied.
func (f File) Present() bool { return f.state != nil }
func (f File) Open() (io.ReadCloser, error) {
if f.state == nil {
return nil, errors.New("upload file is absent")
}
s := f.state
s.mu.Lock()
defer s.mu.Unlock()
if s.closed {
return nil, errors.New("upload file is closed")
}
file, err := os.Open(s.path)
if err != nil {
return nil, err
}
r := &uploadReader{file: file, state: s}
s.readers[r] = true
return r, nil
}
func (r *uploadReader) Read(p []byte) (int, error) { return r.file.Read(p) }
func (r *uploadReader) Close() error {
r.once.Do(func() { r.err = r.file.Close(); r.state.mu.Lock(); delete(r.state.readers, r); r.state.mu.Unlock() })
return r.err
}
func (f File) cleanup() error {
if f.state == nil {
return nil
}
s := f.state
s.mu.Lock()
if s.closed {
s.mu.Unlock()
return nil
}
s.closed = true
readers := make([]*uploadReader, 0, len(s.readers))
for r := range s.readers {
readers = append(readers, r)
}
s.mu.Unlock()
var err error
for _, r := range readers {
err = errors.Join(err, r.Close())
}
return errors.Join(err, os.Remove(s.path))
}

331
contract/multipart.go Normal file
View File

@@ -0,0 +1,331 @@
package contract
import (
"encoding/json"
"errors"
"fmt"
"io"
"mime"
"net/http"
"os"
"path"
"reflect"
"strconv"
"strings"
"sync"
"unicode/utf8"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/openapi"
)
var uploadType = reflect.TypeOf(File{})
// MultipartOptions limits raw request bytes (including headers), part count,
// and individual decoded part sizes. Zero selects bounded defaults. Files are
// spooled to 0600 temporary files; text/JSON parts are buffered up to MaxFieldBytes.
type MultipartOptions struct {
MaxBodyBytes int64
MaxParts int
MaxFieldBytes int64
MaxFileBytes int64
TempDir string
Policy RequestPolicy
}
// Normalized returns validated effective protocol limits without allocating resources.
func (o MultipartOptions) Normalized() (MultipartOptions, error) {
if o.MaxBodyBytes == 0 {
o.MaxBodyBytes = DefaultMaxBodyBytes
}
if o.MaxParts == 0 {
o.MaxParts = 128
}
if o.MaxFieldBytes == 0 {
o.MaxFieldBytes = 64 << 10
}
if o.MaxFileBytes == 0 {
o.MaxFileBytes = o.MaxBodyBytes
}
if o.MaxBodyBytes < 1 || o.MaxBodyBytes > 1<<40 || o.MaxParts < 1 || o.MaxParts > 10000 || o.MaxFieldBytes < 1 || o.MaxFieldBytes > 1<<30 || o.MaxFileBytes < 1 || o.MaxFileBytes > 1<<40 {
return o, errors.New("invalid multipart limits")
}
return o, o.Policy.Validate()
}
func (b *Binder) HasMultipart() bool {
for _, f := range b.root.fields {
if multipartSource(f.source) {
return true
}
}
return false
}
func multipartSource(s string) bool { return s == "file" || s == "multipart" || s == "multipartJSON" }
func jsonSource(s string) bool { return s == "json" || s == "multipartJSON" }
// MultipartSchema describes form parts. JSON fields get an encoding declaration
// in generated OpenAPI; they are decoded with the ordinary strict JSON binder.
func (b *Binder) MultipartSchema() *openapi.Schema {
s := &openapi.Schema{Type: "object", Properties: map[string]*openapi.Schema{}}
no := false
s.AdditionalPropertiesAllowed = &no
for _, f := range b.root.fields {
if multipartSource(f.source) {
s.Properties[f.name] = copySchema(f.value.schema)
if !f.optional && !f.hasDefault {
s.Required = append(s.Required, f.name)
}
}
}
return s
}
func (b *Binder) MultipartJSONFields() []string {
var out []string
for _, f := range b.root.fields {
if f.source == "multipartJSON" {
out = append(out, f.name)
}
}
return out
}
// BindMultipart validates every part before publishing the DTO or entering Logic.
// On success the caller must defer cleanup; on failure all created files/readers
// are cleaned here. The request's Body remains owned by the HTTP server.
func (b *Binder) BindMultipart(r *http.Request, params map[string]string, target any, options MultipartOptions) (cleanup func() error, err error) {
if b == nil || r == nil || b.schemaOnly {
return nil, apierrors.Internal(errors.New("invalid multipart binder"))
}
dst := reflect.ValueOf(target)
if dst.Kind() != reflect.Pointer || dst.IsNil() || dst.Elem().Type() != b.typ {
return nil, apierrors.Internal(errors.New("multipart target mismatch"))
}
o, err := options.Normalized()
if err != nil {
return nil, apierrors.Internal(err)
}
if b.BodySchema() != nil {
return nil, apierrors.Internal(errors.New("multipart request cannot contain json source fields; use multipartJSON"))
}
if r.ContentLength > o.MaxBodyBytes {
return nil, tooLarge()
}
var files []File
var once sync.Once
var closeErr error
release := func() error {
once.Do(func() {
for i := len(files) - 1; i >= 0; i-- {
closeErr = errors.Join(closeErr, files[i].cleanup())
}
})
return closeErr
}
success := false
defer func() {
if !success {
err = errors.Join(err, release())
}
}()
// Bind query/path/header once, independently of multipart body contents.
ordinary := *b
root := *b.root
root.fields = nil
ordinary.root = &root
for _, f := range b.root.fields {
if !multipartSource(f.source) {
root.fields = append(root.fields, f)
}
}
fresh := reflect.New(b.typ)
policy := o.Policy
policy.Body = "auto"
if err = ordinary.BindWithPolicy(r, params, fresh.Interface(), policy); err != nil {
return nil, err
}
request := *r
input := r.Body
if input == nil {
input = http.NoBody
}
request.Body = http.MaxBytesReader(nil, input, o.MaxBodyBytes)
body, err := httpx.NewBody(&request, "multipart")
if err != nil {
return nil, err
}
fields := map[string]field{}
for _, f := range b.root.fields {
if multipartSource(f.source) {
fields[f.name] = f
}
}
text := map[string][]string{}
uploaded := map[string][]File{}
seen := map[string]int{}
policy = MergePolicy(RequestPolicyFromContext(r.Context()), o.Policy)
for count := 0; ; count++ {
part, e := body.Parts.NextRawPart()
if e == io.EOF {
break
}
if e != nil {
return nil, multipartFailure(e)
}
if count >= o.MaxParts {
return nil, tooLarge()
}
f, ok := fields[part.FormName()]
if !ok {
return nil, invalid(errors.New("unknown multipart field"))
}
seen[f.name]++
// Reject encodings that would make declared byte limits ambiguous.
if part.Header.Get("Content-Transfer-Encoding") != "" {
return nil, invalid(errors.New("encoded multipart parts are unsupported"))
}
_, disposition, e := mime.ParseMediaType(part.Header.Get("Content-Disposition"))
if e != nil {
return nil, invalid(e)
}
_, isFile := disposition["filename"]
if (f.source == "file") != isFile {
return nil, invalid(errors.New("multipart field kind mismatch"))
}
limit := o.MaxFieldBytes
if f.source == "file" {
limit = o.MaxFileBytes
if f.maxBytes > 0 && f.maxBytes < limit {
limit = f.maxBytes
}
maxFiles := f.maxFiles
if maxFiles == 0 {
maxFiles = 16
}
if f.value.typ.Kind() != reflect.Slice {
maxFiles = 1
}
if seen[f.name] > maxFiles {
return nil, invalid(errors.New("too many files for multipart field"))
}
name := path.Base(strings.ReplaceAll(part.FileName(), "\\", "/"))
if name == "" || name == "." || len(name) > 255 || !utf8.ValidString(name) || strings.ContainsAny(name, "\x00\r\n") {
return nil, invalid(errors.New("invalid upload filename"))
}
temp, e := os.CreateTemp(o.TempDir, "omni-upload-*")
if e != nil {
return nil, apierrors.Internal(e)
}
file := File{Name: name, ContentType: part.Header.Get("Content-Type"), state: &uploadState{path: temp.Name(), readers: map[*uploadReader]bool{}}}
files = append(files, file)
size, e := io.Copy(temp, io.LimitReader(part, limit+1))
e = errors.Join(e, temp.Close())
if size > limit {
return nil, tooLarge()
}
if e != nil {
return nil, multipartFailure(e)
}
file.Size = size
files[len(files)-1] = file
uploaded[f.name] = append(uploaded[f.name], file)
} else {
if f.maxBytes > 0 && f.maxBytes < limit {
limit = f.maxBytes
}
if seen[f.name] > 1 && (f.source == "multipartJSON" || base(f.value).typ.Kind() != reflect.Slice) {
return nil, invalid(errors.New("duplicate multipart field"))
}
data, e := io.ReadAll(io.LimitReader(part, limit+1))
if int64(len(data)) > limit {
return nil, tooLarge()
}
if e != nil {
return nil, multipartFailure(e)
}
if policy.UTF8 == "reject" && !utf8.Valid(data) {
return nil, invalid(errUTF8)
}
if policy.Charset == "utf8" && part.Header.Get("Content-Type") != "" {
_, p, e := mime.ParseMediaType(part.Header.Get("Content-Type"))
if e != nil || (p["charset"] != "" && !strings.EqualFold(p["charset"], "utf-8")) {
return nil, invalid(errors.New("invalid multipart field charset"))
}
}
text[f.name] = append(text[f.name], strings.ToValidUTF8(string(data), "<22>"))
}
if err = part.Close(); err != nil {
return nil, multipartFailure(err)
}
}
// Consume epilogue as well; it counts against the total request cap.
if err = body.Complete(); err != nil {
return nil, multipartFailure(err)
}
for _, f := range b.root.fields {
if !multipartSource(f.source) {
continue
}
present := seen[f.name] > 0
dest := fresh.Elem().Field(f.index)
if f.source == "file" {
if !present {
if !f.optional {
return nil, invalid(fmt.Errorf("%s: file required", f.name))
}
continue
}
if f.value.typ.Kind() == reflect.Slice {
dest.Set(reflect.ValueOf(uploaded[f.name]))
} else {
dest.Set(reflect.ValueOf(uploaded[f.name][0]))
}
continue
}
var value any
if present {
if f.source == "multipartJSON" {
d := json.NewDecoder(strings.NewReader(text[f.name][0]))
d.UseNumber()
value, err = decodeValue(d, 0)
if err == nil {
_, err = d.Token()
if err == io.EOF {
err = nil
} else if err == nil {
err = errors.New("multiple JSON values")
}
}
} else {
value, err = fromStrings(f.value, text[f.name])
}
if err != nil {
return nil, invalid(fmt.Errorf("%s: %w", f.name, err))
}
}
if err = assignField(f, dest, value, present); err != nil {
return nil, invalid(fmt.Errorf("%s: %w", f.name, err))
}
}
dst.Elem().Set(fresh.Elem())
success = true
return release, nil
}
func multipartFailure(err error) error {
var max *http.MaxBytesError
if errors.As(err, &max) {
return tooLarge()
}
var api *apierrors.APIError
if errors.As(err, &api) {
return err
}
return invalid(err)
}
func fieldLimit(raw string) (int64, error) {
v, err := strconv.ParseInt(raw, 10, 64)
if err != nil || v < 1 || v > 1<<40 {
return 0, errors.New("maxBytes must be 1..1099511627776")
}
return v, nil
}

152
contract/multipart_test.go Normal file
View File

@@ -0,0 +1,152 @@
package contract
import (
"bytes"
"errors"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"io"
"mime/multipart"
"net/http"
"net/http/httptest"
"os"
"strings"
"testing"
)
type uploadInput struct {
Query string `form:"query,optional"`
Name string `multipart:"name" binding:"required,max=10"`
Meta struct {
Count int `json:"count"`
} `multipartJSON:"meta"`
Attachment File `file:"attachment,maxBytes=12"`
Extra []File `file:"extra,optional,maxBytes=8,maxFiles=2"`
}
type uploadPart struct{ name, filename, value string }
func uploadRequest(t *testing.T, parts []uploadPart) (*http.Request, []byte) {
t.Helper()
var data bytes.Buffer
w := multipart.NewWriter(&data)
for _, part := range parts {
if part.filename != "" {
p, e := w.CreateFormFile(part.name, part.filename)
if e != nil {
t.Fatal(e)
}
_, _ = io.WriteString(p, part.value)
} else {
_ = w.WriteField(part.name, part.value)
}
}
_ = w.Close()
r := httptest.NewRequest("POST", "/upload?query=yes", bytes.NewReader(data.Bytes()))
r.Header.Set("Content-Type", w.FormDataContentType())
return r, data.Bytes()
}
func goodParts() []uploadPart {
return []uploadPart{{"name", "", "合同"}, {"meta", "", `{"count":2}`}, {"attachment", "../contract.txt", "hello"}}
}
func assertUploadEmpty(t *testing.T, dir string) {
t.Helper()
files, err := os.ReadDir(dir)
if err != nil || len(files) != 0 {
t.Fatalf("temp files remain: %v %v", files, err)
}
}
func TestMultipartBindingAndLifetime(t *testing.T) {
b := MustNew[uploadInput]()
dir := t.TempDir()
r, _ := uploadRequest(t, append(goodParts(), uploadPart{"extra", "one", "first"}, uploadPart{"extra", "two", "second"}))
var input uploadInput
release, err := b.BindMultipart(r, nil, &input, MultipartOptions{TempDir: dir, Policy: RequestPolicy{UnknownQuery: "reject", UTF8: "reject"}})
if err != nil {
t.Fatal(err)
}
if input.Name != "合同" || input.Query != "yes" || input.Meta.Count != 2 || input.Attachment.Size != 5 || input.Attachment.Name != "contract.txt" || len(input.Extra) != 2 {
t.Fatalf("%+v", input)
}
reader, err := input.Attachment.Open()
if err != nil {
t.Fatal(err)
}
data, err := io.ReadAll(reader)
if err != nil || string(data) != "hello" {
t.Fatal(string(data), err)
}
// An unclosed reader is owned by the upload cleanup as well.
if err = release(); err != nil {
t.Fatal(err)
}
if _, err = reader.Read(make([]byte, 1)); err == nil {
t.Fatal("reader remains open")
}
if _, err = input.Attachment.Open(); err == nil {
t.Fatal("expired upload reopened")
}
if err = release(); err != nil {
t.Fatal(err)
}
assertUploadEmpty(t, dir)
if err = b.Bind(r, nil, &input); err == nil {
t.Fatal("ordinary binding accepted multipart fields")
}
}
func TestMultipartFailuresAreTransactional(t *testing.T) {
b := MustNew[uploadInput]()
for _, tt := range []struct {
name string
parts []uploadPart
options MultipartOptions
status int
}{
{"missing", goodParts()[1:], MultipartOptions{}, 400},
{"unknown", append(goodParts(), uploadPart{"other", "", "x"}), MultipartOptions{}, 400},
{"duplicate", append(goodParts(), uploadPart{"name", "", "x"}), MultipartOptions{}, 400},
{"jsonDuplicate", []uploadPart{{"name", "", "n"}, {"attachment", "f", "x"}, {"meta", "", `{"count":1,"count":2}`}}, MultipartOptions{}, 400},
{"badType", []uploadPart{{"name", "", "n"}, {"attachment", "f", "x"}, {"meta", "", `{"count":"x"}`}}, MultipartOptions{}, 400},
{"badUTF8", append(goodParts()[:1], uploadPart{"meta", "", "{\"count\":\"\xff\"}"}, goodParts()[2]), MultipartOptions{Policy: RequestPolicy{UTF8: "reject"}}, 400},
{"fileSize", []uploadPart{{"attachment", "x", strings.Repeat("x", 13)}}, MultipartOptions{}, 413},
{"fieldSize", goodParts(), MultipartOptions{MaxFieldBytes: 2}, 413},
{"partCount", goodParts(), MultipartOptions{MaxParts: 2}, 413},
{"total", goodParts(), MultipartOptions{MaxBodyBytes: 32}, 413},
{"fileKind", []uploadPart{{"attachment", "", "text"}}, MultipartOptions{}, 400},
{"extraCount", append(goodParts(), uploadPart{"extra", "a", "a"}, uploadPart{"extra", "b", "b"}, uploadPart{"extra", "c", "c"}), MultipartOptions{}, 400},
} {
t.Run(tt.name, func(t *testing.T) {
dir := t.TempDir()
tt.options.TempDir = dir
r, _ := uploadRequest(t, tt.parts)
r.ContentLength = -1
input := uploadInput{Name: "untouched"}
release, err := b.BindMultipart(r, nil, &input, tt.options)
var api *apierrors.APIError
if !errors.As(err, &api) || api.Status != tt.status {
t.Fatalf("error %v", err)
}
if release != nil || input.Name != "untouched" {
t.Fatal("partial binding published")
}
assertUploadEmpty(t, dir)
})
}
}
func TestMultipartTruncationAndEpilogueLimit(t *testing.T) {
b := MustNew[uploadInput]()
for _, epilogue := range []bool{false, true} {
dir := t.TempDir()
r, original := uploadRequest(t, goodParts())
data := original[:len(original)-8]
limit := int64(len(original) + 16)
if epilogue {
data = append(original, bytes.Repeat([]byte("x"), 64)...)
}
r.Body = io.NopCloser(bytes.NewReader(data))
r.ContentLength = -1
var input uploadInput
if _, err := b.BindMultipart(r, nil, &input, MultipartOptions{TempDir: dir, MaxBodyBytes: limit}); err == nil {
t.Fatal("truncation or epilogue bypass accepted")
}
assertUploadEmpty(t, dir)
}
}

105
contract/policy.go Normal file
View File

@@ -0,0 +1,105 @@
package contract
import (
"context"
"errors"
"fmt"
"io"
"unicode/utf8"
)
// RequestPolicy describes protocol strictness, independently of business
// fields. Empty values inherit App defaults, then the historical binder policy.
type RequestPolicy struct {
UnknownQuery string `json:"unknownQuery"` // ignore or reject
UTF8 string `json:"utf8"` // replace or reject
Charset string `json:"charset"` // any or utf8
Body string `json:"body"` // auto, required, optional, none, emptyObject
}
func (p RequestPolicy) Validate() error {
for _, item := range []struct {
name, value string
allowed []string
}{
{"unknownQuery", p.UnknownQuery, []string{"ignore", "reject"}},
{"utf8", p.UTF8, []string{"replace", "reject"}},
{"charset", p.Charset, []string{"any", "utf8"}},
{"body", p.Body, []string{"auto", "required", "optional", "none", "emptyObject"}},
} {
if item.value == "" {
continue
}
ok := false
for _, value := range item.allowed {
ok = ok || value == item.value
}
if !ok {
return fmt.Errorf("request policy: invalid %s", item.name)
}
}
return nil
}
func MergePolicy(base, override RequestPolicy) RequestPolicy {
if override.UnknownQuery != "" {
base.UnknownQuery = override.UnknownQuery
}
if override.UTF8 != "" {
base.UTF8 = override.UTF8
}
if override.Charset != "" {
base.Charset = override.Charset
}
if override.Body != "" {
base.Body = override.Body
}
return base
}
type policyKey struct{}
func WithRequestPolicy(ctx context.Context, p RequestPolicy) context.Context {
if ctx == nil {
ctx = context.Background()
}
return context.WithValue(ctx, policyKey{}, p)
}
func RequestPolicyFromContext(ctx context.Context) RequestPolicy {
if ctx == nil {
return RequestPolicy{}
}
p, _ := ctx.Value(policyKey{}).(RequestPolicy)
return p
}
// utf8Reader checks original bytes in one pass, including runes crossing Read
// boundaries. It never replaces invalid bytes or buffers the complete body.
type utf8Reader struct {
r io.Reader
pending [4]byte
n int
}
var errUTF8 = errors.New("invalid UTF-8 in JSON body")
func (r *utf8Reader) Read(p []byte) (int, error) {
n, err := r.r.Read(p)
for _, b := range p[:n] {
if r.n == 0 && b < utf8.RuneSelf {
continue
}
r.pending[r.n] = b
r.n++
if utf8.FullRune(r.pending[:r.n]) {
ch, size := utf8.DecodeRune(r.pending[:r.n])
if ch == utf8.RuneError && size == 1 {
return n, errUTF8
}
r.n = 0
}
}
if err == io.EOF && r.n != 0 {
return n, errUTF8
}
return n, err
}

104
contract/policy_test.go Normal file
View File

@@ -0,0 +1,104 @@
package contract
import (
"bytes"
"context"
"errors"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
type byteReader struct{ r io.Reader }
func (r byteReader) Read(p []byte) (int, error) {
if len(p) > 1 {
p = p[:1]
}
return r.r.Read(p)
}
func TestBodyPolicyPropagatesOuterLimit(t *testing.T) {
for _, limit := range []int64{0, 1, 4} {
r := httptest.NewRequest("POST", "/", byteReader{strings.NewReader("{} ")})
r.Header.Set("Content-Type", "application/json")
r.Body = http.MaxBytesReader(httptest.NewRecorder(), r.Body, limit)
err := MustNew[struct{}]().BindWithPolicy(r, nil, &struct{}{}, RequestPolicy{Body: "required", UTF8: "reject"})
var e *apierrors.APIError
if !errors.As(err, &e) || e.Status != 413 {
t.Fatalf("limit %d: expected 413, got %v", limit, err)
}
}
}
func TestStrictRequestPolicy(t *testing.T) {
type request struct {
Name string `json:"name"`
Tags []string `form:"tag,optional"`
}
b := MustNew[request]()
strict := RequestPolicy{UnknownQuery: "reject", UTF8: "reject", Charset: "utf8"}
for _, tc := range []struct {
name, url, ct, body string
valid bool
}{
{"valid", "/?tag=a&tag=b", "application/json; charset=UTF-8", `{"name":"企业😀"}`, true},
{"query", "/?typo=1", "application/json", `{"name":"ok"}`, false},
{"charset", "/", "application/json; charset=gbk", `{"name":"ok"}`, false},
{"utf8", "/", "application/json", "{\"name\":\"\xff\"}", false},
{"truncatedRune", "/", "application/json", "{\"name\":\"\xe4\"}", false},
{"tail", "/", "application/json", `{"name":"ok"} {}`, false},
{"tailInvalidByte", "/", "application/json", "{\"name\":\"ok\"} \xff", false},
} {
t.Run(tc.name, func(t *testing.T) {
r := httptest.NewRequest("POST", tc.url, byteReader{strings.NewReader(tc.body)})
r.Header.Set("Content-Type", tc.ct)
var dst request
err := b.BindWithPolicy(r, nil, &dst, strict)
if (err == nil) != tc.valid {
t.Fatal(err)
}
if tc.valid && len(dst.Tags) != 2 {
t.Fatal("query slice rejected")
}
})
}
// Explicit route overrides inherit the rest of the App policy.
r := httptest.NewRequest("POST", "/?typo=1", strings.NewReader(`{"name":"ok"}`))
r.Header.Set("Content-Type", "application/json")
r = r.WithContext(WithRequestPolicy(context.Background(), strict))
if err := b.BindWithPolicy(r, nil, &request{}, RequestPolicy{UnknownQuery: "ignore"}); err != nil {
t.Fatal(err)
}
}
func TestBodyPresencePolicies(t *testing.T) {
b := MustNew[struct{}]()
for _, mode := range []string{"none", "emptyObject", "required", "optional"} {
for _, body := range []string{"", "{}", "null", `{"unexpected":1}`, "{} {}", " "} {
r := httptest.NewRequest("POST", "/", byteReader{strings.NewReader(body)})
r.Header.Set("Content-Type", "application/json")
err := b.BindWithPolicy(r, nil, &struct{}{}, RequestPolicy{Body: mode, UTF8: "reject"})
valid := (mode == "none" && body == "") || (mode != "none" && body == "{}") || (mode == "optional" && body == "")
if (err == nil) != valid {
t.Fatalf("mode=%s body=%q: %v", mode, body, err)
}
}
}
type object struct {
Value string `json:"value"`
}
r := httptest.NewRequest("POST", "/", bytes.NewReader(nil))
if err := MustNew[object]().BindWithPolicy(r, nil, &object{}, RequestPolicy{Body: "optional"}); err != nil {
t.Fatal(err)
}
// Chunked/unknown size is still bounded under strict UTF-8 checking.
r = httptest.NewRequest("POST", "/", byteReader{strings.NewReader(`{"value":"1234567890"}`)})
r.Header.Set("Content-Type", "application/json")
if err := MustNew[object]().BindWithPolicy(r, nil, &object{}, RequestPolicy{UTF8: "reject"}, 8); err == nil {
t.Fatal("chunked over-limit accepted")
}
}

View File

@@ -147,6 +147,10 @@ func responseSchema(n *node) *openapi.Schema {
b := base(n)
if b.scalar != nil {
s := wireSchema(b.scalar.JSONOutput)
s.Description = n.schema.Description
if b.scalar.JSONInput == b.scalar.JSONOutput {
s.Example = n.schema.Example
}
if n.typ.Kind() == reflect.Pointer {
s = nullableSchema(s)
}

View File

@@ -1,5 +1,9 @@
# database
结构化 MySQL 配置、独立 TLS/mTLS、驱动日志和生成项目接入见 [官方 MySQL 适配器](mysql/README.md)。已有 Open/OpenConnector 接口继续支持。
使用 GORM 的项目可显式接入 [gormx 可选适配器](gormx/README.md),复用本包连接池,并将 SQL 日志接入 logx;普通 database 包不导入 GORM。
`database` 包提供基于标准库 `database/sql` 的连接池生命周期,不绑定 ORM 或驱动:
- `Config` / `ConfigFromEnv`:连接池参数和 `GO_OMNI_DB_*` 环境变量。

104
database/gormx/README.md Normal file
View File

@@ -0,0 +1,104 @@
# 可选 GORM 适配器
从 `v0.14.1-preview` 提供。`gormx` 把 GORM 接到 go-omni 已初始化的 `*sql.DB`,保留一份连接池和一处资源所有权。使用 GORM v2 API(本版验收版本为 `gorm.io/gorm v1.31.2`、MySQL dialect `v1.6.0`)。
核心 `omni`、`database`、`database/mysql` 不导入此包;普通项目的编译依赖链不包含 GORM。仓库同一 Go module 的 go.mod 会记录可选适配器依赖,使用方显式导入 `database/gormx` 才编译它。本版没有新增默认 ORM 或 `--orm` 脚手架开关。
## 生成项目接入
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
go-omni new --database mysql --module example.com/billing ./billing
```
按 [MySQL 配置说明](../mysql/README.md) 设置连接信息、TLS 和池大小。在开发者维护的 `internal/svc/serviceContext.go` 中增加 `ORM *gorm.DB` 字段,并填写现有 Init:
```go
import (
"context"
"git.yuntuo.hk/k2safe/go-omni/database/gormx"
"gorm.io/gorm"
)
// 向现有 ServiceContext 添加 ORM *gorm.DB 字段。
// 保留 Runtime/Config/Logger/DB 等现有字段。
func (c *ServiceContext) Init(ctx context.Context) error {
orm, err := gormx.MySQL(ctx, c.DB, gormx.WithLogger(c.Logger))
if err != nil {
return err
}
c.ORM = orm
return nil
}
```
执行 `go mod tidy`。main、runtimeGen、Handler 和路由无需修改;重新生成保留 svc 的字段与 Init。已有项目先 `go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview`,同样接入;v0.14.0 生成项目的目录和生命周期契约不变。
Repository 接收 `*gorm.DB`,查询使用请求 context:
```go
err := repo.db.WithContext(ctx).Where("id = ?", id).First(&item).Error
err = repo.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
// 同一事务内始终使用 tx。
return tx.Create(&item).Error
})
```
现有模型、Hooks、Scopes、事务、命名策略和错误处理继续由项目维护。共享连接池不会把分别发起的 SQL/GORM 事务自动合并;不要在一个事务内切回根 ORM 或 c.DB 执行本应属于该事务的写入。
## 连接池与失败处理
- MySQL 配置、TLS、字符集、会话参数、池大小及健康检查由底层 `database/mysql` 管理,适配器不再设置另一套 DSN 或池参数。
- GORM 方言的版本协商继续执行,自动 Ping 关闭以复用已完成的池初始化。初始化查询使用传入 context,并受池的 PingTimeout 限制;可用 `WithInitTimeout(3*time.Second)` 单独指定。
- 返回成功后不保留启动 context,每次查询用 `WithContext(ctx)`。查询取消、错误、事务提交/回滚保持 GORM 原生语义。
- 对方言初始化失败时 GORM 自动关闭连接池的路径提供保护;错误仍保留 errors.Is/As,原池继续归调用方管理。生成项目的 Init 返回错误时,再由 Runtime 统一回滚该项目资源。
- 服务排空后由 go-omni 关闭原池;`orm.DB()` 返回的仍是那一个池。不要给它重复登记 Close,也不要让 Repository 关闭它。
适配器自身不创建后台 worker、不自动建表、不执行 AutoMigrate、不改写默认事务或全表更新保护,也不替业务重试写入。
## SQL 日志
默认借用当前 context 的 logx,也可显式 `WithLogger(c.Logger)`。每个 ORM 可以使用独立日志实例;不会改动进程全局 Logger。
```go
sqlLog := gormx.DefaultLogConfig()
sqlLog.Level = "info" // silent / error / warn / info
sqlLog.SlowThreshold = 300*time.Millisecond
orm, err := gormx.MySQL(ctx, c.DB,
gormx.WithLogger(c.Logger),
gormx.WithLogConfig(sqlLog),
gormx.WithConfig(gorm.Config{
TranslateError: true,
// NamingStrategy 等原生选项可按项目添加。
}),
)
```
默认记录错误和超过 200 ms 的慢查询,忽略 ErrRecordNotFound 的错误事件;`SlowThreshold: 0` 关闭慢查询事件。事件为 `gorm.query`、`gorm.slowQuery`、`gorm.queryFailed`,携带 durationMs、rowsAffected、sql 和 context 中的 requestId/traceId。GORM 的 `Debug()`/`LogMode` 使用副本,不改变其他请求的日志级别;最终输出还受 logx 自身级别控制。
默认不插入绑定参数、不记录驱动完整错误文案,只记录错误类别;SQL 正文最多保留 4096 字节,截断后附加省略号。明确开启 `IncludeParameters` / `IncludeErrorDetails` 才输出这些内容。日志不会解析和脱敏直接写在原始 SQL 里的字面量,查询仍应使用占位符。仅需要日志适配的既有 GORM 实例,也可使用 `NewLogger(logger, config)` 返回的原生 `logger.Interface`。
`WithConfig` 中若显式设置 `gorm.Config.Logger`,则使用项目自己的 Logger,该 Logger 自行负责脱敏和上下文。配置和 Plugins/ClauseBuilders 容器会复制;用户自定义插件、NamingStrategy 等对象仍由用户保证并发安全。
## 其他方言与高级配置
MySQL 便捷入口封装了下面的通用入口:
```go
orm, err := gormx.Open(ctx, pool,
func(conn gorm.ConnPool) gorm.Dialector {
return mysql.New(mysql.Config{Conn: conn})
},
gormx.WithLogger(logger),
)
```
其他方言可由项目提供相同形式的工厂,必须使用传入的 Conn,不能创建或替换连接池。本版真实数据库验收覆盖 MySQL 8.4,不代表其他数据库已经联调。
不能通过 WithConfig 的 ConnPool/Dialector 覆盖此约定。初始化期间不暴露裸 sql.DB;需要它的自定义插件应在 Open 成功后显式注册。创建额外池或替换 ConnPool 的路由插件不属于本适配器的一池所有权范围。
PrepareStmt 默认关闭;若显式开启,保留 GORM 的缓存行为,容量、TTL 和缓存清理由项目按 GORM 接口管理。当前固定的 GORM 版本,PreparedStmtDB.Close 只清理语句,不提供停止其缓存清理 goroutine 的公开接口;因此应在服务初始化时创建一次 ORM,避免按请求反复创建。适配器不修改第三方缓存实现,也不宣称解决其生命周期限制。
参考:[GORM 现有连接池](https://gorm.io/docs/connecting_to_the_database.html#Existing-database-connection)、[GORM Logger](https://gorm.io/docs/logger.html)。本版验收见 [GORM 验收报告](../../reports/gormAcceptance.md)。

121
database/gormx/gorm.go Normal file
View File

@@ -0,0 +1,121 @@
// Package gormx optionally attaches GORM to a pool owned by GoOmni. It never
// creates a pool, changes pool limits, migrates tables or closes the owner.
package gormx
import (
"context"
"errors"
"maps"
"reflect"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"git.yuntuo.hk/k2safe/go-omni/logx"
gormmysql "gorm.io/driver/mysql"
"gorm.io/gorm"
)
// DialectorFactory must use the supplied connection pool. It must not create a
// pool or retain/use the ORM before Open succeeds. For example, return
// mysql.New(mysql.Config{Conn: conn}). Other dialects can use the same contract.
type DialectorFactory func(conn gorm.ConnPool) gorm.Dialector
type options struct {
config gorm.Config
logger *logx.Logger
logConfig LogConfig
timeout time.Duration
}
type Option func(*options)
// WithConfig preserves native GORM behavior, including naming, hooks, prepared
// statements and transactions. ConnPool/Dialector are supplied by this adapter;
// automatic Ping is disabled because the owning pool already passed startup Ping.
// A custom Config.Logger takes precedence over the logx adapter.
func WithConfig(config gorm.Config) Option { return func(o *options) { o.config = config } }
func WithLogger(logger *logx.Logger) Option { return func(o *options) { o.logger = logger } }
func WithLogConfig(config LogConfig) Option { return func(o *options) { o.logConfig = config } }
func WithInitTimeout(timeout time.Duration) Option { return func(o *options) { o.timeout = timeout } }
// Error exposes a safe initialization summary while preserving errors.Is/As.
type Error struct{ cause error }
func (e *Error) Error() string { return "gormx: initialization failed" }
func (e *Error) Unwrap() error { return e.cause }
// MySQL attaches GORM's MySQL dialect, including version negotiation, to the
// existing pool. Configure TLS, timeouts and connection limits on that pool.
func MySQL(ctx context.Context, pool *database.DB, opts ...Option) (*gorm.DB, error) {
return Open(ctx, pool, func(conn gorm.ConnPool) gorm.Dialector {
return gormmysql.New(gormmysql.Config{Conn: conn})
}, opts...)
}
// Open borrows pool. On failure (including dialect initialization failure),
// the original pool remains owned by the caller. No ORM Close is required.
// The returned ORM does not retain the startup context: each request should use
// WithContext(ctx). Never close the sql.DB returned by orm.DB() independently.
func Open(ctx context.Context, pool *database.DB, factory DialectorFactory, opts ...Option) (*gorm.DB, error) {
if ctx == nil {
ctx = context.Background()
}
if err := ctx.Err(); err != nil {
return nil, &Error{cause: err}
}
if pool == nil || pool.DB == nil || factory == nil {
return nil, errors.New("gormx: pool and dialector factory are required")
}
timeout := pool.PoolConfig().PingTimeout
if timeout == 0 {
timeout = 5 * time.Second
}
o := options{timeout: timeout, logConfig: DefaultLogConfig(), logger: logx.FromContext(ctx)}
for _, option := range opts {
if option != nil {
option(&o)
}
}
if o.timeout <= 0 {
return nil, errors.New("gormx: initialization timeout must be positive")
}
if o.config.ConnPool != nil || o.config.Dialector != nil {
return nil, errors.New("gormx: configure the existing pool through the dialector factory")
}
logger, err := NewLogger(o.logger, o.logConfig)
if err != nil {
return nil, err
}
config := o.config
config.Plugins = maps.Clone(config.Plugins)
config.ClauseBuilders = maps.Clone(config.ClauseBuilders)
if config.Logger == nil {
config.Logger = logger
}
config.DisableAutomaticPing = true
startup, cancel := context.WithTimeout(ctx, o.timeout)
defer cancel()
borrowed := &borrowedPool{db: pool.DB}
borrowed.startup.Store(&startupContext{startup})
config.ConnPool = borrowed
dialect := factory(borrowed)
if dialect == nil || (reflect.ValueOf(dialect).Kind() == reflect.Pointer && reflect.ValueOf(dialect).IsNil()) {
return nil, errors.New("gormx: dialector factory returned nil")
}
orm, err := gorm.Open(dialect, &config)
if err != nil {
return nil, &Error{cause: err}
}
if err = startup.Err(); err != nil {
return nil, &Error{cause: err}
}
// Prepared statements are GORM's own wrapper over the very same pool.
conn := orm.ConnPool
if prepared, ok := conn.(*gorm.PreparedStmtDB); ok {
conn = prepared.ConnPool
}
if actual, ok := conn.(*borrowedPool); !ok || actual != borrowed {
return nil, errors.New("gormx: dialector or plugin replaced the supplied pool")
}
borrowed.startup.Store(nil)
return orm, nil
}

173
database/gormx/gorm_test.go Normal file
View File

@@ -0,0 +1,173 @@
package gormx
import (
"context"
"database/sql/driver"
"errors"
"io"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"git.yuntuo.hk/k2safe/go-omni/logx"
"gorm.io/gorm"
"gorm.io/gorm/clause"
"gorm.io/gorm/schema"
)
type testConnector struct {
closed atomic.Int32
query func(context.Context) error
}
func (c *testConnector) Connect(context.Context) (driver.Conn, error) {
return &testConn{owner: c}, nil
}
func (c *testConnector) Driver() driver.Driver { return testDriver{} }
func (c *testConnector) Close() error { c.closed.Add(1); return nil }
type testDriver struct{}
func (testDriver) Open(string) (driver.Conn, error) { return nil, errors.New("unexpected new pool") }
type testConn struct{ owner *testConnector }
func (c *testConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("unused") }
func (c *testConn) Close() error { return nil }
func (c *testConn) Begin() (driver.Tx, error) { return nil, errors.New("unused") }
func (c *testConn) Ping(context.Context) error { return nil }
func (c *testConn) QueryContext(ctx context.Context, _ string, _ []driver.NamedValue) (driver.Rows, error) {
if c.owner.query != nil {
if err := c.owner.query(ctx); err != nil {
return nil, err
}
}
return &testRows{}, nil
}
type testRows struct{ done bool }
func (r *testRows) Columns() []string { return []string{"version"} }
func (r *testRows) Close() error { return nil }
func (r *testRows) Next(values []driver.Value) error {
if r.done {
return io.EOF
}
r.done = true
values[0] = "8.4.0"
return nil
}
func poolFixture(t *testing.T) (*database.DB, *testConnector) {
t.Helper()
connector := &testConnector{}
pool, err := database.OpenConnector(context.Background(), connector, database.DefaultPoolConfig())
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { _ = pool.Close() })
return pool, connector
}
func TestBorrowedPoolFailureAndStartupDeadline(t *testing.T) {
for _, mode := range []string{"driverError", "deadline", "canceled"} {
t.Run(mode, func(t *testing.T) {
pool, connector := poolFixture(t)
cause := errors.New("private-driver-error")
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
connector.query = func(ctx context.Context) error {
if mode == "driverError" {
return cause
}
<-ctx.Done()
return ctx.Err()
}
if mode == "canceled" {
cancel()
cause = context.Canceled
} else if mode == "deadline" {
cause = context.DeadlineExceeded
}
start := time.Now()
orm, err := MySQL(ctx, pool, WithInitTimeout(20*time.Millisecond))
if orm != nil || !errors.Is(err, cause) {
t.Fatal("lost cause", err)
}
if time.Since(start) > time.Second {
t.Fatal("startup deadline ignored")
}
if connector.closed.Load() != 0 {
t.Fatal("borrowed pool closed after GORM failure")
}
if healthErr := pool.Health(context.Background()); healthErr != nil {
t.Fatal("owner pool unusable", healthErr)
}
if err.Error() == cause.Error() {
t.Fatal("unsafe initialization error")
}
})
}
}
func TestPoolIdentityNativeOptionsAndRequestContext(t *testing.T) {
pool, connector := poolFixture(t)
ctx, cancel := context.WithCancel(context.Background())
cfg := gorm.Config{NamingStrategy: schema.NamingStrategy{TablePrefix: "business_"}, ClauseBuilders: map[string]clause.ClauseBuilder{}}
logger, _ := logx.New(logx.Config{Output: "discard"})
orm, err := MySQL(ctx, pool, WithConfig(cfg), WithLogger(logger))
if err != nil {
t.Fatal(err)
}
cancel() // The startup context must not poison future queries.
got, err := orm.DB()
if err != nil || got != pool.DB {
t.Fatal("pool identity changed", err)
}
if len(cfg.ClauseBuilders) != 0 || cfg.ConnPool != nil || cfg.Logger != nil {
t.Fatal("native input config mutated")
}
if orm.NamingStrategy.TableName("Item") != "business_items" {
t.Fatal("native option lost")
}
if orm.SkipDefaultTransaction || orm.AllowGlobalUpdate {
t.Fatal("transaction safety defaults changed")
}
ctx, cancel = context.WithCancel(context.Background())
cancel()
var v string
if err = orm.WithContext(ctx).Raw("SELECT VERSION()").Scan(&v).Error; !errors.Is(err, context.Canceled) {
t.Fatal(err)
}
if err = orm.WithContext(context.Background()).Raw("SELECT VERSION()").Scan(&v).Error; err != nil || v != "8.4.0" {
t.Fatal(v, err)
}
if connector.closed.Load() != 0 {
t.Fatal("pool unexpectedly closed")
}
if err = pool.Close(); err != nil {
t.Fatal(err)
}
if err = orm.Raw("SELECT VERSION()").Scan(&v).Error; err == nil {
t.Fatal("owner close did not affect GORM")
}
}
func TestRejectedOptionsDoNotTouchPool(t *testing.T) {
for _, option := range []Option{WithInitTimeout(0), WithLogConfig(LogConfig{Level: "bad"}), WithConfig(gorm.Config{ConnPool: &borrowedPool{}})} {
pool, connector := poolFixture(t)
if _, err := MySQL(context.Background(), pool, option); err == nil {
t.Fatal("invalid option accepted")
}
if connector.closed.Load() != 0 {
t.Fatal("pool closed")
}
}
if _, err := MySQL(nil, nil); err == nil {
t.Fatal("nil pool accepted")
}
pool, _ := poolFixture(t)
if _, err := Open(nil, pool, func(gorm.ConnPool) gorm.Dialector { return nil }); err == nil {
t.Fatal("nil dialect accepted")
}
}

View File

@@ -0,0 +1,157 @@
package gormx_test
import (
"bytes"
"context"
"errors"
"net"
"os"
"strconv"
"strings"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database/gormx"
adapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"
"git.yuntuo.hk/k2safe/go-omni/logx"
"gorm.io/gorm"
)
type record struct {
ID int64 `gorm:"primaryKey;autoIncrement:false"`
Name string
}
func (record) TableName() string { return "gorm_adapter_records" }
func TestGORMMySQLTransactionsCancellationAndLogs(t *testing.T) {
if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" {
t.Skip("use scripts/verifyConnector.py --gorm with isolated MySQL")
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
cfg := adapter.DefaultConfig()
host, port, err := net.SplitHostPort(os.Getenv("GO_OMNI_CONNECTOR_ADDR"))
if err != nil {
t.Fatal(err)
}
cfg.Host = host
cfg.Port, _ = strconv.Atoi(port)
cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER")
cfg.Password = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD")
cfg.Database = os.Getenv("GO_OMNI_CONNECTOR_DATABASE")
cfg.TLS.CAFile = os.Getenv("GO_OMNI_CONNECTOR_CA_FILE")
cfg.TLS.ServerName = "localhost"
cfg.Pool.MaxOpenConns = 2
cfg.Pool.MaxIdleConns = 2
pool, err := adapter.Open(ctx, cfg)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
var logs bytes.Buffer
logger, err := logx.New(logx.Config{Writer: &logs})
if err != nil {
t.Fatal(err)
}
defer logger.Close()
logConfig := gormx.DefaultLogConfig()
logConfig.Level = "info"
orm, err := gormx.MySQL(ctx, pool, gormx.WithLogger(logger), gormx.WithLogConfig(logConfig), gormx.WithConfig(gorm.Config{PrepareStmt: true, TranslateError: true}))
if err != nil {
t.Fatal(err)
}
sqlDB, err := orm.DB()
if err != nil || sqlDB != pool.DB {
t.Fatal("pool not shared", err)
}
if sqlDB.Stats().MaxOpenConnections != 2 {
t.Fatal("pool limits changed")
}
var tables int
if err = pool.QueryRowContext(ctx, "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name = 'gorm_adapter_records'").Scan(&tables); err != nil || tables != 0 {
t.Fatal("adapter must not migrate schema", err)
}
if _, err = pool.ExecContext(ctx, "CREATE TABLE gorm_adapter_records (id BIGINT PRIMARY KEY, name VARCHAR(100) UNIQUE NOT NULL)"); err != nil {
t.Fatal(err)
}
defer pool.ExecContext(context.Background(), "DROP TABLE gorm_adapter_records")
requestCtx := logx.WithRequestID(ctx, "gorm-request")
orm = orm.WithContext(requestCtx)
secret := "private-bound-name"
if err = orm.Transaction(func(tx *gorm.DB) error { return tx.Create(&record{ID: 1, Name: secret}).Error }); err != nil {
t.Fatal(err)
}
wanted := errors.New("rollback sentinel")
if err = orm.Transaction(func(tx *gorm.DB) error {
if e := tx.Create(&record{ID: 2, Name: "rollback"}).Error; e != nil {
return e
}
return wanted
}); !errors.Is(err, wanted) {
t.Fatal(err)
}
func() {
defer func() {
if recover() != "rollback panic" {
t.Error("panic changed")
}
}()
_ = orm.Transaction(func(tx *gorm.DB) error {
if e := tx.Create(&record{ID: 3, Name: "panic"}).Error; e != nil {
t.Fatal(e)
}
panic("rollback panic")
})
}()
var count int64
if err = orm.Model(&record{}).Count(&count).Error; err != nil || count != 1 {
t.Fatal("commit/rollback differs", count, err)
}
if err = orm.Transaction(func(tx *gorm.DB) error {
if e := tx.Create(&record{ID: 4, Name: "outer"}).Error; e != nil {
return e
}
if e := tx.Transaction(func(inner *gorm.DB) error {
if e := inner.Create(&record{ID: 5, Name: "inner"}).Error; e != nil {
return e
}
return wanted
}); !errors.Is(e, wanted) {
return errors.New("nested rollback failed")
}
return nil
}); err != nil {
t.Fatal(err)
}
if err = orm.Model(&record{}).Count(&count).Error; err != nil || count != 2 {
t.Fatal(count, err)
}
if err = orm.Create(&record{ID: 6, Name: secret}).Error; !errors.Is(err, gorm.ErrDuplicatedKey) {
t.Fatal("native error translation lost", err)
}
var missing record
if err = orm.First(&missing, "id = ?", 999).Error; !errors.Is(err, gorm.ErrRecordNotFound) {
t.Fatal("not-found semantics changed", err)
}
deadline, stop := context.WithTimeout(requestCtx, 30*time.Millisecond)
start := time.Now()
err = orm.WithContext(deadline).Exec("SELECT SLEEP(3)").Error
stop()
if !errors.Is(err, context.DeadlineExceeded) || time.Since(start) > time.Second {
t.Fatal("query cancellation lost", err)
}
if err = pool.Health(ctx); err != nil {
t.Fatal("pool did not recover", err)
}
if strings.Contains(logs.String(), secret) || !strings.Contains(logs.String(), `"requestId":"gorm-request"`) || !strings.Contains(logs.String(), "gorm.queryFailed") {
t.Fatal("SQL log privacy/correlation differs")
}
var cipher struct {
VariableName string
Value string
}
if err = orm.Raw("SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&cipher).Error; err != nil || cipher.Value == "" {
t.Fatal("GORM bypassed TLS pool", err)
}
}

171
database/gormx/logger.go Normal file
View File

@@ -0,0 +1,171 @@
package gormx
import (
"context"
"errors"
"fmt"
"time"
"unicode/utf8"
"git.yuntuo.hk/k2safe/go-omni/logx"
"gorm.io/gorm"
gormlog "gorm.io/gorm/logger"
)
type LogConfig struct {
Level string `json:"level"` // silent, error, warn (default), info
SlowThreshold time.Duration `json:"slowThreshold"` // zero disables slow-query events
IgnoreRecordNotFound bool `json:"ignoreRecordNotFound"`
IncludeParameters bool `json:"includeParameters"`
IncludeErrorDetails bool `json:"includeErrorDetails"`
MaxSQLBytes int `json:"maxSqlBytes"` // zero uses 4096; bounds emitted SQL text
}
func DefaultLogConfig() LogConfig {
return LogConfig{Level: "warn", SlowThreshold: 200 * time.Millisecond, IgnoreRecordNotFound: true, MaxSQLBytes: 4096}
}
func (c LogConfig) Validate() error {
switch c.Level {
case "", "silent", "error", "warn", "info":
default:
return errors.New("gormx: invalid SQL log level")
}
if c.SlowThreshold < 0 || c.MaxSQLBytes < 0 {
return errors.New("gormx: SQL log limits cannot be negative")
}
return nil
}
type sqlLogger struct {
logger *logx.Logger
config LogConfig
level gormlog.LogLevel
}
// NewLogger adapts logx without changing its level, globals or ownership.
// Bound SQL arguments and raw driver error text are omitted by default. SQL
// literals written directly into a query are not parsed or redacted.
func NewLogger(logger *logx.Logger, config LogConfig) (gormlog.Interface, error) {
if err := config.Validate(); err != nil {
return nil, err
}
if config.MaxSQLBytes == 0 {
config.MaxSQLBytes = 4096
}
level := gormlog.Warn
switch config.Level {
case "silent":
level = gormlog.Silent
case "error":
level = gormlog.Error
case "info":
level = gormlog.Info
}
return &sqlLogger{logger: logger, config: config, level: level}, nil
}
func (l *sqlLogger) LogMode(level gormlog.LogLevel) gormlog.Interface {
copy := *l
if level < gormlog.Silent || level > gormlog.Info {
level = gormlog.Silent
}
copy.level = level
return &copy
}
func (l *sqlLogger) output(ctx context.Context) *logx.Logger {
if l.logger != nil {
return l.logger
}
return logx.FromContext(ctx)
}
func (l *sqlLogger) message(msg string, args []any) string {
if !l.config.IncludeErrorDetails {
copy := append([]any(nil), args...)
for i, v := range copy {
if e, ok := v.(error); ok {
copy[i] = errorKind(e)
}
}
args = copy
}
return clip(fmt.Sprintf(msg, args...), l.config.MaxSQLBytes)
}
func (l *sqlLogger) Info(ctx context.Context, msg string, args ...any) {
if l.level >= gormlog.Info {
l.output(ctx).InfoContext(ctx, "gorm.message", "message", l.message(msg, args))
}
}
func (l *sqlLogger) Warn(ctx context.Context, msg string, args ...any) {
if l.level >= gormlog.Warn {
l.output(ctx).WarnContext(ctx, "gorm.message", "message", l.message(msg, args))
}
}
func (l *sqlLogger) Error(ctx context.Context, msg string, args ...any) {
if l.level >= gormlog.Error {
l.output(ctx).ErrorContext(ctx, "gorm.message", "message", l.message(msg, args))
}
}
func (l *sqlLogger) ParamsFilter(_ context.Context, query string, args ...any) (string, []any) {
if !l.config.IncludeParameters {
return query, nil
}
return query, args
}
func (l *sqlLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
if l.level == gormlog.Silent {
return
}
elapsed := time.Since(begin)
failure := err != nil && !(l.config.IgnoreRecordNotFound && errors.Is(err, gorm.ErrRecordNotFound))
slow := l.config.SlowThreshold > 0 && elapsed > l.config.SlowThreshold
level := gormlog.Info
if failure && l.level >= gormlog.Error {
level = gormlog.Error
} else if slow && l.level >= gormlog.Warn {
level = gormlog.Warn
} else if l.level < gormlog.Info {
return
}
query, rows := fc()
attrs := []any{"durationMs", float64(elapsed) / float64(time.Millisecond), "rowsAffected", rows, "sql", clip(query, l.config.MaxSQLBytes)}
if level == gormlog.Error {
attrs = append(attrs, "errorKind", errorKind(err))
if l.config.IncludeErrorDetails {
attrs = append(attrs, "error", clip(err.Error(), l.config.MaxSQLBytes))
}
}
logger := l.output(ctx)
switch level {
case gormlog.Error:
logger.ErrorContext(ctx, "gorm.queryFailed", attrs...)
case gormlog.Warn:
logger.WarnContext(ctx, "gorm.slowQuery", attrs...)
default:
logger.InfoContext(ctx, "gorm.query", attrs...)
}
}
func errorKind(err error) string {
switch {
case errors.Is(err, context.Canceled):
return "canceled"
case errors.Is(err, context.DeadlineExceeded):
return "deadlineExceeded"
case errors.Is(err, gorm.ErrRecordNotFound):
return "recordNotFound"
default:
return fmt.Sprintf("%T", err)
}
}
func clip(value string, limit int) string {
if len(value) <= limit {
return value
}
value = value[:limit]
for !utf8.ValidString(value) && len(value) > 0 {
value = value[:len(value)-1]
}
return value + "…"
}
var _ gormlog.Interface = (*sqlLogger)(nil)
var _ gorm.ParamsFilter = (*sqlLogger)(nil)

View File

@@ -0,0 +1,92 @@
package gormx
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"strings"
"sync"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/logx"
"gorm.io/gorm"
gormlog "gorm.io/gorm/logger"
)
func TestSQLLoggerLevelsPrivacyAndCorrelation(t *testing.T) {
var out bytes.Buffer
base, _ := logx.New(logx.Config{Writer: &out})
l, _ := NewLogger(base, DefaultLogConfig())
ctx := logx.WithRequestID(logx.WithTraceID(context.Background(), "trace-1"), "request-1")
count := 0
fc := func() (string, int64) { count++; return "SELECT * FROM records WHERE token = ?", 1 }
l.Trace(ctx, time.Now(), fc, nil)
l.Trace(ctx, time.Now(), fc, fmt.Errorf("wrapped: %w", gorm.ErrRecordNotFound))
l.LogMode(gormlog.Silent).Trace(ctx, time.Now(), fc, errors.New("password-value"))
if count != 0 {
t.Fatal("ignored event evaluated SQL")
}
l.Trace(ctx, time.Now().Add(-time.Second), fc, nil)
l.Trace(ctx, time.Now(), fc, errors.New("password-value"))
l.Error(ctx, "failed: %v", errors.New("password-value"))
l.LogMode(gormlog.Info).Trace(ctx, time.Now(), fc, nil)
if count != 3 {
t.Fatal(count)
}
if strings.Contains(out.String(), "password-value") {
t.Fatal("driver detail leaked", out.String())
}
for _, text := range []string{"gorm.slowQuery", "gorm.queryFailed", "gorm.query", `"requestId":"request-1"`, `"traceId":"trace-1"`} {
if !strings.Contains(out.String(), text) {
t.Fatal(text, out.String())
}
}
q, args := l.(gorm.ParamsFilter).ParamsFilter(ctx, "SELECT ?", "private")
if q != "SELECT ?" || len(args) != 0 {
t.Fatal("default SQL arguments exposed")
}
// LogMode clones rather than mutating the shared instance.
l.Trace(ctx, time.Now(), fc, nil)
if count != 3 {
t.Fatal("session mode changed root logger")
}
}
func TestSQLLoggerExplicitDetailsAndConcurrentModes(t *testing.T) {
var out bytes.Buffer
base, _ := logx.New(logx.Config{Writer: &out})
cfg := DefaultLogConfig()
cfg.IncludeParameters = true
cfg.IncludeErrorDetails = true
cfg.Level = "info"
cfg.MaxSQLBytes = 8
l, err := NewLogger(base, cfg)
if err != nil {
t.Fatal(err)
}
_, args := l.(gorm.ParamsFilter).ParamsFilter(context.Background(), "SELECT ?", "value")
if len(args) != 1 {
t.Fatal("explicit arguments dropped")
}
var wg sync.WaitGroup
for i := 0; i < 20; i++ {
wg.Add(1)
go func() {
defer wg.Done()
l.LogMode(gormlog.Error).Trace(context.Background(), time.Now(), func() (string, int64) { return "SELECT 中文标识符", -1 }, errors.New("failure"))
}()
}
wg.Wait()
for _, line := range strings.Split(strings.TrimSpace(out.String()), "\n") {
var event map[string]any
if err = json.Unmarshal([]byte(line), &event); err != nil {
t.Fatal(err)
}
if event["error"] != "failure" || strings.Contains(event["sql"].(string), "<22>") {
t.Fatal(event)
}
}
}

52
database/gormx/pool.go Normal file
View File

@@ -0,0 +1,52 @@
package gormx
import (
"context"
"database/sql"
"sync/atomic"
"gorm.io/gorm"
)
// GORM can call DB().Close on failed initialization. Do not expose the owner's
// sql.DB until initialization has succeeded; SQL methods remain fully usable.
// Dialects using context.Background during version probing are bounded by the
// startup context, which stops affecting queries as soon as Open succeeds.
type borrowedPool struct {
db *sql.DB
startup atomic.Pointer[startupContext]
}
type startupContext struct{ context.Context }
func (p *borrowedPool) operationContext(ctx context.Context) context.Context {
if startup := p.startup.Load(); startup != nil {
return startup.Context
}
return ctx
}
func (p *borrowedPool) GetDBConn() (*sql.DB, error) {
if p.startup.Load() != nil {
return nil, gorm.ErrInvalidDB
}
return p.db, nil
}
func (p *borrowedPool) PrepareContext(ctx context.Context, query string) (*sql.Stmt, error) {
return p.db.PrepareContext(p.operationContext(ctx), query)
}
func (p *borrowedPool) ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error) {
return p.db.ExecContext(p.operationContext(ctx), query, args...)
}
func (p *borrowedPool) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error) {
return p.db.QueryContext(p.operationContext(ctx), query, args...)
}
func (p *borrowedPool) QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row {
return p.db.QueryRowContext(p.operationContext(ctx), query, args...)
}
func (p *borrowedPool) BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error) {
return p.db.BeginTx(p.operationContext(ctx), opts)
}
var _ gorm.ConnPool = (*borrowedPool)(nil)
var _ gorm.TxBeginner = (*borrowedPool)(nil)
var _ gorm.GetDBConnector = (*borrowedPool)(nil)

71
database/mysql/README.md Normal file
View File

@@ -0,0 +1,71 @@
# 官方 MySQL 适配器
本包可选,使用 `database/sql` 和 `go-sql-driver/mysql`,复用框架的唯一连接池、Ping、事务与健康检查。不自动建库/迁移、不重试写操作,也不强制某个 MySQL 版本或业务表。
项目使用 GORM 时,在 svc.Init 中调用 [gormx.MySQL](../gormx/README.md) 即可复用这个池、TLS 和清理流程。
生成项目选择 `--database mysql` 即接入:
```sh
go-omni new --database mysql --module example.com/billing ./billing
```
`etc/config.yaml` 使用结构化配置,环境覆盖优先:
```yaml
database:
host: db.example.com
port: 3306
user: app
database: billing
parseTime: true
location: UTC
charset: utf8mb4
collation: utf8mb4_bin
connectTimeout: 5s
readTimeout: 5s
writeTimeout: 5s
tls:
mode: required
caFile: /etc/certs/database-ca.pem
serverName: db.example.com
pool:
maxOpenConns: 20
maxIdleConns: 5
connMaxLifetime: 30m
connMaxIdleTime: 5m
pingTimeout: 5s
```
通过 `GO_OMNI_DB_PASSWORD` 传密码;还有 HOST、PORT、USER、DATABASE、TLS_CA_FILE、TLS_CERT_FILE、TLS_KEY_FILE、POOL_MAX_OPEN_CONNS 等。业务也可自定义 env 标签。新生成服务不再读取 DSN;独立数据库迁移 CLI 和手写 `database.Open/OpenConnector` 的原接口保留。
默认启用 TLS 并校验系统信任链与主机名;私有 CA 用 caFile,双向 TLS 额外设置 certFile/keyFile。只有明确设置 `tls.mode: disabled` 才使用明文。Unix socket 用 socket,若在 socket 上使用 TLS,必须提供 serverName。不会注册全局 TLS 配置或全局驱动 Logger。
手写入口:
```go
cfg := mysqladapter.DefaultConfig()
// 从项目配置覆盖 cfg;不要打印含密码的完整配置。
db, err := mysqladapter.Open(ctx, cfg, mysqladapter.WithLogger(logger))
if err != nil { return err }
defer db.Close()
// db.DB 是标准 *sql.DB;也可用框架 Health、InTx 等方法。
```
生成项目的 `internal/svc/databaseOptions.go` 是保留的项目扩展位置:
```go
func (c *ServiceContext) DatabaseOptions() []mysqladapter.Option {
return []mysqladapter.Option{
mysqladapter.WithStartupCheck(func(ctx context.Context, db *database.DB) error {
return checkProjectSchema(ctx, db)
}),
}
}
```
检查在 Ping 后运行,共享启动时限;失败或 panic 会关闭新连接池。版本/表结构检查放在这些回调中,不放到框架。回调必须响应 context,并且不能自行关闭借用的 db。依赖尚未完成整体初始化时不要把连接池发布到外部。
`WithDriverConfig(func(*mysql.Config) error)` 支持高级拨号、驱动参数等,配置在回调后克隆;这是项目明确选择的底层扩展。`NewConnector` 返回标准 driver.Connector,供已有基础设施使用;它不接受启动检查选项。任意驱动仍可用核心 `database.OpenConnector`。
错误有安全摘要,errors.Is/As 保留 TLS/驱动/项目检查的原因。配置验证不读证书、不连接数据库;Open 才读取证书、创建池并握手。不要自行打印原始配置或展开可能包含内部信息的 driver 字段给用户。

View File

@@ -0,0 +1,115 @@
package mysql_test
import (
"context"
"crypto/tls"
"errors"
"git.yuntuo.hk/k2safe/go-omni/database"
adapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"
"net"
"os"
"strconv"
"testing"
"time"
)
func TestStructuredMySQLAdapterTLS(t *testing.T) {
if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" {
t.Skip("use scripts/verifyConnector.py with an isolated local MySQL")
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
cfg := adapter.DefaultConfig()
host, port, err := net.SplitHostPort(os.Getenv("GO_OMNI_CONNECTOR_ADDR"))
if err != nil {
t.Fatal(err)
}
cfg.Host = host
cfg.Port, _ = strconv.Atoi(port)
cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER")
cfg.Password = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD")
cfg.Database = os.Getenv("GO_OMNI_CONNECTOR_DATABASE")
cfg.TLS.CAFile = os.Getenv("GO_OMNI_CONNECTOR_CA_FILE")
cfg.TLS.ServerName = "localhost"
cfg.Pool.MaxOpenConns = 2
cfg.Pool.MaxIdleConns = 2
cfg.Pool.PingTimeout = 2 * time.Second
cfg.Session = map[string]string{"time_zone": "'+00:00'"}
checked := false
first, err := adapter.Open(ctx, cfg, adapter.WithStartupCheck(func(ctx context.Context, db *database.DB) error {
var one int
err := db.QueryRowContext(ctx, "SELECT 1").Scan(&one)
checked = one == 1
return err
}))
if err != nil {
t.Fatal(err)
}
defer first.Close()
if !checked {
t.Fatal("startup check not called")
}
cfg.Session = map[string]string{"time_zone": "'+08:00'"}
second, err := adapter.Open(ctx, cfg)
if err != nil {
t.Fatal(err)
}
defer second.Close()
for i, db := range []*database.DB{first, second} {
var key, cipher, zone string
if err = db.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&key, &cipher); err != nil || cipher == "" {
t.Fatal("not encrypted", err)
}
if err = db.QueryRowContext(ctx, "SELECT @@session.time_zone").Scan(&zone); err != nil || zone != []string{"+00:00", "+08:00"}[i] {
t.Fatal("session isolation failed", zone, err)
}
if db.Stats().MaxOpenConnections != 2 {
t.Fatal("pool limit ignored")
}
}
for _, bad := range []string{"untrusted", "hostname"} {
invalid := cfg
if bad == "untrusted" {
invalid.TLS.CAFile = ""
} else {
invalid.TLS.ServerName = "wrong.invalid"
}
db, e := adapter.Open(ctx, invalid)
if db != nil {
_ = db.Close()
t.Fatal("bad certificate accepted")
}
var verify *tls.CertificateVerificationError
if !errors.As(e, &verify) {
t.Fatal("lost TLS verification cause", e)
}
}
cause := errors.New("project schema check failed")
if db, e := adapter.Open(ctx, cfg, adapter.WithStartupCheck(func(context.Context, *database.DB) error { return cause })); db != nil || !errors.Is(e, cause) {
t.Fatal("failed startup escaped", e)
}
if err = first.Health(ctx); err != nil {
t.Fatal("failed pool affected healthy pool", err)
}
// The server requires a client certificate for this separate fixture user.
cfg.User = "omni_mtls"
cfg.Password = ""
cfg.Database = ""
cfg.Session = nil
if db, e := adapter.Open(ctx, cfg); e == nil {
db.Close()
t.Fatal("missing required client certificate accepted")
}
cfg.TLS.CertFile = os.Getenv("GO_OMNI_CONNECTOR_CLIENT_CERT")
cfg.TLS.KeyFile = os.Getenv("GO_OMNI_CONNECTOR_CLIENT_KEY")
mtls, err := adapter.Open(ctx, cfg)
if err != nil {
t.Fatal("mTLS failed", err)
}
if err = mtls.Close(); err != nil {
t.Fatal(err)
}
if err = mtls.Close(); err != nil {
t.Fatal("pool close is not idempotent", err)
}
}

322
database/mysql/mysql.go Normal file
View File

@@ -0,0 +1,322 @@
// Package mysql provides an optional MySQL adapter over database.OpenConnector.
// Each pool owns independent driver settings; no global driver registry changes
// are made. It never migrates tables or retries application transactions.
package mysql
import (
"context"
"crypto/tls"
"crypto/x509"
"database/sql/driver"
"errors"
"fmt"
"io"
"net"
"os"
"regexp"
"strconv"
"strings"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"git.yuntuo.hk/k2safe/go-omni/logx"
mysqldriver "github.com/go-sql-driver/mysql"
)
type Config struct {
Host string `json:"host"`
Port int `json:"port"`
Socket string `json:"socket"`
User string `json:"user"`
Password string `json:"password" env:",secret"`
Database string `json:"database"`
ParseTime bool `json:"parseTime"`
Location string `json:"location"`
Charset string `json:"charset"`
Collation string `json:"collation"`
// Session values are SQL expressions, as required by the driver, for
// example time_zone: "'+00:00'". They are trusted deployment configuration.
Session map[string]string `json:"session"`
ConnectTimeout time.Duration `json:"connectTimeout"`
ReadTimeout time.Duration `json:"readTimeout"`
WriteTimeout time.Duration `json:"writeTimeout"`
MultiStatements bool `json:"multiStatements"`
TLS TLSConfig `json:"tls"`
Pool database.PoolConfig `json:"pool"`
}
type TLSConfig struct {
Mode string `json:"mode"` // required (default) or disabled
CAFile string `json:"caFile"`
CertFile string `json:"certFile"`
KeyFile string `json:"keyFile"`
ServerName string `json:"serverName"`
}
func DefaultConfig() Config {
return Config{Host: "127.0.0.1", Port: 3306, ParseTime: true, Location: "UTC", Charset: "utf8mb4", ConnectTimeout: 5 * time.Second, ReadTimeout: 5 * time.Second, WriteTimeout: 5 * time.Second, TLS: TLSConfig{Mode: "required"}, Pool: database.DefaultPoolConfig()}
}
var identifier = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
// Validate does not read certificate files or open connections. All defaults
// come from DefaultConfig; explicit zero I/O timeouts mean no driver deadline.
func (c Config) Validate() error {
if c.Socket == "" && (c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") || c.Port < 1 || c.Port > 65535) {
return errors.New("mysql: invalid host or port")
}
if strings.ContainsRune(c.Socket, 0) || strings.ContainsRune(c.Database, 0) || strings.ContainsRune(c.User, 0) {
return errors.New("mysql: invalid connection field")
}
if c.ConnectTimeout < 0 || c.ReadTimeout < 0 || c.WriteTimeout < 0 {
return errors.New("mysql: timeouts cannot be negative")
}
if err := c.Pool.Validate(); err != nil {
return err
}
if c.Location != "" {
if _, err := time.LoadLocation(c.Location); err != nil {
return errors.New("mysql: invalid time location")
}
}
if c.Charset != "" && !identifier.MatchString(c.Charset) {
return errors.New("mysql: invalid charset")
}
if c.Collation != "" && !identifier.MatchString(c.Collation) {
return errors.New("mysql: invalid collation")
}
for key := range c.Session {
if !identifier.MatchString(key) {
return errors.New("mysql: invalid session variable name")
}
}
if c.TLS.Mode != "" && c.TLS.Mode != "required" && c.TLS.Mode != "disabled" {
return errors.New("mysql: TLS mode must be required or disabled")
}
if (c.TLS.CertFile == "") != (c.TLS.KeyFile == "") {
return errors.New("mysql: TLS client certificate and key must be configured together")
}
if c.TLS.Mode == "disabled" && (c.TLS.CAFile != "" || c.TLS.CertFile != "" || c.TLS.ServerName != "") {
return errors.New("mysql: TLS fields cannot be used when disabled")
}
if c.Socket != "" && c.TLS.Mode != "disabled" && c.TLS.ServerName == "" {
return errors.New("mysql: TLS over a unix socket requires serverName")
}
return nil
}
// Error has a safe summary; errors.Is/As retain driver/verification causes.
// Do not log raw configuration or a driver's manually extracted error fields.
type Error struct {
Operation string
cause error
}
func (e *Error) Error() string { return "mysql: " + e.Operation + " failed" }
func (e *Error) Unwrap() error { return e.cause }
type Check func(context.Context, *database.DB) error
type options struct {
logger *logx.Logger
checks []Check
configure func(*mysqldriver.Config) error
}
type Option func(*options)
func WithLogger(l *logx.Logger) Option { return func(o *options) { o.logger = l } }
// WithStartupCheck runs after Ping, before ownership reaches the caller. A
// failure closes the pool. Checks must honor ctx and must not close the pool.
func WithStartupCheck(check Check) Option {
return func(o *options) {
if check != nil {
o.checks = append(o.checks, check)
}
}
}
// WithDriverConfig provides an advanced, per-pool escape hatch. Configuration
// is cloned after this callback; keep credentials out of callback errors/logs.
func WithDriverConfig(fn func(*mysqldriver.Config) error) Option {
return func(o *options) { o.configure = fn }
}
func settings(opts []Option) options {
var o options
for _, opt := range opts {
if opt != nil {
opt(&o)
}
}
return o
}
// Open returns the existing Omni pool; .DB is the same standard sql.DB for
// repositories or ORMs. Ping and startup checks share one bounded deadline.
func Open(ctx context.Context, cfg Config, opts ...Option) (db *database.DB, err error) {
if ctx == nil {
ctx = context.Background()
}
if err = ctx.Err(); err != nil {
return nil, err
}
o := settings(opts)
if o.logger == nil {
o.logger = logx.FromContext(ctx)
}
connector, err := connector(cfg, o)
if err != nil {
return nil, err
}
return openPool(ctx, connector, cfg.Pool, o.checks)
}
func openPool(ctx context.Context, connector driver.Connector, pool database.PoolConfig, checks []Check) (db *database.DB, err error) {
checkCtx, cancel := context.WithTimeout(ctx, pool.PingTimeout)
defer cancel()
db, err = database.OpenConnector(checkCtx, connector, pool)
if err != nil {
return nil, &Error{Operation: "connect", cause: err}
}
defer func() {
if p := recover(); p != nil {
_ = db.Close()
panic(p)
}
if err != nil {
err = errors.Join(err, db.Close())
db = nil
}
}()
for _, check := range checks {
if err = check(checkCtx, db); err != nil {
return db, &Error{Operation: "startup check", cause: err}
}
if err = checkCtx.Err(); err != nil {
return db, &Error{Operation: "startup check", cause: err}
}
}
return db, nil
}
// NewConnector builds a standard connector for callers that own pool creation.
// Startup checks are only executed by Open.
func NewConnector(cfg Config, opts ...Option) (driver.Connector, error) {
o := settings(opts)
if len(o.checks) > 0 {
return nil, errors.New("mysql: startup checks require Open, not NewConnector")
}
return connector(cfg, o)
}
func connector(cfg Config, o options) (driver.Connector, error) {
d, err := driverConfig(cfg, o)
if err != nil {
return nil, err
}
c, err := mysqldriver.NewConnector(d)
if err != nil {
return nil, &Error{Operation: "connector configuration", cause: err}
}
return c, nil
}
func driverConfig(cfg Config, o options) (*mysqldriver.Config, error) {
if err := cfg.Validate(); err != nil {
return nil, err
}
d := mysqldriver.NewConfig()
d.User, d.Passwd, d.DBName = cfg.User, cfg.Password, cfg.Database
d.Net, d.Addr = "tcp", net.JoinHostPort(cfg.Host, strconv.Itoa(cfg.Port))
if cfg.Socket != "" {
d.Net, d.Addr = "unix", cfg.Socket
}
d.ParseTime = cfg.ParseTime
if cfg.Location != "" {
d.Loc, _ = time.LoadLocation(cfg.Location)
}
if cfg.Charset != "" {
if err := d.Apply(mysqldriver.Charset(cfg.Charset, cfg.Collation)); err != nil {
return nil, &Error{Operation: "charset", cause: err}
}
} else {
d.Collation = cfg.Collation
}
d.Params = make(map[string]string, len(cfg.Session))
for k, v := range cfg.Session {
d.Params[k] = v
}
d.Timeout, d.ReadTimeout, d.WriteTimeout = cfg.ConnectTimeout, cfg.ReadTimeout, cfg.WriteTimeout
d.MultiStatements = cfg.MultiStatements
logger := o.logger
if logger == nil {
logger = logx.FromContext(nil)
}
d.Logger = driverLog{logger: logger, secret: cfg.Password}
if cfg.TLS.Mode != "disabled" {
name := cfg.TLS.ServerName
if name == "" {
name = cfg.Host
}
t := &tls.Config{MinVersion: tls.VersionTLS12, ServerName: name}
if cfg.TLS.CAFile != "" {
pem, err := readCertificate(cfg.TLS.CAFile)
if err != nil {
return nil, &Error{Operation: "read CA", cause: err}
}
t.RootCAs = x509.NewCertPool()
if !t.RootCAs.AppendCertsFromPEM(pem) {
return nil, errors.New("mysql: invalid CA certificate")
}
}
if cfg.TLS.CertFile != "" {
cert, err := readCertificate(cfg.TLS.CertFile)
if err != nil {
return nil, &Error{Operation: "read client certificate", cause: err}
}
key, err := readCertificate(cfg.TLS.KeyFile)
if err != nil {
return nil, &Error{Operation: "read client key", cause: err}
}
pair, err := tls.X509KeyPair(cert, key)
if err != nil {
return nil, &Error{Operation: "client key pair", cause: err}
}
t.Certificates = []tls.Certificate{pair}
}
d.TLS = t
}
if o.configure != nil {
if err := o.configure(d); err != nil {
return nil, &Error{Operation: "custom configuration", cause: err}
}
}
return d.Clone(), nil
}
func readCertificate(path string) ([]byte, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
data, err := io.ReadAll(io.LimitReader(f, (4<<20)+1))
if err != nil {
return nil, err
}
if len(data) > 4<<20 {
return nil, errors.New("certificate file exceeds 4 MiB")
}
return data, nil
}
type driverLog struct {
logger *logx.Logger
secret string
}
func (l driverLog) Print(values ...any) {
message := fmt.Sprint(values...)
if l.secret != "" {
message = strings.ReplaceAll(message, l.secret, "[REDACTED]")
}
l.logger.Error("database.driver", "driver", "mysql", "error", message)
}

View File

@@ -0,0 +1,146 @@
package mysql
import (
"bytes"
"context"
"crypto/tls"
"database/sql/driver"
"errors"
"net"
"os"
"path/filepath"
"strings"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"git.yuntuo.hk/k2safe/go-omni/logx"
mysqldriver "github.com/go-sql-driver/mysql"
)
func TestConfigurationTLSLoggingAndIsolation(t *testing.T) {
cfg := DefaultConfig()
cfg.User = "test"
cfg.Password = "do-not-print"
cfg.Database = "mixed-name"
cfg.Session = map[string]string{"time_zone": "'+00:00'"}
cfg.Collation = "utf8mb4_bin"
var out bytes.Buffer
logger, _ := logx.New(logx.Config{Writer: &out})
d, err := driverConfig(cfg, settings([]Option{WithLogger(logger)}))
if err != nil {
t.Fatal(err)
}
if d.TLS == nil || d.TLS.MinVersion != tls.VersionTLS12 || d.TLS.ServerName != cfg.Host || d.MultiStatements || d.Params["time_zone"] != "'+00:00'" {
t.Fatal("driver policy differs")
}
cfg.Session["time_zone"] = "'+08:00'"
if d.Params["time_zone"] != "'+00:00'" {
t.Fatal("session map alias")
}
d.Logger.Print("connection ", cfg.Password)
if strings.Contains(out.String(), cfg.Password) || !strings.Contains(out.String(), "[REDACTED]") {
t.Fatal("secret leaked")
}
cfg.TLS.Mode = "disabled"
d2, err := driverConfig(cfg, options{})
if err != nil || d2.TLS != nil || d.TLS == nil {
t.Fatal("TLS configuration crossed instances", err)
}
cfg.TLS.CAFile = "unused"
if err = cfg.Validate(); err == nil {
t.Fatal("disabled TLS accepted CA")
}
cfg = DefaultConfig()
cfg.TLS.CAFile = filepath.Join(t.TempDir(), "missing")
if err = cfg.Validate(); err != nil {
t.Fatal("Validate reads certificates", err)
}
if _, err = NewConnector(cfg); err == nil {
t.Fatal("missing CA accepted")
}
if err = os.WriteFile(cfg.TLS.CAFile, []byte("do-not-print"), 0600); err != nil {
t.Fatal(err)
}
if _, err = NewConnector(cfg); err == nil || strings.Contains(err.Error(), "do-not-print") {
t.Fatal(err)
}
if _, err = NewConnector(DefaultConfig(), WithStartupCheck(func(context.Context, *database.DB) error { return nil })); err == nil {
t.Fatal("unused startup check accepted")
}
}
func TestDriverHandshakeErrorUsesIndependentLogger(t *testing.T) {
var a, b bytes.Buffer
la, _ := logx.New(logx.Config{Writer: &a})
lb, _ := logx.New(logx.Config{Writer: &b})
cfg := DefaultConfig()
cfg.TLS.Mode = "disabled"
cfg.Pool.PingTimeout = time.Second
for i, l := range []*logx.Logger{la, lb} {
before := []int{a.Len(), b.Len()}
db, err := Open(context.Background(), cfg, WithLogger(l), WithDriverConfig(func(d *mysqldriver.Config) error {
d.DialFunc = func(context.Context, string, string) (net.Conn, error) {
reader, writer := net.Pipe()
_ = writer.Close()
return reader, nil
}
return nil
}))
if db != nil || err == nil {
t.Fatal("broken handshake succeeded")
}
sizes := []int{a.Len(), b.Len()}
if sizes[i] <= before[i] || sizes[1-i] != before[1-i] {
t.Fatal("driver log isolation failed")
}
}
}
type testConnector struct{ closed atomic.Int32 }
func (c *testConnector) Connect(context.Context) (driver.Conn, error) { return &testConn{c}, nil }
func (c *testConnector) Driver() driver.Driver { return testDriver{} }
type testDriver struct{}
func (testDriver) Open(string) (driver.Conn, error) { return nil, errors.New("unexpected DSN") }
type testConn struct{ owner *testConnector }
func (c *testConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("unused") }
func (c *testConn) Begin() (driver.Tx, error) { return nil, errors.New("unused") }
func (c *testConn) Ping(context.Context) error { return nil }
func (c *testConn) Close() error { c.owner.closed.Add(1); return nil }
func TestStartupCheckRollbackAndPanic(t *testing.T) {
for _, panics := range []bool{false, true} {
c := &testConnector{}
cause := errors.New("schema failed")
func() {
if panics {
defer func() {
if recover() == nil {
t.Error("panic lost")
}
}()
}
db, err := openPool(context.Background(), c, database.DefaultPoolConfig(), []Check{func(ctx context.Context, db *database.DB) error {
if _, ok := ctx.Deadline(); !ok {
t.Error("unbounded check")
}
if panics {
panic("failed check")
}
return cause
}})
if db != nil || !errors.Is(err, cause) {
t.Error("failed pool escaped or cause lost", err)
}
}()
if c.closed.Load() != 1 {
t.Fatal("failed initialization leaked pool", c.closed.Load())
}
}
}

23
errors/export.go Normal file
View File

@@ -0,0 +1,23 @@
package apierrors
import (
"encoding/json"
"errors"
"io"
)
// ExportCatalog writes normalized, public-only error metadata. Invoke this
// explicitly from project tooling, using the same Catalog function registered
// at runtime. The API CLI reads the artifact; it never executes project code.
func ExportCatalog(w io.Writer, c Catalog) error {
if w == nil {
return errors.New("errors: catalog export writer required")
}
registry, err := NewRegistry(c)
if err != nil {
return err
}
encoder := json.NewEncoder(w)
encoder.SetIndent("", " ")
return encoder.Encode(registry.Catalog())
}

40
errors/export_test.go Normal file
View File

@@ -0,0 +1,40 @@
package apierrors
import (
"bytes"
"errors"
"strings"
"testing"
)
type failedExport struct{ err error }
func (w failedExport) Write([]byte) (int, error) { return 0, w.err }
func TestCatalogExport(t *testing.T) {
catalog := Catalog{DefaultStatus: 200, Definitions: map[string]Definition{"INVALID": {Code: int64(9007199254740993), Message: "invalid"}}}
var out bytes.Buffer
if err := ExportCatalog(&out, catalog); err != nil {
t.Fatal(err)
}
if !strings.Contains(out.String(), "9007199254740993") {
t.Fatal(out.String())
}
var decoded Catalog
if err := DecodeCatalogJSON(out.Bytes(), &decoded); err != nil {
t.Fatal(err)
}
registry, err := NewRegistry(decoded)
if err != nil {
t.Fatal(err)
}
if registry.Resolve(ByCode("INVALID")).Status != 200 {
t.Fatal("status lost")
}
cause := errors.New("write failed")
if err = ExportCatalog(failedExport{cause}, catalog); !errors.Is(err, cause) {
t.Fatal("writer failure lost", err)
}
if err = ExportCatalog(nil, catalog); err == nil {
t.Fatal("nil writer accepted")
}
}

View File

@@ -0,0 +1,61 @@
syntax = "v1"
@app (
defaultResponse: CommonResponse
request: {"unknownQuery":"reject","utf8":"reject","charset":"utf8"}
)
response CommonResponse (
codeType: int32
successCode: 0
errorCode: 9999
messageField: msg
dataMode: always
requestIdField: "-"
)
type Item {
Name string `json:"name"`
}
type Query {
Mode string `form:"mode,optional"`
Tags []string `form:"tags,optional"`
}
type Upload {
Title string `multipart:"title" binding:"required,max=20"`
Metadata Item `multipartJSON:"metadata"`
Attachment File `file:"attachment,maxBytes=1024"`
Extras []File `file:"extras,optional,maxFiles=2,maxBytes=512"`
}
@server (group: demo; middleware: RequireAccess)
service business {
@handler upload
@body (mode: multipart; maxBytes: 4096; maxParts: 6; maxFieldBytes: 128)
post /upload (Upload) returns (Item)
@handler list
get /list (Query) returns ([]Item)
@handler nullable
get /nullable (Query) returns (*Item)
@handler nothing
post /nothing returns (null)
@handler rawNull
@response raw
get /rawNull returns (null)
@handler rawArray
@response raw
get /rawArray returns ([]Item)
@handler empty
@request (body: emptyObject)
post /empty returns (null)
@handler absent
@request (body: none)
post /absent returns (null)
@handler optional
@request (body: optional)
post /optional (Item) returns (Item)
@handler echo
post /echo (Item) returns (Item)
@handler tolerant
@request (unknownQuery: ignore; utf8: replace; charset: any)
post /tolerant (Item) returns (Item)
@handler noContent
delete /noContent
}

View File

@@ -1,4 +1,5 @@
{
"templateVersion": "v0.15.0-preview",
"profile": {
"database": "none",
"storage": "none",
@@ -9,15 +10,17 @@
"module": "example.com/managed",
"entry": "api/managed.api",
"generated": {
"docs/openapi.json": "9eb707aefeefd3f4881c774534e2d143949c0edb1d8a4aabd762f978f6203df1",
"internal/handler/appGen.go": "c6f078932e1a598a46faebba397f4e753db9d8f4f29a02f9b6077a0557d6c14e",
"internal/handler/demo/handlersGen.go": "7a6f9d5e105e44b07196cc540cee3f0562d5f3d1c5f0bc094434db1064d0339c",
"internal/handler/openapiGen.go": "612920ed25e5dc7a4f673be56041d47b70aa3216707ff282e52dfe503c2d705a",
"docs/openapi.json": "68c7e9e0294bf62937aa953a44732288c51571108c922a36a72faa72c4360b5f",
"internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d",
"internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375",
"internal/handler/openapiGen.go": "ae9b794c63e0f3c8f9d200a5fc6977350139d4a6f00b42a26102c8630d907d03",
"internal/handler/responseGen.go": "84397a502bba4777ebfa6fae3d3bc418297acab4f26b04b42e3c31b36c735399",
"internal/handler/routesGen.go": "76c1a5b2644e75e56553dc6e9e862a63128d902dd94807ce7c8aee5491cd1e3e",
"internal/middleware/registryGen.go": "e5f47efa57a1432765ef3419bae59dbf3b4d55b3aac0a6c935ce7a52ba6b47f4",
"internal/types/typesGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344",
"main.go": "a12db5b7df0f2fdca4939a7382daee04c85ddb63ef0bccd1a2ce26b718f42579"
"internal/svc/runtimeGen.go": "29f6afb6614881935099c1845ecb3f7902ee773caa077940f566cb8dfdbcdbb1",
"internal/types/dtoManagedd28645a2af1cGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344",
"internal/types/typesGen.go": "372d67f23a313b3e77d2a3b7a236315db9721bc63350bf1f71db1293e57570d1",
"main.go": "64e26d97de7e7c6c17f47186785edc73d43c23d6c20b50bfc7fff6ef2e914f17"
},
"contract": {
"routes": {

View File

@@ -34,149 +34,18 @@ to fail on unresolved signature errors after generation. Run `go test ./...` too
especially for project type aliases, build tags and changes inside method bodies.
This project is generated from an editable API contract. Business code uses
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
main.go and HTTP composition are continuously generator-owned.
## Framework-managed runtime
## Run
`main.go`, `internal/svc/runtimeGen.go`, generated handlers, `internal/types/*Gen.go`
and OpenAPI are generator-owned. Add project dependencies in `svc.Init(ctx)` and
register each successful initialization with `OnClose`. Runtime owns reverse
cleanup and failure rollback; do not copy locks, constructors or Close methods
into this project. The logger is scoped to this App.
```sh
go mod tidy
go run .
```
GET /health/live and /health/ready check the running service. GET /openapi.json
serves the same document as docs/openapi.json. Bind address defaults to localhost.
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
The greeting starter works immediately; other new operation stubs return 501 until
their business logic is implemented. Authentication hooks also fail closed with 501.
## Develop
1. Edit api/managed.api and its relative imports.
2. Run the same go-omni CLI version as the framework dependency:
```sh
go-omni api validate
go-omni api inspect
go-omni api generate --dry-run
go-omni api generate
go-omni api generate --check
go-omni api export > openapi.json
go test ./...
```
Commands inside a generated project discover the recorded API entry automatically.
--dry-run previews create/update/remove actions without writing files.
The --check command is read-only and suitable for CI. It fails on stale or missing
generated files and missing business scaffolds. go.mod, configuration, Logic,
Repository, svc, middleware hook bodies and API files are created once and then
preserved byte for byte. main.go, Gen.go files and OpenAPI are owned
by the generator; manual edits cause a conflict instead of being overwritten.
Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it.
Avoid editing files while generation is running; api.lock serializes generators.
Routes and types that disappear are removed from generated output. Handwritten
business files remain: remove/adapt obsolete implementations explicitly if an
API change makes their old types or signatures fail to compile. Every regeneration
must be followed by go test ./... . Do not delete service files to change contracts.
## Layout
```text
api/ # editable contract and imported types
main.go # GENERATED startup, lifecycle and route registration
etc/config.yaml # lowerCamelCase configuration
internal/config/ # defaults -> file -> environment
internal/svc/ # dependency initialization and cleanup
internal/types/typesGen.go # generated request/response types
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
internal/handler/<group>/ # generated binding and Logic adapters
internal/logic/<group>/ # developer-owned business operations
internal/repository/<group>/ # developer-owned persistence contracts
internal/middleware/ # named hooks and generated registry
docs/openapi.json # generated OpenAPI 3.0.3
.goOmni/apiManifest.json # ownership and content hashes
```
The API response type describes business data. Declare response Name (...) and
select the App default with @app(defaultResponse: Name) in the entry API.
@server(response: Name) and @response Name override it for groups/routes;
@response raw returns the business JSON directly. Without a template, the default
envelope includes code, message, data and requestId. Generated routes and OpenAPI
use the same contract. 404/405 and ordinary App routes use the declared App default.
No template is selected by declaration/import order. Without @app, existing framework
response defaults apply. An explicit API App default takes precedence over response
options in svc, keeping generated routes and OpenAPI consistent.
Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default.
Use @body(maxBytes: ...) for route limits. @body(mode: raw|multipart; maxBytes: ...)
adds a *httpx.Body argument to Logic, without handwritten handlers. Raw is the exact
limited request stream; Parts supports sequential multipart reading with no temp files.
@produces "application/octet-stream" (or another concrete media type) generates a
Logic result of *httpx.Stream; the handler closes Content and streams it unwrapped.
Signature verification and storage policy belong to Logic/middleware. A consumed
request stream cannot be reread automatically; use one reader owner or explicitly
bounded buffering. Stream reads respect configured server timeouts. Multipart parts
are not DTO form fields: form tags remain URL query parameters. The optional manual
handler/custom binder hooks are preserved, and are needed only for custom protocols.
Fields require exactly one source tag: json, path, form (URL query only), header.
All are required by presence unless optional, omitempty or default is specified.
Zero and false are valid present values; binding:"required" rejects scalar zero.
Use *T with optional when business logic needs to distinguish missing from zero.
Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON
fields and duplicate scalar query/header values are rejected. Arrays in the query
use repeated keys. Enum/range/default and nested fields share the documentation
and binding compiler. Responses are expected to honor their declared constraints;
they are not validated on every response. Do not return nil slices/maps for a
non-nullable collection (initialize empty collections instead).
Go-owned errors can be registered once through svc.ServiceContext.AppOptions with
omni.WithErrors(projecterrors.Catalog()). Generated routes inherit them unless they
declare an independent catalog. Do not repeat Go-owned errors in the API contract.
Alternatively, declare public business errors in error (...) and reference them with @errors.
Use internal/apierrors constructors to share status, code and message with OpenAPI.
Auth declarations connect a bearer/header/cookie scheme to a real named middleware.
The selected auth runs first; @auth none removes inherited declarative auth.
Validation policy remains in the middleware implementation. Public messages must
not contain secrets; error causes may be included in server logs.
svc.New receives the loaded Config and borrowed Logger. Add dependencies there,
then immediately register each cleanup with OnClose. A later initialization failure
runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware
provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated
once before route registration; logging is configured earlier through Config.
The generated main runs HTTP draining before closing svc resources, then the logger.
Do not close borrowed resources or attach duplicate cleanup hooks to App. The startup
context cancels on signals; long-lived clients/workers that requests depend on need
their own lifetime, canceled by OnClose after HTTP draining.
Named middleware is composed in declaration order (first is outermost). Implement
the generated hooks or replace entries in svc; never bypass an auth
hook merely to make the server start. Middleware names do not imply a JWT policy.
This CLI supports a documented go-zero-style subset, not all goctl features.
Database tables and repositories cannot be inferred from an HTTP contract. Add
business repositories through svc. Database and storage options are
composed at creation; the original profile is retained on regeneration.
## Infrastructure
Selected database: none; storage: none; logging: stderr/json.
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
No tables are invented or migrated automatically from request/response DTOs.
The none/memory selection opens no database; implement repositories per business.
With storage enabled, services receive deps.Files, Uploads, Downloads and
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
and authorize before issuing them. No public upload-link route is added.
Storage failures during startup close an already-open database; Close is idempotent.
Database/storage/logging selection is recorded in .goOmni/apiManifest.json.
Changing selection through generation flags is rejected on an existing project;
edit your owned configuration and initialization explicitly, or generate a fresh
project for comparison. New routes still regenerate without repeating these flags.
Configuration, API contracts, svc fields/Init/AppOptions, Logic, Repository and
middleware implementations remain developer-owned. Regeneration preserves them.
Use `go-omni api format` to format local contracts, then `api generate` and
`api generate --check`. See the [framework development guide](../../DEVELOPMENT.md)
for array/null responses, typed multipart, full configuration environment overrides,
shared contracts and v0.13 upgrade steps. This example keeps its raw callback,
streaming upload and download to exercise those existing extension points.

View File

@@ -195,6 +195,12 @@
"demo"
],
"x-omni-max-body-bytes": 64,
"x-omni-request-policy": {
"unknownQuery": "",
"utf8": "",
"charset": "",
"body": "auto"
},
"x-omni-response": "CommonResponse"
}
},
@@ -410,6 +416,12 @@
"tags": [
"demo"
],
"x-omni-request-policy": {
"unknownQuery": "",
"utf8": "",
"charset": "",
"body": ""
},
"x-omni-response": "CommonResponse"
}
},
@@ -589,6 +601,12 @@
"tags": [
"demo"
],
"x-omni-request-policy": {
"unknownQuery": "",
"utf8": "",
"charset": "",
"body": ""
},
"x-omni-response": "CommonResponse"
}
},
@@ -804,6 +822,12 @@
"tags": [
"demo"
],
"x-omni-request-policy": {
"unknownQuery": "",
"utf8": "",
"charset": "",
"body": ""
},
"x-omni-response": "AlternateResponse"
}
},
@@ -992,6 +1016,12 @@
"tags": [
"demo"
],
"x-omni-request-policy": {
"unknownQuery": "",
"utf8": "",
"charset": "",
"body": ""
},
"x-omni-response": "raw"
}
},
@@ -1197,6 +1227,12 @@
"tags": [
"demo"
],
"x-omni-request-policy": {
"unknownQuery": "",
"utf8": "",
"charset": "",
"body": ""
},
"x-omni-response": "CommonResponse"
}
},
@@ -1414,9 +1450,27 @@
"demo"
],
"x-omni-max-body-bytes": 2048,
"x-omni-multipart-limits": {
"maxBytes": 2048,
"maxFieldBytes": 65536,
"maxFileBytes": 2048,
"maxParts": 128
},
"x-omni-request-policy": {
"unknownQuery": "",
"utf8": "",
"charset": "",
"body": "auto"
},
"x-omni-response": "CommonResponse"
}
}
},
"x-omni-contract-sources": {
"files": {
"managed.api": "cec14fe0da47a83804bf461be047df3e004ee470cbc3ea330dabc1537d596672"
},
"shared": {}
},
"x-omni-default-response": "CommonResponse"
}

View File

@@ -2,7 +2,7 @@ module example.com/managed
go 1.22.0
require git.yuntuo.hk/k2safe/go-omni v0.13.0-preview
require git.yuntuo.hk/k2safe/go-omni v0.15.0-preview
require (
github.com/gabriel-vasile/mimetype v1.4.3 // indirect

View File

@@ -1,14 +1,11 @@
// Package settings owns this project's configuration; it adds no business identity model.
package config
import (
"errors"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
"os"
"strings"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
type Config struct {
@@ -26,44 +23,40 @@ func Default() Config {
return Config{Server: server}
}
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
// Pass an empty path to use only defaults and environment.
// Load applies defaults -> file -> typed environment -> validation.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string) (Config, error) {
cfg := Default()
var err error
cfg, err = config.ApplyDefaults(cfg)
if err != nil {
return Config{}, err
}
if path != "" {
cfg, err = config.DecodeFile(path, cfg)
if err != nil {
return Config{}, err
}
}
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_"})
if err != nil {
return Config{}, err
}
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
cfg.Server, err = config.ApplyEnv(cfg.Server, "")
if err != nil {
return Config{}, err
}
// A stream explicitly selected by environment overrides a file profile.
if output := strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")); output != "" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE")) == "" {
cfg.Server.Log.File = ""
cfg.Server.Log.Rotation = logx.RotationConfig{}
}
if err = cfg.Validate(); err != nil {
return Config{}, err
}
return cfg, nil
return cfg, cfg.Validate()
}
func (c Config) Validate() error {
if err := c.Server.Validate(); err != nil {
return err
}
if c.Server.Log.Format != "json" && c.Server.Log.Format != "text" {
return errors.New("log.format must be json or text")
}
switch strings.ToLower(c.Server.Log.Level) {
case "debug", "info", "warn", "warning", "error":
default:
return errors.New("invalid log.level")
}
return nil
}

View File

@@ -4,6 +4,7 @@ package handler
import (
"fmt"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/contract"
"example.com/managed/internal/svc"
"git.yuntuo.hk/k2safe/go-omni/config"
@@ -16,11 +17,13 @@ func NewApp(deps *svc.ServiceContext) (*omni.App, error) {
return nil, fmt.Errorf("service context and lifecycle controller required; see PROJECT.md upgrade instructions")
}
cfg := deps.Config.Server
cfg.LogInstance = deps.Logger
for _, option := range deps.AppOptions() {
if option != nil {
option(&cfg)
}
}
cfg.Request = contract.MergePolicy(cfg.Request, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""})
cfg.Response = newResponseTemplate("{\"body\":null,\"errorBody\":null,\"messageField\":\"msg\",\"requestIdField\":\"-\",\"codeType\":\"int32\",\"successCode\":0,\"errorCode\":9999,\"dataMode\":\"always\"}").Config()
cfg.ResponseEncoder = nil
if err := cfg.Validate(); err != nil {

View File

@@ -36,7 +36,7 @@ var checkRequest = contract.MustNew[types.CheckReq]()
func (h *Handler) Check(c *httpx.Context) error {
var req types.CheckReq
if err := checkRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
if err := checkRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &req, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
return err
}
resp, err := h.logic.Check(c.Context(), &req)
@@ -53,7 +53,7 @@ var otherRequest = contract.MustNew[types.CheckReq]()
func (h *Handler) Other(c *httpx.Context) error {
var req types.CheckReq
if err := otherRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
if err := otherRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &req, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
return err
}
resp, err := h.logic.Other(c.Context(), &req)
@@ -65,7 +65,13 @@ func (h *Handler) Other(c *httpx.Context) error {
}
return c.OK(resp)
}
var rawJSONRequest = contract.MustNew[struct{}]()
func (h *Handler) RawJSON(c *httpx.Context) error {
if err := rawJSONRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
return err
}
resp, err := h.logic.RawJSON(c.Context())
if err != nil {
return err
@@ -75,7 +81,13 @@ func (h *Handler) RawJSON(c *httpx.Context) error {
}
return c.OK(resp)
}
var uploadRequest = contract.MustNew[struct{}]()
func (h *Handler) Upload(c *httpx.Context) error {
if err := uploadRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: "auto"}, 2048); err != nil {
return err
}
body, err := httpx.NewBody(c.Request(), "multipart")
if err != nil {
return err
@@ -92,7 +104,13 @@ func (h *Handler) Upload(c *httpx.Context) error {
}
return c.OK(resp)
}
var callbackRequest = contract.MustNew[struct{}]()
func (h *Handler) Callback(c *httpx.Context) error {
if err := callbackRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: "auto"}, 64); err != nil {
return err
}
body, err := httpx.NewBody(c.Request(), "raw")
if err != nil {
return err
@@ -112,7 +130,13 @@ func (h *Handler) Callback(c *httpx.Context) error {
}
return c.WriteStream(200, "text/plain", resp)
}
var downloadRequest = contract.MustNew[struct{}]()
func (h *Handler) Download(c *httpx.Context) error {
if err := downloadRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
return err
}
resp, err := h.logic.Download(c.Context())
if err != nil {
if resp != nil {
@@ -122,7 +146,13 @@ func (h *Handler) Download(c *httpx.Context) error {
}
return c.WriteStream(200, "application/octet-stream", resp)
}
var slowRequest = contract.MustNew[struct{}]()
func (h *Handler) Slow(c *httpx.Context) error {
if err := slowRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
return err
}
resp, err := h.logic.Slow(c.Context())
if err != nil {
return err

File diff suppressed because one or more lines are too long

View File

@@ -1,39 +0,0 @@
package svc
import "errors"
// Close is idempotent, including startup failure and graceful shutdown.
func (c *ServiceContext) Close() error {
if c == nil {
return nil
}
c.closeOnce.Do(func() {
if c.Lifecycle != nil {
c.Lifecycle.RequestShutdown(nil)
}
c.closeMu.Lock()
c.closed = true
cleanups := c.cleanups
c.cleanups = nil
c.closeMu.Unlock()
for i := len(cleanups) - 1; i >= 0; i-- {
c.closeErr = errors.Join(c.closeErr, cleanups[i]())
}
})
return c.closeErr
}
// OnClose transfers a resource cleanup to this context immediately after creation.
// Hooks run once in reverse order, also when later initialization fails.
func (c *ServiceContext) OnClose(cleanup func() error) {
if cleanup == nil {
return
}
c.closeMu.Lock()
defer c.closeMu.Unlock()
if c.closed {
panic("svc: register cleanup after Close")
}
c.cleanups = append(c.cleanups, cleanup)
}

Some files were not shown because too many files have changed in this diff Show More