Compare commits
4 Commits
v0.13.0-pr
...
v0.14.1-pr
| Author | SHA1 | Date | |
|---|---|---|---|
| b8ab6b3521 | |||
| da8356e18b | |||
| f5478cdc76 | |||
| f9afb8f8dd |
22
API.md
22
API.md
@@ -6,6 +6,8 @@
|
||||
|
||||
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
|
||||
|
||||
v0.14 的 [完整开发指南](DEVELOPMENT.md) 说明响应根类型、`@request`、multipart 字段、注释/格式化、Go 错误目录导出与共享契约。
|
||||
|
||||
## 与 go-zero 的对应关系
|
||||
|
||||
调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式:统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写,不引入 go-zero 或 Gin 运行时依赖。
|
||||
@@ -61,7 +63,7 @@ go-omni new --api user.api --module example.com/userService \
|
||||
|
||||
初次选项记录在 `.goOmni/apiManifest.json`,以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错,因为初始化代码属于业务方;可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。new 与 api new 统一生成契约项目,不再使用旧式 CRUD 目录。
|
||||
|
||||
MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
|
||||
MySQL 使用官方 `database/mysql` 结构化适配器,密码从 `GO_OMNI_DB_PASSWORD` 覆盖,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
|
||||
|
||||
HTTP 请求类型不能推导数据库表结构;迁移和 Repository 由业务自己维护。当前目录的完整示例见 [managedService](examples/managedService/README.md)。MySQL、事务审计和鉴权实现可参考 [旧目录用户管理示例](examples/userService/README.md),接入时按 [项目指南](PROJECT.md) 装配到 config/svc/logic。
|
||||
|
||||
@@ -99,21 +101,21 @@ service user-api {
|
||||
|
||||
- 可省略的 `syntax = "v1"`;`info` 中的 title、version 和描述性元数据。
|
||||
- 单独的 `type X { ... }` 与 `type (...)`;基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。
|
||||
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
|
||||
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和未授权的目录越界。跨服务共享须在 `omni.imports.json` 声明 roots,见 [共享契约](DEVELOPMENT.md#两个服务共享契约)。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
|
||||
- 同一 service 名称的多个块;`@server` 的 group、prefix、middleware、auth、errors、response,group 可以写 `admin/user`。
|
||||
- 每个路由必须有 `@handler`;可选单行/多行 `@doc`、`@auth`、`@errors`、`@response`;GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。
|
||||
- 可省略请求或响应;有响应默认 200,无响应默认 204。Omni 扩展 `@status 201`、`@status 202`;204 不允许响应类型,HEAD 不允许响应体,GET/HEAD 不允许 JSON 请求体。
|
||||
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query,不解析 multipart 或表单 body。query 切片用重复键;header/path 使用标量。
|
||||
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`,或声明式上传的 `multipart`、`multipartJSON`、`file`。`form` 始终表示 URL query;query 切片用重复键,header/path 使用标量。上传示例与限额见 [声明式 multipart](DEVELOPMENT.md#声明式-multipart)。
|
||||
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
|
||||
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
|
||||
|
||||
暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
|
||||
暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、未通过 scalar 显式接入的自定义编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
|
||||
|
||||
业务错误、具名鉴权、多行文档及下述开发命令从 CLI `v0.7.0-preview.1` 提供。公共响应模板从 `v0.8.0-preview` 提供,CLI 与业务项目中的框架依赖需一起升级。
|
||||
|
||||
## 业务错误:声明、代码和文档共用一份定义
|
||||
|
||||
**错误目录也可以只在 Go 中维护。** 从 `v0.11.1-preview` 起,生成路由没有声明错误列表时,继承 `omni.WithErrors(catalog)` 注册的 App 目录;`.api` 继续定义接口和返回格式,无需重复登记错误。全局中间件提前返回同样适用。显式引用 `.api` 错误时才使用路由独立目录,规则见 [目录继承](errors/README.md)。CLI 不执行运行时 Go 配置,因此静态文档不会自动枚举只在 Go 中定义的具体错误。
|
||||
**错误目录也可以只在 Go 中维护。** 从 `v0.11.1-preview` 起,生成路由没有声明错误列表时,继承 `omni.WithErrors(catalog)` 注册的 App 目录;`.api` 继续定义接口和返回格式,无需重复登记错误。全局中间件提前返回同样适用。显式引用 `.api` 错误时才使用路由独立目录,规则见 [目录继承](errors/README.md)。CLI 不执行项目 Go 配置;可显式用 `errors.ExportCatalog` 导出 `omni.errors.json`,供文档合并同一目录,见 [导出步骤](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
|
||||
|
||||
从 `v0.10.0-preview` 起,推荐用 `errorCatalog` 同时定义逻辑标识、对外业务码、提示和 HTTP 状态:
|
||||
|
||||
@@ -258,11 +260,11 @@ go test ./...
|
||||
- 动态 JSON 字段 `any` / `interface{}`,包含切片和 map;外围契约仍严格。
|
||||
- `scalar Name (...)` 显式引用项目类型,声明 JSON 输入/输出类型及文本绑定;支持已有 JSON/Text 转换方法。
|
||||
- `@body (mode: json/raw/multipart; maxBytes: ...)`:按路由限制请求体。
|
||||
- `@binding custom` 生成保留的绑定 hook;`@handlerMode manual` 生成保留的整个 Handler,仅用于自定义协议。raw/multipart 默认生成自动 Handler,将受限流交给 Logic。
|
||||
- `@binding custom` 生成保留的绑定 hook;`@handlerMode manual` 生成保留的整个 Handler,仅用于自定义协议。raw 与未声明上传字段的 multipart 将受限流交给 Logic;声明了上传字段时则交给 Logic 已校验的 DTO。
|
||||
- `@produces "text/plain"` 等声明无 DTO 成功输出;自动模式的 Logic 返回 `*httpx.Stream`,Handler 负责流式发送和关闭。默认状态 200。manual 模式保留项目自己的输出实现。
|
||||
- 自动 multipart Handler 检查未读部分和结束边界;损坏/截断默认返回 400,超限为 413。公共状态、码和文案仍可用项目错误目录配置,见 [流式接入与错误处理](PROJECT.md#自动上传回调与下载)。
|
||||
|
||||
迁移扩展起始于 v0.9.0,原有手写扩展说明见 [迁移扩展指南](MIGRATION.md)。当前根目录入口和自动流式 Handler 要求 CLI 与框架依赖一起使用 v0.13.0-preview;版本说明和新目录维护规则见 [项目指南](PROJECT.md)。不兼容全部 goctl 语法。
|
||||
迁移扩展起始于 v0.9.0,原有手写扩展说明见 [迁移扩展指南](MIGRATION.md)。当前根目录入口和自动流式 Handler 要求 CLI 与框架依赖一起使用 v0.14.0-preview;版本说明和新目录维护规则见 [项目指南](PROJECT.md)。不兼容全部 goctl 语法。
|
||||
|
||||
## 参数语义
|
||||
|
||||
@@ -270,9 +272,9 @@ go test ./...
|
||||
|
||||
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;普通非指针字段拒绝 null(动态 `any` 允许);`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
|
||||
|
||||
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
|
||||
普通 JSON 请求体必须是一个 object,Content-Type 为 application/json;响应根类型另支持数组、nullable 对象和 null。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
|
||||
|
||||
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;非 nullable 集合请返回空集合而不是 nil。
|
||||
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;嵌套的非 nullable 集合请返回空集合而不是 nil,根 `returns ([]Item)` 会把 nil slice 规范为 `[]`。
|
||||
|
||||
## 生成目录和业务边界
|
||||
|
||||
@@ -282,7 +284,7 @@ main.go # 启动、日志、优雅退出
|
||||
etc/config.yaml
|
||||
internal/config/
|
||||
internal/svc/ # 依赖初始化和释放
|
||||
internal/types/typesGen.go
|
||||
internal/types/*Gen.go
|
||||
internal/apierrors/errorsGen.go # 声明错误后生成
|
||||
internal/handler/routesGen.go
|
||||
internal/handler/openapiGen.go
|
||||
|
||||
26
BUSINESS_DEVELOPMENT.md
Normal file
26
BUSINESS_DEVELOPMENT.md
Normal file
@@ -0,0 +1,26 @@
|
||||
# 通用业务开发能力完善计划
|
||||
|
||||
本轮目标:业务维护契约、配置、依赖装配与业务逻辑,框架持续维护 HTTP 接入、契约校验、通用运行期与文档。按实际生成项目和可删除的业务适配点验收,不写入 eTax 业务规则。
|
||||
|
||||
## 实施与验收清单
|
||||
|
||||
- [x] 运行期:资源登记、逆序释放、幂等关闭、失败回滚、故障原因保留归框架;生成入口与项目依赖初始化边界明确。多 App 日志隔离、标准 log.Logger 适配、可信请求元数据与依赖中间件示例。
|
||||
- [x] 配置:项目完整结构体环境覆盖;默认值→文件→环境;嵌套、duration、slice、显式零值、空值策略、必填与脱敏;日志无副作用完整校验。
|
||||
- [x] 数据库:官方 MySQL 结构化配置/TLS/驱动日志适配,继续复用 OpenConnector 和唯一连接池;独立启动检查、失败清理、生成器接入;不内置版本/表结构/业务错误策略。
|
||||
- [x] 响应契约:数组、可空对象、成功 null、独立 HTTP 204;Logic/Handler/OpenAPI/漂移检查一致。
|
||||
- [x] 请求契约:未知 query、JSON UTF-8/charset、无 Body/必填空对象/可选 Body,App 默认与路由覆盖;scalar/any/数组/大小限制一致,普通 JSON 只绑定一次。
|
||||
- [x] 上传:声明文件/文本/JSON 字段,重复/未知/必填/单字段/单文件/总量限制;有界暂存与清理,调用 Logic 前完成普通表单校验;保留流式扩展与 Complete 约定。
|
||||
- [x] 文档工具:字段注释、example/枚举描述进入 DTO 与 OpenAPI;保注释、幂等格式化及 check;显式 Go 错误目录元数据导出与文档合并,不自动执行项目代码。
|
||||
- [x] 共享契约:受控 roots/摘要、循环/深度/文件数/路径保护;分源文件 DTO、多个独立 module 的工作区检查;版本提示与基础模板升级预览,保留业务文件。
|
||||
- [x] 集成:实际生成项目验证新能力、连续生成、业务文件保留、失败清理、nethttp/native 请求与退出;现有错误继承/参数/上传下载回归。
|
||||
- [x] 最终回归:全量 race(含生成器集成)、vet、build、格式化/链接检查;使用说明、迁移步骤及验收报告完整,发布后远程无 replace 验收。
|
||||
|
||||
已发布 `v0.14.0-preview`。逐项验收、可替换的业务适配点与远程报告见 [验收记录](reports/businessDevelopmentAcceptance.md);接入和升级见 [开发指南](DEVELOPMENT.md)。
|
||||
|
||||
## 核心约定
|
||||
|
||||
响应与错误码由项目配置。字段与请求策略由契约表达;基础设施采用可选适配器,保留标准 Connector。所有业务依赖在请求排空后关闭;框架不重试写操作、不自动迁移业务表、不关闭借用资源。
|
||||
|
||||
普通 multipart 为保证进入 Logic 前完成校验,以有界复制暂存到临时文件;框架清理自己创建的资源。纯流式路径明确由调用方在提交持久副作用前完成输入检查。
|
||||
|
||||
不修改 eTax/ePay 业务,不连接既有业务服务器。真实基础设施如需验证,只使用本轮隔离资源。验收报告明确区分单元测试、生成消费者测试与真实服务测试。
|
||||
261
DEVELOPMENT.md
Normal file
261
DEVELOPMENT.md
Normal file
@@ -0,0 +1,261 @@
|
||||
# 通用业务开发指南(v0.14.0-preview)
|
||||
|
||||
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
|
||||
|
||||
## 返回数组、可空对象和 null
|
||||
|
||||
```text
|
||||
@handler list
|
||||
get /items returns ([]Item)
|
||||
|
||||
@handler find
|
||||
get /item returns (*Item)
|
||||
|
||||
@handler save
|
||||
post /item returns (null)
|
||||
|
||||
@handler remove
|
||||
delete /item
|
||||
```
|
||||
|
||||
| 契约 | Logic 返回 | 成功数据 |
|
||||
| --- | --- | --- |
|
||||
| `returns (Item)` | `(*types.Item, error)` | 对象;nil 被视为实现错误 |
|
||||
| `returns ([]Item)` | `([]types.Item, error)` | 数组;nil 统一输出 `[]` |
|
||||
| `returns (*Item)` | `(*types.Item, error)` | 对象或 `null` |
|
||||
| `returns (null)` | `error` | HTTP 200 和空业务数据 |
|
||||
| 不写 returns | `error` | HTTP 204,无响应体 |
|
||||
|
||||
统一响应中的空数据是否显示,仍遵循项目模板的 `dataMode`;需要 `data:null` 就设置 `dataMode: always`。`@response raw` 输出业务数据本身,包括 `[]`、对象或 `null`。可用 `@status 201/202`;204 不能声明响应类型。根类型当前支持以上四种形式,嵌套字段继续支持既有 slice/map/pointer/scalar/any。
|
||||
|
||||
## 契约驱动的请求策略
|
||||
|
||||
入口文件可以声明默认策略,单个接口按需覆盖:
|
||||
|
||||
```text
|
||||
@app (
|
||||
defaultResponse: CommonResponse
|
||||
request: {"unknownQuery":"reject", "utf8":"reject", "charset":"utf8"}
|
||||
)
|
||||
|
||||
@handler create
|
||||
@request (body: required)
|
||||
post /items (CreateReq) returns (Item)
|
||||
|
||||
@handler touch
|
||||
@request (body: emptyObject)
|
||||
post /touch returns (null)
|
||||
```
|
||||
|
||||
- `unknownQuery: reject|ignore`:是否拒绝未声明的 URL 查询字段;重复标量仍拒绝,数组用重复的同名 query。
|
||||
- `utf8: reject|replace`:拒绝原始非法 UTF-8,或采用 JSON 的替换字符行为。不是只检查解码后的字符串。
|
||||
- `charset: utf8|any`:`utf8` 仅接受未声明 charset 或 `utf-8`;不会对其他编码做转码。
|
||||
- `body: auto|required|optional|none|emptyObject`:auto 按 JSON 字段推断;required 要求一个对象;optional 允许完全没有 body,但提供 body 后仍完整校验;none 禁止任何 body 字节;emptyObject 必须是 `{}`。
|
||||
|
||||
未设置的项继承 App 的 `config.Config.Request`,最后采用既有默认行为。`.api` 的 `@app` 显式字段优先于项目配置;路由 `@request` 优先于 App。raw/multipart 使用各自解析流程,不能套 JSON body 策略。普通 JSON 只解析一次;重复/未知 JSON 字段、约束、scalar 和 `any` 仍使用同一编译器。自定义 binder/manual Handler 的输入逻辑由项目显式负责。
|
||||
|
||||
没有 body 与提供 `{}` 不等价。optional 无 body 时 JSON 字段保留零值;需要业务区分是否出现某字段时使用 optional 指针字段。raw/multipart 不继承 App 的 JSON body 策略;GET/HEAD 如遇 App 的 required/optional/emptyObject 默认值,须明确覆盖为 auto 或 none。参数校验不会替代企业权限、记账规则或数据库约束。
|
||||
|
||||
## 声明式 multipart
|
||||
|
||||
```text
|
||||
type UploadReq {
|
||||
Title string `multipart:"title" binding:"required,max=80"`
|
||||
Metadata AttachmentMeta `multipartJSON:"metadata,maxBytes=16384"`
|
||||
Attachment File `file:"attachment,maxBytes=10485760"`
|
||||
Extras []File `file:"extras,optional,maxFiles=3,maxBytes=1048576"`
|
||||
}
|
||||
type AttachmentMeta {
|
||||
Category string `json:"category"`
|
||||
}
|
||||
|
||||
@handler upload
|
||||
@body (mode: multipart; maxBytes: 16777216; maxParts: 8; maxFieldBytes: 65536)
|
||||
post /attachments (UploadReq) returns (AttachmentResp)
|
||||
```
|
||||
|
||||
Logic 直接得到 `*types.UploadReq`,无需维护 Handler。`form` 仍表示 URL query;`multipart` 表示文本表单字段,可用基础类型、文本 scalar 和重复 scalar 数组;`multipartJSON` 是一个严格解码的 JSON 表单字段,可包含对象、数组和已声明的 scalar。
|
||||
|
||||
`File` 提供 `Name`、`ContentType`、`Size`、`Present()` 和 `Open() (io.ReadCloser, error)`。文件名和媒体类型属于客户端元数据,项目继续决定内容检测、存储权限和目标对象名。不要把客户端文件名直接用作存储路径。
|
||||
|
||||
```go
|
||||
reader, err := req.Attachment.Open()
|
||||
if err != nil { return nil, err }
|
||||
defer reader.Close()
|
||||
// 将 reader 传给项目选定的流式存储接口;不要先 io.ReadAll 整个大文件。
|
||||
```
|
||||
|
||||
所有声明字段、重复/未知字段、文件数量和各级限额都会在进入 Logic 前校验。缺少必填项、重复标量/JSON 字段、文件与文本类型混用、畸形 JSON 均失败。JSON 的类型、未知字段和重复 key 检查与普通 JSON 一致。
|
||||
|
||||
默认总量 1 MiB、最多 128 个 part、单文本/JSON 字段 64 KiB;单文件默认受总量约束,文件数组默认最多 16 个。`@body` 可设置 `maxBytes/maxParts/maxFieldBytes/maxFileBytes`;字段上的 `maxBytes/maxFiles` 再收紧对应限额。总量包括 MIME 头和结尾数据;传输编码的 part 被拒绝,避免字节含义不明确。
|
||||
|
||||
文件使用有界复制写入权限为 0600 的临时文件,文本/JSON 字段在各自限额内缓冲。生成 Handler 返回或 panic 时关闭打开的文件 reader 并删除临时文件;不能把请求的 File 留到异步任务使用。失败不会交付半个 DTO,也不会进入 Logic。临时磁盘容量、并发请求上限与 HTTP 超时由部署配置共同约束。
|
||||
|
||||
不声明 multipart 字段时,保留原来的 `body *httpx.Body` 流式入口,不创建临时文件。该方式适合特殊协议;项目应在不可撤销副作用之前完成校验并调用 `body.Complete()`。下载和原始回调继续使用 `@produces` / `@body(mode: raw)`。
|
||||
|
||||
## 完整业务配置的环境覆盖
|
||||
|
||||
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML 文件 → 全结构体环境覆盖 → 既有框架环境别名 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
|
||||
|
||||
```go
|
||||
type ClientConfig struct {
|
||||
Endpoint string `json:"endpoint" env:"CLIENT_ENDPOINT,required"`
|
||||
Token string `json:"-" env:"CLIENT_TOKEN,required,secret,empty=reject"`
|
||||
Timeout time.Duration `json:"timeout" envDefault:"3s"`
|
||||
Retries []int `json:"retries"`
|
||||
Enabled bool `json:"enabled"`
|
||||
}
|
||||
```
|
||||
|
||||
把字段加入项目 `Config` 后,默认环境名取 `GO_OMNI_` 加 JSON 路径的 UPPER_SNAKE_CASE,如 `GO_OMNI_CLIENT_TIMEOUT`。显式 env 名是完整名称,可以使用自己的业务前缀。`env:"-"` 排除整个子树;`json:"-"` 字段只有显式 env 标签才参与。
|
||||
|
||||
duration 使用 `250ms`、`3s`;slice/map 使用 JSON,如 `[1,3,5]`、`{"retry":"2s"}`。支持嵌套、零/false、空数组、自定义 TextUnmarshaler。空环境值默认允许;可选择 `empty=ignore` 或 `empty=reject`。required 要求字符串非空、指针/集合非 nil;0 和 false 可以是合法业务值,范围由项目 Validate 决定。
|
||||
|
||||
手写项目可使用 `config.LoadProject(path, defaults, config.EnvOptions{Prefix:"MY_APP_"})`,或分别调用 `ApplyDefaults/DecodeFile/OverlayEnv`。读取失败不修改输入默认值;诊断不会打印输入值。不要主动打印含密钥的整个配置,或把底层解析错误展开到公开响应。Validate 不打开日志文件、数据库或存储连接;证书的可读性和握手信任在依赖初始化阶段验证。
|
||||
|
||||
## 依赖、日志与中间件
|
||||
|
||||
`internal/svc/runtimeGen.go` 持续由生成器维护。项目的 ServiceContext 嵌入 `*lifecycle.Runtime`,只在 `Init(ctx)` 中装配额外依赖:
|
||||
|
||||
```go
|
||||
func (c *ServiceContext) Init(ctx context.Context) error {
|
||||
client, err := NewClient(c.Config.Client)
|
||||
if err != nil { return err }
|
||||
c.Client = client
|
||||
c.OnClose(client.Close)
|
||||
c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(client)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
每个资源初始化成功就登记清理;后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求,再逆序关闭依赖,最后关闭日志。关闭幂等,单个清理函数 panic/返回错误不会跳过其他清理;初始化 panic 清理后继续传播。项目清理函数仍须自行停止、等待后台任务并保证能返回,框架不能强制杀死任意 goroutine。
|
||||
|
||||
中间件用业务所需的小接口接收依赖,避免反向导入 svc:
|
||||
|
||||
```go
|
||||
// internal/middleware/accessFactory.go
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type AccessChecker interface {
|
||||
CheckAccess(context.Context, string) error
|
||||
}
|
||||
func NewRequireAccess(checker AccessChecker) httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
|
||||
return err
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`.api` 声明对应 middleware 后,保留生成的初始 hook,并另建上述工厂;在 Init 用工厂替换 registry 的同名项即可。该包只依赖接口和 HTTP 抽象,不导入 svc,也不把数据库藏到 context 里。
|
||||
|
||||
生成入口为每个 App 注入独立 Logger,不修改进程默认 Logger。业务可用 `c.Logger.InfoContext(...)`,或请求链中的 `logx.InfoContext(ctx,...)`。标准库/第三方需要 `*log.Logger` 时使用 `c.Logger.Standard(slog.LevelWarn)`;适配器借用原 Logger,不负责关闭。手写多 App 用 `omni.WithLogInstance(logger)` 显式注入。
|
||||
|
||||
`httpx.Metadata(ctx)` 在业务的标准 context 中提供可信代理规则解析后的 ClientIP、HTTP Method 和路由模板;不放入凭据、完整 URL、查询或 body。业务 userId/tenantId 等仍由项目选择,框架不预设身份模型。
|
||||
|
||||
`middleware.ContextErrors("REQUEST_CANCELED", "REQUEST_TIMEOUT")` 可将原始取消/超时错误映射到项目错误目录;显式 APIError 优先,内部 cause 链保留。状态、业务码与提示仍在项目 Catalog 配置。
|
||||
|
||||
## 字段文档、格式化、Go 错误目录导出
|
||||
|
||||
```text
|
||||
type Item {
|
||||
// 企业名称
|
||||
Name string `json:"name" example:"示例企业"`
|
||||
State int `json:"state,options=0|1" example:"1" enumDescriptions:"[\"禁用\",\"启用\"]"` // 状态
|
||||
}
|
||||
```
|
||||
|
||||
字段前/同行注释同步到 Go 注释和 OpenAPI description。显式 `description` 标签优先;example 的基础字段值也检查类型和约束,枚举说明按声明值顺序填写。字符串 example 直接写内容,其他类型写 JSON。自定义 scalar 的示例不能替代其项目解码器验证;输入输出类型不同时,不把输入示例冒充输出示例。
|
||||
|
||||
```sh
|
||||
go-omni api format --api api/service.api
|
||||
go-omni api format --api api/service.api --check
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
```
|
||||
|
||||
格式化保留注释、token 和多行字面量,规范四空格缩进、空行、行首尾空白及相邻单行字段的类型/标签/行尾注释对齐;结果幂等。先校验完整契约,check 不写文件;导入的共享目录不被某个服务的格式化命令修改。DTO 按源 `.api` 输出 `dto<Source><摘要>Gen.go`,都属于 `types` 包,跨文件引用无需改包名。
|
||||
|
||||
Go Catalog 是错误的唯一来源时,显式导出文档元数据:
|
||||
|
||||
```go
|
||||
// 在项目自己的导出命令中;appErrors.Catalog 与 AppOptions 注册同一函数。
|
||||
if err := omnierrors.ExportCatalog(os.Stdout, appErrors.Catalog()); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
```
|
||||
|
||||
```sh
|
||||
go run ./cmd/exportErrors > api/omni.errors.json
|
||||
go-omni api generate
|
||||
```
|
||||
|
||||
文件是派生产物,应纳入版本控制并由业务 CI 检查导出漂移。CLI 自动读取入口目录的 `omni.errors.json`,校验错误目录、合并状态/模板参数/具体示例到 OpenAPI;不会运行项目 Go,不创建另一份运行期注册器。带独立错误目录的路由继续使用自己的目录。没有路由级精确错误列表时,文档描述 App 目录的公共错误集合;特殊自定义 Encoder 的文档由项目显式扩展。
|
||||
|
||||
## 两个服务共享契约
|
||||
|
||||
```text
|
||||
workspace/
|
||||
├── contracts/common/response.api
|
||||
├── services/api/api/service.api
|
||||
├── services/api/api/omni.imports.json
|
||||
├── services/id/api/service.api
|
||||
├── services/id/api/omni.imports.json
|
||||
└── omni.workspace.json
|
||||
```
|
||||
|
||||
两个服务的 `omni.imports.json` 可以分别声明:
|
||||
|
||||
```json
|
||||
{"roots":{"common":{"path":"../../../contracts/common","version":"1.0.0"}}}
|
||||
```
|
||||
|
||||
```text
|
||||
import "@common/response.api"
|
||||
```
|
||||
|
||||
只读取入口目录和显式声明的根。保留真实路径检查、symlink 越界拒绝、循环检测、32 层、64 文件、单文件 1 MiB/总量 8 MiB 限额;最多 16 个共享根。不会下载依赖或复制共享定义进服务。移动目录时显式更新 roots。
|
||||
|
||||
`version` 是项目管理的版本标签。生成文档记录每个文件 SHA-256,以及每个共享根所导入文件集合的摘要;可把文档中的 `x-omni-contract-sources.shared.common.sha256` 填入 root 的 `sha256`,固定输入集合,内容变化后必须显式更新。摘要算法是对“该根内相对路径→文件 SHA-256”的键排序 JSON 对象再做 SHA-256,未导入的文件不参与。
|
||||
|
||||
工作区清单:
|
||||
|
||||
```json
|
||||
{"services":[{"dir":"services/api"},{"dir":"services/id"}]}
|
||||
```
|
||||
|
||||
```sh
|
||||
go-omni workspace format --file omni.workspace.json
|
||||
go-omni workspace generate --file omni.workspace.json
|
||||
go-omni workspace check --file omni.workspace.json
|
||||
```
|
||||
|
||||
各服务保持独立 module;可显式填写各服务的 `api`(相对服务目录)和 `module`。所有服务预检通过后再写;每个服务独立提交文件事务,不承诺跨项目的崩溃原子性。check 汇总格式与生成漂移;业务编译/测试仍在各 module 执行。
|
||||
|
||||
## v0.13 项目升级
|
||||
|
||||
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.14.0-preview(或显式引用当前源码)。
|
||||
|
||||
```sh
|
||||
go-omni api upgrade --dir . --dry-run
|
||||
go-omni api upgrade --dir . --preview .goOmni/upgradePreview
|
||||
# 只有标为 safe 的原始模板才会自动替换;review 阻止全部写入。
|
||||
go-omni api upgrade --dir . --apply
|
||||
go mod tidy
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
go build .
|
||||
```
|
||||
|
||||
自动迁移只替换与 v0.13 原始模板一致的配置/svc 基础文件、移除原始 close.go,并更新最低依赖及生成产物;保留 Logic、Repository、AppOptions、独立命令。迁移失败回滚已改的项目文件。存在业务修改时,仅输出这些基础文件的候选版本和清单,不生成整棵业务代码让人重新拼接。
|
||||
|
||||
定制项目按预览定点迁移:保留依赖字段,把初始化移到 Init,嵌入 Runtime,清理改为 OnClose 登记;移除旧 New/Close/OnClose;合并业务配置和结构化 MySQL 配置,更新 go.mod,再正常 generate。生成器会拒绝遗留构造/关闭实现,避免重复函数或悄悄绕过清理。v0.12 及更早的定制结构不能被精确自动合并,使用同一预览逐项迁移并编译验证。
|
||||
31
PROJECT.md
31
PROJECT.md
@@ -1,6 +1,6 @@
|
||||
# 生成项目与文件维护边界
|
||||
|
||||
v0.13.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
||||
v0.14.1-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
||||
|
||||
## 创建和日常开发
|
||||
|
||||
@@ -47,7 +47,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
|
||||
| `internal/config/config.go` | 自定义配置字段、加载、默认值与校验 |
|
||||
| `internal/svc/serviceContext.go` | 依赖字段和初始化,持有加载后的 Config |
|
||||
| `internal/svc/options.go` | AppOptions、GlobalMiddleware;注册 Go 错误目录和全局扩展 |
|
||||
| `internal/svc/close.go` | 资源关闭;OnClose 为自定义资源提供标准清理入口 |
|
||||
| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 |
|
||||
| `internal/svc/storage.go` | 选用存储时生成,之后保留 |
|
||||
| `internal/logic/<group>/*.go` | 业务方法,使用标准 context.Context |
|
||||
| `internal/repository/<group>/*.go` | 数据访问,按需实现 |
|
||||
@@ -58,7 +58,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
|
||||
|
||||
以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。
|
||||
|
||||
生成器持续管理:`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
|
||||
生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
|
||||
|
||||
Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。
|
||||
|
||||
@@ -66,16 +66,16 @@ Logic 实例在注册时创建并由请求并发复用;请求数据用方法
|
||||
|
||||
## 依赖与启动扩展
|
||||
|
||||
`svc.New(ctx, logger, cfg)` 初始化依赖,可返回错误;Logger 由入口创建并借用。新增客户端时只需要扩展配置和 svc:
|
||||
`svc.New(ctx, logger, cfg)` 初始化依赖,可返回错误;Logger 由入口创建并借用。New 本身由生成器维护;新增客户端时扩展配置和 svc 的 `Init(ctx) error`:
|
||||
|
||||
```go
|
||||
client, err := newClient(cfg.Client)
|
||||
if err != nil { return nil, err }
|
||||
client, err := newClient(c.Config.Client)
|
||||
if err != nil { return err }
|
||||
c.Client = client
|
||||
c.OnClose(client.Close) // 创建成功后立即登记,后续失败也能释放
|
||||
```
|
||||
|
||||
`New` 的失败清理骨架已生成。OnClose 按登记的相反顺序执行;Close 幂等。项目不要重复关闭同一资源,借用的 Logger 也不应由 svc 关闭。构造器在返回错误前若创建了尚未交付的资源,该构造器自己负责释放。
|
||||
ServiceContext 嵌入 `*lifecycle.Runtime`;框架维护锁、清理栈和失败回滚,项目不再复制 Close/OnClose 实现。OnClose 按登记的相反顺序执行;Close 幂等。项目不要重复关闭同一资源,借用的 Logger 也不应由 svc 关闭。构造器在返回错误前若创建了尚未交付的资源,该构造器自己负责释放。
|
||||
|
||||
错误注册写在 `internal/svc/options.go`,错误表可以来自任意项目共用包:
|
||||
|
||||
@@ -85,7 +85,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
|
||||
}
|
||||
```
|
||||
|
||||
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。静态 OpenAPI 不执行 Go,不能自动枚举仅存在于 Go 中的错误目录。
|
||||
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。静态 OpenAPI 不执行 Go;可从同一 Catalog 显式导出 `api/omni.errors.json`,详见 [元数据导出](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
|
||||
|
||||
`GlobalMiddleware()` 提供全局中间件;`Middlewares` 映射提供 `.api` 声明的具名中间件。全部显式调用,不依赖项目 init 注册,也不要求 svc 导入 handler。框架请求链按原有次序运行。
|
||||
|
||||
@@ -140,16 +140,11 @@ server:
|
||||
|
||||
关闭或改名后,旧路径不再由框架保留,项目可以在 `.api` 自己声明。启用的内置 GET 路由与业务路由相同或被业务参数/通配路径覆盖时,在创建 App、开始监听之前报错;不会覆盖业务路由。内置接口仍经过全局中间件;项目在鉴权中间件中明确决定这些路径的访问规则,框架不自动绕过鉴权。
|
||||
|
||||
## 从 v0.12 生成项目升级
|
||||
<a id="从-v012-生成项目升级"></a>
|
||||
|
||||
manifest v2 和现有目录保留。升级同版 CLI 与框架依赖后执行 `api generate`,生成器自动更新 main 和 Handler;`svc`、配置与业务文件仍按字节保留,所以需要在项目拥有的 svc 文件补入以下扩展:
|
||||
## 从 v0.13 或更早版本升级
|
||||
|
||||
1. `serviceContext.go` 导入 `git.yuntuo.hk/k2safe/go-omni/lifecycle`,给 ServiceContext 增加 `Lifecycle *lifecycle.Controller`;创建 c 时用 `lifecycle.New(ctx)` 初始化,并在后续依赖初始化前设置 `ctx = c.Lifecycle.Context()`。
|
||||
2. 增加 `func (c *ServiceContext) RequestShutdown(err error) { c.Lifecycle.RequestShutdown(err) }`。
|
||||
3. 构造失败的 defer 在 `c.Close()` 后保留 `c.Lifecycle.Err()`;Close 的 `closeOnce.Do` 开头调用 `c.Lifecycle.RequestShutdown(nil)`,重复关闭仍由已有 once 保护。
|
||||
4. 配置默认值若来自 `config.Default()`,自动具有默认端点配置;如需禁用或改路径,在 YAML 或项目 Config 中显式设置。手动拼 Config 的项目需要自行填写要启用的端点。
|
||||
|
||||
精确实现可对照 [示例 svc](examples/managedService/internal/svc/serviceContext.go) 和 [关闭逻辑](examples/managedService/internal/svc/close.go),也可在临时目录用新版 CLI 生成参考。不要覆盖原有数据库/客户端初始化;不需要手改 main、Handler 或引入另一套启动器。最后执行 `api generate --check`、`go test ./...`、`go build .`,并验证故障退出。
|
||||
请按 [v0.14 定点迁移说明](DEVELOPMENT.md#v013-项目升级) 使用 `api upgrade`。原始模板可以自动升级;含业务修改的基础文件必须参考局部预览合并。旧 New/Close/OnClose 移到框架 Runtime,项目依赖初始化保留在 Init。不要继续向项目里复制生命周期锁和清理栈。
|
||||
|
||||
## App 默认响应格式
|
||||
|
||||
@@ -193,7 +188,7 @@ post /callback
|
||||
get /download (DownloadReq)
|
||||
```
|
||||
|
||||
- raw/multipart 的 Logic 增加 `body *httpx.Body` 参数。`body.Raw` 是受路由上限约束的原始字节流;`body.Header` 是请求头副本。
|
||||
- 以下为未声明 multipart 字段的流式用法;声明式绑定见 [上传字段](DEVELOPMENT.md#声明式-multipart)。raw/流式 multipart 的 Logic 增加 `body *httpx.Body` 参数。`body.Raw` 是受路由上限约束的原始字节流;`body.Header` 是请求头副本。
|
||||
- multipart 用 `body.Parts.NextPart()` 顺序读取,关闭每个 Part;不调用 ParseMultipartForm,不产生上传临时文件,也不把整个文件放入内存。每个字段/文件的业务要求由 Logic 校验。请求 DTO 可定义 path/form/header;`form` 仍是 URL 查询参数,不能用 json DTO 重复读取上传体。
|
||||
- Parts 是 `*httpx.MultipartReader`,返回的 `*httpx.MultipartPart` 提供 Read、Close、Header、FormName、FileName;NextRawPart 可读取未做 quoted-printable 解码的部分。NextPart 自动关闭上一个未读完的 Part;Close 会报告读取错误。不要混用 Parts 与 Raw 读取同一个 multipart 请求。
|
||||
- multipart 缺少结束边界、头部/内容损坏默认返回 `400 / INVALID_MULTIPART`;超过路由限额仍为 413,媒体类型/缺少 boundary 为 415。框架只提供通用标识和默认提示,项目可以在 Go 错误目录中配置自己的状态、业务码和文案。内部解析原因保留在错误链,不进入公开响应;Logic 自己返回的数据库等错误不会被统一改成 400。
|
||||
@@ -213,7 +208,7 @@ get /download (DownloadReq)
|
||||
|
||||
## 已有服务接入步骤
|
||||
|
||||
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.13.0-preview:
|
||||
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.14.1-preview:
|
||||
|
||||
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
|
||||
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。
|
||||
|
||||
20
README.md
20
README.md
@@ -2,12 +2,16 @@
|
||||
|
||||
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
|
||||
|
||||
**从这份 README 开始接入。** 当前版本 **`v0.13.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
|
||||
**从这份 README 开始接入。** 当前版本 **`v0.14.1-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
|
||||
|
||||
Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
|
||||
|
||||
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。
|
||||
|
||||
新增通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
|
||||
|
||||
使用 GORM 的项目可以选择 [gormx 适配器](database/gormx/README.md),复用数据库初始化和连接池,并将 SQL 日志接入 logx。
|
||||
|
||||
## 选择接入方式
|
||||
|
||||
| 你的项目 | 从哪里开始 |
|
||||
@@ -28,7 +32,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
|
||||
|
||||
```sh
|
||||
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.13.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
|
||||
export PATH="$(go env GOPATH)/bin:$PATH"
|
||||
go-omni --help
|
||||
```
|
||||
@@ -72,7 +76,7 @@ hello/
|
||||
├── internal/
|
||||
│ ├── config/ # 配置读取
|
||||
│ ├── svc/ # 数据库、日志、外部客户端等依赖
|
||||
│ ├── types/typesGen.go # 生成的请求与业务返回类型
|
||||
│ ├── types/*Gen.go # 按 .api 文件生成 DTO
|
||||
│ ├── handler/ # 生成的绑定、响应策略、路由及文档
|
||||
│ ├── logic/greet/ # 在这里实现业务逻辑
|
||||
│ ├── repository/greet/ # 在这里实现数据访问
|
||||
@@ -175,7 +179,7 @@ go-omni new --api catalog.api \
|
||||
--log-output file --log-format json ./catalogFull
|
||||
```
|
||||
|
||||
该命令在 `catalog.api` 所在目录执行,目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护,API 模板不自动建表;MySQL DSN 通过 `GO_OMNI_DB_DSN` 提供。存储还可选 `s3`、`r2`、`oss`,按生成项目的配置说明提供桶、端点和凭据。完整生成项目示例见 [managedService](examples/managedService/README.md)。
|
||||
该命令在 `catalog.api` 所在目录执行,目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护,API 模板不自动建表;MySQL 使用结构化配置,密码通过 `GO_OMNI_DB_PASSWORD` 提供。存储还可选 `s3`、`r2`、`oss`,按生成项目的配置说明提供桶、端点和凭据。完整生成项目示例见 [managedService](examples/managedService/README.md)。
|
||||
|
||||
`go-omni new` 与 `go-omni api new` 统一生成契约项目。需要手写项目时,可直接引用框架包;生成器不再提供另一套 CRUD 启动目录。
|
||||
|
||||
@@ -184,7 +188,7 @@ go-omni new --api catalog.api \
|
||||
在业务项目的 **go.mod 所在目录**执行:
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.13.0-preview
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview
|
||||
```
|
||||
|
||||
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
|
||||
@@ -382,7 +386,7 @@ return apierrors.TextTooLong("企业名称", 100)
|
||||
|
||||
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
|
||||
|
||||
已有项目使用 `driver.Connector` 时,直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 等驱动的独立 TLS、Logger、时区与拨号参数由项目构造 Connector;框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN,也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。
|
||||
已有项目使用 `driver.Connector` 时,直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 可直接用 [database/mysql](database/mysql/README.md) 配置独立 TLS、Logger、时区与拨号参数;任意驱动也可由项目构造 Connector。框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN,也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。
|
||||
|
||||
| 能力 | 包与接入说明 |
|
||||
| --- | --- |
|
||||
@@ -409,7 +413,7 @@ API 项目的数据库迁移需要单独维护并显式执行;启动不会自
|
||||
| `GO_OMNI_SHUTDOWN_TIMEOUT` | 优雅退出等待时间,默认 `10s` |
|
||||
| `GO_OMNI_ENABLE_LOGGER` / `GO_OMNI_LOG_LEVEL` | 访问日志开关、日志级别 |
|
||||
| `GO_OMNI_TRUSTED_PROXIES` | 可解析转发地址头的可信代理 IP/CIDR,逗号分隔 |
|
||||
| `GO_OMNI_DB_DSN` | 选择 MySQL 初始化时的数据库连接配置 |
|
||||
| `GO_OMNI_DB_HOST/USER/PASSWORD/DATABASE` | 官方 MySQL 适配器的结构化连接配置 |
|
||||
|
||||
例如在生成项目中切换 native:
|
||||
|
||||
@@ -454,7 +458,7 @@ go test ./...
|
||||
|
||||
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
|
||||
|
||||
CLI 与业务 module 固定使用 `v0.13.0-preview`。新布局不兼容旧 manifest;旧试验项目需在新目录生成并搬入业务文件,不能直接覆盖手写入口。新布局内部升级再执行上述生成与测试步骤,手写框架项目只需依赖升级及业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
||||
CLI 与业务 module 固定使用 `v0.14.1-preview`。v0.14.0 项目直接升级依赖与 CLI,GORM 按需导入;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
||||
|
||||
## 按需深入
|
||||
|
||||
|
||||
53
RELEASE.md
53
RELEASE.md
@@ -1,4 +1,53 @@
|
||||
# v0.13.0-preview
|
||||
# v0.14.1-preview
|
||||
|
||||
增加可选 `database/gormx`:GORM 复用 go-omni 已初始化的标准池、TLS、限额、健康检查和关闭流程,业务继续使用原生 GORM 模型与事务。核心 App/数据库包的编译依赖链不引入 GORM。
|
||||
|
||||
在 svc.Init 中调用 `gormx.MySQL(ctx, c.DB, gormx.WithLogger(c.Logger))`;通用 Open 工厂保留其他方言入口。适配器保护借用池不被初始化失败路径关闭,并为初始化查询设置时限。查询使用 WithContext,SQL 日志接入 logx,请求标识、慢查询和错误分类贯通;默认不输出绑定参数与驱动完整错误文本。
|
||||
|
||||
v0.14.0 项目更新 module 与 CLI 即可,不改生成目录或 main。模板最低运行时保持 v0.14.0;修正新版 CLI 把最低兼容版本误当自身版本、导致同版消费者出现版本提示的问题。其他数据库方言和管理额外池的插件未纳入本次实库验收,原生 PrepareStmt 的生命周期限制见 [接入说明](database/gormx/README.md)。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
|
||||
```
|
||||
|
||||
全量 race(含生成消费者)、vet、build,以及隔离 MySQL 的事务/取消/日志/生成项目接入通过,详见 [GORM 验收](reports/gormAcceptance.md)。
|
||||
|
||||
---
|
||||
|
||||
# v0.14.0-preview(历史版本)
|
||||
|
||||
本版补齐契约驱动的业务开发与通用基础设施。项目继续维护接口、配置、依赖装配、Logic、SQL 和业务规则;生成器维护入口、请求绑定、路由、响应与文档。
|
||||
|
||||
## 安装与升级
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
新生成器与运行时一起升级。已有 v0.13 项目先运行 `go-omni api upgrade --dry-run`,再按 [定点迁移步骤](DEVELOPMENT.md#v013-项目升级) 处理;不会静默覆盖自定义 svc/config。运行期的新构造器由生成器维护,项目初始化迁入 `svc.Init`,使用 `OnClose` 登记资源。
|
||||
|
||||
## 本版能力
|
||||
|
||||
- `.api` 支持根数组、可空对象、成功 null;真正 204 独立保留。生成 Logic、Handler、OpenAPI 和漂移检查使用相同语义。
|
||||
- 请求契约控制未知 query、原始 UTF-8、charset、必须/可选/禁止 body 和严格空对象。路由覆盖 App 策略,普通 JSON 只解析一次。
|
||||
- multipart 声明文件、文本和 JSON 表单字段。Logic 收到校验后的 DTO;文件有界暂存、流式读取、请求结束清理,保留底层 raw/multipart/下载扩展。
|
||||
- 完整项目配置支持环境覆盖、显式零值、空值策略、必填、嵌套及复杂类型。独立校验先于日志和连接创建。
|
||||
- `lifecycle.Runtime` 统一逆序清理、幂等关闭、失败回滚及故障保留;App Logger 独立,支持标准 `log.Logger` 与可信请求元数据。提供依赖中间件装配示例和可选 context 错误映射。
|
||||
- 官方 `database/mysql` 适配器提供结构化配置、每连接池独立 TLS/mTLS、驱动日志、启动检查与失败清理。继续返回同一 `database.DB`,不自动迁移表或重试业务事务。
|
||||
- 字段注释、示例和枚举说明进入 DTO/OpenAPI;`api format` 幂等格式化。Go 错误目录可显式导出供文档使用,无需复制错误定义。
|
||||
- 显式共享契约 roots、来源摘要、分文件 DTO、多服务 workspace 命令、基础模板升级预览;保留目录边界及业务文件所有权。
|
||||
|
||||
详见 [业务开发指南](DEVELOPMENT.md)、[数据库接入](database/mysql/README.md) 和 [验收记录](reports/businessDevelopmentAcceptance.md)。
|
||||
|
||||
全量 race(含生成项目)、vet、build 与本地进程验收通过。发布后的空缓存远程消费者和隔离 MySQL 验收也均通过;CLI/框架同版且无本地 replace。标签固定在 `f5478cdc7662704b164637649305361e52f7c598`,报告单独提交。
|
||||
|
||||
仍需业务维护:契约、配置、svc 依赖创建/探针/清理登记、错误目录、Logic、Repository 和权限中间件。特殊协议可使用保留的扩展点。声明式上传的暂存磁盘和并发量需配置容量;共享契约版本标签由项目管理,框架校验摘要但不下载依赖。此版仍为 preview,不代表 eTax/ePay 已完成业务迁移。
|
||||
|
||||
---
|
||||
|
||||
# v0.13.0-preview(历史版本)
|
||||
|
||||
本版补齐项目依赖主动退出与内置接口配置。ID 租约、数据库或后台任务等故障由项目判断,框架通过统一生命周期完成退出;不内置业务规则。
|
||||
|
||||
@@ -33,7 +82,7 @@ python3 scripts/verifyManagedProject.py --report performance/lifecycleEndpointsL
|
||||
python3 scripts/verifyManagedRelease.py --version v0.13.0-preview --report performance/lifecycleEndpointsRemote.json
|
||||
```
|
||||
|
||||
远程验收在发布后执行,报告单独提交,不改写发布标签。此版仍为 preview;具体业务的租约失效保护、鉴权与事务回归由接入项目验证。资源清理顺序不代表能强行结束忽略 context 取消的业务 goroutine;项目 Close 必须停止并等待自己启动的后台任务。
|
||||
发布后的远程空缓存验收已通过,耗时 26.55 秒;[机器报告](performance/lifecycleEndpointsRemote.json) 确认 CLI 和生成项目使用同版远程 module,无本地 replace。报告单独提交,发布标签保持在代码提交 `f0991d3ff248e4cb32812c368f0f8ba28740a6c7`。此版仍为 preview;具体业务的租约失效保护、鉴权与事务回归由接入项目验证。资源清理顺序不代表能强行结束忽略 context 取消的业务 goroutine;项目 Close 必须停止并等待自己启动的后台任务。
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# GoOmni 成熟度计划
|
||||
|
||||
当前进展:`v0.14.0-preview` 已完成契约驱动的业务开发能力、通用 Runtime/配置和官方 MySQL 适配器,并通过全量 race、实际生成服务及空缓存远程验收。新项目/升级以 [README](README.md) 和 [开发指南](DEVELOPMENT.md) 为准;下方为历史阶段记录,完整本轮验收见 [报告](reports/businessDevelopmentAcceptance.md)。具体 eTax/ePay 业务迁移与业务回归仍由接入项目完成。
|
||||
|
||||
## 1. 新项目启动与发布流程(已完成,v0.1.1)
|
||||
|
||||
- 修正脚手架固定版本引用,保留显式本地开发选项。
|
||||
|
||||
58
app/logIsolation_test.go
Normal file
58
app/logIsolation_test.go
Normal file
@@ -0,0 +1,58 @@
|
||||
package app_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
func TestAppLogAndRequestMetadataIsolation(t *testing.T) {
|
||||
var a, b bytes.Buffer
|
||||
la, _ := logx.New(logx.Config{Writer: &a})
|
||||
lb, _ := logx.New(logx.Config{Writer: &b})
|
||||
apps := []*omni.App{
|
||||
omni.New(omni.WithLogger(true), omni.WithLogInstance(la), omni.WithTrustedProxies("127.0.0.1/32")),
|
||||
omni.New(omni.WithLogger(true), omni.WithLogInstance(lb)),
|
||||
}
|
||||
for i, app := range apps {
|
||||
app.GET("/probe/:id", func(c *httpx.Context) error {
|
||||
ctx := c.Context()
|
||||
meta := httpx.Metadata(ctx)
|
||||
expected := "127.0.0.1"
|
||||
if i == 0 {
|
||||
expected = "203.0.113.7"
|
||||
}
|
||||
if meta.ClientIP != expected || meta.Route != "/probe/:id" || meta.Method != "GET" {
|
||||
t.Error(meta)
|
||||
}
|
||||
logx.InfoContext(ctx, "business", "app", i)
|
||||
clone := c.Clone(context.WithValue(ctx, struct{}{}, true), httptest.NewRecorder())
|
||||
if httpx.Metadata(clone.Context()) != meta {
|
||||
t.Error("metadata lost")
|
||||
}
|
||||
return c.OK(nil)
|
||||
})
|
||||
}
|
||||
for _, app := range apps {
|
||||
r := httptest.NewRequest("GET", "/probe/42", nil)
|
||||
r.RemoteAddr = "127.0.0.1:1234"
|
||||
r.Header.Set("X-Forwarded-For", "203.0.113.7")
|
||||
app.ServeHTTP(httptest.NewRecorder(), r)
|
||||
}
|
||||
for i, out := range []string{a.String(), b.String()} {
|
||||
if !strings.Contains(out, "http.request") || !strings.Contains(out, "business") || strings.Contains(out, `"app":`+string(rune('0'+(1-i)))) {
|
||||
t.Fatal(out)
|
||||
}
|
||||
}
|
||||
_ = apps[0].Close()
|
||||
lb.Info("still open")
|
||||
if !strings.Contains(b.String(), "still open") {
|
||||
t.Fatal("borrowed logger closed")
|
||||
}
|
||||
}
|
||||
@@ -3,7 +3,9 @@ package app
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
@@ -26,10 +28,14 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid
|
||||
}
|
||||
}
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
request = request.WithContext(logx.WithLogger(contract.WithRequestPolicy(request.Context(), a.config.Request), a.config.LogInstance))
|
||||
if len(a.config.TrustedProxies) > 0 {
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
|
||||
}
|
||||
requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context()))
|
||||
requestContext.WithContext(httpx.WithRequestMetadata(request.Context(), httpx.RequestMetadata{
|
||||
ClientIP: requestContext.ClientIP(), Method: request.Method, Route: httpx.RoutePattern(request.Context()),
|
||||
}))
|
||||
if provider, ok := a.responseEncoder.(response.ErrorRegistryProvider); ok {
|
||||
requestContext.SetErrorRegistry(provider.ErrorRegistry())
|
||||
}
|
||||
|
||||
@@ -1,12 +1,14 @@
|
||||
# go-omni CLI
|
||||
|
||||
v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目录>|--apply]`,以及 `workspace format|generate|check --file omni.workspace.json`。详见 [通用开发指南](../../DEVELOPMENT.md)。
|
||||
|
||||
新项目采用根目录 `main.go` 与 `internal/config、svc、logic、handler、types`。完整职责、启动扩展、流式接口和生成文件边界见 [项目指南](../../PROJECT.md)。
|
||||
|
||||
## 创建项目
|
||||
|
||||
```sh
|
||||
# 仓库读取权限与 PATH 配置见根目录 README
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.13.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
|
||||
go-omni new --module example.com/hello ./hello
|
||||
|
||||
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
|
||||
@@ -41,14 +43,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
|
||||
|
||||
## 版本与迁移
|
||||
|
||||
CLI 与框架依赖固定使用 v0.13.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
|
||||
CLI 与框架依赖固定使用 v0.14.1-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
|
||||
|
||||
```sh
|
||||
# 在 go-omni 源码仓库中执行
|
||||
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
|
||||
```
|
||||
|
||||
新模板需要 v0.13.0-preview 的依赖退出与端点配置能力,继续使用 manifest v2。v0.12 项目需按 [升级步骤](../../PROJECT.md#从-v012-生成项目升级) 补入 svc 生命周期字段和方法,生成器保留这些项目文件。不兼容旧目录,旧生成项目请在新目录重新生成后移入业务代码,不覆盖手写 main。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
|
||||
模板最低需要 v0.14.0-preview 的通用 Runtime、配置和绑定能力,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
|
||||
|
||||
## 数据库迁移命令
|
||||
|
||||
|
||||
@@ -15,16 +15,19 @@ func runAPI(args []string) error {
|
||||
return nil
|
||||
}
|
||||
command := args[0]
|
||||
if command == "upgrade" {
|
||||
return runUpgrade(args[1:])
|
||||
}
|
||||
if command == "go" {
|
||||
command = "generate"
|
||||
}
|
||||
if command == "check" {
|
||||
command = "validate"
|
||||
}
|
||||
if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" {
|
||||
if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" && command != "format" {
|
||||
return fmt.Errorf("unknown api command %q", command)
|
||||
}
|
||||
readOnly := command == "validate" || command == "inspect" || command == "export"
|
||||
readOnly := command == "validate" || command == "inspect" || command == "export" || command == "format"
|
||||
o := apigen.Options{}
|
||||
for i := 1; i < len(args); i++ {
|
||||
arg := args[i]
|
||||
@@ -33,7 +36,7 @@ func runAPI(args []string) error {
|
||||
return nil
|
||||
}
|
||||
if arg == "--check" || arg == "--dry-run" {
|
||||
if command != "generate" {
|
||||
if command != "generate" && !(command == "format" && arg == "--check") {
|
||||
return fmt.Errorf("%s belongs to api generate", arg)
|
||||
}
|
||||
if arg == "--check" {
|
||||
@@ -108,6 +111,14 @@ func runAPI(args []string) error {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if command == "format" {
|
||||
changes, err := apigen.FormatProject(o.API, o.Check)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("API format: %d files updated\n", len(changes))
|
||||
return nil
|
||||
}
|
||||
if readOnly {
|
||||
spec, err := apigen.Parse(o.API)
|
||||
if err != nil {
|
||||
|
||||
91
cmd/go-omni/apigen/business_test.go
Normal file
91
cmd/go-omni/apigen/business_test.go
Normal file
@@ -0,0 +1,91 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestBusinessContract(t *testing.T) {
|
||||
spec, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/business.api"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var doc map[string]any
|
||||
if err = json.Unmarshal(c.Document, &doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
paths := doc["paths"].(map[string]any)
|
||||
body := paths["/optional"].(map[string]any)["post"].(map[string]any)["requestBody"].(map[string]any)
|
||||
if body["required"] != false {
|
||||
t.Fatal("optional body documented as required")
|
||||
}
|
||||
for _, path := range []string{"/empty", "/absent"} {
|
||||
if paths[path].(map[string]any)["post"].(map[string]any)["x-omni-request-policy"] == nil {
|
||||
t.Fatal("missing policy metadata")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestBusinessGeneratedProject(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "business")
|
||||
o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/business.api"), Module: "example.com/business", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, name := range []string{"upload", "nullable", "nothing", "rawNull", "rawArray", "empty", "absent", "optional", "echo", "tolerant", "noContent"} {
|
||||
writeFile(t, filepath.Join(dir, "internal/logic/demo/"+name+".go"), "package demo\n")
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "internal/logic/demo/list.go"), readFile(t, "testdata/business/operations.go"))
|
||||
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/business/consumer_test.go"))
|
||||
writeFile(t, filepath.Join(dir, "internal/middleware/accessFactory.go"), readFile(t, "testdata/business/accessFactory.go"))
|
||||
writeFile(t, filepath.Join(dir, "internal/svc/client.go"), readFile(t, "testdata/business/client.go"))
|
||||
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n Client *accessClient", 1)
|
||||
source = strings.Replace(source, "return nil", "return c.initAccess(ctx)", 1)
|
||||
writeFile(t, svcFile, source)
|
||||
o.API = filepath.Join(dir, "api/business.api")
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-race", "-mod=mod", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("%v\n%s", err, out)
|
||||
}
|
||||
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
|
||||
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
raw, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatal(err, string(raw))
|
||||
}
|
||||
var module struct {
|
||||
Version, Sum string
|
||||
Replace any
|
||||
}
|
||||
if err = json.Unmarshal(raw, &module); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
|
||||
t.Fatal("not a fixed remote consumer")
|
||||
}
|
||||
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
|
||||
}
|
||||
}
|
||||
69
cmd/go-omni/apigen/catalogDocument.go
Normal file
69
cmd/go-omni/apigen/catalogDocument.go
Normal file
@@ -0,0 +1,69 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
const catalogDocumentFile = "omni.errors.json"
|
||||
|
||||
func readDocumentCatalog(root string, spec *Spec) error {
|
||||
path := filepath.Join(root, catalogDocumentFile)
|
||||
stat, err := os.Lstat(path)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !stat.Mode().IsRegular() {
|
||||
return errors.New("api: error metadata must be a regular file")
|
||||
}
|
||||
file, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer file.Close()
|
||||
raw, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(raw) > 1<<20 {
|
||||
return errors.New("api: error metadata exceeds 1 MiB")
|
||||
}
|
||||
var catalog apierrors.Catalog
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &catalog); err != nil {
|
||||
return err
|
||||
}
|
||||
registry, err := apierrors.NewRegistry(catalog)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
normalized := registry.Catalog()
|
||||
spec.DocumentCatalog = &normalized
|
||||
spec.Sources[catalogDocumentFile] = raw
|
||||
spec.SourcePaths[catalogDocumentFile] = path
|
||||
return nil
|
||||
}
|
||||
func validateDocumentCatalog(spec *Spec) error {
|
||||
if spec.DocumentCatalog == nil {
|
||||
return nil
|
||||
}
|
||||
for _, r := range spec.Routes {
|
||||
if len(r.Errors) > 0 {
|
||||
continue
|
||||
}
|
||||
policy, err := response.New(responseConfig(spec, r))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err = policy.ValidateErrors(*spec.DocumentCatalog); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -40,8 +40,17 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
if prepareErr != nil {
|
||||
return nil, prepareErr
|
||||
}
|
||||
if err := validateDocumentCatalog(spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := spec.RequestPolicy.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
definitions := map[string]Type{}
|
||||
for _, t := range spec.Types {
|
||||
if t.Name == "File" {
|
||||
return nil, errors.New("File is reserved for multipart uploads")
|
||||
}
|
||||
if t.Name == "ContractScalars" && len(spec.Scalars) > 0 {
|
||||
return nil, errors.New("ContractScalars is reserved when declaring scalars")
|
||||
}
|
||||
@@ -61,7 +70,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
resolving := map[string]bool{}
|
||||
var resolve func(string, int) (reflect.Type, error)
|
||||
var expression func(ast.Expr, int) (reflect.Type, error)
|
||||
primitives := map[string]reflect.Type{"any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||
primitives := map[string]reflect.Type{"File": reflect.TypeOf(contract.File{}), "any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
|
||||
if depth > 32 {
|
||||
return nil, errors.New("type nesting exceeds 32")
|
||||
@@ -138,7 +147,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err)
|
||||
}
|
||||
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)})
|
||||
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(fieldTag(f))})
|
||||
}
|
||||
t := reflect.StructOf(fields)
|
||||
resolved[name] = t
|
||||
@@ -162,7 +171,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
}
|
||||
handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{}
|
||||
middlewareSymbols := map[string]string{}
|
||||
for _, r := range spec.Routes {
|
||||
for routeIndex, r := range spec.Routes {
|
||||
if !goName.MatchString(r.Handler) {
|
||||
return nil, fmt.Errorf("%s: invalid handler name", r.Position)
|
||||
}
|
||||
@@ -233,14 +242,26 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
if err := validateRouteExtensions(r); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||
}
|
||||
effectivePolicy := contract.MergePolicy(spec.RequestPolicy, r.RequestPolicy)
|
||||
if (r.Method == "get" || r.Method == "head") && effectivePolicy.Body != "" && effectivePolicy.Body != "auto" && effectivePolicy.Body != "none" {
|
||||
return nil, errors.New("GET/HEAD cannot inherit a JSON body policy; override body with auto or none")
|
||||
}
|
||||
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
|
||||
if r.Request != "" {
|
||||
b, ok := c.Bindings[r.Request]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown request %s", r.Request)
|
||||
}
|
||||
r.BoundMultipart = b.HasMultipart() && !r.CustomBinding && !r.Manual
|
||||
spec.Routes[routeIndex].BoundMultipart = r.BoundMultipart
|
||||
if b.HasMultipart() && r.BodyMode != "multipart" {
|
||||
return nil, errors.New("multipart fields require @body (mode: multipart)")
|
||||
}
|
||||
op.Parameters = b.Parameters()
|
||||
op.Request = b.BodySchema()
|
||||
if op.Request != nil && (effectivePolicy.Body == "none" || effectivePolicy.Body == "emptyObject") {
|
||||
return nil, errors.New("request policy conflicts with JSON fields")
|
||||
}
|
||||
if op.Request != nil && (r.BodyMode == "raw" || r.BodyMode == "multipart") {
|
||||
return nil, errors.New("raw/multipart request DTOs may declare only path/form/header fields; use the generated body stream for content")
|
||||
}
|
||||
@@ -250,11 +271,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
}
|
||||
var response *openapi.Schema
|
||||
if r.Response != "" {
|
||||
b, ok := c.Bindings[r.Response]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown response %s", r.Response)
|
||||
}
|
||||
s, err := b.ResponseSchema()
|
||||
s, err := responseDataSchema(r.Response, c.Bindings)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -295,7 +312,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
if err = decoder.Decode(&doc); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
decorateDocument(doc, spec)
|
||||
decorateDocument(doc, spec, c.Bindings)
|
||||
c.Document, err = json.MarshalIndent(doc, "", " ")
|
||||
return c, err
|
||||
}
|
||||
|
||||
@@ -90,7 +90,8 @@ func snapshotContract(spec *Spec) *contractSnapshot {
|
||||
}
|
||||
}
|
||||
visit(r.Request)
|
||||
visit(r.Response)
|
||||
_, responseName := responseShape(r.Response)
|
||||
visit(responseName)
|
||||
op := routeContract{
|
||||
Method: strings.ToUpper(r.Method), Path: r.Path, Group: r.Group, Handler: r.Handler,
|
||||
Request: r.Request, Response: r.Response, BodyMode: r.BodyMode, Produces: r.Produces,
|
||||
@@ -223,14 +224,14 @@ func expectedLogicSignature(r Route) string {
|
||||
if r.Request != "" {
|
||||
params = append(params, "*types."+r.Request)
|
||||
}
|
||||
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
|
||||
if streamsBody(r) {
|
||||
params = append(params, "*httpx.Body")
|
||||
}
|
||||
results := []string{"error"}
|
||||
if r.Produces != "" {
|
||||
results = []string{"*httpx.Stream", "error"}
|
||||
} else if r.Response != "" {
|
||||
results = []string{"*types." + r.Response, "error"}
|
||||
} else if result := responseGoType(r.Response); result != "" {
|
||||
results = []string{result, "error"}
|
||||
}
|
||||
return displaySignature(upper(r.Handler), params, results)
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ package apigen
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"sort"
|
||||
@@ -119,7 +120,8 @@ func prepareMetadata(original *Spec) (*Spec, error) {
|
||||
return &spec, nil
|
||||
}
|
||||
|
||||
func decorateDocument(doc map[string]any, spec *Spec) {
|
||||
func decorateDocument(doc map[string]any, spec *Spec, bindings map[string]*contract.Binder) {
|
||||
doc["x-omni-contract-sources"] = sourceMetadata(spec)
|
||||
if spec.DefaultResponse != "" {
|
||||
doc["x-omni-default-response"] = spec.DefaultResponse
|
||||
}
|
||||
@@ -144,7 +146,12 @@ func decorateDocument(doc map[string]any, spec *Spec) {
|
||||
}
|
||||
doc["components"].(map[string]any)["securitySchemes"] = schemes
|
||||
definitions := map[string]ErrorDefinition{}
|
||||
registry := apierrors.MustRegistry(errorCatalog(spec, nil))
|
||||
declaredRegistry := apierrors.MustRegistry(errorCatalog(spec, nil))
|
||||
var documentRegistry *apierrors.Registry
|
||||
if spec.DocumentCatalog != nil {
|
||||
documentRegistry = apierrors.MustRegistry(*spec.DocumentCatalog)
|
||||
doc["x-omni-app-error-catalog"] = documentRegistry.Catalog()
|
||||
}
|
||||
for _, e := range spec.Errors {
|
||||
definitions[e.Name] = e
|
||||
}
|
||||
@@ -152,6 +159,7 @@ func decorateDocument(doc map[string]any, spec *Spec) {
|
||||
doc["x-omni-errors"] = spec.Errors
|
||||
}
|
||||
for _, r := range spec.Routes {
|
||||
registry := declaredRegistry
|
||||
policy := response.MustNew(responseConfig(spec, r))
|
||||
cfg := policy.Config()
|
||||
parts := strings.Split(r.Path, "/")
|
||||
@@ -161,7 +169,16 @@ func decorateDocument(doc map[string]any, spec *Spec) {
|
||||
}
|
||||
}
|
||||
op := doc["paths"].(map[string]any)[strings.Join(parts, "/")].(map[string]any)[r.Method].(map[string]any)
|
||||
decorateRequestOptions(op, r)
|
||||
requestRoute := r
|
||||
requestRoute.RequestPolicy = contract.MergePolicy(spec.RequestPolicy, r.RequestPolicy)
|
||||
decorateRequestOptions(op, requestRoute)
|
||||
if b := bindings[r.Request]; b != nil && b.HasMultipart() {
|
||||
encoding := map[string]any{}
|
||||
for _, name := range b.MultipartJSONFields() {
|
||||
encoding[name] = map[string]any{"contentType": "application/json"}
|
||||
}
|
||||
op["requestBody"] = map[string]any{"required": true, "content": map[string]any{"multipart/form-data": map[string]any{"schema": b.MultipartSchema(), "encoding": encoding}}}
|
||||
}
|
||||
if r.ResponseFormat != "" {
|
||||
op["x-omni-response"] = r.ResponseFormat
|
||||
}
|
||||
@@ -193,6 +210,13 @@ func decorateDocument(doc map[string]any, spec *Spec) {
|
||||
e := definitions[name]
|
||||
grouped[e.Status] = append(grouped[e.Status], e)
|
||||
}
|
||||
if len(r.Errors) == 0 && documentRegistry != nil {
|
||||
registry = documentRegistry
|
||||
op["x-omni-error-catalog-scope"] = "app"
|
||||
for key, definition := range registry.Catalog().Definitions {
|
||||
grouped[definition.Status] = append(grouped[definition.Status], ErrorDefinition{Name: key, Code: key, Status: definition.Status, Message: definition.Message, BusinessCode: definition.Code, Params: definition.Params})
|
||||
}
|
||||
}
|
||||
if len(r.Errors) > 0 {
|
||||
op["x-omni-errors"] = r.Errors
|
||||
}
|
||||
|
||||
127
cmd/go-omni/apigen/documentation_test.go
Normal file
127
cmd/go-omni/apigen/documentation_test.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFieldDocumentationAndFormat(t *testing.T) {
|
||||
source := "// 项目说明\n type Req{\n// 状态\n State int `json:\"state,options=0|1\" example:\"1\" enumDescriptions:\"[\\\"禁用\\\",\\\"启用\\\"]\"` // 状态值\n /* 第一行\n 第二行 */\nName string `json:\"name\" example:\"企业名称\"`\n}\nservice demo{\n @handler echo\npost /echo (Req) returns (Req)\n}\n"
|
||||
formatted, err := Format([]byte(source))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again, err := Format(formatted)
|
||||
if err != nil || !bytes.Equal(formatted, again) {
|
||||
t.Fatal("format is not idempotent", err, string(again))
|
||||
}
|
||||
if !bytes.Contains(formatted, []byte(" // 状态\n State")) {
|
||||
t.Fatal(string(formatted))
|
||||
}
|
||||
spec, err := parseText(t, string(formatted))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if spec.Types[0].Fields[0].Description != "状态\n状态值" || spec.Types[0].Fields[1].Description != "第一行\n第二行" {
|
||||
t.Fatalf("%+v", spec.Types[0].Fields)
|
||||
}
|
||||
schema, err := c.Bindings["Req"].ResponseSchema()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
state := schema.Properties["state"]
|
||||
if state.Description != "状态\n状态值" || len(state.EnumDescriptions) != 2 || state.Example != json.Number("1") {
|
||||
t.Fatalf("%+v", state)
|
||||
}
|
||||
dir := t.TempDir()
|
||||
entry := filepath.Join(dir, "demo.api")
|
||||
writeFile(t, entry, source)
|
||||
if _, err = FormatProject(entry, true); err == nil {
|
||||
t.Fatal("check accepted drift")
|
||||
}
|
||||
if readFile(t, entry) != source {
|
||||
t.Fatal("check wrote source")
|
||||
}
|
||||
if _, err = FormatProject(entry, false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = FormatProject(entry, true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
project := filepath.Join(t.TempDir(), "demo")
|
||||
if _, err = Generate(Options{API: entry, Dir: project, Module: "example.com/docs", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dto := readFile(t, filepath.Join(project, "internal/types/"+dtoFile("demo.api:1:1")))
|
||||
if !strings.Contains(dto, "// 状态值") || !strings.Contains(dto, `description:"状态\n状态值"`) {
|
||||
t.Fatal(dto)
|
||||
}
|
||||
// Invalid examples fail before project output is created.
|
||||
bad := strings.Replace(source, "example:\"1\"", "example:\"9\"", 1)
|
||||
writeFile(t, entry, bad)
|
||||
if _, err = FormatProject(entry, false); err == nil {
|
||||
t.Fatal("invalid enum example accepted")
|
||||
}
|
||||
if data, _ := os.ReadFile(entry); string(data) != bad {
|
||||
t.Fatal("invalid source changed")
|
||||
}
|
||||
}
|
||||
|
||||
func TestFormatAlignsFieldsAndPreservesTokens(t *testing.T) {
|
||||
source := []byte("type Req {\n Name string `json:\"name\"` // 名称\n Sequence int64 `json:\"sequence\"` // 编号\n}\n")
|
||||
out, err := Format(source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again, err := Format(out)
|
||||
if err != nil || !bytes.Equal(out, again) {
|
||||
t.Fatal("alignment not idempotent", err)
|
||||
}
|
||||
lines := strings.Split(string(out), "\n")
|
||||
if strings.Index(lines[1], "`") != strings.Index(lines[2], "`") || strings.Index(lines[1], "//") != strings.Index(lines[2], "//") {
|
||||
t.Fatal("fields not aligned", string(out))
|
||||
}
|
||||
for _, literal := range []string{"`json:\"name\"`", "`json:\"sequence\"`", "// 名称", "// 编号"} {
|
||||
if !bytes.Contains(out, []byte(literal)) {
|
||||
t.Fatal("literal changed", literal)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRequestPolicyCannotCreateGETBody(t *testing.T) {
|
||||
for _, source := range []string{
|
||||
"service demo { @handler get\n @request(body: required)\n get /x }",
|
||||
"@app(request: {\"body\":\"optional\"})\n service demo { @handler get\n get /x }",
|
||||
} {
|
||||
spec, err := parseText(t, source)
|
||||
if err == nil {
|
||||
_, err = Compile(spec)
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("GET body policy accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestFormatLiteralPreservation(t *testing.T) {
|
||||
source := []byte("type Demo {\n Body string `json:\"body\"\ndescription:\"literal space\"`\n}\n")
|
||||
out, err := Format(source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !bytes.Contains(out, []byte("`json:\"body\"\ndescription:\"literal space\"`")) {
|
||||
t.Fatal(string(out))
|
||||
}
|
||||
for _, src := range []string{"type X {", "}", "info (title:\"unterminated)"} {
|
||||
if _, err := Format([]byte(src)); err == nil {
|
||||
t.Fatal("invalid input accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
207
cmd/go-omni/apigen/format.go
Normal file
207
cmd/go-omni/apigen/format.go
Normal file
@@ -0,0 +1,207 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"text/scanner"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// Format normalizes indentation and blank lines without rewriting tokens,
|
||||
// comments, quoted metadata, struct tags or multiline literal contents.
|
||||
func Format(source []byte) ([]byte, error) {
|
||||
if len(source) > 1<<20 {
|
||||
return nil, errors.New("api: source size limit exceeded")
|
||||
}
|
||||
var scan scanner.Scanner
|
||||
scan.Init(bytes.NewReader(source))
|
||||
scan.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments
|
||||
var scanErr error
|
||||
scan.Error = func(s *scanner.Scanner, msg string) {
|
||||
if scanErr == nil {
|
||||
scanErr = fmt.Errorf("%s: %s", s.Position, msg)
|
||||
}
|
||||
}
|
||||
depths := map[int]int{}
|
||||
protected := map[int]bool{}
|
||||
depth := 0
|
||||
for k := scan.Scan(); k != scanner.EOF; k = scan.Scan() {
|
||||
line := scan.Position.Line
|
||||
value := scan.TokenText()
|
||||
if k == '}' || k == ')' {
|
||||
depth--
|
||||
if depth < 0 {
|
||||
return nil, errors.New("api: unmatched closing delimiter")
|
||||
}
|
||||
}
|
||||
if _, ok := depths[line]; !ok {
|
||||
depths[line] = depth
|
||||
}
|
||||
// Preserve interior lines exactly, including the closing line of a literal.
|
||||
count := strings.Count(value, "\n")
|
||||
for n := 1; n <= count; n++ {
|
||||
protected[line+n] = true
|
||||
}
|
||||
if k == '{' || k == '(' {
|
||||
depth++
|
||||
}
|
||||
}
|
||||
if scanErr != nil {
|
||||
return nil, scanErr
|
||||
}
|
||||
if depth != 0 {
|
||||
return nil, errors.New("api: unclosed delimiter")
|
||||
}
|
||||
lines := strings.Split(string(source), "\n")
|
||||
out := make([]string, 0, len(lines))
|
||||
outProtected := map[int]bool{}
|
||||
blank := true
|
||||
lastDepth := 0
|
||||
for i, line := range lines {
|
||||
if protected[i+1] {
|
||||
out = append(out, line)
|
||||
outProtected[len(out)] = true
|
||||
blank = false
|
||||
continue
|
||||
}
|
||||
trimmed := strings.TrimSpace(line)
|
||||
if trimmed == "" {
|
||||
if !blank {
|
||||
out = append(out, "")
|
||||
blank = true
|
||||
}
|
||||
continue
|
||||
}
|
||||
d, ok := depths[i+1]
|
||||
if ok {
|
||||
lastDepth = d
|
||||
}
|
||||
out = append(out, strings.Repeat(" ", lastDepth)+trimmed)
|
||||
blank = false
|
||||
}
|
||||
for len(out) > 0 && out[len(out)-1] == "" {
|
||||
out = out[:len(out)-1]
|
||||
}
|
||||
alignFieldLines(out, outProtected)
|
||||
return []byte(strings.Join(out, "\n") + "\n"), nil
|
||||
}
|
||||
|
||||
// Only complete, single-line fields are aligned. Literal bytes and multiline
|
||||
// comment contents are never candidates, even when they resemble a field.
|
||||
var singleLineField = regexp.MustCompile("^([ \\t]*)([A-Z][A-Za-z0-9_]*)[ \\t]+([^ \\t`]+)[ \\t]+(`[^`]*`)[ \\t]*(//.*)?$")
|
||||
|
||||
func alignFieldLines(lines []string, protected map[int]bool) {
|
||||
for start := 0; start < len(lines); {
|
||||
var fields [][]string
|
||||
widths := [3]int{}
|
||||
end := start
|
||||
for end < len(lines) && !protected[end+1] {
|
||||
parts := singleLineField.FindStringSubmatch(lines[end])
|
||||
if parts == nil || (len(fields) > 0 && parts[1] != fields[0][1]) {
|
||||
break
|
||||
}
|
||||
for col := range widths {
|
||||
if n := utf8.RuneCountInString(parts[col+2]); n > widths[col] {
|
||||
widths[col] = n
|
||||
}
|
||||
}
|
||||
fields = append(fields, parts)
|
||||
end++
|
||||
}
|
||||
for i, parts := range fields {
|
||||
line := parts[1]
|
||||
for col := range widths {
|
||||
line += parts[col+2]
|
||||
if col < 2 || parts[5] != "" {
|
||||
line += strings.Repeat(" ", widths[col]-utf8.RuneCountInString(parts[col+2])+1)
|
||||
}
|
||||
}
|
||||
lines[start+i] = line + parts[5]
|
||||
}
|
||||
if end == start {
|
||||
end++
|
||||
}
|
||||
start = end
|
||||
}
|
||||
}
|
||||
|
||||
// FormatProject validates the full contract before writing any local source.
|
||||
// Check mode is read-only and reports every file needing formatting.
|
||||
func FormatProject(entry string, check bool) ([]string, error) {
|
||||
spec, err := Parse(entry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, err = Compile(spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
real, err := filepath.EvalSymlinks(entry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
root := filepath.Dir(real)
|
||||
names := make([]string, 0, len(spec.Sources))
|
||||
for name := range spec.Sources {
|
||||
if strings.HasPrefix(name, "@") || filepath.Ext(name) != ".api" {
|
||||
continue
|
||||
}
|
||||
names = append(names, name)
|
||||
}
|
||||
sort.Strings(names)
|
||||
next := map[string][]byte{}
|
||||
var changed []string
|
||||
for _, name := range names {
|
||||
data, err := Format(spec.Sources[name])
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", name, err)
|
||||
}
|
||||
if !bytes.Equal(data, spec.Sources[name]) {
|
||||
next[name] = data
|
||||
changed = append(changed, name)
|
||||
}
|
||||
}
|
||||
if check {
|
||||
if len(changed) > 0 {
|
||||
return changed, fmt.Errorf("API format drift: %s", strings.Join(changed, ", "))
|
||||
}
|
||||
return changed, nil
|
||||
}
|
||||
for _, name := range changed {
|
||||
path := filepath.Join(root, filepath.FromSlash(name))
|
||||
old, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
return changed, err
|
||||
}
|
||||
if !bytes.Equal(old, spec.Sources[name]) {
|
||||
return changed, fmt.Errorf("API source changed during formatting: %s", name)
|
||||
}
|
||||
stat, err := os.Stat(path)
|
||||
if err != nil {
|
||||
return changed, err
|
||||
}
|
||||
f, err := os.CreateTemp(filepath.Dir(path), ".omni-format-*")
|
||||
if err != nil {
|
||||
return changed, err
|
||||
}
|
||||
tmp := f.Name()
|
||||
err = f.Chmod(stat.Mode().Perm())
|
||||
if err == nil {
|
||||
_, err = f.Write(next[name])
|
||||
}
|
||||
err = errors.Join(err, f.Close())
|
||||
if err == nil {
|
||||
err = os.Rename(tmp, path)
|
||||
}
|
||||
_ = os.Remove(tmp)
|
||||
if err != nil {
|
||||
return changed, err
|
||||
}
|
||||
}
|
||||
return changed, nil
|
||||
}
|
||||
@@ -19,10 +19,11 @@ import (
|
||||
)
|
||||
|
||||
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
const minimumVersion = "v0.13.0-preview"
|
||||
const minimumVersion = "v0.14.0-preview"
|
||||
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||
|
||||
type Options struct {
|
||||
lockHeld bool
|
||||
Database, Storage string
|
||||
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
|
||||
Check bool
|
||||
@@ -95,6 +96,9 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
return Output{}, err
|
||||
}
|
||||
o.Dir = directory
|
||||
if err = checkRuntimeExtension(directory); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
for _, r := range c.Spec.Routes {
|
||||
if !r.Manual {
|
||||
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
|
||||
@@ -109,6 +113,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
return Output{}, err
|
||||
}
|
||||
seeds := map[string][]byte{}
|
||||
requiredVersion := ""
|
||||
moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod"))
|
||||
if readErr == nil {
|
||||
mf, err := modfile.Parse("go.mod", moduleData, nil)
|
||||
@@ -120,7 +125,6 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
}
|
||||
o.Module = mf.Module.Mod.Path
|
||||
found := false
|
||||
requiredVersion := ""
|
||||
for _, r := range mf.Require {
|
||||
if r.Mod.Path == frameworkModule {
|
||||
found = true
|
||||
@@ -141,13 +145,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
if !filepath.IsAbs(path) {
|
||||
path = filepath.Join(directory, path)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(path, "lifecycle", "controller.go")); err == nil {
|
||||
if _, err := os.Stat(filepath.Join(path, "lifecycle", "runtime.go")); err == nil {
|
||||
localCurrent = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !localCurrent {
|
||||
return Output{}, fmt.Errorf("existing framework dependency predates dependency shutdown and configurable endpoint support; use %s or a current local replacement", minimumVersion)
|
||||
return Output{}, fmt.Errorf("existing framework dependency predates the generated Runtime, request policy and structured MySQL support; use %s or a current local replacement", minimumVersion)
|
||||
}
|
||||
}
|
||||
if o.FrameworkDir != "" {
|
||||
@@ -191,7 +195,16 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
// from its own api/ tree, including all imports.
|
||||
entry := "api/" + c.Spec.Entry
|
||||
for name, data := range c.Spec.Sources {
|
||||
seeds["api/"+name] = data
|
||||
if !strings.HasPrefix(name, "@") {
|
||||
seeds["api/"+name] = data
|
||||
}
|
||||
}
|
||||
if len(c.Spec.SharedRoots) > 0 {
|
||||
raw, e := sharedSeeds(c.Spec, directory)
|
||||
if e != nil {
|
||||
return Output{}, e
|
||||
}
|
||||
seeds["api/"+importsFile] = raw
|
||||
}
|
||||
if o.API != "" {
|
||||
source, _ := filepath.Abs(o.API)
|
||||
@@ -200,6 +213,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
rel, e := filepath.Rel(targetReal, sourceReal)
|
||||
if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
entry = filepath.ToSlash(rel)
|
||||
delete(seeds, "api/"+importsFile)
|
||||
for name := range c.Spec.Sources {
|
||||
delete(seeds, "api/"+name)
|
||||
}
|
||||
@@ -208,7 +222,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
// Source collisions must never leave a copied contract different from the
|
||||
// generated code. On updates, use the developer-owned API in the project.
|
||||
for name, data := range seeds {
|
||||
if strings.HasSuffix(name, ".api") {
|
||||
if strings.HasSuffix(name, ".api") || filepath.Base(name) == catalogDocumentFile || filepath.Base(name) == importsFile {
|
||||
current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name)))
|
||||
if e == nil && string(current) != string(data) {
|
||||
return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name)
|
||||
@@ -227,7 +241,16 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
return apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec))
|
||||
out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld)
|
||||
expectedVersion := minimumVersion
|
||||
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) {
|
||||
expectedVersion = info.Main.Version
|
||||
}
|
||||
if requiredVersion != "" && requiredVersion != expectedVersion {
|
||||
out.Diagnostics = append(out.Diagnostics, Diagnostic{Severity: "review", Code: "frameworkVersion", Path: "go.mod", Before: requiredVersion, After: expectedVersion, Message: "project dependency differs from this CLI version; use matching CLI/framework versions, or verify the explicit local replacement"})
|
||||
sortDiagnostics(out.Diagnostics)
|
||||
}
|
||||
return out, err
|
||||
}
|
||||
|
||||
func moduleFile(o Options) ([]byte, error) {
|
||||
@@ -336,6 +359,13 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
}
|
||||
|
||||
types := "package types\n\n"
|
||||
needsUpload := false
|
||||
for _, b := range c.Bindings {
|
||||
needsUpload = needsUpload || b.HasMultipart()
|
||||
}
|
||||
if needsUpload {
|
||||
types += fmt.Sprintf("import %q\n", frameworkModule+"/contract")
|
||||
}
|
||||
if len(c.Spec.Scalars) > 0 {
|
||||
types += "import("
|
||||
for i, d := range c.Spec.Scalars {
|
||||
@@ -350,12 +380,28 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
registry += "}}\n"
|
||||
put(generated, "internal/types/scalarsGen.go", generatedHeader+registry)
|
||||
}
|
||||
if needsUpload {
|
||||
types += "type File = contract.File\n"
|
||||
}
|
||||
bySource := map[string]string{}
|
||||
for _, t := range c.Spec.Types {
|
||||
types += "type " + t.Name + " struct {\n"
|
||||
for _, f := range t.Fields {
|
||||
types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag)
|
||||
name := dtoFile(t.Position)
|
||||
if bySource[name] == "" {
|
||||
bySource[name] = "package types\n\n"
|
||||
}
|
||||
types += "}\n"
|
||||
text := "type " + t.Name + " struct {\n"
|
||||
for _, f := range t.Fields {
|
||||
if f.Description != "" {
|
||||
for _, line := range strings.Split(f.Description, "\n") {
|
||||
text += "// " + line + "\n"
|
||||
}
|
||||
}
|
||||
text += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, fieldTag(f))
|
||||
}
|
||||
bySource[name] += text + "}\n"
|
||||
}
|
||||
for name, text := range bySource {
|
||||
put(generated, "internal/types/"+name, generatedHeader+text)
|
||||
}
|
||||
put(generated, "internal/types/typesGen.go", generatedHeader+types)
|
||||
put(generated, "docs/openapi.json", string(c.Document)+"\n")
|
||||
@@ -408,8 +454,8 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
continue
|
||||
}
|
||||
hasAutomatic = true
|
||||
needsTypes = needsTypes || r.Request != "" || r.Response != ""
|
||||
needsBinder = needsBinder || (r.Request != "" && !r.CustomBinding)
|
||||
needsTypes = needsTypes || r.Request != "" || responseGoType(r.Response) != ""
|
||||
needsBinder = needsBinder || !r.CustomBinding
|
||||
}
|
||||
text += fmt.Sprintf("%q;%q;", frameworkModule+"/httpx", o.Module+"/internal/svc")
|
||||
if needsTypes {
|
||||
@@ -442,12 +488,18 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Manual.go", fmt.Sprintf("package %s\nimport(%q;apierrors %q)\n// %s owns raw/multipart binding and response output. h.deps holds project dependencies.\n// This file is preserved on regeneration.\nfunc(h *Handler)%s(c *httpx.Context)error{return apierrors.New(501,\"NOT_IMPLEMENTED\",\"manual handler is not implemented\")}\n", packageName, frameworkModule+"/httpx", frameworkModule+"/errors", name, name))
|
||||
continue
|
||||
}
|
||||
if r.Request != "" && !r.CustomBinding {
|
||||
if !r.CustomBinding {
|
||||
scalarArgs := ""
|
||||
if len(c.Spec.Scalars) > 0 {
|
||||
scalarArgs = "types.ContractScalars()..."
|
||||
}
|
||||
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s](%s)\n", lower(r.Handler), r.Request, scalarArgs)
|
||||
requestType := "struct{}"
|
||||
if r.Request != "" {
|
||||
requestType = "types." + r.Request
|
||||
} else {
|
||||
scalarArgs = ""
|
||||
}
|
||||
text += fmt.Sprintf("var %sRequest = contract.MustNew[%s](%s)\n", lower(r.Handler), requestType, scalarArgs)
|
||||
}
|
||||
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
|
||||
call := "c.Context()"
|
||||
@@ -455,16 +507,32 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
if r.CustomBinding {
|
||||
text += fmt.Sprintf("var req types.%s\nif err:=h.bind%s(c,&req);err!=nil{return err}\n", r.Request, name)
|
||||
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Bind.go", fmt.Sprintf("package %s\nimport(%q;%q;apierrors %q)\n// bind%s is project-owned; repeated generation preserves it.\nfunc(h *Handler)bind%s(c *httpx.Context,req *types.%s)error{return apierrors.New(501,\"BINDING_NOT_IMPLEMENTED\",\"custom binding is not implemented\")}\n", packageName, frameworkModule+"/httpx", o.Module+"/internal/types", frameworkModule+"/errors", name, name, r.Request))
|
||||
} else if r.BoundMultipart {
|
||||
text += fmt.Sprintf("var req types.%s\ncleanup,err:=%sRequest.BindMultipart(c.Request(),httpx.RouteParams(c.Context()),&req,contract.MultipartOptions{MaxBodyBytes:%d,MaxParts:%d,MaxFieldBytes:%d,MaxFileBytes:%d,Policy:%s});if err!=nil{return err};defer cleanup()\n", r.Request, lower(r.Handler), r.MaxBodyBytes, r.MaxParts, r.MaxFieldBytes, r.MaxFileBytes, policyLiteral(r.RequestPolicy))
|
||||
} else {
|
||||
limit := ""
|
||||
if r.MaxBodyBytes > 0 {
|
||||
limit = fmt.Sprintf(",%d", r.MaxBodyBytes)
|
||||
}
|
||||
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req%s);err!=nil{return err}\n", r.Request, lower(r.Handler), limit)
|
||||
policy := r.RequestPolicy
|
||||
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
|
||||
policy.Body = "auto"
|
||||
}
|
||||
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.BindWithPolicy(c.Request(),httpx.RouteParams(c.Context()),&req,%s%s);err!=nil{return err}\n", r.Request, lower(r.Handler), policyLiteral(policy), limit)
|
||||
}
|
||||
call += ", &req"
|
||||
} else {
|
||||
limit := ""
|
||||
if r.MaxBodyBytes > 0 {
|
||||
limit = fmt.Sprintf(",%d", r.MaxBodyBytes)
|
||||
}
|
||||
policy := r.RequestPolicy
|
||||
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
|
||||
policy.Body = "auto"
|
||||
}
|
||||
text += fmt.Sprintf("if err:=%sRequest.BindWithPolicy(c.Request(),httpx.RouteParams(c.Context()),&struct{}{},%s%s);err!=nil{return err}\n", lower(r.Handler), policyLiteral(policy), limit)
|
||||
}
|
||||
streamingBody := r.BodyMode == "raw" || r.BodyMode == "multipart"
|
||||
streamingBody := streamsBody(r)
|
||||
if streamingBody {
|
||||
text += fmt.Sprintf("body,err:=httpx.NewBody(c.Request(),%q);if err!=nil{return err}\n", r.BodyMode)
|
||||
call += ",body"
|
||||
@@ -475,8 +543,15 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
text += "if err:=body.Complete();err!=nil{if resp!=nil{_ =resp.Close()};return err}\n"
|
||||
}
|
||||
text += fmt.Sprintf("return c.WriteStream(%d,%q,resp)\n}\n", r.Status, r.Produces)
|
||||
} else if r.Response != "" {
|
||||
text += fmt.Sprintf("resp,err:=h.logic.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name)
|
||||
} else if responseGoType(r.Response) != "" {
|
||||
text += fmt.Sprintf("resp,err:=h.logic.%s(%s);if err!=nil{return err};\n", name, call)
|
||||
kind, _ := responseShape(r.Response)
|
||||
if kind == "object" {
|
||||
text += fmt.Sprintf("if resp==nil{return fmt.Errorf(%q)}\n", name+" returned a nil response")
|
||||
}
|
||||
if kind == "array" {
|
||||
text += "if resp==nil{resp=make(" + responseGoType(r.Response) + ",0)}\n"
|
||||
}
|
||||
if streamingBody {
|
||||
text += "if err:=body.Complete();err!=nil{return err}\n"
|
||||
}
|
||||
@@ -487,10 +562,15 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
if streamingBody {
|
||||
text += "if err:=body.Complete();err!=nil{return err};"
|
||||
}
|
||||
text += "return c.NoContent()\n}\n"
|
||||
if r.Response == "null" {
|
||||
method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status]
|
||||
text += "return c." + method + "(nil)\n}\n"
|
||||
} else {
|
||||
text += "return c.NoContent()\n}\n"
|
||||
}
|
||||
}
|
||||
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
|
||||
if r.Request != "" || r.Response != "" {
|
||||
if r.Request != "" || responseGoType(r.Response) != "" {
|
||||
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||
}
|
||||
if streamingBody || r.Produces != "" {
|
||||
@@ -503,7 +583,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}"
|
||||
} else {
|
||||
service += "// TODO: implement business behavior and use s.deps for dependencies.\n"
|
||||
if r.Response != "" || r.Produces != "" {
|
||||
if responseGoType(r.Response) != "" || r.Produces != "" {
|
||||
service += "return nil, "
|
||||
} else {
|
||||
service += "return "
|
||||
@@ -549,7 +629,11 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
return nil, nil, err
|
||||
}
|
||||
for name, data := range infra {
|
||||
seeds[name] = data
|
||||
if strings.HasSuffix(name, "Gen.go") {
|
||||
generated[name] = append([]byte(generatedHeader), data...)
|
||||
} else {
|
||||
seeds[name] = data
|
||||
}
|
||||
}
|
||||
if o.Storage == "local" {
|
||||
for _, r := range c.Spec.Routes {
|
||||
@@ -564,7 +648,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
}
|
||||
func hasResponse(routes []Route) bool {
|
||||
for _, r := range routes {
|
||||
if r.Response != "" && !r.Manual && r.Produces == "" {
|
||||
if kind, _ := responseShape(r.Response); r.Response != "" && kind == "object" && !r.Manual && r.Produces == "" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
@@ -575,17 +659,41 @@ func signature(r Route) string {
|
||||
if r.Request != "" {
|
||||
s += ",req *types." + r.Request
|
||||
}
|
||||
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
|
||||
if streamsBody(r) {
|
||||
s += ",body *httpx.Body"
|
||||
}
|
||||
s += ") "
|
||||
if r.Produces != "" {
|
||||
return s + "(*httpx.Stream,error)"
|
||||
}
|
||||
if r.Response != "" {
|
||||
s += "(*types." + r.Response + ",error)"
|
||||
if result := responseGoType(r.Response); result != "" {
|
||||
s += "(" + result + ",error)"
|
||||
} else {
|
||||
s += "error"
|
||||
}
|
||||
return s
|
||||
}
|
||||
|
||||
func streamsBody(r Route) bool {
|
||||
return r.BodyMode == "raw" || (r.BodyMode == "multipart" && !r.BoundMultipart)
|
||||
}
|
||||
|
||||
func dtoFile(position string) string {
|
||||
source := position
|
||||
for i := 0; i < 2; i++ {
|
||||
if idx := strings.LastIndex(source, ":"); idx >= 0 {
|
||||
source = source[:idx]
|
||||
}
|
||||
}
|
||||
stem := strings.TrimSuffix(filepath.Base(source), ".api")
|
||||
var b strings.Builder
|
||||
for _, r := range stem {
|
||||
if r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' {
|
||||
b.WriteRune(r)
|
||||
}
|
||||
}
|
||||
if b.Len() == 0 {
|
||||
b.WriteString("contract")
|
||||
}
|
||||
return "dto" + upper(b.String()) + digest([]byte(source))[:12] + "Gen.go"
|
||||
}
|
||||
|
||||
@@ -42,7 +42,7 @@ func checkLogic(root, module string, contract *contractSnapshot, seeds map[strin
|
||||
typeExtensions := false
|
||||
if entries, err := os.ReadDir(filepath.Join(root, "internal/types")); err == nil {
|
||||
for _, e := range entries {
|
||||
if sourceFile(e.Name()) && e.Name() != "typesGen.go" && e.Name() != "scalarsGen.go" {
|
||||
if sourceFile(e.Name()) && !ownedPath("internal/types/"+e.Name()) {
|
||||
typeExtensions = true
|
||||
}
|
||||
}
|
||||
|
||||
@@ -82,7 +82,7 @@ func TestManagedGeneratedProject(t *testing.T) {
|
||||
writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1))
|
||||
svcPath := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
source := strings.Replace(readFile(t, svcPath), "type ServiceContext struct {", "type ServiceContext struct {\n Client *Client", 1)
|
||||
source = strings.Replace(source, "return c, nil", "if err=initClient(ctx,c);err!=nil{return nil,err}\n return c,nil", 1)
|
||||
source = strings.Replace(source, "return nil", "return initClient(ctx,c)", 1)
|
||||
writeFile(t, svcPath, source)
|
||||
// Independent tooling stays outside generator ownership.
|
||||
writeFile(t, filepath.Join(dir, "cmd/migrate/main.go"), "package main\nfunc main(){}\n")
|
||||
|
||||
@@ -5,6 +5,8 @@ package apigen
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
@@ -14,6 +16,10 @@ import (
|
||||
)
|
||||
|
||||
type Spec struct {
|
||||
DocumentCatalog *apierrors.Catalog
|
||||
SharedRoots map[string]SharedRoot
|
||||
SourcePaths map[string]string
|
||||
RequestPolicy contract.RequestPolicy
|
||||
DefaultResponse string
|
||||
appDeclared bool
|
||||
Title, Version, Service, Description string
|
||||
@@ -32,8 +38,12 @@ type Type struct {
|
||||
Fields []Field
|
||||
Position string
|
||||
}
|
||||
type Field struct{ Name, Type, Tag string }
|
||||
type Field struct{ Name, Type, Tag, Description string }
|
||||
type Route struct {
|
||||
BoundMultipart bool
|
||||
MaxParts int
|
||||
MaxFieldBytes, MaxFileBytes int64
|
||||
RequestPolicy contract.RequestPolicy
|
||||
Manual, CustomBinding bool
|
||||
BodyMode, Produces string
|
||||
MaxBodyBytes int64
|
||||
@@ -51,6 +61,7 @@ type Route struct {
|
||||
}
|
||||
type token struct {
|
||||
text, position string
|
||||
comment string
|
||||
kind rune
|
||||
space bool
|
||||
}
|
||||
@@ -71,11 +82,15 @@ func Parse(path string) (*Spec, error) {
|
||||
return nil, err
|
||||
}
|
||||
root := filepath.Dir(entry)
|
||||
spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||
roots, err := readSharedRoots(root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
spec := &Spec{Version: "1.0.0", SharedRoots: roots, SourcePaths: map[string]string{}, Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||
visiting, visited := map[string]bool{}, map[string]bool{}
|
||||
total := 0
|
||||
var visit func(string, int) error
|
||||
visit = func(name string, depth int) error {
|
||||
var visit func(string, string, int) error
|
||||
visit = func(name, alias string, depth int) error {
|
||||
if depth > 32 || len(visited)+len(visiting) >= 64 {
|
||||
return errors.New("api: import limit exceeded")
|
||||
}
|
||||
@@ -83,9 +98,16 @@ func Parse(path string) (*Spec, error) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel, err := filepath.Rel(root, real)
|
||||
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return errors.New("api: imports must stay within the entry API directory")
|
||||
sourceRoot := root
|
||||
if alias != "" {
|
||||
sourceRoot = roots[alias].Path
|
||||
}
|
||||
rel, err := sourceWithin(sourceRoot, real)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if alias != "" {
|
||||
rel = "@" + alias + "/" + rel
|
||||
}
|
||||
if filepath.Ext(real) != ".api" {
|
||||
return errors.New("api: expected .api file")
|
||||
@@ -120,18 +142,37 @@ func Parse(path string) (*Spec, error) {
|
||||
return err
|
||||
}
|
||||
spec.Sources[filepath.ToSlash(rel)] = data
|
||||
spec.SourcePaths[filepath.ToSlash(rel)] = real
|
||||
for _, imp := range p.imports {
|
||||
if filepath.IsAbs(imp) || strings.Contains(imp, "\\") {
|
||||
return errors.New("api: imports must use relative slash paths")
|
||||
}
|
||||
if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil {
|
||||
nextAlias := alias
|
||||
next := filepath.Join(filepath.Dir(real), filepath.FromSlash(imp))
|
||||
if strings.HasPrefix(imp, "@") {
|
||||
var sub string
|
||||
var ok bool
|
||||
nextAlias, sub, ok = strings.Cut(strings.TrimPrefix(imp, "@"), "/")
|
||||
declared, found := roots[nextAlias]
|
||||
if !ok || !found || sub == "" || strings.HasPrefix(sub, "/") || strings.HasPrefix(sub, "../") || filepath.ToSlash(filepath.Clean(sub)) != sub {
|
||||
return errors.New("api: import needs a declared @root and a clean relative path")
|
||||
}
|
||||
next = filepath.Join(declared.Path, filepath.FromSlash(sub))
|
||||
}
|
||||
if err = visit(next, nextAlias, depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
visited[real] = true
|
||||
return nil
|
||||
}
|
||||
if err = visit(entry, 0); err != nil {
|
||||
if err = visit(entry, "", 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = readDocumentCatalog(root, spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = validateSharedPins(spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if spec.Service == "" || len(spec.Routes) == 0 {
|
||||
@@ -146,7 +187,7 @@ func lex(name, source string) ([]token, error) {
|
||||
var s scanner.Scanner
|
||||
s.Init(strings.NewReader(source))
|
||||
s.Filename = name
|
||||
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments
|
||||
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments
|
||||
s.Whitespace ^= 1 << '\n'
|
||||
var scanErr error
|
||||
s.Error = func(s *scanner.Scanner, message string) {
|
||||
@@ -156,9 +197,29 @@ func lex(name, source string) ([]token, error) {
|
||||
}
|
||||
var out []token
|
||||
end := 0
|
||||
pending := ""
|
||||
lastLine := 0
|
||||
lastToken := -1
|
||||
for {
|
||||
kind := s.Scan()
|
||||
out = append(out, token{s.TokenText(), s.Position.String(), kind, s.Position.Offset > end})
|
||||
if kind == scanner.Comment {
|
||||
comment := cleanComment(s.TokenText())
|
||||
if lastToken >= 0 && s.Position.Line == lastLine {
|
||||
out[lastToken].comment = joinComment(out[lastToken].comment, comment)
|
||||
} else {
|
||||
pending = joinComment(pending, comment)
|
||||
}
|
||||
end = s.Position.Offset + len(s.TokenText())
|
||||
continue
|
||||
}
|
||||
current := token{text: s.TokenText(), position: s.Position.String(), kind: kind, space: s.Position.Offset > end}
|
||||
if kind != '\n' {
|
||||
current.comment = pending
|
||||
pending = ""
|
||||
lastLine = s.Position.Line
|
||||
lastToken = len(out)
|
||||
}
|
||||
out = append(out, current)
|
||||
end = s.Position.Offset + len(s.TokenText())
|
||||
if kind == scanner.EOF {
|
||||
break
|
||||
@@ -315,8 +376,20 @@ func (p *parser) parse() error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(m) != 1 || m["defaultResponse"] == "" {
|
||||
return p.err("@app requires only defaultResponse: TemplateName")
|
||||
if len(m) == 0 {
|
||||
return p.err("@app requires defaultResponse or request")
|
||||
}
|
||||
for key, value := range m {
|
||||
switch key {
|
||||
case "defaultResponse":
|
||||
case "request":
|
||||
p.spec.RequestPolicy, err = decodeRequestPolicy(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
default:
|
||||
return p.err("unsupported @app property " + key)
|
||||
}
|
||||
}
|
||||
p.spec.DefaultResponse = m["defaultResponse"]
|
||||
p.spec.appDeclared = true
|
||||
@@ -422,6 +495,7 @@ func (p *parser) parseType() error {
|
||||
}
|
||||
typ := Type{Name: name, Position: pos}
|
||||
for p.skip(); p.peek() != "}"; p.skip() {
|
||||
description := p.tokens[p.at].comment
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -444,11 +518,13 @@ func (p *parser) parseType() error {
|
||||
if p.tokens[p.at].kind != scanner.RawString {
|
||||
return p.err("field source tag required")
|
||||
}
|
||||
tag, err := strconv.Unquote(p.next().text)
|
||||
tagToken := p.next()
|
||||
description = joinComment(description, tagToken.comment)
|
||||
tag, err := strconv.Unquote(tagToken.text)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag})
|
||||
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag, Description: description})
|
||||
}
|
||||
p.next()
|
||||
p.spec.Types = append(p.spec.Types, typ)
|
||||
@@ -496,6 +572,12 @@ func (p *parser) parseService(server map[string]string) error {
|
||||
}
|
||||
seen[annotation] = true
|
||||
switch annotation {
|
||||
case "request":
|
||||
var props map[string]string
|
||||
props, e = p.metadata()
|
||||
if e == nil {
|
||||
route.RequestPolicy, e = parseRequestPolicy(props)
|
||||
}
|
||||
case "handlerMode":
|
||||
mode, err := p.ident()
|
||||
if err != nil {
|
||||
@@ -589,9 +671,48 @@ func (p *parser) routeType() (string, error) {
|
||||
if err := p.expect("("); err != nil {
|
||||
return "", err
|
||||
}
|
||||
prefix := ""
|
||||
if p.peek() == "[" {
|
||||
p.next()
|
||||
if err := p.expect("]"); err != nil {
|
||||
return "", err
|
||||
}
|
||||
prefix = "[]"
|
||||
} else if p.peek() == "*" {
|
||||
p.next()
|
||||
prefix = "*"
|
||||
}
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, p.expect(")")
|
||||
return prefix + name, p.expect(")")
|
||||
}
|
||||
|
||||
func joinComment(a, b string) string {
|
||||
if a == "" {
|
||||
return b
|
||||
}
|
||||
if b == "" {
|
||||
return a
|
||||
}
|
||||
return a + "\n" + b
|
||||
}
|
||||
func cleanComment(s string) string {
|
||||
if strings.HasPrefix(s, "//") {
|
||||
return strings.TrimSpace(strings.TrimPrefix(s, "//"))
|
||||
}
|
||||
s = strings.TrimSuffix(strings.TrimPrefix(s, "/*"), "*/")
|
||||
lines := strings.Split(s, "\n")
|
||||
for i, line := range lines {
|
||||
lines[i] = strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "*"))
|
||||
}
|
||||
return strings.TrimSpace(strings.Join(lines, "\n"))
|
||||
}
|
||||
func fieldTag(f Field) string {
|
||||
tags, err := contract.ParseTags(f.Tag)
|
||||
if err != nil || f.Description == "" || tags["description"] != "" {
|
||||
return f.Tag
|
||||
}
|
||||
return f.Tag + " description:" + strconv.Quote(f.Description)
|
||||
}
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"mime"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -16,6 +18,21 @@ func (p *parser) parseBody(r *Route) error {
|
||||
}
|
||||
for key, value := range props {
|
||||
switch key {
|
||||
case "maxParts":
|
||||
r.MaxParts, err = strconv.Atoi(value)
|
||||
if err != nil || r.MaxParts < 1 || r.MaxParts > 10000 {
|
||||
return p.err("maxParts must be 1..10000")
|
||||
}
|
||||
case "maxFieldBytes", "maxFileBytes":
|
||||
limit, e := strconv.ParseInt(value, 10, 64)
|
||||
if e != nil || limit < 1 || limit > 1<<30 {
|
||||
return p.err("part limits must be 1..1073741824 bytes")
|
||||
}
|
||||
if key == "maxFieldBytes" {
|
||||
r.MaxFieldBytes = limit
|
||||
} else {
|
||||
r.MaxFileBytes = limit
|
||||
}
|
||||
case "mode":
|
||||
r.BodyMode = value
|
||||
case "maxBytes":
|
||||
@@ -33,6 +50,16 @@ func (p *parser) parseBody(r *Route) error {
|
||||
return nil
|
||||
}
|
||||
func validateRouteExtensions(r Route) error {
|
||||
if (r.MaxParts > 0 || r.MaxFieldBytes > 0 || r.MaxFileBytes > 0) && r.BodyMode != "multipart" {
|
||||
return errors.New("part limits require multipart mode")
|
||||
}
|
||||
|
||||
if err := r.RequestPolicy.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if (r.BodyMode == "raw" || r.BodyMode == "multipart") && r.RequestPolicy.Body != "" && r.RequestPolicy.Body != "auto" {
|
||||
return errors.New("raw/multipart cannot declare a JSON body policy")
|
||||
}
|
||||
if r.BodyMode != "" && r.BodyMode != "json" && r.BodyMode != "raw" && r.BodyMode != "multipart" {
|
||||
return errors.New("body mode must be json, raw or multipart")
|
||||
}
|
||||
@@ -42,6 +69,9 @@ func validateRouteExtensions(r Route) error {
|
||||
if r.MaxBodyBytes > 0 && (r.Method == "get" || r.Method == "head") {
|
||||
return errors.New("GET/HEAD cannot declare a body limit")
|
||||
}
|
||||
if (r.Method == "get" || r.Method == "head") && r.RequestPolicy.Body != "" && r.RequestPolicy.Body != "auto" && r.RequestPolicy.Body != "none" {
|
||||
return errors.New("GET/HEAD cannot declare a JSON body policy")
|
||||
}
|
||||
if r.Produces != "" {
|
||||
media, params, err := mime.ParseMediaType(r.Produces)
|
||||
if err != nil || len(params) != 0 || media != r.Produces || strings.Contains(media, "*") {
|
||||
@@ -58,6 +88,23 @@ func validateRouteExtensions(r Route) error {
|
||||
}
|
||||
|
||||
func decorateRequestOptions(op map[string]any, r Route) {
|
||||
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
|
||||
r.RequestPolicy.Body = "auto"
|
||||
}
|
||||
op["x-omni-request-policy"] = r.RequestPolicy
|
||||
if r.BodyMode == "multipart" {
|
||||
limits, _ := (contract.MultipartOptions{MaxBodyBytes: r.MaxBodyBytes, MaxParts: r.MaxParts, MaxFieldBytes: r.MaxFieldBytes, MaxFileBytes: r.MaxFileBytes}).Normalized()
|
||||
op["x-omni-multipart-limits"] = map[string]any{"maxBytes": limits.MaxBodyBytes, "maxParts": limits.MaxParts, "maxFieldBytes": limits.MaxFieldBytes, "maxFileBytes": limits.MaxFileBytes}
|
||||
}
|
||||
if r.BodyMode != "raw" && r.BodyMode != "multipart" {
|
||||
if body, ok := op["requestBody"].(map[string]any); ok && r.RequestPolicy.Body == "optional" {
|
||||
body["required"] = false
|
||||
}
|
||||
if _, ok := op["requestBody"]; !ok && (r.RequestPolicy.Body == "emptyObject" || r.RequestPolicy.Body == "required" || r.RequestPolicy.Body == "optional") {
|
||||
op["requestBody"] = map[string]any{"required": r.RequestPolicy.Body != "optional", "content": map[string]any{"application/json": map[string]any{"schema": map[string]any{"type": "object", "additionalProperties": false}}}}
|
||||
}
|
||||
}
|
||||
|
||||
if r.Manual {
|
||||
op["x-omni-handler-mode"] = "manual"
|
||||
}
|
||||
@@ -87,3 +134,21 @@ func decorateRequestOptions(op map[string]any, r Route) {
|
||||
op["responses"].(map[string]any)[strconv.Itoa(r.Status)] = map[string]any{"description": "Success", "content": map[string]any{r.Produces: map[string]any{"schema": schema}}}
|
||||
}
|
||||
}
|
||||
|
||||
func parseRequestPolicy(props map[string]string) (contract.RequestPolicy, error) {
|
||||
data, err := json.Marshal(props)
|
||||
if err != nil {
|
||||
return contract.RequestPolicy{}, err
|
||||
}
|
||||
return decodeRequestPolicy(string(data))
|
||||
}
|
||||
func decodeRequestPolicy(raw string) (contract.RequestPolicy, error) {
|
||||
var p contract.RequestPolicy
|
||||
if err := apierrors.DecodeCatalogJSON([]byte(raw), &p); err != nil {
|
||||
return p, fmt.Errorf("request policy: %w", err)
|
||||
}
|
||||
return p, p.Validate()
|
||||
}
|
||||
func policyLiteral(p contract.RequestPolicy) string {
|
||||
return fmt.Sprintf("contract.RequestPolicy{UnknownQuery:%q,UTF8:%q,Charset:%q,Body:%q}", p.UnknownQuery, p.UTF8, p.Charset, p.Body)
|
||||
}
|
||||
|
||||
54
cmd/go-omni/apigen/responseShape.go
Normal file
54
cmd/go-omni/apigen/responseShape.go
Normal file
@@ -0,0 +1,54 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Response roots deliberately allow named DTOs, []DTO, *DTO and null. Keeping
|
||||
// these four shapes explicit makes nil/empty behavior uniform across encoders.
|
||||
func responseShape(value string) (kind, name string) {
|
||||
if value == "null" {
|
||||
return "null", ""
|
||||
}
|
||||
if strings.HasPrefix(value, "[]") {
|
||||
return "array", strings.TrimPrefix(value, "[]")
|
||||
}
|
||||
if strings.HasPrefix(value, "*") {
|
||||
return "nullable", strings.TrimPrefix(value, "*")
|
||||
}
|
||||
return "object", value
|
||||
}
|
||||
func responseGoType(value string) string {
|
||||
kind, name := responseShape(value)
|
||||
if kind == "null" || value == "" {
|
||||
return ""
|
||||
}
|
||||
if kind == "array" {
|
||||
return "[]types." + name
|
||||
}
|
||||
return "*types." + name
|
||||
}
|
||||
func responseDataSchema(value string, bindings map[string]*contract.Binder) (*openapi.Schema, error) {
|
||||
kind, name := responseShape(value)
|
||||
if kind == "null" {
|
||||
return &openapi.Schema{Type: "object", Nullable: true, Enum: []any{nil}}, nil
|
||||
}
|
||||
b, ok := bindings[name]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, *Name or null", name)
|
||||
}
|
||||
s, err := b.ResponseSchema()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch kind {
|
||||
case "array":
|
||||
return &openapi.Schema{Type: "array", Items: s}, nil
|
||||
case "nullable":
|
||||
s.Nullable = true
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
150
cmd/go-omni/apigen/shared.go
Normal file
150
cmd/go-omni/apigen/shared.go
Normal file
@@ -0,0 +1,150 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const importsFile = "omni.imports.json"
|
||||
|
||||
// SharedRoot explicitly authorizes reads from one canonical local directory.
|
||||
// Version is a project-owned label; SHA256 optionally pins the imported source
|
||||
// set, not a remote tag. The CLI never downloads or executes shared contracts.
|
||||
type SharedRoot struct {
|
||||
Path string `json:"path"`
|
||||
Version string `json:"version,omitempty"`
|
||||
SHA256 string `json:"sha256,omitempty"`
|
||||
}
|
||||
type sharedConfig struct {
|
||||
Roots map[string]SharedRoot `json:"roots"`
|
||||
}
|
||||
|
||||
func readSharedRoots(root string) (map[string]SharedRoot, error) {
|
||||
stat, err := os.Lstat(filepath.Join(root, importsFile))
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !stat.Mode().IsRegular() {
|
||||
return nil, errors.New("api: shared root configuration must be a regular file")
|
||||
}
|
||||
f, err := os.Open(filepath.Join(root, importsFile))
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
raw, err := io.ReadAll(io.LimitReader(f, 65537))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(raw) > 65536 {
|
||||
return nil, errors.New("api: shared root config too large")
|
||||
}
|
||||
var cfg sharedConfig
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(cfg.Roots) > 16 {
|
||||
return nil, errors.New("api: at most 16 shared roots")
|
||||
}
|
||||
for name, r := range cfg.Roots {
|
||||
if !lowerName.MatchString(name) || r.Path == "" || len(r.Version) > 128 {
|
||||
return nil, errors.New("api: invalid shared root")
|
||||
}
|
||||
if r.SHA256 != "" && (len(r.SHA256) != 64 || strings.Trim(r.SHA256, "0123456789abcdef") != "") {
|
||||
return nil, errors.New("api: shared sha256 must be 64 lowercase hex characters")
|
||||
}
|
||||
path := r.Path
|
||||
if !filepath.IsAbs(path) {
|
||||
path = filepath.Join(root, path)
|
||||
}
|
||||
r.Path, err = filepath.EvalSymlinks(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
r.Path, err = filepath.Abs(r.Path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
stat, err := os.Stat(r.Path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !stat.IsDir() {
|
||||
return nil, errors.New("api: shared root must be a directory")
|
||||
}
|
||||
cfg.Roots[name] = r
|
||||
}
|
||||
return cfg.Roots, nil
|
||||
}
|
||||
func sourceWithin(root, name string) (string, error) {
|
||||
real, err := filepath.EvalSymlinks(name)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
rel, err := filepath.Rel(root, real)
|
||||
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return "", errors.New("api: imports must stay within their declared contract root")
|
||||
}
|
||||
return filepath.ToSlash(rel), nil
|
||||
}
|
||||
func sharedHash(sources map[string][]byte, alias string) string {
|
||||
values := map[string]string{}
|
||||
for name, data := range sources {
|
||||
if strings.HasPrefix(name, "@"+alias+"/") {
|
||||
values[strings.TrimPrefix(name, "@"+alias+"/")] = digest(data)
|
||||
}
|
||||
}
|
||||
raw, _ := json.Marshal(values)
|
||||
return digest(raw)
|
||||
}
|
||||
func sourceMetadata(spec *Spec) map[string]any {
|
||||
hashes := map[string]string{}
|
||||
for name, data := range spec.Sources {
|
||||
hashes[name] = digest(data)
|
||||
}
|
||||
shared := map[string]any{}
|
||||
for alias, r := range spec.SharedRoots {
|
||||
shared[alias] = map[string]string{"version": r.Version, "sha256": sharedHash(spec.Sources, alias)}
|
||||
}
|
||||
return map[string]any{"files": hashes, "shared": shared}
|
||||
}
|
||||
func sharedSeeds(spec *Spec, directory string) ([]byte, error) {
|
||||
cfg := sharedConfig{Roots: map[string]SharedRoot{}}
|
||||
for name, r := range spec.SharedRoots {
|
||||
rel, err := filepath.Rel(filepath.Join(directory, "api"), r.Path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
r.Path = filepath.ToSlash(rel)
|
||||
cfg.Roots[name] = r
|
||||
}
|
||||
data, err := json.MarshalIndent(cfg, "", " ")
|
||||
return append(data, '\n'), err
|
||||
}
|
||||
func validateSharedPins(spec *Spec) error {
|
||||
names := make([]string, 0, len(spec.SharedRoots))
|
||||
for name := range spec.SharedRoots {
|
||||
names = append(names, name)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, name := range names {
|
||||
r := spec.SharedRoots[name]
|
||||
if r.SHA256 != "" && r.SHA256 != sharedHash(spec.Sources, name) {
|
||||
return fmt.Errorf("api: shared contract %s checksum differs (got %s)", name, sharedHash(spec.Sources, name))
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
96
cmd/go-omni/apigen/shared_test.go
Normal file
96
cmd/go-omni/apigen/shared_test.go
Normal file
@@ -0,0 +1,96 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestControlledSharedContracts(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
shared := filepath.Join(root, "shared")
|
||||
writeFile(t, filepath.Join(shared, "response.api"), "response Common (codeType: int32; successCode: 0; errorCode: 9999; dataMode: always)\ntype Item { Name string `json:\"name\"` }\n")
|
||||
for _, service := range []string{"api", "id"} {
|
||||
dir := filepath.Join(root, service)
|
||||
entry := filepath.Join(dir, "api/service.api")
|
||||
writeFile(t, entry, "import \"@common/response.api\"\n@app(defaultResponse: Common)\nservice demo {\n@handler get\nget /get returns (Item)\n}\n")
|
||||
config := `{"roots":{"common":{"path":"../../shared","version":"1.0.0"}}}`
|
||||
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), config)
|
||||
spec, err := Parse(entry)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(spec.Sources) != 2 || len(spec.SourcePaths) != 2 {
|
||||
t.Fatalf("%+v", spec)
|
||||
}
|
||||
pin := sharedHash(spec.Sources, "common")
|
||||
config = strings.Replace(config, `"version":"1.0.0"`, `"version":"1.0.0","sha256":"`+pin+`"`, 1)
|
||||
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), config)
|
||||
o := Options{Dir: dir, API: entry, Module: "example.com/" + service, FrameworkDir: frameworkRoot(t)}
|
||||
if _, err = Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if _, err = Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(dir, "api/@common")); !os.IsNotExist(err) {
|
||||
t.Fatal("shared contract copied into service")
|
||||
}
|
||||
document := readFile(t, filepath.Join(dir, "docs/openapi.json"))
|
||||
if !strings.Contains(document, pin) || !strings.Contains(document, "@common/response.api") {
|
||||
t.Fatal("missing provenance")
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), strings.Replace(config, pin, strings.Repeat("0", 64), 1))
|
||||
if _, err = Parse(entry); err == nil {
|
||||
t.Fatal("checksum mismatch accepted")
|
||||
}
|
||||
}
|
||||
// An undeclared alias, traversal, symlink escape and cycle are all rejected.
|
||||
for _, imp := range []string{"@missing/response.api", "@common/../outside.api", "../shared/response.api", "@common/escape.api", "@common/cycle.api"} {
|
||||
local := filepath.Join(root, "consumer")
|
||||
writeFile(t, filepath.Join(local, "omni.imports.json"), `{"roots":{"common":{"path":"../shared"}}}`)
|
||||
writeFile(t, filepath.Join(root, "outside.api"), "type Outside {}\n")
|
||||
_ = os.Symlink(filepath.Join(root, "outside.api"), filepath.Join(shared, "escape.api"))
|
||||
writeFile(t, filepath.Join(shared, "cycle.api"), "import \"cycle.api\"\n")
|
||||
entry := filepath.Join(local, "service.api")
|
||||
writeFile(t, entry, "import "+strconvQuote(imp)+"\nservice demo{\n@handler get\nget /get\n}\n")
|
||||
if _, err := Parse(entry); err == nil {
|
||||
t.Fatal("unsafe import accepted", imp)
|
||||
}
|
||||
}
|
||||
}
|
||||
func strconvQuote(value string) string { data, _ := json.Marshal(value); return string(data) }
|
||||
func TestCatalogDocumentExportDoesNotGenerateRuntimeRegistry(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
entry := filepath.Join(dir, "service.api")
|
||||
writeFile(t, entry, "@app(defaultResponse: Common)\nresponse Common(codeType: int32; successCode: 0; errorCode: 9999; messageField: msg; dataMode: always)\nservice demo{\n@handler get\nget /get returns (null)\n}\n")
|
||||
var exported bytes.Buffer
|
||||
err := apierrors.ExportCatalog(&exported, apierrors.Catalog{DefaultStatus: 200, Definitions: map[string]apierrors.Definition{"TOO_LONG": {Code: 400002, Message: "{label}不能超过{max}个字符", Params: []apierrors.Parameter{{Name: "label", Type: "string", Example: "企业名称"}, {Name: "max", Type: "int", Example: 100}}}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "omni.errors.json"), exported.String())
|
||||
spec, err := Parse(entry)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(c.Spec.Errors) != 0 || !strings.Contains(string(c.Document), "企业名称不能超过100个字符") {
|
||||
t.Fatal(string(c.Document))
|
||||
}
|
||||
target := filepath.Join(t.TempDir(), "catalog")
|
||||
if _, err = Generate(Options{Dir: target, API: entry, Module: "example.com/catalog", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(target, "internal/apierrors/errorsGen.go")); !os.IsNotExist(err) {
|
||||
t.Fatal("document metadata became a runtime error registry")
|
||||
}
|
||||
}
|
||||
@@ -20,8 +20,11 @@ import (
|
||||
func NewApp(deps *svc.ServiceContext)(*omni.App,error){
|
||||
if deps==nil || deps.Lifecycle==nil{return nil,fmt.Errorf("service context and lifecycle controller required; see PROJECT.md upgrade instructions")}
|
||||
cfg:=deps.Config.Server
|
||||
cfg.LogInstance=deps.Logger
|
||||
for _,option:=range deps.AppOptions(){if option!=nil{option(&cfg)}}
|
||||
`, frameworkModule, frameworkModule+"/httpx", frameworkModule+"/config", frameworkModule+"/health", o.Module+"/internal/svc")
|
||||
source += fmt.Sprintf("cfg.Request=contract.MergePolicy(cfg.Request,%#v)\n", spec.RequestPolicy)
|
||||
source = strings.Replace(source, "\"fmt\"", "\"fmt\";"+fmt.Sprintf("%q", frameworkModule+"/contract"), 1)
|
||||
if spec.DefaultResponse != "" {
|
||||
for _, r := range spec.Responses {
|
||||
if r.Name == spec.DefaultResponse {
|
||||
|
||||
@@ -28,14 +28,10 @@ func run(ctx context.Context,path string)(runErr error){
|
||||
cfg,err:=config.Load(path);if err!=nil{return err}
|
||||
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||
logx.SetDefault(logger)
|
||||
deps,err:=svc.New(ctx,logger,cfg);if err!=nil{return err}
|
||||
// RunContext drains HTTP requests before returning. This scope owns dependencies
|
||||
// and closes them exactly once, before closing the borrowed logger.
|
||||
defer func(){
|
||||
runErr=errors.Join(runErr,deps.Close())
|
||||
if cause:=deps.Lifecycle.Err();cause!=nil && !errors.Is(runErr,cause){runErr=errors.Join(runErr,cause)}
|
||||
}()
|
||||
defer func(){runErr=deps.Finish(runErr)}()
|
||||
app,err:=handler.NewApp(deps);if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,app.Close())}()
|
||||
logger.InfoContext(ctx,"service.starting","address",app.Config().Address)
|
||||
@@ -111,6 +107,16 @@ and produce REVIEW notices. This source scan does not execute project code or
|
||||
replace go test ./... . Snapshot-based REVIEW notices describe changes since the
|
||||
last successful generation; they do not persist as an acknowledgement checklist.
|
||||
|
||||
## Contract tools
|
||||
|
||||
api format [--check] preserves comments and normalizes indentation. api upgrade
|
||||
previews changed infrastructure; --apply only replaces recognized prior seeds.
|
||||
Use @request policy overrides and @app(request: {...}) for query/JSON strictness.
|
||||
Explicitly exported api/omni.errors.json documents the Go-owned error catalog;
|
||||
the CLI never executes project Go. omni.imports.json permits named shared roots;
|
||||
workspace format/generate/check processes explicitly listed independent services.
|
||||
Full reference: https://git.yuntuo.hk/k2safe/go-omni/src/branch/main/DEVELOPMENT.md
|
||||
|
||||
## Layout
|
||||
|
||||
`+"```text"+`
|
||||
@@ -119,7 +125,8 @@ main.go # GENERATED startup, lifecycle and route registrati
|
||||
etc/config.yaml # lowerCamelCase configuration
|
||||
internal/config/ # defaults -> file -> environment
|
||||
internal/svc/ # dependency initialization and cleanup
|
||||
internal/types/typesGen.go # generated request/response types
|
||||
internal/types/*Gen.go # DTOs split by source API, one types package
|
||||
internal/svc/runtimeGen.go # GENERATED lifecycle and infrastructure startup
|
||||
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
|
||||
internal/handler/<group>/ # generated binding and Logic adapters
|
||||
internal/logic/<group>/ # developer-owned business operations
|
||||
@@ -138,7 +145,7 @@ use the same contract. 404/405 and ordinary App routes use the declared App defa
|
||||
No template is selected by declaration/import order. Without @app, existing framework
|
||||
response defaults apply. An explicit API App default takes precedence over response
|
||||
options in svc, keeping generated routes and OpenAPI consistent.
|
||||
Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default.
|
||||
Object responses reject nil; []DTO maps nil to []; *DTO permits null; returns(null) is a successful null value. No returns means 204. Body DTOs use application/json and have a 1 MiB limit by default.
|
||||
Use @body(maxBytes: ...) for route limits. @body(mode: raw|multipart; maxBytes: ...)
|
||||
adds a *httpx.Body argument to Logic, without handwritten handlers. Raw is the exact
|
||||
limited request stream; Parts supports sequential multipart reading with no temp files.
|
||||
@@ -146,11 +153,10 @@ limited request stream; Parts supports sequential multipart reading with no temp
|
||||
Logic result of *httpx.Stream; the handler closes Content and streams it unwrapped.
|
||||
Signature verification and storage policy belong to Logic/middleware. A consumed
|
||||
request stream cannot be reread automatically; use one reader owner or explicitly
|
||||
bounded buffering. Stream reads respect configured server timeouts. Multipart parts
|
||||
are not DTO form fields: form tags remain URL query parameters. The optional manual
|
||||
bounded buffering. Stream reads respect configured server timeouts. Declare multipart/multipartJSON/file DTO fields for validated uploads; without those fields the existing streaming Body is passed. form tags remain URL query parameters. The optional manual
|
||||
handler/custom binder hooks are preserved, and are needed only for custom protocols.
|
||||
|
||||
Fields require exactly one source tag: json, path, form (URL query only), header.
|
||||
Fields require one source tag: json, path, form (URL query), header, multipart, multipartJSON or file. Multipart fields select full automatic binding before Logic; File/[]File are request-scoped disk-backed uploads. Use File.Open to stream them; generated cleanup owns their lifetime.
|
||||
All are required by presence unless optional, omitempty or default is specified.
|
||||
Zero and false are valid present values; binding:"required" rejects scalar zero.
|
||||
Use *T with optional when business logic needs to distinguish missing from zero.
|
||||
@@ -171,7 +177,7 @@ The selected auth runs first; @auth none removes inherited declarative auth.
|
||||
Validation policy remains in the middleware implementation. Public messages must
|
||||
not contain secrets; error causes may be included in server logs.
|
||||
|
||||
svc.New receives the loaded Config and borrowed Logger. Add dependencies there,
|
||||
Generated svc.New receives Config and a borrowed Logger. Add dependencies in svc.Init,
|
||||
then immediately register each cleanup with OnClose. A later initialization failure
|
||||
runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware
|
||||
provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated
|
||||
@@ -208,7 +214,7 @@ func infrastructureREADME(o Options) string {
|
||||
|
||||
Selected database: %s; storage: %s; logging: %s/%s.
|
||||
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
|
||||
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||
Use structured database/mysql configuration and GO_OMNI_DB_PASSWORD. Other connection/pool/TLS settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||
No tables are invented or migrated automatically from request/response DTOs.
|
||||
The none/memory selection opens no database; implement repositories per business.
|
||||
|
||||
|
||||
22
cmd/go-omni/apigen/testdata/business/accessFactory.go
vendored
Normal file
22
cmd/go-omni/apigen/testdata/business/accessFactory.go
vendored
Normal file
@@ -0,0 +1,22 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
// This package deliberately imports no svc; the dependency is supplied explicitly.
|
||||
type AccessChecker interface {
|
||||
CheckAccess(context.Context, string) error
|
||||
}
|
||||
|
||||
func NewRequireAccess(checker AccessChecker) httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
|
||||
return err
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
32
cmd/go-omni/apigen/testdata/business/client.go
vendored
Normal file
32
cmd/go-omni/apigen/testdata/business/client.go
vendored
Normal file
@@ -0,0 +1,32 @@
|
||||
package svc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"example.com/business/internal/middleware"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"sync/atomic"
|
||||
)
|
||||
|
||||
// A local fixture checks wiring; production authentication is project-owned.
|
||||
type accessClient struct{ closed atomic.Bool }
|
||||
|
||||
func (c *accessClient) Close() error { c.closed.Store(true); return nil }
|
||||
func (c *accessClient) CheckAccess(ctx context.Context, token string) error {
|
||||
if c.closed.Load() {
|
||||
return apierrors.New(503, "CLOSED", "closed")
|
||||
}
|
||||
if httpx.Metadata(ctx).Route == "" {
|
||||
return apierrors.New(500, "NO_METADATA", "missing route")
|
||||
}
|
||||
if token == "reject" {
|
||||
return apierrors.New(403, "DENIED", "denied")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (c *ServiceContext) initAccess(ctx context.Context) error {
|
||||
c.Client = &accessClient{}
|
||||
c.OnClose(c.Client.Close)
|
||||
c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(c.Client)
|
||||
return nil
|
||||
}
|
||||
119
cmd/go-omni/apigen/testdata/business/consumer_test.go
vendored
Normal file
119
cmd/go-omni/apigen/testdata/business/consumer_test.go
vendored
Normal file
@@ -0,0 +1,119 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"example.com/business/internal/config"
|
||||
"example.com/business/internal/handler"
|
||||
"example.com/business/internal/svc"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestShapesAndPolicy(t *testing.T) {
|
||||
cfg := config.Default()
|
||||
cfg.Server.Log.Writer = io.Discard
|
||||
logger, err := logx.New(cfg.Server.Log)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer logger.Close()
|
||||
deps, err := svc.New(context.Background(), logger, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer deps.Close()
|
||||
app, err := handler.NewApp(deps)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer app.Close()
|
||||
for _, tt := range []struct {
|
||||
method, path, body, media string
|
||||
status int
|
||||
data string
|
||||
raw bool
|
||||
}{
|
||||
{"GET", "/list", "", "", 200, `[]`, false},
|
||||
{"GET", "/list?mode=full&tags=a&tags=b", "", "", 200, `[{"name":"one"}]`, false},
|
||||
{"GET", "/nullable", "", "", 200, `null`, false},
|
||||
{"GET", "/nullable?mode=full", "", "", 200, `{"name":"one"}`, false},
|
||||
{"POST", "/nothing", "", "", 200, `null`, false},
|
||||
{"GET", "/rawNull", "", "", 200, `null`, true},
|
||||
{"GET", "/rawArray", "", "", 200, `[]`, true},
|
||||
{"DELETE", "/noContent", "", "", 204, "", true},
|
||||
{"POST", "/empty", `{}`, "application/json", 200, `null`, false},
|
||||
{"POST", "/empty", ``, "application/json", 400, "", false},
|
||||
{"POST", "/empty", `{"unknown":1}`, "application/json", 400, "", false},
|
||||
{"POST", "/absent", ``, "", 200, `null`, false},
|
||||
{"POST", "/absent", `{}`, "application/json", 400, "", false},
|
||||
{"POST", "/optional", ``, "", 200, `{"name":""}`, false},
|
||||
{"POST", "/optional", `{}`, "application/json", 400, "", false},
|
||||
{"POST", "/echo", `{"name":"ok"}`, "application/json", 200, `{"name":"ok"}`, false},
|
||||
{"POST", "/echo", `{"name":"ok"}`, "application/json; charset=latin1", 415, "", false},
|
||||
{"POST", "/echo", "{\"name\":\"\xff\"}", "application/json", 400, "", false},
|
||||
{"GET", "/list?unknown=1", "", "", 400, "", false},
|
||||
{"GET", "/rawNull?unknown=1", "", "", 400, "", false},
|
||||
{"POST", "/tolerant?unknown=1", "{\"name\":\"\xff\"}", "application/json; charset=latin1", 200, `{"name":"<22>"}`, false},
|
||||
} {
|
||||
t.Run(tt.method+tt.path+tt.body+tt.media, func(t *testing.T) {
|
||||
r := httptest.NewRequest(tt.method, tt.path, strings.NewReader(tt.body))
|
||||
if tt.media != "" {
|
||||
r.Header.Set("Content-Type", tt.media)
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
app.ServeHTTP(w, r)
|
||||
if w.Code != tt.status {
|
||||
t.Fatalf("%d %s", w.Code, w.Body)
|
||||
}
|
||||
if tt.status >= 400 {
|
||||
return
|
||||
}
|
||||
data := strings.TrimSpace(w.Body.String())
|
||||
if !tt.raw {
|
||||
var obj map[string]json.RawMessage
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &obj); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v, ok := obj["data"]
|
||||
if !ok {
|
||||
t.Fatal("missing data")
|
||||
}
|
||||
data = string(v)
|
||||
}
|
||||
if data != tt.data {
|
||||
t.Fatalf("data %s expected %s", data, tt.data)
|
||||
}
|
||||
})
|
||||
}
|
||||
var body bytes.Buffer
|
||||
form := multipart.NewWriter(&body)
|
||||
_ = form.WriteField("title", "title")
|
||||
_ = form.WriteField("metadata", `{"name":"meta"}`)
|
||||
f, err := form.CreateFormFile("attachment", "a.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, _ = io.WriteString(f, "content")
|
||||
_ = form.Close()
|
||||
r := httptest.NewRequest("POST", "/upload", &body)
|
||||
r.Header.Set("Content-Type", form.FormDataContentType())
|
||||
w := httptest.NewRecorder()
|
||||
app.ServeHTTP(w, r)
|
||||
if w.Code != 200 || !strings.Contains(w.Body.String(), "title:meta:content") {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
denied := httptest.NewRequest("GET", "/list", nil)
|
||||
denied.Header.Set("Authorization", "reject")
|
||||
result := httptest.NewRecorder()
|
||||
app.ServeHTTP(result, denied)
|
||||
if result.Code != 403 {
|
||||
t.Fatal("injected middleware did not run", result.Code, result.Body.String())
|
||||
}
|
||||
|
||||
}
|
||||
42
cmd/go-omni/apigen/testdata/business/operations.go
vendored
Normal file
42
cmd/go-omni/apigen/testdata/business/operations.go
vendored
Normal file
@@ -0,0 +1,42 @@
|
||||
package demo
|
||||
|
||||
import (
|
||||
"context"
|
||||
"example.com/business/internal/types"
|
||||
"io"
|
||||
)
|
||||
|
||||
func (s *Logic) List(ctx context.Context, req *types.Query) ([]types.Item, error) {
|
||||
if req.Mode == "full" {
|
||||
return []types.Item{{Name: "one"}}, nil
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
func (s *Logic) Nullable(ctx context.Context, req *types.Query) (*types.Item, error) {
|
||||
if req.Mode == "full" {
|
||||
return &types.Item{Name: "one"}, nil
|
||||
}
|
||||
return nil, nil
|
||||
}
|
||||
func (s *Logic) Nothing(ctx context.Context) error { return nil }
|
||||
func (s *Logic) RawNull(ctx context.Context) error { return nil }
|
||||
func (s *Logic) RawArray(ctx context.Context) ([]types.Item, error) { return nil, nil }
|
||||
func (s *Logic) Empty(ctx context.Context) error { return nil }
|
||||
func (s *Logic) Absent(ctx context.Context) error { return nil }
|
||||
func (s *Logic) Optional(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil }
|
||||
func (s *Logic) Echo(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil }
|
||||
func (s *Logic) Tolerant(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil }
|
||||
func (s *Logic) NoContent(ctx context.Context) error { return nil }
|
||||
|
||||
func (s *Logic) Upload(ctx context.Context, req *types.Upload) (*types.Item, error) {
|
||||
f, err := req.Attachment.Open()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(f)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &types.Item{Name: req.Title + ":" + req.Metadata.Name + ":" + string(data)}, nil
|
||||
}
|
||||
296
cmd/go-omni/apigen/upgrade.go
Normal file
296
cmd/go-omni/apigen/upgrade.go
Normal file
@@ -0,0 +1,296 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||
"go/ast"
|
||||
"go/format"
|
||||
goparser "go/parser"
|
||||
gotoken "go/token"
|
||||
"golang.org/x/mod/modfile"
|
||||
"golang.org/x/mod/semver"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type UpgradeChange struct {
|
||||
Path, Action string
|
||||
Manual bool
|
||||
before, after []byte
|
||||
mode os.FileMode
|
||||
exists bool
|
||||
}
|
||||
type UpgradePlan struct {
|
||||
Changes []UpgradeChange
|
||||
Module, Entry string
|
||||
}
|
||||
|
||||
// PlanUpgrade compares infrastructure against the exact previous release seeds.
|
||||
// A custom file is never classified as safe to replace. Logic, middleware,
|
||||
// options and repositories outside the changed templates are left untouched.
|
||||
func PlanUpgrade(dir string) (UpgradePlan, error) {
|
||||
var plan UpgradePlan
|
||||
root := canonical(dir)
|
||||
if err := safePath(root, manifestPath); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(root, manifestPath))
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
var m manifest
|
||||
if err = json.Unmarshal(raw, &m); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
if m.Version != 2 || m.Profile == nil {
|
||||
return plan, errors.New("upgrade requires a managed v0.13+ project")
|
||||
}
|
||||
if filepath.Ext(m.Entry) != ".api" {
|
||||
return plan, errors.New("upgrade requires a valid API entry")
|
||||
}
|
||||
if err = safePath(root, m.Entry); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
plan.Module, plan.Entry = m.Module, m.Entry
|
||||
if semver.IsValid(m.TemplateVersion) && semver.Compare(m.TemplateVersion, minimumVersion) >= 0 {
|
||||
return plan, nil
|
||||
}
|
||||
p := m.Profile
|
||||
legacy, err := scaffold.LegacyAPIInfrastructure(m.Module, p.Database, p.Storage, p.LogOutput, p.LogFormat)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
current, err := scaffold.APIInfrastructure(m.Module, p.Database, p.Storage, p.LogOutput, p.LogFormat)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
names := map[string]bool{}
|
||||
for name := range legacy {
|
||||
names[name] = true
|
||||
}
|
||||
for name := range current {
|
||||
if !strings.HasSuffix(name, "Gen.go") {
|
||||
names[name] = true
|
||||
}
|
||||
}
|
||||
sorted := make([]string, 0, len(names))
|
||||
for name := range names {
|
||||
sorted = append(sorted, name)
|
||||
}
|
||||
sort.Strings(sorted)
|
||||
for _, name := range sorted {
|
||||
// options.go is unchanged by this release. Its application customizations remain untouched.
|
||||
previous, next := legacy[name], current[name]
|
||||
if strings.HasSuffix(name, ".go") {
|
||||
if previous != nil {
|
||||
previous, err = format.Source(previous)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
}
|
||||
if next != nil {
|
||||
next, err = format.Source(next)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
}
|
||||
}
|
||||
if bytes.Equal(previous, next) {
|
||||
continue
|
||||
}
|
||||
if err = safePath(root, name); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
existing, readErr := os.ReadFile(filepath.Join(root, name))
|
||||
if readErr != nil && !errors.Is(readErr, os.ErrNotExist) {
|
||||
return plan, readErr
|
||||
}
|
||||
exists := readErr == nil
|
||||
if bytes.Equal(existing, next) {
|
||||
continue
|
||||
}
|
||||
mode := os.FileMode(0644)
|
||||
if exists {
|
||||
stat, e := os.Stat(filepath.Join(root, name))
|
||||
if e != nil {
|
||||
return plan, e
|
||||
}
|
||||
mode = stat.Mode().Perm()
|
||||
}
|
||||
action := "update"
|
||||
if next == nil {
|
||||
action = "remove"
|
||||
} else if !exists {
|
||||
action = "create"
|
||||
}
|
||||
plan.Changes = append(plan.Changes, UpgradeChange{Path: name, Action: action, Manual: exists && !bytes.Equal(existing, previous), before: existing, after: next, mode: mode, exists: exists})
|
||||
}
|
||||
if err = safePath(root, "go.mod"); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
raw, err = os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
mf, err := modfile.Parse("go.mod", raw, nil)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
for _, r := range mf.Require {
|
||||
if r.Mod.Path == frameworkModule && semver.Compare(r.Mod.Version, minimumVersion) < 0 {
|
||||
if err = mf.AddRequire(frameworkModule, minimumVersion); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
next, e := mf.Format()
|
||||
if e != nil {
|
||||
return plan, e
|
||||
}
|
||||
stat, e := os.Stat(filepath.Join(root, "go.mod"))
|
||||
if e != nil {
|
||||
return plan, e
|
||||
}
|
||||
plan.Changes = append(plan.Changes, UpgradeChange{Path: "go.mod", Action: "update", before: raw, after: next, exists: true, mode: stat.Mode().Perm()})
|
||||
break
|
||||
}
|
||||
}
|
||||
return plan, nil
|
||||
}
|
||||
|
||||
// PreviewUpgrade writes only proposed infrastructure files into an empty directory.
|
||||
// It never mutates the service or emits a replacement business tree.
|
||||
func PreviewUpgrade(dir, destination string) (UpgradePlan, error) {
|
||||
plan, err := PlanUpgrade(dir)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
if entries, e := os.ReadDir(destination); e == nil && len(entries) > 0 {
|
||||
return plan, errors.New("upgrade preview directory must be empty")
|
||||
} else if e != nil && !errors.Is(e, os.ErrNotExist) {
|
||||
return plan, e
|
||||
}
|
||||
for _, c := range plan.Changes {
|
||||
if c.after == nil {
|
||||
continue
|
||||
}
|
||||
if err = safePath(destination, c.Path); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
if err = atomicWrite(destination, c.Path, c.after, 0644); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
}
|
||||
note := "# Infrastructure upgrade preview\n\nOnly untouched v0.13 seed files can be replaced automatically. Custom files require a targeted edit; this preview is never copied over the project automatically.\n\nMove dependency initialization into ServiceContext.Init; embed *lifecycle.Runtime; register cleanup with OnClose. Remove old New/Close/OnClose implementations after preserving their project work in Init or cleanup hooks. Use structured database/mysql.Config and DatabaseOptions. Preserve business configuration fields, AppOptions, middleware and Logic.\n\n"
|
||||
for _, c := range plan.Changes {
|
||||
note += fmt.Sprintf("- %s %s (manual review: %t)\n", c.Action, c.Path, c.Manual)
|
||||
}
|
||||
if err = safePath(destination, "UPGRADE.md"); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
return plan, atomicWrite(destination, "UPGRADE.md", []byte(note), 0644)
|
||||
}
|
||||
|
||||
// ApplyUpgrade applies only recognized seed changes under the same generation
|
||||
// lock. A generation failure restores every changed developer file. Any manual
|
||||
// review item blocks all writes; use PreviewUpgrade and migrate that file first.
|
||||
func ApplyUpgrade(dir string) (plan UpgradePlan, err error) {
|
||||
root := canonical(dir)
|
||||
if err = safePath(root, ".goOmni/api.lock"); err != nil {
|
||||
return plan, err
|
||||
}
|
||||
lock, err := os.OpenFile(filepath.Join(root, ".goOmni/api.lock"), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
_ = lock.Close()
|
||||
defer os.Remove(filepath.Join(root, ".goOmni/api.lock"))
|
||||
plan, err = PlanUpgrade(root)
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
for _, c := range plan.Changes {
|
||||
if c.Manual {
|
||||
return plan, fmt.Errorf("upgrade: %s contains project changes; inspect an upgrade preview and migrate it explicitly", c.Path)
|
||||
}
|
||||
}
|
||||
var changed []UpgradeChange
|
||||
defer func() {
|
||||
if err != nil {
|
||||
for i := len(changed) - 1; i >= 0; i-- {
|
||||
c := changed[i]
|
||||
if c.exists {
|
||||
err = errors.Join(err, atomicWrite(root, c.Path, c.before, c.mode))
|
||||
} else {
|
||||
e := os.Remove(filepath.Join(root, c.Path))
|
||||
if !errors.Is(e, os.ErrNotExist) {
|
||||
err = errors.Join(err, e)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}()
|
||||
for _, c := range plan.Changes {
|
||||
if c.after == nil {
|
||||
err = os.Remove(filepath.Join(root, c.Path))
|
||||
} else {
|
||||
err = atomicWrite(root, c.Path, c.after, c.mode)
|
||||
}
|
||||
if err != nil {
|
||||
return plan, err
|
||||
}
|
||||
changed = append(changed, c)
|
||||
}
|
||||
preview, e := Generate(Options{Dir: root, API: filepath.Join(root, plan.Entry), DryRun: true, lockHeld: true})
|
||||
if e != nil {
|
||||
return plan, e
|
||||
}
|
||||
if diagnosticErrors(preview.Diagnostics) > 0 {
|
||||
return plan, errors.New("upgrade: Logic checks failed; resolve diagnostics before applying")
|
||||
}
|
||||
_, err = Generate(Options{Dir: root, API: filepath.Join(root, plan.Entry), lockHeld: true})
|
||||
return plan, err
|
||||
}
|
||||
|
||||
func checkRuntimeExtension(root string) error {
|
||||
directory := filepath.Join(root, "internal/svc")
|
||||
files, err := os.ReadDir(directory)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range files {
|
||||
if !sourceFile(entry.Name()) || strings.HasSuffix(entry.Name(), "Gen.go") {
|
||||
continue
|
||||
}
|
||||
path := filepath.Join(directory, entry.Name())
|
||||
file, err := goparser.ParseFile(gotoken.NewFileSet(), path, nil, goparser.SkipObjectResolution)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, decl := range file.Decls {
|
||||
fn, ok := decl.(*ast.FuncDecl)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
if fn.Recv == nil && fn.Name.Name == "New" {
|
||||
return fmt.Errorf("%s owns New; run go-omni api upgrade --dir %s and move project initialization into ServiceContext.Init", path, root)
|
||||
}
|
||||
if fn.Recv != nil && (fn.Name.Name == "Close" || fn.Name.Name == "OnClose" || fn.Name.Name == "Finish") {
|
||||
receiver := fn.Recv.List[0].Type
|
||||
if p, ok := receiver.(*ast.StarExpr); ok {
|
||||
receiver = p.X
|
||||
}
|
||||
if id, ok := receiver.(*ast.Ident); ok && id.Name == "ServiceContext" {
|
||||
return fmt.Errorf("%s overrides framework resource lifecycle; inspect go-omni api upgrade and register cleanup in Init", path)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
124
cmd/go-omni/apigen/upgrade_test.go
Normal file
124
cmd/go-omni/apigen/upgrade_test.go
Normal file
@@ -0,0 +1,124 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||
"go/format"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func oldInfrastructureProject(t *testing.T) string {
|
||||
t.Helper()
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
previous, err := scaffold.LegacyAPIInfrastructure("example.com/upgrade", "none", "none", "stderr", "json")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for name, content := range previous {
|
||||
if strings.HasSuffix(name, ".go") {
|
||||
content, err = format.Source(content)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, name), string(content))
|
||||
}
|
||||
var m manifest
|
||||
if err = json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
delete(m.Generated, "internal/svc/runtimeGen.go")
|
||||
m.TemplateVersion = ""
|
||||
_ = os.Remove(filepath.Join(dir, "internal/svc/runtimeGen.go"))
|
||||
raw, _ := json.Marshal(m)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
path := filepath.Join(dir, "go.mod")
|
||||
writeFile(t, path, strings.ReplaceAll(readFile(t, path), minimumVersion, "v0.13.0-preview"))
|
||||
return dir
|
||||
}
|
||||
|
||||
func TestUpgradeRejectsEscapingEntry(t *testing.T) {
|
||||
dir := oldInfrastructureProject(t)
|
||||
var m manifest
|
||||
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Entry = "../outside.api"
|
||||
raw, _ := json.Marshal(m)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
before := snapshot(t, dir)
|
||||
if _, err := ApplyUpgrade(dir); err == nil {
|
||||
t.Fatal("escaping API entry accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("invalid upgrade wrote project")
|
||||
}
|
||||
}
|
||||
func TestSafeInfrastructureUpgrade(t *testing.T) {
|
||||
dir := oldInfrastructureProject(t)
|
||||
before := snapshot(t, dir)
|
||||
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}); err == nil {
|
||||
t.Fatal("legacy constructor accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("failed generate wrote files")
|
||||
}
|
||||
plan, err := PlanUpgrade(dir)
|
||||
if err != nil || len(plan.Changes) == 0 {
|
||||
t.Fatal(plan, err)
|
||||
}
|
||||
for _, c := range plan.Changes {
|
||||
if c.Manual {
|
||||
t.Fatal(c.Path)
|
||||
}
|
||||
}
|
||||
preview := filepath.Join(t.TempDir(), "preview")
|
||||
if _, err = PreviewUpgrade(dir, preview); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("preview changed project")
|
||||
}
|
||||
logic := readFile(t, filepath.Join(dir, "internal/logic/greet/greet.go"))
|
||||
if _, err = ApplyUpgrade(dir); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(dir, "internal/svc/close.go")); !os.IsNotExist(err) {
|
||||
t.Fatal("old lifecycle remains")
|
||||
}
|
||||
if readFile(t, filepath.Join(dir, "internal/logic/greet/greet.go")) != logic {
|
||||
t.Fatal("business changed")
|
||||
}
|
||||
if _, err = Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
plan, err = PlanUpgrade(dir)
|
||||
if err != nil || len(plan.Changes) != 0 {
|
||||
t.Fatal("upgrade not idempotent", plan, err)
|
||||
}
|
||||
}
|
||||
func TestUpgradePreservesCustomFilesAndRollsBack(t *testing.T) {
|
||||
for _, custom := range []bool{true, false} {
|
||||
dir := oldInfrastructureProject(t)
|
||||
if custom {
|
||||
path := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
writeFile(t, path, readFile(t, path)+"\n// custom dependency\n")
|
||||
} else {
|
||||
path := filepath.Join(dir, "internal/logic/greet/greet.go")
|
||||
writeFile(t, path, "package greet;func(s *Logic)Greet(){}")
|
||||
}
|
||||
before := snapshot(t, dir)
|
||||
if _, err := ApplyUpgrade(dir); err == nil {
|
||||
t.Fatal("unsafe upgrade accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("failed upgrade changed project")
|
||||
}
|
||||
}
|
||||
}
|
||||
126
cmd/go-omni/apigen/workspace.go
Normal file
126
cmd/go-omni/apigen/workspace.go
Normal file
@@ -0,0 +1,126 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type WorkspaceService struct {
|
||||
Dir string `json:"dir"`
|
||||
API string `json:"api,omitempty"`
|
||||
Module string `json:"module,omitempty"`
|
||||
}
|
||||
type WorkspaceConfig struct {
|
||||
Services []WorkspaceService `json:"services"`
|
||||
}
|
||||
|
||||
// Workspace operates independent Go modules listed in an explicit local file.
|
||||
// All projects are preflighted before generate/format writes. Each generator
|
||||
// transaction is atomic within its project; no cross-module transaction is promised.
|
||||
func Workspace(path, command string, w io.Writer) error {
|
||||
if command != "generate" && command != "check" && command != "format" {
|
||||
return errors.New("workspace command must be generate, check or format")
|
||||
}
|
||||
if w == nil {
|
||||
w = io.Discard
|
||||
}
|
||||
real, err := filepath.Abs(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
stat, err := os.Lstat(real)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !stat.Mode().IsRegular() || stat.Size() > 65536 {
|
||||
return errors.New("workspace configuration must be a regular file of at most 64 KiB")
|
||||
}
|
||||
raw, err := os.ReadFile(real)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var cfg WorkspaceConfig
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(cfg.Services) == 0 || len(cfg.Services) > 64 {
|
||||
return errors.New("workspace must list 1..64 services")
|
||||
}
|
||||
root := canonical(filepath.Dir(real))
|
||||
var options []Options
|
||||
seen := map[string]bool{}
|
||||
for _, service := range cfg.Services {
|
||||
if service.Dir == "" || service.Dir == "." {
|
||||
return errors.New("workspace requires distinct relative service directories")
|
||||
}
|
||||
if err = safePath(root, service.Dir+"/.goOmniWorkspaceCheck"); err != nil {
|
||||
return err
|
||||
}
|
||||
dir := filepath.Join(root, filepath.FromSlash(service.Dir))
|
||||
key := strings.ToLower(dir)
|
||||
for prior := range seen {
|
||||
if prior == key || strings.HasPrefix(key, prior+string(filepath.Separator)) || strings.HasPrefix(prior, key+string(filepath.Separator)) {
|
||||
return errors.New("workspace service directories overlap")
|
||||
}
|
||||
}
|
||||
seen[key] = true
|
||||
entry := service.API
|
||||
if entry == "" {
|
||||
entry, err = ProjectEntry(dir)
|
||||
} else {
|
||||
if err = safePath(dir, entry); err != nil {
|
||||
return err
|
||||
}
|
||||
entry = filepath.Join(dir, entry)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
options = append(options, Options{API: entry, Dir: dir, Module: service.Module, DryRun: true})
|
||||
}
|
||||
var failures error
|
||||
for _, o := range options {
|
||||
if command == "format" {
|
||||
spec, e := Parse(o.API)
|
||||
if e == nil {
|
||||
_, e = Compile(spec)
|
||||
}
|
||||
err = e
|
||||
} else {
|
||||
_, err = Generate(o)
|
||||
}
|
||||
if err != nil {
|
||||
failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err))
|
||||
}
|
||||
}
|
||||
if failures != nil {
|
||||
return failures
|
||||
}
|
||||
for _, o := range options {
|
||||
switch command {
|
||||
case "format":
|
||||
_, err = FormatProject(o.API, false)
|
||||
case "generate":
|
||||
o.DryRun = false
|
||||
_, err = Generate(o)
|
||||
case "check":
|
||||
_, fmtErr := FormatProject(o.API, true)
|
||||
o.DryRun = false
|
||||
o.Check = true
|
||||
_, genErr := Generate(o)
|
||||
err = errors.Join(fmtErr, genErr)
|
||||
}
|
||||
if err != nil {
|
||||
failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err))
|
||||
fmt.Fprintf(w, "FAIL %s\n", o.Dir)
|
||||
} else {
|
||||
fmt.Fprintf(w, "OK %s\n", o.Dir)
|
||||
}
|
||||
}
|
||||
return failures
|
||||
}
|
||||
43
cmd/go-omni/apigen/workspace_test.go
Normal file
43
cmd/go-omni/apigen/workspace_test.go
Normal file
@@ -0,0 +1,43 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"io"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestWorkspaceIndependentModules(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
for _, name := range []string{"api", "id"} {
|
||||
dir := filepath.Join(root, "services", name)
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
config := filepath.Join(root, "omni.workspace.json")
|
||||
writeFile(t, config, `{"services":[{"dir":"services/api"},{"dir":"services/id"}]}`)
|
||||
for _, op := range []string{"format", "generate", "check", "generate", "check"} {
|
||||
if err := Workspace(config, op, io.Discard); err != nil {
|
||||
t.Fatal(op, err)
|
||||
}
|
||||
}
|
||||
// A bad second service must not leave the first service partially regenerated.
|
||||
first := filepath.Join(root, "services/api/api/service.api")
|
||||
writeFile(t, first, strings.Replace(readFile(t, first), "Greet a visitor", "Greet", 1))
|
||||
second := filepath.Join(root, "services/id/api/service.api")
|
||||
writeFile(t, second, "invalid source")
|
||||
before := snapshot(t, root)
|
||||
if err := Workspace(config, "generate", io.Discard); err == nil {
|
||||
t.Fatal("invalid workspace accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("preflight failure changed project")
|
||||
}
|
||||
for _, bad := range []string{`{"services":[{"dir":"../outside"}]}`, `{"services":[{"dir":"services/api"},{"dir":"services/api/nested"}]}`, `{"services":[{"dir":"services/api","dir":"services/id"}]}`} {
|
||||
writeFile(t, config, bad)
|
||||
if err := Workspace(config, "check", io.Discard); err == nil {
|
||||
t.Fatal("invalid workspace accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -16,12 +16,13 @@ import (
|
||||
const manifestPath = ".goOmni/apiManifest.json"
|
||||
|
||||
type manifest struct {
|
||||
Profile *projectProfile `json:"profile,omitempty"`
|
||||
Version int `json:"version"`
|
||||
Module string `json:"module"`
|
||||
Entry string `json:"entry"`
|
||||
Generated map[string]string `json:"generated"`
|
||||
Contract *contractSnapshot `json:"contract,omitempty"`
|
||||
TemplateVersion string `json:"templateVersion,omitempty"`
|
||||
Profile *projectProfile `json:"profile,omitempty"`
|
||||
Version int `json:"version"`
|
||||
Module string `json:"module"`
|
||||
Entry string `json:"entry"`
|
||||
Generated map[string]string `json:"generated"`
|
||||
Contract *contractSnapshot `json:"contract,omitempty"`
|
||||
}
|
||||
type change struct {
|
||||
name string
|
||||
@@ -30,7 +31,7 @@ type change struct {
|
||||
mode os.FileMode
|
||||
}
|
||||
|
||||
func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile, contract *contractSnapshot) (Output, error) {
|
||||
func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile, contract *contractSnapshot, lockHeld ...bool) (Output, error) {
|
||||
var result Output
|
||||
if info, err := os.Lstat(root); err == nil {
|
||||
if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() {
|
||||
@@ -61,7 +62,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
|
||||
if err := safePath(root, manifestPath); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if !check && !dryRun {
|
||||
if !check && !dryRun && !(len(lockHeld) > 0 && lockHeld[0]) {
|
||||
if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
|
||||
return result, err
|
||||
}
|
||||
@@ -108,7 +109,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
next := manifest{Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
|
||||
next := manifest{TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
|
||||
for name, data := range generated {
|
||||
next.Generated[name] = digest(data)
|
||||
}
|
||||
@@ -271,7 +272,10 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
|
||||
}
|
||||
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
|
||||
func ownedPath(name string) bool {
|
||||
if name == "main.go" || name == "internal/handler/appGen.go" || name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" {
|
||||
if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) {
|
||||
return true
|
||||
}
|
||||
if name == "main.go" || name == "internal/svc/runtimeGen.go" || name == "internal/handler/appGen.go" || name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" {
|
||||
return true
|
||||
}
|
||||
if name == "internal/apierrors/errorsGen.go" || name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {
|
||||
|
||||
@@ -18,6 +18,11 @@ func main() {
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "workspace":
|
||||
if err := runWorkspace(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni workspace:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "api":
|
||||
if err := runAPI(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni api:", err)
|
||||
@@ -170,6 +175,9 @@ func usage() {
|
||||
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api new [--module <module>] <directory> Create a runnable API project")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api generate [--api <file.api>] [--dir <project>] [--check|--dry-run]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api format [--api <file.api>] [--check]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api upgrade [--dir <project>] [--dry-run|--preview <directory>|--apply]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni workspace <format|generate|check> [--file omni.workspace.json]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api inspect [--api <file.api>] Show routes, auth and business errors")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api export [--api <file.api>] Write OpenAPI JSON to stdout")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api validate --api <file.api> Validate definitions and OpenAPI")
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
)
|
||||
|
||||
func TestNewWithRemoteFrameworkVersion(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--framework-version", "v0.13.0-preview"}, {"--framework-version=v0.13.0-preview"}} {
|
||||
for _, flag := range [][]string{{"--framework-version", "v0.14.0-preview"}, {"--framework-version=v0.14.0-preview"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
args := append([]string{"--module", "example.com/billing"}, flag...)
|
||||
if err := runNew(append(args, project)); err != nil {
|
||||
@@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(content), "go-omni v0.13.0-preview") || strings.Contains(string(content), "replace ") {
|
||||
if !strings.Contains(string(content), "go-omni v0.14.0-preview") || strings.Contains(string(content), "replace ") {
|
||||
t.Fatalf("CLI did not pin remote dependency: %s", content)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,17 +8,37 @@ import (
|
||||
// APIInfrastructure renders shared infrastructure without inventing business
|
||||
// tables or routes. The API generator applies these as developer-owned seeds.
|
||||
func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
|
||||
return apiInfrastructure(module, database, storage, logOutput, logFormat, false)
|
||||
}
|
||||
|
||||
// LegacyAPIInfrastructure renders only the known v0.13 seed baseline for a safe upgrade.
|
||||
func LegacyAPIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
|
||||
return apiInfrastructure(module, database, storage, logOutput, logFormat, true)
|
||||
}
|
||||
func apiInfrastructure(module, database, storage, logOutput, logFormat string, legacy bool) (map[string][]byte, error) {
|
||||
data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true}
|
||||
files := map[string]string{
|
||||
"internal/svc/close.go": "templates/api-close.go.tmpl",
|
||||
"internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl",
|
||||
"internal/svc/options.go": "templates/api-options.go.tmpl",
|
||||
"internal/config/config.go": "templates/modules-settings.go.tmpl",
|
||||
"internal/config/config.go": "templates/api-config.go.tmpl",
|
||||
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
|
||||
"internal/svc/serviceContext.go": "templates/api-context.go.tmpl",
|
||||
}
|
||||
if database == "mysql" {
|
||||
files["internal/svc/databaseOptions.go"] = "templates/api-databaseOptions.go.tmpl"
|
||||
}
|
||||
if storage != "none" {
|
||||
files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||
}
|
||||
if legacy {
|
||||
delete(files, "internal/svc/runtimeGen.go")
|
||||
delete(files, "internal/svc/databaseOptions.go")
|
||||
files["internal/svc/close.go"] = "templates/api-close.go.tmpl"
|
||||
files["internal/config/config.go"] = "templates/modules-settings.go.tmpl"
|
||||
for name, source := range files {
|
||||
files[name] = strings.Replace(source, "templates/", "templates/legacyV013/", 1)
|
||||
}
|
||||
}
|
||||
result := make(map[string][]byte, len(files))
|
||||
for name, source := range files {
|
||||
content, err := render(source, data)
|
||||
|
||||
104
cmd/go-omni/scaffold/templates/api-config.go.tmpl
Normal file
104
cmd/go-omni/scaffold/templates/api-config.go.tmpl
Normal file
@@ -0,0 +1,104 @@
|
||||
package settings
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
|
||||
{{if ne .Storage "none"}}"errors"
|
||||
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
)
|
||||
type Config struct {
|
||||
Server config.Config `json:"server"`
|
||||
{{if eq .Database "mysql"}}Database mysqladapter.Config `json:"database" env:"GO_OMNI_DB"`{{end}}
|
||||
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage" env:"GO_OMNI_STORAGE"`{{end}}
|
||||
}
|
||||
func Default() Config {
|
||||
server:=config.Default()
|
||||
server.Address="127.0.0.1:8080"
|
||||
server.EnableLogger=true
|
||||
server.SlowRequestThreshold=500*time.Millisecond
|
||||
server.Log.Service={{printf "%q" .Module}}
|
||||
server.Log.Format={{printf "%q" .LogFormat}}
|
||||
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
|
||||
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
|
||||
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
|
||||
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:mysqladapter.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
|
||||
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
|
||||
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
|
||||
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
|
||||
{{if eq .Storage "r2"}}Region:"auto",{{end}}
|
||||
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
|
||||
}{{end}}}
|
||||
}
|
||||
|
||||
// Load applies defaults -> file -> typed environment -> validation.
|
||||
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
|
||||
func Load(path string)(Config,error){
|
||||
cfg:=Default();var err error
|
||||
cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err}
|
||||
if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}}
|
||||
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_"});if err!=nil{return Config{},err}
|
||||
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
|
||||
cfg.Server,err=config.ApplyEnv(cfg.Server,"");if err!=nil{return Config{},err}
|
||||
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
|
||||
return cfg,cfg.Validate()
|
||||
}
|
||||
func(c Config)Validate()error{
|
||||
if err:=c.Server.Validate();err!=nil{return err}
|
||||
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return err}{{end}}
|
||||
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return err}{{end}}
|
||||
return nil
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
type StorageConfig struct {
|
||||
Provider string `json:"provider"`
|
||||
Root string `json:"root"`
|
||||
PublicURL string `json:"publicUrl"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
Region string `json:"region"`
|
||||
Bucket string `json:"bucket"`
|
||||
AccountID string `json:"accountId"`
|
||||
Secure bool `json:"secure"`
|
||||
MaxBytes int64 `json:"maxBytes"`
|
||||
ContentTypes []string `json:"contentTypes"`
|
||||
LinkTTL time.Duration `json:"linkTtl"`
|
||||
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||
// Secrets are environment-only and cannot be populated by config files.
|
||||
FilesToken string `json:"-" env:"GO_OMNI_FILES_TOKEN,secret"`
|
||||
SigningKey string `json:"-" env:"GO_OMNI_STORAGE_SIGNING_KEY,secret"`
|
||||
AccessKey string `json:"-" env:"GO_OMNI_STORAGE_ACCESS_KEY,secret"`
|
||||
SecretKey string `json:"-" env:"GO_OMNI_STORAGE_SECRET_KEY,secret"`
|
||||
SessionToken string `json:"-" env:"GO_OMNI_STORAGE_SESSION_TOKEN,secret"`
|
||||
}
|
||||
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||
func (s StorageConfig) Validate()error {
|
||||
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
|
||||
if err:=s.Policy().Validate();err!=nil{return err}
|
||||
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||
{{if eq .Storage "local"}}
|
||||
if s.Root=="" {return errors.New("storage.root is required")}
|
||||
u,err:=url.Parse(s.PublicURL)
|
||||
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||
{{else}}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||
{{if eq .Storage "r2"}}
|
||||
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||
{{else if eq .Storage "s3"}}
|
||||
u,err:=url.Parse("https://"+s.Endpoint)
|
||||
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
|
||||
{{else if eq .Storage "oss"}}
|
||||
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
|
||||
if !s.Secure{return errors.New("OSS requires secure=true")}
|
||||
{{end}}
|
||||
{{end}}
|
||||
return nil
|
||||
}
|
||||
{{end}}
|
||||
@@ -1,26 +1,20 @@
|
||||
package servicecontext
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
{{if ne .Storage "none"}}"net/http"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if eq .Database "mysql"}}"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"github.com/go-sql-driver/mysql"{{end}}
|
||||
{{if eq .Database "mysql"}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"{{.Module}}/internal/settings"
|
||||
"{{.Module}}/internal/middleware"
|
||||
)
|
||||
// Context owns infrastructure, while Logger is borrowed from main.
|
||||
// Add business repositories here; HTTP contracts do not define database tables.
|
||||
// Context holds project dependencies. Runtime owns cleanup and shutdown.
|
||||
// Logger is borrowed; the generated entry closes it after dependencies.
|
||||
type Context struct {
|
||||
Lifecycle *lifecycle.Controller
|
||||
*lifecycle.Runtime
|
||||
Config settings.Config
|
||||
cleanups []func()error
|
||||
Logger *logx.Logger
|
||||
Health *health.Checker
|
||||
Middlewares map[string]httpx.Middleware
|
||||
@@ -31,38 +25,9 @@ type Context struct {
|
||||
Downloads storage.Presigner
|
||||
UploadLinks storage.UploadPresigner
|
||||
FileHTTP http.Handler
|
||||
storageClose func()error
|
||||
{{end}}
|
||||
closeMu sync.Mutex
|
||||
closed bool
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
}
|
||||
// Pass the settings loaded by main. The optional form supports lightweight
|
||||
// tests without infrastructure; selected adapters still require valid settings.
|
||||
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
|
||||
if ctx==nil{ctx=context.Background()}
|
||||
if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))}
|
||||
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
|
||||
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||
_=cfg
|
||||
checker,err:=health.New();if err!=nil{return nil,err}
|
||||
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Lifecycle:lifecycle.New(ctx)}
|
||||
ctx=c.Lifecycle.Context()
|
||||
defer func(){if err!=nil{err=errors.Join(err,c.Close());if cause:=c.Lifecycle.Err();cause!=nil && !errors.Is(err,cause){err=errors.Join(err,cause)}}}()
|
||||
{{if eq .Database "mysql"}}
|
||||
if err=cfg.Validate();err!=nil{return nil,err}
|
||||
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
|
||||
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
|
||||
dsn.ParseTime=true;dsn.Loc=time.UTC
|
||||
cfg.Database.DSN=dsn.FormatDSN()
|
||||
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
|
||||
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
|
||||
{{end}}
|
||||
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
|
||||
return c,nil
|
||||
}
|
||||
|
||||
// RequestShutdown reports a fatal dependency failure, or a normal stop when
|
||||
// err is nil. It returns immediately; generated main drains and cleans up.
|
||||
func(c *Context)RequestShutdown(err error){c.Lifecycle.RequestShutdown(err)}
|
||||
// Init is project-owned. Add dependencies here and immediately register each
|
||||
// cleanup with c.OnClose. Returning an error automatically rolls everything back.
|
||||
// Generated infrastructure is ready here. Never create another App or logger.
|
||||
func(c *Context)Init(ctx context.Context)error {return nil}
|
||||
|
||||
@@ -0,0 +1,5 @@
|
||||
package servicecontext
|
||||
import mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"
|
||||
// DatabaseOptions supplies explicit driver settings or bounded startup checks.
|
||||
// Keep schema/version requirements in this project; do not open another pool.
|
||||
func(c *Context)DatabaseOptions()[]mysqladapter.Option{return nil}
|
||||
33
cmd/go-omni/scaffold/templates/api-runtime.go.tmpl
Normal file
33
cmd/go-omni/scaffold/templates/api-runtime.go.tmpl
Normal file
@@ -0,0 +1,33 @@
|
||||
package servicecontext
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
|
||||
"{{.Module}}/internal/settings"
|
||||
"{{.Module}}/internal/middleware"
|
||||
)
|
||||
// New is generator-owned. Customize dependencies in ServiceContext.Init.
|
||||
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
|
||||
if ctx==nil{ctx=context.Background()}
|
||||
if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))}
|
||||
if logger==nil{return nil,errors.New("service context requires a logger")}
|
||||
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
|
||||
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||
if err=cfg.Validate();err!=nil{return nil,err}
|
||||
checker,err:=health.New();if err!=nil{return nil,err}
|
||||
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Runtime:lifecycle.NewRuntime(ctx)}
|
||||
ctx=logx.WithLogger(c.Context(),logger)
|
||||
defer func(){if p:=recover();p!=nil{_ = c.Close();panic(p)};if err!=nil{err=c.Finish(err)}}()
|
||||
{{if eq .Database "mysql"}}
|
||||
options:=append([]mysqladapter.Option{mysqladapter.WithLogger(logger)},c.DatabaseOptions()...)
|
||||
c.DB,err=mysqladapter.Open(ctx,cfg.Database,options...);if err!=nil{return nil,err}
|
||||
c.OnClose(c.DB.Close)
|
||||
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.Pool.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
|
||||
{{end}}
|
||||
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
|
||||
if err=c.Init(ctx);err!=nil{return nil,err}
|
||||
return c,nil
|
||||
}
|
||||
23
cmd/go-omni/scaffold/templates/legacyV013/api-close.go.tmpl
Normal file
23
cmd/go-omni/scaffold/templates/legacyV013/api-close.go.tmpl
Normal file
@@ -0,0 +1,23 @@
|
||||
package servicecontext
|
||||
import "errors"
|
||||
// Close is idempotent, including startup failure and graceful shutdown.
|
||||
func(c *Context)Close()error{
|
||||
if c==nil{return nil}
|
||||
c.closeOnce.Do(func(){
|
||||
if c.Lifecycle!=nil{c.Lifecycle.RequestShutdown(nil)}
|
||||
c.closeMu.Lock();c.closed=true;cleanups:=c.cleanups;c.cleanups=nil;c.closeMu.Unlock()
|
||||
for i:=len(cleanups)-1;i>=0;i--{c.closeErr=errors.Join(c.closeErr,cleanups[i]())}
|
||||
{{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}}
|
||||
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||
})
|
||||
return c.closeErr
|
||||
}
|
||||
|
||||
// OnClose transfers a resource cleanup to this context immediately after creation.
|
||||
// Hooks run once in reverse order, also when later initialization fails.
|
||||
func(c *Context)OnClose(cleanup func()error) {
|
||||
if cleanup==nil{return}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
if c.closed{panic("svc: register cleanup after Close")}
|
||||
c.cleanups=append(c.cleanups,cleanup)
|
||||
}
|
||||
@@ -0,0 +1,68 @@
|
||||
package servicecontext
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
{{if ne .Storage "none"}}"net/http"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if eq .Database "mysql"}}"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"github.com/go-sql-driver/mysql"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"{{.Module}}/internal/settings"
|
||||
"{{.Module}}/internal/middleware"
|
||||
)
|
||||
// Context owns infrastructure, while Logger is borrowed from main.
|
||||
// Add business repositories here; HTTP contracts do not define database tables.
|
||||
type Context struct {
|
||||
Lifecycle *lifecycle.Controller
|
||||
Config settings.Config
|
||||
cleanups []func()error
|
||||
Logger *logx.Logger
|
||||
Health *health.Checker
|
||||
Middlewares map[string]httpx.Middleware
|
||||
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||
{{if ne .Storage "none"}}
|
||||
Files storage.Store
|
||||
Uploads *storage.Uploader
|
||||
Downloads storage.Presigner
|
||||
UploadLinks storage.UploadPresigner
|
||||
FileHTTP http.Handler
|
||||
storageClose func()error
|
||||
{{end}}
|
||||
closeMu sync.Mutex
|
||||
closed bool
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
}
|
||||
// Pass the settings loaded by main. The optional form supports lightweight
|
||||
// tests without infrastructure; selected adapters still require valid settings.
|
||||
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
|
||||
if ctx==nil{ctx=context.Background()}
|
||||
if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))}
|
||||
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
|
||||
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||
_=cfg
|
||||
checker,err:=health.New();if err!=nil{return nil,err}
|
||||
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Lifecycle:lifecycle.New(ctx)}
|
||||
ctx=c.Lifecycle.Context()
|
||||
defer func(){if err!=nil{err=errors.Join(err,c.Close());if cause:=c.Lifecycle.Err();cause!=nil && !errors.Is(err,cause){err=errors.Join(err,cause)}}}()
|
||||
{{if eq .Database "mysql"}}
|
||||
if err=cfg.Validate();err!=nil{return nil,err}
|
||||
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
|
||||
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
|
||||
dsn.ParseTime=true;dsn.Loc=time.UTC
|
||||
cfg.Database.DSN=dsn.FormatDSN()
|
||||
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
|
||||
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
|
||||
{{end}}
|
||||
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
|
||||
return c,nil
|
||||
}
|
||||
|
||||
// RequestShutdown reports a fatal dependency failure, or a normal stop when
|
||||
// err is nil. It returns immediately; generated main drains and cleans up.
|
||||
func(c *Context)RequestShutdown(err error){c.Lifecycle.RequestShutdown(err)}
|
||||
@@ -0,0 +1,11 @@
|
||||
package servicecontext
|
||||
import (
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
// AppOptions runs before routes are registered. Register project error catalogs
|
||||
// here with omni.WithErrors. It must not create resources or another App.
|
||||
func(c *Context)AppOptions()[]omni.Option{return nil}
|
||||
// GlobalMiddleware runs for all endpoints, including App 404/405 fallbacks.
|
||||
// Named route middleware lives in Middlewares and is selected by the API contract.
|
||||
func(c *Context)GlobalMiddleware()[]httpx.Middleware{return nil}
|
||||
@@ -0,0 +1,59 @@
|
||||
# Non-secret settings only. Environment variables override these values.
|
||||
server:
|
||||
address: '127.0.0.1:8080'
|
||||
enableLogger: true
|
||||
slowRequestThreshold: 500ms
|
||||
shutdownTimeout: 10s
|
||||
{{if .API}} endpoints:
|
||||
liveness:
|
||||
enabled: true
|
||||
path: /health/live
|
||||
responseMode: app
|
||||
readiness:
|
||||
enabled: true
|
||||
path: /health/ready
|
||||
responseMode: app
|
||||
openapi:
|
||||
enabled: true
|
||||
path: /openapi.json
|
||||
{{end}}
|
||||
log:
|
||||
service: {{printf "%q" .Module}}
|
||||
level: info
|
||||
format: {{.LogFormat}}
|
||||
{{if eq .LogOutput "file"}} file: './var/log/server.log'
|
||||
rotation:
|
||||
enabled: true
|
||||
maxSizeMb: 100
|
||||
maxBackups: 10
|
||||
maxAgeDays: 7
|
||||
compress: true
|
||||
{{else}} output: {{.LogOutput}}
|
||||
{{end}} redactKeys: []
|
||||
{{if ne .Storage "none"}}storage:
|
||||
provider: {{.Storage}}
|
||||
maxBytes: 33554432
|
||||
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||
linkTtl: 5m
|
||||
requestTimeout: 30s
|
||||
{{if eq .Storage "local"}} root: './var/files'
|
||||
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
|
||||
secure: true
|
||||
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
|
||||
region: auto
|
||||
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
|
||||
region: us-east-1
|
||||
{{else}} region: cn-hangzhou
|
||||
endpoint: '' # empty uses the official regional endpoint
|
||||
{{end}}{{end}}{{end}}
|
||||
|
||||
{{if and .API (eq .Database "mysql")}}database:
|
||||
driver: mysql
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||
{{end}}
|
||||
@@ -0,0 +1,140 @@
|
||||
// Package settings owns this project's configuration; it adds no business identity model.
|
||||
package settings
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
{{if ne .Storage "none"}}"fmt"
|
||||
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||
"strconv"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Server config.Config `json:"server"`
|
||||
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
|
||||
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
server:=config.Default()
|
||||
server.Address="127.0.0.1:8080"
|
||||
server.EnableLogger=true
|
||||
server.SlowRequestThreshold=500*time.Millisecond
|
||||
server.Log.Service={{printf "%q" .Module}}
|
||||
server.Log.Format={{printf "%q" .LogFormat}}
|
||||
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
|
||||
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
|
||||
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
|
||||
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
|
||||
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
|
||||
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
|
||||
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
|
||||
{{if eq .Storage "r2"}}Region:"auto",{{end}}
|
||||
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
|
||||
}{{end}}}
|
||||
}
|
||||
|
||||
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||
// Pass an empty path to use only defaults and environment.
|
||||
func Load(path string)(Config,error) {
|
||||
cfg:=Default()
|
||||
var err error
|
||||
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
|
||||
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
|
||||
if err!=nil{return Config{},err}
|
||||
// A stream explicitly selected by environment overrides a file profile.
|
||||
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
|
||||
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
|
||||
{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
|
||||
if err=cfg.Validate();err!=nil{return Config{},err}
|
||||
return cfg,nil
|
||||
}
|
||||
func (c Config) Validate()error {
|
||||
{{if and .API (eq .Database "mysql")}}
|
||||
if err:=c.Database.Validate();err!=nil{return err}
|
||||
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
|
||||
{{end}}
|
||||
if err:=c.Server.Validate();err!=nil{return err}
|
||||
if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")}
|
||||
switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")}
|
||||
{{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}}
|
||||
}
|
||||
|
||||
{{if ne .Storage "none"}}
|
||||
type StorageConfig struct {
|
||||
Provider string `json:"provider"`
|
||||
Root string `json:"root"`
|
||||
PublicURL string `json:"publicUrl"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
Region string `json:"region"`
|
||||
Bucket string `json:"bucket"`
|
||||
AccountID string `json:"accountId"`
|
||||
Secure bool `json:"secure"`
|
||||
MaxBytes int64 `json:"maxBytes"`
|
||||
ContentTypes []string `json:"contentTypes"`
|
||||
LinkTTL time.Duration `json:"linkTtl"`
|
||||
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||
// Secrets are environment-only and cannot be populated by config files.
|
||||
FilesToken string `json:"-"`
|
||||
SigningKey string `json:"-"`
|
||||
AccessKey string `json:"-"`
|
||||
SecretKey string `json:"-"`
|
||||
SessionToken string `json:"-"`
|
||||
}
|
||||
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||
func (s *StorageConfig) applyEnv()error {
|
||||
values:=map[string]*string{
|
||||
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
|
||||
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
|
||||
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
|
||||
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
|
||||
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
|
||||
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
|
||||
}
|
||||
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
|
||||
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
|
||||
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
|
||||
}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
|
||||
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
|
||||
return nil
|
||||
}
|
||||
func (s StorageConfig) Validate()error {
|
||||
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
|
||||
if err:=s.Policy().Validate();err!=nil{return err}
|
||||
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||
{{if eq .Storage "local"}}
|
||||
if s.Root=="" {return errors.New("storage.root is required")}
|
||||
u,err:=url.Parse(s.PublicURL)
|
||||
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||
{{else}}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||
{{if eq .Storage "r2"}}
|
||||
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||
{{else if eq .Storage "s3"}}
|
||||
u,err:=url.Parse("https://"+s.Endpoint)
|
||||
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
|
||||
{{else if eq .Storage "oss"}}
|
||||
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
|
||||
if !s.Secure{return errors.New("OSS requires secure=true")}
|
||||
{{end}}
|
||||
{{end}}
|
||||
return nil
|
||||
}
|
||||
{{end}}
|
||||
@@ -0,0 +1,79 @@
|
||||
package servicecontext
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
{{if eq .Storage "local"}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
{{else}}
|
||||
"net"
|
||||
"net/http"
|
||||
"time"
|
||||
{{if eq .Storage "oss"}}
|
||||
ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss"
|
||||
ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
|
||||
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
|
||||
{{else}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/s3"
|
||||
"github.com/minio/minio-go/v7/pkg/credentials"
|
||||
{{end}}{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"{{.Module}}/internal/settings"
|
||||
)
|
||||
|
||||
// InitStorage is called once, before serving requests. The composition root
|
||||
// owns provider clients; business services receive only capability interfaces.
|
||||
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
|
||||
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
if c.closed{return errors.New("service dependencies are closed")}
|
||||
if err:=ctx.Err();err!=nil{return err}
|
||||
if c.Files!=nil{return errors.New("storage already initialized")}
|
||||
if err:=cfg.Validate();err!=nil{return err}
|
||||
{{if eq .Storage "local"}}
|
||||
files,err:=local.New(cfg.Root,cfg.MaxBytes)
|
||||
if err!=nil{return err}
|
||||
signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer
|
||||
{{else}}
|
||||
// Dedicated bounded pool, never the process-wide default transport.
|
||||
transport:=&http.Transport{
|
||||
Proxy:http.ProxyFromEnvironment,
|
||||
DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext,
|
||||
ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32,
|
||||
IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second,
|
||||
ResponseHeaderTimeout:cfg.RequestTimeout,
|
||||
}
|
||||
initialized:=false
|
||||
defer func(){if !initialized{transport.CloseIdleConnections()}}()
|
||||
{{if eq .Storage "oss"}}
|
||||
sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region).
|
||||
WithSignatureVersion(ossSDK.SignatureVersionV4).
|
||||
WithAdditionalHeaders([]string{"content-length"}).
|
||||
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)).
|
||||
WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout})
|
||||
if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)}
|
||||
files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes)
|
||||
{{else}}
|
||||
creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)
|
||||
{{if eq .Storage "r2"}}
|
||||
options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds)
|
||||
if err!=nil{return err}
|
||||
options.MaxBytes,options.Transport=cfg.MaxBytes,transport
|
||||
{{else}}
|
||||
options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport}
|
||||
{{end}}
|
||||
files,err:=s3.Open(options)
|
||||
{{end}}
|
||||
if err!=nil{return err}
|
||||
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
|
||||
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
|
||||
initialized=true
|
||||
{{end}}
|
||||
return nil
|
||||
}
|
||||
@@ -49,11 +49,16 @@ server:
|
||||
{{end}}{{end}}{{end}}
|
||||
|
||||
{{if and .API (eq .Database "mysql")}}database:
|
||||
driver: mysql
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||
host: 127.0.0.1
|
||||
port: 3306
|
||||
database: '' # GO_OMNI_DB_DATABASE
|
||||
user: '' # GO_OMNI_DB_USER; password via GO_OMNI_DB_PASSWORD
|
||||
tls:
|
||||
mode: required
|
||||
pool:
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
{{end}}
|
||||
|
||||
@@ -25,8 +25,8 @@ import (
|
||||
// owns provider clients; business services receive only capability interfaces.
|
||||
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
|
||||
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
if c.closed{return errors.New("service dependencies are closed")}
|
||||
{{if .API}}if c.Runtime==nil || c.Context().Err()!=nil{return errors.New("service dependencies are closed")}{{else}}c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
if c.closed{return errors.New("service dependencies are closed")}{{end}}
|
||||
if err:=ctx.Err();err!=nil{return err}
|
||||
if c.Files!=nil{return errors.New("storage already initialized")}
|
||||
if err:=cfg.Validate();err!=nil{return err}
|
||||
@@ -72,7 +72,7 @@ func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)err
|
||||
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
|
||||
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
|
||||
{{if .API}}c.OnClose(func()error{transport.CloseIdleConnections();return nil}){{else}}c.storageClose=func()error{transport.CloseIdleConnections();return nil}{{end}}
|
||||
initialized=true
|
||||
{{end}}
|
||||
return nil
|
||||
|
||||
@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||
// downloaded module version, which also has a module checksum.
|
||||
const defaultFrameworkVersion = "v0.13.0-preview"
|
||||
const defaultFrameworkVersion = "v0.14.1-preview"
|
||||
|
||||
// This target is not a claim that the release already exists. Source users
|
||||
// select a local framework checkout until the module templates are published.
|
||||
|
||||
44
cmd/go-omni/upgrade.go
Normal file
44
cmd/go-omni/upgrade.go
Normal file
@@ -0,0 +1,44 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||
"io"
|
||||
)
|
||||
|
||||
func runUpgrade(args []string) error {
|
||||
flags := flag.NewFlagSet("upgrade", flag.ContinueOnError)
|
||||
flags.SetOutput(io.Discard)
|
||||
dir := flags.String("dir", ".", "service directory")
|
||||
apply := flags.Bool("apply", false, "apply only recognized baseline changes")
|
||||
preview := flags.String("preview", "", "write proposed infrastructure into an empty directory")
|
||||
dry := flags.Bool("dry-run", false, "show the upgrade plan")
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() != 0 {
|
||||
return errors.New("unexpected upgrade argument")
|
||||
}
|
||||
if *apply && (*preview != "" || *dry) {
|
||||
return errors.New("--apply cannot be combined with preview/dry-run")
|
||||
}
|
||||
var plan apigen.UpgradePlan
|
||||
var err error
|
||||
if *apply {
|
||||
plan, err = apigen.ApplyUpgrade(*dir)
|
||||
} else if *preview != "" {
|
||||
plan, err = apigen.PreviewUpgrade(*dir, *preview)
|
||||
} else {
|
||||
plan, err = apigen.PlanUpgrade(*dir)
|
||||
}
|
||||
for _, c := range plan.Changes {
|
||||
kind := "safe"
|
||||
if c.Manual {
|
||||
kind = "review"
|
||||
}
|
||||
fmt.Printf("%-6s %-6s %s\n", kind, c.Action, c.Path)
|
||||
}
|
||||
return err
|
||||
}
|
||||
25
cmd/go-omni/workspace.go
Normal file
25
cmd/go-omni/workspace.go
Normal file
@@ -0,0 +1,25 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"flag"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||
"io"
|
||||
"os"
|
||||
)
|
||||
|
||||
func runWorkspace(args []string) error {
|
||||
if len(args) < 1 {
|
||||
return errors.New("usage: go-omni workspace <format|generate|check> --file omni.workspace.json")
|
||||
}
|
||||
flags := flag.NewFlagSet("workspace", flag.ContinueOnError)
|
||||
flags.SetOutput(io.Discard)
|
||||
path := flags.String("file", "omni.workspace.json", "workspace configuration")
|
||||
if err := flags.Parse(args[1:]); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() != 0 {
|
||||
return errors.New("unexpected workspace argument")
|
||||
}
|
||||
return apigen.Workspace(*path, args[0], os.Stdout)
|
||||
}
|
||||
81
config/clone.go
Normal file
81
config/clone.go
Normal file
@@ -0,0 +1,81 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Clone only configuration data. json:"-" capabilities remain borrowed and
|
||||
// are never traversed (log writers, clients and mutex-bearing runtime objects).
|
||||
// Unexported immutable implementation details such as time.Time are copied.
|
||||
func cloneConfiguration(v reflect.Value, depth int) (reflect.Value, error) {
|
||||
if depth > 64 {
|
||||
return reflect.Value{}, errors.New("config: defaults contain cycles or excessive nesting")
|
||||
}
|
||||
result := reflect.New(v.Type()).Elem()
|
||||
result.Set(v)
|
||||
switch v.Kind() {
|
||||
case reflect.Struct:
|
||||
for i := 0; i < v.NumField(); i++ {
|
||||
field := v.Type().Field(i)
|
||||
if field.PkgPath != "" || strings.Split(field.Tag.Get("json"), ",")[0] == "-" {
|
||||
continue
|
||||
}
|
||||
child, err := cloneConfiguration(v.Field(i), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
result.Field(i).Set(child)
|
||||
}
|
||||
case reflect.Pointer:
|
||||
if !v.IsNil() {
|
||||
child, err := cloneConfiguration(v.Elem(), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
result.Set(reflect.New(v.Type().Elem()))
|
||||
result.Elem().Set(child)
|
||||
}
|
||||
case reflect.Interface:
|
||||
if !v.IsNil() {
|
||||
child, err := cloneConfiguration(v.Elem(), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
result.Set(child)
|
||||
}
|
||||
case reflect.Slice:
|
||||
if !v.IsNil() {
|
||||
result = reflect.MakeSlice(v.Type(), v.Len(), v.Len())
|
||||
for i := 0; i < v.Len(); i++ {
|
||||
child, err := cloneConfiguration(v.Index(i), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
result.Index(i).Set(child)
|
||||
}
|
||||
}
|
||||
case reflect.Array:
|
||||
for i := 0; i < v.Len(); i++ {
|
||||
child, err := cloneConfiguration(v.Index(i), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
result.Index(i).Set(child)
|
||||
}
|
||||
case reflect.Map:
|
||||
if !v.IsNil() {
|
||||
result = reflect.MakeMapWithSize(v.Type(), v.Len())
|
||||
iter := v.MapRange()
|
||||
for iter.Next() {
|
||||
child, err := cloneConfiguration(iter.Value(), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
result.SetMapIndex(iter.Key(), child)
|
||||
}
|
||||
}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
@@ -26,20 +27,22 @@ type TLSConfig struct {
|
||||
|
||||
// Config controls the default runtime behavior of an App.
|
||||
type Config struct {
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
HTTP HTTPConfig `json:"http"`
|
||||
Endpoints EndpointsConfig `json:"endpoints"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Log logx.Config `json:"log"`
|
||||
Errors apierrors.Catalog `json:"errors"`
|
||||
Response response.Config `json:"response"`
|
||||
ResponseEncoder response.Encoder `json:"-"`
|
||||
Request contract.RequestPolicy `json:"request"`
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
HTTP HTTPConfig `json:"http"`
|
||||
Endpoints EndpointsConfig `json:"endpoints"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Log logx.Config `json:"log"`
|
||||
Errors apierrors.Catalog `json:"errors"`
|
||||
Response response.Config `json:"response"`
|
||||
ResponseEncoder response.Encoder `json:"-"`
|
||||
LogInstance *logx.Logger `json:"-"`
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
@@ -55,6 +58,12 @@ func Default() Config {
|
||||
|
||||
// Validate checks configuration values before an application starts.
|
||||
func (c Config) Validate() error {
|
||||
if err := c.Request.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Log.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Endpoints.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -221,3 +230,10 @@ func WithErrors(catalog apierrors.Catalog) Option {
|
||||
catalog = catalog.Clone()
|
||||
return func(cfg *Config) { cfg.Errors = catalog.Clone() }
|
||||
}
|
||||
|
||||
// WithLogInstance borrows a logger for this App without changing process logging.
|
||||
func WithLogInstance(logger *logx.Logger) Option { return func(c *Config) { c.LogInstance = logger } }
|
||||
|
||||
func WithRequestPolicy(policy contract.RequestPolicy) Option {
|
||||
return func(c *Config) { c.Request = policy }
|
||||
}
|
||||
|
||||
@@ -22,6 +22,23 @@ import (
|
||||
// field names for both formats, rejects unknown fields and accepts durations
|
||||
// such as "3s". Application-specific validation remains with the caller.
|
||||
func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||
result, err := decodeFile(path, defaults)
|
||||
if err != nil {
|
||||
var zero T
|
||||
return zero, &DecodeError{cause: err}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// DecodeError keeps input values out of public diagnostics while preserving
|
||||
// errors.Is/As for explicitly handled underlying I/O and syntax errors.
|
||||
type DecodeError struct{ cause error }
|
||||
|
||||
func (e *DecodeError) Error() string {
|
||||
return "config: cannot decode file; check its path, format, field names and value types"
|
||||
}
|
||||
func (e *DecodeError) Unwrap() error { return e.cause }
|
||||
func decodeFile[T any](path string, defaults T) (T, error) {
|
||||
var zero T
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
@@ -64,15 +81,13 @@ func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
// Deep copy defaults so errors cannot partially mutate caller-owned maps.
|
||||
original, err := json.Marshal(defaults)
|
||||
// Clone configuration without serializing borrowed json:"-" capabilities.
|
||||
var result T
|
||||
original, err := cloneConfiguration(reflect.ValueOf(&defaults).Elem(), 0)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var result T
|
||||
if err = json.Unmarshal(original, &result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
reflect.ValueOf(&result).Elem().Set(original)
|
||||
dec := json.NewDecoder(bytes.NewReader(encoded))
|
||||
dec.DisallowUnknownFields()
|
||||
if err = dec.Decode(&result); err != nil {
|
||||
@@ -115,6 +130,17 @@ func durations(v any, t reflect.Type) (any, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Map && t.Key().Kind() == reflect.String {
|
||||
if values, ok := v.(map[string]any); ok {
|
||||
for key, value := range values {
|
||||
converted, err := durations(value, t.Elem())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
values[key] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Slice {
|
||||
if items, ok := v.([]any); ok {
|
||||
for n, item := range items {
|
||||
|
||||
290
config/project.go
Normal file
290
config/project.go
Normal file
@@ -0,0 +1,290 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"math"
|
||||
"os"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// EnvOptions controls typed project configuration overlays. Explicit env tags
|
||||
// are absolute names; otherwise JSON names become PREFIX_NESTED_FIELD names.
|
||||
// Empty is allow (default), ignore or reject. No input values appear in errors.
|
||||
type EnvOptions struct {
|
||||
Prefix string
|
||||
Empty string
|
||||
Lookup func(string) (string, bool)
|
||||
}
|
||||
|
||||
// ApplyDefaults applies envDefault tags to zero fields before file overlays.
|
||||
func ApplyDefaults[T any](base T) (T, error) { return projectOverlay(base, EnvOptions{}, true) }
|
||||
|
||||
// LoadProject applies code/tag defaults, a strict optional file, environment,
|
||||
// then the project's Validate method, before any logger or dependency is opened.
|
||||
// envDefault is only applied to zero-valued defaults before reading the file.
|
||||
func LoadProject[T any](path string, defaults T, options EnvOptions) (T, error) {
|
||||
var zero T
|
||||
cfg, err := projectOverlay(defaults, options, true)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if path != "" {
|
||||
cfg, err = DecodeFile(path, cfg)
|
||||
if err != nil {
|
||||
return zero, errors.New("config: cannot decode file; check its path, format, field names and value types")
|
||||
}
|
||||
}
|
||||
cfg, err = OverlayEnv(cfg, options)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if valid, ok := any(&cfg).(interface{ Validate() error }); ok {
|
||||
if err = valid.Validate(); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
// OverlayEnv overlays the entire project struct without mutating the input.
|
||||
// Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a
|
||||
// subtree. required means nonempty string/non-nil pointer or collection; false
|
||||
// and zero are valid scalars. Use project Validate for semantic constraints.
|
||||
// Slices/maps use JSON syntax, avoiding ambiguous comma and empty semantics.
|
||||
func OverlayEnv[T any](base T, options EnvOptions) (T, error) {
|
||||
return projectOverlay(base, options, false)
|
||||
}
|
||||
|
||||
var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
|
||||
var durationType = reflect.TypeOf(time.Duration(0))
|
||||
|
||||
func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error) {
|
||||
var zero T
|
||||
if options.Lookup == nil {
|
||||
options.Lookup = os.LookupEnv
|
||||
}
|
||||
if options.Empty == "" {
|
||||
options.Empty = "allow"
|
||||
}
|
||||
if options.Empty != "allow" && options.Empty != "ignore" && options.Empty != "reject" {
|
||||
return zero, errors.New("config: empty policy must be allow, ignore or reject")
|
||||
}
|
||||
v := reflect.ValueOf(&base).Elem()
|
||||
if v.Kind() != reflect.Struct {
|
||||
return zero, errors.New("config: project configuration must be a struct")
|
||||
}
|
||||
result := reflect.New(v.Type()).Elem()
|
||||
result.Set(v)
|
||||
seen := map[string]bool{}
|
||||
var walk func(reflect.Value, string, int) error
|
||||
walk = func(value reflect.Value, prefix string, depth int) error {
|
||||
if depth > 32 {
|
||||
return errors.New("config: nesting exceeds 32")
|
||||
}
|
||||
for i := 0; i < value.NumField(); i++ {
|
||||
field := value.Type().Field(i)
|
||||
dst := value.Field(i)
|
||||
if !dst.CanSet() || field.PkgPath != "" {
|
||||
continue
|
||||
}
|
||||
tag := strings.Split(field.Tag.Get("env"), ",")
|
||||
if tag[0] == "-" {
|
||||
continue
|
||||
}
|
||||
jsonName := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||
if jsonName == "-" && tag[0] == "" {
|
||||
continue
|
||||
}
|
||||
if jsonName == "" {
|
||||
jsonName = field.Name
|
||||
}
|
||||
key := prefix + envName(jsonName)
|
||||
if tag[0] != "" {
|
||||
key = tag[0]
|
||||
}
|
||||
empty, required := options.Empty, false
|
||||
for _, opt := range tag[1:] {
|
||||
switch opt {
|
||||
case "required":
|
||||
required = true
|
||||
case "secret", "":
|
||||
case "empty=allow":
|
||||
empty = "allow"
|
||||
case "empty=ignore":
|
||||
empty = "ignore"
|
||||
case "empty=reject":
|
||||
empty = "reject"
|
||||
default:
|
||||
return fmt.Errorf("config: invalid env tag for %s", key)
|
||||
}
|
||||
}
|
||||
if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) {
|
||||
if err := walk(dst, key+"_", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if dst.Kind() == reflect.Pointer && dst.Type().Elem().Kind() == reflect.Struct && !dst.Type().Implements(textType) {
|
||||
// Clone before traversing so a failed overlay never edits defaults.
|
||||
child := reflect.New(dst.Type().Elem())
|
||||
if !dst.IsNil() {
|
||||
child.Elem().Set(dst.Elem())
|
||||
}
|
||||
if err := walk(child.Elem(), key+"_", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
if !dst.IsNil() || !child.Elem().IsZero() {
|
||||
dst.Set(child)
|
||||
}
|
||||
if required && !defaults && dst.IsNil() {
|
||||
return fmt.Errorf("config: %s is required", key)
|
||||
}
|
||||
continue
|
||||
}
|
||||
// Unexported runtime capabilities are skipped unless explicitly tagged.
|
||||
if dst.Kind() == reflect.Func || dst.Kind() == reflect.Chan || dst.Kind() == reflect.Interface {
|
||||
if tag[0] != "" {
|
||||
return fmt.Errorf("config: %s has unsupported environment type", key)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if seen[key] {
|
||||
return fmt.Errorf("config: duplicate environment key %s", key)
|
||||
}
|
||||
seen[key] = true
|
||||
raw, present := options.Lookup(key)
|
||||
if defaults {
|
||||
raw, present = field.Tag.Lookup("envDefault")
|
||||
present = present && dst.IsZero()
|
||||
}
|
||||
if present && raw == "" && !defaults {
|
||||
if empty == "reject" {
|
||||
return fmt.Errorf("config: %s must not be empty", key)
|
||||
}
|
||||
if empty == "ignore" {
|
||||
present = false
|
||||
}
|
||||
}
|
||||
if present {
|
||||
parsed := reflect.New(dst.Type()).Elem()
|
||||
if err := parseEnvValue(parsed, raw); err != nil {
|
||||
return fmt.Errorf("config: %s must be a valid %s", key, dst.Type())
|
||||
}
|
||||
dst.Set(parsed)
|
||||
}
|
||||
if required && !defaults && emptyRequired(dst) {
|
||||
return fmt.Errorf("config: %s is required", key)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := walk(result, options.Prefix, 0); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result.Interface().(T), nil
|
||||
}
|
||||
|
||||
func emptyRequired(v reflect.Value) bool {
|
||||
switch v.Kind() {
|
||||
case reflect.String:
|
||||
return v.Len() == 0
|
||||
case reflect.Pointer, reflect.Map, reflect.Slice:
|
||||
return v.IsNil()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func parseEnvValue(v reflect.Value, raw string) (err error) {
|
||||
defer func() {
|
||||
if recover() != nil {
|
||||
err = errors.New("invalid custom value")
|
||||
}
|
||||
}()
|
||||
if v.Kind() == reflect.Pointer {
|
||||
v.Set(reflect.New(v.Type().Elem()))
|
||||
return parseEnvValue(v.Elem(), raw)
|
||||
}
|
||||
if v.CanAddr() && v.Addr().Type().Implements(textType) {
|
||||
return v.Addr().Interface().(encoding.TextUnmarshaler).UnmarshalText([]byte(raw))
|
||||
}
|
||||
if v.Type() == durationType {
|
||||
d, err := time.ParseDuration(raw)
|
||||
if err == nil {
|
||||
v.SetInt(int64(d))
|
||||
}
|
||||
return err
|
||||
}
|
||||
switch v.Kind() {
|
||||
case reflect.String:
|
||||
v.SetString(raw)
|
||||
return nil
|
||||
case reflect.Bool:
|
||||
x, err := strconv.ParseBool(raw)
|
||||
if err == nil {
|
||||
v.SetBool(x)
|
||||
}
|
||||
return err
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
|
||||
x, err := strconv.ParseInt(raw, 10, v.Type().Bits())
|
||||
if err == nil {
|
||||
v.SetInt(x)
|
||||
}
|
||||
return err
|
||||
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
x, err := strconv.ParseUint(raw, 10, v.Type().Bits())
|
||||
if err == nil {
|
||||
v.SetUint(x)
|
||||
}
|
||||
return err
|
||||
case reflect.Float32, reflect.Float64:
|
||||
x, err := strconv.ParseFloat(raw, v.Type().Bits())
|
||||
if err == nil && (math.IsNaN(x) || math.IsInf(x, 0)) {
|
||||
return errors.New("nonfinite value")
|
||||
}
|
||||
if err == nil {
|
||||
v.SetFloat(x)
|
||||
}
|
||||
return err
|
||||
case reflect.Slice, reflect.Map:
|
||||
var value any
|
||||
if err := apierrors.DecodeCatalogJSON([]byte(raw), &value); err != nil {
|
||||
return err
|
||||
}
|
||||
value, err := durations(value, v.Type())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(encoded))
|
||||
decoder.DisallowUnknownFields()
|
||||
return decoder.Decode(v.Addr().Interface())
|
||||
}
|
||||
return errors.New("unsupported environment value")
|
||||
}
|
||||
|
||||
func envName(s string) string {
|
||||
r := []rune(s)
|
||||
var out strings.Builder
|
||||
for i, c := range r {
|
||||
if i > 0 && unicode.IsUpper(c) && (unicode.IsLower(r[i-1]) || unicode.IsDigit(r[i-1]) || (i+1 < len(r) && unicode.IsLower(r[i+1]))) {
|
||||
out.WriteByte('_')
|
||||
}
|
||||
if c == '-' {
|
||||
c = '_'
|
||||
}
|
||||
out.WriteRune(unicode.ToUpper(c))
|
||||
}
|
||||
return out.String()
|
||||
}
|
||||
61
config/projectAdvanced_test.go
Normal file
61
config/projectAdvanced_test.go
Normal file
@@ -0,0 +1,61 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestConfigFilePreservesDefaultsWithoutSharingMutableData(t *testing.T) {
|
||||
type cfg struct {
|
||||
Writer io.Writer `json:"-"`
|
||||
Secret string `json:"-"`
|
||||
Values map[string]int64 `json:"values"`
|
||||
Timers map[string]time.Duration `json:"timers"`
|
||||
ID int64 `json:"id"`
|
||||
}
|
||||
var writer bytes.Buffer
|
||||
base := cfg{Writer: &writer, Secret: "borrowedSecret", Values: map[string]int64{"old": 7}, ID: 9007199254740993}
|
||||
file := filepath.Join(t.TempDir(), "config.json")
|
||||
if err := os.WriteFile(file, []byte(`{"values":{"new":2},"timers":{"retry":"2s"}}`), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
next, err := DecodeFile(file, base)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if next.Writer != &writer || next.Secret != base.Secret || next.ID != base.ID || next.Timers["retry"] != 2*time.Second || len(base.Values) != 1 {
|
||||
t.Fatal("defaults lost or mutated")
|
||||
}
|
||||
next.Values["old"] = 9
|
||||
if base.Values["old"] != 7 {
|
||||
t.Fatal("defaults shared map")
|
||||
}
|
||||
if err = os.WriteFile(file, []byte(`{"values":{"secretValue":"credential-super-secret"}}`), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = DecodeFile(file, base); err == nil || strings.Contains(err.Error(), "credential-super-secret") {
|
||||
t.Fatal("decode error leaked contents", err)
|
||||
}
|
||||
}
|
||||
func TestEnvCollectionPrecisionAndDurationMaps(t *testing.T) {
|
||||
type cfg struct {
|
||||
IDs []int64 `json:"ids"`
|
||||
Timers map[string]time.Duration `json:"timers"`
|
||||
}
|
||||
lookup := func(key string) (string, bool) {
|
||||
v, ok := map[string]string{"IDS": "[9007199254740993]", "TIMERS": `{"backoff":"250ms"}`}[key]
|
||||
return v, ok
|
||||
}
|
||||
c, err := OverlayEnv(cfg{}, EnvOptions{Lookup: lookup})
|
||||
if err != nil || c.IDs[0] != 9007199254740993 || c.Timers["backoff"] != 250*time.Millisecond {
|
||||
t.Fatal(c, err)
|
||||
}
|
||||
if _, err = OverlayEnv(cfg{}, EnvOptions{Lookup: func(key string) (string, bool) { return `{"x":"1s","x":"2s"}`, key == "TIMERS" }}); err == nil {
|
||||
t.Fatal("duplicate map key accepted")
|
||||
}
|
||||
}
|
||||
124
config/project_test.go
Normal file
124
config/project_test.go
Normal file
@@ -0,0 +1,124 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type projectNested struct {
|
||||
ReadTimeout time.Duration `json:"readTimeout"`
|
||||
Names []string `json:"names"`
|
||||
}
|
||||
type projectConfig struct {
|
||||
Nested *projectNested `json:"nested"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Count int `json:"count" envDefault:"12"`
|
||||
Password string `json:"-" env:"ETAX_PASSWORD,secret,required,empty=reject"`
|
||||
Label string `json:"label" env:",empty=ignore"`
|
||||
}
|
||||
|
||||
func (c projectConfig) Validate() error {
|
||||
if c.Count < 0 {
|
||||
return errors.New("count must be nonnegative")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func TestProjectConfigurationOrderAndValues(t *testing.T) {
|
||||
f := filepath.Join(t.TempDir(), "config.yaml")
|
||||
if err := os.WriteFile(f, []byte("count: 3\nenabled: true\nnested:\n readTimeout: 2s\n names: [file]\nlabel: keep\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
env := map[string]string{"ETAX_PASSWORD": " password ", "APP_ENABLED": "false", "APP_COUNT": "0", "APP_NESTED_READ_TIMEOUT": "7s", "APP_NESTED_NAMES": "[]", "APP_LABEL": ""}
|
||||
options := EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
|
||||
cfg, err := LoadProject(f, projectConfig{}, options)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Enabled || cfg.Count != 0 || cfg.Password != " password " || cfg.Nested.ReadTimeout != 7*time.Second || cfg.Nested.Names == nil || len(cfg.Nested.Names) != 0 || cfg.Label != "keep" {
|
||||
t.Fatal("overlay mismatch")
|
||||
}
|
||||
delete(env, "APP_COUNT")
|
||||
cfg, err = LoadProject("", projectConfig{}, options)
|
||||
if err != nil || cfg.Count != 12 {
|
||||
t.Fatal(cfg.Count, err)
|
||||
}
|
||||
env["APP_COUNT"] = "-1"
|
||||
if _, err = LoadProject(f, projectConfig{}, options); err == nil {
|
||||
t.Fatal("project Validate not called")
|
||||
}
|
||||
}
|
||||
|
||||
func TestProjectOverlayIsTransactionalAndRedacted(t *testing.T) {
|
||||
base := projectConfig{Nested: &projectNested{ReadTimeout: time.Second, Names: []string{"old"}}, Password: "original"}
|
||||
env := map[string]string{"APP_NESTED_READ_TIMEOUT": "2s", "APP_NESTED_NAMES": "[\"new\"]", "APP_COUNT": "secret-value"}
|
||||
_, err := OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }})
|
||||
if err == nil || strings.Contains(err.Error(), "secret-value") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if base.Nested.ReadTimeout != time.Second || !reflect.DeepEqual(base.Nested.Names, []string{"old"}) {
|
||||
t.Fatal("default pointer modified")
|
||||
}
|
||||
delete(env, "APP_COUNT")
|
||||
env["ETAX_PASSWORD"] = ""
|
||||
if _, err = OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}); err == nil {
|
||||
t.Fatal("empty secret accepted")
|
||||
}
|
||||
type required struct {
|
||||
Token string `env:"TOKEN,required"`
|
||||
}
|
||||
if _, err = OverlayEnv(required{}, EnvOptions{Lookup: func(string) (string, bool) { return "", false }}); err == nil {
|
||||
t.Fatal("required absent")
|
||||
}
|
||||
type duplicate struct {
|
||||
A string `env:"SAME"`
|
||||
B string `env:"SAME"`
|
||||
}
|
||||
if _, err = OverlayEnv(duplicate{}, EnvOptions{}); err == nil {
|
||||
t.Fatal("duplicate env binding accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnvironmentCollectionAndEmptySemantics(t *testing.T) {
|
||||
type sample struct {
|
||||
Values []time.Duration
|
||||
Flag bool
|
||||
Text string
|
||||
Optional *int
|
||||
}
|
||||
cases := []struct {
|
||||
raw string
|
||||
valid bool
|
||||
}{{`["1s","2s"]`, true}, {`["invalid-secret"]`, false}}
|
||||
for _, tc := range cases {
|
||||
cfg, err := OverlayEnv(sample{}, EnvOptions{Lookup: func(k string) (string, bool) {
|
||||
if k == "VALUES" {
|
||||
return tc.raw, true
|
||||
}
|
||||
return "", false
|
||||
}})
|
||||
if (err == nil) != tc.valid {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tc.valid && !reflect.DeepEqual(cfg.Values, []time.Duration{time.Second, 2 * time.Second}) {
|
||||
t.Fatal(cfg)
|
||||
}
|
||||
if err != nil && strings.Contains(err.Error(), "secret") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, mode := range []string{"allow", "ignore", "reject"} {
|
||||
cfg, err := OverlayEnv(sample{Text: "old"}, EnvOptions{Empty: mode, Lookup: func(k string) (string, bool) { return "", k == "TEXT" }})
|
||||
if mode == "reject" {
|
||||
if err == nil {
|
||||
t.Fatal("empty not rejected")
|
||||
}
|
||||
} else if err != nil || (mode == "allow" && cfg.Text != "") || (mode == "ignore" && cfg.Text != "old") {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
172
contract/bind.go
172
contract/bind.go
@@ -1,6 +1,7 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
@@ -11,6 +12,7 @@ import (
|
||||
"net/url"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
)
|
||||
@@ -19,9 +21,17 @@ import (
|
||||
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
|
||||
// parameters never overwrite path/body/header values. form means URL query only.
|
||||
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
|
||||
return b.BindWithPolicy(r, params, target, RequestPolicy{}, maxBytes...)
|
||||
}
|
||||
|
||||
// BindWithPolicy merges route overrides over the App request policy.
|
||||
func (b *Binder) BindWithPolicy(r *http.Request, params map[string]string, target any, policy RequestPolicy, maxBytes ...int64) error {
|
||||
if b == nil || r == nil || b.schemaOnly {
|
||||
return apierrors.Internal(errors.New("contract: nil binder or request"))
|
||||
}
|
||||
if b.HasMultipart() {
|
||||
return apierrors.Internal(errors.New("multipart fields require BindMultipart"))
|
||||
}
|
||||
v := reflect.ValueOf(target)
|
||||
if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ {
|
||||
return apierrors.Internal(errors.New("contract: target type mismatch"))
|
||||
@@ -33,54 +43,9 @@ func (b *Binder) Bind(r *http.Request, params map[string]string, target any, max
|
||||
if len(maxBytes) == 1 {
|
||||
limit = maxBytes[0]
|
||||
}
|
||||
fresh := reflect.New(b.typ).Elem()
|
||||
body := map[string]any{}
|
||||
if len(b.root.schema.Properties) > 0 {
|
||||
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
|
||||
if err != nil || ct != "application/json" {
|
||||
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json")
|
||||
}
|
||||
if r.Body == nil {
|
||||
return invalid(errors.New("body required"))
|
||||
}
|
||||
if r.ContentLength > limit {
|
||||
return tooLarge()
|
||||
}
|
||||
reader := &io.LimitedReader{R: r.Body, N: limit + 1}
|
||||
dec := json.NewDecoder(reader)
|
||||
dec.UseNumber()
|
||||
value, err := decodeValue(dec, 0)
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != nil {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) {
|
||||
return tooLarge()
|
||||
}
|
||||
return invalid(err)
|
||||
}
|
||||
_, err = dec.Token()
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != io.EOF {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) {
|
||||
return tooLarge()
|
||||
}
|
||||
return invalid(errors.New("exactly one JSON value required"))
|
||||
}
|
||||
var ok bool
|
||||
body, ok = value.(map[string]any)
|
||||
if !ok {
|
||||
return invalid(errors.New("body must be an object"))
|
||||
}
|
||||
for key := range body {
|
||||
if _, ok = b.root.schema.Properties[key]; !ok {
|
||||
return invalid(fmt.Errorf("unknown body field %s", key))
|
||||
}
|
||||
}
|
||||
policy = MergePolicy(RequestPolicyFromContext(r.Context()), policy)
|
||||
if err := policy.Validate(); err != nil {
|
||||
return apierrors.Internal(err)
|
||||
}
|
||||
query := url.Values{}
|
||||
if r.URL != nil {
|
||||
@@ -90,10 +55,121 @@ func (b *Binder) Bind(r *http.Request, params map[string]string, target any, max
|
||||
return invalid(errors.New("malformed query"))
|
||||
}
|
||||
}
|
||||
if policy.UnknownQuery == "reject" {
|
||||
allowed := map[string]bool{}
|
||||
for _, f := range b.root.fields {
|
||||
if f.source == "form" {
|
||||
allowed[f.name] = true
|
||||
}
|
||||
}
|
||||
for key := range query {
|
||||
if !allowed[key] {
|
||||
return invalid(fmt.Errorf("unknown query field %s", key))
|
||||
}
|
||||
}
|
||||
}
|
||||
fresh := reflect.New(b.typ).Elem()
|
||||
body := map[string]any{}
|
||||
bodyAbsent := false
|
||||
bodyMode := policy.Body
|
||||
if bodyMode == "" || bodyMode == "auto" {
|
||||
if len(b.root.schema.Properties) > 0 {
|
||||
bodyMode = "required"
|
||||
} else {
|
||||
bodyMode = "auto"
|
||||
}
|
||||
}
|
||||
if (bodyMode == "none" || bodyMode == "emptyObject") && len(b.root.schema.Properties) > 0 {
|
||||
return apierrors.Internal(errors.New("request body policy conflicts with JSON fields"))
|
||||
}
|
||||
if bodyMode == "none" {
|
||||
if r.Body != nil {
|
||||
var one [1]byte
|
||||
n, err := io.ReadFull(r.Body, one[:])
|
||||
if n > 0 {
|
||||
return invalid(errors.New("request body is forbidden"))
|
||||
}
|
||||
if err != io.EOF {
|
||||
return invalid(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if bodyMode == "required" || bodyMode == "optional" || bodyMode == "emptyObject" {
|
||||
var bodyReader io.Reader = r.Body
|
||||
if bodyReader == nil {
|
||||
bodyReader = strings.NewReader("")
|
||||
}
|
||||
if r.ContentLength > limit {
|
||||
return tooLarge()
|
||||
}
|
||||
reader := &io.LimitedReader{R: bodyReader, N: limit + 1}
|
||||
var checked io.Reader = reader
|
||||
if policy.UTF8 == "reject" {
|
||||
checked = &utf8Reader{r: checked}
|
||||
}
|
||||
buffered := bufio.NewReader(checked)
|
||||
_, peekErr := buffered.Peek(1)
|
||||
absent := peekErr == io.EOF
|
||||
bodyAbsent = absent
|
||||
if absent && bodyMode != "optional" {
|
||||
return invalid(errors.New("body required"))
|
||||
}
|
||||
if peekErr != nil && !absent {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(peekErr, &maxErr) {
|
||||
return tooLarge()
|
||||
}
|
||||
return invalid(peekErr)
|
||||
}
|
||||
if !absent {
|
||||
ct, mediaParams, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
|
||||
if err != nil || ct != "application/json" || (policy.Charset == "utf8" && mediaParams["charset"] != "" && !strings.EqualFold(mediaParams["charset"], "utf-8")) {
|
||||
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json with an allowed charset")
|
||||
}
|
||||
dec := json.NewDecoder(buffered)
|
||||
dec.UseNumber()
|
||||
value, err := decodeValue(dec, 0)
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != nil {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) {
|
||||
return tooLarge()
|
||||
}
|
||||
return invalid(err)
|
||||
}
|
||||
_, err = dec.Token()
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != io.EOF {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) {
|
||||
return tooLarge()
|
||||
}
|
||||
return invalid(errors.New("exactly one JSON value required"))
|
||||
}
|
||||
var ok bool
|
||||
body, ok = value.(map[string]any)
|
||||
if !ok {
|
||||
return invalid(errors.New("body must be an object"))
|
||||
}
|
||||
for key := range body {
|
||||
if _, ok = b.root.schema.Properties[key]; !ok {
|
||||
return invalid(fmt.Errorf("unknown body field %s", key))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for _, f := range b.root.fields {
|
||||
var value any
|
||||
var present bool
|
||||
if f.source == "json" {
|
||||
if bodyAbsent && bodyMode == "optional" {
|
||||
continue
|
||||
}
|
||||
value, present = body[f.name]
|
||||
} else {
|
||||
var raw []string
|
||||
|
||||
@@ -36,6 +36,8 @@ type node struct {
|
||||
}
|
||||
type field struct {
|
||||
index int
|
||||
maxBytes int64
|
||||
maxFiles int
|
||||
name, source string
|
||||
value *node
|
||||
optional bool
|
||||
@@ -113,6 +115,13 @@ func (c compiler) compile(t reflect.Type, root bool, stack map[reflect.Type]bool
|
||||
if t.Kind() != reflect.Pointer && (t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler)) {
|
||||
return nil, errors.New("contract: custom JSON marshalers require explicit Scalar registration")
|
||||
}
|
||||
if t == uploadType {
|
||||
if root {
|
||||
return nil, errors.New("upload File cannot be a root DTO")
|
||||
}
|
||||
n.schema = &openapi.Schema{Type: "string", Format: "binary"}
|
||||
return n, nil
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Interface:
|
||||
if t.NumMethod() != 0 {
|
||||
@@ -223,7 +232,7 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
|
||||
var sourceTag string
|
||||
for key, value := range tags {
|
||||
switch key {
|
||||
case "json", "path", "form", "header":
|
||||
case "json", "path", "form", "header", "file", "multipart", "multipartJSON":
|
||||
if value == "-" {
|
||||
return f, errors.New("ignored fields are not part of an API contract")
|
||||
}
|
||||
@@ -232,6 +241,8 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
|
||||
}
|
||||
f.source = key
|
||||
sourceTag = value
|
||||
case "description", "example", "enumDescriptions":
|
||||
// Documentation is validated after the field constraints.
|
||||
case "binding":
|
||||
f.binding = value
|
||||
default:
|
||||
@@ -254,6 +265,45 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
|
||||
return f, err
|
||||
}
|
||||
f.value = n
|
||||
if f.source == "file" {
|
||||
if sf.Type != uploadType && (sf.Type.Kind() != reflect.Slice || sf.Type.Elem() != uploadType) {
|
||||
return f, errors.New("file field requires File or []File")
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, option := range parts[1:] {
|
||||
key, v, _ := strings.Cut(option, "=")
|
||||
if seen[key] {
|
||||
return f, errors.New("duplicate file option")
|
||||
}
|
||||
seen[key] = true
|
||||
switch key {
|
||||
case "optional":
|
||||
if option != key {
|
||||
return f, errors.New("optional takes no value")
|
||||
}
|
||||
f.optional = true
|
||||
case "maxBytes":
|
||||
f.maxBytes, err = fieldLimit(v)
|
||||
if err != nil {
|
||||
return f, err
|
||||
}
|
||||
case "maxFiles":
|
||||
f.maxFiles, err = strconv.Atoi(v)
|
||||
if err != nil || f.maxFiles < 1 || f.maxFiles > 10000 || sf.Type.Kind() != reflect.Slice {
|
||||
return f, errors.New("maxFiles requires []File and a limit 1..10000")
|
||||
}
|
||||
default:
|
||||
return f, fmt.Errorf("unsupported file option %q", key)
|
||||
}
|
||||
}
|
||||
if f.binding != "" {
|
||||
return f, errors.New("file constraints use optional/maxBytes/maxFiles")
|
||||
}
|
||||
return f, applyDocumentation(&f, tags)
|
||||
}
|
||||
if containsUpload(n) {
|
||||
return f, errors.New("File requires file source")
|
||||
}
|
||||
opaque := base(n).scalar
|
||||
isList := base(n).typ.Kind() == reflect.Slice && opaque == nil
|
||||
if isList {
|
||||
@@ -263,25 +313,35 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
|
||||
if isList && (f.source == "path" || f.source == "header") {
|
||||
return f, errors.New("path/header values must be scalars")
|
||||
}
|
||||
if f.source != "json" && !opaque.Text {
|
||||
if !jsonSource(f.source) && !opaque.Text {
|
||||
return f, errors.New("custom scalar needs text: true and UnmarshalText for path/query/header")
|
||||
}
|
||||
if f.binding != "" {
|
||||
return f, errors.New("custom scalar constraints belong in its decoder; use optional/pointer for presence and null")
|
||||
}
|
||||
for _, option := range parts[1:] {
|
||||
if strings.HasPrefix(option, "maxBytes=") && multipartSource(f.source) {
|
||||
f.maxBytes, err = fieldLimit(strings.TrimPrefix(option, "maxBytes="))
|
||||
if err != nil {
|
||||
return f, err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if option != "optional" && option != "omitempty" {
|
||||
return f, errors.New("custom scalar supports only optional/omitempty field options")
|
||||
return f, errors.New("custom scalar supports optional/omitempty and multipart maxBytes")
|
||||
}
|
||||
f.optional = true
|
||||
}
|
||||
if f.source == "path" && (f.optional || sf.Type.Kind() == reflect.Pointer) {
|
||||
return f, errors.New("path fields must be required non-pointer values")
|
||||
}
|
||||
if err := applyDocumentation(&f, tags); err != nil {
|
||||
return f, err
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
scalar := base(n).typ.Kind()
|
||||
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
|
||||
if !jsonSource(f.source) && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
|
||||
return f, errors.New("path/query/header values must be scalars or scalar slices")
|
||||
}
|
||||
if f.source == "path" && !isScalar(scalar) {
|
||||
@@ -303,6 +363,14 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
|
||||
return f, errors.New("optional takes no value")
|
||||
}
|
||||
f.optional = true
|
||||
case "maxBytes":
|
||||
if !multipartSource(f.source) {
|
||||
return f, errors.New("maxBytes field option is only for multipart")
|
||||
}
|
||||
f.maxBytes, err = fieldLimit(value)
|
||||
if err != nil {
|
||||
return f, err
|
||||
}
|
||||
case "default":
|
||||
if !isScalar(scalar) {
|
||||
return f, errors.New("default requires a scalar")
|
||||
@@ -354,6 +422,9 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl
|
||||
return f, fmt.Errorf("default violates validation: %w", err)
|
||||
}
|
||||
}
|
||||
if err := applyDocumentation(&f, tags); err != nil {
|
||||
return f, err
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
|
||||
@@ -432,7 +503,7 @@ func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
|
||||
func (b *Binder) Parameters() []openapi.Parameter {
|
||||
var out []openapi.Parameter
|
||||
for _, f := range b.root.fields {
|
||||
if f.source == "json" {
|
||||
if f.source == "json" || multipartSource(f.source) {
|
||||
continue
|
||||
}
|
||||
in := f.source
|
||||
@@ -445,7 +516,7 @@ func (b *Binder) Parameters() []openapi.Parameter {
|
||||
} else if base(f.value).typ.Kind() == reflect.Slice && base(base(f.value).elem).scalar != nil {
|
||||
schema.Items = &openapi.Schema{Type: "string"}
|
||||
}
|
||||
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema})
|
||||
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema, Description: f.value.schema.Description})
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -455,3 +526,10 @@ func copySchema(s *openapi.Schema) *openapi.Schema {
|
||||
_ = json.Unmarshal(data, &out)
|
||||
return &out
|
||||
}
|
||||
|
||||
func containsUpload(n *node) bool {
|
||||
if n.typ == uploadType {
|
||||
return true
|
||||
}
|
||||
return n.elem != nil && containsUpload(n.elem)
|
||||
}
|
||||
|
||||
174
contract/documentation.go
Normal file
174
contract/documentation.go
Normal file
@@ -0,0 +1,174 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Documentation is part of the contract compiler, so CLI and runtime agree.
|
||||
func applyDocumentation(f *field, tags map[string]string) error {
|
||||
s := f.value.schema
|
||||
if f.maxBytes > 0 {
|
||||
v := f.maxBytes
|
||||
s.MaxBytes = &v
|
||||
}
|
||||
if f.source == "file" && f.value.typ.Kind() == reflect.Slice {
|
||||
v := f.maxFiles
|
||||
if v == 0 {
|
||||
v = 16
|
||||
}
|
||||
s.MaxItems = &v
|
||||
}
|
||||
s.Description = tags["description"]
|
||||
if raw, ok := tags["enumDescriptions"]; ok {
|
||||
if err := json.Unmarshal([]byte(raw), &s.EnumDescriptions); err != nil {
|
||||
return errors.New("enumDescriptions must be a JSON string array")
|
||||
}
|
||||
count := len(s.Enum)
|
||||
if s.Nullable && count > 0 && s.Enum[count-1] == nil {
|
||||
count--
|
||||
}
|
||||
if count == 0 || len(s.EnumDescriptions) != count {
|
||||
return errors.New("enumDescriptions must describe every declared enum value")
|
||||
}
|
||||
for _, v := range s.EnumDescriptions {
|
||||
if strings.TrimSpace(v) == "" {
|
||||
return errors.New("enum description cannot be empty")
|
||||
}
|
||||
}
|
||||
}
|
||||
if raw, ok := tags["example"]; ok {
|
||||
var value any
|
||||
if base(f.value).typ.Kind() == reflect.String && base(f.value).scalar == nil {
|
||||
value = raw
|
||||
} else {
|
||||
decoder := json.NewDecoder(strings.NewReader(raw))
|
||||
decoder.UseNumber()
|
||||
var err error
|
||||
value, err = decodeValue(decoder, 0)
|
||||
if err != nil {
|
||||
return errors.New("invalid example JSON")
|
||||
}
|
||||
if _, err = decoder.Token(); err != io.EOF {
|
||||
return errors.New("example requires one value")
|
||||
}
|
||||
}
|
||||
// A schema-only custom scalar never executes project decoders.
|
||||
if !documentationHasScalar(f.value) {
|
||||
v := reflect.New(f.value.typ).Elem()
|
||||
if err := setValue(f.value, v, value); err != nil {
|
||||
return fmt.Errorf("example: %w", err)
|
||||
}
|
||||
if err := checkField(*f, v); err != nil {
|
||||
return fmt.Errorf("example: %w", err)
|
||||
}
|
||||
} else if err := checkDocumentationScalarTree(f.value, value); err != nil {
|
||||
return err
|
||||
}
|
||||
s.Example = value
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func documentationHasScalar(n *node) bool {
|
||||
if n.scalar != nil {
|
||||
return true
|
||||
}
|
||||
if n.elem != nil && documentationHasScalar(n.elem) {
|
||||
return true
|
||||
}
|
||||
for _, f := range n.fields {
|
||||
if documentationHasScalar(f.value) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Metadata never invokes application decoders, including nested scalar types.
|
||||
func checkDocumentationScalarTree(n *node, value any) error {
|
||||
if value == nil {
|
||||
if n.typ.Kind() == reflect.Pointer || n.typ.Kind() == reflect.Interface {
|
||||
return nil
|
||||
}
|
||||
return errors.New("example null requires a nullable field")
|
||||
}
|
||||
if n.typ.Kind() == reflect.Pointer {
|
||||
return checkDocumentationScalarTree(n.elem, value)
|
||||
}
|
||||
if n.scalar != nil {
|
||||
for _, kind := range strings.Split(n.scalar.JSONInput, "|") {
|
||||
if scalarAccepts(strings.TrimSpace(kind), value) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return errors.New("example does not match scalar input")
|
||||
}
|
||||
if !documentationHasScalar(n) {
|
||||
v := reflect.New(n.typ).Elem()
|
||||
return setValue(n, v, value)
|
||||
}
|
||||
switch n.typ.Kind() {
|
||||
case reflect.Slice:
|
||||
values, ok := value.([]any)
|
||||
if !ok {
|
||||
return errors.New("example requires an array")
|
||||
}
|
||||
if n.schema.MinItems != nil && len(values) < *n.schema.MinItems || n.schema.MaxItems != nil && len(values) > *n.schema.MaxItems {
|
||||
return errors.New("example violates item limits")
|
||||
}
|
||||
for _, v := range values {
|
||||
if err := checkDocumentationScalarTree(n.elem, v); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case reflect.Map:
|
||||
values, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return errors.New("example requires an object")
|
||||
}
|
||||
for _, v := range values {
|
||||
if err := checkDocumentationScalarTree(n.elem, v); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case reflect.Struct:
|
||||
values, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return errors.New("example requires an object")
|
||||
}
|
||||
declared := map[string]bool{}
|
||||
for _, f := range n.fields {
|
||||
declared[f.name] = true
|
||||
v, ok := values[f.name]
|
||||
if !ok {
|
||||
if !f.optional && !f.hasDefault {
|
||||
return errors.New("example missing field")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if err := checkDocumentationScalarTree(f.value, v); err != nil {
|
||||
return err
|
||||
}
|
||||
if !documentationHasScalar(f.value) {
|
||||
typed := reflect.New(f.value.typ).Elem()
|
||||
if err := setValue(f.value, typed, v); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := checkField(f, typed); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
for name := range values {
|
||||
if !declared[name] {
|
||||
return errors.New("unknown example field")
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
55
contract/documentation_test.go
Normal file
55
contract/documentation_test.go
Normal file
@@ -0,0 +1,55 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"reflect"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type documentationScalar string
|
||||
|
||||
var documentationDecodeCalls int
|
||||
|
||||
func (v documentationScalar) MarshalJSON() ([]byte, error) { return json.Marshal(string(v)) }
|
||||
func (v *documentationScalar) UnmarshalJSON(raw []byte) error {
|
||||
documentationDecodeCalls++
|
||||
var s string
|
||||
err := json.Unmarshal(raw, &s)
|
||||
*v = documentationScalar(s)
|
||||
return err
|
||||
}
|
||||
func TestDocumentationNeverRunsNestedScalarDecoders(t *testing.T) {
|
||||
type nested struct {
|
||||
ID documentationScalar `json:"id"`
|
||||
}
|
||||
type input struct {
|
||||
Value nested `json:"value" example:"{\"id\":\"123\"}"`
|
||||
}
|
||||
documentationDecodeCalls = 0
|
||||
scalar := ScalarOf[documentationScalar]("string", "string", false)
|
||||
if _, err := New[input](scalar); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Describe(reflect.TypeOf(input{}), scalar); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if documentationDecodeCalls != 0 {
|
||||
t.Fatal("metadata invoked business decoder")
|
||||
}
|
||||
type bad struct {
|
||||
Value nested `json:"value" example:"{\"id\":123}"`
|
||||
}
|
||||
if _, err := Describe(reflect.TypeOf(bad{}), scalar); err == nil {
|
||||
t.Fatal("invalid scalar wire example accepted")
|
||||
}
|
||||
type constrained struct {
|
||||
ID documentationScalar `json:"id"`
|
||||
Count int `json:"count,range=[1:3]"`
|
||||
}
|
||||
type invalidConstraint struct {
|
||||
Value constrained `json:"value" example:"{\"id\":\"123\",\"count\":9}"`
|
||||
}
|
||||
if _, err := Describe(reflect.TypeOf(invalidConstraint{}), scalar); err == nil {
|
||||
t.Fatal("nested scalar bypassed another field's example constraints")
|
||||
}
|
||||
}
|
||||
80
contract/file.go
Normal file
80
contract/file.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// File is a request-scoped upload, staged on disk using a bounded copy buffer.
|
||||
// Name and ContentType are untrusted client metadata. Open streams content;
|
||||
// readers and the temporary file are closed when the generated handler returns.
|
||||
// Persist wanted content before returning. Retaining File does not extend its lifetime.
|
||||
type File struct {
|
||||
Name string
|
||||
ContentType string
|
||||
Size int64
|
||||
state *uploadState
|
||||
}
|
||||
type uploadState struct {
|
||||
mu sync.Mutex
|
||||
path string
|
||||
closed bool
|
||||
readers map[*uploadReader]bool
|
||||
}
|
||||
type uploadReader struct {
|
||||
file *os.File
|
||||
once sync.Once
|
||||
err error
|
||||
state *uploadState
|
||||
}
|
||||
|
||||
// Present reports whether the optional upload was supplied.
|
||||
func (f File) Present() bool { return f.state != nil }
|
||||
|
||||
func (f File) Open() (io.ReadCloser, error) {
|
||||
if f.state == nil {
|
||||
return nil, errors.New("upload file is absent")
|
||||
}
|
||||
s := f.state
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if s.closed {
|
||||
return nil, errors.New("upload file is closed")
|
||||
}
|
||||
file, err := os.Open(s.path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
r := &uploadReader{file: file, state: s}
|
||||
s.readers[r] = true
|
||||
return r, nil
|
||||
}
|
||||
func (r *uploadReader) Read(p []byte) (int, error) { return r.file.Read(p) }
|
||||
func (r *uploadReader) Close() error {
|
||||
r.once.Do(func() { r.err = r.file.Close(); r.state.mu.Lock(); delete(r.state.readers, r); r.state.mu.Unlock() })
|
||||
return r.err
|
||||
}
|
||||
func (f File) cleanup() error {
|
||||
if f.state == nil {
|
||||
return nil
|
||||
}
|
||||
s := f.state
|
||||
s.mu.Lock()
|
||||
if s.closed {
|
||||
s.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
s.closed = true
|
||||
readers := make([]*uploadReader, 0, len(s.readers))
|
||||
for r := range s.readers {
|
||||
readers = append(readers, r)
|
||||
}
|
||||
s.mu.Unlock()
|
||||
var err error
|
||||
for _, r := range readers {
|
||||
err = errors.Join(err, r.Close())
|
||||
}
|
||||
return errors.Join(err, os.Remove(s.path))
|
||||
}
|
||||
331
contract/multipart.go
Normal file
331
contract/multipart.go
Normal file
@@ -0,0 +1,331 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"mime"
|
||||
"net/http"
|
||||
"os"
|
||||
"path"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"unicode/utf8"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
)
|
||||
|
||||
var uploadType = reflect.TypeOf(File{})
|
||||
|
||||
// MultipartOptions limits raw request bytes (including headers), part count,
|
||||
// and individual decoded part sizes. Zero selects bounded defaults. Files are
|
||||
// spooled to 0600 temporary files; text/JSON parts are buffered up to MaxFieldBytes.
|
||||
type MultipartOptions struct {
|
||||
MaxBodyBytes int64
|
||||
MaxParts int
|
||||
MaxFieldBytes int64
|
||||
MaxFileBytes int64
|
||||
TempDir string
|
||||
Policy RequestPolicy
|
||||
}
|
||||
|
||||
// Normalized returns validated effective protocol limits without allocating resources.
|
||||
func (o MultipartOptions) Normalized() (MultipartOptions, error) {
|
||||
if o.MaxBodyBytes == 0 {
|
||||
o.MaxBodyBytes = DefaultMaxBodyBytes
|
||||
}
|
||||
if o.MaxParts == 0 {
|
||||
o.MaxParts = 128
|
||||
}
|
||||
if o.MaxFieldBytes == 0 {
|
||||
o.MaxFieldBytes = 64 << 10
|
||||
}
|
||||
if o.MaxFileBytes == 0 {
|
||||
o.MaxFileBytes = o.MaxBodyBytes
|
||||
}
|
||||
if o.MaxBodyBytes < 1 || o.MaxBodyBytes > 1<<40 || o.MaxParts < 1 || o.MaxParts > 10000 || o.MaxFieldBytes < 1 || o.MaxFieldBytes > 1<<30 || o.MaxFileBytes < 1 || o.MaxFileBytes > 1<<40 {
|
||||
return o, errors.New("invalid multipart limits")
|
||||
}
|
||||
return o, o.Policy.Validate()
|
||||
}
|
||||
func (b *Binder) HasMultipart() bool {
|
||||
for _, f := range b.root.fields {
|
||||
if multipartSource(f.source) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
func multipartSource(s string) bool { return s == "file" || s == "multipart" || s == "multipartJSON" }
|
||||
func jsonSource(s string) bool { return s == "json" || s == "multipartJSON" }
|
||||
|
||||
// MultipartSchema describes form parts. JSON fields get an encoding declaration
|
||||
// in generated OpenAPI; they are decoded with the ordinary strict JSON binder.
|
||||
func (b *Binder) MultipartSchema() *openapi.Schema {
|
||||
s := &openapi.Schema{Type: "object", Properties: map[string]*openapi.Schema{}}
|
||||
no := false
|
||||
s.AdditionalPropertiesAllowed = &no
|
||||
for _, f := range b.root.fields {
|
||||
if multipartSource(f.source) {
|
||||
s.Properties[f.name] = copySchema(f.value.schema)
|
||||
if !f.optional && !f.hasDefault {
|
||||
s.Required = append(s.Required, f.name)
|
||||
}
|
||||
}
|
||||
}
|
||||
return s
|
||||
}
|
||||
func (b *Binder) MultipartJSONFields() []string {
|
||||
var out []string
|
||||
for _, f := range b.root.fields {
|
||||
if f.source == "multipartJSON" {
|
||||
out = append(out, f.name)
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// BindMultipart validates every part before publishing the DTO or entering Logic.
|
||||
// On success the caller must defer cleanup; on failure all created files/readers
|
||||
// are cleaned here. The request's Body remains owned by the HTTP server.
|
||||
func (b *Binder) BindMultipart(r *http.Request, params map[string]string, target any, options MultipartOptions) (cleanup func() error, err error) {
|
||||
if b == nil || r == nil || b.schemaOnly {
|
||||
return nil, apierrors.Internal(errors.New("invalid multipart binder"))
|
||||
}
|
||||
dst := reflect.ValueOf(target)
|
||||
if dst.Kind() != reflect.Pointer || dst.IsNil() || dst.Elem().Type() != b.typ {
|
||||
return nil, apierrors.Internal(errors.New("multipart target mismatch"))
|
||||
}
|
||||
o, err := options.Normalized()
|
||||
if err != nil {
|
||||
return nil, apierrors.Internal(err)
|
||||
}
|
||||
if b.BodySchema() != nil {
|
||||
return nil, apierrors.Internal(errors.New("multipart request cannot contain json source fields; use multipartJSON"))
|
||||
}
|
||||
if r.ContentLength > o.MaxBodyBytes {
|
||||
return nil, tooLarge()
|
||||
}
|
||||
var files []File
|
||||
var once sync.Once
|
||||
var closeErr error
|
||||
release := func() error {
|
||||
once.Do(func() {
|
||||
for i := len(files) - 1; i >= 0; i-- {
|
||||
closeErr = errors.Join(closeErr, files[i].cleanup())
|
||||
}
|
||||
})
|
||||
return closeErr
|
||||
}
|
||||
success := false
|
||||
defer func() {
|
||||
if !success {
|
||||
err = errors.Join(err, release())
|
||||
}
|
||||
}()
|
||||
// Bind query/path/header once, independently of multipart body contents.
|
||||
ordinary := *b
|
||||
root := *b.root
|
||||
root.fields = nil
|
||||
ordinary.root = &root
|
||||
for _, f := range b.root.fields {
|
||||
if !multipartSource(f.source) {
|
||||
root.fields = append(root.fields, f)
|
||||
}
|
||||
}
|
||||
fresh := reflect.New(b.typ)
|
||||
policy := o.Policy
|
||||
policy.Body = "auto"
|
||||
if err = ordinary.BindWithPolicy(r, params, fresh.Interface(), policy); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
request := *r
|
||||
input := r.Body
|
||||
if input == nil {
|
||||
input = http.NoBody
|
||||
}
|
||||
request.Body = http.MaxBytesReader(nil, input, o.MaxBodyBytes)
|
||||
body, err := httpx.NewBody(&request, "multipart")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
fields := map[string]field{}
|
||||
for _, f := range b.root.fields {
|
||||
if multipartSource(f.source) {
|
||||
fields[f.name] = f
|
||||
}
|
||||
}
|
||||
text := map[string][]string{}
|
||||
uploaded := map[string][]File{}
|
||||
seen := map[string]int{}
|
||||
policy = MergePolicy(RequestPolicyFromContext(r.Context()), o.Policy)
|
||||
for count := 0; ; count++ {
|
||||
part, e := body.Parts.NextRawPart()
|
||||
if e == io.EOF {
|
||||
break
|
||||
}
|
||||
if e != nil {
|
||||
return nil, multipartFailure(e)
|
||||
}
|
||||
if count >= o.MaxParts {
|
||||
return nil, tooLarge()
|
||||
}
|
||||
f, ok := fields[part.FormName()]
|
||||
if !ok {
|
||||
return nil, invalid(errors.New("unknown multipart field"))
|
||||
}
|
||||
seen[f.name]++
|
||||
// Reject encodings that would make declared byte limits ambiguous.
|
||||
if part.Header.Get("Content-Transfer-Encoding") != "" {
|
||||
return nil, invalid(errors.New("encoded multipart parts are unsupported"))
|
||||
}
|
||||
_, disposition, e := mime.ParseMediaType(part.Header.Get("Content-Disposition"))
|
||||
if e != nil {
|
||||
return nil, invalid(e)
|
||||
}
|
||||
_, isFile := disposition["filename"]
|
||||
if (f.source == "file") != isFile {
|
||||
return nil, invalid(errors.New("multipart field kind mismatch"))
|
||||
}
|
||||
limit := o.MaxFieldBytes
|
||||
if f.source == "file" {
|
||||
limit = o.MaxFileBytes
|
||||
if f.maxBytes > 0 && f.maxBytes < limit {
|
||||
limit = f.maxBytes
|
||||
}
|
||||
maxFiles := f.maxFiles
|
||||
if maxFiles == 0 {
|
||||
maxFiles = 16
|
||||
}
|
||||
if f.value.typ.Kind() != reflect.Slice {
|
||||
maxFiles = 1
|
||||
}
|
||||
if seen[f.name] > maxFiles {
|
||||
return nil, invalid(errors.New("too many files for multipart field"))
|
||||
}
|
||||
name := path.Base(strings.ReplaceAll(part.FileName(), "\\", "/"))
|
||||
if name == "" || name == "." || len(name) > 255 || !utf8.ValidString(name) || strings.ContainsAny(name, "\x00\r\n") {
|
||||
return nil, invalid(errors.New("invalid upload filename"))
|
||||
}
|
||||
temp, e := os.CreateTemp(o.TempDir, "omni-upload-*")
|
||||
if e != nil {
|
||||
return nil, apierrors.Internal(e)
|
||||
}
|
||||
file := File{Name: name, ContentType: part.Header.Get("Content-Type"), state: &uploadState{path: temp.Name(), readers: map[*uploadReader]bool{}}}
|
||||
files = append(files, file)
|
||||
size, e := io.Copy(temp, io.LimitReader(part, limit+1))
|
||||
e = errors.Join(e, temp.Close())
|
||||
if size > limit {
|
||||
return nil, tooLarge()
|
||||
}
|
||||
if e != nil {
|
||||
return nil, multipartFailure(e)
|
||||
}
|
||||
file.Size = size
|
||||
files[len(files)-1] = file
|
||||
uploaded[f.name] = append(uploaded[f.name], file)
|
||||
} else {
|
||||
if f.maxBytes > 0 && f.maxBytes < limit {
|
||||
limit = f.maxBytes
|
||||
}
|
||||
if seen[f.name] > 1 && (f.source == "multipartJSON" || base(f.value).typ.Kind() != reflect.Slice) {
|
||||
return nil, invalid(errors.New("duplicate multipart field"))
|
||||
}
|
||||
data, e := io.ReadAll(io.LimitReader(part, limit+1))
|
||||
if int64(len(data)) > limit {
|
||||
return nil, tooLarge()
|
||||
}
|
||||
if e != nil {
|
||||
return nil, multipartFailure(e)
|
||||
}
|
||||
if policy.UTF8 == "reject" && !utf8.Valid(data) {
|
||||
return nil, invalid(errUTF8)
|
||||
}
|
||||
if policy.Charset == "utf8" && part.Header.Get("Content-Type") != "" {
|
||||
_, p, e := mime.ParseMediaType(part.Header.Get("Content-Type"))
|
||||
if e != nil || (p["charset"] != "" && !strings.EqualFold(p["charset"], "utf-8")) {
|
||||
return nil, invalid(errors.New("invalid multipart field charset"))
|
||||
}
|
||||
}
|
||||
text[f.name] = append(text[f.name], strings.ToValidUTF8(string(data), "<22>"))
|
||||
}
|
||||
if err = part.Close(); err != nil {
|
||||
return nil, multipartFailure(err)
|
||||
}
|
||||
}
|
||||
// Consume epilogue as well; it counts against the total request cap.
|
||||
if err = body.Complete(); err != nil {
|
||||
return nil, multipartFailure(err)
|
||||
}
|
||||
for _, f := range b.root.fields {
|
||||
if !multipartSource(f.source) {
|
||||
continue
|
||||
}
|
||||
present := seen[f.name] > 0
|
||||
dest := fresh.Elem().Field(f.index)
|
||||
if f.source == "file" {
|
||||
if !present {
|
||||
if !f.optional {
|
||||
return nil, invalid(fmt.Errorf("%s: file required", f.name))
|
||||
}
|
||||
continue
|
||||
}
|
||||
if f.value.typ.Kind() == reflect.Slice {
|
||||
dest.Set(reflect.ValueOf(uploaded[f.name]))
|
||||
} else {
|
||||
dest.Set(reflect.ValueOf(uploaded[f.name][0]))
|
||||
}
|
||||
continue
|
||||
}
|
||||
var value any
|
||||
if present {
|
||||
if f.source == "multipartJSON" {
|
||||
d := json.NewDecoder(strings.NewReader(text[f.name][0]))
|
||||
d.UseNumber()
|
||||
value, err = decodeValue(d, 0)
|
||||
if err == nil {
|
||||
_, err = d.Token()
|
||||
if err == io.EOF {
|
||||
err = nil
|
||||
} else if err == nil {
|
||||
err = errors.New("multiple JSON values")
|
||||
}
|
||||
}
|
||||
} else {
|
||||
value, err = fromStrings(f.value, text[f.name])
|
||||
}
|
||||
if err != nil {
|
||||
return nil, invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||
}
|
||||
}
|
||||
if err = assignField(f, dest, value, present); err != nil {
|
||||
return nil, invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||
}
|
||||
}
|
||||
dst.Elem().Set(fresh.Elem())
|
||||
success = true
|
||||
return release, nil
|
||||
}
|
||||
func multipartFailure(err error) error {
|
||||
var max *http.MaxBytesError
|
||||
if errors.As(err, &max) {
|
||||
return tooLarge()
|
||||
}
|
||||
var api *apierrors.APIError
|
||||
if errors.As(err, &api) {
|
||||
return err
|
||||
}
|
||||
return invalid(err)
|
||||
}
|
||||
func fieldLimit(raw string) (int64, error) {
|
||||
v, err := strconv.ParseInt(raw, 10, 64)
|
||||
if err != nil || v < 1 || v > 1<<40 {
|
||||
return 0, errors.New("maxBytes must be 1..1099511627776")
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
152
contract/multipart_test.go
Normal file
152
contract/multipart_test.go
Normal file
@@ -0,0 +1,152 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"io"
|
||||
"mime/multipart"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type uploadInput struct {
|
||||
Query string `form:"query,optional"`
|
||||
Name string `multipart:"name" binding:"required,max=10"`
|
||||
Meta struct {
|
||||
Count int `json:"count"`
|
||||
} `multipartJSON:"meta"`
|
||||
Attachment File `file:"attachment,maxBytes=12"`
|
||||
Extra []File `file:"extra,optional,maxBytes=8,maxFiles=2"`
|
||||
}
|
||||
type uploadPart struct{ name, filename, value string }
|
||||
|
||||
func uploadRequest(t *testing.T, parts []uploadPart) (*http.Request, []byte) {
|
||||
t.Helper()
|
||||
var data bytes.Buffer
|
||||
w := multipart.NewWriter(&data)
|
||||
for _, part := range parts {
|
||||
if part.filename != "" {
|
||||
p, e := w.CreateFormFile(part.name, part.filename)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
_, _ = io.WriteString(p, part.value)
|
||||
} else {
|
||||
_ = w.WriteField(part.name, part.value)
|
||||
}
|
||||
}
|
||||
_ = w.Close()
|
||||
r := httptest.NewRequest("POST", "/upload?query=yes", bytes.NewReader(data.Bytes()))
|
||||
r.Header.Set("Content-Type", w.FormDataContentType())
|
||||
return r, data.Bytes()
|
||||
}
|
||||
func goodParts() []uploadPart {
|
||||
return []uploadPart{{"name", "", "合同"}, {"meta", "", `{"count":2}`}, {"attachment", "../contract.txt", "hello"}}
|
||||
}
|
||||
func assertUploadEmpty(t *testing.T, dir string) {
|
||||
t.Helper()
|
||||
files, err := os.ReadDir(dir)
|
||||
if err != nil || len(files) != 0 {
|
||||
t.Fatalf("temp files remain: %v %v", files, err)
|
||||
}
|
||||
}
|
||||
func TestMultipartBindingAndLifetime(t *testing.T) {
|
||||
b := MustNew[uploadInput]()
|
||||
dir := t.TempDir()
|
||||
r, _ := uploadRequest(t, append(goodParts(), uploadPart{"extra", "one", "first"}, uploadPart{"extra", "two", "second"}))
|
||||
var input uploadInput
|
||||
release, err := b.BindMultipart(r, nil, &input, MultipartOptions{TempDir: dir, Policy: RequestPolicy{UnknownQuery: "reject", UTF8: "reject"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if input.Name != "合同" || input.Query != "yes" || input.Meta.Count != 2 || input.Attachment.Size != 5 || input.Attachment.Name != "contract.txt" || len(input.Extra) != 2 {
|
||||
t.Fatalf("%+v", input)
|
||||
}
|
||||
reader, err := input.Attachment.Open()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
data, err := io.ReadAll(reader)
|
||||
if err != nil || string(data) != "hello" {
|
||||
t.Fatal(string(data), err)
|
||||
}
|
||||
// An unclosed reader is owned by the upload cleanup as well.
|
||||
if err = release(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = reader.Read(make([]byte, 1)); err == nil {
|
||||
t.Fatal("reader remains open")
|
||||
}
|
||||
if _, err = input.Attachment.Open(); err == nil {
|
||||
t.Fatal("expired upload reopened")
|
||||
}
|
||||
if err = release(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
assertUploadEmpty(t, dir)
|
||||
if err = b.Bind(r, nil, &input); err == nil {
|
||||
t.Fatal("ordinary binding accepted multipart fields")
|
||||
}
|
||||
}
|
||||
func TestMultipartFailuresAreTransactional(t *testing.T) {
|
||||
b := MustNew[uploadInput]()
|
||||
for _, tt := range []struct {
|
||||
name string
|
||||
parts []uploadPart
|
||||
options MultipartOptions
|
||||
status int
|
||||
}{
|
||||
{"missing", goodParts()[1:], MultipartOptions{}, 400},
|
||||
{"unknown", append(goodParts(), uploadPart{"other", "", "x"}), MultipartOptions{}, 400},
|
||||
{"duplicate", append(goodParts(), uploadPart{"name", "", "x"}), MultipartOptions{}, 400},
|
||||
{"jsonDuplicate", []uploadPart{{"name", "", "n"}, {"attachment", "f", "x"}, {"meta", "", `{"count":1,"count":2}`}}, MultipartOptions{}, 400},
|
||||
{"badType", []uploadPart{{"name", "", "n"}, {"attachment", "f", "x"}, {"meta", "", `{"count":"x"}`}}, MultipartOptions{}, 400},
|
||||
{"badUTF8", append(goodParts()[:1], uploadPart{"meta", "", "{\"count\":\"\xff\"}"}, goodParts()[2]), MultipartOptions{Policy: RequestPolicy{UTF8: "reject"}}, 400},
|
||||
{"fileSize", []uploadPart{{"attachment", "x", strings.Repeat("x", 13)}}, MultipartOptions{}, 413},
|
||||
{"fieldSize", goodParts(), MultipartOptions{MaxFieldBytes: 2}, 413},
|
||||
{"partCount", goodParts(), MultipartOptions{MaxParts: 2}, 413},
|
||||
{"total", goodParts(), MultipartOptions{MaxBodyBytes: 32}, 413},
|
||||
{"fileKind", []uploadPart{{"attachment", "", "text"}}, MultipartOptions{}, 400},
|
||||
{"extraCount", append(goodParts(), uploadPart{"extra", "a", "a"}, uploadPart{"extra", "b", "b"}, uploadPart{"extra", "c", "c"}), MultipartOptions{}, 400},
|
||||
} {
|
||||
t.Run(tt.name, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
tt.options.TempDir = dir
|
||||
r, _ := uploadRequest(t, tt.parts)
|
||||
r.ContentLength = -1
|
||||
input := uploadInput{Name: "untouched"}
|
||||
release, err := b.BindMultipart(r, nil, &input, tt.options)
|
||||
var api *apierrors.APIError
|
||||
if !errors.As(err, &api) || api.Status != tt.status {
|
||||
t.Fatalf("error %v", err)
|
||||
}
|
||||
if release != nil || input.Name != "untouched" {
|
||||
t.Fatal("partial binding published")
|
||||
}
|
||||
assertUploadEmpty(t, dir)
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestMultipartTruncationAndEpilogueLimit(t *testing.T) {
|
||||
b := MustNew[uploadInput]()
|
||||
for _, epilogue := range []bool{false, true} {
|
||||
dir := t.TempDir()
|
||||
r, original := uploadRequest(t, goodParts())
|
||||
data := original[:len(original)-8]
|
||||
limit := int64(len(original) + 16)
|
||||
if epilogue {
|
||||
data = append(original, bytes.Repeat([]byte("x"), 64)...)
|
||||
}
|
||||
r.Body = io.NopCloser(bytes.NewReader(data))
|
||||
r.ContentLength = -1
|
||||
var input uploadInput
|
||||
if _, err := b.BindMultipart(r, nil, &input, MultipartOptions{TempDir: dir, MaxBodyBytes: limit}); err == nil {
|
||||
t.Fatal("truncation or epilogue bypass accepted")
|
||||
}
|
||||
assertUploadEmpty(t, dir)
|
||||
}
|
||||
}
|
||||
105
contract/policy.go
Normal file
105
contract/policy.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// RequestPolicy describes protocol strictness, independently of business
|
||||
// fields. Empty values inherit App defaults, then the historical binder policy.
|
||||
type RequestPolicy struct {
|
||||
UnknownQuery string `json:"unknownQuery"` // ignore or reject
|
||||
UTF8 string `json:"utf8"` // replace or reject
|
||||
Charset string `json:"charset"` // any or utf8
|
||||
Body string `json:"body"` // auto, required, optional, none, emptyObject
|
||||
}
|
||||
|
||||
func (p RequestPolicy) Validate() error {
|
||||
for _, item := range []struct {
|
||||
name, value string
|
||||
allowed []string
|
||||
}{
|
||||
{"unknownQuery", p.UnknownQuery, []string{"ignore", "reject"}},
|
||||
{"utf8", p.UTF8, []string{"replace", "reject"}},
|
||||
{"charset", p.Charset, []string{"any", "utf8"}},
|
||||
{"body", p.Body, []string{"auto", "required", "optional", "none", "emptyObject"}},
|
||||
} {
|
||||
if item.value == "" {
|
||||
continue
|
||||
}
|
||||
ok := false
|
||||
for _, value := range item.allowed {
|
||||
ok = ok || value == item.value
|
||||
}
|
||||
if !ok {
|
||||
return fmt.Errorf("request policy: invalid %s", item.name)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func MergePolicy(base, override RequestPolicy) RequestPolicy {
|
||||
if override.UnknownQuery != "" {
|
||||
base.UnknownQuery = override.UnknownQuery
|
||||
}
|
||||
if override.UTF8 != "" {
|
||||
base.UTF8 = override.UTF8
|
||||
}
|
||||
if override.Charset != "" {
|
||||
base.Charset = override.Charset
|
||||
}
|
||||
if override.Body != "" {
|
||||
base.Body = override.Body
|
||||
}
|
||||
return base
|
||||
}
|
||||
|
||||
type policyKey struct{}
|
||||
|
||||
func WithRequestPolicy(ctx context.Context, p RequestPolicy) context.Context {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
return context.WithValue(ctx, policyKey{}, p)
|
||||
}
|
||||
func RequestPolicyFromContext(ctx context.Context) RequestPolicy {
|
||||
if ctx == nil {
|
||||
return RequestPolicy{}
|
||||
}
|
||||
p, _ := ctx.Value(policyKey{}).(RequestPolicy)
|
||||
return p
|
||||
}
|
||||
|
||||
// utf8Reader checks original bytes in one pass, including runes crossing Read
|
||||
// boundaries. It never replaces invalid bytes or buffers the complete body.
|
||||
type utf8Reader struct {
|
||||
r io.Reader
|
||||
pending [4]byte
|
||||
n int
|
||||
}
|
||||
|
||||
var errUTF8 = errors.New("invalid UTF-8 in JSON body")
|
||||
|
||||
func (r *utf8Reader) Read(p []byte) (int, error) {
|
||||
n, err := r.r.Read(p)
|
||||
for _, b := range p[:n] {
|
||||
if r.n == 0 && b < utf8.RuneSelf {
|
||||
continue
|
||||
}
|
||||
r.pending[r.n] = b
|
||||
r.n++
|
||||
if utf8.FullRune(r.pending[:r.n]) {
|
||||
ch, size := utf8.DecodeRune(r.pending[:r.n])
|
||||
if ch == utf8.RuneError && size == 1 {
|
||||
return n, errUTF8
|
||||
}
|
||||
r.n = 0
|
||||
}
|
||||
}
|
||||
if err == io.EOF && r.n != 0 {
|
||||
return n, errUTF8
|
||||
}
|
||||
return n, err
|
||||
}
|
||||
104
contract/policy_test.go
Normal file
104
contract/policy_test.go
Normal file
@@ -0,0 +1,104 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type byteReader struct{ r io.Reader }
|
||||
|
||||
func (r byteReader) Read(p []byte) (int, error) {
|
||||
if len(p) > 1 {
|
||||
p = p[:1]
|
||||
}
|
||||
return r.r.Read(p)
|
||||
}
|
||||
|
||||
func TestBodyPolicyPropagatesOuterLimit(t *testing.T) {
|
||||
for _, limit := range []int64{0, 1, 4} {
|
||||
r := httptest.NewRequest("POST", "/", byteReader{strings.NewReader("{} ")})
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Body = http.MaxBytesReader(httptest.NewRecorder(), r.Body, limit)
|
||||
err := MustNew[struct{}]().BindWithPolicy(r, nil, &struct{}{}, RequestPolicy{Body: "required", UTF8: "reject"})
|
||||
var e *apierrors.APIError
|
||||
if !errors.As(err, &e) || e.Status != 413 {
|
||||
t.Fatalf("limit %d: expected 413, got %v", limit, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestStrictRequestPolicy(t *testing.T) {
|
||||
type request struct {
|
||||
Name string `json:"name"`
|
||||
Tags []string `form:"tag,optional"`
|
||||
}
|
||||
b := MustNew[request]()
|
||||
strict := RequestPolicy{UnknownQuery: "reject", UTF8: "reject", Charset: "utf8"}
|
||||
for _, tc := range []struct {
|
||||
name, url, ct, body string
|
||||
valid bool
|
||||
}{
|
||||
{"valid", "/?tag=a&tag=b", "application/json; charset=UTF-8", `{"name":"企业😀"}`, true},
|
||||
{"query", "/?typo=1", "application/json", `{"name":"ok"}`, false},
|
||||
{"charset", "/", "application/json; charset=gbk", `{"name":"ok"}`, false},
|
||||
{"utf8", "/", "application/json", "{\"name\":\"\xff\"}", false},
|
||||
{"truncatedRune", "/", "application/json", "{\"name\":\"\xe4\"}", false},
|
||||
{"tail", "/", "application/json", `{"name":"ok"} {}`, false},
|
||||
{"tailInvalidByte", "/", "application/json", "{\"name\":\"ok\"} \xff", false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
r := httptest.NewRequest("POST", tc.url, byteReader{strings.NewReader(tc.body)})
|
||||
r.Header.Set("Content-Type", tc.ct)
|
||||
var dst request
|
||||
err := b.BindWithPolicy(r, nil, &dst, strict)
|
||||
if (err == nil) != tc.valid {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tc.valid && len(dst.Tags) != 2 {
|
||||
t.Fatal("query slice rejected")
|
||||
}
|
||||
})
|
||||
}
|
||||
// Explicit route overrides inherit the rest of the App policy.
|
||||
r := httptest.NewRequest("POST", "/?typo=1", strings.NewReader(`{"name":"ok"}`))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r = r.WithContext(WithRequestPolicy(context.Background(), strict))
|
||||
if err := b.BindWithPolicy(r, nil, &request{}, RequestPolicy{UnknownQuery: "ignore"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBodyPresencePolicies(t *testing.T) {
|
||||
b := MustNew[struct{}]()
|
||||
for _, mode := range []string{"none", "emptyObject", "required", "optional"} {
|
||||
for _, body := range []string{"", "{}", "null", `{"unexpected":1}`, "{} {}", " "} {
|
||||
r := httptest.NewRequest("POST", "/", byteReader{strings.NewReader(body)})
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
err := b.BindWithPolicy(r, nil, &struct{}{}, RequestPolicy{Body: mode, UTF8: "reject"})
|
||||
valid := (mode == "none" && body == "") || (mode != "none" && body == "{}") || (mode == "optional" && body == "")
|
||||
if (err == nil) != valid {
|
||||
t.Fatalf("mode=%s body=%q: %v", mode, body, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
type object struct {
|
||||
Value string `json:"value"`
|
||||
}
|
||||
r := httptest.NewRequest("POST", "/", bytes.NewReader(nil))
|
||||
if err := MustNew[object]().BindWithPolicy(r, nil, &object{}, RequestPolicy{Body: "optional"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Chunked/unknown size is still bounded under strict UTF-8 checking.
|
||||
r = httptest.NewRequest("POST", "/", byteReader{strings.NewReader(`{"value":"1234567890"}`)})
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
if err := MustNew[object]().BindWithPolicy(r, nil, &object{}, RequestPolicy{UTF8: "reject"}, 8); err == nil {
|
||||
t.Fatal("chunked over-limit accepted")
|
||||
}
|
||||
}
|
||||
@@ -147,6 +147,10 @@ func responseSchema(n *node) *openapi.Schema {
|
||||
b := base(n)
|
||||
if b.scalar != nil {
|
||||
s := wireSchema(b.scalar.JSONOutput)
|
||||
s.Description = n.schema.Description
|
||||
if b.scalar.JSONInput == b.scalar.JSONOutput {
|
||||
s.Example = n.schema.Example
|
||||
}
|
||||
if n.typ.Kind() == reflect.Pointer {
|
||||
s = nullableSchema(s)
|
||||
}
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# database
|
||||
|
||||
结构化 MySQL 配置、独立 TLS/mTLS、驱动日志和生成项目接入见 [官方 MySQL 适配器](mysql/README.md)。已有 Open/OpenConnector 接口继续支持。
|
||||
|
||||
使用 GORM 的项目可显式接入 [gormx 可选适配器](gormx/README.md),复用本包连接池,并将 SQL 日志接入 logx;普通 database 包不导入 GORM。
|
||||
|
||||
`database` 包提供基于标准库 `database/sql` 的连接池生命周期,不绑定 ORM 或驱动:
|
||||
|
||||
- `Config` / `ConfigFromEnv`:连接池参数和 `GO_OMNI_DB_*` 环境变量。
|
||||
|
||||
104
database/gormx/README.md
Normal file
104
database/gormx/README.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# 可选 GORM 适配器
|
||||
|
||||
从 `v0.14.1-preview` 提供。`gormx` 把 GORM 接到 go-omni 已初始化的 `*sql.DB`,保留一份连接池和一处资源所有权。使用 GORM v2 API(本版验收版本为 `gorm.io/gorm v1.31.2`、MySQL dialect `v1.6.0`)。
|
||||
|
||||
核心 `omni`、`database`、`database/mysql` 不导入此包;普通项目的编译依赖链不包含 GORM。仓库同一 Go module 的 go.mod 会记录可选适配器依赖,使用方显式导入 `database/gormx` 才编译它。本版没有新增默认 ORM 或 `--orm` 脚手架开关。
|
||||
|
||||
## 生成项目接入
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
|
||||
go-omni new --database mysql --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
按 [MySQL 配置说明](../mysql/README.md) 设置连接信息、TLS 和池大小。在开发者维护的 `internal/svc/serviceContext.go` 中增加 `ORM *gorm.DB` 字段,并填写现有 Init:
|
||||
|
||||
```go
|
||||
import (
|
||||
"context"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database/gormx"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// 向现有 ServiceContext 添加 ORM *gorm.DB 字段。
|
||||
// 保留 Runtime/Config/Logger/DB 等现有字段。
|
||||
func (c *ServiceContext) Init(ctx context.Context) error {
|
||||
orm, err := gormx.MySQL(ctx, c.DB, gormx.WithLogger(c.Logger))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.ORM = orm
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
执行 `go mod tidy`。main、runtimeGen、Handler 和路由无需修改;重新生成保留 svc 的字段与 Init。已有项目先 `go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview`,同样接入;v0.14.0 生成项目的目录和生命周期契约不变。
|
||||
|
||||
Repository 接收 `*gorm.DB`,查询使用请求 context:
|
||||
|
||||
```go
|
||||
err := repo.db.WithContext(ctx).Where("id = ?", id).First(&item).Error
|
||||
|
||||
err = repo.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
// 同一事务内始终使用 tx。
|
||||
return tx.Create(&item).Error
|
||||
})
|
||||
```
|
||||
|
||||
现有模型、Hooks、Scopes、事务、命名策略和错误处理继续由项目维护。共享连接池不会把分别发起的 SQL/GORM 事务自动合并;不要在一个事务内切回根 ORM 或 c.DB 执行本应属于该事务的写入。
|
||||
|
||||
## 连接池与失败处理
|
||||
|
||||
- MySQL 配置、TLS、字符集、会话参数、池大小及健康检查由底层 `database/mysql` 管理,适配器不再设置另一套 DSN 或池参数。
|
||||
- GORM 方言的版本协商继续执行,自动 Ping 关闭以复用已完成的池初始化。初始化查询使用传入 context,并受池的 PingTimeout 限制;可用 `WithInitTimeout(3*time.Second)` 单独指定。
|
||||
- 返回成功后不保留启动 context,每次查询用 `WithContext(ctx)`。查询取消、错误、事务提交/回滚保持 GORM 原生语义。
|
||||
- 对方言初始化失败时 GORM 自动关闭连接池的路径提供保护;错误仍保留 errors.Is/As,原池继续归调用方管理。生成项目的 Init 返回错误时,再由 Runtime 统一回滚该项目资源。
|
||||
- 服务排空后由 go-omni 关闭原池;`orm.DB()` 返回的仍是那一个池。不要给它重复登记 Close,也不要让 Repository 关闭它。
|
||||
|
||||
适配器自身不创建后台 worker、不自动建表、不执行 AutoMigrate、不改写默认事务或全表更新保护,也不替业务重试写入。
|
||||
|
||||
## SQL 日志
|
||||
|
||||
默认借用当前 context 的 logx,也可显式 `WithLogger(c.Logger)`。每个 ORM 可以使用独立日志实例;不会改动进程全局 Logger。
|
||||
|
||||
```go
|
||||
sqlLog := gormx.DefaultLogConfig()
|
||||
sqlLog.Level = "info" // silent / error / warn / info
|
||||
sqlLog.SlowThreshold = 300*time.Millisecond
|
||||
|
||||
orm, err := gormx.MySQL(ctx, c.DB,
|
||||
gormx.WithLogger(c.Logger),
|
||||
gormx.WithLogConfig(sqlLog),
|
||||
gormx.WithConfig(gorm.Config{
|
||||
TranslateError: true,
|
||||
// NamingStrategy 等原生选项可按项目添加。
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
默认记录错误和超过 200 ms 的慢查询,忽略 ErrRecordNotFound 的错误事件;`SlowThreshold: 0` 关闭慢查询事件。事件为 `gorm.query`、`gorm.slowQuery`、`gorm.queryFailed`,携带 durationMs、rowsAffected、sql 和 context 中的 requestId/traceId。GORM 的 `Debug()`/`LogMode` 使用副本,不改变其他请求的日志级别;最终输出还受 logx 自身级别控制。
|
||||
|
||||
默认不插入绑定参数、不记录驱动完整错误文案,只记录错误类别;SQL 正文最多保留 4096 字节,截断后附加省略号。明确开启 `IncludeParameters` / `IncludeErrorDetails` 才输出这些内容。日志不会解析和脱敏直接写在原始 SQL 里的字面量,查询仍应使用占位符。仅需要日志适配的既有 GORM 实例,也可使用 `NewLogger(logger, config)` 返回的原生 `logger.Interface`。
|
||||
|
||||
`WithConfig` 中若显式设置 `gorm.Config.Logger`,则使用项目自己的 Logger,该 Logger 自行负责脱敏和上下文。配置和 Plugins/ClauseBuilders 容器会复制;用户自定义插件、NamingStrategy 等对象仍由用户保证并发安全。
|
||||
|
||||
## 其他方言与高级配置
|
||||
|
||||
MySQL 便捷入口封装了下面的通用入口:
|
||||
|
||||
```go
|
||||
orm, err := gormx.Open(ctx, pool,
|
||||
func(conn gorm.ConnPool) gorm.Dialector {
|
||||
return mysql.New(mysql.Config{Conn: conn})
|
||||
},
|
||||
gormx.WithLogger(logger),
|
||||
)
|
||||
```
|
||||
|
||||
其他方言可由项目提供相同形式的工厂,必须使用传入的 Conn,不能创建或替换连接池。本版真实数据库验收覆盖 MySQL 8.4,不代表其他数据库已经联调。
|
||||
|
||||
不能通过 WithConfig 的 ConnPool/Dialector 覆盖此约定。初始化期间不暴露裸 sql.DB;需要它的自定义插件应在 Open 成功后显式注册。创建额外池或替换 ConnPool 的路由插件不属于本适配器的一池所有权范围。
|
||||
|
||||
PrepareStmt 默认关闭;若显式开启,保留 GORM 的缓存行为,容量、TTL 和缓存清理由项目按 GORM 接口管理。当前固定的 GORM 版本,PreparedStmtDB.Close 只清理语句,不提供停止其缓存清理 goroutine 的公开接口;因此应在服务初始化时创建一次 ORM,避免按请求反复创建。适配器不修改第三方缓存实现,也不宣称解决其生命周期限制。
|
||||
|
||||
参考:[GORM 现有连接池](https://gorm.io/docs/connecting_to_the_database.html#Existing-database-connection)、[GORM Logger](https://gorm.io/docs/logger.html)。本版验收见 [GORM 验收报告](../../reports/gormAcceptance.md)。
|
||||
121
database/gormx/gorm.go
Normal file
121
database/gormx/gorm.go
Normal file
@@ -0,0 +1,121 @@
|
||||
// Package gormx optionally attaches GORM to a pool owned by GoOmni. It never
|
||||
// creates a pool, changes pool limits, migrates tables or closes the owner.
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"maps"
|
||||
"reflect"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
gormmysql "gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// DialectorFactory must use the supplied connection pool. It must not create a
|
||||
// pool or retain/use the ORM before Open succeeds. For example, return
|
||||
// mysql.New(mysql.Config{Conn: conn}). Other dialects can use the same contract.
|
||||
type DialectorFactory func(conn gorm.ConnPool) gorm.Dialector
|
||||
|
||||
type options struct {
|
||||
config gorm.Config
|
||||
logger *logx.Logger
|
||||
logConfig LogConfig
|
||||
timeout time.Duration
|
||||
}
|
||||
type Option func(*options)
|
||||
|
||||
// WithConfig preserves native GORM behavior, including naming, hooks, prepared
|
||||
// statements and transactions. ConnPool/Dialector are supplied by this adapter;
|
||||
// automatic Ping is disabled because the owning pool already passed startup Ping.
|
||||
// A custom Config.Logger takes precedence over the logx adapter.
|
||||
func WithConfig(config gorm.Config) Option { return func(o *options) { o.config = config } }
|
||||
func WithLogger(logger *logx.Logger) Option { return func(o *options) { o.logger = logger } }
|
||||
func WithLogConfig(config LogConfig) Option { return func(o *options) { o.logConfig = config } }
|
||||
func WithInitTimeout(timeout time.Duration) Option { return func(o *options) { o.timeout = timeout } }
|
||||
|
||||
// Error exposes a safe initialization summary while preserving errors.Is/As.
|
||||
type Error struct{ cause error }
|
||||
|
||||
func (e *Error) Error() string { return "gormx: initialization failed" }
|
||||
func (e *Error) Unwrap() error { return e.cause }
|
||||
|
||||
// MySQL attaches GORM's MySQL dialect, including version negotiation, to the
|
||||
// existing pool. Configure TLS, timeouts and connection limits on that pool.
|
||||
func MySQL(ctx context.Context, pool *database.DB, opts ...Option) (*gorm.DB, error) {
|
||||
return Open(ctx, pool, func(conn gorm.ConnPool) gorm.Dialector {
|
||||
return gormmysql.New(gormmysql.Config{Conn: conn})
|
||||
}, opts...)
|
||||
}
|
||||
|
||||
// Open borrows pool. On failure (including dialect initialization failure),
|
||||
// the original pool remains owned by the caller. No ORM Close is required.
|
||||
// The returned ORM does not retain the startup context: each request should use
|
||||
// WithContext(ctx). Never close the sql.DB returned by orm.DB() independently.
|
||||
func Open(ctx context.Context, pool *database.DB, factory DialectorFactory, opts ...Option) (*gorm.DB, error) {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, &Error{cause: err}
|
||||
}
|
||||
if pool == nil || pool.DB == nil || factory == nil {
|
||||
return nil, errors.New("gormx: pool and dialector factory are required")
|
||||
}
|
||||
timeout := pool.PoolConfig().PingTimeout
|
||||
if timeout == 0 {
|
||||
timeout = 5 * time.Second
|
||||
}
|
||||
o := options{timeout: timeout, logConfig: DefaultLogConfig(), logger: logx.FromContext(ctx)}
|
||||
for _, option := range opts {
|
||||
if option != nil {
|
||||
option(&o)
|
||||
}
|
||||
}
|
||||
if o.timeout <= 0 {
|
||||
return nil, errors.New("gormx: initialization timeout must be positive")
|
||||
}
|
||||
if o.config.ConnPool != nil || o.config.Dialector != nil {
|
||||
return nil, errors.New("gormx: configure the existing pool through the dialector factory")
|
||||
}
|
||||
logger, err := NewLogger(o.logger, o.logConfig)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
config := o.config
|
||||
config.Plugins = maps.Clone(config.Plugins)
|
||||
config.ClauseBuilders = maps.Clone(config.ClauseBuilders)
|
||||
if config.Logger == nil {
|
||||
config.Logger = logger
|
||||
}
|
||||
config.DisableAutomaticPing = true
|
||||
startup, cancel := context.WithTimeout(ctx, o.timeout)
|
||||
defer cancel()
|
||||
borrowed := &borrowedPool{db: pool.DB}
|
||||
borrowed.startup.Store(&startupContext{startup})
|
||||
config.ConnPool = borrowed
|
||||
dialect := factory(borrowed)
|
||||
if dialect == nil || (reflect.ValueOf(dialect).Kind() == reflect.Pointer && reflect.ValueOf(dialect).IsNil()) {
|
||||
return nil, errors.New("gormx: dialector factory returned nil")
|
||||
}
|
||||
orm, err := gorm.Open(dialect, &config)
|
||||
if err != nil {
|
||||
return nil, &Error{cause: err}
|
||||
}
|
||||
if err = startup.Err(); err != nil {
|
||||
return nil, &Error{cause: err}
|
||||
}
|
||||
// Prepared statements are GORM's own wrapper over the very same pool.
|
||||
conn := orm.ConnPool
|
||||
if prepared, ok := conn.(*gorm.PreparedStmtDB); ok {
|
||||
conn = prepared.ConnPool
|
||||
}
|
||||
if actual, ok := conn.(*borrowedPool); !ok || actual != borrowed {
|
||||
return nil, errors.New("gormx: dialector or plugin replaced the supplied pool")
|
||||
}
|
||||
borrowed.startup.Store(nil)
|
||||
return orm, nil
|
||||
}
|
||||
173
database/gormx/gorm_test.go
Normal file
173
database/gormx/gorm_test.go
Normal file
@@ -0,0 +1,173 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql/driver"
|
||||
"errors"
|
||||
"io"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
"gorm.io/gorm/schema"
|
||||
)
|
||||
|
||||
type testConnector struct {
|
||||
closed atomic.Int32
|
||||
query func(context.Context) error
|
||||
}
|
||||
|
||||
func (c *testConnector) Connect(context.Context) (driver.Conn, error) {
|
||||
return &testConn{owner: c}, nil
|
||||
}
|
||||
func (c *testConnector) Driver() driver.Driver { return testDriver{} }
|
||||
func (c *testConnector) Close() error { c.closed.Add(1); return nil }
|
||||
|
||||
type testDriver struct{}
|
||||
|
||||
func (testDriver) Open(string) (driver.Conn, error) { return nil, errors.New("unexpected new pool") }
|
||||
|
||||
type testConn struct{ owner *testConnector }
|
||||
|
||||
func (c *testConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("unused") }
|
||||
func (c *testConn) Close() error { return nil }
|
||||
func (c *testConn) Begin() (driver.Tx, error) { return nil, errors.New("unused") }
|
||||
func (c *testConn) Ping(context.Context) error { return nil }
|
||||
func (c *testConn) QueryContext(ctx context.Context, _ string, _ []driver.NamedValue) (driver.Rows, error) {
|
||||
if c.owner.query != nil {
|
||||
if err := c.owner.query(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return &testRows{}, nil
|
||||
}
|
||||
|
||||
type testRows struct{ done bool }
|
||||
|
||||
func (r *testRows) Columns() []string { return []string{"version"} }
|
||||
func (r *testRows) Close() error { return nil }
|
||||
func (r *testRows) Next(values []driver.Value) error {
|
||||
if r.done {
|
||||
return io.EOF
|
||||
}
|
||||
r.done = true
|
||||
values[0] = "8.4.0"
|
||||
return nil
|
||||
}
|
||||
func poolFixture(t *testing.T) (*database.DB, *testConnector) {
|
||||
t.Helper()
|
||||
connector := &testConnector{}
|
||||
pool, err := database.OpenConnector(context.Background(), connector, database.DefaultPoolConfig())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = pool.Close() })
|
||||
return pool, connector
|
||||
}
|
||||
|
||||
func TestBorrowedPoolFailureAndStartupDeadline(t *testing.T) {
|
||||
for _, mode := range []string{"driverError", "deadline", "canceled"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
pool, connector := poolFixture(t)
|
||||
cause := errors.New("private-driver-error")
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
connector.query = func(ctx context.Context) error {
|
||||
if mode == "driverError" {
|
||||
return cause
|
||||
}
|
||||
<-ctx.Done()
|
||||
return ctx.Err()
|
||||
}
|
||||
if mode == "canceled" {
|
||||
cancel()
|
||||
cause = context.Canceled
|
||||
} else if mode == "deadline" {
|
||||
cause = context.DeadlineExceeded
|
||||
}
|
||||
start := time.Now()
|
||||
orm, err := MySQL(ctx, pool, WithInitTimeout(20*time.Millisecond))
|
||||
if orm != nil || !errors.Is(err, cause) {
|
||||
t.Fatal("lost cause", err)
|
||||
}
|
||||
if time.Since(start) > time.Second {
|
||||
t.Fatal("startup deadline ignored")
|
||||
}
|
||||
if connector.closed.Load() != 0 {
|
||||
t.Fatal("borrowed pool closed after GORM failure")
|
||||
}
|
||||
if healthErr := pool.Health(context.Background()); healthErr != nil {
|
||||
t.Fatal("owner pool unusable", healthErr)
|
||||
}
|
||||
if err.Error() == cause.Error() {
|
||||
t.Fatal("unsafe initialization error")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPoolIdentityNativeOptionsAndRequestContext(t *testing.T) {
|
||||
pool, connector := poolFixture(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cfg := gorm.Config{NamingStrategy: schema.NamingStrategy{TablePrefix: "business_"}, ClauseBuilders: map[string]clause.ClauseBuilder{}}
|
||||
logger, _ := logx.New(logx.Config{Output: "discard"})
|
||||
orm, err := MySQL(ctx, pool, WithConfig(cfg), WithLogger(logger))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cancel() // The startup context must not poison future queries.
|
||||
got, err := orm.DB()
|
||||
if err != nil || got != pool.DB {
|
||||
t.Fatal("pool identity changed", err)
|
||||
}
|
||||
if len(cfg.ClauseBuilders) != 0 || cfg.ConnPool != nil || cfg.Logger != nil {
|
||||
t.Fatal("native input config mutated")
|
||||
}
|
||||
if orm.NamingStrategy.TableName("Item") != "business_items" {
|
||||
t.Fatal("native option lost")
|
||||
}
|
||||
if orm.SkipDefaultTransaction || orm.AllowGlobalUpdate {
|
||||
t.Fatal("transaction safety defaults changed")
|
||||
}
|
||||
ctx, cancel = context.WithCancel(context.Background())
|
||||
cancel()
|
||||
var v string
|
||||
if err = orm.WithContext(ctx).Raw("SELECT VERSION()").Scan(&v).Error; !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = orm.WithContext(context.Background()).Raw("SELECT VERSION()").Scan(&v).Error; err != nil || v != "8.4.0" {
|
||||
t.Fatal(v, err)
|
||||
}
|
||||
if connector.closed.Load() != 0 {
|
||||
t.Fatal("pool unexpectedly closed")
|
||||
}
|
||||
if err = pool.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = orm.Raw("SELECT VERSION()").Scan(&v).Error; err == nil {
|
||||
t.Fatal("owner close did not affect GORM")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectedOptionsDoNotTouchPool(t *testing.T) {
|
||||
for _, option := range []Option{WithInitTimeout(0), WithLogConfig(LogConfig{Level: "bad"}), WithConfig(gorm.Config{ConnPool: &borrowedPool{}})} {
|
||||
pool, connector := poolFixture(t)
|
||||
if _, err := MySQL(context.Background(), pool, option); err == nil {
|
||||
t.Fatal("invalid option accepted")
|
||||
}
|
||||
if connector.closed.Load() != 0 {
|
||||
t.Fatal("pool closed")
|
||||
}
|
||||
}
|
||||
if _, err := MySQL(nil, nil); err == nil {
|
||||
t.Fatal("nil pool accepted")
|
||||
}
|
||||
pool, _ := poolFixture(t)
|
||||
if _, err := Open(nil, pool, func(gorm.ConnPool) gorm.Dialector { return nil }); err == nil {
|
||||
t.Fatal("nil dialect accepted")
|
||||
}
|
||||
}
|
||||
157
database/gormx/integration_test.go
Normal file
157
database/gormx/integration_test.go
Normal file
@@ -0,0 +1,157 @@
|
||||
package gormx_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database/gormx"
|
||||
adapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type record struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement:false"`
|
||||
Name string
|
||||
}
|
||||
|
||||
func (record) TableName() string { return "gorm_adapter_records" }
|
||||
|
||||
func TestGORMMySQLTransactionsCancellationAndLogs(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" {
|
||||
t.Skip("use scripts/verifyConnector.py --gorm with isolated MySQL")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
defer cancel()
|
||||
cfg := adapter.DefaultConfig()
|
||||
host, port, err := net.SplitHostPort(os.Getenv("GO_OMNI_CONNECTOR_ADDR"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Host = host
|
||||
cfg.Port, _ = strconv.Atoi(port)
|
||||
cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER")
|
||||
cfg.Password = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD")
|
||||
cfg.Database = os.Getenv("GO_OMNI_CONNECTOR_DATABASE")
|
||||
cfg.TLS.CAFile = os.Getenv("GO_OMNI_CONNECTOR_CA_FILE")
|
||||
cfg.TLS.ServerName = "localhost"
|
||||
cfg.Pool.MaxOpenConns = 2
|
||||
cfg.Pool.MaxIdleConns = 2
|
||||
pool, err := adapter.Open(ctx, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
var logs bytes.Buffer
|
||||
logger, err := logx.New(logx.Config{Writer: &logs})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer logger.Close()
|
||||
logConfig := gormx.DefaultLogConfig()
|
||||
logConfig.Level = "info"
|
||||
orm, err := gormx.MySQL(ctx, pool, gormx.WithLogger(logger), gormx.WithLogConfig(logConfig), gormx.WithConfig(gorm.Config{PrepareStmt: true, TranslateError: true}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sqlDB, err := orm.DB()
|
||||
if err != nil || sqlDB != pool.DB {
|
||||
t.Fatal("pool not shared", err)
|
||||
}
|
||||
if sqlDB.Stats().MaxOpenConnections != 2 {
|
||||
t.Fatal("pool limits changed")
|
||||
}
|
||||
var tables int
|
||||
if err = pool.QueryRowContext(ctx, "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name = 'gorm_adapter_records'").Scan(&tables); err != nil || tables != 0 {
|
||||
t.Fatal("adapter must not migrate schema", err)
|
||||
}
|
||||
if _, err = pool.ExecContext(ctx, "CREATE TABLE gorm_adapter_records (id BIGINT PRIMARY KEY, name VARCHAR(100) UNIQUE NOT NULL)"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.ExecContext(context.Background(), "DROP TABLE gorm_adapter_records")
|
||||
requestCtx := logx.WithRequestID(ctx, "gorm-request")
|
||||
orm = orm.WithContext(requestCtx)
|
||||
secret := "private-bound-name"
|
||||
if err = orm.Transaction(func(tx *gorm.DB) error { return tx.Create(&record{ID: 1, Name: secret}).Error }); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wanted := errors.New("rollback sentinel")
|
||||
if err = orm.Transaction(func(tx *gorm.DB) error {
|
||||
if e := tx.Create(&record{ID: 2, Name: "rollback"}).Error; e != nil {
|
||||
return e
|
||||
}
|
||||
return wanted
|
||||
}); !errors.Is(err, wanted) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
func() {
|
||||
defer func() {
|
||||
if recover() != "rollback panic" {
|
||||
t.Error("panic changed")
|
||||
}
|
||||
}()
|
||||
_ = orm.Transaction(func(tx *gorm.DB) error {
|
||||
if e := tx.Create(&record{ID: 3, Name: "panic"}).Error; e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
panic("rollback panic")
|
||||
})
|
||||
}()
|
||||
var count int64
|
||||
if err = orm.Model(&record{}).Count(&count).Error; err != nil || count != 1 {
|
||||
t.Fatal("commit/rollback differs", count, err)
|
||||
}
|
||||
if err = orm.Transaction(func(tx *gorm.DB) error {
|
||||
if e := tx.Create(&record{ID: 4, Name: "outer"}).Error; e != nil {
|
||||
return e
|
||||
}
|
||||
if e := tx.Transaction(func(inner *gorm.DB) error {
|
||||
if e := inner.Create(&record{ID: 5, Name: "inner"}).Error; e != nil {
|
||||
return e
|
||||
}
|
||||
return wanted
|
||||
}); !errors.Is(e, wanted) {
|
||||
return errors.New("nested rollback failed")
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = orm.Model(&record{}).Count(&count).Error; err != nil || count != 2 {
|
||||
t.Fatal(count, err)
|
||||
}
|
||||
if err = orm.Create(&record{ID: 6, Name: secret}).Error; !errors.Is(err, gorm.ErrDuplicatedKey) {
|
||||
t.Fatal("native error translation lost", err)
|
||||
}
|
||||
var missing record
|
||||
if err = orm.First(&missing, "id = ?", 999).Error; !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
t.Fatal("not-found semantics changed", err)
|
||||
}
|
||||
deadline, stop := context.WithTimeout(requestCtx, 30*time.Millisecond)
|
||||
start := time.Now()
|
||||
err = orm.WithContext(deadline).Exec("SELECT SLEEP(3)").Error
|
||||
stop()
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(start) > time.Second {
|
||||
t.Fatal("query cancellation lost", err)
|
||||
}
|
||||
if err = pool.Health(ctx); err != nil {
|
||||
t.Fatal("pool did not recover", err)
|
||||
}
|
||||
if strings.Contains(logs.String(), secret) || !strings.Contains(logs.String(), `"requestId":"gorm-request"`) || !strings.Contains(logs.String(), "gorm.queryFailed") {
|
||||
t.Fatal("SQL log privacy/correlation differs")
|
||||
}
|
||||
var cipher struct {
|
||||
VariableName string
|
||||
Value string
|
||||
}
|
||||
if err = orm.Raw("SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&cipher).Error; err != nil || cipher.Value == "" {
|
||||
t.Fatal("GORM bypassed TLS pool", err)
|
||||
}
|
||||
}
|
||||
171
database/gormx/logger.go
Normal file
171
database/gormx/logger.go
Normal file
@@ -0,0 +1,171 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
gormlog "gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
type LogConfig struct {
|
||||
Level string `json:"level"` // silent, error, warn (default), info
|
||||
SlowThreshold time.Duration `json:"slowThreshold"` // zero disables slow-query events
|
||||
IgnoreRecordNotFound bool `json:"ignoreRecordNotFound"`
|
||||
IncludeParameters bool `json:"includeParameters"`
|
||||
IncludeErrorDetails bool `json:"includeErrorDetails"`
|
||||
MaxSQLBytes int `json:"maxSqlBytes"` // zero uses 4096; bounds emitted SQL text
|
||||
}
|
||||
|
||||
func DefaultLogConfig() LogConfig {
|
||||
return LogConfig{Level: "warn", SlowThreshold: 200 * time.Millisecond, IgnoreRecordNotFound: true, MaxSQLBytes: 4096}
|
||||
}
|
||||
func (c LogConfig) Validate() error {
|
||||
switch c.Level {
|
||||
case "", "silent", "error", "warn", "info":
|
||||
default:
|
||||
return errors.New("gormx: invalid SQL log level")
|
||||
}
|
||||
if c.SlowThreshold < 0 || c.MaxSQLBytes < 0 {
|
||||
return errors.New("gormx: SQL log limits cannot be negative")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type sqlLogger struct {
|
||||
logger *logx.Logger
|
||||
config LogConfig
|
||||
level gormlog.LogLevel
|
||||
}
|
||||
|
||||
// NewLogger adapts logx without changing its level, globals or ownership.
|
||||
// Bound SQL arguments and raw driver error text are omitted by default. SQL
|
||||
// literals written directly into a query are not parsed or redacted.
|
||||
func NewLogger(logger *logx.Logger, config LogConfig) (gormlog.Interface, error) {
|
||||
if err := config.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if config.MaxSQLBytes == 0 {
|
||||
config.MaxSQLBytes = 4096
|
||||
}
|
||||
level := gormlog.Warn
|
||||
switch config.Level {
|
||||
case "silent":
|
||||
level = gormlog.Silent
|
||||
case "error":
|
||||
level = gormlog.Error
|
||||
case "info":
|
||||
level = gormlog.Info
|
||||
}
|
||||
return &sqlLogger{logger: logger, config: config, level: level}, nil
|
||||
}
|
||||
func (l *sqlLogger) LogMode(level gormlog.LogLevel) gormlog.Interface {
|
||||
copy := *l
|
||||
if level < gormlog.Silent || level > gormlog.Info {
|
||||
level = gormlog.Silent
|
||||
}
|
||||
copy.level = level
|
||||
return ©
|
||||
}
|
||||
func (l *sqlLogger) output(ctx context.Context) *logx.Logger {
|
||||
if l.logger != nil {
|
||||
return l.logger
|
||||
}
|
||||
return logx.FromContext(ctx)
|
||||
}
|
||||
func (l *sqlLogger) message(msg string, args []any) string {
|
||||
if !l.config.IncludeErrorDetails {
|
||||
copy := append([]any(nil), args...)
|
||||
for i, v := range copy {
|
||||
if e, ok := v.(error); ok {
|
||||
copy[i] = errorKind(e)
|
||||
}
|
||||
}
|
||||
args = copy
|
||||
}
|
||||
return clip(fmt.Sprintf(msg, args...), l.config.MaxSQLBytes)
|
||||
}
|
||||
func (l *sqlLogger) Info(ctx context.Context, msg string, args ...any) {
|
||||
if l.level >= gormlog.Info {
|
||||
l.output(ctx).InfoContext(ctx, "gorm.message", "message", l.message(msg, args))
|
||||
}
|
||||
}
|
||||
func (l *sqlLogger) Warn(ctx context.Context, msg string, args ...any) {
|
||||
if l.level >= gormlog.Warn {
|
||||
l.output(ctx).WarnContext(ctx, "gorm.message", "message", l.message(msg, args))
|
||||
}
|
||||
}
|
||||
func (l *sqlLogger) Error(ctx context.Context, msg string, args ...any) {
|
||||
if l.level >= gormlog.Error {
|
||||
l.output(ctx).ErrorContext(ctx, "gorm.message", "message", l.message(msg, args))
|
||||
}
|
||||
}
|
||||
|
||||
func (l *sqlLogger) ParamsFilter(_ context.Context, query string, args ...any) (string, []any) {
|
||||
if !l.config.IncludeParameters {
|
||||
return query, nil
|
||||
}
|
||||
return query, args
|
||||
}
|
||||
func (l *sqlLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
|
||||
if l.level == gormlog.Silent {
|
||||
return
|
||||
}
|
||||
elapsed := time.Since(begin)
|
||||
failure := err != nil && !(l.config.IgnoreRecordNotFound && errors.Is(err, gorm.ErrRecordNotFound))
|
||||
slow := l.config.SlowThreshold > 0 && elapsed > l.config.SlowThreshold
|
||||
level := gormlog.Info
|
||||
if failure && l.level >= gormlog.Error {
|
||||
level = gormlog.Error
|
||||
} else if slow && l.level >= gormlog.Warn {
|
||||
level = gormlog.Warn
|
||||
} else if l.level < gormlog.Info {
|
||||
return
|
||||
}
|
||||
query, rows := fc()
|
||||
attrs := []any{"durationMs", float64(elapsed) / float64(time.Millisecond), "rowsAffected", rows, "sql", clip(query, l.config.MaxSQLBytes)}
|
||||
if level == gormlog.Error {
|
||||
attrs = append(attrs, "errorKind", errorKind(err))
|
||||
if l.config.IncludeErrorDetails {
|
||||
attrs = append(attrs, "error", clip(err.Error(), l.config.MaxSQLBytes))
|
||||
}
|
||||
}
|
||||
logger := l.output(ctx)
|
||||
switch level {
|
||||
case gormlog.Error:
|
||||
logger.ErrorContext(ctx, "gorm.queryFailed", attrs...)
|
||||
case gormlog.Warn:
|
||||
logger.WarnContext(ctx, "gorm.slowQuery", attrs...)
|
||||
default:
|
||||
logger.InfoContext(ctx, "gorm.query", attrs...)
|
||||
}
|
||||
}
|
||||
func errorKind(err error) string {
|
||||
switch {
|
||||
case errors.Is(err, context.Canceled):
|
||||
return "canceled"
|
||||
case errors.Is(err, context.DeadlineExceeded):
|
||||
return "deadlineExceeded"
|
||||
case errors.Is(err, gorm.ErrRecordNotFound):
|
||||
return "recordNotFound"
|
||||
default:
|
||||
return fmt.Sprintf("%T", err)
|
||||
}
|
||||
}
|
||||
func clip(value string, limit int) string {
|
||||
if len(value) <= limit {
|
||||
return value
|
||||
}
|
||||
value = value[:limit]
|
||||
for !utf8.ValidString(value) && len(value) > 0 {
|
||||
value = value[:len(value)-1]
|
||||
}
|
||||
return value + "…"
|
||||
}
|
||||
|
||||
var _ gormlog.Interface = (*sqlLogger)(nil)
|
||||
var _ gorm.ParamsFilter = (*sqlLogger)(nil)
|
||||
92
database/gormx/logger_test.go
Normal file
92
database/gormx/logger_test.go
Normal file
@@ -0,0 +1,92 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
gormlog "gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
func TestSQLLoggerLevelsPrivacyAndCorrelation(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
base, _ := logx.New(logx.Config{Writer: &out})
|
||||
l, _ := NewLogger(base, DefaultLogConfig())
|
||||
ctx := logx.WithRequestID(logx.WithTraceID(context.Background(), "trace-1"), "request-1")
|
||||
count := 0
|
||||
fc := func() (string, int64) { count++; return "SELECT * FROM records WHERE token = ?", 1 }
|
||||
l.Trace(ctx, time.Now(), fc, nil)
|
||||
l.Trace(ctx, time.Now(), fc, fmt.Errorf("wrapped: %w", gorm.ErrRecordNotFound))
|
||||
l.LogMode(gormlog.Silent).Trace(ctx, time.Now(), fc, errors.New("password-value"))
|
||||
if count != 0 {
|
||||
t.Fatal("ignored event evaluated SQL")
|
||||
}
|
||||
l.Trace(ctx, time.Now().Add(-time.Second), fc, nil)
|
||||
l.Trace(ctx, time.Now(), fc, errors.New("password-value"))
|
||||
l.Error(ctx, "failed: %v", errors.New("password-value"))
|
||||
l.LogMode(gormlog.Info).Trace(ctx, time.Now(), fc, nil)
|
||||
if count != 3 {
|
||||
t.Fatal(count)
|
||||
}
|
||||
if strings.Contains(out.String(), "password-value") {
|
||||
t.Fatal("driver detail leaked", out.String())
|
||||
}
|
||||
for _, text := range []string{"gorm.slowQuery", "gorm.queryFailed", "gorm.query", `"requestId":"request-1"`, `"traceId":"trace-1"`} {
|
||||
if !strings.Contains(out.String(), text) {
|
||||
t.Fatal(text, out.String())
|
||||
}
|
||||
}
|
||||
q, args := l.(gorm.ParamsFilter).ParamsFilter(ctx, "SELECT ?", "private")
|
||||
if q != "SELECT ?" || len(args) != 0 {
|
||||
t.Fatal("default SQL arguments exposed")
|
||||
}
|
||||
// LogMode clones rather than mutating the shared instance.
|
||||
l.Trace(ctx, time.Now(), fc, nil)
|
||||
if count != 3 {
|
||||
t.Fatal("session mode changed root logger")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSQLLoggerExplicitDetailsAndConcurrentModes(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
base, _ := logx.New(logx.Config{Writer: &out})
|
||||
cfg := DefaultLogConfig()
|
||||
cfg.IncludeParameters = true
|
||||
cfg.IncludeErrorDetails = true
|
||||
cfg.Level = "info"
|
||||
cfg.MaxSQLBytes = 8
|
||||
l, err := NewLogger(base, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, args := l.(gorm.ParamsFilter).ParamsFilter(context.Background(), "SELECT ?", "value")
|
||||
if len(args) != 1 {
|
||||
t.Fatal("explicit arguments dropped")
|
||||
}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
l.LogMode(gormlog.Error).Trace(context.Background(), time.Now(), func() (string, int64) { return "SELECT 中文标识符", -1 }, errors.New("failure"))
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
for _, line := range strings.Split(strings.TrimSpace(out.String()), "\n") {
|
||||
var event map[string]any
|
||||
if err = json.Unmarshal([]byte(line), &event); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if event["error"] != "failure" || strings.Contains(event["sql"].(string), "<22>") {
|
||||
t.Fatal(event)
|
||||
}
|
||||
}
|
||||
}
|
||||
52
database/gormx/pool.go
Normal file
52
database/gormx/pool.go
Normal file
@@ -0,0 +1,52 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"sync/atomic"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// GORM can call DB().Close on failed initialization. Do not expose the owner's
|
||||
// sql.DB until initialization has succeeded; SQL methods remain fully usable.
|
||||
// Dialects using context.Background during version probing are bounded by the
|
||||
// startup context, which stops affecting queries as soon as Open succeeds.
|
||||
type borrowedPool struct {
|
||||
db *sql.DB
|
||||
startup atomic.Pointer[startupContext]
|
||||
}
|
||||
|
||||
type startupContext struct{ context.Context }
|
||||
|
||||
func (p *borrowedPool) operationContext(ctx context.Context) context.Context {
|
||||
if startup := p.startup.Load(); startup != nil {
|
||||
return startup.Context
|
||||
}
|
||||
return ctx
|
||||
}
|
||||
func (p *borrowedPool) GetDBConn() (*sql.DB, error) {
|
||||
if p.startup.Load() != nil {
|
||||
return nil, gorm.ErrInvalidDB
|
||||
}
|
||||
return p.db, nil
|
||||
}
|
||||
func (p *borrowedPool) PrepareContext(ctx context.Context, query string) (*sql.Stmt, error) {
|
||||
return p.db.PrepareContext(p.operationContext(ctx), query)
|
||||
}
|
||||
func (p *borrowedPool) ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error) {
|
||||
return p.db.ExecContext(p.operationContext(ctx), query, args...)
|
||||
}
|
||||
func (p *borrowedPool) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error) {
|
||||
return p.db.QueryContext(p.operationContext(ctx), query, args...)
|
||||
}
|
||||
func (p *borrowedPool) QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row {
|
||||
return p.db.QueryRowContext(p.operationContext(ctx), query, args...)
|
||||
}
|
||||
func (p *borrowedPool) BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error) {
|
||||
return p.db.BeginTx(p.operationContext(ctx), opts)
|
||||
}
|
||||
|
||||
var _ gorm.ConnPool = (*borrowedPool)(nil)
|
||||
var _ gorm.TxBeginner = (*borrowedPool)(nil)
|
||||
var _ gorm.GetDBConnector = (*borrowedPool)(nil)
|
||||
71
database/mysql/README.md
Normal file
71
database/mysql/README.md
Normal file
@@ -0,0 +1,71 @@
|
||||
# 官方 MySQL 适配器
|
||||
|
||||
本包可选,使用 `database/sql` 和 `go-sql-driver/mysql`,复用框架的唯一连接池、Ping、事务与健康检查。不自动建库/迁移、不重试写操作,也不强制某个 MySQL 版本或业务表。
|
||||
|
||||
项目使用 GORM 时,在 svc.Init 中调用 [gormx.MySQL](../gormx/README.md) 即可复用这个池、TLS 和清理流程。
|
||||
|
||||
生成项目选择 `--database mysql` 即接入:
|
||||
|
||||
```sh
|
||||
go-omni new --database mysql --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
`etc/config.yaml` 使用结构化配置,环境覆盖优先:
|
||||
|
||||
```yaml
|
||||
database:
|
||||
host: db.example.com
|
||||
port: 3306
|
||||
user: app
|
||||
database: billing
|
||||
parseTime: true
|
||||
location: UTC
|
||||
charset: utf8mb4
|
||||
collation: utf8mb4_bin
|
||||
connectTimeout: 5s
|
||||
readTimeout: 5s
|
||||
writeTimeout: 5s
|
||||
tls:
|
||||
mode: required
|
||||
caFile: /etc/certs/database-ca.pem
|
||||
serverName: db.example.com
|
||||
pool:
|
||||
maxOpenConns: 20
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
```
|
||||
|
||||
通过 `GO_OMNI_DB_PASSWORD` 传密码;还有 HOST、PORT、USER、DATABASE、TLS_CA_FILE、TLS_CERT_FILE、TLS_KEY_FILE、POOL_MAX_OPEN_CONNS 等。业务也可自定义 env 标签。新生成服务不再读取 DSN;独立数据库迁移 CLI 和手写 `database.Open/OpenConnector` 的原接口保留。
|
||||
|
||||
默认启用 TLS 并校验系统信任链与主机名;私有 CA 用 caFile,双向 TLS 额外设置 certFile/keyFile。只有明确设置 `tls.mode: disabled` 才使用明文。Unix socket 用 socket,若在 socket 上使用 TLS,必须提供 serverName。不会注册全局 TLS 配置或全局驱动 Logger。
|
||||
|
||||
手写入口:
|
||||
|
||||
```go
|
||||
cfg := mysqladapter.DefaultConfig()
|
||||
// 从项目配置覆盖 cfg;不要打印含密码的完整配置。
|
||||
db, err := mysqladapter.Open(ctx, cfg, mysqladapter.WithLogger(logger))
|
||||
if err != nil { return err }
|
||||
defer db.Close()
|
||||
// db.DB 是标准 *sql.DB;也可用框架 Health、InTx 等方法。
|
||||
```
|
||||
|
||||
生成项目的 `internal/svc/databaseOptions.go` 是保留的项目扩展位置:
|
||||
|
||||
```go
|
||||
func (c *ServiceContext) DatabaseOptions() []mysqladapter.Option {
|
||||
return []mysqladapter.Option{
|
||||
mysqladapter.WithStartupCheck(func(ctx context.Context, db *database.DB) error {
|
||||
return checkProjectSchema(ctx, db)
|
||||
}),
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
检查在 Ping 后运行,共享启动时限;失败或 panic 会关闭新连接池。版本/表结构检查放在这些回调中,不放到框架。回调必须响应 context,并且不能自行关闭借用的 db。依赖尚未完成整体初始化时不要把连接池发布到外部。
|
||||
|
||||
`WithDriverConfig(func(*mysql.Config) error)` 支持高级拨号、驱动参数等,配置在回调后克隆;这是项目明确选择的底层扩展。`NewConnector` 返回标准 driver.Connector,供已有基础设施使用;它不接受启动检查选项。任意驱动仍可用核心 `database.OpenConnector`。
|
||||
|
||||
错误有安全摘要,errors.Is/As 保留 TLS/驱动/项目检查的原因。配置验证不读证书、不连接数据库;Open 才读取证书、创建池并握手。不要自行打印原始配置或展开可能包含内部信息的 driver 字段给用户。
|
||||
115
database/mysql/integration_test.go
Normal file
115
database/mysql/integration_test.go
Normal file
@@ -0,0 +1,115 @@
|
||||
package mysql_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
adapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"
|
||||
"net"
|
||||
"os"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestStructuredMySQLAdapterTLS(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" {
|
||||
t.Skip("use scripts/verifyConnector.py with an isolated local MySQL")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
defer cancel()
|
||||
cfg := adapter.DefaultConfig()
|
||||
host, port, err := net.SplitHostPort(os.Getenv("GO_OMNI_CONNECTOR_ADDR"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Host = host
|
||||
cfg.Port, _ = strconv.Atoi(port)
|
||||
cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER")
|
||||
cfg.Password = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD")
|
||||
cfg.Database = os.Getenv("GO_OMNI_CONNECTOR_DATABASE")
|
||||
cfg.TLS.CAFile = os.Getenv("GO_OMNI_CONNECTOR_CA_FILE")
|
||||
cfg.TLS.ServerName = "localhost"
|
||||
cfg.Pool.MaxOpenConns = 2
|
||||
cfg.Pool.MaxIdleConns = 2
|
||||
cfg.Pool.PingTimeout = 2 * time.Second
|
||||
cfg.Session = map[string]string{"time_zone": "'+00:00'"}
|
||||
checked := false
|
||||
first, err := adapter.Open(ctx, cfg, adapter.WithStartupCheck(func(ctx context.Context, db *database.DB) error {
|
||||
var one int
|
||||
err := db.QueryRowContext(ctx, "SELECT 1").Scan(&one)
|
||||
checked = one == 1
|
||||
return err
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer first.Close()
|
||||
if !checked {
|
||||
t.Fatal("startup check not called")
|
||||
}
|
||||
cfg.Session = map[string]string{"time_zone": "'+08:00'"}
|
||||
second, err := adapter.Open(ctx, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer second.Close()
|
||||
for i, db := range []*database.DB{first, second} {
|
||||
var key, cipher, zone string
|
||||
if err = db.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&key, &cipher); err != nil || cipher == "" {
|
||||
t.Fatal("not encrypted", err)
|
||||
}
|
||||
if err = db.QueryRowContext(ctx, "SELECT @@session.time_zone").Scan(&zone); err != nil || zone != []string{"+00:00", "+08:00"}[i] {
|
||||
t.Fatal("session isolation failed", zone, err)
|
||||
}
|
||||
if db.Stats().MaxOpenConnections != 2 {
|
||||
t.Fatal("pool limit ignored")
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"untrusted", "hostname"} {
|
||||
invalid := cfg
|
||||
if bad == "untrusted" {
|
||||
invalid.TLS.CAFile = ""
|
||||
} else {
|
||||
invalid.TLS.ServerName = "wrong.invalid"
|
||||
}
|
||||
db, e := adapter.Open(ctx, invalid)
|
||||
if db != nil {
|
||||
_ = db.Close()
|
||||
t.Fatal("bad certificate accepted")
|
||||
}
|
||||
var verify *tls.CertificateVerificationError
|
||||
if !errors.As(e, &verify) {
|
||||
t.Fatal("lost TLS verification cause", e)
|
||||
}
|
||||
}
|
||||
cause := errors.New("project schema check failed")
|
||||
if db, e := adapter.Open(ctx, cfg, adapter.WithStartupCheck(func(context.Context, *database.DB) error { return cause })); db != nil || !errors.Is(e, cause) {
|
||||
t.Fatal("failed startup escaped", e)
|
||||
}
|
||||
if err = first.Health(ctx); err != nil {
|
||||
t.Fatal("failed pool affected healthy pool", err)
|
||||
}
|
||||
// The server requires a client certificate for this separate fixture user.
|
||||
cfg.User = "omni_mtls"
|
||||
cfg.Password = ""
|
||||
cfg.Database = ""
|
||||
cfg.Session = nil
|
||||
if db, e := adapter.Open(ctx, cfg); e == nil {
|
||||
db.Close()
|
||||
t.Fatal("missing required client certificate accepted")
|
||||
}
|
||||
cfg.TLS.CertFile = os.Getenv("GO_OMNI_CONNECTOR_CLIENT_CERT")
|
||||
cfg.TLS.KeyFile = os.Getenv("GO_OMNI_CONNECTOR_CLIENT_KEY")
|
||||
mtls, err := adapter.Open(ctx, cfg)
|
||||
if err != nil {
|
||||
t.Fatal("mTLS failed", err)
|
||||
}
|
||||
if err = mtls.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = mtls.Close(); err != nil {
|
||||
t.Fatal("pool close is not idempotent", err)
|
||||
}
|
||||
}
|
||||
322
database/mysql/mysql.go
Normal file
322
database/mysql/mysql.go
Normal file
@@ -0,0 +1,322 @@
|
||||
// Package mysql provides an optional MySQL adapter over database.OpenConnector.
|
||||
// Each pool owns independent driver settings; no global driver registry changes
|
||||
// are made. It never migrates tables or retries application transactions.
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"database/sql/driver"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"os"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
mysqldriver "github.com/go-sql-driver/mysql"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Host string `json:"host"`
|
||||
Port int `json:"port"`
|
||||
Socket string `json:"socket"`
|
||||
User string `json:"user"`
|
||||
Password string `json:"password" env:",secret"`
|
||||
Database string `json:"database"`
|
||||
ParseTime bool `json:"parseTime"`
|
||||
Location string `json:"location"`
|
||||
Charset string `json:"charset"`
|
||||
Collation string `json:"collation"`
|
||||
// Session values are SQL expressions, as required by the driver, for
|
||||
// example time_zone: "'+00:00'". They are trusted deployment configuration.
|
||||
Session map[string]string `json:"session"`
|
||||
ConnectTimeout time.Duration `json:"connectTimeout"`
|
||||
ReadTimeout time.Duration `json:"readTimeout"`
|
||||
WriteTimeout time.Duration `json:"writeTimeout"`
|
||||
MultiStatements bool `json:"multiStatements"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
Pool database.PoolConfig `json:"pool"`
|
||||
}
|
||||
type TLSConfig struct {
|
||||
Mode string `json:"mode"` // required (default) or disabled
|
||||
CAFile string `json:"caFile"`
|
||||
CertFile string `json:"certFile"`
|
||||
KeyFile string `json:"keyFile"`
|
||||
ServerName string `json:"serverName"`
|
||||
}
|
||||
|
||||
func DefaultConfig() Config {
|
||||
return Config{Host: "127.0.0.1", Port: 3306, ParseTime: true, Location: "UTC", Charset: "utf8mb4", ConnectTimeout: 5 * time.Second, ReadTimeout: 5 * time.Second, WriteTimeout: 5 * time.Second, TLS: TLSConfig{Mode: "required"}, Pool: database.DefaultPoolConfig()}
|
||||
}
|
||||
|
||||
var identifier = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
|
||||
|
||||
// Validate does not read certificate files or open connections. All defaults
|
||||
// come from DefaultConfig; explicit zero I/O timeouts mean no driver deadline.
|
||||
func (c Config) Validate() error {
|
||||
if c.Socket == "" && (c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") || c.Port < 1 || c.Port > 65535) {
|
||||
return errors.New("mysql: invalid host or port")
|
||||
}
|
||||
if strings.ContainsRune(c.Socket, 0) || strings.ContainsRune(c.Database, 0) || strings.ContainsRune(c.User, 0) {
|
||||
return errors.New("mysql: invalid connection field")
|
||||
}
|
||||
if c.ConnectTimeout < 0 || c.ReadTimeout < 0 || c.WriteTimeout < 0 {
|
||||
return errors.New("mysql: timeouts cannot be negative")
|
||||
}
|
||||
if err := c.Pool.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if c.Location != "" {
|
||||
if _, err := time.LoadLocation(c.Location); err != nil {
|
||||
return errors.New("mysql: invalid time location")
|
||||
}
|
||||
}
|
||||
if c.Charset != "" && !identifier.MatchString(c.Charset) {
|
||||
return errors.New("mysql: invalid charset")
|
||||
}
|
||||
if c.Collation != "" && !identifier.MatchString(c.Collation) {
|
||||
return errors.New("mysql: invalid collation")
|
||||
}
|
||||
for key := range c.Session {
|
||||
if !identifier.MatchString(key) {
|
||||
return errors.New("mysql: invalid session variable name")
|
||||
}
|
||||
}
|
||||
if c.TLS.Mode != "" && c.TLS.Mode != "required" && c.TLS.Mode != "disabled" {
|
||||
return errors.New("mysql: TLS mode must be required or disabled")
|
||||
}
|
||||
if (c.TLS.CertFile == "") != (c.TLS.KeyFile == "") {
|
||||
return errors.New("mysql: TLS client certificate and key must be configured together")
|
||||
}
|
||||
if c.TLS.Mode == "disabled" && (c.TLS.CAFile != "" || c.TLS.CertFile != "" || c.TLS.ServerName != "") {
|
||||
return errors.New("mysql: TLS fields cannot be used when disabled")
|
||||
}
|
||||
if c.Socket != "" && c.TLS.Mode != "disabled" && c.TLS.ServerName == "" {
|
||||
return errors.New("mysql: TLS over a unix socket requires serverName")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Error has a safe summary; errors.Is/As retain driver/verification causes.
|
||||
// Do not log raw configuration or a driver's manually extracted error fields.
|
||||
type Error struct {
|
||||
Operation string
|
||||
cause error
|
||||
}
|
||||
|
||||
func (e *Error) Error() string { return "mysql: " + e.Operation + " failed" }
|
||||
func (e *Error) Unwrap() error { return e.cause }
|
||||
|
||||
type Check func(context.Context, *database.DB) error
|
||||
type options struct {
|
||||
logger *logx.Logger
|
||||
checks []Check
|
||||
configure func(*mysqldriver.Config) error
|
||||
}
|
||||
type Option func(*options)
|
||||
|
||||
func WithLogger(l *logx.Logger) Option { return func(o *options) { o.logger = l } }
|
||||
|
||||
// WithStartupCheck runs after Ping, before ownership reaches the caller. A
|
||||
// failure closes the pool. Checks must honor ctx and must not close the pool.
|
||||
func WithStartupCheck(check Check) Option {
|
||||
return func(o *options) {
|
||||
if check != nil {
|
||||
o.checks = append(o.checks, check)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// WithDriverConfig provides an advanced, per-pool escape hatch. Configuration
|
||||
// is cloned after this callback; keep credentials out of callback errors/logs.
|
||||
func WithDriverConfig(fn func(*mysqldriver.Config) error) Option {
|
||||
return func(o *options) { o.configure = fn }
|
||||
}
|
||||
|
||||
func settings(opts []Option) options {
|
||||
var o options
|
||||
for _, opt := range opts {
|
||||
if opt != nil {
|
||||
opt(&o)
|
||||
}
|
||||
}
|
||||
return o
|
||||
}
|
||||
|
||||
// Open returns the existing Omni pool; .DB is the same standard sql.DB for
|
||||
// repositories or ORMs. Ping and startup checks share one bounded deadline.
|
||||
func Open(ctx context.Context, cfg Config, opts ...Option) (db *database.DB, err error) {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if err = ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
o := settings(opts)
|
||||
if o.logger == nil {
|
||||
o.logger = logx.FromContext(ctx)
|
||||
}
|
||||
connector, err := connector(cfg, o)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return openPool(ctx, connector, cfg.Pool, o.checks)
|
||||
}
|
||||
|
||||
func openPool(ctx context.Context, connector driver.Connector, pool database.PoolConfig, checks []Check) (db *database.DB, err error) {
|
||||
checkCtx, cancel := context.WithTimeout(ctx, pool.PingTimeout)
|
||||
defer cancel()
|
||||
db, err = database.OpenConnector(checkCtx, connector, pool)
|
||||
if err != nil {
|
||||
return nil, &Error{Operation: "connect", cause: err}
|
||||
}
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
_ = db.Close()
|
||||
panic(p)
|
||||
}
|
||||
if err != nil {
|
||||
err = errors.Join(err, db.Close())
|
||||
db = nil
|
||||
}
|
||||
}()
|
||||
for _, check := range checks {
|
||||
if err = check(checkCtx, db); err != nil {
|
||||
return db, &Error{Operation: "startup check", cause: err}
|
||||
}
|
||||
if err = checkCtx.Err(); err != nil {
|
||||
return db, &Error{Operation: "startup check", cause: err}
|
||||
}
|
||||
}
|
||||
return db, nil
|
||||
}
|
||||
|
||||
// NewConnector builds a standard connector for callers that own pool creation.
|
||||
// Startup checks are only executed by Open.
|
||||
func NewConnector(cfg Config, opts ...Option) (driver.Connector, error) {
|
||||
o := settings(opts)
|
||||
if len(o.checks) > 0 {
|
||||
return nil, errors.New("mysql: startup checks require Open, not NewConnector")
|
||||
}
|
||||
return connector(cfg, o)
|
||||
}
|
||||
func connector(cfg Config, o options) (driver.Connector, error) {
|
||||
d, err := driverConfig(cfg, o)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c, err := mysqldriver.NewConnector(d)
|
||||
if err != nil {
|
||||
return nil, &Error{Operation: "connector configuration", cause: err}
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
|
||||
func driverConfig(cfg Config, o options) (*mysqldriver.Config, error) {
|
||||
if err := cfg.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
d := mysqldriver.NewConfig()
|
||||
d.User, d.Passwd, d.DBName = cfg.User, cfg.Password, cfg.Database
|
||||
d.Net, d.Addr = "tcp", net.JoinHostPort(cfg.Host, strconv.Itoa(cfg.Port))
|
||||
if cfg.Socket != "" {
|
||||
d.Net, d.Addr = "unix", cfg.Socket
|
||||
}
|
||||
d.ParseTime = cfg.ParseTime
|
||||
if cfg.Location != "" {
|
||||
d.Loc, _ = time.LoadLocation(cfg.Location)
|
||||
}
|
||||
if cfg.Charset != "" {
|
||||
if err := d.Apply(mysqldriver.Charset(cfg.Charset, cfg.Collation)); err != nil {
|
||||
return nil, &Error{Operation: "charset", cause: err}
|
||||
}
|
||||
} else {
|
||||
d.Collation = cfg.Collation
|
||||
}
|
||||
d.Params = make(map[string]string, len(cfg.Session))
|
||||
for k, v := range cfg.Session {
|
||||
d.Params[k] = v
|
||||
}
|
||||
d.Timeout, d.ReadTimeout, d.WriteTimeout = cfg.ConnectTimeout, cfg.ReadTimeout, cfg.WriteTimeout
|
||||
d.MultiStatements = cfg.MultiStatements
|
||||
logger := o.logger
|
||||
if logger == nil {
|
||||
logger = logx.FromContext(nil)
|
||||
}
|
||||
d.Logger = driverLog{logger: logger, secret: cfg.Password}
|
||||
if cfg.TLS.Mode != "disabled" {
|
||||
name := cfg.TLS.ServerName
|
||||
if name == "" {
|
||||
name = cfg.Host
|
||||
}
|
||||
t := &tls.Config{MinVersion: tls.VersionTLS12, ServerName: name}
|
||||
if cfg.TLS.CAFile != "" {
|
||||
pem, err := readCertificate(cfg.TLS.CAFile)
|
||||
if err != nil {
|
||||
return nil, &Error{Operation: "read CA", cause: err}
|
||||
}
|
||||
t.RootCAs = x509.NewCertPool()
|
||||
if !t.RootCAs.AppendCertsFromPEM(pem) {
|
||||
return nil, errors.New("mysql: invalid CA certificate")
|
||||
}
|
||||
}
|
||||
if cfg.TLS.CertFile != "" {
|
||||
cert, err := readCertificate(cfg.TLS.CertFile)
|
||||
if err != nil {
|
||||
return nil, &Error{Operation: "read client certificate", cause: err}
|
||||
}
|
||||
key, err := readCertificate(cfg.TLS.KeyFile)
|
||||
if err != nil {
|
||||
return nil, &Error{Operation: "read client key", cause: err}
|
||||
}
|
||||
pair, err := tls.X509KeyPair(cert, key)
|
||||
if err != nil {
|
||||
return nil, &Error{Operation: "client key pair", cause: err}
|
||||
}
|
||||
t.Certificates = []tls.Certificate{pair}
|
||||
}
|
||||
d.TLS = t
|
||||
}
|
||||
if o.configure != nil {
|
||||
if err := o.configure(d); err != nil {
|
||||
return nil, &Error{Operation: "custom configuration", cause: err}
|
||||
}
|
||||
}
|
||||
return d.Clone(), nil
|
||||
}
|
||||
|
||||
func readCertificate(path string) ([]byte, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(io.LimitReader(f, (4<<20)+1))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(data) > 4<<20 {
|
||||
return nil, errors.New("certificate file exceeds 4 MiB")
|
||||
}
|
||||
return data, nil
|
||||
}
|
||||
|
||||
type driverLog struct {
|
||||
logger *logx.Logger
|
||||
secret string
|
||||
}
|
||||
|
||||
func (l driverLog) Print(values ...any) {
|
||||
message := fmt.Sprint(values...)
|
||||
if l.secret != "" {
|
||||
message = strings.ReplaceAll(message, l.secret, "[REDACTED]")
|
||||
}
|
||||
l.logger.Error("database.driver", "driver", "mysql", "error", message)
|
||||
}
|
||||
146
database/mysql/mysql_test.go
Normal file
146
database/mysql/mysql_test.go
Normal file
@@ -0,0 +1,146 @@
|
||||
package mysql
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"database/sql/driver"
|
||||
"errors"
|
||||
"net"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
mysqldriver "github.com/go-sql-driver/mysql"
|
||||
)
|
||||
|
||||
func TestConfigurationTLSLoggingAndIsolation(t *testing.T) {
|
||||
cfg := DefaultConfig()
|
||||
cfg.User = "test"
|
||||
cfg.Password = "do-not-print"
|
||||
cfg.Database = "mixed-name"
|
||||
cfg.Session = map[string]string{"time_zone": "'+00:00'"}
|
||||
cfg.Collation = "utf8mb4_bin"
|
||||
var out bytes.Buffer
|
||||
logger, _ := logx.New(logx.Config{Writer: &out})
|
||||
d, err := driverConfig(cfg, settings([]Option{WithLogger(logger)}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if d.TLS == nil || d.TLS.MinVersion != tls.VersionTLS12 || d.TLS.ServerName != cfg.Host || d.MultiStatements || d.Params["time_zone"] != "'+00:00'" {
|
||||
t.Fatal("driver policy differs")
|
||||
}
|
||||
cfg.Session["time_zone"] = "'+08:00'"
|
||||
if d.Params["time_zone"] != "'+00:00'" {
|
||||
t.Fatal("session map alias")
|
||||
}
|
||||
d.Logger.Print("connection ", cfg.Password)
|
||||
if strings.Contains(out.String(), cfg.Password) || !strings.Contains(out.String(), "[REDACTED]") {
|
||||
t.Fatal("secret leaked")
|
||||
}
|
||||
cfg.TLS.Mode = "disabled"
|
||||
d2, err := driverConfig(cfg, options{})
|
||||
if err != nil || d2.TLS != nil || d.TLS == nil {
|
||||
t.Fatal("TLS configuration crossed instances", err)
|
||||
}
|
||||
cfg.TLS.CAFile = "unused"
|
||||
if err = cfg.Validate(); err == nil {
|
||||
t.Fatal("disabled TLS accepted CA")
|
||||
}
|
||||
cfg = DefaultConfig()
|
||||
cfg.TLS.CAFile = filepath.Join(t.TempDir(), "missing")
|
||||
if err = cfg.Validate(); err != nil {
|
||||
t.Fatal("Validate reads certificates", err)
|
||||
}
|
||||
if _, err = NewConnector(cfg); err == nil {
|
||||
t.Fatal("missing CA accepted")
|
||||
}
|
||||
if err = os.WriteFile(cfg.TLS.CAFile, []byte("do-not-print"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = NewConnector(cfg); err == nil || strings.Contains(err.Error(), "do-not-print") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = NewConnector(DefaultConfig(), WithStartupCheck(func(context.Context, *database.DB) error { return nil })); err == nil {
|
||||
t.Fatal("unused startup check accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDriverHandshakeErrorUsesIndependentLogger(t *testing.T) {
|
||||
var a, b bytes.Buffer
|
||||
la, _ := logx.New(logx.Config{Writer: &a})
|
||||
lb, _ := logx.New(logx.Config{Writer: &b})
|
||||
cfg := DefaultConfig()
|
||||
cfg.TLS.Mode = "disabled"
|
||||
cfg.Pool.PingTimeout = time.Second
|
||||
for i, l := range []*logx.Logger{la, lb} {
|
||||
before := []int{a.Len(), b.Len()}
|
||||
db, err := Open(context.Background(), cfg, WithLogger(l), WithDriverConfig(func(d *mysqldriver.Config) error {
|
||||
d.DialFunc = func(context.Context, string, string) (net.Conn, error) {
|
||||
reader, writer := net.Pipe()
|
||||
_ = writer.Close()
|
||||
return reader, nil
|
||||
}
|
||||
return nil
|
||||
}))
|
||||
if db != nil || err == nil {
|
||||
t.Fatal("broken handshake succeeded")
|
||||
}
|
||||
sizes := []int{a.Len(), b.Len()}
|
||||
if sizes[i] <= before[i] || sizes[1-i] != before[1-i] {
|
||||
t.Fatal("driver log isolation failed")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type testConnector struct{ closed atomic.Int32 }
|
||||
|
||||
func (c *testConnector) Connect(context.Context) (driver.Conn, error) { return &testConn{c}, nil }
|
||||
func (c *testConnector) Driver() driver.Driver { return testDriver{} }
|
||||
|
||||
type testDriver struct{}
|
||||
|
||||
func (testDriver) Open(string) (driver.Conn, error) { return nil, errors.New("unexpected DSN") }
|
||||
|
||||
type testConn struct{ owner *testConnector }
|
||||
|
||||
func (c *testConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("unused") }
|
||||
func (c *testConn) Begin() (driver.Tx, error) { return nil, errors.New("unused") }
|
||||
func (c *testConn) Ping(context.Context) error { return nil }
|
||||
func (c *testConn) Close() error { c.owner.closed.Add(1); return nil }
|
||||
|
||||
func TestStartupCheckRollbackAndPanic(t *testing.T) {
|
||||
for _, panics := range []bool{false, true} {
|
||||
c := &testConnector{}
|
||||
cause := errors.New("schema failed")
|
||||
func() {
|
||||
if panics {
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Error("panic lost")
|
||||
}
|
||||
}()
|
||||
}
|
||||
db, err := openPool(context.Background(), c, database.DefaultPoolConfig(), []Check{func(ctx context.Context, db *database.DB) error {
|
||||
if _, ok := ctx.Deadline(); !ok {
|
||||
t.Error("unbounded check")
|
||||
}
|
||||
if panics {
|
||||
panic("failed check")
|
||||
}
|
||||
return cause
|
||||
}})
|
||||
if db != nil || !errors.Is(err, cause) {
|
||||
t.Error("failed pool escaped or cause lost", err)
|
||||
}
|
||||
}()
|
||||
if c.closed.Load() != 1 {
|
||||
t.Fatal("failed initialization leaked pool", c.closed.Load())
|
||||
}
|
||||
}
|
||||
}
|
||||
23
errors/export.go
Normal file
23
errors/export.go
Normal file
@@ -0,0 +1,23 @@
|
||||
package apierrors
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
)
|
||||
|
||||
// ExportCatalog writes normalized, public-only error metadata. Invoke this
|
||||
// explicitly from project tooling, using the same Catalog function registered
|
||||
// at runtime. The API CLI reads the artifact; it never executes project code.
|
||||
func ExportCatalog(w io.Writer, c Catalog) error {
|
||||
if w == nil {
|
||||
return errors.New("errors: catalog export writer required")
|
||||
}
|
||||
registry, err := NewRegistry(c)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
encoder := json.NewEncoder(w)
|
||||
encoder.SetIndent("", " ")
|
||||
return encoder.Encode(registry.Catalog())
|
||||
}
|
||||
40
errors/export_test.go
Normal file
40
errors/export_test.go
Normal file
@@ -0,0 +1,40 @@
|
||||
package apierrors
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type failedExport struct{ err error }
|
||||
|
||||
func (w failedExport) Write([]byte) (int, error) { return 0, w.err }
|
||||
func TestCatalogExport(t *testing.T) {
|
||||
catalog := Catalog{DefaultStatus: 200, Definitions: map[string]Definition{"INVALID": {Code: int64(9007199254740993), Message: "invalid"}}}
|
||||
var out bytes.Buffer
|
||||
if err := ExportCatalog(&out, catalog); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(out.String(), "9007199254740993") {
|
||||
t.Fatal(out.String())
|
||||
}
|
||||
var decoded Catalog
|
||||
if err := DecodeCatalogJSON(out.Bytes(), &decoded); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
registry, err := NewRegistry(decoded)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if registry.Resolve(ByCode("INVALID")).Status != 200 {
|
||||
t.Fatal("status lost")
|
||||
}
|
||||
cause := errors.New("write failed")
|
||||
if err = ExportCatalog(failedExport{cause}, catalog); !errors.Is(err, cause) {
|
||||
t.Fatal("writer failure lost", err)
|
||||
}
|
||||
if err = ExportCatalog(nil, catalog); err == nil {
|
||||
t.Fatal("nil writer accepted")
|
||||
}
|
||||
}
|
||||
61
examples/contracts/business.api
Normal file
61
examples/contracts/business.api
Normal file
@@ -0,0 +1,61 @@
|
||||
syntax = "v1"
|
||||
@app (
|
||||
defaultResponse: CommonResponse
|
||||
request: {"unknownQuery":"reject","utf8":"reject","charset":"utf8"}
|
||||
)
|
||||
response CommonResponse (
|
||||
codeType: int32
|
||||
successCode: 0
|
||||
errorCode: 9999
|
||||
messageField: msg
|
||||
dataMode: always
|
||||
requestIdField: "-"
|
||||
)
|
||||
type Item {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
type Query {
|
||||
Mode string `form:"mode,optional"`
|
||||
Tags []string `form:"tags,optional"`
|
||||
}
|
||||
type Upload {
|
||||
Title string `multipart:"title" binding:"required,max=20"`
|
||||
Metadata Item `multipartJSON:"metadata"`
|
||||
Attachment File `file:"attachment,maxBytes=1024"`
|
||||
Extras []File `file:"extras,optional,maxFiles=2,maxBytes=512"`
|
||||
}
|
||||
@server (group: demo; middleware: RequireAccess)
|
||||
service business {
|
||||
@handler upload
|
||||
@body (mode: multipart; maxBytes: 4096; maxParts: 6; maxFieldBytes: 128)
|
||||
post /upload (Upload) returns (Item)
|
||||
|
||||
@handler list
|
||||
get /list (Query) returns ([]Item)
|
||||
@handler nullable
|
||||
get /nullable (Query) returns (*Item)
|
||||
@handler nothing
|
||||
post /nothing returns (null)
|
||||
@handler rawNull
|
||||
@response raw
|
||||
get /rawNull returns (null)
|
||||
@handler rawArray
|
||||
@response raw
|
||||
get /rawArray returns ([]Item)
|
||||
@handler empty
|
||||
@request (body: emptyObject)
|
||||
post /empty returns (null)
|
||||
@handler absent
|
||||
@request (body: none)
|
||||
post /absent returns (null)
|
||||
@handler optional
|
||||
@request (body: optional)
|
||||
post /optional (Item) returns (Item)
|
||||
@handler echo
|
||||
post /echo (Item) returns (Item)
|
||||
@handler tolerant
|
||||
@request (unknownQuery: ignore; utf8: replace; charset: any)
|
||||
post /tolerant (Item) returns (Item)
|
||||
@handler noContent
|
||||
delete /noContent
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
{
|
||||
"templateVersion": "v0.14.0-preview",
|
||||
"profile": {
|
||||
"database": "none",
|
||||
"storage": "none",
|
||||
@@ -9,15 +10,17 @@
|
||||
"module": "example.com/managed",
|
||||
"entry": "api/managed.api",
|
||||
"generated": {
|
||||
"docs/openapi.json": "9eb707aefeefd3f4881c774534e2d143949c0edb1d8a4aabd762f978f6203df1",
|
||||
"internal/handler/appGen.go": "c6f078932e1a598a46faebba397f4e753db9d8f4f29a02f9b6077a0557d6c14e",
|
||||
"internal/handler/demo/handlersGen.go": "7a6f9d5e105e44b07196cc540cee3f0562d5f3d1c5f0bc094434db1064d0339c",
|
||||
"internal/handler/openapiGen.go": "612920ed25e5dc7a4f673be56041d47b70aa3216707ff282e52dfe503c2d705a",
|
||||
"docs/openapi.json": "68c7e9e0294bf62937aa953a44732288c51571108c922a36a72faa72c4360b5f",
|
||||
"internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d",
|
||||
"internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375",
|
||||
"internal/handler/openapiGen.go": "ae9b794c63e0f3c8f9d200a5fc6977350139d4a6f00b42a26102c8630d907d03",
|
||||
"internal/handler/responseGen.go": "84397a502bba4777ebfa6fae3d3bc418297acab4f26b04b42e3c31b36c735399",
|
||||
"internal/handler/routesGen.go": "76c1a5b2644e75e56553dc6e9e862a63128d902dd94807ce7c8aee5491cd1e3e",
|
||||
"internal/middleware/registryGen.go": "e5f47efa57a1432765ef3419bae59dbf3b4d55b3aac0a6c935ce7a52ba6b47f4",
|
||||
"internal/types/typesGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344",
|
||||
"main.go": "a12db5b7df0f2fdca4939a7382daee04c85ddb63ef0bccd1a2ce26b718f42579"
|
||||
"internal/svc/runtimeGen.go": "78ffd0b2a196fbc29e29473919d1341e8a2f508f6594ad148c0b399fbcc40018",
|
||||
"internal/types/dtoManagedd28645a2af1cGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344",
|
||||
"internal/types/typesGen.go": "372d67f23a313b3e77d2a3b7a236315db9721bc63350bf1f71db1293e57570d1",
|
||||
"main.go": "8a4051c0d51fc14ddf22c36d09de706f02499817accc937f0785e3d0fb5e2aa5"
|
||||
},
|
||||
"contract": {
|
||||
"routes": {
|
||||
|
||||
@@ -34,149 +34,18 @@ to fail on unresolved signature errors after generation. Run `go test ./...` too
|
||||
especially for project type aliases, build tags and changes inside method bodies.
|
||||
|
||||
|
||||
This project is generated from an editable API contract. Business code uses
|
||||
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
|
||||
main.go and HTTP composition are continuously generator-owned.
|
||||
## Framework-managed runtime
|
||||
|
||||
## Run
|
||||
`main.go`, `internal/svc/runtimeGen.go`, generated handlers, `internal/types/*Gen.go`
|
||||
and OpenAPI are generator-owned. Add project dependencies in `svc.Init(ctx)` and
|
||||
register each successful initialization with `OnClose`. Runtime owns reverse
|
||||
cleanup and failure rollback; do not copy locks, constructors or Close methods
|
||||
into this project. The logger is scoped to this App.
|
||||
|
||||
```sh
|
||||
go mod tidy
|
||||
go run .
|
||||
```
|
||||
|
||||
GET /health/live and /health/ready check the running service. GET /openapi.json
|
||||
serves the same document as docs/openapi.json. Bind address defaults to localhost.
|
||||
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
|
||||
The greeting starter works immediately; other new operation stubs return 501 until
|
||||
their business logic is implemented. Authentication hooks also fail closed with 501.
|
||||
|
||||
## Develop
|
||||
|
||||
1. Edit api/managed.api and its relative imports.
|
||||
2. Run the same go-omni CLI version as the framework dependency:
|
||||
|
||||
```sh
|
||||
go-omni api validate
|
||||
go-omni api inspect
|
||||
go-omni api generate --dry-run
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go-omni api export > openapi.json
|
||||
go test ./...
|
||||
```
|
||||
|
||||
Commands inside a generated project discover the recorded API entry automatically.
|
||||
--dry-run previews create/update/remove actions without writing files.
|
||||
The --check command is read-only and suitable for CI. It fails on stale or missing
|
||||
generated files and missing business scaffolds. go.mod, configuration, Logic,
|
||||
Repository, svc, middleware hook bodies and API files are created once and then
|
||||
preserved byte for byte. main.go, Gen.go files and OpenAPI are owned
|
||||
by the generator; manual edits cause a conflict instead of being overwritten.
|
||||
Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it.
|
||||
Avoid editing files while generation is running; api.lock serializes generators.
|
||||
|
||||
Routes and types that disappear are removed from generated output. Handwritten
|
||||
business files remain: remove/adapt obsolete implementations explicitly if an
|
||||
API change makes their old types or signatures fail to compile. Every regeneration
|
||||
must be followed by go test ./... . Do not delete service files to change contracts.
|
||||
|
||||
## Layout
|
||||
|
||||
```text
|
||||
api/ # editable contract and imported types
|
||||
main.go # GENERATED startup, lifecycle and route registration
|
||||
etc/config.yaml # lowerCamelCase configuration
|
||||
internal/config/ # defaults -> file -> environment
|
||||
internal/svc/ # dependency initialization and cleanup
|
||||
internal/types/typesGen.go # generated request/response types
|
||||
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
|
||||
internal/handler/<group>/ # generated binding and Logic adapters
|
||||
internal/logic/<group>/ # developer-owned business operations
|
||||
internal/repository/<group>/ # developer-owned persistence contracts
|
||||
internal/middleware/ # named hooks and generated registry
|
||||
docs/openapi.json # generated OpenAPI 3.0.3
|
||||
.goOmni/apiManifest.json # ownership and content hashes
|
||||
```
|
||||
|
||||
The API response type describes business data. Declare response Name (...) and
|
||||
select the App default with @app(defaultResponse: Name) in the entry API.
|
||||
@server(response: Name) and @response Name override it for groups/routes;
|
||||
@response raw returns the business JSON directly. Without a template, the default
|
||||
envelope includes code, message, data and requestId. Generated routes and OpenAPI
|
||||
use the same contract. 404/405 and ordinary App routes use the declared App default.
|
||||
No template is selected by declaration/import order. Without @app, existing framework
|
||||
response defaults apply. An explicit API App default takes precedence over response
|
||||
options in svc, keeping generated routes and OpenAPI consistent.
|
||||
Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default.
|
||||
Use @body(maxBytes: ...) for route limits. @body(mode: raw|multipart; maxBytes: ...)
|
||||
adds a *httpx.Body argument to Logic, without handwritten handlers. Raw is the exact
|
||||
limited request stream; Parts supports sequential multipart reading with no temp files.
|
||||
@produces "application/octet-stream" (or another concrete media type) generates a
|
||||
Logic result of *httpx.Stream; the handler closes Content and streams it unwrapped.
|
||||
Signature verification and storage policy belong to Logic/middleware. A consumed
|
||||
request stream cannot be reread automatically; use one reader owner or explicitly
|
||||
bounded buffering. Stream reads respect configured server timeouts. Multipart parts
|
||||
are not DTO form fields: form tags remain URL query parameters. The optional manual
|
||||
handler/custom binder hooks are preserved, and are needed only for custom protocols.
|
||||
|
||||
Fields require exactly one source tag: json, path, form (URL query only), header.
|
||||
All are required by presence unless optional, omitempty or default is specified.
|
||||
Zero and false are valid present values; binding:"required" rejects scalar zero.
|
||||
Use *T with optional when business logic needs to distinguish missing from zero.
|
||||
Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON
|
||||
fields and duplicate scalar query/header values are rejected. Arrays in the query
|
||||
use repeated keys. Enum/range/default and nested fields share the documentation
|
||||
and binding compiler. Responses are expected to honor their declared constraints;
|
||||
they are not validated on every response. Do not return nil slices/maps for a
|
||||
non-nullable collection (initialize empty collections instead).
|
||||
|
||||
Go-owned errors can be registered once through svc.ServiceContext.AppOptions with
|
||||
omni.WithErrors(projecterrors.Catalog()). Generated routes inherit them unless they
|
||||
declare an independent catalog. Do not repeat Go-owned errors in the API contract.
|
||||
Alternatively, declare public business errors in error (...) and reference them with @errors.
|
||||
Use internal/apierrors constructors to share status, code and message with OpenAPI.
|
||||
Auth declarations connect a bearer/header/cookie scheme to a real named middleware.
|
||||
The selected auth runs first; @auth none removes inherited declarative auth.
|
||||
Validation policy remains in the middleware implementation. Public messages must
|
||||
not contain secrets; error causes may be included in server logs.
|
||||
|
||||
svc.New receives the loaded Config and borrowed Logger. Add dependencies there,
|
||||
then immediately register each cleanup with OnClose. A later initialization failure
|
||||
runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware
|
||||
provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated
|
||||
once before route registration; logging is configured earlier through Config.
|
||||
The generated main runs HTTP draining before closing svc resources, then the logger.
|
||||
Do not close borrowed resources or attach duplicate cleanup hooks to App. The startup
|
||||
context cancels on signals; long-lived clients/workers that requests depend on need
|
||||
their own lifetime, canceled by OnClose after HTTP draining.
|
||||
|
||||
Named middleware is composed in declaration order (first is outermost). Implement
|
||||
the generated hooks or replace entries in svc; never bypass an auth
|
||||
hook merely to make the server start. Middleware names do not imply a JWT policy.
|
||||
|
||||
This CLI supports a documented go-zero-style subset, not all goctl features.
|
||||
Database tables and repositories cannot be inferred from an HTTP contract. Add
|
||||
business repositories through svc. Database and storage options are
|
||||
composed at creation; the original profile is retained on regeneration.
|
||||
|
||||
## Infrastructure
|
||||
|
||||
Selected database: none; storage: none; logging: stderr/json.
|
||||
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
|
||||
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||
No tables are invented or migrated automatically from request/response DTOs.
|
||||
The none/memory selection opens no database; implement repositories per business.
|
||||
|
||||
With storage enabled, services receive deps.Files, Uploads, Downloads and
|
||||
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
|
||||
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
|
||||
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
|
||||
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
|
||||
and authorize before issuing them. No public upload-link route is added.
|
||||
Storage failures during startup close an already-open database; Close is idempotent.
|
||||
|
||||
Database/storage/logging selection is recorded in .goOmni/apiManifest.json.
|
||||
Changing selection through generation flags is rejected on an existing project;
|
||||
edit your owned configuration and initialization explicitly, or generate a fresh
|
||||
project for comparison. New routes still regenerate without repeating these flags.
|
||||
Configuration, API contracts, svc fields/Init/AppOptions, Logic, Repository and
|
||||
middleware implementations remain developer-owned. Regeneration preserves them.
|
||||
Use `go-omni api format` to format local contracts, then `api generate` and
|
||||
`api generate --check`. See the [framework development guide](../../DEVELOPMENT.md)
|
||||
for array/null responses, typed multipart, full configuration environment overrides,
|
||||
shared contracts and v0.13 upgrade steps. This example keeps its raw callback,
|
||||
streaming upload and download to exercise those existing extension points.
|
||||
|
||||
@@ -195,6 +195,12 @@
|
||||
"demo"
|
||||
],
|
||||
"x-omni-max-body-bytes": 64,
|
||||
"x-omni-request-policy": {
|
||||
"unknownQuery": "",
|
||||
"utf8": "",
|
||||
"charset": "",
|
||||
"body": "auto"
|
||||
},
|
||||
"x-omni-response": "CommonResponse"
|
||||
}
|
||||
},
|
||||
@@ -410,6 +416,12 @@
|
||||
"tags": [
|
||||
"demo"
|
||||
],
|
||||
"x-omni-request-policy": {
|
||||
"unknownQuery": "",
|
||||
"utf8": "",
|
||||
"charset": "",
|
||||
"body": ""
|
||||
},
|
||||
"x-omni-response": "CommonResponse"
|
||||
}
|
||||
},
|
||||
@@ -589,6 +601,12 @@
|
||||
"tags": [
|
||||
"demo"
|
||||
],
|
||||
"x-omni-request-policy": {
|
||||
"unknownQuery": "",
|
||||
"utf8": "",
|
||||
"charset": "",
|
||||
"body": ""
|
||||
},
|
||||
"x-omni-response": "CommonResponse"
|
||||
}
|
||||
},
|
||||
@@ -804,6 +822,12 @@
|
||||
"tags": [
|
||||
"demo"
|
||||
],
|
||||
"x-omni-request-policy": {
|
||||
"unknownQuery": "",
|
||||
"utf8": "",
|
||||
"charset": "",
|
||||
"body": ""
|
||||
},
|
||||
"x-omni-response": "AlternateResponse"
|
||||
}
|
||||
},
|
||||
@@ -992,6 +1016,12 @@
|
||||
"tags": [
|
||||
"demo"
|
||||
],
|
||||
"x-omni-request-policy": {
|
||||
"unknownQuery": "",
|
||||
"utf8": "",
|
||||
"charset": "",
|
||||
"body": ""
|
||||
},
|
||||
"x-omni-response": "raw"
|
||||
}
|
||||
},
|
||||
@@ -1197,6 +1227,12 @@
|
||||
"tags": [
|
||||
"demo"
|
||||
],
|
||||
"x-omni-request-policy": {
|
||||
"unknownQuery": "",
|
||||
"utf8": "",
|
||||
"charset": "",
|
||||
"body": ""
|
||||
},
|
||||
"x-omni-response": "CommonResponse"
|
||||
}
|
||||
},
|
||||
@@ -1414,9 +1450,27 @@
|
||||
"demo"
|
||||
],
|
||||
"x-omni-max-body-bytes": 2048,
|
||||
"x-omni-multipart-limits": {
|
||||
"maxBytes": 2048,
|
||||
"maxFieldBytes": 65536,
|
||||
"maxFileBytes": 2048,
|
||||
"maxParts": 128
|
||||
},
|
||||
"x-omni-request-policy": {
|
||||
"unknownQuery": "",
|
||||
"utf8": "",
|
||||
"charset": "",
|
||||
"body": "auto"
|
||||
},
|
||||
"x-omni-response": "CommonResponse"
|
||||
}
|
||||
}
|
||||
},
|
||||
"x-omni-contract-sources": {
|
||||
"files": {
|
||||
"managed.api": "cec14fe0da47a83804bf461be047df3e004ee470cbc3ea330dabc1537d596672"
|
||||
},
|
||||
"shared": {}
|
||||
},
|
||||
"x-omni-default-response": "CommonResponse"
|
||||
}
|
||||
|
||||
@@ -2,7 +2,7 @@ module example.com/managed
|
||||
|
||||
go 1.22.0
|
||||
|
||||
require git.yuntuo.hk/k2safe/go-omni v0.13.0-preview
|
||||
require git.yuntuo.hk/k2safe/go-omni v0.14.0-preview
|
||||
|
||||
require (
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||
|
||||
@@ -1,14 +1,11 @@
|
||||
// Package settings owns this project's configuration; it adds no business identity model.
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
@@ -26,44 +23,40 @@ func Default() Config {
|
||||
return Config{Server: server}
|
||||
}
|
||||
|
||||
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||
// Pass an empty path to use only defaults and environment.
|
||||
// Load applies defaults -> file -> typed environment -> validation.
|
||||
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
|
||||
func Load(path string) (Config, error) {
|
||||
cfg := Default()
|
||||
var err error
|
||||
cfg, err = config.ApplyDefaults(cfg)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
if path != "" {
|
||||
cfg, err = config.DecodeFile(path, cfg)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
}
|
||||
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_"})
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
|
||||
cfg.Server, err = config.ApplyEnv(cfg.Server, "")
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
// A stream explicitly selected by environment overrides a file profile.
|
||||
if output := strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")); output != "" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE")) == "" {
|
||||
cfg.Server.Log.File = ""
|
||||
cfg.Server.Log.Rotation = logx.RotationConfig{}
|
||||
}
|
||||
|
||||
if err = cfg.Validate(); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return cfg, nil
|
||||
return cfg, cfg.Validate()
|
||||
}
|
||||
func (c Config) Validate() error {
|
||||
|
||||
if err := c.Server.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if c.Server.Log.Format != "json" && c.Server.Log.Format != "text" {
|
||||
return errors.New("log.format must be json or text")
|
||||
}
|
||||
switch strings.ToLower(c.Server.Log.Level) {
|
||||
case "debug", "info", "warn", "warning", "error":
|
||||
default:
|
||||
return errors.New("invalid log.level")
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -4,6 +4,7 @@ package handler
|
||||
import (
|
||||
"fmt"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
|
||||
"example.com/managed/internal/svc"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
@@ -16,11 +17,13 @@ func NewApp(deps *svc.ServiceContext) (*omni.App, error) {
|
||||
return nil, fmt.Errorf("service context and lifecycle controller required; see PROJECT.md upgrade instructions")
|
||||
}
|
||||
cfg := deps.Config.Server
|
||||
cfg.LogInstance = deps.Logger
|
||||
for _, option := range deps.AppOptions() {
|
||||
if option != nil {
|
||||
option(&cfg)
|
||||
}
|
||||
}
|
||||
cfg.Request = contract.MergePolicy(cfg.Request, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""})
|
||||
cfg.Response = newResponseTemplate("{\"body\":null,\"errorBody\":null,\"messageField\":\"msg\",\"requestIdField\":\"-\",\"codeType\":\"int32\",\"successCode\":0,\"errorCode\":9999,\"dataMode\":\"always\"}").Config()
|
||||
cfg.ResponseEncoder = nil
|
||||
if err := cfg.Validate(); err != nil {
|
||||
|
||||
@@ -36,7 +36,7 @@ var checkRequest = contract.MustNew[types.CheckReq]()
|
||||
|
||||
func (h *Handler) Check(c *httpx.Context) error {
|
||||
var req types.CheckReq
|
||||
if err := checkRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||
if err := checkRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &req, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := h.logic.Check(c.Context(), &req)
|
||||
@@ -53,7 +53,7 @@ var otherRequest = contract.MustNew[types.CheckReq]()
|
||||
|
||||
func (h *Handler) Other(c *httpx.Context) error {
|
||||
var req types.CheckReq
|
||||
if err := otherRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||
if err := otherRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &req, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := h.logic.Other(c.Context(), &req)
|
||||
@@ -65,7 +65,13 @@ func (h *Handler) Other(c *httpx.Context) error {
|
||||
}
|
||||
return c.OK(resp)
|
||||
}
|
||||
|
||||
var rawJSONRequest = contract.MustNew[struct{}]()
|
||||
|
||||
func (h *Handler) RawJSON(c *httpx.Context) error {
|
||||
if err := rawJSONRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := h.logic.RawJSON(c.Context())
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -75,7 +81,13 @@ func (h *Handler) RawJSON(c *httpx.Context) error {
|
||||
}
|
||||
return c.OK(resp)
|
||||
}
|
||||
|
||||
var uploadRequest = contract.MustNew[struct{}]()
|
||||
|
||||
func (h *Handler) Upload(c *httpx.Context) error {
|
||||
if err := uploadRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: "auto"}, 2048); err != nil {
|
||||
return err
|
||||
}
|
||||
body, err := httpx.NewBody(c.Request(), "multipart")
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -92,7 +104,13 @@ func (h *Handler) Upload(c *httpx.Context) error {
|
||||
}
|
||||
return c.OK(resp)
|
||||
}
|
||||
|
||||
var callbackRequest = contract.MustNew[struct{}]()
|
||||
|
||||
func (h *Handler) Callback(c *httpx.Context) error {
|
||||
if err := callbackRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: "auto"}, 64); err != nil {
|
||||
return err
|
||||
}
|
||||
body, err := httpx.NewBody(c.Request(), "raw")
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -112,7 +130,13 @@ func (h *Handler) Callback(c *httpx.Context) error {
|
||||
}
|
||||
return c.WriteStream(200, "text/plain", resp)
|
||||
}
|
||||
|
||||
var downloadRequest = contract.MustNew[struct{}]()
|
||||
|
||||
func (h *Handler) Download(c *httpx.Context) error {
|
||||
if err := downloadRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := h.logic.Download(c.Context())
|
||||
if err != nil {
|
||||
if resp != nil {
|
||||
@@ -122,7 +146,13 @@ func (h *Handler) Download(c *httpx.Context) error {
|
||||
}
|
||||
return c.WriteStream(200, "application/octet-stream", resp)
|
||||
}
|
||||
|
||||
var slowRequest = contract.MustNew[struct{}]()
|
||||
|
||||
func (h *Handler) Slow(c *httpx.Context) error {
|
||||
if err := slowRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil {
|
||||
return err
|
||||
}
|
||||
resp, err := h.logic.Slow(c.Context())
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,39 +0,0 @@
|
||||
package svc
|
||||
|
||||
import "errors"
|
||||
|
||||
// Close is idempotent, including startup failure and graceful shutdown.
|
||||
func (c *ServiceContext) Close() error {
|
||||
if c == nil {
|
||||
return nil
|
||||
}
|
||||
c.closeOnce.Do(func() {
|
||||
if c.Lifecycle != nil {
|
||||
c.Lifecycle.RequestShutdown(nil)
|
||||
}
|
||||
c.closeMu.Lock()
|
||||
c.closed = true
|
||||
cleanups := c.cleanups
|
||||
c.cleanups = nil
|
||||
c.closeMu.Unlock()
|
||||
for i := len(cleanups) - 1; i >= 0; i-- {
|
||||
c.closeErr = errors.Join(c.closeErr, cleanups[i]())
|
||||
}
|
||||
|
||||
})
|
||||
return c.closeErr
|
||||
}
|
||||
|
||||
// OnClose transfers a resource cleanup to this context immediately after creation.
|
||||
// Hooks run once in reverse order, also when later initialization fails.
|
||||
func (c *ServiceContext) OnClose(cleanup func() error) {
|
||||
if cleanup == nil {
|
||||
return
|
||||
}
|
||||
c.closeMu.Lock()
|
||||
defer c.closeMu.Unlock()
|
||||
if c.closed {
|
||||
panic("svc: register cleanup after Close")
|
||||
}
|
||||
c.cleanups = append(c.cleanups, cleanup)
|
||||
}
|
||||
56
examples/managedService/internal/svc/runtimeGen.go
Normal file
56
examples/managedService/internal/svc/runtimeGen.go
Normal file
@@ -0,0 +1,56 @@
|
||||
// Code generated by go-omni api. DO NOT EDIT.
|
||||
package svc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
|
||||
"example.com/managed/internal/config"
|
||||
"example.com/managed/internal/middleware"
|
||||
)
|
||||
|
||||
// New is generator-owned. Customize dependencies in ServiceContext.Init.
|
||||
func New(ctx context.Context, logger *logx.Logger, configs ...config.Config) (result *ServiceContext, err error) {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if err = ctx.Err(); err != nil {
|
||||
return nil, errors.Join(err, lifecycle.Cause(ctx))
|
||||
}
|
||||
if logger == nil {
|
||||
return nil, errors.New("service context requires a logger")
|
||||
}
|
||||
if len(configs) > 1 {
|
||||
return nil, errors.New("expected at most one configuration")
|
||||
}
|
||||
cfg := config.Default()
|
||||
if len(configs) == 1 {
|
||||
cfg = configs[0]
|
||||
}
|
||||
if err = cfg.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
checker, err := health.New()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := &ServiceContext{Config: cfg, Logger: logger, Health: checker, Middlewares: middleware.Defaults(), Runtime: lifecycle.NewRuntime(ctx)}
|
||||
ctx = logx.WithLogger(c.Context(), logger)
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
_ = c.Close()
|
||||
panic(p)
|
||||
}
|
||||
if err != nil {
|
||||
err = c.Finish(err)
|
||||
}
|
||||
}()
|
||||
|
||||
if err = c.Init(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return c, nil
|
||||
}
|
||||
@@ -2,66 +2,25 @@ package svc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
|
||||
"example.com/managed/internal/config"
|
||||
"example.com/managed/internal/middleware"
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
// Context owns infrastructure, while Logger is borrowed from main.
|
||||
// Add business repositories here; HTTP contracts do not define database tables.
|
||||
// Context holds project dependencies. Runtime owns cleanup and shutdown.
|
||||
// Logger is borrowed; the generated entry closes it after dependencies.
|
||||
type ServiceContext struct {
|
||||
Lifecycle *lifecycle.Controller
|
||||
*lifecycle.Runtime
|
||||
Config config.Config
|
||||
cleanups []func() error
|
||||
Logger *logx.Logger
|
||||
Health *health.Checker
|
||||
Middlewares map[string]httpx.Middleware
|
||||
|
||||
closeMu sync.Mutex
|
||||
closed bool
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
}
|
||||
|
||||
// Pass the settings loaded by main. The optional form supports lightweight
|
||||
// tests without infrastructure; selected adapters still require valid config.
|
||||
func New(ctx context.Context, logger *logx.Logger, configs ...config.Config) (result *ServiceContext, err error) {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if err = ctx.Err(); err != nil {
|
||||
return nil, errors.Join(err, lifecycle.Cause(ctx))
|
||||
}
|
||||
if len(configs) > 1 {
|
||||
return nil, errors.New("expected at most one configuration")
|
||||
}
|
||||
cfg := config.Default()
|
||||
if len(configs) == 1 {
|
||||
cfg = configs[0]
|
||||
}
|
||||
_ = cfg
|
||||
checker, err := health.New()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := &ServiceContext{Config: cfg, Logger: logger, Health: checker, Middlewares: middleware.Defaults(), Lifecycle: lifecycle.New(ctx)}
|
||||
ctx = c.Lifecycle.Context()
|
||||
defer func() {
|
||||
if err != nil {
|
||||
err = errors.Join(err, c.Close())
|
||||
if cause := c.Lifecycle.Err(); cause != nil && !errors.Is(err, cause) {
|
||||
err = errors.Join(err, cause)
|
||||
}
|
||||
}
|
||||
}()
|
||||
|
||||
return c, nil
|
||||
}
|
||||
|
||||
func (c *ServiceContext) RequestShutdown(err error) { c.Lifecycle.RequestShutdown(err) }
|
||||
// Init is project-owned. Add dependencies here and immediately register each
|
||||
// cleanup with c.OnClose. Returning an error automatically rolls everything back.
|
||||
// Generated infrastructure is ready here. Never create another App or logger.
|
||||
func (c *ServiceContext) Init(ctx context.Context) error { return nil }
|
||||
|
||||
@@ -0,0 +1,9 @@
|
||||
// Code generated by go-omni api. DO NOT EDIT.
|
||||
package types
|
||||
|
||||
type CheckReq struct {
|
||||
Kind string `form:"kind,optional"`
|
||||
}
|
||||
type Receipt struct {
|
||||
Bytes int64 `json:"bytes"`
|
||||
}
|
||||
@@ -1,9 +1,2 @@
|
||||
// Code generated by go-omni api. DO NOT EDIT.
|
||||
package types
|
||||
|
||||
type CheckReq struct {
|
||||
Kind string `form:"kind,optional"`
|
||||
}
|
||||
type Receipt struct {
|
||||
Bytes int64 `json:"bytes"`
|
||||
}
|
||||
|
||||
@@ -34,19 +34,13 @@ func run(ctx context.Context, path string) (runErr error) {
|
||||
return err
|
||||
}
|
||||
defer func() { runErr = errors.Join(runErr, logger.Close()) }()
|
||||
logx.SetDefault(logger)
|
||||
deps, err := svc.New(ctx, logger, cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// RunContext drains HTTP requests before returning. This scope owns dependencies
|
||||
// and closes them exactly once, before closing the borrowed logger.
|
||||
defer func() {
|
||||
runErr = errors.Join(runErr, deps.Close())
|
||||
if cause := deps.Lifecycle.Err(); cause != nil && !errors.Is(runErr, cause) {
|
||||
runErr = errors.Join(runErr, cause)
|
||||
}
|
||||
}()
|
||||
defer func() { runErr = deps.Finish(runErr) }()
|
||||
app, err := handler.NewApp(deps)
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user