Compare commits
12 Commits
v0.14.0-pr
...
v0.19.0-pr
| Author | SHA1 | Date | |
|---|---|---|---|
| fd83235ce7 | |||
| ad9257d202 | |||
| 4563a6354f | |||
| 5fc82509a1 | |||
| 8b588a24cd | |||
| 8493f9dae9 | |||
| 64e3bd6cfc | |||
| 1f2d8b6a6a | |||
| 16deead70e | |||
| adcb6f60b6 | |||
| b8ab6b3521 | |||
| da8356e18b |
8
AGENTS.md
Normal file
8
AGENTS.md
Normal file
@@ -0,0 +1,8 @@
|
||||
# 开发约定
|
||||
|
||||
- 框架保持通用,不内置接入项目的业务类型、字段、错误码或业务规则。
|
||||
- 2026-09-25 用户要求:每次功能开发完成,必须同步维护根 README.md。README 需要覆盖当前全部公开功能,解释用途、基本用法,并提供可运行示例或链接到 examples 内的完整示例。
|
||||
- 新增、调整和删除功能都必须同步文档、示例、测试与发布说明。删除功能要明确说明已移除内容、影响范围和迁移步骤,不能继续把历史用法当作当前能力推荐。
|
||||
- examples 使用通用场景,避免依赖业务项目、私有数据库或线上服务。必要的外部基础设施配置应明确说明。
|
||||
- 2026-09-25 用户决定:移除 mappings/mappingFiles 及结构映射代码生成机制,改用可选 copyx 工具包。不得恢复另一套 JSON 或 Go 映射声明语言。
|
||||
- 深拷贝的自定义转换结果不得与输入共享可变引用;发现共享应报错,不能降级成浅拷贝或静默接受。
|
||||
@@ -13,7 +13,9 @@
|
||||
- [x] 文档工具:字段注释、example/枚举描述进入 DTO 与 OpenAPI;保注释、幂等格式化及 check;显式 Go 错误目录元数据导出与文档合并,不自动执行项目代码。
|
||||
- [x] 共享契约:受控 roots/摘要、循环/深度/文件数/路径保护;分源文件 DTO、多个独立 module 的工作区检查;版本提示与基础模板升级预览,保留业务文件。
|
||||
- [x] 集成:实际生成项目验证新能力、连续生成、业务文件保留、失败清理、nethttp/native 请求与退出;现有错误继承/参数/上传下载回归。
|
||||
- [ ] 最终回归:全量 race(含生成器集成)、vet、build、格式化/链接检查;使用说明、迁移步骤及验收报告完整,发布后远程无 replace 验收。
|
||||
- [x] 最终回归:全量 race(含生成器集成)、vet、build、格式化/链接检查;使用说明、迁移步骤及验收报告完整,发布后远程无 replace 验收。
|
||||
|
||||
已发布 `v0.14.0-preview`。逐项验收、可替换的业务适配点与远程报告见 [验收记录](reports/businessDevelopmentAcceptance.md);接入和升级见 [开发指南](DEVELOPMENT.md)。
|
||||
|
||||
## 核心约定
|
||||
|
||||
|
||||
@@ -1,5 +1,14 @@
|
||||
# Changelog
|
||||
|
||||
## v0.19.0-preview(2026-09-25)
|
||||
|
||||
- 新增通用可选 copyx:构造器、浅/深拷贝、精确类型转换、字段规则、nil/零值策略、并发复用。
|
||||
- 深拷贝拒绝自定义结果共享源可变引用;原子提交目标、保留错误链、限制遍历及估算分配。
|
||||
- 移除 JSON 结构映射配置与生成;提供明确迁移诊断,保留公共字段组合等其他生成能力。
|
||||
- 同步 README 全部能力索引、COPY.md、可运行示例和开发文档约定。
|
||||
- 验收与不兼容迁移见 [发布说明](RELEASE.md)、[copyx 验收](reports/copyxAcceptance.md)。v0.14–v0.18 的完整变更保留在发布说明历史段落。
|
||||
|
||||
|
||||
## v0.13.0-preview(2026-09-24)
|
||||
|
||||
- 新增通用 lifecycle.Controller,svc.RequestShutdown 支持依赖主动通知正常停止或致命故障,通知不阻塞,首个故障原因保留;生成入口排空 HTTP 后关闭依赖与日志,故障退出码为 1。
|
||||
|
||||
141
COPY.md
Normal file
141
COPY.md
Normal file
@@ -0,0 +1,141 @@
|
||||
# 可选拷贝工具 copyx
|
||||
|
||||
v0.19 用普通 Go 调用取代 JSON 结构映射生成。`copyx` 只依赖标准库,不需要注册到 App,不执行 JSON 往返,也不会自动暴露数据库模型。普通 Logic 直接使用 `.api` 生成的 DTO;只有存在独立业务类型或需要隔离可变数据时才用它。
|
||||
|
||||
完整可运行代码:[examples/copying](examples/copying)。在框架根目录运行:
|
||||
|
||||
```sh
|
||||
go run ./examples/copying
|
||||
go test -race ./copyx ./examples/copying
|
||||
```
|
||||
|
||||
## 最简单的调用
|
||||
|
||||
```go
|
||||
import "git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
|
||||
type Request struct { Name string; Enabled bool }
|
||||
type Input struct { Name string; Enabled bool }
|
||||
|
||||
req := Request{Name: "sample", Enabled: false}
|
||||
var input Input
|
||||
if err := copyx.Copy(&input, req); err != nil { return err }
|
||||
```
|
||||
|
||||
同名、同类型字段直接赋值。默认不会跳过 `false`、`0`、空字符串。目标字段没有来源时报错;多余源字段默认忽略,使用 `WithStrictFields()` 可要求源字段也全部使用或显式忽略。
|
||||
|
||||
默认是浅拷贝:同类型指针、map、slice 共享引用,符合普通 Go 赋值。JSON tag 不参与字段匹配。不会自动调用方法、SQL Scanner/Valuer,或把 `int32`、`int64`、string、命名标量互相强转。
|
||||
|
||||
新字段:普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时,两边各声明一次;同名同类型不增加拷贝配置。检查在运行时发生,实际类型对必须有测试,不能把运行时工具当作旧生成器的编译期字段覆盖检查。
|
||||
|
||||
## 深拷贝与构造器
|
||||
|
||||
```go
|
||||
snapshot, err := copyx.Clone(source) // 保持源的静态类型,始终深拷贝
|
||||
if err != nil { return err }
|
||||
|
||||
copier, err := copyx.New(copyx.WithDeepCopy(), copyx.WithStrictFields())
|
||||
if err != nil { return err }
|
||||
var result Output
|
||||
if err := copier.Copy(&result, source); err != nil { return err }
|
||||
```
|
||||
|
||||
`Copy` 的目标必须是非 nil 指针;`Clone[T]` 返回 `(T, error)`。重复执行时在初始化阶段创建 Copier 并复用;配置之后不可变,每次调用独立持有遍历状态,可以并发使用。源、目标的并发读写仍由调用方同步,自定义函数也须支持并发。
|
||||
|
||||
深拷贝会分配指针、map、slice,递归处理结构、数组、接口中的具体值。nil 集合与非 nil 空集合保持区别;slice 结果容量为长度,不复制长度之外的隐藏元素。普通循环和重复指针可处理。循环图应保留根指针,使用 `Clone(sourcePointer)` 或 `Copy(&targetPointer, sourcePointer)`;将循环指针根压成值结构会明确报错。
|
||||
|
||||
它保证结果不引用输入中的可变对象,不保证保留所有底层内存布局关系:不同重叠 slice 视图会分别复制,内部指针指向某个切片元素的关系也不重建。字符串和零尺寸值可安全共享;`time.Time` 按 Go 不可变值使用(Location 不做独立复制)。不要通过 unsafe 或 uintptr 隐藏可变引用后要求工具识别。
|
||||
|
||||
## 字段差异与自定义转换
|
||||
|
||||
```go
|
||||
type Input struct { Name string; UpdatedAt time.Time; InternalID string }
|
||||
type View struct { DisplayName string; UpdatedAt string; LocalNote string }
|
||||
|
||||
copier, err := copyx.New(
|
||||
copyx.WithDeepCopy(),
|
||||
copyx.WithStrictFields(),
|
||||
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
|
||||
copyx.WithIgnoreSource[Input, View]("InternalID"),
|
||||
copyx.WithIgnoreTarget[Input, View]("LocalNote"),
|
||||
copyx.WithConverter(func(v time.Time) (string, error) {
|
||||
return v.UTC().Format(time.RFC3339), nil
|
||||
}),
|
||||
)
|
||||
if err != nil { return err }
|
||||
out := View{LocalNote: "preserve"}
|
||||
if err := copier.Copy(&out, input); err != nil { return err }
|
||||
```
|
||||
|
||||
字段规则的类型参数必须是结构类型,字段名使用导出的 Go 字段名。构造时检查不存在的字段、重名及冲突。不支持字符串路径表达式;复杂嵌套差异为该嵌套类型注册 converter,或直接写 Go。
|
||||
|
||||
`WithIgnoreSource` 只是声明该源字段不参与使用;目标同名字段若仍需保留,还需 `WithIgnoreTarget`。忽略目标会保留原目标值,Clone 中则为零值。深拷贝时如果保留值仍与源共享可变引用,也会拒绝。
|
||||
|
||||
转换器的签名是 `func(S) (T, error)`,按**精确类型对**匹配,不能重复注册。金额、编号、日期格式由项目明确转换。需要逐字段不同规则时,给整个源/目标结构注册函数;函数返回独立的新目标。无需另一份映射配置或全局注册。
|
||||
|
||||
精确 converter 优先于 nil/零值策略,能收到 typed nil;例如 `func(*Money) (Money, error)` 可自行决定 nil 是否允许。绑定到 `any` 字段的 converter 能接收 nil interface,但顶层未带类型的 `nil` 没有精确类型可匹配。浅拷贝中已经直接共享的同类型容器不会继续逐项转换其内部;需要遍历内部时选择深拷贝,或为该容器类型注册 converter。
|
||||
|
||||
## 深拷贝不能原样返回可变引用
|
||||
|
||||
```go
|
||||
copier, err := copyx.New(
|
||||
copyx.WithDeepCopy(),
|
||||
copyx.WithConverter(func(v []string) ([]string, error) {
|
||||
return v, nil // 会返回 ErrSharedReference
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
检查覆盖完整源图与转换结果,包括新父对象引用旧子对象、原 map、重叠子切片、`source[:0]` 的可写容量、指向源数组元素的指针,以及未导出字段中的引用。会检查每个 converter 的结果和最终目标,不允许静默降级为浅拷贝。
|
||||
|
||||
返回相同字符串、整数并不会报错,它们不是可变引用。正确切片 converter 可以写:
|
||||
|
||||
```go
|
||||
copyx.WithConverter(func(v []string) ([]string, error) {
|
||||
if v == nil { return nil, nil }
|
||||
out := make([]string, len(v))
|
||||
copy(out, v)
|
||||
return out, nil
|
||||
})
|
||||
```
|
||||
|
||||
第三方 Decimal/自定义 Money 如果含未导出的指针,默认深拷贝拒绝;注册该类型的 converter,使用它自己的构造/Clone API 创建独立值,避免转成 float64 损失精度。框架不内置某一金额库的规则。无法独立复制的类型直接返回错误。
|
||||
|
||||
## nil、零值和错误
|
||||
|
||||
| 选项 | 行为 |
|
||||
| --- | --- |
|
||||
| 默认 `NilError` | nil → 可空目标保留 nil;nil → 非可空目标报 `ErrNil` |
|
||||
| `WithNilPolicy(NilZero)` | 显式允许 nil → 零值;可空目标仍为 nil |
|
||||
| `WithNilPolicy(NilSkip)` | nil 保留已有目标;新集合元素没有旧目标,保留元素零值 |
|
||||
| `WithSkipZero()` | 显式跳过 Go 零值,包括 `false`、`0`、`""`;非 nil 空集合不是 Go 零值 |
|
||||
|
||||
所有 nil 策略都不把不兼容类型自动变成兼容类型。非 nil 指针与值之间可以按对应元素转换;nil 到非可空目标仍需选择策略。局部更新不能默认使用 NilZero/SkipZero,它们可能改变清空字段或布尔值的业务语义。
|
||||
|
||||
工具只能复制绑定后的 Go 值:如果输入类型已经将“缺失”和“JSON null”合并为 nil,无法恢复两者区别。需要三态时,应先设计包含 presence 信息的类型和绑定逻辑。
|
||||
|
||||
赋值使用临时结果,成功后才替换目标;转换失败不会交付部分结果。`*copyx.Error` 包含 Go 字段/索引路径,`errors.Is/As` 保留内部错误链,工具不主动在错误中加入源值。converter 自己返回的错误文本仍由项目负责。
|
||||
|
||||
converter 必须只读输入;其外部副作用、输入修改无法回滚,panic 原样传播。框架只检查与本次输入的共享引用,无法证明 converter 未返回外部全局缓存的引用。它是受信 Go 代码扩展,不是可运行不可信代码的沙箱。
|
||||
|
||||
## 支持范围与资源限制
|
||||
|
||||
- 支持结构体、指针、interface、slice、同长度 array、map。map key 转换后冲突或不可比较则报错。
|
||||
- 导出匿名结构字段可按提升后的无歧义名称访问;源嵌入指针为 nil 且需要其字段时明确报错。跨类型目标包含私有字段时拒绝;同类型私有纯值可作为整体保留。
|
||||
- 不克隆文件、数据库连接、网络连接、锁、原子状态。已知标准库资源/同步结构按值复制也拒绝;深拷贝中的非 nil func、chan、unsafe.Pointer 拒绝。其他封装资源类型应由项目排除,工具不能识别所有自定义句柄。
|
||||
- 单次遍历深度上限 256、节点上限 200,000;内置分配的估算预算为 64 MiB,超限返回 `ErrLimit`。预算不是进程堆内存硬限制,reflect 元数据和 converter 自行分配不计入。大对象流式读写应使用 IO/存储接口。
|
||||
- 只缓存不可变类型元数据,不缓存请求值、用户 converter 或实例策略。没有 `unsafe` 写入、JSON 往返、隐式数值截断或方法调用。
|
||||
- 这是运行时反射工具,会比直接 Go 赋值多检查与分配。热点应复用 Copier,并用真实数据测量;性能记录见 [本版验收](reports/copyxAcceptance.md)。
|
||||
|
||||
## 从旧 mapping 迁移
|
||||
|
||||
v0.19 **彻底移除**映射配置加载、Go 类型加载、映射代码生成及 `pointerResult` 等附属选项。公共 `.api` 字段组合、scalar、中间件工厂、Repository 开关继续存在。
|
||||
|
||||
1. 先检查每个转换是否必要。普通 Logic 可直接使用生成 DTO;独立核心保留自己的类型。
|
||||
2. 将 `mapping.Xxx(...)` 调用改为普通 Go 或小型 `copyx` helper。字段改名、nil、金额规则写在该 helper 同一处。指针返回可直接 `var out *Response; err := copier.Copy(&out, input)`。
|
||||
3. 用真实类型对测试缺失/null/零值/空数组、字段覆盖、金额精度和错误链。需要源字段新增时提醒遗漏,启用 `WithStrictFields()` 并测试。
|
||||
4. 从 `omni.gen.json` 删除 `mappings` / `mappingFiles`,删除无其他用途的映射 JSON 文件。
|
||||
5. 确认已无调用,再删除 `internal/mapping/mappingsGen.go`。如之前手改过文件,先迁出手写内容。生成器遇到该旧文件会阻止生成并提示迁移,**不会静默删除**;其余业务文件照常保留。
|
||||
6. 升级框架与 CLI 到同版,执行 generate、generate --check、go test ./...;多个服务再执行 workspace generate/check。新 manifest 自动去掉已移除文件的所有权记录。
|
||||
|
||||
历史发布记录保留旧能力说明,仅用于理解旧版本,不代表 v0.19 继续支持。
|
||||
127
DEVELOPMENT.md
127
DEVELOPMENT.md
@@ -1,7 +1,9 @@
|
||||
# 通用业务开发指南(v0.14.0-preview)
|
||||
# 通用业务开发指南(v0.19.0-preview)
|
||||
|
||||
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
|
||||
|
||||
公共字段组合、自动中间件工厂和 Repository 开关,见 [生成能力指南](GENERATION.md)。必要的结构转换使用可选 [copyx](COPY.md),v0.19 已移除旧 JSON 映射生成。
|
||||
|
||||
## 返回数组、可空对象和 null
|
||||
|
||||
```text
|
||||
@@ -21,12 +23,17 @@ delete /item
|
||||
| 契约 | Logic 返回 | 成功数据 |
|
||||
| --- | --- | --- |
|
||||
| `returns (Item)` | `(*types.Item, error)` | 对象;nil 被视为实现错误 |
|
||||
| `returns ([]Item)` | `([]types.Item, error)` | 数组;nil 统一输出 `[]` |
|
||||
| `returns ([]Item)` | `([]types.Item, error)` | 对象数组;nil 统一输出 `[]` |
|
||||
| `returns ([]string)` | `([]string, error)` | 字符串数组 |
|
||||
| `returns ([]int64)` / `([]bool)` | `([]int64, error)` / `([]bool, error)` | 数值 / 布尔数组 |
|
||||
| `returns ([]any)` | `([]any, error)` | 动态 JSON 数组,元素允许对象、数组、标量和 null |
|
||||
| `returns (*Item)` | `(*types.Item, error)` | 对象或 `null` |
|
||||
| `returns (null)` | `error` | HTTP 200 和空业务数据 |
|
||||
| 不写 returns | `error` | HTTP 204,无响应体 |
|
||||
|
||||
统一响应中的空数据是否显示,仍遵循项目模板的 `dataMode`;需要 `data:null` 就设置 `dataMode: always`。`@response raw` 输出业务数据本身,包括 `[]`、对象或 `null`。可用 `@status 201/202`;204 不能声明响应类型。根类型当前支持以上四种形式,嵌套字段继续支持既有 slice/map/pointer/scalar/any。
|
||||
统一响应中的空数据是否显示,仍遵循项目模板的 `dataMode`;需要 `data:null` 就设置 `dataMode: always`。`@response raw` 输出业务数据本身,包括 `[]`、对象或 `null`。可用 `@status 201/202`;204 不能声明响应类型。嵌套字段继续支持既有 slice/map/pointer/scalar/any。
|
||||
|
||||
基础类型数组还支持 int/int8/int16/int32、uint/uint16/uint32/uint64、float32/float64。nil 根数组都输出 `[]`;`[]any` 在 OpenAPI 中是 `type: array, items: {}`,不限制元素形状,但元素必须可被标准 JSON 编码器序列化。`[]uint8`(Go 的 byte slice)会编码为 base64,故明确拒绝;数字数组可用 `[]int32`,二进制用下载接口。当前不支持根嵌套数组或根自定义 scalar 数组。
|
||||
|
||||
## 契约驱动的请求策略
|
||||
|
||||
@@ -113,6 +120,23 @@ duration 使用 `250ms`、`3s`;slice/map 使用 JSON,如 `[1,3,5]`、`{"retr
|
||||
|
||||
手写项目可使用 `config.LoadProject(path, defaults, config.EnvOptions{Prefix:"MY_APP_"})`,或分别调用 `ApplyDefaults/DecodeFile/OverlayEnv`。读取失败不修改输入默认值;诊断不会打印输入值。不要主动打印含密钥的整个配置,或把底层解析错误展开到公开响应。Validate 不打开日志文件、数据库或存储连接;证书的可读性和握手信任在依赖初始化阶段验证。
|
||||
|
||||
### 一次覆盖完整配置与 HTTP/日志短名称
|
||||
|
||||
```go
|
||||
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{
|
||||
Prefix: "MY_APP_",
|
||||
Empty: "reject", // allow / ignore / reject,项目自行选择
|
||||
Aliases: config.ServerEnvAliases("MY_APP_", "server"),
|
||||
// Lookup 可注入测试或项目配置源;默认 os.LookupEnv。
|
||||
})
|
||||
```
|
||||
|
||||
不要再紧接着调用 `ApplyEnv(cfg.Server, ...)`。上面的调用同时处理完整配置和 HTTP_ADDR、SERVICE、LOG_LEVEL 等框架短名称。例如 `MY_APP_SERVER_ADDRESS` 与 `MY_APP_HTTP_ADDR` 指向同一字段,**完整名称优先**;未设置时才读取短名称。完整名称显式设空仍优先:allow 交给该字段解析(string 可清空;数字等仍要求合法值),ignore 保留文件/默认值,reject 报错。不会因 ignore 再退回别名。字段 `env` 标签的 empty 规则优先于全局规则。
|
||||
|
||||
自定义别名用 `EnvOptions.Aliases`,键是完整环境名、值是按优先级排列的别名数组。两种名称都使用同一 Lookup、空值策略和类型转换;布尔 false、数值 0 与未设置不同。集合始终使用 JSON,包括 `MY_APP_TRUSTED_PROXIES='["127.0.0.1"]'`。
|
||||
|
||||
新项目模板一次调用 OverlayEnv,默认 `Empty: "ignore"`。旧 `ApplyEnv` 保留原有忽略空值、修剪空白及 TRUSTED_PROXIES 逗号列表行为。已有项目的 `internal/config` 属于开发者文件,升级不覆盖它;按上例合并原来的两次覆盖调用后,保留项目自己的 Validate。日志输出的显式完整名称也优先于短名称。
|
||||
|
||||
## 依赖、日志与中间件
|
||||
|
||||
`internal/svc/runtimeGen.go` 持续由生成器维护。项目的 ServiceContext 嵌入 `*lifecycle.Runtime`,只在 `Init(ctx)` 中装配额外依赖:
|
||||
@@ -123,40 +147,15 @@ func (c *ServiceContext) Init(ctx context.Context) error {
|
||||
if err != nil { return err }
|
||||
c.Client = client
|
||||
c.OnClose(client.Close)
|
||||
c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(client)
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
每个资源初始化成功就登记清理;后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求,再逆序关闭依赖,最后关闭日志。关闭幂等,单个清理函数 panic/返回错误不会跳过其他清理;初始化 panic 清理后继续传播。项目清理函数仍须自行停止、等待后台任务并保证能返回,框架不能强制杀死任意 goroutine。
|
||||
每个资源初始化成功就登记清理;后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求,再执行后台组件的 Stop/Drain 阶段,确认收尾后逆序关闭依赖,最后关闭日志。关闭幂等,单个清理函数 panic/返回错误不会跳过其他清理;初始化 panic 清理后继续传播。后台组件按 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭) 登记;无法确认收尾完成时保留共享依赖并异常退出,不能把取消等同于任务已经结束。
|
||||
|
||||
中间件用业务所需的小接口接收依赖,避免反向导入 svc:
|
||||
新项目使用中间件工厂:项目维护 `middleware.Dependencies` 小接口与 `New<Name>(deps) (httpx.Middleware, error)`,svc 提供对应访问器。生成的 runtime 在 Init 成功后自动调用工厂并注册,不再按名字替换 map;工厂失败会释放已经登记的依赖。示例与现有项目迁移步骤见 [中间件工厂](GENERATION.md#中间件工厂自动注册)。
|
||||
|
||||
```go
|
||||
// internal/middleware/accessFactory.go
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"context"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type AccessChecker interface {
|
||||
CheckAccess(context.Context, string) error
|
||||
}
|
||||
func NewRequireAccess(checker AccessChecker) httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
|
||||
return err
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`.api` 声明对应 middleware 后,保留生成的初始 hook,并另建上述工厂;在 Init 用工厂替换 registry 的同名项即可。该包只依赖接口和 HTTP 抽象,不导入 svc,也不把数据库藏到 context 里。
|
||||
v0.16 项目及 `omni.gen.json` 中选择 `middleware: handlers` 的项目继续使用原有 `Defaults()`,可以在 Init 设置 `c.Middlewares["RequireAccess"]`。更新时不会重写这部分业务代码;只有显式切换到 factories 后才使用新流程。
|
||||
|
||||
生成入口为每个 App 注入独立 Logger,不修改进程默认 Logger。业务可用 `c.Logger.InfoContext(...)`,或请求链中的 `logx.InfoContext(ctx,...)`。标准库/第三方需要 `*log.Logger` 时使用 `c.Logger.Standard(slog.LevelWarn)`;适配器借用原 Logger,不负责关闭。手写多 App 用 `omni.WithLogInstance(logger)` 显式注入。
|
||||
|
||||
@@ -185,21 +184,9 @@ go-omni api generate --check
|
||||
|
||||
格式化保留注释、token 和多行字面量,规范四空格缩进、空行、行首尾空白及相邻单行字段的类型/标签/行尾注释对齐;结果幂等。先校验完整契约,check 不写文件;导入的共享目录不被某个服务的格式化命令修改。DTO 按源 `.api` 输出 `dto<Source><摘要>Gen.go`,都属于 `types` 包,跨文件引用无需改包名。
|
||||
|
||||
Go Catalog 是错误的唯一来源时,显式导出文档元数据:
|
||||
Go Catalog 是错误的唯一来源时,优先使用 workspace 的显式导出配置,不再维护导出/复制脚本。AppOptions 仍通过 `omni.WithErrors(apierrors.Catalog())` 注册同一个 Go 函数,运行期只注册一次。单个项目也可继续使用 `errors.ExportCatalog` 导出入口目录的 `api/omni.errors.json`;此文件和下面的共享 source 二选一。
|
||||
|
||||
```go
|
||||
// 在项目自己的导出命令中;appErrors.Catalog 与 AppOptions 注册同一函数。
|
||||
if err := omnierrors.ExportCatalog(os.Stdout, appErrors.Catalog()); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
```
|
||||
|
||||
```sh
|
||||
go run ./cmd/exportErrors > api/omni.errors.json
|
||||
go-omni api generate
|
||||
```
|
||||
|
||||
文件是派生产物,应纳入版本控制并由业务 CI 检查导出漂移。CLI 自动读取入口目录的 `omni.errors.json`,校验错误目录、合并状态/模板参数/具体示例到 OpenAPI;不会运行项目 Go,不创建另一份运行期注册器。带独立错误目录的路由继续使用自己的目录。没有路由级精确错误列表时,文档描述 App 目录的公共错误集合;特殊自定义 Encoder 的文档由项目显式扩展。
|
||||
OpenAPI 自动把有收益的重复响应、schema 和示例移到 `components.responses/schemas/examples`,通过标准 `$ref` 引用。HTTP 200 同时含成功和业务失败时保留两种 schema、业务码、模板参数示例和 data:null;业务响应内容不会改变。消费 OpenAPI 的工具需要支持标准本地引用。
|
||||
|
||||
## 两个服务共享契约
|
||||
|
||||
@@ -241,9 +228,57 @@ go-omni workspace check --file omni.workspace.json
|
||||
|
||||
各服务保持独立 module;可显式填写各服务的 `api`(相对服务目录)和 `module`。所有服务预检通过后再写;每个服务独立提交文件事务,不承诺跨项目的崩溃原子性。check 汇总格式与生成漂移;业务编译/测试仍在各 module 执行。
|
||||
|
||||
### 同一份 Go 错误目录供多个服务生成
|
||||
|
||||
在 workspace 清单显式声明提供目录的 Go 包和函数:
|
||||
|
||||
```json
|
||||
{
|
||||
"errorCatalogs": [{
|
||||
"dir": ".",
|
||||
"package": "example.com/project/internal/apierrors",
|
||||
"function": "Catalog",
|
||||
"output": "contracts/common/errors.json"
|
||||
}],
|
||||
"services": [{"dir":"services/api"}, {"dir":"services/id"}]
|
||||
}
|
||||
```
|
||||
|
||||
`dir` 相对 workspace,必须包含 go.mod;函数签名为 `func Catalog() errors.Catalog`。允许 module 内的 internal 包。先准备 output 的父目录。生成器构建一个临时导出程序,使用 Go overlay、临时 go.mod/go.sum 和 `-mod=readonly`,不会往项目增加导出命令或修改 go.mod/go.sum。最多 16 个提供方,每个有 90 秒构建/执行总时限和 1 MiB stdout 限制。错误消息不回显程序输出。
|
||||
|
||||
只有显式运行 **workspace generate/check** 会执行清单指定的 Go 包及其依赖初始化;不要指向不信任的包。提供目录的函数应只构造目录,不连接数据库、启动服务、写 stdout 或修改项目。框架不扫描其他包,也不通过 shell 执行配置字符串。普通 api validate/export/generate 和 workspace format 只读取现有元数据。
|
||||
|
||||
各服务在自己的 `api/omni.imports.json` 引用共享 JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"roots": {"common":{"path":"../../../contracts/common","version":"1.0.0"}},
|
||||
"errors": {
|
||||
"source": "@common/errors.json",
|
||||
"include": ["USERNAME_INVALID", "INTERNAL_ERROR"],
|
||||
"routes": {"GET /permissions":["INTERNAL_ERROR"]}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`include` 省略或 null 表示全部目录,`[]` 表示不列业务错误;routes 按 **大写方法 + 空格 + .api 原路径**(如 `GET /users/:id`)覆盖服务级列表。错误标识、路由拼错或重复标识会明确报错。路由已经通过 `.api @errors` 声明独立运行期目录时,不能再给它设置文档 routes 列表。
|
||||
|
||||
**这些列表只决定文档展示,不修改 App 错误目录、路由继承或运行期可返回的错误。** `.api` 不需要重复定义 Go 错误表。状态、业务码、提示和模板参数仍来自同一 Catalog;特殊自定义 Encoder 的文档由项目显式扩展。
|
||||
|
||||
```sh
|
||||
# 首次生成导出一次共享目录,所有服务预检通过才写入
|
||||
# 共享 JSON 不复制到各服务;已有 api/omni.errors.json 请显式移除后改用 source。
|
||||
go-omni workspace generate --file omni.workspace.json
|
||||
go-omni workspace format --file omni.workspace.json
|
||||
go-omni workspace generate --file omni.workspace.json
|
||||
go-omni workspace check --file omni.workspace.json
|
||||
```
|
||||
|
||||
共享 JSON 应纳入版本控制。check 在内存重新导出并比较 JSON,然后检查所有服务;缺失或过期时失败,不更新项目文件。Go 构建缓存仍可能更新;显式提供方的自有副作用不属于生成器可隔离范围。业务 CI 仍执行各 module 的编译与测试。目录函数本身有变化时重新 generate,受控共享根的固定 SHA256 如需改变仍由项目显式更新。
|
||||
|
||||
## v0.13 项目升级
|
||||
|
||||
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.14.0-preview(或显式引用当前源码)。
|
||||
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.19.0-preview(或显式引用当前源码)。
|
||||
|
||||
```sh
|
||||
go-omni api upgrade --dir . --dry-run
|
||||
|
||||
116
GENERATION.md
Normal file
116
GENERATION.md
Normal file
@@ -0,0 +1,116 @@
|
||||
# 公共字段、中间件工厂与生成选项
|
||||
|
||||
本页对应 `v0.19.0-preview`。CLI 与框架依赖固定到同一版本,安装见 [README](README.md#准备环境与安装)。
|
||||
|
||||
## 一份项目生成配置
|
||||
|
||||
新项目根目录的 `omni.gen.json` 只描述生成选项:
|
||||
|
||||
```json
|
||||
{
|
||||
"repository": "none",
|
||||
"middleware": "factories"
|
||||
}
|
||||
```
|
||||
|
||||
- `repository: none` 默认不创建空 Repository 文件;`group` 按 group 补齐可编辑骨架。已有文件始终保留;切回 none 不会删除业务文件。SQL、ORM 和仓储划分由项目决定。
|
||||
- `middleware: factories` 是新项目默认值,自动调用带依赖的工厂。`handlers` 保留原有 `Defaults()` 及手动注册方式,供既有项目迁移。
|
||||
- 该文件与 `.api` 一起提交。generate/check/dry-run 和 workspace 采用相同规则;validate/export/format 只处理接口契约。
|
||||
- 旧 manifest 没有配置时补 `middleware: handlers`、`repository: none`。已有配置不能直接删除,避免意外切换中间件模式。
|
||||
- v0.19 已删除 `mappings` 和 `mappingFiles`;遇到旧配置会明确报错,不读取子映射文件或执行映射生成。处理步骤见 [copyx 迁移](COPY.md#从旧-mapping-迁移)。
|
||||
|
||||
## 公共字段组合
|
||||
|
||||
```text
|
||||
type EmployeeFields {
|
||||
// 员工姓名。
|
||||
Name string `json:"name" binding:"max=80"`
|
||||
Salary Money `json:"salary"`
|
||||
}
|
||||
|
||||
type EnterpriseRequest {
|
||||
EmployeeFields
|
||||
}
|
||||
|
||||
type AdminRequest {
|
||||
EmployeeFields
|
||||
CompanyID string `path:"companyId"`
|
||||
}
|
||||
```
|
||||
|
||||
`Money` 由项目按既有 scalar 语法声明。组合行只有已声明的对象类型名,不加指针或字段 tag。公共对象可以通过已有相对 import 或声明过的共享根 `@common/...` 引入。
|
||||
|
||||
生成器展开为普通 Go 字段。JSON 保持平铺;字段的绑定来源、校验、注释、可空性和 OpenAPI 使用同一份展开结果。不会在 JSON 中增加 `EmployeeFields` 一层。重复 Go 字段、重复来源字段名、循环组合、未知对象和过深展开均失败;即使重复字段类型相同,也不能隐式覆盖。要改变某个字段的规则,应拆出更小的公共组合。
|
||||
|
||||
允许多层组合,上限 32 层、单类型 4096 字段、全契约展开 65536 字段。Go 匿名指针嵌入并非本功能;生成 DTO 中不会使用匿名嵌入。
|
||||
|
||||
## DTO 与业务类型
|
||||
|
||||
普通 Logic 直接接收生成的 Request、返回生成的 Response,不额外创建相同结构。只有独立核心或明确不同的输入/输出职责才保留业务类型。
|
||||
|
||||
必要转换直接写 Go,或用可选 `copyx.Copy(&target, source)`。同名同类型字段自动赋值,字段改名和类型变化通过 Go 参数、转换函数明确设置,见 [完整示例](examples/copying/main.go)。无需 JSON 映射配置、包路径字符串或额外生成文件。
|
||||
|
||||
`copyx` 在运行时检查类型和字段,**不提供旧映射生成阶段的静态字段覆盖保证**;CI 应给实际类型对添加测试。普通业务字段新增通常只改 `.api` 和 Logic;独立 DTO/领域类型都需要此字段时,两边各加一次,同名同类型无需追加拷贝配置。
|
||||
|
||||
## 中间件工厂自动注册
|
||||
|
||||
`.api` 沿用 `middleware: RequireAccess` 或 auth 对应的中间件声明。新项目会生成可编辑的 `internal/middleware/requireAccessFactory.go`,只需实现一次:
|
||||
|
||||
```go
|
||||
// internal/middleware/dependencies.go,由项目维护。
|
||||
type AccessChecker interface {
|
||||
CheckAccess(context.Context, string) error
|
||||
}
|
||||
type Dependencies interface {
|
||||
AccessChecker() AccessChecker
|
||||
}
|
||||
|
||||
// internal/middleware/requireAccessFactory.go,由项目维护。
|
||||
func NewRequireAccess(deps Dependencies) (httpx.Middleware, error) {
|
||||
checker := deps.AccessChecker()
|
||||
if checker == nil { return nil, errors.New("access checker is required") }
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
|
||||
return err
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}, nil
|
||||
}
|
||||
|
||||
// internal/svc/access.go,由项目维护;ServiceContext.Init 初始化 c.Auth。
|
||||
func (c *ServiceContext) AccessChecker() middleware.AccessChecker { return c.Auth }
|
||||
```
|
||||
|
||||
编译器检查 ServiceContext 是否实现依赖接口,middleware 不导入 svc。多个工厂可以共用这些小接口。所需 imports 和具体鉴权服务仍属于项目。
|
||||
|
||||
启动顺序为:框架基础依赖 → `svc.Init` → 生成的 `middleware.New(c)` 调用各具名工厂 → 创建 App/注册路由 → 监听。中间件实例创建一次,由请求并发复用;不要把单次请求信息存在实例字段里。工厂之间没有初始化顺序依赖;路由执行顺序仍以 `.api` 声明为准。
|
||||
|
||||
工厂返回错误或 nil 中间件均阻止监听;错误链保留,已登记资源按现有生命周期清理。工厂 panic 也先清理依赖再传播。资源初始化和清理登记放在 svc.Init,工厂借用资源;HTTP 请求排空之前不会关闭这些依赖。
|
||||
|
||||
factory 模式不再在 Init 写 `Middlewares["..."]`;残留非空手写注册表会明确报错,防止生成注册静默覆盖它。未实现的工厂让启动失败;未实现的 Logic 仍返回 501。
|
||||
|
||||
### 既有项目迁移
|
||||
|
||||
1. 首次更新保持 `middleware: handlers`,现有 hook 和手写 Configure 不变。
|
||||
2. 改为 factories 并重新生成,会补 `dependencies.go` 和 `*Factory.go`,保留原来的手写文件。
|
||||
3. 将旧实现迁入工厂,提供 Dependencies 的访问器;移除 Init 中按名字覆盖 map 的代码。
|
||||
4. 旧函数不再需要时由项目删除。若新工厂符号与旧文件中的同名函数冲突,整理旧文件;生成器不覆盖业务代码。
|
||||
5. `generate --check` 与 `go test ./...` 检查后再运行。工厂没有全部实现前,服务不会启动。
|
||||
|
||||
## 文件维护边界
|
||||
|
||||
| 项目维护 | 生成器持续维护 |
|
||||
| --- | --- |
|
||||
| `.api` 公共字段与接口 | 展开后的 DTO、Handler、OpenAPI |
|
||||
| `omni.gen.json` 的 Repository/中间件选项 | 对应骨架和注册代码;已有手写骨架不覆盖 |
|
||||
| 独立 Go 业务类型、scalar、可选 copyx 调用和测试 | 不生成结构转换文件 |
|
||||
| middleware Dependencies、具名工厂、svc 访问器 | 中间件 registry、启动调用 |
|
||||
| Logic、Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 |
|
||||
|
||||
独立三端工资契约测试位于 `cmd/go-omni/apigen/testdata/composition`,不导入任何接入项目。它验证公共字段、Money scalar、三端 Handler、copyx、重新生成保留业务内容,以及新增字段不需要更新转换规则。
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1
|
||||
```
|
||||
18
PROJECT.md
18
PROJECT.md
@@ -1,6 +1,6 @@
|
||||
# 生成项目与文件维护边界
|
||||
|
||||
v0.14.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
||||
v0.19.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
||||
|
||||
## 创建和日常开发
|
||||
|
||||
@@ -38,6 +38,8 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
|
||||
|
||||
这是轻量源码分析,不运行项目 init、不执行 Go 命令、不下载依赖。项目自定义类型别名、嵌入方法或 build tags 等无法可靠判断时,会给 REVIEW 并交给 Go 编译器验证;不把这些情况强行判为签名错误。它不检查方法体是否引用已删除字段,也不证明业务逻辑正确。CI 中应同时执行 `go-omni api generate --check` 和 `go test ./...`,按项目实际构建参数测试。当前版本首次接手缺少快照的 manifest v2 时,会明确提示建立基线,不猜测旧契约;Logic 签名检查仍会执行。
|
||||
|
||||
本版新增生成选项、公共字段组合和映射,详见 [生成能力指南](GENERATION.md)。
|
||||
|
||||
## 哪些文件由用户维护
|
||||
|
||||
| 文件 | 首次生成后由项目维护 |
|
||||
@@ -50,8 +52,10 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
|
||||
| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 |
|
||||
| `internal/svc/storage.go` | 选用存储时生成,之后保留 |
|
||||
| `internal/logic/<group>/*.go` | 业务方法,使用标准 context.Context |
|
||||
| `internal/repository/<group>/*.go` | 数据访问,按需实现 |
|
||||
| `internal/middleware/<name>.go` | 项目鉴权/权限等具名中间件实现 |
|
||||
| `internal/repository/<group>/*.go` | 数据访问;默认不补空骨架,可选 group 模式 |
|
||||
| `internal/middleware/*Factory.go`、`dependencies.go` | 中间件工厂与依赖小接口,svc 实现访问器 |
|
||||
| `omni.gen.json` | Repository 和中间件生成选项;不再声明结构映射 |
|
||||
| 可选的 Go 转换函数 / copyx 调用 | 只有独立类型确实需要转换时维护,见 [copyx](COPY.md) |
|
||||
| 项目 Go 错误包 | 自行维护 Catalog 和常量,可跨服务共用 |
|
||||
| 自定义标量文件、`*Bind.go`、`*Manual.go` | 只有显式选择相应扩展时才生成/使用 |
|
||||
| 项目测试、迁移、部署脚本、go.mod/go.sum | 项目所有,生成器不重置其内容 |
|
||||
@@ -85,7 +89,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
|
||||
}
|
||||
```
|
||||
|
||||
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。静态 OpenAPI 不执行 Go;可从同一 Catalog 显式导出 `api/omni.errors.json`,详见 [元数据导出](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
|
||||
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。普通 `.api` 编译不执行 Go;可在 workspace 显式配置 Catalog 函数,统一导出共享文档元数据,也可沿用 `api/omni.errors.json`,详见 [元数据导出](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
|
||||
|
||||
`GlobalMiddleware()` 提供全局中间件;`Middlewares` 映射提供 `.api` 声明的具名中间件。全部显式调用,不依赖项目 init 注册,也不要求 svc 导入 handler。框架请求链按原有次序运行。
|
||||
|
||||
@@ -93,7 +97,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
|
||||
|
||||
最终配置校验包含自定义响应编码器与错误目录的组合。码类型/映射冲突、编码器已绑定另一份目录等问题在创建 App 前返回错误,进入入口的清理流程,不等到构造 App 时才 panic。
|
||||
|
||||
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后按框架规则强制关闭连接。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,在 OnClose 中取消。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
|
||||
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后按框架规则强制关闭连接。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,使用 WorkContext 并登记成对的 OnStop/OnDrain,详见 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭)。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
|
||||
|
||||
## 依赖主动申请退出
|
||||
|
||||
@@ -106,7 +110,7 @@ c.RequestShutdown(err)
|
||||
|
||||
这只是通知,立即返回,不等待 HTTP 排空,也不直接关闭数据库。生成入口停止接收新请求,等待在途请求完成;超过 `shutdownTimeout` 后取消请求并关闭连接,再执行 svc.Close,最后关闭日志。非空故障原因保留在返回错误链中,进程退出码为 1;`RequestShutdown(nil)` 或正常 SIGINT/SIGTERM 在没有其他故障时退出码为 0。
|
||||
|
||||
多个依赖可并发通知,关闭流程仅执行一次,首个非空故障原因保留。正常信号已开始排空时收到的故障也不会被清理动作覆盖。初始化期间的通知同样有效,不依赖 HTTP 监听已经启动。业务应在 Close/OnClose 中停止并等待自己启动的监控任务,再结束资源清理;关闭完成以后才报告的故障无法参与已经结束的进程结果。
|
||||
多个依赖可并发通知,关闭流程仅执行一次,首个非空故障原因保留。正常信号已开始排空时收到的故障也不会被清理动作覆盖。初始化期间的通知同样有效,不依赖 HTTP 监听已经启动。后台监控任务应通过 OnStop/OnDrain 停止并等待,再进入资源清理;关闭完成以后才报告的故障无法参与已经结束的进程结果。
|
||||
|
||||
`Lifecycle *lifecycle.Controller` 是 svc 持有的通用控制器,初始化时创建,生成入口使用它的 Context。该 Context 表示服务正在停止,不代表数据库已经关闭。不能把它当作在途请求的 Context,也不要在它取消时立刻关闭请求仍依赖的资源。发号器失去租约后拒绝发号等业务安全规则仍由项目实现。
|
||||
|
||||
@@ -208,7 +212,7 @@ get /download (DownloadReq)
|
||||
|
||||
## 已有服务接入步骤
|
||||
|
||||
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.14.0-preview:
|
||||
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.19.0-preview:
|
||||
|
||||
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
|
||||
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。
|
||||
|
||||
101
README.md
101
README.md
@@ -2,13 +2,57 @@
|
||||
|
||||
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
|
||||
|
||||
**从这份 README 开始接入。** 当前版本 **`v0.14.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
|
||||
**从这份 README 开始接入。** 当前版本 **`v0.19.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
|
||||
|
||||
**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。
|
||||
|
||||
普通接口的流程是:**修改 `.api` → 生成 Request/Response → 写 Logic → 测试**。有独立职责的业务类型才需要转换;可直接写 Go,也可以按需使用 [copyx](COPY.md)。公共字段组合、中间件工厂、Repository 开关继续支持,见 [生成能力指南](GENERATION.md)。
|
||||
|
||||
Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
|
||||
|
||||
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。
|
||||
|
||||
新增通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
|
||||
契约支持 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录,以及统一环境变量别名覆盖。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。
|
||||
|
||||
已有通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
|
||||
|
||||
后台组件可在 svc 登记 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭),先停止接收工作并等待收尾,再关闭共享资源;无需绑定队列或 ORM。
|
||||
|
||||
使用 GORM 的项目可以选择 [gormx 适配器](database/gormx/README.md),复用数据库初始化和连接池,并将 SQL 日志接入 logx。
|
||||
|
||||
## 当前功能与使用入口
|
||||
|
||||
下表是当前公开能力的索引。每项给出调用入口和完整说明;模块按需使用,`omni.New()` 不会自动初始化数据库、Redis、ORM、队列或云存储。示例中的 import 统一以 `git.yuntuo.hk/k2safe/go-omni/` 开头。
|
||||
|
||||
| 功能与用途 | 怎么用 | 解释与示例 |
|
||||
| --- | --- | --- |
|
||||
| 项目、接口与文档生成 | `go-omni new`;改 `.api` 后 `api generate`;`--check` 检查漂移 | [下方快速开始](#新项目生成并运行)、[CLI](cmd/go-omni/README.md)、[完整生成项目](examples/managedService/README.md) |
|
||||
| 公共字段与跨服务共享契约 | `.api` 组合对象字段;`omni.imports.json` 声明共享根;workspace 批量生成 | [公共字段示例](GENERATION.md#公共字段组合)、[共享契约与 workspace](DEVELOPMENT.md) |
|
||||
| 中间件依赖工厂、可选 Repository | `omni.gen.json` 选择 factories / none;在 `New<Name>(deps)` 写一次实现 | [配置与完整工厂示例](GENERATION.md) |
|
||||
| 路由、分组、请求上下文 | `omni.New()`、`app.GET` / `Group` / `Use`;业务使用 `context.Context` | [基本服务](examples/basic/main.go)、[已有项目接入](#已有项目按模块接入) |
|
||||
| HTTP 网络与扩展适配 | 配置 transport 为 nethttp / native / http2;自定义 Router/Runner | [网络层配置与契约](transport/README.md)、[性能与限制](PERFORMANCE.md) |
|
||||
| HTTP 老接口桥接 | 通过官方 Handler/Middleware 适配器接入标准 `net/http`,跟踪实际响应状态 | [接入说明](MIGRATION.md)、[桥接/上传/自定义 ID 示例](examples/compatibility/main.go) |
|
||||
| 请求校验、动态 JSON、自定义标量 | `.api` 声明来源/校验/scalar;手写契约用 `contract.New[T]`;简单绑定可用 `validate.JSON/Query/URI` | [语法与例子](API.md)、[请求策略](DEVELOPMENT.md) |
|
||||
| 数组、null、原始请求、上传下载 | `.api` 声明数组/可空返回、body/multipart 与大小限制;必要时使用 Bind/Manual 扩展 | [契约示例](examples/contracts)、[含上传/回调/下载的项目](examples/managedService/README.md) |
|
||||
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
|
||||
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
|
||||
| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
|
||||
| 结构化业务日志 | `logx.Info/Error`、`logx.From(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
|
||||
| 文件配置与环境覆盖 | 默认值→JSON/YAML→环境;完整业务结构用 `config.LoadProject[T]` / 环境别名 | [配置详解与例子](DEVELOPMENT.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
|
||||
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
|
||||
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
|
||||
| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
|
||||
| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
|
||||
| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
|
||||
| 出站 HTTP、熔断 | `httpc.New` 复用连接池与截止时间;`resilience.New` 隔离故障依赖 | [HTTP 用法](MODULES.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
|
||||
| CORS、日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
|
||||
| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
|
||||
| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
|
||||
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
|
||||
| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [观测用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
|
||||
| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
|
||||
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
|
||||
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
|
||||
|
||||
## 选择接入方式
|
||||
|
||||
@@ -30,7 +74,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
|
||||
|
||||
```sh
|
||||
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview
|
||||
export PATH="$(go env GOPATH)/bin:$PATH"
|
||||
go-omni --help
|
||||
```
|
||||
@@ -71,21 +115,22 @@ hello/
|
||||
├── api/service.api # 维护接口定义,也可 import 其他 .api
|
||||
├── main.go # 生成器维护:启动、路由注册、优雅退出
|
||||
├── etc/config.yaml # 项目配置
|
||||
├── omni.gen.json # 项目维护的生成选项与可选映射
|
||||
├── internal/
|
||||
│ ├── config/ # 配置读取
|
||||
│ ├── svc/ # 数据库、日志、外部客户端等依赖
|
||||
│ ├── types/*Gen.go # 按 .api 文件生成 DTO
|
||||
│ ├── handler/ # 生成的绑定、响应策略、路由及文档
|
||||
│ ├── logic/greet/ # 在这里实现业务逻辑
|
||||
│ ├── repository/greet/ # 在这里实现数据访问
|
||||
│ └── middleware/ # 声明鉴权等中间件后生成对应 hook
|
||||
│ ├── repository/ # 按需创建,默认不生成空骨架
|
||||
│ └── middleware/ # 声明鉴权等中间件后生成工厂
|
||||
├── docs/openapi.json # 生成的 OpenAPI 3.0.3
|
||||
└── .goOmni/apiManifest.json # 生成文件所有权与校验信息,一起提交 Git
|
||||
```
|
||||
|
||||
修改 `api/service.api` 后重新生成,业务写在 `internal/logic`,数据访问写在 `internal/repository`,依赖在 `internal/svc` 装配。声明业务错误后还会生成 `internal/apierrors/errorsGen.go`。
|
||||
|
||||
`api new` 的问候接口可以直接运行;自定义接口新生成的 Logic 和鉴权 hook 返回 **501**,需要实现后才可用。生成器不会根据 HTTP 字段猜数据库表或业务规则。
|
||||
`api new` 的问候接口可以直接运行;自定义接口未实现的 Logic 返回 **501**。新项目的具名中间件工厂未实现时直接阻止启动;v0.16 的原有 handler 模式仍返回 501。生成器不会根据 HTTP 字段猜数据库表或业务规则。
|
||||
|
||||
### 从自己的接口定义生成
|
||||
|
||||
@@ -186,7 +231,7 @@ go-omni new --api catalog.api \
|
||||
在业务项目的 **go.mod 所在目录**执行:
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.14.0-preview
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview
|
||||
```
|
||||
|
||||
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
|
||||
@@ -310,7 +355,7 @@ auth UserAuth (
|
||||
)
|
||||
```
|
||||
|
||||
生成器会创建 `internal/middleware/auth.go`,由项目实现凭据校验和身份注入。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的 hook 返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。
|
||||
新项目生成 `internal/middleware/authFactory.go`,由项目实现 `NewAuth(deps)`;生成器负责初始化和注册。v0.16 或 handler 模式继续使用 `internal/middleware/auth.go`。凭据校验和身份注入由项目实现。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的工厂阻止启动,原有 hook 模式返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。
|
||||
|
||||
业务错误可以集中定义,业务只选择发生了哪个错误。**HTTP 状态由项目配置,业务失败可以返回 HTTP 200**;框架不根据 HTTP 状态单独判断成功与否。
|
||||
|
||||
@@ -380,6 +425,38 @@ return apierrors.TextTooLong("企业名称", 100)
|
||||
|
||||
更详细的配置、作用域和升级步骤见 [统一错误目录](errors/README.md)。
|
||||
|
||||
### 可选结构转换与深拷贝
|
||||
|
||||
普通接口不需要任何拷贝配置,直接使用生成的 Request/Response。独立业务结构之间有转换需求时:
|
||||
|
||||
```go
|
||||
// import "git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
var input DomainInput
|
||||
if err := copyx.Copy(&input, req); err != nil { return err }
|
||||
|
||||
// 源与目标之间的指针、map、slice 需要隔离时:
|
||||
snapshot, err := copyx.Clone(source)
|
||||
if err != nil { return err }
|
||||
```
|
||||
|
||||
默认浅拷贝,同名同类型字段自动赋值,保留零值;默认不会把 nil 填成非可空目标的零值。不要为使用工具新增一套重复模型。
|
||||
|
||||
需要重复使用或自定义规则时创建 Copier:
|
||||
|
||||
```go
|
||||
copier, err := copyx.New(
|
||||
copyx.WithDeepCopy(),
|
||||
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
|
||||
copyx.WithConverter(func(v time.Time) (string, error) {
|
||||
return v.UTC().Format(time.RFC3339), nil
|
||||
}),
|
||||
)
|
||||
if err != nil { return err }
|
||||
if err := copier.Copy(&view, input); err != nil { return err }
|
||||
```
|
||||
|
||||
自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错;字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md),完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查,类型对需测试;深拷贝不适用于连接、锁等资源。
|
||||
|
||||
### 数据库、存储及其他模块
|
||||
|
||||
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
|
||||
@@ -454,9 +531,9 @@ go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
|
||||
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
|
||||
|
||||
CLI 与业务 module 固定使用 `v0.14.0-preview`。新布局不兼容旧 manifest;旧试验项目需在新目录生成并搬入业务文件,不能直接覆盖手写入口。新布局内部升级再执行上述生成与测试步骤,手写框架项目只需依赖升级及业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
||||
CLI 与业务 module 固定使用 `v0.19.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
||||
|
||||
## 按需深入
|
||||
|
||||
@@ -472,3 +549,7 @@ CLI 与业务 module 固定使用 `v0.14.0-preview`。新布局不兼容旧 mani
|
||||
| 网络性能、测试条件与适用范围 | [性能说明](PERFORMANCE.md)、[HTTP/2 验收](transport/native/HTTP2.md) |
|
||||
|
||||
当前版本可用于固定版本接入与业务验证,仍标记为 preview。`.api` 借鉴 go-zero 的接口开发方式,支持范围以本仓库文档为准,不是完整 goctl 语法兼容层;HTTP/2 保持实验状态,OSS/R2 云账号实网验收不属于本次发布范围。
|
||||
|
||||
## 后续开发约定
|
||||
|
||||
每次功能开发完成,同步更新这份功能索引、用法说明、`examples/` 和发布记录;删除或改变功能必须注明影响与迁移。代码、示例和文档一起回归后再发布。约定已记录在 [AGENTS.md](AGENTS.md),后续开发继续遵循。
|
||||
|
||||
145
RELEASE.md
145
RELEASE.md
@@ -1,4 +1,145 @@
|
||||
# v0.14.0-preview
|
||||
# v0.19.0-preview
|
||||
|
||||
将可选结构转换改为直接 Go 调用,减少业务项目需要维护的配置。普通接口继续使用 `.api` 生成 DTO,不要求增加独立模型或任何拷贝规则。
|
||||
|
||||
- 新增只依赖标准库的 `copyx`:Copy、Clone、可复用 Copier;浅/深拷贝、严格字段覆盖、nil/零值策略、字段改名、忽略字段、精确类型 converter。没有全局注册和隐式数字强转。
|
||||
- 深拷贝检查整个输入和输出图。自定义 converter 原样返回指针、map、slice,或新对象仍嵌套旧引用时返回错误;保留普通循环图,拒绝资源对象及超出遍历/估算分配预算的输入。
|
||||
- 转换失败不提交部分目标;保留错误路径和原始错误链。并发复用只共享不可变类型元数据;源/目标读写和 converter 副作用由调用方负责。
|
||||
- **不兼容变更:删除 mappings / mappingFiles、Go 类型加载与映射代码生成。** 检出旧配置/旧生成文件时给出迁移提示;不会静默删除业务正在使用的代码。公共字段组合、scalar、中间件工厂、Repository 选项保留。
|
||||
- README 补齐当前能力入口、解释和用法,新增 [完整 copyx 指南](COPY.md)、[可运行示例](examples/copying);[AGENTS.md](AGENTS.md) 记录每次修改同步文档/示例/移除说明的约定。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview
|
||||
# 如用过旧 mapping,先按 COPY.md 完成迁移:
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
迁移步骤和语义边界见 [copyx](COPY.md#从旧-mapping-迁移)。新工具做运行时检查,不能声称具备旧生成器的编译期字段覆盖保证;实际类型对必须有测试。独立三端工资示例验证金额精度、缺失/null/零值、空集合、错误链和字段新增;不修改或代替接入项目的完整业务回归。
|
||||
|
||||
46 个测试包的全量 race(含实际生成项目)、vet/build、Go 1.22 兼容测试通过;30 秒 fuzz 无失败,14 种基准场景各测量三轮。验收条件、性能数据及本轮未覆盖的外部环境见 [v0.19 验收记录](reports/copyxAcceptance.md)。本版仍为 preview。
|
||||
|
||||
---
|
||||
|
||||
以下是历史版本记录,其中 mapping 功能已在 v0.19 移除,不应继续作为当前接入方式。
|
||||
|
||||
# v0.18.0-preview(历史版本)
|
||||
|
||||
补齐可空嵌套映射和按模块拆分配置,减少重复转换代码,业务空值规则仍由项目显式选择:
|
||||
|
||||
- 嵌套 mapping 支持 `*S → *T` 保留 nil、`S → *T`,以及指定 nil zero/error 后的 `*S → T`;同样适用于 slice/等长数组元素。错误保留字段路径、索引和内部错误链,失败不返回部分结果。
|
||||
- `mappingFiles` 显式引用项目内的映射文件,支持跨文件向前引用;统一检查重名、循环、类型与字段覆盖。禁止多层导入、路径越界和隐式覆盖,生成期间核对全部配置输入。
|
||||
- 同类型 `*T → T` 可显式配置 nil zero/error;现有 converter 保持完整控制,不默认放宽类型或混入领域规则。
|
||||
- `pointerResult: true` 额外生成 `<Name>Ptr(S) (*T,error)`,原值函数保留;失败返回 nil 与原错误,辅助函数名冲突在生成前拒绝。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.18.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.18.0-preview
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
v0.17 的单文件 mappings、现有 converter 和业务文件继续保留。配置拆分时移动完整映射条目,主配置显式列出子文件;不保留重复定义。新能力均为可选生成配置,不增加运行时依赖。nil zero 表示 Go 零值并跳过子映射;业务必填、金额精度、编号格式和快照复制仍由项目负责。
|
||||
|
||||
使用与迁移见 [生成能力指南](GENERATION.md),回归和发布后验收见 [验收记录](reports/nullableMappingAcceptance.md)。本版仍为 preview。
|
||||
|
||||
全量 44 个测试包的 race、vet、build 和实际进程验收通过;空缓存远程 CLI 与 8 组实际生成消费者回归也通过,同版依赖且无本地 replace。标签固定在 `4563a6354fac19f22cd8d407fd76d98b8cc9c406`。
|
||||
|
||||
---
|
||||
|
||||
# v0.17.0-preview(历史版本)
|
||||
|
||||
减少项目里的重复声明、字段赋值和中间件装配代码,业务规则继续由项目控制:
|
||||
|
||||
- `.api` 公共字段组合:保持 JSON 平铺,DTO、绑定、校验和 OpenAPI 同步;字段冲突、循环和超限明确报错。
|
||||
- 可选结构映射:在 `omni.gen.json` 显式声明源与目标,生成编译期检查的普通 Go 赋值。不同字段名、金额、时间和指针转换由项目指定函数;遗漏字段与类型错误在写文件前失败。不使用 JSON 往返或运行时反射。
|
||||
- 带依赖的中间件工厂:svc 初始化依赖后自动调用 `New<Name>(deps)` 并注册;错误、nil 或 panic 阻止启动,已登记资源按生命周期清理。
|
||||
- Repository 骨架默认关闭,可按 group 启用;已有业务文件始终保留。生成选项与映射纳入重新生成和 `--check`。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
新项目默认使用 factories;已有项目首次升级保留 handlers 模式,需要显式迁移后再切换。映射按需开启,先准备业务类型、转换函数和依赖,再声明映射;普通引用赋值保留 Go 语义,缺失/null 语义由项目类型与转换函数定义。
|
||||
|
||||
使用方法、文件归属和升级步骤见 [生成能力指南](GENERATION.md),回归与发布验收见 [验收记录](reports/generationExperienceAcceptance.md)。本版仍为 preview;框架验收不等于业务项目完成迁移。
|
||||
|
||||
全量 44 个测试包的 race、vet、build 和实际进程验收通过;发布后的空缓存远程 CLI 与 7 组生成消费者回归也通过,同版依赖且无本地 replace。标签固定在 `8b588a24cd653dac77dd68f6777e0db367981f19`。
|
||||
|
||||
---
|
||||
|
||||
# v0.16.0-preview(历史版本)
|
||||
|
||||
补齐通用契约与生成体验,业务响应协议继续由项目定义:
|
||||
|
||||
- `returns([]string/[]int64/[]bool/[]any)` 及其他支持的基础类型数组,贯通 Logic、Handler、OpenAPI 和生成检查。nil 根数组输出 `[]`;DTO 数组、nullable、null、raw 和 204 保持原语义。字节数组明确拒绝 base64/数组歧义。
|
||||
- OpenAPI 以标准 components / `$ref` 共享重复响应、schema、示例;保留同一 HTTP 200 下的成功与失败、模板参数、数字业务码和 data:null。
|
||||
- workspace 显式配置 Go Catalog 提供方,一次导出共享 JSON,多服务通过受控根引用。支持服务/路由的文档错误列表,运行期 App 目录继承不变。check 核对导出漂移并保持项目文件不变。
|
||||
- EnvOptions.Aliases + ServerEnvAliases 统一完整配置与框架短名称的 Lookup、类型转换、空值规则。完整名称优先,显式空值不等于未设置;旧 ApplyEnv 的空值/逗号列表行为保留。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.16.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.16.0-preview
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
旧项目 config/Logic/svc 继续由开发者维护,不会被重新生成覆盖。已有环境配置需按 [一次覆盖完整配置](DEVELOPMENT.md#一次覆盖完整配置与-http日志短名称) 合并 OverlayEnv 与 ApplyEnv;共享错误目录接入见 [workspace 配置](DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。已有手写权限 Handler 切换自动生成前,先迁出实现并显式移除 Manual 文件,生成器不会静默删除。
|
||||
|
||||
全量 race(含实际生成消费者)、vet、build、根入口/退出/再生成验证通过。eTax 只读契约验收:API 文档 20,751,149 → 847,171 字节,ID 文档 1,008,210 → 392,481 字节;70 个操作展开引用后与原文档完全一致。空缓存远程 CLI / 固定版本消费者验收也已通过,没有本地 replace;标签固定在 `64e3bd6cfced225476d58e95037249cd0a4fe695`。详见 [验收记录](reports/contractFollowupAcceptance.md)。这不代表已完成业务项目迁移。
|
||||
|
||||
---
|
||||
|
||||
# v0.15.0-preview(历史版本)
|
||||
|
||||
增加通用后台分阶段关闭:生成入口先完成 HTTP 排空,再运行全部 OnStop、全部 OnDrain,确认组件收尾后逆序释放 OnClose 资源,最后关闭日志。适用于任意后台组件;不引入 Orbit,不改变业务事务或任务一致性策略。
|
||||
|
||||
- svc 登记成对、同名的 OnStop/OnDrain,Context 用于早期退出通知,WorkContext 供已接受工作在收尾期间使用。同一阶段并发执行,存在内部先后依赖的组件组合登记。
|
||||
- server.lifecycle 配置 stopTimeout、drainTimeout、cancelTimeout。阶段超时取消工作并给未返回钩子收尾窗口;错误与阶段/组件名称可通过错误链追踪。
|
||||
- 不能确认后台已经结束时,返回 ErrResourcesRetained,保留共享资源和借用日志,生成进程非零退出。未返回的 Stop 不会与自己的 Drain 重叠;错误、panic 不阻止其他组件尝试收尾。
|
||||
- task.Runner 增加 StopAccepting/Wait,原 Shutdown/Close 继续可用。核心 lifecycle 不依赖 task 或其他适配器。
|
||||
- 初始化和监听失败复用同一流程;重复关闭只执行一次。生成器继续保留配置、svc 和业务代码。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.15.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.15.0-preview
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
v0.14.x 无需重写 svc/config;生成器更新 main/runtimeGen,未配置时使用默认时限。也可 api upgrade --dry-run/--apply 更新依赖与生成文件,保留已有项目扩展。需要阶段保证的旧后台 OnClose 登记,应显式迁到 Stop/Drain;v0.13 及更早版本仍按原迁移指南处理。
|
||||
|
||||
全量 race(含实际生成项目)、vet、build 和隔离 MySQL/GORM 验收已通过。发布后的空缓存远程生成项目与真实 MySQL/GORM 验收也通过,CLI/框架同版且无本地 replace;标签固定在 `16deead70e860052c5b629511207578c03f9b2cb`。详见 [生命周期接入](lifecycle/README.md#后台组件分阶段关闭) 与 [验收记录](reports/phasedLifecycleAcceptance.md)。Go 无法强制结束任意 goroutine;OnClose 仍须能返回,HTTP 处理器仍须响应取消。此版不会将未知的第三方组件自动改造成优雅关闭组件。
|
||||
|
||||
---
|
||||
|
||||
# v0.14.1-preview(历史版本)
|
||||
|
||||
增加可选 `database/gormx`:GORM 复用 go-omni 已初始化的标准池、TLS、限额、健康检查和关闭流程,业务继续使用原生 GORM 模型与事务。核心 App/数据库包的编译依赖链不引入 GORM。
|
||||
|
||||
在 svc.Init 中调用 `gormx.MySQL(ctx, c.DB, gormx.WithLogger(c.Logger))`;通用 Open 工厂保留其他方言入口。适配器保护借用池不被初始化失败路径关闭,并为初始化查询设置时限。查询使用 WithContext,SQL 日志接入 logx,请求标识、慢查询和错误分类贯通;默认不输出绑定参数与驱动完整错误文本。
|
||||
|
||||
v0.14.0 项目更新 module 与 CLI 即可,不改生成目录或 main。模板最低运行时保持 v0.14.0;修正新版 CLI 把最低兼容版本误当自身版本、导致同版消费者出现版本提示的问题。其他数据库方言和管理额外池的插件未纳入本次实库验收,原生 PrepareStmt 的生命周期限制见 [接入说明](database/gormx/README.md)。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
|
||||
```
|
||||
|
||||
全量 race(含生成消费者)、vet、build,以及隔离 MySQL 的事务/取消/日志/生成项目接入通过。发布后的空缓存远程 MySQL/GORM 和生成项目验收均通过,无本地 replace,CLI 与框架使用同版。标签固定在 `b8ab6b35219d53d6de392c920fe7ecbe9a76dade`,详见 [GORM 验收](reports/gormAcceptance.md)。
|
||||
|
||||
---
|
||||
|
||||
# v0.14.0-preview(历史版本)
|
||||
|
||||
本版补齐契约驱动的业务开发与通用基础设施。项目继续维护接口、配置、依赖装配、Logic、SQL 和业务规则;生成器维护入口、请求绑定、路由、响应与文档。
|
||||
|
||||
@@ -24,6 +165,8 @@ go-omni new --module example.com/billing ./billing
|
||||
|
||||
详见 [业务开发指南](DEVELOPMENT.md)、[数据库接入](database/mysql/README.md) 和 [验收记录](reports/businessDevelopmentAcceptance.md)。
|
||||
|
||||
全量 race(含生成项目)、vet、build 与本地进程验收通过。发布后的空缓存远程消费者和隔离 MySQL 验收也均通过;CLI/框架同版且无本地 replace。标签固定在 `f5478cdc7662704b164637649305361e52f7c598`,报告单独提交。
|
||||
|
||||
仍需业务维护:契约、配置、svc 依赖创建/探针/清理登记、错误目录、Logic、Repository 和权限中间件。特殊协议可使用保留的扩展点。声明式上传的暂存磁盘和并发量需配置容量;共享契约版本标签由项目管理,框架校验摘要但不下载依赖。此版仍为 preview,不代表 eTax/ePay 已完成业务迁移。
|
||||
|
||||
---
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
# GoOmni 成熟度计划
|
||||
|
||||
当前发布基线为 v0.19.0-preview:普通接口通过 `.api` → DTO → Logic 开发;独立类型需要转换时可选 copyx。旧 JSON mapping 生成已移除,见 [迁移说明](COPY.md)。当前能力、使用入口和限制以 [README](README.md) 与 [发布记录](RELEASE.md) 为准。下文保留阶段性建设记录。
|
||||
|
||||
|
||||
当前进展:`v0.14.0-preview` 已完成契约驱动的业务开发能力、通用 Runtime/配置和官方 MySQL 适配器,并通过全量 race、实际生成服务及空缓存远程验收。新项目/升级以 [README](README.md) 和 [开发指南](DEVELOPMENT.md) 为准;下方为历史阶段记录,完整本轮验收见 [报告](reports/businessDevelopmentAcceptance.md)。具体 eTax/ePay 业务迁移与业务回归仍由接入项目完成。
|
||||
|
||||
## 1. 新项目启动与发布流程(已完成,v0.1.1)
|
||||
|
||||
- 修正脚手架固定版本引用,保留显式本地开发选项。
|
||||
|
||||
@@ -1,5 +1,11 @@
|
||||
# go-omni CLI
|
||||
|
||||
v0.19 移除结构映射生成:`omni.gen.json` 只保留 Repository 和中间件选项。普通接口直接使用生成 DTO;需要类型转换时用普通 Go 或可选 [copyx](../../COPY.md)。旧 `mappings` / `mappingFiles` 必须按迁移步骤清理,CLI 不会静默删除仍在调用的旧生成文件。
|
||||
|
||||
公共字段组合、中间件工厂和按需 Repository 继续支持,见 [生成能力指南](../../GENERATION.md)。
|
||||
|
||||
v0.16 增加基础类型/动态 JSON 数组返回、OpenAPI 公共引用和 workspace Go Catalog 统一导出。见 [接入与筛选规则](../../DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。
|
||||
|
||||
v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目录>|--apply]`,以及 `workspace format|generate|check --file omni.workspace.json`。详见 [通用开发指南](../../DEVELOPMENT.md)。
|
||||
|
||||
新项目采用根目录 `main.go` 与 `internal/config、svc、logic、handler、types`。完整职责、启动扩展、流式接口和生成文件边界见 [项目指南](../../PROJECT.md)。
|
||||
@@ -8,14 +14,14 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目
|
||||
|
||||
```sh
|
||||
# 仓库读取权限与 PATH 配置见根目录 README
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview
|
||||
go-omni new --module example.com/hello ./hello
|
||||
|
||||
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
|
||||
go-omni new --api service.api --module example.com/example ./example
|
||||
```
|
||||
|
||||
`new` 与 `api new` 统一生成契约项目;不再提供另一套 CRUD 启动目录。未实现的 Logic / 中间件骨架返回 501。进入项目执行 `go mod tidy`、`go run .`、`go build .`。
|
||||
`new` 与 `api new` 统一生成契约项目;不再提供另一套 CRUD 启动目录。未实现的 Logic 返回 501;中间件工厂未实现时阻止启动,既有 handler 模式仍返回 501。进入项目执行 `go mod tidy`、`go run .`、`go build .`。
|
||||
|
||||
参数:`--database none|memory|mysql`(默认 none;memory 等同 none),`--storage none|local|s3|r2|oss`,`--log-output stdout|stderr|file`,`--log-format json|text`。基础设施按需生成,连接凭据由环境提供,不自动创建数据库、表或云资源。选择 MySQL 不会凭空生成业务表或 Repository 查询。
|
||||
|
||||
@@ -43,14 +49,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
|
||||
|
||||
## 版本与迁移
|
||||
|
||||
CLI 与框架依赖固定使用 v0.14.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
|
||||
CLI 与框架依赖固定使用 v0.19.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
|
||||
|
||||
```sh
|
||||
# 在 go-omni 源码仓库中执行
|
||||
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
|
||||
```
|
||||
|
||||
新模板需要 v0.14.0-preview 的通用 Runtime、配置和绑定能力,继续使用 manifest v2。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
|
||||
当前模板与依赖以 v0.19.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
|
||||
|
||||
## 数据库迁移命令
|
||||
|
||||
|
||||
@@ -170,6 +170,7 @@ func TestRegenerationAndConflicts(t *testing.T) {
|
||||
servicePath := filepath.Join(dir, "internal/logic/greet/greet.go")
|
||||
service := readFile(t, servicePath) + "\n// user edit must survive\n"
|
||||
writeFile(t, servicePath, service)
|
||||
writeFile(t, filepath.Join(dir, "internal/repository/greet/repository.go"), "package greet\n")
|
||||
for _, file := range []string{"etc/config.yaml", "internal/svc/serviceContext.go", "internal/repository/greet/repository.go"} {
|
||||
body := readFile(t, filepath.Join(dir, file))
|
||||
if strings.HasSuffix(file, ".go") {
|
||||
@@ -301,6 +302,7 @@ func TestGeneratedProjectsBuildAndRun(t *testing.T) {
|
||||
if starter {
|
||||
_, err = New(o)
|
||||
} else {
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
|
||||
o.API = filepath.Join(root, "examples/contracts/user.api")
|
||||
_, err = Generate(o)
|
||||
}
|
||||
|
||||
104
cmd/go-omni/apigen/arrayResponse_test.go
Normal file
104
cmd/go-omni/apigen/arrayResponse_test.go
Normal file
@@ -0,0 +1,104 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestPrimitiveArrayContracts(t *testing.T) {
|
||||
for _, name := range []string{"string", "bool", "int", "int8", "int16", "int32", "int64", "uint", "uint16", "uint32", "uint64", "float32", "float64", "any"} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
spec, err := parseText(t, fmt.Sprintf("service arrays{\n@handler list\nget /list returns ([]%s)\n}", name))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
schema := operation(t, document(t, c), "/list", "get")["responses"].(map[string]any)["200"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
|
||||
data := schema["properties"].(map[string]any)["data"].(map[string]any)
|
||||
if data["type"] != "array" || data["items"] == nil {
|
||||
t.Fatal(data)
|
||||
}
|
||||
if name == "any" && len(data["items"].(map[string]any)) != 0 {
|
||||
t.Fatal("dynamic JSON was constrained")
|
||||
}
|
||||
if responseGoType("[]"+name) != "[]"+name || responseNeedsTypes("[]"+name) {
|
||||
t.Fatal("wrong Go signature/import")
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, name := range []string{"uint8", "File", "Unknown", "[]string"} {
|
||||
spec, err := parseText(t, fmt.Sprintf("service arrays{\n@handler list\nget /list returns ([]%s)\n}", name))
|
||||
if err == nil {
|
||||
_, err = Compile(spec)
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("unsupported root accepted", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrimitiveArrayConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "arrays")
|
||||
o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/arrays.api"), Module: "example.com/arrays", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, name := range []string{"strings", "integers", "dynamic", "empty", "raw"} {
|
||||
writeFile(t, filepath.Join(dir, "internal/logic/arrays/"+name+".go"), "package arrays\n")
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "internal/logic/arrays/values.go"), readFile(t, "testdata/arrays/values.go"))
|
||||
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/arrays/consumer_test.go"))
|
||||
o.API = filepath.Join(dir, "api/arrays.api")
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(readFile(t, filepath.Join(dir, "internal/logic/arrays/values.go")), "企业") {
|
||||
t.Fatal("logic overwritten")
|
||||
}
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-race", "-mod=mod", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("%v\n%s", err, out)
|
||||
}
|
||||
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
|
||||
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
raw, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatal(err, string(raw))
|
||||
}
|
||||
var module struct {
|
||||
Version, Sum string
|
||||
Replace any
|
||||
}
|
||||
if err = json.Unmarshal(raw, &module); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
|
||||
t.Fatal("not a fixed remote consumer")
|
||||
}
|
||||
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
|
||||
}
|
||||
|
||||
}
|
||||
@@ -40,6 +40,7 @@ func TestBusinessGeneratedProject(t *testing.T) {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "business")
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
|
||||
o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/business.api"), Module: "example.com/business", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
|
||||
@@ -2,41 +2,73 @@ package apigen
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const catalogDocumentFile = "omni.errors.json"
|
||||
|
||||
func readDocumentCatalog(root string, spec *Spec) error {
|
||||
path := filepath.Join(root, catalogDocumentFile)
|
||||
stat, err := os.Lstat(path)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil
|
||||
// DocumentErrors selects documentation only. Runtime App inheritance and
|
||||
// route-local @errors declarations are never changed by this metadata.
|
||||
// A nil Include lists all errors; [] lists none. Routes override Include.
|
||||
type DocumentErrors struct {
|
||||
Source string `json:"source"`
|
||||
Include []string `json:"include"`
|
||||
Routes map[string][]string `json:"routes,omitempty"`
|
||||
}
|
||||
|
||||
func readDocumentCatalog(root string, spec *Spec, overrides map[string][]byte) error {
|
||||
name := catalogDocumentFile
|
||||
path := filepath.Join(root, name)
|
||||
// Shared metadata uses the same explicitly authorized roots as .api imports.
|
||||
|
||||
if selection := spec.DocumentErrors; selection != nil {
|
||||
if _, err := os.Lstat(path); !errors.Is(err, os.ErrNotExist) {
|
||||
return errors.New("api: use either local omni.errors.json or shared errors.source, not both")
|
||||
}
|
||||
name = selection.Source
|
||||
alias, sub, ok := strings.Cut(strings.TrimPrefix(name, "@"), "/")
|
||||
declared, found := spec.SharedRoots[alias]
|
||||
if !strings.HasPrefix(name, "@") || !ok || !found || sub == "" || filepath.Ext(sub) != ".json" {
|
||||
return errors.New("api: errors.source must name a JSON file in a declared @root")
|
||||
}
|
||||
if err := safePath(declared.Path, sub); err != nil {
|
||||
return err
|
||||
}
|
||||
path = filepath.Join(declared.Path, filepath.FromSlash(sub))
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !stat.Mode().IsRegular() {
|
||||
return errors.New("api: error metadata must be a regular file")
|
||||
}
|
||||
file, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer file.Close()
|
||||
raw, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
|
||||
if err != nil {
|
||||
return err
|
||||
raw, ok := overrides[path]
|
||||
if !ok {
|
||||
stat, err := os.Lstat(path)
|
||||
if errors.Is(err, os.ErrNotExist) && spec.DocumentErrors == nil {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !stat.Mode().IsRegular() {
|
||||
return errors.New("api: error metadata must be a regular file")
|
||||
}
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer f.Close()
|
||||
raw, err = io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if len(raw) > 1<<20 {
|
||||
return errors.New("api: error metadata exceeds 1 MiB")
|
||||
}
|
||||
var catalog apierrors.Catalog
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &catalog); err != nil {
|
||||
if err := apierrors.DecodeCatalogJSON(raw, &catalog); err != nil {
|
||||
return err
|
||||
}
|
||||
registry, err := apierrors.NewRegistry(catalog)
|
||||
@@ -45,10 +77,70 @@ func readDocumentCatalog(root string, spec *Spec) error {
|
||||
}
|
||||
normalized := registry.Catalog()
|
||||
spec.DocumentCatalog = &normalized
|
||||
spec.Sources[catalogDocumentFile] = raw
|
||||
spec.SourcePaths[catalogDocumentFile] = path
|
||||
spec.Sources[name] = raw
|
||||
spec.SourcePaths[name] = path
|
||||
return validateDocumentSelection(spec)
|
||||
}
|
||||
|
||||
func documentRouteCatalog(spec *Spec, route Route) apierrors.Catalog {
|
||||
catalog := spec.DocumentCatalog.Clone()
|
||||
if spec.DocumentErrors == nil {
|
||||
return catalog
|
||||
}
|
||||
include := spec.DocumentErrors.Include
|
||||
if selected, ok := spec.DocumentErrors.Routes[strings.ToUpper(route.Method)+" "+route.Path]; ok {
|
||||
include = selected
|
||||
}
|
||||
if include != nil {
|
||||
selected := map[string]apierrors.Definition{}
|
||||
for _, key := range include {
|
||||
selected[key] = catalog.Definitions[key]
|
||||
}
|
||||
catalog.Definitions = selected
|
||||
}
|
||||
return catalog
|
||||
}
|
||||
|
||||
func validateDocumentSelection(spec *Spec) error {
|
||||
cfg := spec.DocumentErrors
|
||||
if cfg == nil {
|
||||
return nil
|
||||
}
|
||||
validate := func(keys []string) error {
|
||||
seen := map[string]bool{}
|
||||
for _, key := range keys {
|
||||
if _, ok := spec.DocumentCatalog.Definitions[key]; !ok {
|
||||
return fmt.Errorf("api: unknown documentation error %q", key)
|
||||
}
|
||||
if seen[key] {
|
||||
return fmt.Errorf("api: duplicate documentation error %q", key)
|
||||
}
|
||||
seen[key] = true
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := validate(cfg.Include); err != nil {
|
||||
return err
|
||||
}
|
||||
routes := map[string]Route{}
|
||||
for _, r := range spec.Routes {
|
||||
routes[strings.ToUpper(r.Method)+" "+r.Path] = r
|
||||
}
|
||||
for key, keys := range cfg.Routes {
|
||||
r, ok := routes[key]
|
||||
if !ok {
|
||||
return fmt.Errorf("api: unknown documentation route %q", key)
|
||||
}
|
||||
if len(r.Errors) > 0 {
|
||||
return fmt.Errorf("api: %s already declares route-local errors", key)
|
||||
}
|
||||
if err := validate(keys); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateDocumentCatalog(spec *Spec) error {
|
||||
if spec.DocumentCatalog == nil {
|
||||
return nil
|
||||
@@ -61,7 +153,7 @@ func validateDocumentCatalog(spec *Spec) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err = policy.ValidateErrors(*spec.DocumentCatalog); err != nil {
|
||||
if err = policy.ValidateErrors(documentRouteCatalog(spec, r)); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
@@ -40,6 +40,9 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
if prepareErr != nil {
|
||||
return nil, prepareErr
|
||||
}
|
||||
if err := expandFields(spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := validateDocumentCatalog(spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -70,7 +73,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
resolving := map[string]bool{}
|
||||
var resolve func(string, int) (reflect.Type, error)
|
||||
var expression func(ast.Expr, int) (reflect.Type, error)
|
||||
primitives := map[string]reflect.Type{"File": reflect.TypeOf(contract.File{}), "any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||
|
||||
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
|
||||
if depth > 32 {
|
||||
return nil, errors.New("type nesting exceeds 32")
|
||||
@@ -80,7 +83,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
if t, ok := scalarTypes[e.Name]; ok {
|
||||
return t, nil
|
||||
}
|
||||
if t, ok := primitives[e.Name]; ok {
|
||||
if t, ok := primitiveTypes[e.Name]; ok {
|
||||
return t, nil
|
||||
}
|
||||
return resolve(e.Name, depth+1)
|
||||
@@ -313,6 +316,7 @@ func Compile(spec *Spec) (*Compiled, error) {
|
||||
return nil, err
|
||||
}
|
||||
decorateDocument(doc, spec, c.Bindings)
|
||||
compactDocument(doc)
|
||||
c.Document, err = json.MarshalIndent(doc, "", " ")
|
||||
return c, err
|
||||
}
|
||||
|
||||
102
cmd/go-omni/apigen/composition.go
Normal file
102
cmd/go-omni/apigen/composition.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// expandFields copies definitions rather than embedding Go structs. Binding,
|
||||
// validation, DTOs, snapshots and OpenAPI all consume the same flattened fields.
|
||||
func expandFields(spec *Spec) error {
|
||||
definitions := map[string]Type{}
|
||||
for _, t := range spec.Types {
|
||||
if _, exists := definitions[t.Name]; exists {
|
||||
return fmt.Errorf("%s: duplicate type %s", t.Position, t.Name)
|
||||
}
|
||||
definitions[t.Name] = t
|
||||
}
|
||||
expanded := map[string]Type{}
|
||||
heights := map[string]int{}
|
||||
visiting := map[string]bool{}
|
||||
total := 0
|
||||
var expand func(string, int) (Type, error)
|
||||
expand = func(name string, depth int) (Type, error) {
|
||||
if t, ok := expanded[name]; ok {
|
||||
if depth+heights[name] > 32 {
|
||||
return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name)
|
||||
}
|
||||
return t, nil
|
||||
}
|
||||
if visiting[name] {
|
||||
return Type{}, fmt.Errorf("field composition cycle at %s", name)
|
||||
}
|
||||
if depth > 32 {
|
||||
return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name)
|
||||
}
|
||||
t, ok := definitions[name]
|
||||
if !ok {
|
||||
return Type{}, fmt.Errorf("unknown composed object type %s", name)
|
||||
}
|
||||
visiting[name] = true
|
||||
defer delete(visiting, name)
|
||||
fields := []Field{}
|
||||
height := 0
|
||||
seen, wire := map[string]bool{}, map[string]bool{}
|
||||
for _, f := range t.Fields {
|
||||
add := []Field{f}
|
||||
if f.Type == "" {
|
||||
child, err := expand(f.Name, depth+1)
|
||||
if err != nil {
|
||||
return Type{}, fmt.Errorf("%s.%s: %w", name, f.Name, err)
|
||||
}
|
||||
add = child.Fields
|
||||
if heights[f.Name]+1 > height {
|
||||
height = heights[f.Name] + 1
|
||||
}
|
||||
if height > 32 {
|
||||
return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name)
|
||||
}
|
||||
}
|
||||
for _, field := range add {
|
||||
if seen[field.Name] {
|
||||
return Type{}, fmt.Errorf("%s: conflicting composed field %s", name, field.Name)
|
||||
}
|
||||
seen[field.Name] = true
|
||||
for _, source := range []string{"json", "form", "path", "header", "multipart", "multipartJSON", "file"} {
|
||||
tag := strings.Split(reflect.StructTag(field.Tag).Get(source), ",")[0]
|
||||
if tag == "" || tag == "-" {
|
||||
continue
|
||||
}
|
||||
if source == "header" {
|
||||
tag = strings.ToLower(tag)
|
||||
}
|
||||
key := source + ":" + tag
|
||||
if wire[key] {
|
||||
return Type{}, fmt.Errorf("%s: conflicting composed field tag %s", name, key)
|
||||
}
|
||||
wire[key] = true
|
||||
}
|
||||
fields = append(fields, field)
|
||||
total++
|
||||
if len(fields) > 4096 || total > 65536 {
|
||||
return Type{}, fmt.Errorf("field composition expansion limit exceeded")
|
||||
}
|
||||
}
|
||||
}
|
||||
t.Fields = fields
|
||||
expanded[name] = t
|
||||
heights[name] = height
|
||||
return t, nil
|
||||
}
|
||||
result := make([]Type, 0, len(spec.Types))
|
||||
for _, t := range spec.Types {
|
||||
out, err := expand(t.Name, 0)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result = append(result, out)
|
||||
}
|
||||
spec.Types = result
|
||||
return nil
|
||||
}
|
||||
163
cmd/go-omni/apigen/composition_test.go
Normal file
163
cmd/go-omni/apigen/composition_test.go
Normal file
@@ -0,0 +1,163 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestFieldComposition(t *testing.T) {
|
||||
source := readFile(t, "testdata/composition/service.api")
|
||||
spec, err := parseText(t, source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
first, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
second, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !bytes.Equal(first.Document, second.Document) || len(spec.Types[1].Fields) != 1 {
|
||||
t.Fatal("composition mutates input")
|
||||
}
|
||||
for _, name := range []string{"EnterpriseRequest", "TenantRequest", "AdminRequest"} {
|
||||
schema := first.Bindings[name].BodySchema()
|
||||
raw, _ := json.Marshal(schema)
|
||||
if !bytes.Contains(raw, []byte("Employee display name.")) || bytes.Contains(raw, []byte("SalaryFields")) {
|
||||
t.Fatalf("flattened schema/comment missing: %s", raw)
|
||||
}
|
||||
}
|
||||
cases := []struct{ name, decl, want string }{
|
||||
{"unknown", "type Req {\n Missing\n}", "unknown composed"},
|
||||
{"cycle", "type Req {\n Other\n}\ntype Other {\n Req\n}", "cycle"},
|
||||
{"conflict", "type Base {\n Name string `json:\"name\"`\n}\ntype Req {\n Base\n Name string `json:\"other\"`\n}", "conflicting"},
|
||||
{"wireConflict", "type Base {\n Name string `json:\"name\"`\n}\ntype Req {\n Base\n Other string `json:\"name\"`\n}", "conflicting"},
|
||||
{"tagged", "type Req {\n Other `json:\"other\"`\n}", "cannot carry"},
|
||||
{"diamond", "type Base {\n Name string `json:\"name\"`\n}\ntype Left {\n Base\n}\ntype Right {\n Base\n}\ntype Req {\n Left\n Right\n}", "conflicting"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
s, e := parseText(t, tc.decl+"\nservice demo{\n@handler post\npost / (Req)\n}")
|
||||
if e == nil {
|
||||
_, e = Compile(s)
|
||||
}
|
||||
if e == nil || !strings.Contains(e.Error(), tc.want) {
|
||||
t.Fatal(e)
|
||||
}
|
||||
})
|
||||
}
|
||||
// Imports can contain reusable definitions and retain the existing root policy.
|
||||
dir := t.TempDir()
|
||||
writeFile(t, filepath.Join(dir, "base.api"), "type Base {\n Name string `json:\"name\"`\n}\n")
|
||||
writeFile(t, filepath.Join(dir, "service.api"), "import \"base.api\"\ntype Req {\n Base\n}\nservice demo{\n@handler post\npost / (Req)\n}")
|
||||
if _, err := FormatProject(filepath.Join(dir, "service.api"), false); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := FormatProject(filepath.Join(dir, "service.api"), true); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRepositoryGenerationOptions(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/repository", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/repository")); !os.IsNotExist(err) {
|
||||
t.Fatal("default creates repository")
|
||||
}
|
||||
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"repository":"group","middleware":"factories"}`)
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
repo := filepath.Join(dir, "internal/repository/greet/repository.go")
|
||||
original := readFile(t, repo) + "\n// business SQL belongs here\n"
|
||||
writeFile(t, repo, original)
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"repository":"none","middleware":"factories"}`)
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if readFile(t, repo) != original {
|
||||
t.Fatal("repository removed or overwritten")
|
||||
}
|
||||
for _, raw := range []string{`{"repository":"oops"}`, `{"middleware":"oops"}`, `{"unknown":true}`, `{"repository":"none","repository":"group"}`} {
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), raw)
|
||||
before := snapshot(t, dir)
|
||||
if _, err := Generate(o); err == nil {
|
||||
t.Fatal("accepted", raw)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("invalid config wrote files")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestCompositionExpansionBound(t *testing.T) {
|
||||
var source strings.Builder
|
||||
source.WriteString("type Base {\n Value string `json:\"value\"`\n}\n")
|
||||
last := "Base"
|
||||
for i := 0; i < 35; i++ {
|
||||
name := fmt.Sprintf("Level%d", i)
|
||||
fmt.Fprintf(&source, "type %s {\n %s\n}\n", name, last)
|
||||
last = name
|
||||
}
|
||||
// Reverse order forces traversal instead of hitting pre-expanded definitions.
|
||||
spec, err := parseText(t, source.String()+"service demo{\n@handler post\npost / ("+last+")\n}")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for i, j := 0, len(spec.Types)-1; i < j; i, j = i+1, j-1 {
|
||||
spec.Types[i], spec.Types[j] = spec.Types[j], spec.Types[i]
|
||||
}
|
||||
if _, err = Compile(spec); err == nil {
|
||||
t.Fatal("unbounded composition accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerationConfigMissingAndLegacy(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/config", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||
if err := os.Remove(filepath.Join(dir, generatorConfigFile)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := snapshot(t, dir)
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "restore") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("missing config changed project")
|
||||
}
|
||||
// A genuine old manifest has never tracked a generator config.
|
||||
var old manifest
|
||||
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &old); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
old.GeneratorConfig = false
|
||||
raw, _ := json.Marshal(old)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, exists, err := readGeneratorConfig(dir)
|
||||
if err != nil || !exists || cfg.Middleware != "handlers" || cfg.Repository != "none" {
|
||||
t.Fatal(cfg, exists, err)
|
||||
}
|
||||
if !strings.Contains(readFile(t, filepath.Join(dir, "internal/svc/runtimeGen.go")), "middleware.Defaults()") {
|
||||
t.Fatal("legacy registration replaced")
|
||||
}
|
||||
}
|
||||
@@ -213,7 +213,7 @@ func decorateDocument(doc map[string]any, spec *Spec, bindings map[string]*contr
|
||||
if len(r.Errors) == 0 && documentRegistry != nil {
|
||||
registry = documentRegistry
|
||||
op["x-omni-error-catalog-scope"] = "app"
|
||||
for key, definition := range registry.Catalog().Definitions {
|
||||
for key, definition := range documentRouteCatalog(spec, r).Definitions {
|
||||
grouped[definition.Status] = append(grouped[definition.Status], ErrorDefinition{Name: key, Code: key, Status: definition.Status, Message: definition.Message, BusinessCode: definition.Code, Params: definition.Params})
|
||||
}
|
||||
}
|
||||
|
||||
95
cmd/go-omni/apigen/copyConsumer_test.go
Normal file
95
cmd/go-omni/apigen/copyConsumer_test.go
Normal file
@@ -0,0 +1,95 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func runConsumerGo(t *testing.T, dir string, args ...string) {
|
||||
t.Helper()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", args...)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("go %v: %v\n%s", args, err, out)
|
||||
}
|
||||
}
|
||||
func TestCopyxThreeEndpointConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for path, fixture := range map[string]string{
|
||||
"internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go", "internal/convert/copy.go": "copy.go",
|
||||
"internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go",
|
||||
} {
|
||||
writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture))
|
||||
}
|
||||
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1)
|
||||
source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1)
|
||||
writeFile(t, svcFile, source)
|
||||
configFile := filepath.Join(dir, "internal/config/config.go")
|
||||
writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1))
|
||||
for _, group := range []string{"Enterprise", "Tenant", "Admin"} {
|
||||
text := fmt.Sprintf(`package %s
|
||||
import("context";"example.com/composition/internal/types";"example.com/composition/internal/convert";"example.com/composition/internal/domain")
|
||||
func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){
|
||||
input,err:=convert.Employee(*req);if err!=nil{return nil,err}
|
||||
result,err:=convert.Response(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil
|
||||
}
|
||||
`, strings.ToLower(group), group)
|
||||
writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text)
|
||||
}
|
||||
runConsumerGo(t, dir, "mod", "tidy")
|
||||
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
before := snapshot(t, dir)
|
||||
o.DryRun = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.DryRun = false
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("generation changed business files")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/mapping")); !os.IsNotExist(err) {
|
||||
t.Fatal("mapping directory generated")
|
||||
}
|
||||
// Adding a common field requires one contract edit and one independent
|
||||
// business field. Three endpoint converters need no new rules.
|
||||
contract := readFile(t, o.API)
|
||||
contract = strings.Replace(contract, "type SalaryFields {", "type SalaryFields {\n Note string `json:\"note,optional\"`", 1)
|
||||
writeFile(t, o.API, contract)
|
||||
domainPath := filepath.Join(dir, "internal/domain/employee.go")
|
||||
writeFile(t, domainPath, strings.Replace(readFile(t, domainPath), "type Employee struct {", "type Employee struct {\n Note string", 1))
|
||||
o.Check = false
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
|
||||
runConsumerGo(t, dir, "build", ".")
|
||||
}
|
||||
86
cmd/go-omni/apigen/documentRefs.go
Normal file
86
cmd/go-omni/apigen/documentRefs.go
Normal file
@@ -0,0 +1,86 @@
|
||||
package apigen
|
||||
|
||||
import "encoding/json"
|
||||
|
||||
// compactDocument only replaces standard OpenAPI Reference Object positions.
|
||||
// Payloads (including examples containing a literal "$ref") are never walked.
|
||||
// Mixed HTTP 200 responses retain both success and error alternatives.
|
||||
func compactDocument(doc map[string]any) {
|
||||
type slot struct {
|
||||
parent map[string]any
|
||||
key string
|
||||
}
|
||||
components := doc["components"].(map[string]any)
|
||||
var responses, schemas, examples []slot
|
||||
for _, path := range doc["paths"].(map[string]any) {
|
||||
for _, value := range path.(map[string]any) {
|
||||
op, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
rs, _ := op["responses"].(map[string]any)
|
||||
for status := range rs {
|
||||
responses = append(responses, slot{rs, status})
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
factor := func(kind string, slots []slot) {
|
||||
counts := map[string]int{}
|
||||
for _, s := range slots {
|
||||
raw, _ := json.Marshal(s.parent[s.key])
|
||||
counts[string(raw)]++
|
||||
}
|
||||
shared, _ := components[kind].(map[string]any)
|
||||
if shared == nil {
|
||||
shared = map[string]any{}
|
||||
}
|
||||
for _, s := range slots {
|
||||
raw, _ := json.Marshal(s.parent[s.key])
|
||||
count := counts[string(raw)]
|
||||
if count < 2 {
|
||||
continue
|
||||
}
|
||||
name := "Omni_" + digest(raw)
|
||||
ref := map[string]any{"$ref": "#/components/" + kind + "/" + name}
|
||||
refJSON, _ := json.Marshal(ref)
|
||||
// Keep small objects inline if components would increase size.
|
||||
if (count-1)*len(raw) <= count*len(refJSON)+len(name)+4 {
|
||||
continue
|
||||
}
|
||||
shared[name] = s.parent[s.key]
|
||||
s.parent[s.key] = map[string]any{"$ref": "#/components/" + kind + "/" + name}
|
||||
}
|
||||
if len(shared) > 0 {
|
||||
components[kind] = shared
|
||||
}
|
||||
}
|
||||
|
||||
// Share whole responses before counting inner objects. Otherwise a catalog
|
||||
// copied across 100 routes can create hundreds of unnecessary components
|
||||
// that only one shared response ultimately uses.
|
||||
factor("responses", responses)
|
||||
collectMedia := func(value any) {
|
||||
resp := value.(map[string]any)
|
||||
content, _ := resp["content"].(map[string]any)
|
||||
for _, value := range content {
|
||||
media := value.(map[string]any)
|
||||
if media["schema"] != nil {
|
||||
schemas = append(schemas, slot{media, "schema"})
|
||||
}
|
||||
es, _ := media["examples"].(map[string]any)
|
||||
for name := range es {
|
||||
examples = append(examples, slot{es, name})
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, s := range responses {
|
||||
collectMedia(s.parent[s.key])
|
||||
}
|
||||
sharedResponses, _ := components["responses"].(map[string]any)
|
||||
for _, resp := range sharedResponses {
|
||||
collectMedia(resp)
|
||||
}
|
||||
factor("examples", examples)
|
||||
factor("schemas", schemas)
|
||||
}
|
||||
107
cmd/go-omni/apigen/documentRefs_test.go
Normal file
107
cmd/go-omni/apigen/documentRefs_test.go
Normal file
@@ -0,0 +1,107 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSharedErrorDocumentReferences(t *testing.T) {
|
||||
source := `@app(defaultResponse: Common)
|
||||
response Common(codeType:int32;successCode:0;errorCode:9999;messageField:msg;dataMode:always)
|
||||
service demo {
|
||||
`
|
||||
for i := 0; i < 16; i++ {
|
||||
kind := "string"
|
||||
if i%2 == 0 {
|
||||
kind = "int64"
|
||||
}
|
||||
source += fmt.Sprintf("@handler list%d\nget /list%d returns ([]%s)\n", i, i, kind)
|
||||
}
|
||||
source += "}\n"
|
||||
spec, err := parseText(t, source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
catalog := apierrors.Catalog{DefaultStatus: 400, Definitions: map[string]apierrors.Definition{}}
|
||||
for i := 0; i < 80; i++ {
|
||||
status := 400
|
||||
if i%2 == 0 {
|
||||
status = 200
|
||||
}
|
||||
catalog.Definitions[fmt.Sprintf("ERROR_%d", i)] = apierrors.Definition{Status: status, Code: 400000 + i, Message: strings.Repeat("说明", 40) + "{label}不能超过{max}个字符", Params: []apierrors.Parameter{{Name: "label", Type: "string", Example: "企业名称"}, {Name: "max", Type: "int", Example: 100}}}
|
||||
}
|
||||
spec.DocumentCatalog = &catalog
|
||||
compiled, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var raw map[string]any
|
||||
if err = json.Unmarshal(compiled.Document, &raw); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
components := raw["components"].(map[string]any)
|
||||
for _, kind := range []string{"responses", "examples"} {
|
||||
if len(components[kind].(map[string]any)) == 0 {
|
||||
t.Fatal("missing shared", kind)
|
||||
}
|
||||
}
|
||||
expanded := document(t, compiled)
|
||||
for i := 0; i < 16; i++ {
|
||||
responses := operation(t, expanded, fmt.Sprintf("/list%d", i), "get")["responses"].(map[string]any)
|
||||
for _, status := range []string{"200", "400"} {
|
||||
media := responses[status].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
|
||||
if len(media["examples"].(map[string]any)) != 40 {
|
||||
t.Fatal("examples lost")
|
||||
}
|
||||
if status == "200" {
|
||||
choices := media["schema"].(map[string]any)["anyOf"].([]any)
|
||||
if len(choices) != 2 || choices[0].(map[string]any)["properties"].(map[string]any)["data"].(map[string]any)["type"] != "array" {
|
||||
t.Fatal("200 success shape lost")
|
||||
}
|
||||
}
|
||||
for _, example := range media["examples"].(map[string]any) {
|
||||
value := example.(map[string]any)["value"].(map[string]any)
|
||||
if _, ok := value["data"]; !ok || value["data"] != nil || !strings.Contains(value["msg"].(string), "企业名称不能超过100个字符") {
|
||||
t.Fatal(value)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
expandedBytes, _ := json.MarshalIndent(expanded, "", " ")
|
||||
if len(compiled.Document)*5 > len(expandedBytes) {
|
||||
t.Fatalf("poor dedup: %d vs %d", len(compiled.Document), len(expandedBytes))
|
||||
}
|
||||
again, err := Compile(spec)
|
||||
if err != nil || string(again.Document) != string(compiled.Document) {
|
||||
t.Fatal("non-deterministic document", err)
|
||||
}
|
||||
before, _ := json.Marshal(raw)
|
||||
compactDocument(raw)
|
||||
data, _ := json.Marshal(raw)
|
||||
if string(data) != string(before) {
|
||||
t.Fatal("compaction not idempotent")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSharedSchemasAndLiteralExampleReference(t *testing.T) {
|
||||
doc := map[string]any{"components": map[string]any{}, "paths": map[string]any{}}
|
||||
paths := doc["paths"].(map[string]any)
|
||||
for _, name := range []string{"first", "second", "third"} {
|
||||
schema := map[string]any{"type": "object", "description": strings.Repeat("schema description ", 40)}
|
||||
example := map[string]any{"summary": strings.Repeat("example ", 50), "value": map[string]any{"$ref": "literal business field"}}
|
||||
paths["/"+name] = map[string]any{"get": map[string]any{"responses": map[string]any{"200": map[string]any{"description": name, "content": map[string]any{"application/json": map[string]any{"schema": schema, "examples": map[string]any{"same": example}}}}}}}
|
||||
}
|
||||
compactDocument(doc)
|
||||
components := doc["components"].(map[string]any)
|
||||
if len(components["schemas"].(map[string]any)) != 1 {
|
||||
t.Fatal("schema not shared")
|
||||
}
|
||||
for _, example := range components["examples"].(map[string]any) {
|
||||
if example.(map[string]any)["value"].(map[string]any)["$ref"] != "literal business field" {
|
||||
t.Fatal("payload reference interpreted")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -19,11 +19,13 @@ import (
|
||||
)
|
||||
|
||||
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
const minimumVersion = "v0.14.0-preview"
|
||||
const minimumVersion = "v0.19.0-preview"
|
||||
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||
|
||||
type Options struct {
|
||||
generation GeneratorConfig
|
||||
lockHeld bool
|
||||
documentCatalogs map[string][]byte
|
||||
Database, Storage string
|
||||
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
|
||||
Check bool
|
||||
@@ -48,7 +50,7 @@ func Generate(o Options) (Output, error) {
|
||||
if o.API == "" || o.Dir == "" {
|
||||
return Output{}, errors.New("--api and --dir are required")
|
||||
}
|
||||
spec, err := Parse(o.API)
|
||||
spec, err := parseWithCatalogs(o.API, o.documentCatalogs)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
@@ -96,6 +98,11 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
return Output{}, err
|
||||
}
|
||||
o.Dir = directory
|
||||
genConfig, hasGenConfig, err := readGeneratorConfig(directory)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
o.generation = genConfig
|
||||
if err = checkRuntimeExtension(directory); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
@@ -145,13 +152,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
if !filepath.IsAbs(path) {
|
||||
path = filepath.Join(directory, path)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(path, "lifecycle", "runtime.go")); err == nil {
|
||||
if source, err := os.ReadFile(filepath.Join(path, "config", "env.go")); err == nil && strings.Contains(string(source), "func ServerEnvAliases(") {
|
||||
localCurrent = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !localCurrent {
|
||||
return Output{}, fmt.Errorf("existing framework dependency predates the generated Runtime, request policy and structured MySQL support; use %s or a current local replacement", minimumVersion)
|
||||
return Output{}, fmt.Errorf("existing framework dependency predates the generated configuration helpers; use %s or a current local replacement", minimumVersion)
|
||||
}
|
||||
}
|
||||
if o.FrameworkDir != "" {
|
||||
@@ -191,6 +198,10 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
for name, data := range base {
|
||||
seeds[name] = data
|
||||
}
|
||||
if !hasGenConfig {
|
||||
raw, _ := json.MarshalIndent(genConfig, "", " ")
|
||||
seeds[generatorConfigFile] = append(raw, '\n')
|
||||
}
|
||||
// The API source remains developer-owned. A relocated project regenerates
|
||||
// from its own api/ tree, including all imports.
|
||||
entry := "api/" + c.Spec.Entry
|
||||
@@ -241,9 +252,22 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
currentConfig, stillExists, err := readGeneratorConfig(directory)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
beforeConfig, _ := json.Marshal(genConfig)
|
||||
afterConfig, _ := json.Marshal(currentConfig)
|
||||
if hasGenConfig != stillExists || genConfig.inputDigest != currentConfig.inputDigest || string(beforeConfig) != string(afterConfig) {
|
||||
return Output{}, errors.New("generation configuration changed during generation; retry")
|
||||
}
|
||||
out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld)
|
||||
if requiredVersion != "" && requiredVersion != minimumVersion {
|
||||
out.Diagnostics = append(out.Diagnostics, Diagnostic{Severity: "review", Code: "frameworkVersion", Path: "go.mod", Before: requiredVersion, After: minimumVersion, Message: "project dependency differs from this CLI template version; use matching CLI/framework versions, or verify the explicit local replacement"})
|
||||
expectedVersion := minimumVersion
|
||||
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) {
|
||||
expectedVersion = info.Main.Version
|
||||
}
|
||||
if requiredVersion != "" && requiredVersion != expectedVersion {
|
||||
out.Diagnostics = append(out.Diagnostics, Diagnostic{Severity: "review", Code: "frameworkVersion", Path: "go.mod", Before: requiredVersion, After: expectedVersion, Message: "project dependency differs from this CLI version; use matching CLI/framework versions, or verify the explicit local replacement"})
|
||||
sortDiagnostics(out.Diagnostics)
|
||||
}
|
||||
return out, err
|
||||
@@ -450,7 +474,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
continue
|
||||
}
|
||||
hasAutomatic = true
|
||||
needsTypes = needsTypes || r.Request != "" || responseGoType(r.Response) != ""
|
||||
needsTypes = needsTypes || r.Request != "" || responseNeedsTypes(r.Response)
|
||||
needsBinder = needsBinder || !r.CustomBinding
|
||||
}
|
||||
text += fmt.Sprintf("%q;%q;", frameworkModule+"/httpx", o.Module+"/internal/svc")
|
||||
@@ -566,7 +590,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
}
|
||||
}
|
||||
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
|
||||
if r.Request != "" || responseGoType(r.Response) != "" {
|
||||
if r.Request != "" || responseNeedsTypes(r.Response) {
|
||||
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||
}
|
||||
if streamingBody || r.Produces != "" {
|
||||
@@ -597,7 +621,9 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
}
|
||||
put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text)
|
||||
put(seeds, "internal/logic/"+group+"/logic.go", fmt.Sprintf("package %s\nimport %q\ntype Logic struct {deps *svc.ServiceContext}\nfunc New(deps *svc.ServiceContext)*Logic{return &Logic{deps:deps}}\n", packageName, o.Module+"/internal/svc"))
|
||||
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
|
||||
if o.generation.Repository == "group" {
|
||||
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
|
||||
}
|
||||
}
|
||||
put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n")
|
||||
registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n"
|
||||
@@ -606,6 +632,9 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m)))
|
||||
}
|
||||
put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n")
|
||||
if o.generation.Middleware == "factories" {
|
||||
renderMiddlewareFactories(middlewareNames, generated, seeds)
|
||||
}
|
||||
for name, source := range baseTemplates {
|
||||
content := strings.ReplaceAll(source, "MODULE", o.Module)
|
||||
content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat)
|
||||
@@ -625,6 +654,16 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
|
||||
return nil, nil, err
|
||||
}
|
||||
for name, data := range infra {
|
||||
if o.generation.Middleware == "factories" && name == "internal/svc/runtimeGen.go" {
|
||||
formatted, err := format.Source(data)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
source := string(formatted)
|
||||
source = strings.Replace(source, "Middlewares: middleware.Defaults(),", "", 1)
|
||||
source = strings.Replace(source, "return c, nil", "if len(c.Middlewares) != 0 { return nil, errors.New(\"factory middleware is registered automatically; remove manual Middlewares assignments from Init\") }\nc.Middlewares, err = middleware.New(c)\nif err != nil { return nil, err }\nreturn c, nil", 1)
|
||||
data = []byte(source)
|
||||
}
|
||||
if strings.HasSuffix(name, "Gen.go") {
|
||||
generated[name] = append([]byte(generatedHeader), data...)
|
||||
} else {
|
||||
|
||||
110
cmd/go-omni/apigen/generatorConfig.go
Normal file
110
cmd/go-omni/apigen/generatorConfig.go
Normal file
@@ -0,0 +1,110 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
)
|
||||
|
||||
const generatorConfigFile = "omni.gen.json"
|
||||
|
||||
// GeneratorConfig contains developer-owned scaffolding options only.
|
||||
type GeneratorConfig struct {
|
||||
Repository string `json:"repository,omitempty"`
|
||||
Middleware string `json:"middleware,omitempty"`
|
||||
inputDigest string
|
||||
}
|
||||
|
||||
func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
|
||||
cfg := GeneratorConfig{Repository: "none", Middleware: "factories"}
|
||||
if err := safePath(root, generatorConfigFile); err != nil {
|
||||
return cfg, false, err
|
||||
}
|
||||
stat, err := os.Lstat(filepath.Join(root, generatorConfigFile))
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
// Existing projects keep their handwritten middleware signature until they
|
||||
// explicitly select factories; do not rewrite or adopt developer files.
|
||||
if raw, e := os.ReadFile(filepath.Join(root, manifestPath)); e == nil {
|
||||
var previous manifest
|
||||
if err := json.Unmarshal(raw, &previous); err != nil {
|
||||
return cfg, false, err
|
||||
}
|
||||
if previous.GeneratorConfig {
|
||||
return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options before regenerating")
|
||||
}
|
||||
cfg.Middleware = "handlers"
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
return cfg, false, e
|
||||
}
|
||||
return cfg, false, nil
|
||||
}
|
||||
if err != nil {
|
||||
return cfg, false, err
|
||||
}
|
||||
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
|
||||
return cfg, false, errors.New("omni.gen.json must be a regular file of at most 1 MiB")
|
||||
}
|
||||
raw, err := readGeneratorConfigFile(root, generatorConfigFile)
|
||||
if err != nil {
|
||||
return cfg, false, err
|
||||
}
|
||||
var keys map[string]json.RawMessage
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &keys); err != nil {
|
||||
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
|
||||
}
|
||||
for _, key := range []string{"mappings", "mappingFiles"} {
|
||||
if _, exists := keys[key]; exists {
|
||||
return cfg, false, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go, remove mappings/mappingFiles from omni.gen.json, then remove internal/mapping/mappingsGen.go; see COPY.md")
|
||||
}
|
||||
}
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
|
||||
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
|
||||
}
|
||||
digest := sha256.Sum256(raw)
|
||||
cfg.inputDigest = fmt.Sprintf("%x", digest)
|
||||
if cfg.Repository != "none" && cfg.Repository != "group" {
|
||||
return cfg, false, errors.New("omni.gen.json: repository must be none or group")
|
||||
}
|
||||
if cfg.Middleware != "factories" && cfg.Middleware != "handlers" {
|
||||
return cfg, false, errors.New("omni.gen.json: middleware must be factories or handlers")
|
||||
}
|
||||
return cfg, true, nil
|
||||
}
|
||||
|
||||
// Read through a bounded reader as well as checking stat: a growing file cannot
|
||||
// make configuration loading unbounded. safePath also rejects symlink escapes.
|
||||
func readGeneratorConfigFile(root, name string) ([]byte, error) {
|
||||
if err := safePath(root, name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
f, err := os.Open(filepath.Join(root, filepath.FromSlash(name)))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", name, err)
|
||||
}
|
||||
defer f.Close()
|
||||
stat, err := f.Stat()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
|
||||
return nil, fmt.Errorf("%s must be a regular file of at most 1 MiB", name)
|
||||
}
|
||||
raw, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(raw) > 1<<20 {
|
||||
return nil, fmt.Errorf("%s exceeds 1 MiB", name)
|
||||
}
|
||||
if trimmed := strings.TrimSpace(string(raw)); !strings.HasPrefix(trimmed, "{") {
|
||||
return nil, fmt.Errorf("%s must contain a JSON object", name)
|
||||
}
|
||||
return raw, nil
|
||||
}
|
||||
96
cmd/go-omni/apigen/generatorConfig_test.go
Normal file
96
cmd/go-omni/apigen/generatorConfig_test.go
Normal file
@@ -0,0 +1,96 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestWorkspaceRejectsRemovedMappingWithoutWrites(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
for _, name := range []string{"one", "two"} {
|
||||
if _, err := New(Options{Dir: filepath.Join(root, name), Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
firstAPI := filepath.Join(root, "one/api/service.api")
|
||||
writeFile(t, firstAPI, strings.Replace(readFile(t, firstAPI), "Greet a visitor", "Changed description", 1))
|
||||
writeFile(t, filepath.Join(root, "two", generatorConfigFile), `{"mappingFiles":["../../private.json"]}`)
|
||||
workspace := filepath.Join(root, "omni.workspace.json")
|
||||
writeFile(t, workspace, `{"services":[{"dir":"one"},{"dir":"two"}]}`)
|
||||
before := snapshot(t, root)
|
||||
for _, operation := range []string{"generate", "check"} {
|
||||
if err := Workspace(workspace, operation, io.Discard); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
|
||||
t.Fatal(operation, err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("workspace migration failure wrote files")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemovedMappingConfiguration(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, raw := range []string{`{"mappings":[]}`, `{"mappingFiles":[]}`, `{"mappings":null}`, `{"mappingFiles":["../../private.json"]}`} {
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), raw)
|
||||
before := snapshot(t, dir)
|
||||
for _, o := range []Options{{Dir: dir}, {Dir: dir, Check: true}, {Dir: dir, DryRun: true}} {
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("rejected config wrote project")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestOldMappingFileRequiresExplicitMigration(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
name := "internal/mapping/mappingsGen.go"
|
||||
body := generatedHeader + "package mapping\n"
|
||||
writeFile(t, filepath.Join(dir, name), body)
|
||||
var m manifest
|
||||
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Generated[name] = digest([]byte(body))
|
||||
raw, _ := json.Marshal(m)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
before := snapshot(t, dir)
|
||||
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||
for _, mode := range []string{"generate", "check", "dry-run"} {
|
||||
o.Check = mode == "check"
|
||||
o.DryRun = mode == "dry-run"
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("legacy file deleted or project changed")
|
||||
}
|
||||
}
|
||||
if err := os.Remove(filepath.Join(dir, name)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = false
|
||||
o.DryRun = false
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(readFile(t, filepath.Join(dir, manifestPath)), name) {
|
||||
t.Fatal("old ownership retained")
|
||||
}
|
||||
}
|
||||
@@ -71,6 +71,7 @@ func TestManagedGeneratedProject(t *testing.T) {
|
||||
files := map[string]string{
|
||||
"consumer_test.go": "consumer_test.go", "internal/svc/options.go": "options.go", "internal/svc/client.go": "client.go", "internal/logic/demo/check.go": "operations.go",
|
||||
"shutdown_test.go": "shutdown_test.go",
|
||||
"phases_test.go": "phases_test.go", "internal/svc/background.go": "background.go",
|
||||
}
|
||||
for path, fixture := range files {
|
||||
writeFile(t, filepath.Join(dir, path), readFile(t, filepath.Join("testdata/managed", fixture)))
|
||||
@@ -79,7 +80,7 @@ func TestManagedGeneratedProject(t *testing.T) {
|
||||
writeFile(t, filepath.Join(dir, "internal/logic/demo", name+".go"), "package demo\n")
|
||||
}
|
||||
configPath := filepath.Join(dir, "internal/config/config.go")
|
||||
writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1))
|
||||
writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n BackgroundMode string `json:\"backgroundMode\"`\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1))
|
||||
svcPath := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
source := strings.Replace(readFile(t, svcPath), "type ServiceContext struct {", "type ServiceContext struct {\n Client *Client", 1)
|
||||
source = strings.Replace(source, "return nil", "return initClient(ctx,c)", 1)
|
||||
@@ -169,12 +170,15 @@ func generateManagedTestProject(t *testing.T, o Options) error {
|
||||
_, err := Generate(o)
|
||||
return err
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
args := []string{"api", "generate", "--api", o.API, "--dir", o.Dir, "--module", o.Module}
|
||||
if o.Check {
|
||||
args = append(args, "--check")
|
||||
}
|
||||
if o.DryRun {
|
||||
args = append(args, "--dry-run")
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, cli, args...)
|
||||
cmd.Dir = filepath.Dir(o.Dir)
|
||||
if data, err := cmd.CombinedOutput(); err != nil {
|
||||
|
||||
@@ -74,7 +74,7 @@ func document(t *testing.T, c *Compiled) map[string]any {
|
||||
if err := json.Unmarshal(c.Document, &doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return doc
|
||||
return expandDocumentRefs(t, doc, doc).(map[string]any)
|
||||
}
|
||||
func operation(t *testing.T, doc map[string]any, path, method string) map[string]any {
|
||||
t.Helper()
|
||||
@@ -238,6 +238,7 @@ func TestDeclaredContractConsumer(t *testing.T) {
|
||||
source := filepath.Join(root, "input/service.api")
|
||||
writeFile(t, source, declaredAPI)
|
||||
dir := filepath.Join(root, "project")
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
|
||||
if _, err := Generate(Options{API: source, Dir: dir, Module: "example.com/dx", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -303,3 +304,21 @@ func TestAuthMetadataWithParameterNamePrefixes(t *testing.T) {
|
||||
t.Fatal("auth metadata lost")
|
||||
}
|
||||
}
|
||||
|
||||
// Existing semantic assertions resolve the standard local references emitted
|
||||
// by the compiler. documentRefs_test additionally checks the actual references.
|
||||
func expandDocumentRefs(t *testing.T, doc map[string]any, v any) any {
|
||||
t.Helper()
|
||||
switch v:=v.(type){
|
||||
case map[string]any:
|
||||
if ref,ok:=v["$ref"].(string);ok && strings.HasPrefix(ref,"#/components/") && len(v)==1 {
|
||||
var target any=doc
|
||||
for _,part:=range strings.Split(strings.TrimPrefix(ref,"#/"),"/"){m,ok:=target.(map[string]any);if !ok{t.Fatal("invalid reference",ref)};target=m[part]}
|
||||
if target==nil{t.Fatal("dangling reference",ref)};return expandDocumentRefs(t,doc,target)
|
||||
}
|
||||
out:=map[string]any{};for k,x:=range v{out[k]=expandDocumentRefs(t,doc,x)};return out
|
||||
case []any:
|
||||
out:=make([]any,len(v));for i,x:=range v{out[i]=expandDocumentRefs(t,doc,x)};return out
|
||||
default:return v
|
||||
}
|
||||
}
|
||||
|
||||
19
cmd/go-omni/apigen/middlewareFactory.go
Normal file
19
cmd/go-omni/apigen/middlewareFactory.go
Normal file
@@ -0,0 +1,19 @@
|
||||
package apigen
|
||||
|
||||
import "fmt"
|
||||
|
||||
func renderMiddlewareFactories(names []string, generated, seeds map[string][]byte) {
|
||||
// All project-specific types live behind this project-owned interface. svc
|
||||
// implements it; middleware never imports svc (and cannot form that cycle).
|
||||
seeds["internal/middleware/dependencies.go"] = []byte("package middleware\n// Dependencies is implemented by *svc.ServiceContext. Add typed accessors here\n// and implement them on ServiceContext. Factories borrow dependencies; register\n// their cleanup in ServiceContext.Init, not in a middleware factory.\ntype Dependencies interface {}\n")
|
||||
text := fmt.Sprintf("package middleware\nimport(\"fmt\";%q)\n// New constructs declared middleware after ServiceContext.Init succeeds.\nfunc New(deps Dependencies)(map[string]httpx.Middleware,error){\nresult:=map[string]httpx.Middleware{}\n", frameworkModule+"/httpx")
|
||||
if len(names) == 0 {
|
||||
text = fmt.Sprintf("package middleware\nimport %q\nfunc New(deps Dependencies)(map[string]httpx.Middleware,error){\nresult:=map[string]httpx.Middleware{}\n", frameworkModule+"/httpx")
|
||||
}
|
||||
for i, name := range names {
|
||||
delete(seeds, "internal/middleware/"+lower(name)+".go")
|
||||
text += fmt.Sprintf("m%d,err:=New%s(deps)\nif err!=nil{return nil,fmt.Errorf(%q,err)}\nif m%d==nil{return nil,fmt.Errorf(%q)}\nresult[%q]=m%d\n", i, upper(name), "middleware "+name+": %w", i, "middleware "+name+": factory returned nil", name, i)
|
||||
seeds["internal/middleware/"+lower(name)+"Factory.go"] = []byte(fmt.Sprintf("package middleware\nimport(\"errors\";%q)\n// New%s is project-owned. Implement this factory once; registration is generated.\n// Dependencies have already been initialized. Do not acquire owned resources here.\nfunc New%s(deps Dependencies)(httpx.Middleware,error){\nreturn nil,errors.New(%q)\n}\n", frameworkModule+"/httpx", upper(name), upper(name), "implement "+name+" middleware factory"))
|
||||
}
|
||||
generated["internal/middleware/registryGen.go"] = []byte(generatedHeader + text + "return result,nil\n}\n")
|
||||
}
|
||||
@@ -17,6 +17,7 @@ import (
|
||||
|
||||
type Spec struct {
|
||||
DocumentCatalog *apierrors.Catalog
|
||||
DocumentErrors *DocumentErrors
|
||||
SharedRoots map[string]SharedRoot
|
||||
SourcePaths map[string]string
|
||||
RequestPolicy contract.RequestPolicy
|
||||
@@ -38,6 +39,8 @@ type Type struct {
|
||||
Fields []Field
|
||||
Position string
|
||||
}
|
||||
|
||||
// An empty Type denotes a named field composition, expanded before compilation.
|
||||
type Field struct{ Name, Type, Tag, Description string }
|
||||
type Route struct {
|
||||
BoundMultipart bool
|
||||
@@ -72,7 +75,9 @@ type parser struct {
|
||||
imports []string
|
||||
}
|
||||
|
||||
func Parse(path string) (*Spec, error) {
|
||||
func Parse(path string) (*Spec, error) { return parseWithCatalogs(path, nil) }
|
||||
|
||||
func parseWithCatalogs(path string, catalogs map[string][]byte) (*Spec, error) {
|
||||
entry, err := filepath.Abs(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -82,11 +87,12 @@ func Parse(path string) (*Spec, error) {
|
||||
return nil, err
|
||||
}
|
||||
root := filepath.Dir(entry)
|
||||
roots, err := readSharedRoots(root)
|
||||
shared, err := readSharedConfig(root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
spec := &Spec{Version: "1.0.0", SharedRoots: roots, SourcePaths: map[string]string{}, Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||
roots := shared.Roots
|
||||
spec := &Spec{Version: "1.0.0", DocumentErrors: shared.Errors, SharedRoots: roots, SourcePaths: map[string]string{}, Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||
visiting, visited := map[string]bool{}, map[string]bool{}
|
||||
total := 0
|
||||
var visit func(string, string, int) error
|
||||
@@ -169,7 +175,7 @@ func Parse(path string) (*Spec, error) {
|
||||
if err = visit(entry, "", 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = readDocumentCatalog(root, spec); err != nil {
|
||||
if err = readDocumentCatalog(root, spec, catalogs); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = validateSharedPins(spec); err != nil {
|
||||
@@ -513,7 +519,11 @@ func (p *parser) parseType() error {
|
||||
}
|
||||
}
|
||||
if expression == "" {
|
||||
return p.err("field type required; embedded types are not supported")
|
||||
if p.tokens[p.at].kind == scanner.RawString {
|
||||
return p.err("composed types cannot carry field tags")
|
||||
}
|
||||
typ.Fields = append(typ.Fields, Field{Name: name, Description: description})
|
||||
continue
|
||||
}
|
||||
if p.tokens[p.at].kind != scanner.RawString {
|
||||
return p.err("field source tag required")
|
||||
|
||||
@@ -73,6 +73,8 @@ func TestConfiguration(t *testing.T){
|
||||
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
|
||||
t.Setenv("GO_OMNI_SERVER_LOG_FILE","./explicit.log");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!="./explicit.log"{t.Fatalf("canonical log file: %v",err)}
|
||||
t.Setenv("GO_OMNI_SERVER_LOG_OUTPUT","stderr");cfg,err=Load("");if err!=nil||cfg.Server.Log.Output!="stderr"{t.Fatalf("canonical output: %v",err)}
|
||||
_=os.Getenv("unused")
|
||||
}`
|
||||
endpoint := ""
|
||||
|
||||
@@ -4,11 +4,12 @@ import (
|
||||
"fmt"
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Response roots deliberately allow named DTOs, []DTO, *DTO and null. Keeping
|
||||
// these four shapes explicit makes nil/empty behavior uniform across encoders.
|
||||
// Response roots allow named DTOs, []DTO, []primitive, *DTO and null.
|
||||
// A nil root slice is always encoded as an empty JSON array.
|
||||
func responseShape(value string) (kind, name string) {
|
||||
if value == "null" {
|
||||
return "null", ""
|
||||
@@ -27,6 +28,9 @@ func responseGoType(value string) string {
|
||||
return ""
|
||||
}
|
||||
if kind == "array" {
|
||||
if primitiveTypes[name] != nil {
|
||||
return "[]" + name
|
||||
}
|
||||
return "[]types." + name
|
||||
}
|
||||
return "*types." + name
|
||||
@@ -36,9 +40,25 @@ func responseDataSchema(value string, bindings map[string]*contract.Binder) (*op
|
||||
if kind == "null" {
|
||||
return &openapi.Schema{Type: "object", Nullable: true, Enum: []any{nil}}, nil
|
||||
}
|
||||
if kind == "array" && primitiveTypes[name] != nil {
|
||||
if name == "File" {
|
||||
return nil, fmt.Errorf("File is request-only; use a response DTO")
|
||||
}
|
||||
// Reuse the contract compiler, including integer bounds and byte-slice rules.
|
||||
t := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: reflect.SliceOf(primitiveTypes[name]), Tag: `json:"value"`}})
|
||||
b, err := contract.Describe(t)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s, err := b.ResponseSchema()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return s.Properties["value"], nil
|
||||
}
|
||||
b, ok := bindings[name]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, *Name or null", name)
|
||||
return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, []string/[]int64/[]bool/[]any, *Name or null", name)
|
||||
}
|
||||
s, err := b.ResponseSchema()
|
||||
if err != nil {
|
||||
@@ -52,3 +72,7 @@ func responseDataSchema(value string, bindings map[string]*contract.Binder) (*op
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
var primitiveTypes = map[string]reflect.Type{"File": reflect.TypeOf(contract.File{}), "any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||
|
||||
func responseNeedsTypes(value string) bool { return strings.Contains(responseGoType(value), "types.") }
|
||||
|
||||
@@ -251,6 +251,7 @@ func TestResponseTemplateConsumer(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
|
||||
generate(source, true)
|
||||
writeFile(t, filepath.Join(dir, "response_test.go"), responseConsumerTest)
|
||||
for _, method := range []string{"Item", "Callback", "Partner"} {
|
||||
|
||||
@@ -23,48 +23,49 @@ type SharedRoot struct {
|
||||
SHA256 string `json:"sha256,omitempty"`
|
||||
}
|
||||
type sharedConfig struct {
|
||||
Roots map[string]SharedRoot `json:"roots"`
|
||||
Roots map[string]SharedRoot `json:"roots"`
|
||||
Errors *DocumentErrors `json:"errors,omitempty"`
|
||||
}
|
||||
|
||||
func readSharedRoots(root string) (map[string]SharedRoot, error) {
|
||||
func readSharedConfig(root string) (sharedConfig, error) {
|
||||
stat, err := os.Lstat(filepath.Join(root, importsFile))
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil, nil
|
||||
return sharedConfig{}, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return sharedConfig{}, err
|
||||
}
|
||||
if !stat.Mode().IsRegular() {
|
||||
return nil, errors.New("api: shared root configuration must be a regular file")
|
||||
return sharedConfig{}, errors.New("api: shared root configuration must be a regular file")
|
||||
}
|
||||
f, err := os.Open(filepath.Join(root, importsFile))
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil, nil
|
||||
return sharedConfig{}, nil
|
||||
}
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return sharedConfig{}, err
|
||||
}
|
||||
defer f.Close()
|
||||
raw, err := io.ReadAll(io.LimitReader(f, 65537))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return sharedConfig{}, err
|
||||
}
|
||||
if len(raw) > 65536 {
|
||||
return nil, errors.New("api: shared root config too large")
|
||||
return sharedConfig{}, errors.New("api: shared root config too large")
|
||||
}
|
||||
var cfg sharedConfig
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
|
||||
return nil, err
|
||||
return sharedConfig{}, err
|
||||
}
|
||||
if len(cfg.Roots) > 16 {
|
||||
return nil, errors.New("api: at most 16 shared roots")
|
||||
return sharedConfig{}, errors.New("api: at most 16 shared roots")
|
||||
}
|
||||
for name, r := range cfg.Roots {
|
||||
if !lowerName.MatchString(name) || r.Path == "" || len(r.Version) > 128 {
|
||||
return nil, errors.New("api: invalid shared root")
|
||||
return sharedConfig{}, errors.New("api: invalid shared root")
|
||||
}
|
||||
if r.SHA256 != "" && (len(r.SHA256) != 64 || strings.Trim(r.SHA256, "0123456789abcdef") != "") {
|
||||
return nil, errors.New("api: shared sha256 must be 64 lowercase hex characters")
|
||||
return sharedConfig{}, errors.New("api: shared sha256 must be 64 lowercase hex characters")
|
||||
}
|
||||
path := r.Path
|
||||
if !filepath.IsAbs(path) {
|
||||
@@ -72,22 +73,22 @@ func readSharedRoots(root string) (map[string]SharedRoot, error) {
|
||||
}
|
||||
r.Path, err = filepath.EvalSymlinks(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return sharedConfig{}, err
|
||||
}
|
||||
r.Path, err = filepath.Abs(r.Path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return sharedConfig{}, err
|
||||
}
|
||||
stat, err := os.Stat(r.Path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
return sharedConfig{}, err
|
||||
}
|
||||
if !stat.IsDir() {
|
||||
return nil, errors.New("api: shared root must be a directory")
|
||||
return sharedConfig{}, errors.New("api: shared root must be a directory")
|
||||
}
|
||||
cfg.Roots[name] = r
|
||||
}
|
||||
return cfg.Roots, nil
|
||||
return cfg, nil
|
||||
}
|
||||
func sourceWithin(root, name string) (string, error) {
|
||||
real, err := filepath.EvalSymlinks(name)
|
||||
@@ -122,7 +123,7 @@ func sourceMetadata(spec *Spec) map[string]any {
|
||||
return map[string]any{"files": hashes, "shared": shared}
|
||||
}
|
||||
func sharedSeeds(spec *Spec, directory string) ([]byte, error) {
|
||||
cfg := sharedConfig{Roots: map[string]SharedRoot{}}
|
||||
cfg := sharedConfig{Roots: map[string]SharedRoot{}, Errors: spec.DocumentErrors}
|
||||
for name, r := range spec.SharedRoots {
|
||||
rel, err := filepath.Rel(filepath.Join(directory, "api"), r.Path)
|
||||
if err != nil {
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"os/signal"
|
||||
"syscall"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"MODULE/internal/config"
|
||||
"MODULE/internal/handler"
|
||||
"MODULE/internal/svc"
|
||||
@@ -27,7 +28,9 @@ func main(){
|
||||
func run(ctx context.Context,path string)(runErr error){
|
||||
cfg,err:=config.Load(path);if err!=nil{return err}
|
||||
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||
// Unjoined background work may still log. The owning process exits on this
|
||||
// error; do not close its borrowed logger before that work has finished.
|
||||
defer func(){if !errors.Is(runErr,lifecycle.ErrResourcesRetained){runErr=errors.Join(runErr,logger.Close())}}()
|
||||
deps,err:=svc.New(ctx,logger,cfg);if err!=nil{return err}
|
||||
// RunContext drains HTTP requests before returning. This scope owns dependencies
|
||||
// and closes them exactly once, before closing the borrowed logger.
|
||||
@@ -48,6 +51,17 @@ This project is generated from an editable API contract. Business code uses
|
||||
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
|
||||
main.go and HTTP composition are continuously generator-owned.
|
||||
|
||||
omni.gen.json selects repository: none (default) or group, and middleware:
|
||||
factories for new projects. Named middleware factories run after svc.Init;
|
||||
implement Dependencies accessors on ServiceContext and New<Name>(deps) once.
|
||||
Unimplemented/nil factories fail startup and clean registered dependencies.
|
||||
Existing projects retain middleware: handlers until explicitly migrated.
|
||||
Ordinary Logic uses the generated Request/Response types directly. Where an
|
||||
independent business type is needed, use ordinary Go or the optional copyx package.
|
||||
There are no mappings/mappingFiles settings or generated mapping files in v0.19.
|
||||
See COPY.md in the framework repository for copy policies and migration steps,
|
||||
and GENERATION.md for field composition and middleware factories.
|
||||
|
||||
## Run
|
||||
|
||||
`+"```sh"+`
|
||||
@@ -130,8 +144,9 @@ internal/svc/runtimeGen.go # GENERATED lifecycle and infrastructure startup
|
||||
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
|
||||
internal/handler/<group>/ # generated binding and Logic adapters
|
||||
internal/logic/<group>/ # developer-owned business operations
|
||||
internal/repository/<group>/ # developer-owned persistence contracts
|
||||
internal/middleware/ # named hooks and generated registry
|
||||
internal/repository/<group>/ # optional; repository: group creates editable seeds
|
||||
internal/middleware/ # typed dependency interface, factories and registry
|
||||
omni.gen.json # developer-owned repository/middleware options
|
||||
docs/openapi.json # generated OpenAPI 3.0.3
|
||||
.goOmni/apiManifest.json # ownership and content hashes
|
||||
`+"```"+`
|
||||
@@ -182,7 +197,16 @@ then immediately register each cleanup with OnClose. A later initialization fail
|
||||
runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware
|
||||
provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated
|
||||
once before route registration; logging is configured earlier through Config.
|
||||
The generated main runs HTTP draining before closing svc resources, then the logger.
|
||||
The generated main runs HTTP draining, all OnStop hooks, all OnDrain hooks, resource
|
||||
OnClose callbacks, then logger close. Stop/drain hooks within a phase run concurrently;
|
||||
combine components with ordering dependencies into one pair. Pair OnStop/OnDrain
|
||||
with the same name. OnDrain returns nil only after final work and acknowledgements
|
||||
have finished. Use WorkContext for accepted background work; Context is an early
|
||||
shutdown notification. server.lifecycle configures stopTimeout (5s), drainTimeout
|
||||
(10s), and cancelTimeout (5s grace per timed-out phase). Zero uses defaults.
|
||||
Timeout/failure cancels WorkContext. If hooks cannot prove completion, resources
|
||||
and the borrowed logger are retained and the generated process exits unsuccessfully.
|
||||
Errors retain their causes. No automatic retry of close or force killing of goroutines.
|
||||
Do not close borrowed resources or attach duplicate cleanup hooks to App. The startup
|
||||
context cancels on signals; long-lived clients/workers that requests depend on need
|
||||
their own lifetime, canceled by OnClose after HTTP draining.
|
||||
|
||||
54
cmd/go-omni/apigen/testdata/arrays/consumer_test.go
vendored
Normal file
54
cmd/go-omni/apigen/testdata/arrays/consumer_test.go
vendored
Normal file
@@ -0,0 +1,54 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"example.com/arrays/internal/config"
|
||||
"example.com/arrays/internal/handler"
|
||||
"example.com/arrays/internal/svc"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestArrayWire(t *testing.T) {
|
||||
cfg := config.Default()
|
||||
cfg.Server.Log.Writer = io.Discard
|
||||
logger, err := logx.New(cfg.Server.Log)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer logger.Close()
|
||||
deps, err := svc.New(context.Background(), logger, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer deps.Close()
|
||||
app, err := handler.NewApp(deps)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer app.Close()
|
||||
for path, want := range map[string]string{
|
||||
"/strings": `["企业","备注"]`, "/integers": `[9223372036854775807,-3]`, "/dynamic": `["text",7,true,null,{"name":"企业"},[false,2]]`, "/empty": `[]`, "/raw": `[true,false]`,
|
||||
} {
|
||||
w := httptest.NewRecorder()
|
||||
app.ServeHTTP(w, httptest.NewRequest("GET", path, nil))
|
||||
got := strings.TrimSpace(w.Body.String())
|
||||
if w.Code != 200 {
|
||||
t.Fatal(got)
|
||||
}
|
||||
if path != "/raw" {
|
||||
var obj map[string]json.RawMessage
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &obj); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got = string(obj["data"])
|
||||
}
|
||||
if got != want {
|
||||
t.Fatalf("%s got %s want %s", path, got, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
13
cmd/go-omni/apigen/testdata/arrays/values.go
vendored
Normal file
13
cmd/go-omni/apigen/testdata/arrays/values.go
vendored
Normal file
@@ -0,0 +1,13 @@
|
||||
package arrays
|
||||
|
||||
import "context"
|
||||
|
||||
func (s *Logic) Strings(context.Context) ([]string, error) { return []string{"企业", "备注"}, nil }
|
||||
func (s *Logic) Integers(context.Context) ([]int64, error) {
|
||||
return []int64{9223372036854775807, -3}, nil
|
||||
}
|
||||
func (s *Logic) Dynamic(context.Context) ([]any, error) {
|
||||
return []any{"text", 7, true, nil, map[string]any{"name": "企业"}, []any{false, 2}}, nil
|
||||
}
|
||||
func (s *Logic) Empty(context.Context) ([]any, error) { return nil, nil }
|
||||
func (s *Logic) Raw(context.Context) ([]bool, error) { return []bool{true, false}, nil }
|
||||
30
cmd/go-omni/apigen/testdata/composition/accessFactory.go
vendored
Normal file
30
cmd/go-omni/apigen/testdata/composition/accessFactory.go
vendored
Normal file
@@ -0,0 +1,30 @@
|
||||
package middleware
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
var ErrFactory = errors.New("factory fault")
|
||||
|
||||
func NewAccess(deps Dependencies) (httpx.Middleware, error) {
|
||||
if deps.Access() == nil {
|
||||
return nil, errors.New("dependencies not initialized")
|
||||
}
|
||||
switch deps.FactoryMode() {
|
||||
case "fail":
|
||||
return nil, ErrFactory
|
||||
case "nil":
|
||||
return nil, nil
|
||||
case "panic":
|
||||
panic(ErrFactory)
|
||||
}
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
if err := deps.Access().Check(c.Context()); err != nil {
|
||||
return err
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}, nil
|
||||
}
|
||||
52
cmd/go-omni/apigen/testdata/composition/amount.go
vendored
Normal file
52
cmd/go-omni/apigen/testdata/composition/amount.go
vendored
Normal file
@@ -0,0 +1,52 @@
|
||||
package amount
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math/big"
|
||||
"regexp"
|
||||
)
|
||||
|
||||
// This fixture's business policy is exact decimal HALF_UP. The framework does
|
||||
// not choose its precision, rounding mode or currency.
|
||||
type Money string
|
||||
|
||||
var decimalPattern = regexp.MustCompile(`^-?[0-9]+(\.[0-9]+)?$`)
|
||||
|
||||
func (m Money) String() string {
|
||||
if m == "" {
|
||||
return "0.00"
|
||||
}
|
||||
return string(m)
|
||||
}
|
||||
func (m Money) MarshalJSON() ([]byte, error) { return json.Marshal(m.String()) }
|
||||
func (m *Money) UnmarshalJSON(data []byte) error {
|
||||
var text string
|
||||
if err := json.Unmarshal(data, &text); err != nil {
|
||||
return err
|
||||
}
|
||||
if !decimalPattern.MatchString(text) {
|
||||
return errors.New("invalid decimal")
|
||||
}
|
||||
value, ok := new(big.Rat).SetString(text)
|
||||
if !ok {
|
||||
return errors.New("invalid decimal")
|
||||
}
|
||||
value.Mul(value, big.NewRat(100, 1))
|
||||
sign := value.Sign()
|
||||
value.Abs(value)
|
||||
cents, rem := new(big.Int), new(big.Int)
|
||||
cents.QuoRem(value.Num(), value.Denom(), rem)
|
||||
if rem.Mul(rem, big.NewInt(2)).Cmp(value.Denom()) >= 0 {
|
||||
cents.Add(cents, big.NewInt(1))
|
||||
}
|
||||
whole, fraction := new(big.Int), new(big.Int)
|
||||
whole.QuoRem(cents, big.NewInt(100), fraction)
|
||||
prefix := ""
|
||||
if sign < 0 && cents.Sign() != 0 {
|
||||
prefix = "-"
|
||||
}
|
||||
*m = Money(fmt.Sprintf("%s%s.%02d", prefix, whole.String(), fraction.Int64()))
|
||||
return nil
|
||||
}
|
||||
36
cmd/go-omni/apigen/testdata/composition/client.go
vendored
Normal file
36
cmd/go-omni/apigen/testdata/composition/client.go
vendored
Normal file
@@ -0,0 +1,36 @@
|
||||
package svc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"example.com/composition/internal/middleware"
|
||||
"sync/atomic"
|
||||
)
|
||||
|
||||
var Closes atomic.Int64
|
||||
var ErrInit = errors.New("init fault")
|
||||
|
||||
type Client struct{ closed atomic.Bool }
|
||||
|
||||
func (c *Client) Check(ctx context.Context) error {
|
||||
if c.closed.Load() {
|
||||
return errors.New("client closed before HTTP drained")
|
||||
}
|
||||
return ctx.Err()
|
||||
}
|
||||
func (c *ServiceContext) initClient(ctx context.Context) error {
|
||||
c.client = &Client{}
|
||||
c.OnClose(func() error {
|
||||
if !c.client.closed.CompareAndSwap(false, true) {
|
||||
return errors.New("double close")
|
||||
}
|
||||
Closes.Add(1)
|
||||
return nil
|
||||
})
|
||||
if c.Config.FactoryMode == "initFail" {
|
||||
return ErrInit
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (c *ServiceContext) Access() middleware.AccessChecker { return c.client }
|
||||
func (c *ServiceContext) FactoryMode() string { return c.Config.FactoryMode }
|
||||
217
cmd/go-omni/apigen/testdata/composition/consumer_test.go
vendored
Normal file
217
cmd/go-omni/apigen/testdata/composition/consumer_test.go
vendored
Normal file
@@ -0,0 +1,217 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
"example.com/composition/internal/amount"
|
||||
"example.com/composition/internal/config"
|
||||
"example.com/composition/internal/convert"
|
||||
"example.com/composition/internal/domain"
|
||||
"example.com/composition/internal/handler"
|
||||
"example.com/composition/internal/middleware"
|
||||
"example.com/composition/internal/svc"
|
||||
"example.com/composition/internal/types"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
func logger(t *testing.T) *logx.Logger {
|
||||
t.Helper()
|
||||
cfg := logx.Default()
|
||||
cfg.Writer = io.Discard
|
||||
l, e := logx.New(cfg)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
t.Cleanup(func() { _ = l.Close() })
|
||||
return l
|
||||
}
|
||||
func TestHTTPComposition(t *testing.T) {
|
||||
deps, e := svc.New(context.Background(), logger(t))
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer deps.Close()
|
||||
app, e := handler.NewApp(deps)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer app.Close()
|
||||
for _, path := range []string{"/enterprise", "/tenant/123", "/admin/123"} {
|
||||
for _, tc := range []struct {
|
||||
name, body string
|
||||
status int
|
||||
salary, nameValue string
|
||||
employer any
|
||||
}{
|
||||
{"exact", `{"name":"employee","salary":"90071992547409.91","employer":null,"enabled":false}`, 200, "90071992547409.91", "employee", nil},
|
||||
{"round", `{"salary":"1.005","employer":"0"}`, 200, "1.01", "", "0.00"},
|
||||
{"missing", `{}`, 200, "0.00", "", nil},
|
||||
{"null", `{"name":null,"salary":null,"employer":null,"enabled":null}`, 200, "0.00", "", nil},
|
||||
{"zero", `{"salary":"0","enabled":false}`, 200, "0.00", "", nil},
|
||||
{"invalid", `{"salary":"abc"}`, 400, "", "", nil},
|
||||
{"empty", `{"salary":""}`, 400, "", "", nil},
|
||||
{"numeric", `{"salary":100}`, 400, "", "", nil},
|
||||
{"flat", `{"SalaryFields":{"salary":"1"}}`, 400, "", "", nil},
|
||||
{"validation", `{"name":"` + strings.Repeat("x", 81) + `"}`, 400, "", "", nil},
|
||||
} {
|
||||
t.Run(path+tc.name, func(t *testing.T) {
|
||||
req := httptest.NewRequest("POST", path, strings.NewReader(tc.body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
w := httptest.NewRecorder()
|
||||
app.Engine().ServeHTTP(w, req)
|
||||
if w.Code != tc.status {
|
||||
t.Fatalf("status=%d body=%s", w.Code, w.Body.String())
|
||||
}
|
||||
if w.Code == 200 {
|
||||
var out map[string]any
|
||||
if e := json.Unmarshal(w.Body.Bytes(), &out); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if out["salary"] != tc.salary || out["employeeName"] != tc.nameValue || out["employer"] != tc.employer || out["id"] != "9007199254740993" {
|
||||
t.Fatal(out)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestFullFormAndCopySemantics(t *testing.T) {
|
||||
name := "employee"
|
||||
money := amount.Money("2.00")
|
||||
yes := true
|
||||
req := types.EnterpriseRequest{Name: &name, Salary: &money, Enabled: &yes, Labels: []string{"one"}}
|
||||
first, e := convert.Employee(req)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
second, e := convert.Employee(types.EnterpriseRequest{})
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if first.EmployeeName != "employee" || !first.Enabled || second.EmployeeName != "" || second.Enabled || second.Salary.String() != "0.00" || second.Employer != nil {
|
||||
t.Fatal("full-form update became patch behavior")
|
||||
}
|
||||
// Direct assignments retain standard Go aliasing, including scalar pointers.
|
||||
first.Labels[0] = "shared"
|
||||
if req.Labels[0] != "shared" {
|
||||
t.Fatal("unexpected implicit deep copy")
|
||||
}
|
||||
req.Employer = &money
|
||||
first, e = convert.Employee(req)
|
||||
if e != nil || first.Employer != req.Employer {
|
||||
t.Fatal("nullable pointer changed", e)
|
||||
}
|
||||
bad := "reject"
|
||||
failed, e := convert.Employee(types.EnterpriseRequest{Name: &bad})
|
||||
if !errors.Is(e, convert.ErrName) || failed.EmployeeName != "" || !strings.Contains(e.Error(), "EmployeeName") {
|
||||
t.Fatal("converter cause/zero result lost", e)
|
||||
}
|
||||
empty, e := convert.ToBatch(domain.Batch{})
|
||||
if e != nil || empty.Rows != nil {
|
||||
t.Fatal("nil slice changed", e)
|
||||
}
|
||||
src := domain.Batch{Rows: []domain.Row{}, Fixed: [2]domain.Row{{Value: "a"}, {Value: "b"}}, Optional: &name}
|
||||
out, e := convert.ToBatch(src)
|
||||
if e != nil || out.Rows == nil || len(out.Rows) != 0 || out.Fixed[1].Value != "b" || out.Optional != src.Optional {
|
||||
t.Fatal(out, e)
|
||||
}
|
||||
src.Rows = []domain.Row{{Value: "one"}}
|
||||
out, e = convert.ToBatch(src)
|
||||
if e != nil || out.Rows[0].Value != "one" {
|
||||
t.Fatal(out, e)
|
||||
}
|
||||
out.Rows[0].Value = "changed"
|
||||
if src.Rows[0].Value != "one" {
|
||||
t.Fatal("element mapping reused input slice")
|
||||
}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 32; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
result, err := convert.Employee(types.EnterpriseRequest{Name: &name, Salary: &money})
|
||||
if err != nil || result.EmployeeName != name || result.Salary != money {
|
||||
t.Error(result, err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
}
|
||||
func TestFactoryLifecycle(t *testing.T) {
|
||||
for _, mode := range []string{"fail", "nil", "initFail", "panic"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
before := svc.Closes.Load()
|
||||
cfg := config.Default()
|
||||
cfg.FactoryMode = mode
|
||||
if mode == "panic" {
|
||||
func() {
|
||||
defer func() {
|
||||
if recover() != middleware.ErrFactory {
|
||||
t.Error("panic cause changed")
|
||||
}
|
||||
}()
|
||||
_, _ = svc.New(context.Background(), logger(t), cfg)
|
||||
}()
|
||||
} else {
|
||||
deps, err := svc.New(context.Background(), logger(t), cfg)
|
||||
if deps != nil || err == nil {
|
||||
t.Fatal("factory did not fail closed")
|
||||
}
|
||||
if mode == "fail" && !errors.Is(err, middleware.ErrFactory) {
|
||||
t.Fatal("factory cause lost", err)
|
||||
}
|
||||
if mode == "initFail" && !errors.Is(err, svc.ErrInit) {
|
||||
t.Fatal("init cause lost", err)
|
||||
}
|
||||
}
|
||||
if svc.Closes.Load() != before+1 {
|
||||
t.Fatal("dependency not closed exactly once")
|
||||
}
|
||||
})
|
||||
}
|
||||
deps, e := svc.New(context.Background(), logger(t))
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e = deps.Access().Check(context.Background()); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
before := svc.Closes.Load()
|
||||
if e = deps.Close(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e = deps.Close(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if svc.Closes.Load() != before+1 {
|
||||
t.Fatal("normal close repeated")
|
||||
}
|
||||
}
|
||||
func TestGeneratedEntryListenFailure(t *testing.T) {
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer listener.Close()
|
||||
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||
if err = os.WriteFile(path, []byte("server:\n address: "+listener.Addr().String()+"\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := svc.Closes.Load()
|
||||
if err = run(context.Background(), path); err == nil {
|
||||
t.Fatal("listen should fail")
|
||||
}
|
||||
if svc.Closes.Load() != before+1 {
|
||||
t.Fatal("listen failure did not close dependencies")
|
||||
}
|
||||
}
|
||||
21
cmd/go-omni/apigen/testdata/composition/converters.go
vendored
Normal file
21
cmd/go-omni/apigen/testdata/composition/converters.go
vendored
Normal file
@@ -0,0 +1,21 @@
|
||||
package convert
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
var ErrName = errors.New("rejected name")
|
||||
|
||||
// Explicit project rules: full-form omission clears strings/bools and uses zero
|
||||
// for required money. Nullable employer amounts remain nil by direct assignment.
|
||||
func Name(value *string) (string, error) {
|
||||
if value == nil {
|
||||
return "", nil
|
||||
}
|
||||
if *value == "reject" {
|
||||
return "", ErrName
|
||||
}
|
||||
return *value, nil
|
||||
}
|
||||
func ID(value int64) string { return strconv.FormatInt(value, 10) }
|
||||
29
cmd/go-omni/apigen/testdata/composition/copy.go
vendored
Normal file
29
cmd/go-omni/apigen/testdata/composition/copy.go
vendored
Normal file
@@ -0,0 +1,29 @@
|
||||
package convert
|
||||
|
||||
import (
|
||||
"example.com/composition/internal/domain"
|
||||
"example.com/composition/internal/types"
|
||||
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
)
|
||||
|
||||
// The three endpoints reuse one ordinary Go helper and their existing scalar.
|
||||
// There is no field list or package/type name JSON to keep in sync.
|
||||
func Employee[S any](source S) (domain.Employee, error) {
|
||||
var target domain.Employee
|
||||
err := copyx.Copy(&target, source,
|
||||
copyx.WithFieldNames[S, domain.Employee](map[string]string{"Name": "EmployeeName"}),
|
||||
copyx.WithNilPolicy(copyx.NilZero),
|
||||
copyx.WithConverter(Name),
|
||||
)
|
||||
return target, err
|
||||
}
|
||||
func Response(source domain.Result) (types.EmployeeResponse, error) {
|
||||
var target types.EmployeeResponse
|
||||
err := copyx.Copy(&target, source, copyx.WithConverter(func(v int64) (string, error) { return ID(v), nil }))
|
||||
return target, err
|
||||
}
|
||||
func ToBatch(source domain.Batch) (domain.BatchDTO, error) {
|
||||
var target domain.BatchDTO
|
||||
err := copyx.Copy(&target, source)
|
||||
return target, err
|
||||
}
|
||||
9
cmd/go-omni/apigen/testdata/composition/dependencies.go
vendored
Normal file
9
cmd/go-omni/apigen/testdata/composition/dependencies.go
vendored
Normal file
@@ -0,0 +1,9 @@
|
||||
package middleware
|
||||
|
||||
import "context"
|
||||
|
||||
type AccessChecker interface{ Check(context.Context) error }
|
||||
type Dependencies interface {
|
||||
Access() AccessChecker
|
||||
FactoryMode() string
|
||||
}
|
||||
29
cmd/go-omni/apigen/testdata/composition/domain.go
vendored
Normal file
29
cmd/go-omni/apigen/testdata/composition/domain.go
vendored
Normal file
@@ -0,0 +1,29 @@
|
||||
package domain
|
||||
|
||||
import "example.com/composition/internal/amount"
|
||||
|
||||
type Employee struct {
|
||||
EmployeeName string
|
||||
Salary amount.Money
|
||||
Employer *amount.Money
|
||||
Enabled bool
|
||||
Labels []string
|
||||
}
|
||||
type Result struct {
|
||||
Employee
|
||||
ID int64
|
||||
}
|
||||
|
||||
// Distinct container item types exercise explicit Go collection copying.
|
||||
type Row struct{ Value string }
|
||||
type RowDTO struct{ Value string }
|
||||
type Batch struct {
|
||||
Rows []Row
|
||||
Fixed [2]Row
|
||||
Optional *string
|
||||
}
|
||||
type BatchDTO struct {
|
||||
Rows []RowDTO
|
||||
Fixed [2]RowDTO
|
||||
Optional *string
|
||||
}
|
||||
63
cmd/go-omni/apigen/testdata/composition/service.api
vendored
Normal file
63
cmd/go-omni/apigen/testdata/composition/service.api
vendored
Normal file
@@ -0,0 +1,63 @@
|
||||
scalar Money (
|
||||
goImport: "example.com/composition/internal/amount"
|
||||
goType: Money
|
||||
jsonInput: string
|
||||
jsonOutput: string
|
||||
)
|
||||
|
||||
type SalaryFields {
|
||||
// Employee display name.
|
||||
Name *string `json:"name,optional" binding:"max=80"`
|
||||
Salary *Money `json:"salary,optional"`
|
||||
Employer *Money `json:"employer,optional"`
|
||||
Enabled *bool `json:"enabled,optional"`
|
||||
Labels []string `json:"labels,optional"`
|
||||
}
|
||||
|
||||
type EnterpriseRequest {
|
||||
SalaryFields
|
||||
}
|
||||
type TenantRequest {
|
||||
SalaryFields
|
||||
CompanyID string `path:"companyId"`
|
||||
}
|
||||
type AdminRequest {
|
||||
SalaryFields
|
||||
CompanyID string `path:"companyId"`
|
||||
}
|
||||
type EmployeeResponse {
|
||||
ID string `json:"id"`
|
||||
EmployeeName string `json:"employeeName"`
|
||||
Salary Money `json:"salary"`
|
||||
Employer *Money `json:"employer"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Labels []string `json:"labels"`
|
||||
}
|
||||
|
||||
@server (
|
||||
group: enterprise
|
||||
middleware: Access
|
||||
response: raw
|
||||
)
|
||||
service composition {
|
||||
@handler save
|
||||
post /enterprise (EnterpriseRequest) returns (EmployeeResponse)
|
||||
}
|
||||
@server (
|
||||
group: tenant
|
||||
middleware: Access
|
||||
response: raw
|
||||
)
|
||||
service composition {
|
||||
@handler save
|
||||
post /tenant/:companyId (TenantRequest) returns (EmployeeResponse)
|
||||
}
|
||||
@server (
|
||||
group: admin
|
||||
middleware: Access
|
||||
response: raw
|
||||
)
|
||||
service composition {
|
||||
@handler save
|
||||
post /admin/:companyId (AdminRequest) returns (EmployeeResponse)
|
||||
}
|
||||
49
cmd/go-omni/apigen/testdata/managed/background.go
vendored
Normal file
49
cmd/go-omni/apigen/testdata/managed/background.go
vendored
Normal file
@@ -0,0 +1,49 @@
|
||||
package svc
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/task"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
)
|
||||
|
||||
type BackgroundState struct {
|
||||
Stop, Drain, Release, Started chan struct{}
|
||||
Wrote atomic.Bool
|
||||
Future *task.Future
|
||||
}
|
||||
|
||||
var Background atomic.Pointer[BackgroundState]
|
||||
|
||||
func initBackground(c *ServiceContext) error {
|
||||
state := &BackgroundState{Stop: make(chan struct{}), Drain: make(chan struct{}), Release: make(chan struct{}), Started: make(chan struct{})}
|
||||
baseline := Closed.Load()
|
||||
runner, err := task.New(task.Config{Workers: 1, QueueSize: 1, TaskTimeout: time.Minute, OnError: func(string, error) {}})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.OnStop("jobs", func(ctx context.Context) error { close(state.Stop); return runner.StopAccepting(ctx) })
|
||||
c.OnDrain("jobs", func(ctx context.Context) error { close(state.Drain); return runner.Wait(ctx) })
|
||||
state.Future, err = runner.Submit(c.WorkContext(), "finish", func(ctx context.Context) error {
|
||||
close(state.Started)
|
||||
if c.Config.BackgroundMode == "auto" {
|
||||
<-state.Stop
|
||||
} else {
|
||||
<-state.Release
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
if Closed.Load() != baseline {
|
||||
return errors.New("shared client closed before background write")
|
||||
}
|
||||
state.Wrote.Store(true)
|
||||
return nil
|
||||
})
|
||||
if err == nil {
|
||||
<-state.Started
|
||||
}
|
||||
Background.Store(state)
|
||||
return err
|
||||
}
|
||||
@@ -20,6 +20,11 @@ func initClient(ctx context.Context, c *ServiceContext) error {
|
||||
Opened.Add(1)
|
||||
c.Client = &Client{}
|
||||
c.OnClose(func() error { Closed.Add(1); c.Logger.Info("test.client.closed"); return nil })
|
||||
if c.Config.BackgroundMode != "" {
|
||||
if err := initBackground(c); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if c.Config.FailInit {
|
||||
return errors.New("client initialization failed")
|
||||
}
|
||||
|
||||
155
cmd/go-omni/apigen/testdata/managed/phases_test.go
vendored
Normal file
155
cmd/go-omni/apigen/testdata/managed/phases_test.go
vendored
Normal file
@@ -0,0 +1,155 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"example.com/managed/internal/svc"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func phasedConfig(t *testing.T, address, mode string, fail bool) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(t.TempDir(), "config.json")
|
||||
drainTimeout := "1s"
|
||||
if mode == "stuck" {
|
||||
drainTimeout = "50ms"
|
||||
}
|
||||
data, _ := json.Marshal(map[string]any{"backgroundMode": mode, "failInit": fail, "server": map[string]any{"address": address, "lifecycle": map[string]string{"stopTimeout": "1s", "drainTimeout": drainTimeout, "cancelTimeout": "50ms"}}})
|
||||
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
func await(t *testing.T, ch <-chan struct{}) {
|
||||
t.Helper()
|
||||
select {
|
||||
case <-ch:
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("phase did not arrive")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGeneratedBackgroundPhases(t *testing.T) {
|
||||
for _, transport := range []string{"nethttp", "native"} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_TRANSPORT", transport)
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
addr := listener.Addr().String()
|
||||
_ = listener.Close()
|
||||
svc.Started = make(chan struct{}, 1)
|
||||
svc.Release = make(chan struct{})
|
||||
before := svc.Closed.Load()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
path := phasedConfig(t, addr, "hold", false)
|
||||
go func() { done <- run(ctx, path) }()
|
||||
waitReady(t, "http://"+addr, done)
|
||||
state := svc.Background.Load()
|
||||
defer func() {
|
||||
select {
|
||||
case <-state.Release:
|
||||
default:
|
||||
close(state.Release)
|
||||
}
|
||||
}()
|
||||
requestDone := make(chan error, 1)
|
||||
go func() {
|
||||
client := http.Client{Timeout: 3 * time.Second}
|
||||
resp, e := client.Get("http://" + addr + "/slow")
|
||||
if e == nil {
|
||||
_, _ = io.Copy(io.Discard, resp.Body)
|
||||
_ = resp.Body.Close()
|
||||
if resp.StatusCode != 200 {
|
||||
e = errors.New("HTTP drain failed")
|
||||
}
|
||||
}
|
||||
requestDone <- e
|
||||
}()
|
||||
await(t, svc.Started)
|
||||
deps := svc.Current.Load()
|
||||
deps.RequestShutdown(svc.FatalDependency)
|
||||
if deps.WorkContext().Err() != nil {
|
||||
t.Fatal("accepted work canceled at signal")
|
||||
}
|
||||
select {
|
||||
case <-state.Stop:
|
||||
t.Fatal("stopped background before HTTP finished")
|
||||
default:
|
||||
}
|
||||
close(svc.Release)
|
||||
if err := <-requestDone; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
await(t, state.Drain)
|
||||
if svc.Closed.Load() != before {
|
||||
t.Fatal("client closed during background drain")
|
||||
}
|
||||
close(state.Release)
|
||||
if err := <-done; !errors.Is(err, svc.FatalDependency) || errors.Is(err, lifecycle.ErrResourcesRetained) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !state.Wrote.Load() || svc.Closed.Load() != before+1 {
|
||||
t.Fatal("missing final write or cleanup")
|
||||
}
|
||||
if err := state.Future.Wait(context.Background()); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestGeneratedBackgroundFailureCleanup(t *testing.T) {
|
||||
for _, failInit := range []bool{false, true} {
|
||||
t.Run(map[bool]string{true: "initialization", false: "listen"}[failInit], func(t *testing.T) {
|
||||
l, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer l.Close()
|
||||
before := svc.Closed.Load()
|
||||
err = run(context.Background(), phasedConfig(t, l.Addr().String(), "auto", failInit))
|
||||
if err == nil || errors.Is(err, lifecycle.ErrResourcesRetained) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
state := svc.Background.Load()
|
||||
if !state.Wrote.Load() || svc.Closed.Load() != before+1 {
|
||||
t.Fatal("failure cleanup did not join background first")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestGeneratedBackgroundTimeoutProcess(t *testing.T) {
|
||||
if path := os.Getenv("OMNI_PHASE_TIMEOUT_CONFIG"); path != "" {
|
||||
os.Args = []string{os.Args[0], "-config", path}
|
||||
main()
|
||||
return
|
||||
}
|
||||
path := phasedConfig(t, "127.0.0.1:0", "stuck", true)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestGeneratedBackgroundTimeoutProcess$")
|
||||
cmd.Env = append(os.Environ(), "OMNI_PHASE_TIMEOUT_CONFIG="+path)
|
||||
output, err := cmd.CombinedOutput()
|
||||
var exit *exec.ExitError
|
||||
if !errors.As(err, &exit) || exit.ExitCode() != 1 || ctx.Err() != nil {
|
||||
t.Fatal(err, string(output))
|
||||
}
|
||||
if strings.Contains(string(output), "test.client.closed") || !strings.Contains(string(output), "shared resources retained") || !strings.Contains(string(output), "client initialization failed") {
|
||||
t.Fatal(string(output))
|
||||
}
|
||||
}
|
||||
@@ -78,6 +78,12 @@ func PlanUpgrade(dir string) (UpgradePlan, error) {
|
||||
names[name] = true
|
||||
}
|
||||
}
|
||||
// v0.14 already owns config/svc extensions. Only bump the module here;
|
||||
// Generate updates generator-owned main/runtime files without replaying the
|
||||
// v0.13 seed migration over developer configuration and dependency code.
|
||||
if semver.IsValid(m.TemplateVersion) && semver.Compare(m.TemplateVersion, "v0.14.0-preview") >= 0 {
|
||||
names = map[string]bool{}
|
||||
}
|
||||
sorted := make([]string, 0, len(names))
|
||||
for name := range names {
|
||||
sorted = append(sorted, name)
|
||||
|
||||
@@ -60,6 +60,49 @@ func TestUpgradeRejectsEscapingEntry(t *testing.T) {
|
||||
t.Fatal("invalid upgrade wrote project")
|
||||
}
|
||||
}
|
||||
|
||||
func TestUpgradeV014PreservesExtensions(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "service")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var m manifest
|
||||
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.TemplateVersion = "v0.14.0-preview"
|
||||
raw, _ := json.Marshal(m)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
mod := filepath.Join(dir, "go.mod")
|
||||
writeFile(t, mod, strings.ReplaceAll(readFile(t, mod), minimumVersion, "v0.14.1-preview"))
|
||||
owned := map[string]string{}
|
||||
for _, name := range []string{"internal/svc/serviceContext.go", "internal/config/config.go", "etc/config.yaml"} {
|
||||
path := filepath.Join(dir, name)
|
||||
value := readFile(t, path)
|
||||
if strings.HasSuffix(name, ".go") {
|
||||
value += "\n// project extension kept\n"
|
||||
} else {
|
||||
value += "\n# project configuration kept\n"
|
||||
}
|
||||
writeFile(t, path, value)
|
||||
owned[path] = value
|
||||
}
|
||||
plan, err := PlanUpgrade(dir)
|
||||
if err != nil || len(plan.Changes) != 1 || plan.Changes[0].Path != "go.mod" {
|
||||
t.Fatal(plan, err)
|
||||
}
|
||||
if _, err := ApplyUpgrade(dir); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for path, want := range owned {
|
||||
if readFile(t, path) != want {
|
||||
t.Fatal("developer file modified", path)
|
||||
}
|
||||
}
|
||||
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestSafeInfrastructureUpgrade(t *testing.T) {
|
||||
dir := oldInfrastructureProject(t)
|
||||
before := snapshot(t, dir)
|
||||
|
||||
@@ -16,7 +16,8 @@ type WorkspaceService struct {
|
||||
Module string `json:"module,omitempty"`
|
||||
}
|
||||
type WorkspaceConfig struct {
|
||||
Services []WorkspaceService `json:"services"`
|
||||
Services []WorkspaceService `json:"services"`
|
||||
ErrorCatalogs []WorkspaceCatalog `json:"errorCatalogs,omitempty"`
|
||||
}
|
||||
|
||||
// Workspace operates independent Go modules listed in an explicit local file.
|
||||
@@ -83,6 +84,16 @@ func Workspace(path, command string, w io.Writer) error {
|
||||
}
|
||||
options = append(options, Options{API: entry, Dir: dir, Module: service.Module, DryRun: true})
|
||||
}
|
||||
var catalogs map[string][]byte
|
||||
if command != "format" {
|
||||
catalogs, err = prepareWorkspaceCatalogs(root, cfg.ErrorCatalogs, command == "check")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for i := range options {
|
||||
options[i].documentCatalogs = catalogs
|
||||
}
|
||||
}
|
||||
var failures error
|
||||
for _, o := range options {
|
||||
if command == "format" {
|
||||
@@ -101,6 +112,13 @@ func Workspace(path, command string, w io.Writer) error {
|
||||
if failures != nil {
|
||||
return failures
|
||||
}
|
||||
if command == "generate" {
|
||||
for path, data := range catalogs {
|
||||
if err = writeWorkspaceCatalog(path, data); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, o := range options {
|
||||
switch command {
|
||||
case "format":
|
||||
|
||||
217
cmd/go-omni/apigen/workspaceCatalog.go
Normal file
217
cmd/go-omni/apigen/workspaceCatalog.go
Normal file
@@ -0,0 +1,217 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
gotoken "go/token"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"golang.org/x/mod/modfile"
|
||||
"golang.org/x/mod/module"
|
||||
)
|
||||
|
||||
// WorkspaceCatalog is an explicitly trusted Go provider: func() errors.Catalog.
|
||||
// Only workspace generate/check execute providers. Parsing an .api never does.
|
||||
type WorkspaceCatalog struct {
|
||||
Dir string `json:"dir"`
|
||||
Package string `json:"package"`
|
||||
Function string `json:"function"`
|
||||
Output string `json:"output"`
|
||||
}
|
||||
|
||||
type limitedCatalogBuffer struct {
|
||||
bytes.Buffer
|
||||
limit int
|
||||
}
|
||||
|
||||
func (b *limitedCatalogBuffer) Write(p []byte) (int, error) {
|
||||
if len(p) > b.limit-b.Len() {
|
||||
return 0, errors.New("catalog provider output exceeds limit")
|
||||
}
|
||||
return b.Buffer.Write(p)
|
||||
}
|
||||
|
||||
func exportWorkspaceCatalog(root string, cfg WorkspaceCatalog) ([]byte, error) {
|
||||
if cfg.Dir == "" || cfg.Output == "" || !gotoken.IsIdentifier(cfg.Function) || !gotoken.IsExported(cfg.Function) || module.CheckImportPath(cfg.Package) != nil {
|
||||
return nil, errors.New("workspace catalog requires dir, an importable package, an exported function and output")
|
||||
}
|
||||
if err := safePath(root, filepath.ToSlash(filepath.Join(cfg.Dir, ".goOmniExportCheck"))); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := safePath(root, cfg.Output); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if filepath.Ext(cfg.Output) != ".json" {
|
||||
return nil, errors.New("workspace catalog output must be .json")
|
||||
}
|
||||
dir := filepath.Join(root, filepath.FromSlash(cfg.Dir))
|
||||
if stat, err := os.Stat(filepath.Join(dir, "go.mod")); err != nil || !stat.Mode().IsRegular() {
|
||||
return nil, errors.New("workspace catalog dir must contain go.mod")
|
||||
}
|
||||
temp, err := os.MkdirTemp("", "go-omni-catalog-*")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer os.RemoveAll(temp)
|
||||
// Overlay a virtual Go file under the provider module. This allows internal
|
||||
// packages without creating files or changing go.mod/go.sum in that module.
|
||||
virtual := filepath.Join(dir, ".goOmniCatalogExport", "main.go")
|
||||
if _, err := os.Lstat(filepath.Dir(virtual)); !errors.Is(err, os.ErrNotExist) {
|
||||
return nil, errors.New("reserved catalog export path already exists")
|
||||
}
|
||||
source := fmt.Sprintf("package main\nimport (\"os\"; provider %q; omnierrors %q)\nfunc main(){if err:=omnierrors.ExportCatalog(os.Stdout,provider.%s());err!=nil{os.Exit(1)}}\n", cfg.Package, frameworkModule+"/errors", cfg.Function)
|
||||
backing := filepath.Join(temp, "main.go")
|
||||
if err = os.WriteFile(backing, []byte(source), 0600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
overlay, _ := json.Marshal(map[string]any{"Replace": map[string]string{virtual: backing}})
|
||||
overlayPath := filepath.Join(temp, "overlay.json")
|
||||
if err = os.WriteFile(overlayPath, overlay, 0600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Keep all module metadata writes in the temporary directory as well.
|
||||
// -mod=readonly alone must not be relied on to protect every go.sum update.
|
||||
moduleData, err := os.ReadFile(filepath.Join(dir, "go.mod"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mf, err := modfile.Parse("go.mod", moduleData, nil)
|
||||
if err != nil {
|
||||
return nil, errors.New("workspace catalog has invalid go.mod")
|
||||
}
|
||||
replacements := append([]*modfile.Replace(nil), mf.Replace...)
|
||||
for _, r := range replacements {
|
||||
if r.New.Version == "" && !filepath.IsAbs(r.New.Path) {
|
||||
if err = mf.AddReplace(r.Old.Path, r.Old.Version, filepath.Join(dir, r.New.Path), ""); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
moduleData, err = mf.Format()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
modPath := filepath.Join(temp, "go.mod")
|
||||
if err = os.WriteFile(modPath, moduleData, 0600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if sums, e := os.ReadFile(filepath.Join(dir, "go.sum")); e == nil {
|
||||
if err = os.WriteFile(filepath.Join(temp, "go.sum"), sums, 0600); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
return nil, e
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
|
||||
executable := filepath.Join(temp, "catalogExport")
|
||||
// Build first, then execute the binary directly: cancellation kills the
|
||||
// provider itself instead of only terminating a go-run wrapper process.
|
||||
build := exec.CommandContext(ctx, "go", "build", "-mod=readonly", "-modfile", modPath, "-overlay", overlayPath, "-o", executable, virtual)
|
||||
build.Dir = dir
|
||||
build.Env = append(os.Environ(), "GOWORK=off")
|
||||
build.WaitDelay = 2 * time.Second
|
||||
build.Stdout = &limitedCatalogBuffer{limit: 64 << 10}
|
||||
build.Stderr = &limitedCatalogBuffer{limit: 64 << 10}
|
||||
if err = build.Run(); err != nil {
|
||||
return nil, fmt.Errorf("workspace catalog %s.%s: build failed (%v); check module dependencies and func() errors.Catalog signature", cfg.Package, cfg.Function, err)
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, executable)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
cmd.WaitDelay = 2 * time.Second
|
||||
stdout := &limitedCatalogBuffer{limit: 1 << 20}
|
||||
stderr := &limitedCatalogBuffer{limit: 64 << 10}
|
||||
cmd.Stdout = stdout
|
||||
cmd.Stderr = stderr
|
||||
if err = cmd.Run(); err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return nil, fmt.Errorf("workspace catalog %s: %w", cfg.Package, ctx.Err())
|
||||
}
|
||||
// Provider output can contain arbitrary application data. Do not echo it.
|
||||
return nil, fmt.Errorf("workspace catalog %s.%s: execution failed (%v); provider must return errors.Catalog without writing stdout", cfg.Package, cfg.Function, err)
|
||||
}
|
||||
var catalog apierrors.Catalog
|
||||
if err = apierrors.DecodeCatalogJSON(stdout.Bytes(), &catalog); err != nil {
|
||||
return nil, errors.New("workspace catalog provider did not emit a valid catalog")
|
||||
}
|
||||
var normalized bytes.Buffer
|
||||
if err = apierrors.ExportCatalog(&normalized, catalog); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return normalized.Bytes(), nil
|
||||
}
|
||||
|
||||
func prepareWorkspaceCatalogs(root string, configs []WorkspaceCatalog, check bool) (map[string][]byte, error) {
|
||||
if len(configs) > 16 {
|
||||
return nil, errors.New("workspace supports at most 16 catalog providers")
|
||||
}
|
||||
result := map[string][]byte{}
|
||||
for _, cfg := range configs {
|
||||
if err := safePath(root, cfg.Output); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
path := filepath.Join(root, filepath.FromSlash(cfg.Output))
|
||||
for prior := range result {
|
||||
if strings.EqualFold(prior, path) {
|
||||
return nil, errors.New("workspace catalog outputs overlap")
|
||||
}
|
||||
}
|
||||
data, err := exportWorkspaceCatalog(root, cfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if check {
|
||||
old, err := os.ReadFile(path)
|
||||
if err != nil || !bytes.Equal(old, data) {
|
||||
return nil, fmt.Errorf("workspace catalog drift: %s; run workspace generate", cfg.Output)
|
||||
}
|
||||
}
|
||||
result[path] = data
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func writeWorkspaceCatalog(path string, data []byte) error {
|
||||
mode := os.FileMode(0644)
|
||||
if stat, err := os.Lstat(path); err == nil {
|
||||
if !stat.Mode().IsRegular() {
|
||||
return errors.New("workspace catalog output must be a regular file")
|
||||
}
|
||||
mode = stat.Mode().Perm()
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
old, err := os.ReadFile(path)
|
||||
if err == nil && bytes.Equal(old, data) {
|
||||
return nil
|
||||
}
|
||||
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
f, err := os.CreateTemp(filepath.Dir(path), ".omni-catalog-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(f.Name())
|
||||
if err = f.Chmod(mode); err != nil {
|
||||
_ = f.Close()
|
||||
return err
|
||||
}
|
||||
_, writeErr := io.Copy(f, bytes.NewReader(data))
|
||||
err = errors.Join(writeErr, f.Close())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(f.Name(), path)
|
||||
}
|
||||
168
cmd/go-omni/apigen/workspaceCatalog_test.go
Normal file
168
cmd/go-omni/apigen/workspaceCatalog_test.go
Normal file
@@ -0,0 +1,168 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestWorkspaceSharedGoCatalog(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeFile(t, filepath.Join(root, "go.mod"), fmt.Sprintf("module example.com/catalogs\n\ngo 1.24.0\nrequire %s v0.15.0-preview\nreplace %s => %s\n", frameworkModule, frameworkModule, strconvQuote(frameworkRoot(t))))
|
||||
writeFile(t, filepath.Join(root, "internal/apierrors/catalog.go"), `package apierrors
|
||||
import errors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
func Catalog() errors.Catalog {return errors.Catalog{DefaultStatus:400,Definitions:map[string]errors.Definition{
|
||||
"INVALID":{Code:"INVALID",Message:"invalid input"},"MISSING":{Status:404,Code:"MISSING",Message:"missing"},
|
||||
}}}
|
||||
`)
|
||||
if err := os.MkdirAll(filepath.Join(root, "contracts"), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, name := range []string{"api", "id"} {
|
||||
dir := filepath.Join(root, "services", name)
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), `{"roots":{"common":{"path":"../../../contracts"}},"errors":{"source":"@common/errors.json","include":["INVALID"],"routes":{"GET /api/greet/:name":["MISSING"]}}}`)
|
||||
}
|
||||
workspace := filepath.Join(root, "omni.workspace.json")
|
||||
writeFile(t, workspace, `{"errorCatalogs":[{"dir":".","package":"example.com/catalogs/internal/apierrors","function":"Catalog","output":"contracts/errors.json"}],"services":[{"dir":"services/api"},{"dir":"services/id"}]}`)
|
||||
if err := Workspace(workspace, "generate", io.Discard); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Workspace(workspace, "format", io.Discard); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := Workspace(workspace, "generate", io.Discard); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := snapshot(t, root)
|
||||
if err := Workspace(workspace, "check", io.Discard); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("check wrote project files")
|
||||
}
|
||||
for _, name := range []string{"api", "id"} {
|
||||
entry := filepath.Join(root, "services", name, "api/service.api")
|
||||
spec, err := Parse(entry)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
op := operation(t, document(t, c), "/api/greet/{name}", "get")
|
||||
if op["responses"].(map[string]any)["404"] == nil || len(spec.Errors) != 0 || len(spec.Routes[0].Errors) != 0 {
|
||||
t.Fatal("selection changed runtime catalog")
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(filepath.Dir(entry), catalogDocumentFile)); !os.IsNotExist(err) {
|
||||
t.Fatal("shared catalog was copied")
|
||||
}
|
||||
}
|
||||
shared := filepath.Join(root, "contracts/errors.json")
|
||||
data := readFile(t, shared)
|
||||
writeFile(t, shared, strings.Replace(data, "invalid input", "stale", 1))
|
||||
before = snapshot(t, root)
|
||||
if err := Workspace(workspace, "check", io.Discard); err == nil || !strings.Contains(err.Error(), "catalog drift") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("drift check wrote files")
|
||||
}
|
||||
if err := Workspace(workspace, "generate", io.Discard); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got := readFile(t, shared); got != data {
|
||||
t.Fatal("catalog not refreshed")
|
||||
}
|
||||
// Failed provider/preflight may not update even the shared metadata.
|
||||
source := filepath.Join(root, "internal/apierrors/catalog.go")
|
||||
writeFile(t, source, readFile(t, source)+"\ninvalid go source")
|
||||
before = snapshot(t, root)
|
||||
if err := Workspace(workspace, "generate", io.Discard); err == nil {
|
||||
t.Fatal("provider failure ignored")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("provider failure mutated project")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDocumentSelectionBoundaries(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
local := filepath.Join(root, "api")
|
||||
shared := filepath.Join(root, "shared")
|
||||
entry := filepath.Join(local, "service.api")
|
||||
writeFile(t, entry, "service demo {\n@handler list\nget /list returns ([]string)\n}\n")
|
||||
writeFile(t, filepath.Join(shared, "errors.json"), `{"defaultStatus":400,"definitions":{"INVALID":{"code":400002,"message":"invalid"}}}`)
|
||||
for _, cfg := range []string{
|
||||
`{"source":"@common/errors.json","include":["UNKNOWN"]}`,
|
||||
`{"source":"@common/errors.json","routes":{"GET /absent":["INVALID"]}}`,
|
||||
`{"source":"@common/../api/service.api"}`,
|
||||
`{"source":"../shared/errors.json"}`,
|
||||
} {
|
||||
writeFile(t, filepath.Join(local, importsFile), `{"roots":{"common":{"path":"../shared"}},"errors":`+cfg+`}`)
|
||||
if _, err := Parse(entry); err == nil {
|
||||
t.Fatal("bad selection accepted", cfg)
|
||||
}
|
||||
}
|
||||
writeFile(t, filepath.Join(local, importsFile), `{"roots":{"common":{"path":"../shared"}},"errors":{"source":"@common/errors.json","include":[]}}`)
|
||||
spec, err := Parse(entry)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(documentRouteCatalog(spec, spec.Routes[0]).Definitions) != 0 {
|
||||
t.Fatal("empty include ignored")
|
||||
}
|
||||
seeded, err := sharedSeeds(spec, filepath.Join(root, "generated"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var cfg sharedConfig
|
||||
if err = json.Unmarshal(seeded, &cfg); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Errors.Include == nil || !bytes.Contains(seeded, []byte(`"include": []`)) {
|
||||
t.Fatal("empty selection lost during relocation")
|
||||
}
|
||||
}
|
||||
|
||||
func TestWorkspaceCatalogPathsAndProviderValidation(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeFile(t, filepath.Join(root, "go.mod"), "module example.com/test\ngo 1.24.0\n")
|
||||
good := WorkspaceCatalog{Dir: ".", Package: "example.com/test/internal/errors", Function: "Catalog", Output: "errors.json"}
|
||||
cases := []WorkspaceCatalog{}
|
||||
for _, change := range []func(*WorkspaceCatalog){
|
||||
func(c *WorkspaceCatalog) { c.Dir = "../outside" },
|
||||
func(c *WorkspaceCatalog) { c.Output = "../outside.json" },
|
||||
func(c *WorkspaceCatalog) { c.Package = "example.com/pkg;bad" },
|
||||
func(c *WorkspaceCatalog) { c.Function = "Catalog()" },
|
||||
func(c *WorkspaceCatalog) { c.Output = "go.mod" },
|
||||
} {
|
||||
c := good
|
||||
change(&c)
|
||||
cases = append(cases, c)
|
||||
}
|
||||
for _, c := range cases {
|
||||
if _, err := exportWorkspaceCatalog(root, c); err == nil {
|
||||
t.Fatal("bad provider accepted", c)
|
||||
}
|
||||
}
|
||||
outside := filepath.Join(t.TempDir(), "errors.json")
|
||||
writeFile(t, outside, `{}`)
|
||||
if err := os.Symlink(outside, filepath.Join(root, "errors.json")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := prepareWorkspaceCatalogs(root, []WorkspaceCatalog{good}, false); err == nil {
|
||||
t.Fatal("symlink output accepted")
|
||||
}
|
||||
if readFile(t, outside) != `{}` {
|
||||
t.Fatal("outside file modified")
|
||||
}
|
||||
}
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
const manifestPath = ".goOmni/apiManifest.json"
|
||||
|
||||
type manifest struct {
|
||||
GeneratorConfig bool `json:"generatorConfig,omitempty"`
|
||||
TemplateVersion string `json:"templateVersion,omitempty"`
|
||||
Profile *projectProfile `json:"profile,omitempty"`
|
||||
Version int `json:"version"`
|
||||
@@ -109,13 +110,24 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
next := manifest{TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
|
||||
next := manifest{GeneratorConfig: true, TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
|
||||
for name, data := range generated {
|
||||
next.Generated[name] = digest(data)
|
||||
}
|
||||
var changes []change
|
||||
// Check ALL previous outputs, even stale ones, before writing any new file.
|
||||
for name, hash := range old.Generated {
|
||||
// Migration guard only; old mappings are never regenerated or silently
|
||||
// deleted. Once the project removes them, forget the stale ownership.
|
||||
if name == "internal/mapping/mappingsGen.go" {
|
||||
if err = safePath(root, name); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if _, e := os.Lstat(filepath.Join(root, name)); !errors.Is(e, os.ErrNotExist) {
|
||||
return result, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go and remove internal/mapping/mappingsGen.go before generating; see COPY.md")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !ownedPath(name) {
|
||||
return result, fmt.Errorf("api: invalid owned path %q", name)
|
||||
}
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
)
|
||||
|
||||
func TestNewWithRemoteFrameworkVersion(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--framework-version", "v0.14.0-preview"}, {"--framework-version=v0.14.0-preview"}} {
|
||||
for _, flag := range [][]string{{"--framework-version", "v0.19.0-preview"}, {"--framework-version=v0.19.0-preview"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
args := append([]string{"--module", "example.com/billing"}, flag...)
|
||||
if err := runNew(append(args, project)); err != nil {
|
||||
@@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(content), "go-omni v0.14.0-preview") || strings.Contains(string(content), "replace ") {
|
||||
if !strings.Contains(string(content), "go-omni v0.19.0-preview") || strings.Contains(string(content), "replace ") {
|
||||
t.Fatalf("CLI did not pin remote dependency: %s", content)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package settings
|
||||
import (
|
||||
"os"
|
||||
"strings"
|
||||
{{if and (not .API) (ne .Storage "none")}}"strings"{{end}}
|
||||
"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
@@ -41,10 +41,10 @@ func Load(path string)(Config,error){
|
||||
cfg:=Default();var err error
|
||||
cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err}
|
||||
if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}}
|
||||
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_"});if err!=nil{return Config{},err}
|
||||
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
|
||||
cfg.Server,err=config.ApplyEnv(cfg.Server,"");if err!=nil{return Config{},err}
|
||||
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
|
||||
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_",Empty:"ignore",Aliases:config.ServerEnvAliases("GO_OMNI_","server")});if err!=nil{return Config{},err}
|
||||
output,outputSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_OUTPUT");if !outputSet {output=os.Getenv("GO_OMNI_LOG_OUTPUT")}
|
||||
file,fileSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_FILE");if !fileSet {file=os.Getenv("GO_OMNI_LOG_FILE")}
|
||||
if output!="" && file=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
|
||||
return cfg,cfg.Validate()
|
||||
}
|
||||
func(c Config)Validate()error{
|
||||
|
||||
@@ -28,6 +28,8 @@ type Context struct {
|
||||
{{end}}
|
||||
}
|
||||
// Init is project-owned. Add dependencies here and immediately register each
|
||||
// cleanup with c.OnClose. Returning an error automatically rolls everything back.
|
||||
// cleanup with c.OnClose. Background components use paired OnStop/OnDrain hooks
|
||||
// and WorkContext. Failed initialization stops/drains before releasing resources;
|
||||
// unjoined background work returns ErrResourcesRetained instead of closing its dependencies.
|
||||
// Generated infrastructure is ready here. Never create another App or logger.
|
||||
func(c *Context)Init(ctx context.Context)error {return nil}
|
||||
|
||||
@@ -18,7 +18,8 @@ func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(res
|
||||
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||
if err=cfg.Validate();err!=nil{return nil,err}
|
||||
checker,err:=health.New();if err!=nil{return nil,err}
|
||||
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Runtime:lifecycle.NewRuntime(ctx)}
|
||||
runtime,err:=lifecycle.NewRuntimeWithConfig(logx.WithLogger(ctx,logger),cfg.Server.Lifecycle);if err!=nil{return nil,err}
|
||||
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Runtime:runtime}
|
||||
ctx=logx.WithLogger(c.Context(),logger)
|
||||
defer func(){if p:=recover();p!=nil{_ = c.Close();panic(p)};if err!=nil{err=c.Finish(err)}}()
|
||||
{{if eq .Database "mysql"}}
|
||||
|
||||
@@ -4,7 +4,11 @@ server:
|
||||
enableLogger: true
|
||||
slowRequestThreshold: 500ms
|
||||
shutdownTimeout: 10s
|
||||
{{if .API}} endpoints:
|
||||
{{if .API}} lifecycle:
|
||||
stopTimeout: 5s
|
||||
drainTimeout: 10s
|
||||
cancelTimeout: 5s
|
||||
endpoints:
|
||||
liveness:
|
||||
enabled: true
|
||||
path: /health/live
|
||||
|
||||
@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||
// downloaded module version, which also has a module checksum.
|
||||
const defaultFrameworkVersion = "v0.14.0-preview"
|
||||
const defaultFrameworkVersion = "v0.19.0-preview"
|
||||
|
||||
// This target is not a claim that the release already exists. Source users
|
||||
// select a local framework checkout until the module templates are published.
|
||||
|
||||
66
config/aliases_test.go
Normal file
66
config/aliases_test.go
Normal file
@@ -0,0 +1,66 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestUnifiedEnvAliases(t *testing.T) {
|
||||
type project struct {
|
||||
Server Config `json:"server"`
|
||||
Label string `json:"label"`
|
||||
}
|
||||
base := project{Server: Default(), Label: "base"}
|
||||
for _, policy := range []string{"allow", "ignore", "reject"} {
|
||||
t.Run(policy, func(t *testing.T) {
|
||||
env := map[string]string{"TEST_HTTP_ADDR": "127.0.0.1:8099", "TEST_LOG_SERVICE": "", "TEST_SERVICE": "", "TEST_LABEL": ""}
|
||||
opts := EnvOptions{Prefix: "TEST_", Empty: policy, Aliases: ServerEnvAliases("TEST_", "server"), Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
|
||||
got, err := OverlayEnv(base, opts)
|
||||
if policy == "reject" {
|
||||
if err == nil || !strings.Contains(err.Error(), "empty") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Server.Address != "127.0.0.1:8099" {
|
||||
t.Fatal("alias ignored")
|
||||
}
|
||||
if policy == "allow" && got.Label != "" || policy == "ignore" && got.Label != "base" {
|
||||
t.Fatal("empty policy differs")
|
||||
}
|
||||
env["TEST_SERVER_ADDRESS"] = "127.0.0.1:9001"
|
||||
env["TEST_HTTP_READ_TIMEOUT"] = "3s"
|
||||
env["TEST_ENABLE_LOGGER"] = "false"
|
||||
env["TEST_TRUSTED_PROXIES"] = `["127.0.0.1"]`
|
||||
got, err = OverlayEnv(base, opts)
|
||||
if err != nil || got.Server.Address != "127.0.0.1:9001" || got.Server.HTTP.ReadTimeout != 3*time.Second || got.Server.EnableLogger || len(got.Server.TrustedProxies) != 1 {
|
||||
t.Fatal(got, err)
|
||||
}
|
||||
env["TEST_SERVER_ADDRESS"] = ""
|
||||
got, err = OverlayEnv(base, opts)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
expected := ""
|
||||
if policy == "ignore" {
|
||||
expected = base.Server.Address
|
||||
}
|
||||
if got.Server.Address != expected {
|
||||
t.Fatal("empty canonical unexpectedly fell through to alias")
|
||||
}
|
||||
})
|
||||
}
|
||||
if base.Server.Address != ":8080" || base.Label != "base" {
|
||||
t.Fatal("defaults mutated")
|
||||
}
|
||||
}
|
||||
func TestAliasTypeErrorsDoNotExposeValue(t *testing.T) {
|
||||
opts := EnvOptions{Prefix: "X_", Empty: "reject", Aliases: ServerEnvAliases("X_", ""), Lookup: func(k string) (string, bool) { return "private-invalid", k == "X_HTTP_READ_TIMEOUT" }}
|
||||
if _, err := OverlayEnv(Default(), opts); err == nil || strings.Contains(err.Error(), "private-invalid") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
@@ -27,22 +28,23 @@ type TLSConfig struct {
|
||||
|
||||
// Config controls the default runtime behavior of an App.
|
||||
type Config struct {
|
||||
Request contract.RequestPolicy `json:"request"`
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
HTTP HTTPConfig `json:"http"`
|
||||
Endpoints EndpointsConfig `json:"endpoints"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Log logx.Config `json:"log"`
|
||||
Errors apierrors.Catalog `json:"errors"`
|
||||
Response response.Config `json:"response"`
|
||||
ResponseEncoder response.Encoder `json:"-"`
|
||||
LogInstance *logx.Logger `json:"-"`
|
||||
Request contract.RequestPolicy `json:"request"`
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
HTTP HTTPConfig `json:"http"`
|
||||
Endpoints EndpointsConfig `json:"endpoints"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Lifecycle lifecycle.ShutdownConfig `json:"lifecycle"`
|
||||
Log logx.Config `json:"log"`
|
||||
Errors apierrors.Catalog `json:"errors"`
|
||||
Response response.Config `json:"response"`
|
||||
ResponseEncoder response.Encoder `json:"-"`
|
||||
LogInstance *logx.Logger `json:"-"`
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
@@ -51,6 +53,7 @@ func Default() Config {
|
||||
Transport: TransportNetHTTP,
|
||||
RequestIDHeader: defaultRequestIDHeader,
|
||||
ShutdownTimeout: 10 * time.Second,
|
||||
Lifecycle: lifecycle.DefaultShutdownConfig(),
|
||||
Endpoints: DefaultEndpoints(),
|
||||
Log: logx.Default(),
|
||||
}
|
||||
@@ -58,6 +61,9 @@ func Default() Config {
|
||||
|
||||
// Validate checks configuration values before an application starts.
|
||||
func (c Config) Validate() error {
|
||||
if err := c.Lifecycle.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Request.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
168
config/env.go
168
config/env.go
@@ -1,135 +1,83 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
const defaultEnvPrefix = "GO_OMNI_"
|
||||
|
||||
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
|
||||
// database settings remain owned by database.ConfigFromEnv.
|
||||
func LoadFromEnv(prefix string) (Config, error) {
|
||||
return applyEnv(Default(), prefix)
|
||||
// ServerEnvAliases returns the documented short names for Config fields.
|
||||
// field is the JSON path of Config in a project (usually "server"), or ""
|
||||
// for Config itself. Canonical names win over aliases. Both use the same
|
||||
// EnvOptions.Empty/Lookup and typed parser; collections use JSON syntax.
|
||||
func ServerEnvAliases(prefix, field string) map[string][]string {
|
||||
root := prefix
|
||||
if field != "" {
|
||||
for _, part := range strings.Split(field, ".") {
|
||||
root += envName(part) + "_"
|
||||
}
|
||||
}
|
||||
names := map[string]string{
|
||||
"ADDRESS": "HTTP_ADDR", "TRANSPORT": "TRANSPORT", "TLS_CERT_FILE": "TLS_CERT_FILE", "TLS_KEY_FILE": "TLS_KEY_FILE",
|
||||
"REQUEST_ID_HEADER": "REQUEST_ID_HEADER", "ENABLE_LOGGER": "ENABLE_LOGGER", "SLOW_REQUEST_THRESHOLD": "SLOW_REQUEST_THRESHOLD",
|
||||
"TRUSTED_PROXIES": "TRUSTED_PROXIES", "SHUTDOWN_TIMEOUT": "SHUTDOWN_TIMEOUT",
|
||||
"LOG_LEVEL": "LOG_LEVEL", "LOG_FORMAT": "LOG_FORMAT", "LOG_OUTPUT": "LOG_OUTPUT", "LOG_FILE": "LOG_FILE", "LOG_SERVICE": "SERVICE", "LOG_ENVIRONMENT": "ENVIRONMENT", "LOG_ADD_SOURCE": "LOG_ADD_SOURCE",
|
||||
"HTTP_READ_HEADER_TIMEOUT": "HTTP_READ_HEADER_TIMEOUT", "HTTP_READ_TIMEOUT": "HTTP_READ_TIMEOUT", "HTTP_WRITE_TIMEOUT": "HTTP_WRITE_TIMEOUT", "HTTP_IDLE_TIMEOUT": "HTTP_IDLE_TIMEOUT", "HTTP_MAX_HEADER_BYTES": "HTTP_MAX_HEADER_BYTES",
|
||||
"LIFECYCLE_STOP_TIMEOUT": "LIFECYCLE_STOP_TIMEOUT", "LIFECYCLE_DRAIN_TIMEOUT": "LIFECYCLE_DRAIN_TIMEOUT", "LIFECYCLE_CANCEL_TIMEOUT": "LIFECYCLE_CANCEL_TIMEOUT",
|
||||
}
|
||||
for _, endpoint := range []string{"LIVENESS", "READINESS", "OPENAPI"} {
|
||||
for _, part := range []string{"ENABLED", "PATH", "RESPONSE_MODE"} {
|
||||
if endpoint == "OPENAPI" && part == "RESPONSE_MODE" {
|
||||
continue
|
||||
}
|
||||
name := "ENDPOINTS_" + endpoint + "_" + part
|
||||
names[name] = name
|
||||
}
|
||||
}
|
||||
aliases := map[string][]string{}
|
||||
for key, alias := range names {
|
||||
if root+key != prefix+alias {
|
||||
aliases[root+key] = []string{prefix + alias}
|
||||
}
|
||||
}
|
||||
return aliases
|
||||
}
|
||||
|
||||
// ApplyEnv overlays nonempty environment values on a caller's defaults or file
|
||||
// settings, then validates the result. Caller-owned slices are not retained.
|
||||
// LoadFromEnv loads defaults using the legacy short-name conventions.
|
||||
func LoadFromEnv(prefix string) (Config, error) { return ApplyEnv(Default(), prefix) }
|
||||
|
||||
// ApplyEnv retains legacy whitespace trimming, ignored empty variables and
|
||||
// comma-separated TRUSTED_PROXIES. New projects should use OverlayEnv with
|
||||
// ServerEnvAliases for one explicit empty-value policy across the whole config.
|
||||
func ApplyEnv(defaults Config, prefix string) (Config, error) {
|
||||
defaults.Errors = defaults.Errors.Clone()
|
||||
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
|
||||
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
|
||||
return applyEnv(defaults, prefix)
|
||||
}
|
||||
|
||||
func applyEnv(cfg Config, prefix string) (Config, error) {
|
||||
prefix = strings.TrimSpace(prefix)
|
||||
if prefix == "" {
|
||||
prefix = defaultEnvPrefix
|
||||
}
|
||||
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
|
||||
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
|
||||
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
|
||||
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
|
||||
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
|
||||
cfg.TrustedProxies = strings.Split(value, ",")
|
||||
for i := range cfg.TrustedProxies {
|
||||
cfg.TrustedProxies[i] = strings.TrimSpace(cfg.TrustedProxies[i])
|
||||
}
|
||||
}
|
||||
cfg.Log.Level = envString(prefix+"LOG_LEVEL", cfg.Log.Level)
|
||||
cfg.Log.Format = envString(prefix+"LOG_FORMAT", cfg.Log.Format)
|
||||
cfg.Log.Output = envString(prefix+"LOG_OUTPUT", cfg.Log.Output)
|
||||
cfg.Log.File = envString(prefix+"LOG_FILE", cfg.Log.File)
|
||||
cfg.Log.Service = envString(prefix+"SERVICE", cfg.Log.Service)
|
||||
cfg.Log.Environment = envString(prefix+"ENVIRONMENT", cfg.Log.Environment)
|
||||
|
||||
var err error
|
||||
for _, endpoint := range []struct {
|
||||
name string
|
||||
enabled *bool
|
||||
path *string
|
||||
mode *string
|
||||
}{
|
||||
{"LIVENESS", &cfg.Endpoints.Liveness.Enabled, &cfg.Endpoints.Liveness.Path, &cfg.Endpoints.Liveness.ResponseMode},
|
||||
{"READINESS", &cfg.Endpoints.Readiness.Enabled, &cfg.Endpoints.Readiness.Path, &cfg.Endpoints.Readiness.ResponseMode},
|
||||
{"OPENAPI", &cfg.Endpoints.OpenAPI.Enabled, &cfg.Endpoints.OpenAPI.Path, nil},
|
||||
} {
|
||||
key := prefix + "ENDPOINTS_" + endpoint.name
|
||||
*endpoint.enabled, err = envBool(key+"_ENABLED", *endpoint.enabled)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
*endpoint.path = envString(key+"_PATH", *endpoint.path)
|
||||
if endpoint.mode != nil {
|
||||
*endpoint.mode = envString(key+"_RESPONSE_MODE", *endpoint.mode)
|
||||
}
|
||||
}
|
||||
for _, field := range []struct {
|
||||
name string
|
||||
value *time.Duration
|
||||
}{
|
||||
{"HTTP_READ_HEADER_TIMEOUT", &cfg.HTTP.ReadHeaderTimeout},
|
||||
{"HTTP_READ_TIMEOUT", &cfg.HTTP.ReadTimeout},
|
||||
{"HTTP_WRITE_TIMEOUT", &cfg.HTTP.WriteTimeout},
|
||||
{"HTTP_IDLE_TIMEOUT", &cfg.HTTP.IdleTimeout},
|
||||
} {
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + field.name)); value != "" {
|
||||
*field.value, err = time.ParseDuration(value)
|
||||
if err != nil {
|
||||
return Config{}, fmt.Errorf("%s%s must be a duration: %w", prefix, field.name, err)
|
||||
lookup := func(key string) (string, bool) {
|
||||
raw, ok := os.LookupEnv(key)
|
||||
raw = strings.TrimSpace(raw)
|
||||
if ok && raw != "" && key == prefix+"TRUSTED_PROXIES" && !strings.HasPrefix(raw, "[") {
|
||||
values := strings.Split(raw, ",")
|
||||
for i := range values {
|
||||
values[i] = strings.TrimSpace(values[i])
|
||||
}
|
||||
data, _ := json.Marshal(values)
|
||||
raw = string(data)
|
||||
}
|
||||
return raw, ok
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "HTTP_MAX_HEADER_BYTES")); value != "" {
|
||||
cfg.HTTP.MaxHeaderBytes, err = strconv.Atoi(value)
|
||||
if err != nil {
|
||||
return Config{}, fmt.Errorf("%sHTTP_MAX_HEADER_BYTES must be an integer: %w", prefix, err)
|
||||
}
|
||||
}
|
||||
if cfg.EnableLogger, err = envBool(prefix+"ENABLE_LOGGER", cfg.EnableLogger); err != nil {
|
||||
defaults.Errors = defaults.Errors.Clone()
|
||||
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
|
||||
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
|
||||
cfg, err := OverlayEnv(defaults, EnvOptions{Prefix: prefix, Empty: "ignore", Lookup: lookup, Aliases: ServerEnvAliases(prefix, "")})
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "SLOW_REQUEST_THRESHOLD")); value != "" {
|
||||
cfg.SlowRequestThreshold, err = time.ParseDuration(value)
|
||||
if err != nil {
|
||||
return Config{}, fmt.Errorf("%sSLOW_REQUEST_THRESHOLD must be a duration: %w", prefix, err)
|
||||
}
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
|
||||
cfg.ShutdownTimeout, err = time.ParseDuration(value)
|
||||
if err != nil {
|
||||
return Config{}, fmt.Errorf("%sSHUTDOWN_TIMEOUT must be a duration: %w", prefix, err)
|
||||
}
|
||||
}
|
||||
if err := cfg.Validate(); err != nil {
|
||||
if err = cfg.Validate(); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
|
||||
func envString(key, fallback string) string {
|
||||
if value := strings.TrimSpace(os.Getenv(key)); value != "" {
|
||||
return value
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func envBool(key string, fallback bool) (bool, error) {
|
||||
value := strings.TrimSpace(os.Getenv(key))
|
||||
if value == "" {
|
||||
return fallback, nil
|
||||
}
|
||||
parsed, err := strconv.ParseBool(value)
|
||||
if err != nil {
|
||||
return false, fmt.Errorf("%s must be a boolean: %w", key, err)
|
||||
}
|
||||
return parsed, nil
|
||||
}
|
||||
|
||||
@@ -159,7 +159,7 @@ func LoadFile(path, prefix string) (Config, error) {
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return applyEnv(cfg, prefix)
|
||||
return ApplyEnv(cfg, prefix)
|
||||
}
|
||||
|
||||
// Reloader only accepts log.level changes. Static changes and invalid files
|
||||
|
||||
37
config/lifecycle_test.go
Normal file
37
config/lifecycle_test.go
Normal file
@@ -0,0 +1,37 @@
|
||||
package config_test
|
||||
|
||||
import (
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestLifecycleConfiguration(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||
if err := os.WriteFile(path, []byte("lifecycle:\n stopTimeout: 2s\n drainTimeout: 3s\n cancelTimeout: 4s\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.LoadFile(path, "PHASE_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Lifecycle != (lifecycle.ShutdownConfig{StopTimeout: 2 * time.Second, DrainTimeout: 3 * time.Second, CancelTimeout: 4 * time.Second}) {
|
||||
t.Fatal(cfg.Lifecycle)
|
||||
}
|
||||
for _, key := range []string{"STOP_TIMEOUT", "DRAIN_TIMEOUT", "CANCEL_TIMEOUT"} {
|
||||
t.Setenv("PHASE_TEST_LIFECYCLE_"+key, "6s")
|
||||
}
|
||||
cfg, err = config.ApplyEnv(cfg, "PHASE_TEST_")
|
||||
if err != nil || cfg.Lifecycle.StopTimeout != 6*time.Second || cfg.Lifecycle.DrainTimeout != 6*time.Second || cfg.Lifecycle.CancelTimeout != 6*time.Second {
|
||||
t.Fatal(cfg.Lifecycle, err)
|
||||
}
|
||||
for _, v := range []string{"bad", "-1s"} {
|
||||
t.Setenv("PHASE_TEST_LIFECYCLE_DRAIN_TIMEOUT", v)
|
||||
if _, err := config.ApplyEnv(cfg, "PHASE_TEST_"); err == nil {
|
||||
t.Fatal("invalid duration accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -23,6 +23,9 @@ type EnvOptions struct {
|
||||
Prefix string
|
||||
Empty string
|
||||
Lookup func(string) (string, bool)
|
||||
// Aliases maps canonical absolute environment names to ordered fallback names.
|
||||
// A present canonical value wins, including an explicit empty value.
|
||||
Aliases map[string][]string
|
||||
}
|
||||
|
||||
// ApplyDefaults applies envDefault tags to zero fields before file overlays.
|
||||
@@ -162,13 +165,22 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
|
||||
}
|
||||
seen[key] = true
|
||||
raw, present := options.Lookup(key)
|
||||
selectedKey := key
|
||||
if !present && !defaults {
|
||||
for _, alias := range options.Aliases[key] {
|
||||
if raw, present = options.Lookup(alias); present {
|
||||
selectedKey = alias
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if defaults {
|
||||
raw, present = field.Tag.Lookup("envDefault")
|
||||
present = present && dst.IsZero()
|
||||
}
|
||||
if present && raw == "" && !defaults {
|
||||
if empty == "reject" {
|
||||
return fmt.Errorf("config: %s must not be empty", key)
|
||||
return fmt.Errorf("config: %s must not be empty", selectedKey)
|
||||
}
|
||||
if empty == "ignore" {
|
||||
present = false
|
||||
@@ -177,7 +189,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
|
||||
if present {
|
||||
parsed := reflect.New(dst.Type()).Elem()
|
||||
if err := parseEnvValue(parsed, raw); err != nil {
|
||||
return fmt.Errorf("config: %s must be a valid %s", key, dst.Type())
|
||||
return fmt.Errorf("config: %s must be a valid %s", selectedKey, dst.Type())
|
||||
}
|
||||
dst.Set(parsed)
|
||||
}
|
||||
|
||||
117
copyx/benchmark_test.go
Normal file
117
copyx/benchmark_test.go
Normal file
@@ -0,0 +1,117 @@
|
||||
package copyx_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
)
|
||||
|
||||
func BenchmarkScenarios(b *testing.B) {
|
||||
type request struct {
|
||||
Name string
|
||||
Count int64
|
||||
Active bool
|
||||
}
|
||||
type result struct {
|
||||
Label string
|
||||
Count int64
|
||||
Active bool
|
||||
}
|
||||
source := request{"sample", 9007199254740993, true}
|
||||
option := copyx.WithFieldNames[request, result](map[string]string{"Name": "Label"})
|
||||
copier, err := copyx.New(option)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
for _, reuse := range []bool{false, true} {
|
||||
b.Run(fmt.Sprintf("struct/reuse%t", reuse), func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var target result
|
||||
var err error
|
||||
if reuse {
|
||||
err = copier.Copy(&target, source)
|
||||
} else {
|
||||
err = copyx.Copy(&target, source, option)
|
||||
}
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
deep, _ := copyx.New(copyx.WithDeepCopy())
|
||||
b.Run("bytes64KiB", func(b *testing.B) {
|
||||
src := make([]byte, 64<<10)
|
||||
b.SetBytes(int64(len(src)))
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst []byte
|
||||
if err := deep.Copy(&dst, src); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
src := map[string][]int{}
|
||||
for i := 0; i < 100; i++ {
|
||||
src[fmt.Sprint(i)] = []int{1, 2, 3}
|
||||
}
|
||||
b.Run("map100", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst map[string][]int
|
||||
if err := deep.Copy(&dst, src); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
cycle := &detail{Values: []int{1, 2, 3}}
|
||||
cycle.Next = cycle
|
||||
b.Run("cycle", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst *detail
|
||||
if err := deep.Copy(&dst, cycle); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
b.Run("parallelCycle", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
b.RunParallel(func(pb *testing.PB) {
|
||||
for pb.Next() {
|
||||
var dst *detail
|
||||
if err := deep.Copy(&dst, cycle); err != nil {
|
||||
b.Error(err)
|
||||
return
|
||||
}
|
||||
}
|
||||
})
|
||||
})
|
||||
for _, fresh := range []bool{false, true} {
|
||||
b.Run(fmt.Sprintf("converter/fresh%t", fresh), func(b *testing.B) {
|
||||
c, err := copyx.New(copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) ([]int, error) {
|
||||
if fresh {
|
||||
return append([]int(nil), v...), nil
|
||||
}
|
||||
return v, nil
|
||||
}))
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
src := []int{1, 2, 3}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst []int
|
||||
err := c.Copy(&dst, src)
|
||||
if fresh && err != nil || !fresh && !errors.Is(err, copyx.ErrSharedReference) {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
537
copyx/copy.go
Normal file
537
copyx/copy.go
Normal file
@@ -0,0 +1,537 @@
|
||||
package copyx
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"runtime"
|
||||
"sort"
|
||||
"sync"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrType = errors.New("copyx: incompatible types")
|
||||
ErrNil = errors.New("copyx: nil cannot fill a non-nullable target")
|
||||
ErrField = errors.New("copyx: unmatched field")
|
||||
ErrUnsupported = errors.New("copyx: unsupported value; provide an explicit data converter")
|
||||
ErrSharedReference = errors.New("copyx: deep copy retains a mutable source reference")
|
||||
ErrLimit = errors.New("copyx: traversal limit exceeded")
|
||||
)
|
||||
|
||||
// Error describes a field/index path without recording the source value.
|
||||
type Error struct {
|
||||
Path string
|
||||
Cause error
|
||||
}
|
||||
|
||||
func (e *Error) Error() string { return e.Path + ": " + e.Cause.Error() }
|
||||
func (e *Error) Unwrap() error { return e.Cause }
|
||||
func at(path string, err error) error { return &Error{Path: path, Cause: err} }
|
||||
|
||||
func Copy(target, source any, options ...Option) error {
|
||||
c, err := New(options...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Copy(target, source)
|
||||
}
|
||||
|
||||
// Clone always makes a deep copy. Nil and empty collections stay distinct.
|
||||
func Clone[T any](source T, options ...Option) (T, error) {
|
||||
var target T
|
||||
options = append(append([]Option(nil), options...), WithDeepCopy())
|
||||
err := Copy(&target, source, options...)
|
||||
return target, err
|
||||
}
|
||||
|
||||
// Copy stages the complete result and only assigns target on success. User
|
||||
// converter side effects (including mutation of source) cannot be rolled back.
|
||||
// Caller must synchronize concurrent access to source and target themselves.
|
||||
func (c *Copier) Copy(target, source any) error {
|
||||
// Alias indexes contain addresses only. Keep the original graph alive until
|
||||
// validation and commit finish so the GC cannot recycle any indexed address.
|
||||
defer func() { runtime.KeepAlive(source) }()
|
||||
if c == nil {
|
||||
return errors.New("copyx: nil Copier")
|
||||
}
|
||||
dst := reflect.ValueOf(target)
|
||||
if !dst.IsValid() || dst.Kind() != reflect.Pointer || dst.IsNil() {
|
||||
return errors.New("copyx: target must be a non-nil pointer")
|
||||
}
|
||||
src := reflect.ValueOf(source)
|
||||
s := state{options: &c.options, seen: map[visit]reflect.Value{}}
|
||||
if c.options.deep {
|
||||
var err error
|
||||
s.graph, err = sourceGraph(src)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if src.IsValid() && src.Kind() == reflect.Pointer && !src.IsNil() && dst.Elem().Kind() == reflect.Struct {
|
||||
id := identity(src)
|
||||
s.valueRoot = &id
|
||||
}
|
||||
}
|
||||
out, err := s.walk(src, dst.Elem(), dst.Elem().Type(), "$", 0)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if s.graph != nil {
|
||||
if err = s.graph.audit(out); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
dst.Elem().Set(out)
|
||||
return nil
|
||||
}
|
||||
|
||||
type visit struct {
|
||||
reference
|
||||
target reflect.Type
|
||||
}
|
||||
type state struct {
|
||||
options *settings
|
||||
seen map[visit]reflect.Value
|
||||
graph *graph
|
||||
nodes int
|
||||
allocated uint64
|
||||
valueRoot *reference
|
||||
}
|
||||
|
||||
func (s *state) reserve(count int, size uintptr, path string) error {
|
||||
if count < 0 || (count > 0 && uint64(size) > maxCopyBytes/uint64(count)) {
|
||||
return at(path, ErrLimit)
|
||||
}
|
||||
s.allocated += uint64(count) * uint64(size)
|
||||
if s.allocated > maxCopyBytes {
|
||||
return at(path, ErrLimit)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func nullable(k reflect.Kind) bool {
|
||||
switch k {
|
||||
case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan:
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
func baseValue(base reflect.Value, t reflect.Type) reflect.Value {
|
||||
if base.IsValid() && base.Type() == t {
|
||||
return base
|
||||
}
|
||||
return reflect.Zero(t)
|
||||
}
|
||||
func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) {
|
||||
s.nodes++
|
||||
if depth > maxDepth || s.nodes > maxNodes {
|
||||
return reflect.Value{}, at(path, ErrLimit)
|
||||
}
|
||||
base = baseValue(base, t)
|
||||
if !shallowSafe(t) {
|
||||
return reflect.Value{}, at(path, ErrUnsupported)
|
||||
}
|
||||
if src.IsValid() && !src.CanInterface() {
|
||||
return reflect.Value{}, at(path, ErrUnsupported)
|
||||
}
|
||||
// Exact converters also receive typed nils and zeros. No method calls,
|
||||
// Scanner/Valuer invocation or implicit numeric conversion is attempted.
|
||||
if src.IsValid() {
|
||||
if convert, ok := s.options.converters[pair{src.Type(), t}]; ok {
|
||||
out, err := convert(src)
|
||||
if err != nil {
|
||||
return reflect.Value{}, at(path, err)
|
||||
}
|
||||
if s.graph != nil {
|
||||
if err = s.graph.audit(out); err != nil {
|
||||
return reflect.Value{}, at(path, err)
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
}
|
||||
if !src.IsValid() || (nullable(src.Kind()) && src.IsNil()) {
|
||||
if src.IsValid() && !s.nilCompatible(src.Type(), t, map[pair]bool{}) {
|
||||
return reflect.Value{}, at(path, ErrType)
|
||||
}
|
||||
if s.options.nilPolicy == NilSkip {
|
||||
return base, nil
|
||||
}
|
||||
if nullable(t.Kind()) || s.options.nilPolicy == NilZero {
|
||||
return reflect.Zero(t), nil
|
||||
}
|
||||
return reflect.Value{}, at(path, ErrNil)
|
||||
}
|
||||
if s.options.skipZero && src.IsZero() {
|
||||
if !s.nilCompatible(src.Type(), t, map[pair]bool{}) {
|
||||
return reflect.Value{}, at(path, ErrType)
|
||||
}
|
||||
return base, nil
|
||||
}
|
||||
if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && !s.options.skipZero && s.options.nilPolicy != NilSkip {
|
||||
if !s.options.deep || simpleValue(t) {
|
||||
return src, nil
|
||||
}
|
||||
}
|
||||
if src.Kind() == reflect.Interface {
|
||||
return s.walk(src.Elem(), base, t, path, depth+1)
|
||||
}
|
||||
if t.Kind() == reflect.Interface {
|
||||
if !src.Type().Implements(t) {
|
||||
return reflect.Value{}, at(path, ErrType)
|
||||
}
|
||||
out, err := s.walk(src, reflect.Value{}, src.Type(), path, depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
v := reflect.New(t).Elem()
|
||||
v.Set(out)
|
||||
return v, nil
|
||||
}
|
||||
if s.options.deep && forbidden(t) {
|
||||
return reflect.Value{}, at(path, ErrUnsupported)
|
||||
}
|
||||
if src.Kind() == reflect.Pointer && t.Kind() != reflect.Pointer {
|
||||
return s.walk(src.Elem(), base, t, path, depth+1)
|
||||
}
|
||||
if t.Kind() == reflect.Pointer {
|
||||
if !s.options.deep && src.Type() == t {
|
||||
return src, nil
|
||||
}
|
||||
if src.Kind() == reflect.Pointer {
|
||||
key := visit{identity(src), t}
|
||||
if s.valueRoot != nil && key.reference == *s.valueRoot {
|
||||
return reflect.Value{}, at(path, fmt.Errorf("%w: cyclic root must retain its pointer; use Clone(source) or Copy(&targetPointer, source)", ErrUnsupported))
|
||||
}
|
||||
if prior, ok := s.seen[key]; ok {
|
||||
return prior, nil
|
||||
}
|
||||
}
|
||||
if err := s.reserve(1, t.Elem().Size(), path); err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out := reflect.New(t.Elem())
|
||||
if out.Type() != t {
|
||||
out = out.Convert(t)
|
||||
}
|
||||
if src.Kind() == reflect.Pointer {
|
||||
key := visit{identity(src), t}
|
||||
s.seen[key] = out
|
||||
src = src.Elem()
|
||||
}
|
||||
var childBase reflect.Value
|
||||
if !base.IsNil() {
|
||||
childBase = base.Elem()
|
||||
}
|
||||
value, err := s.walk(src, childBase, t.Elem(), path, depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out.Elem().Set(value)
|
||||
return out, nil
|
||||
}
|
||||
if src.Kind() != t.Kind() {
|
||||
return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t))
|
||||
}
|
||||
if t == timeType && src.Type() == t {
|
||||
return src, nil
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Struct:
|
||||
return s.structure(src, base, t, path, depth)
|
||||
case reflect.Slice:
|
||||
if !s.options.deep && src.Type() == t {
|
||||
return src, nil
|
||||
}
|
||||
key := visit{identity(src), t}
|
||||
if prior, ok := s.seen[key]; ok {
|
||||
return prior, nil
|
||||
}
|
||||
if err := s.reserve(src.Len(), t.Elem().Size(), path); err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out := reflect.MakeSlice(t, src.Len(), src.Len())
|
||||
s.seen[key] = out
|
||||
if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && simpleValue(t.Elem()) {
|
||||
reflect.Copy(out, src)
|
||||
return out, nil
|
||||
}
|
||||
for i := 0; i < src.Len(); i++ {
|
||||
v, err := s.walk(src.Index(i), reflect.Value{}, t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out.Index(i).Set(v)
|
||||
}
|
||||
return out, nil
|
||||
case reflect.Array:
|
||||
if src.Len() != t.Len() {
|
||||
return reflect.Value{}, at(path, ErrType)
|
||||
}
|
||||
if err := s.reserve(1, t.Size(), path); err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out := reflect.New(t).Elem()
|
||||
for i := 0; i < src.Len(); i++ {
|
||||
v, err := s.walk(src.Index(i), base.Index(i), t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out.Index(i).Set(v)
|
||||
}
|
||||
return out, nil
|
||||
case reflect.Map:
|
||||
if !s.options.deep && src.Type() == t {
|
||||
return src, nil
|
||||
}
|
||||
key := visit{identity(src), t}
|
||||
if prior, ok := s.seen[key]; ok {
|
||||
return prior, nil
|
||||
}
|
||||
if err := s.reserve(src.Len(), t.Key().Size()+t.Elem().Size()+32, path); err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out := reflect.MakeMapWithSize(t, src.Len())
|
||||
s.seen[key] = out
|
||||
it := src.MapRange()
|
||||
for it.Next() {
|
||||
k, err := s.walk(it.Key(), reflect.Value{}, t.Key(), path+"[key]", depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
if !k.Comparable() {
|
||||
return reflect.Value{}, at(path+"[key]", ErrType)
|
||||
}
|
||||
if out.MapIndex(k).IsValid() {
|
||||
return reflect.Value{}, at(path+"[key]", errors.New("copyx: key conversion collision"))
|
||||
}
|
||||
v, err := s.walk(it.Value(), reflect.Value{}, t.Elem(), path+"[value]", depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out.SetMapIndex(k, v)
|
||||
}
|
||||
return out, nil
|
||||
case reflect.Func, reflect.Chan, reflect.UnsafePointer:
|
||||
if s.options.deep {
|
||||
return reflect.Value{}, at(path, ErrUnsupported)
|
||||
}
|
||||
}
|
||||
if src.Type() != t {
|
||||
return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t))
|
||||
}
|
||||
return src, nil
|
||||
}
|
||||
|
||||
var fieldTypes sync.Map
|
||||
|
||||
func exportedFields(t reflect.Type) map[string]reflect.StructField {
|
||||
if known, ok := fieldTypes.Load(t); ok {
|
||||
return known.(map[string]reflect.StructField)
|
||||
}
|
||||
out := map[string]reflect.StructField{}
|
||||
for _, f := range reflect.VisibleFields(t) {
|
||||
if f.PkgPath != "" {
|
||||
continue
|
||||
}
|
||||
kind := f.Type.Kind()
|
||||
if kind == reflect.Pointer {
|
||||
kind = f.Type.Elem().Kind()
|
||||
}
|
||||
if f.Anonymous && kind == reflect.Struct {
|
||||
continue
|
||||
}
|
||||
out[f.Name] = f
|
||||
}
|
||||
actual, _ := fieldTypes.LoadOrStore(t, out)
|
||||
return actual.(map[string]reflect.StructField)
|
||||
}
|
||||
|
||||
func (s *state) nilCompatible(from, to reflect.Type, seen map[pair]bool) bool {
|
||||
key := pair{from, to}
|
||||
if seen[key] {
|
||||
return true
|
||||
}
|
||||
seen[key] = true
|
||||
if from == to {
|
||||
return true
|
||||
}
|
||||
if _, ok := s.options.converters[key]; ok {
|
||||
return true
|
||||
}
|
||||
if to.Kind() == reflect.Interface {
|
||||
return from.Implements(to)
|
||||
}
|
||||
if from.Kind() == reflect.Pointer {
|
||||
return s.nilCompatible(from.Elem(), to, seen)
|
||||
}
|
||||
if to.Kind() == reflect.Pointer {
|
||||
return s.nilCompatible(from, to.Elem(), seen)
|
||||
}
|
||||
if from.Kind() != to.Kind() {
|
||||
return false
|
||||
}
|
||||
switch from.Kind() {
|
||||
case reflect.Struct:
|
||||
// Nil must not hide a structurally incompatible type pair. There are no
|
||||
// field values to visit, but coverage and conversion types still apply.
|
||||
for i := 0; i < to.NumField(); i++ {
|
||||
if to.Field(i).PkgPath != "" {
|
||||
return false
|
||||
}
|
||||
}
|
||||
sources, targets := exportedFields(from), exportedFields(to)
|
||||
rules := s.options.fields[key]
|
||||
if rules == nil {
|
||||
rules = &fields{}
|
||||
}
|
||||
reverse, used := map[string]string{}, map[string]bool{}
|
||||
for a, b := range rules.names {
|
||||
reverse[b] = a
|
||||
}
|
||||
for name, target := range targets {
|
||||
if rules.ignoreTarget[name] {
|
||||
continue
|
||||
}
|
||||
srcName := name
|
||||
if renamed, ok := reverse[name]; ok {
|
||||
srcName = renamed
|
||||
}
|
||||
source, ok := sources[srcName]
|
||||
if !ok || rules.ignoreSource[srcName] || !s.nilCompatible(source.Type, target.Type, seen) {
|
||||
return false
|
||||
}
|
||||
used[srcName] = true
|
||||
}
|
||||
if s.options.strict {
|
||||
for name := range sources {
|
||||
if !used[name] && !rules.ignoreSource[name] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
return true
|
||||
case reflect.Slice:
|
||||
return s.nilCompatible(from.Elem(), to.Elem(), seen)
|
||||
case reflect.Map:
|
||||
return s.nilCompatible(from.Key(), to.Key(), seen) && s.nilCompatible(from.Elem(), to.Elem(), seen)
|
||||
case reflect.Array:
|
||||
return from.Len() == to.Len() && s.nilCompatible(from.Elem(), to.Elem(), seen)
|
||||
}
|
||||
return false
|
||||
}
|
||||
func fieldValue(v reflect.Value, index []int) (reflect.Value, bool) {
|
||||
for _, i := range index {
|
||||
if v.Kind() == reflect.Pointer {
|
||||
if v.IsNil() {
|
||||
return reflect.Value{}, false
|
||||
}
|
||||
v = v.Elem()
|
||||
}
|
||||
v = v.Field(i)
|
||||
}
|
||||
return v, true
|
||||
}
|
||||
func setField(v reflect.Value, index []int, value reflect.Value) error {
|
||||
for j, i := range index {
|
||||
field := v.Field(i)
|
||||
if !field.CanSet() {
|
||||
return ErrUnsupported
|
||||
}
|
||||
if j == len(index)-1 {
|
||||
field.Set(value)
|
||||
return nil
|
||||
}
|
||||
if field.Kind() == reflect.Pointer {
|
||||
// Detach embedded destination pointers before editing promoted fields.
|
||||
fresh := reflect.New(field.Type().Elem())
|
||||
if !field.IsNil() {
|
||||
fresh.Elem().Set(field.Elem())
|
||||
}
|
||||
field.Set(fresh)
|
||||
v = fresh.Elem()
|
||||
} else {
|
||||
v = field
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (s *state) structure(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) {
|
||||
if err := s.reserve(1, t.Size(), path); err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
out := reflect.New(t).Elem()
|
||||
out.Set(base)
|
||||
if src.Type() == t {
|
||||
// Private scalar state may be assigned as part of its whole value; mutable
|
||||
// private state requires a converter, never unsafe writes or silent loss.
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
f := t.Field(i)
|
||||
if f.PkgPath != "" && s.options.deep && privateMutable(f.Type) {
|
||||
return reflect.Value{}, at(path+"."+f.Name, ErrUnsupported)
|
||||
}
|
||||
}
|
||||
out.Set(src)
|
||||
} else {
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
if t.Field(i).PkgPath != "" {
|
||||
return reflect.Value{}, at(path, ErrUnsupported)
|
||||
}
|
||||
}
|
||||
}
|
||||
from, to := exportedFields(src.Type()), exportedFields(t)
|
||||
rules := s.options.fields[pair{src.Type(), t}]
|
||||
if rules == nil {
|
||||
rules = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}}
|
||||
}
|
||||
reverse := map[string]string{}
|
||||
for a, b := range rules.names {
|
||||
reverse[b] = a
|
||||
}
|
||||
names := make([]string, 0, len(to))
|
||||
for n := range to {
|
||||
names = append(names, n)
|
||||
}
|
||||
sort.Strings(names)
|
||||
used := map[string]bool{}
|
||||
for _, name := range names {
|
||||
target := to[name]
|
||||
old, _ := fieldValue(base, target.Index)
|
||||
if rules.ignoreTarget[name] {
|
||||
if !old.IsValid() {
|
||||
old = reflect.Zero(target.Type)
|
||||
}
|
||||
if err := setField(out, target.Index, old); err != nil {
|
||||
return reflect.Value{}, at(path+"."+name, err)
|
||||
}
|
||||
continue
|
||||
}
|
||||
srcName := name
|
||||
if renamed, ok := reverse[name]; ok {
|
||||
srcName = renamed
|
||||
}
|
||||
source, ok := from[srcName]
|
||||
if !ok || rules.ignoreSource[srcName] {
|
||||
return reflect.Value{}, at(path+"."+name, ErrField)
|
||||
}
|
||||
used[srcName] = true
|
||||
v, ok := fieldValue(src, source.Index)
|
||||
if !ok {
|
||||
return reflect.Value{}, at(path+"."+name, ErrNil)
|
||||
}
|
||||
value, err := s.walk(v, old, target.Type, path+"."+name, depth+1)
|
||||
if err != nil {
|
||||
return reflect.Value{}, err
|
||||
}
|
||||
if err = setField(out, target.Index, value); err != nil {
|
||||
return reflect.Value{}, at(path+"."+name, err)
|
||||
}
|
||||
}
|
||||
if s.options.strict {
|
||||
for name := range from {
|
||||
if !used[name] && !rules.ignoreSource[name] {
|
||||
return reflect.Value{}, at(path+"."+name, ErrField)
|
||||
}
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
448
copyx/copy_test.go
Normal file
448
copyx/copy_test.go
Normal file
@@ -0,0 +1,448 @@
|
||||
package copyx_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
)
|
||||
|
||||
type detail struct {
|
||||
Values []int
|
||||
Index map[string][]int
|
||||
Next *detail
|
||||
}
|
||||
|
||||
func TestDeepGraph(t *testing.T) {
|
||||
src := &detail{Values: []int{1, 2}, Index: map[string][]int{"a": {3}}}
|
||||
src.Next = src
|
||||
dst, err := copyx.Clone(src)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if dst == src || dst.Next != dst {
|
||||
t.Fatal("pointer cycle not preserved")
|
||||
}
|
||||
dst.Values[0] = 9
|
||||
dst.Index["a"][0] = 8
|
||||
if src.Values[0] != 1 || src.Index["a"][0] != 3 {
|
||||
t.Fatal("deep alias")
|
||||
}
|
||||
m := map[string]any{}
|
||||
m["self"] = m
|
||||
n, err := copyx.Clone(m)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
n["self"].(map[string]any)["new"] = 1
|
||||
if n["new"] != 1 || m["new"] != nil {
|
||||
t.Fatal("map cycle changed")
|
||||
}
|
||||
x := []any{nil}
|
||||
x[0] = x
|
||||
y, err := copyx.Clone(x)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
y[0].([]any)[0] = "changed"
|
||||
if y[0] != "changed" {
|
||||
t.Fatal("slice cycle changed")
|
||||
}
|
||||
if _, ok := x[0].([]any); !ok {
|
||||
t.Fatal("source slice changed")
|
||||
}
|
||||
}
|
||||
func TestShallowAndNil(t *testing.T) {
|
||||
src := detail{Values: []int{1}}
|
||||
var dst detail
|
||||
if err := copyx.Copy(&dst, src); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dst.Values[0] = 2
|
||||
if src.Values[0] != 2 {
|
||||
t.Fatal("shallow unexpectedly detached")
|
||||
}
|
||||
for _, slice := range [][]int{nil, {}} {
|
||||
out, err := copyx.Clone(slice)
|
||||
if err != nil || (out == nil) != (slice == nil) {
|
||||
t.Fatal(out, err)
|
||||
}
|
||||
}
|
||||
for _, m := range []map[string]int{nil, {}} {
|
||||
out, err := copyx.Clone(m)
|
||||
if err != nil || (out == nil) != (m == nil) {
|
||||
t.Fatal(out, err)
|
||||
}
|
||||
}
|
||||
var p *int
|
||||
n := 7
|
||||
if err := copyx.Copy(&n, p); !errors.Is(err, copyx.ErrNil) || n != 7 {
|
||||
t.Fatal(n, err)
|
||||
}
|
||||
if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilSkip)); err != nil || n != 7 {
|
||||
t.Fatal(n, err)
|
||||
}
|
||||
if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilZero)); err != nil || n != 0 {
|
||||
t.Fatal(n, err)
|
||||
}
|
||||
x := 3
|
||||
out := &x
|
||||
if err := copyx.Copy(&out, p); err != nil || out != nil {
|
||||
t.Fatal(out, err)
|
||||
}
|
||||
flag := true
|
||||
if err := copyx.Copy(&flag, false); err != nil || flag {
|
||||
t.Fatal(flag, err)
|
||||
}
|
||||
flag = true
|
||||
if err := copyx.Copy(&flag, false, copyx.WithSkipZero()); err != nil || !flag {
|
||||
t.Fatal(flag, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConverterAliasesRejected(t *testing.T) {
|
||||
for _, kind := range []string{"same", "nested", "subslice", "capacity", "fieldPointer", "private", "map", "sibling"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
src := &detail{Values: []int{1, 2, 3}, Index: map[string][]int{"x": {1}}}
|
||||
var err error
|
||||
switch kind {
|
||||
case "same":
|
||||
_, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return v, nil }))
|
||||
case "nested":
|
||||
_, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return &detail{Values: v.Values}, nil }))
|
||||
case "subslice":
|
||||
_, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[1:], nil }))
|
||||
case "capacity":
|
||||
_, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[:0], nil }))
|
||||
case "map":
|
||||
_, err = copyx.Clone(src.Index, copyx.WithConverter(func(v map[string][]int) (map[string][]int, error) { return v, nil }))
|
||||
case "fieldPointer":
|
||||
var dst *int
|
||||
err = copyx.Copy(&dst, src.Values, copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) (*int, error) { return &v[1], nil }))
|
||||
case "private":
|
||||
type opaque struct{ data []int }
|
||||
v := opaque{src.Values}
|
||||
_, err = copyx.Clone(v, copyx.WithConverter(func(v opaque) (opaque, error) { return v, nil }))
|
||||
case "sibling":
|
||||
type object struct{ A, B []int }
|
||||
v := object{[]int{1}, []int{2}}
|
||||
_, err = copyx.Clone(v, copyx.WithConverter(func(_ []int) ([]int, error) { return v.B, nil }))
|
||||
}
|
||||
if !errors.Is(err, copyx.ErrSharedReference) {
|
||||
t.Fatalf("got %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
_, err := copyx.Clone(4, copyx.WithConverter(func(v int) (int, error) { return v, nil }))
|
||||
if err != nil {
|
||||
t.Fatal("equal primitive rejected", err)
|
||||
}
|
||||
_, err = copyx.Clone("same", copyx.WithConverter(func(v string) (string, error) { return v, nil }))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
good, err := copyx.Clone([]int{1}, copyx.WithConverter(func(v []int) ([]int, error) { return append([]int(nil), v...), nil }))
|
||||
if err != nil || good[0] != 1 {
|
||||
t.Fatal(good, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPrivateAndUnsupported(t *testing.T) {
|
||||
type money struct{ cents *int64 }
|
||||
n := int64(9007199254740991)
|
||||
src := money{&n}
|
||||
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrUnsupported) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out, err := copyx.Clone(src, copyx.WithConverter(func(v money) (money, error) { n := *v.cents; return money{&n}, nil }))
|
||||
if err != nil || *out.cents != n || out.cents == src.cents {
|
||||
t.Fatal(out, err)
|
||||
}
|
||||
tm := time.Now()
|
||||
cloned, err := copyx.Clone(tm)
|
||||
if err != nil || !cloned.Equal(tm) {
|
||||
t.Fatal(cloned, err)
|
||||
}
|
||||
for _, v := range []any{make(chan int), func() {}, &sync.Mutex{}, sync.Mutex{}} {
|
||||
if _, err := copyx.Clone(v); !errors.Is(err, copyx.ErrUnsupported) {
|
||||
t.Fatalf("%T: %v", v, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNilTypeMismatchAndInterfaceConverter(t *testing.T) {
|
||||
var source *string
|
||||
target := 7
|
||||
if err := copyx.Copy(&target, source, copyx.WithNilPolicy(copyx.NilZero)); !errors.Is(err, copyx.ErrType) || target != 7 {
|
||||
t.Fatal(target, err)
|
||||
}
|
||||
var list []int
|
||||
var targetMap map[string]int
|
||||
if err := copyx.Copy(&targetMap, list); !errors.Is(err, copyx.ErrType) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
type input struct{ Value any }
|
||||
type output struct{ Value string }
|
||||
var out output
|
||||
err := copyx.Copy(&out, input{}, copyx.WithConverter(func(v any) (string, error) {
|
||||
if v == nil {
|
||||
return "nil handled", nil
|
||||
}
|
||||
return "value", nil
|
||||
}))
|
||||
if err != nil || out.Value != "nil handled" {
|
||||
t.Fatal(out, err)
|
||||
}
|
||||
var lock sync.Mutex
|
||||
if err := copyx.Copy(&lock, sync.Mutex{}); !errors.Is(err, copyx.ErrUnsupported) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
type locked struct {
|
||||
mu sync.Mutex
|
||||
Value int
|
||||
}
|
||||
var l locked
|
||||
if err := copyx.Copy(&l, locked{}); !errors.Is(err, copyx.ErrUnsupported) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAllocationLimitAndPointerRoot(t *testing.T) {
|
||||
src := make([]byte, (64<<20)+1)
|
||||
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
n := &detail{}
|
||||
n.Next = n
|
||||
var value detail
|
||||
if err := copyx.Copy(&value, n, copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrUnsupported) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var p *detail
|
||||
if err := copyx.Copy(&p, n, copyx.WithDeepCopy()); err != nil || p.Next != p || p == n {
|
||||
t.Fatal(p, err)
|
||||
}
|
||||
}
|
||||
func TestFieldsConvertersAndAtomicity(t *testing.T) {
|
||||
type input struct {
|
||||
Name string
|
||||
Age int
|
||||
Extra string
|
||||
}
|
||||
type output struct {
|
||||
Label string
|
||||
Age int
|
||||
Local string
|
||||
}
|
||||
rename := map[string]string{"Name": "Label"}
|
||||
c, err := copyx.New(copyx.WithFieldNames[input, output](rename), copyx.WithIgnoreTarget[input, output]("Local"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rename["Name"] = "Missing"
|
||||
dst := output{Local: "preserved"}
|
||||
if err = c.Copy(&dst, input{Name: "hello", Age: 2}); err != nil || dst.Label != "hello" || dst.Local != "preserved" {
|
||||
t.Fatal(dst, err)
|
||||
}
|
||||
if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local")); !errors.Is(err, copyx.ErrField) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local"), copyx.WithIgnoreSource[input, output]("Extra")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = copyx.Copy(&dst, input{}); !errors.Is(err, copyx.ErrField) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = copyx.New(copyx.WithFieldNames[input, output](map[string]string{"Nope": "Age"})); err == nil {
|
||||
t.Fatal("invalid rule accepted")
|
||||
}
|
||||
if _, err = copyx.New(copyx.WithConverter(func(v int) (string, error) { return "", nil }), copyx.WithConverter(func(v int) (string, error) { return "", nil })); err == nil {
|
||||
t.Fatal("duplicate converter accepted")
|
||||
}
|
||||
|
||||
sentinel := errors.New("conversion failed")
|
||||
type node struct {
|
||||
A []int
|
||||
Z string
|
||||
}
|
||||
old := node{[]int{8}, "old"}
|
||||
before := old.A
|
||||
err = copyx.Copy(&old, node{[]int{1}, "bad"}, copyx.WithDeepCopy(), copyx.WithConverter(func(v string) (string, error) { return "", sentinel }))
|
||||
if !errors.Is(err, sentinel) || !strings.Contains(err.Error(), "Z") || old.Z != "old" || old.A[0] != 8 || &old.A[0] != &before[0] {
|
||||
t.Fatal(old, err)
|
||||
}
|
||||
// Exact typed nil converter must get control before the global nil policy.
|
||||
var ptr *int
|
||||
var value int
|
||||
err = copyx.Copy(&value, ptr, copyx.WithConverter(func(v *int) (int, error) {
|
||||
if v == nil {
|
||||
return 0, sentinel
|
||||
}
|
||||
return *v, nil
|
||||
}))
|
||||
if !errors.Is(err, sentinel) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var small int8 = 3
|
||||
if err = copyx.Copy(&small, int64(300)); !errors.Is(err, copyx.ErrType) || small != 3 {
|
||||
t.Fatal(small, err)
|
||||
}
|
||||
}
|
||||
|
||||
type Embedded struct {
|
||||
A int
|
||||
Z string
|
||||
}
|
||||
|
||||
func TestEmbeddedDestinationDetached(t *testing.T) {
|
||||
type target struct{ *Embedded }
|
||||
type source struct {
|
||||
A int
|
||||
Z int
|
||||
}
|
||||
ptr := &Embedded{5, "keep"}
|
||||
dst := target{ptr}
|
||||
if err := copyx.Copy(&dst, source{6, 7}); !errors.Is(err, copyx.ErrType) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ptr.A != 5 || dst.Embedded != ptr {
|
||||
t.Fatal("failure mutated embedded target")
|
||||
}
|
||||
if err := copyx.Copy(&dst, Embedded{6, "new"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ptr.A != 5 || dst.A != 6 {
|
||||
t.Fatal("success mutated old embedded object")
|
||||
}
|
||||
}
|
||||
func TestConcurrentCopier(t *testing.T) {
|
||||
c, err := copyx.New(copyx.WithDeepCopy())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
src := &detail{Values: []int{7}, Index: map[string][]int{"a": {9}}}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 64; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
for n := 0; n < 20; n++ {
|
||||
var dst *detail
|
||||
if err := c.Copy(&dst, src); err != nil {
|
||||
t.Error(err)
|
||||
return
|
||||
}
|
||||
dst.Values[0] = n
|
||||
if src.Values[0] != 7 {
|
||||
t.Error("shared")
|
||||
}
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
}
|
||||
func TestBoundAndCollections(t *testing.T) {
|
||||
src := &detail{}
|
||||
p := src
|
||||
for i := 0; i < 300; i++ {
|
||||
p.Next = &detail{}
|
||||
p = p.Next
|
||||
}
|
||||
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
type from struct{ Value string }
|
||||
type to struct{ Value string }
|
||||
var out []to
|
||||
if err := copyx.Copy(&out, []from{{"a"}, {"b"}}); err != nil || len(out) != 2 || out[1].Value != "b" {
|
||||
t.Fatal(out, err)
|
||||
}
|
||||
var array [2]to
|
||||
if err := copyx.Copy(&array, [2]from{{"a"}, {"b"}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := copyx.Copy(&array, [1]from{{"a"}}); !errors.Is(err, copyx.ErrType) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var outMap map[string]int
|
||||
err := copyx.Copy(&outMap, map[int]int{1: 1, 2: 2}, copyx.WithConverter(func(int) (string, error) { return "same", nil }))
|
||||
if err == nil || outMap != nil {
|
||||
t.Fatal(outMap, err)
|
||||
}
|
||||
}
|
||||
func FuzzClone(f *testing.F) {
|
||||
f.Add([]byte{1, 2, 3})
|
||||
f.Add([]byte{})
|
||||
f.Fuzz(func(t *testing.T, data []byte) {
|
||||
if len(data) > 1024 {
|
||||
data = data[:1024]
|
||||
}
|
||||
src := map[string]any{"bytes": data, "other": []string{"x"}}
|
||||
out, err := copyx.Clone(src)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(src, out) {
|
||||
t.Fatal("changed value")
|
||||
}
|
||||
if len(data) > 0 {
|
||||
out["bytes"].([]byte)[0] ^= 255
|
||||
if data[0] == out["bytes"].([]byte)[0] {
|
||||
t.Fatal("alias")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func TestTraversalNodeLimit(t *testing.T) {
|
||||
source := make([]any, 200001)
|
||||
if _, err := copyx.Clone(source); !errors.Is(err, copyx.ErrLimit) {
|
||||
t.Fatal("expected bounded traversal", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNilCannotHideFieldMismatch(t *testing.T) {
|
||||
type source struct{ Value string }
|
||||
type target struct{ Value int }
|
||||
var src *source
|
||||
var dst *target
|
||||
for _, policy := range []copyx.NilPolicy{copyx.NilError, copyx.NilZero, copyx.NilSkip} {
|
||||
if err := copyx.Copy(&dst, src, copyx.WithNilPolicy(policy)); !errors.Is(err, copyx.ErrType) {
|
||||
t.Fatal("nil concealed incompatible fields", err)
|
||||
}
|
||||
}
|
||||
if err := copyx.Copy(&dst, src, copyx.WithConverter(func(string) (int, error) { return 0, nil })); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
text := "keep"
|
||||
if err := copyx.Copy(&text, 0, copyx.WithSkipZero()); !errors.Is(err, copyx.ErrType) || text != "keep" {
|
||||
t.Fatal("zero concealed incompatible types", text, err)
|
||||
}
|
||||
}
|
||||
func BenchmarkCopy(b *testing.B) {
|
||||
for _, n := range []int{1, 100, 1000} {
|
||||
for _, deep := range []bool{false, true} {
|
||||
b.Run(fmt.Sprintf("fields%d/deep%t", n, deep), func(b *testing.B) {
|
||||
src := detail{Values: make([]int, n), Index: map[string][]int{"one": {1, 2}}}
|
||||
var opts []copyx.Option
|
||||
if deep {
|
||||
opts = append(opts, copyx.WithDeepCopy())
|
||||
}
|
||||
c, _ := copyx.New(opts...)
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst detail
|
||||
if err := c.Copy(&dst, src); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
270
copyx/graph.go
Normal file
270
copyx/graph.go
Normal file
@@ -0,0 +1,270 @@
|
||||
package copyx
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const maxDepth = 256
|
||||
const maxNodes = 200000
|
||||
const maxCopyBytes = 64 << 20
|
||||
|
||||
var timeType = reflect.TypeFor[time.Time]()
|
||||
|
||||
// Immutable type metadata is shared, never values or converter registrations.
|
||||
var valueTypes sync.Map
|
||||
|
||||
func simpleValue(t reflect.Type) bool {
|
||||
if known, ok := valueTypes.Load(t); ok {
|
||||
return known.(bool)
|
||||
}
|
||||
result := t == timeType
|
||||
if !result {
|
||||
switch t.Kind() {
|
||||
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
|
||||
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr,
|
||||
reflect.Float32, reflect.Float64, reflect.Complex64, reflect.Complex128, reflect.String:
|
||||
result = true
|
||||
case reflect.Array:
|
||||
result = simpleValue(t.Elem())
|
||||
case reflect.Struct:
|
||||
result = !forbidden(t)
|
||||
for i := 0; result && i < t.NumField(); i++ {
|
||||
result = simpleValue(t.Field(i).Type)
|
||||
}
|
||||
}
|
||||
}
|
||||
valueTypes.Store(t, result)
|
||||
return result
|
||||
}
|
||||
|
||||
type reference struct {
|
||||
kind reflect.Kind
|
||||
typ reflect.Type
|
||||
ptr uintptr
|
||||
length, capacity int
|
||||
}
|
||||
|
||||
func identity(v reflect.Value) reference {
|
||||
r := reference{kind: v.Kind(), typ: v.Type(), ptr: uintptr(v.UnsafePointer())}
|
||||
if v.Kind() == reflect.Slice {
|
||||
r.length = v.Len()
|
||||
r.capacity = v.Cap()
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
type span struct{ start, end uintptr }
|
||||
type graph struct {
|
||||
maps map[uintptr]bool
|
||||
spans []span
|
||||
maxEnd []uintptr
|
||||
}
|
||||
|
||||
// References are inspected without reading private values through Interface or
|
||||
// using unsafe writes. Slice capacity matters: returning source[:0] is an alias.
|
||||
func scanGraph(v reflect.Value, visit func(reflect.Value, string) error) error {
|
||||
seen := map[reference]bool{}
|
||||
nodes := 0
|
||||
var walk func(reflect.Value, string, int) error
|
||||
walk = func(v reflect.Value, path string, depth int) error {
|
||||
nodes++
|
||||
if depth > maxDepth || nodes > maxNodes {
|
||||
return at(path, ErrLimit)
|
||||
}
|
||||
if !v.IsValid() {
|
||||
return nil
|
||||
}
|
||||
if err := visit(v, path); err != nil {
|
||||
return err
|
||||
}
|
||||
if simpleValue(v.Type()) {
|
||||
return nil
|
||||
} // time.Time is a documented immutable value.
|
||||
switch v.Kind() {
|
||||
case reflect.Interface:
|
||||
if !v.IsNil() {
|
||||
return walk(v.Elem(), path, depth+1)
|
||||
}
|
||||
case reflect.Pointer, reflect.Map, reflect.Slice:
|
||||
if v.IsNil() {
|
||||
return nil
|
||||
}
|
||||
id := identity(v)
|
||||
if seen[id] {
|
||||
return nil
|
||||
}
|
||||
seen[id] = true
|
||||
switch v.Kind() {
|
||||
case reflect.Pointer:
|
||||
return walk(v.Elem(), path, depth+1)
|
||||
case reflect.Map:
|
||||
it := v.MapRange()
|
||||
for it.Next() {
|
||||
if err := walk(it.Key(), path+"[key]", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := walk(it.Value(), path+"[value]", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case reflect.Slice:
|
||||
if simpleValue(v.Type().Elem()) {
|
||||
return nil
|
||||
}
|
||||
all := v.Slice(0, v.Cap())
|
||||
for i := 0; i < all.Len(); i++ {
|
||||
if err := walk(all.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
case reflect.Struct:
|
||||
for i := 0; i < v.NumField(); i++ {
|
||||
if err := walk(v.Field(i), path+"."+v.Type().Field(i).Name, depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case reflect.Array:
|
||||
for i := 0; i < v.Len(); i++ {
|
||||
if err := walk(v.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return walk(v, "$", 0)
|
||||
}
|
||||
func mutableSpan(v reflect.Value) (span, bool) {
|
||||
var n uintptr
|
||||
switch v.Kind() {
|
||||
case reflect.Pointer:
|
||||
if v.IsNil() {
|
||||
return span{}, false
|
||||
}
|
||||
n = v.Type().Elem().Size()
|
||||
case reflect.Slice:
|
||||
if v.IsNil() {
|
||||
return span{}, false
|
||||
}
|
||||
n = uintptr(v.Cap()) * v.Type().Elem().Size()
|
||||
default:
|
||||
return span{}, false
|
||||
}
|
||||
if n == 0 {
|
||||
return span{}, false
|
||||
}
|
||||
start := uintptr(v.UnsafePointer())
|
||||
return span{start, start + n}, true
|
||||
}
|
||||
func sourceGraph(v reflect.Value) (*graph, error) {
|
||||
g := &graph{maps: map[uintptr]bool{}}
|
||||
err := scanGraph(v, func(v reflect.Value, _ string) error {
|
||||
if r, ok := mutableSpan(v); ok {
|
||||
g.spans = append(g.spans, r)
|
||||
}
|
||||
if v.Kind() == reflect.Map && !v.IsNil() {
|
||||
g.maps[uintptr(v.UnsafePointer())] = true
|
||||
}
|
||||
return nil
|
||||
})
|
||||
sort.Slice(g.spans, func(i, j int) bool { return g.spans[i].start < g.spans[j].start })
|
||||
var end uintptr
|
||||
for _, r := range g.spans {
|
||||
if r.end > end {
|
||||
end = r.end
|
||||
}
|
||||
g.maxEnd = append(g.maxEnd, end)
|
||||
}
|
||||
return g, err
|
||||
}
|
||||
func (g *graph) audit(v reflect.Value) error {
|
||||
return scanGraph(v, func(v reflect.Value, path string) error {
|
||||
if forbidden(v.Type()) {
|
||||
return at(path, ErrUnsupported)
|
||||
}
|
||||
switch v.Kind() {
|
||||
case reflect.Func, reflect.Chan:
|
||||
if !v.IsNil() {
|
||||
return at(path, ErrUnsupported)
|
||||
}
|
||||
case reflect.UnsafePointer:
|
||||
if v.Pointer() != 0 {
|
||||
return at(path, ErrUnsupported)
|
||||
}
|
||||
}
|
||||
if v.Kind() == reflect.Map && !v.IsNil() && g.maps[uintptr(v.UnsafePointer())] {
|
||||
return at(path, ErrSharedReference)
|
||||
}
|
||||
if r, ok := mutableSpan(v); ok {
|
||||
i := sort.Search(len(g.spans), func(i int) bool { return g.spans[i].start >= r.end })
|
||||
if i > 0 && g.maxEnd[i-1] > r.start {
|
||||
return at(path, ErrSharedReference)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
func forbidden(t reflect.Type) bool {
|
||||
// Lock/resource implementations are not data. A custom converter may turn
|
||||
// them into a different data type but may not manufacture cloned handles.
|
||||
switch t.PkgPath() {
|
||||
case "sync", "sync/atomic", "os", "net", "database/sql":
|
||||
return t.Kind() == reflect.Struct
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
// Shallow copies may share handles through pointers, but must not copy lock or
|
||||
// resource state by value, including state inside a containing struct/array.
|
||||
var safeValueTypes sync.Map
|
||||
|
||||
func shallowSafe(t reflect.Type) bool {
|
||||
if known, ok := safeValueTypes.Load(t); ok {
|
||||
return known.(bool)
|
||||
}
|
||||
result := true
|
||||
if forbidden(t) {
|
||||
safeValueTypes.Store(t, false)
|
||||
return false
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Struct:
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
if !shallowSafe(t.Field(i).Type) {
|
||||
result = false
|
||||
break
|
||||
}
|
||||
}
|
||||
case reflect.Array:
|
||||
result = shallowSafe(t.Elem())
|
||||
}
|
||||
safeValueTypes.Store(t, result)
|
||||
return result
|
||||
}
|
||||
|
||||
func privateMutable(t reflect.Type) bool {
|
||||
if t == timeType {
|
||||
return false
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan, reflect.UnsafePointer:
|
||||
return true
|
||||
case reflect.Array:
|
||||
return privateMutable(t.Elem())
|
||||
case reflect.Struct:
|
||||
if forbidden(t) {
|
||||
return true
|
||||
}
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
if privateMutable(t.Field(i).Type) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
169
copyx/options.go
Normal file
169
copyx/options.go
Normal file
@@ -0,0 +1,169 @@
|
||||
// Package copyx copies Go values without JSON, generated code or global registration.
|
||||
package copyx
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
)
|
||||
|
||||
// NilPolicy controls nil input. NilError preserves nil for nullable targets and
|
||||
// rejects nil to non-nullable targets. NilSkip retains the existing target.
|
||||
type NilPolicy uint8
|
||||
|
||||
const (
|
||||
NilError NilPolicy = iota
|
||||
NilZero
|
||||
NilSkip
|
||||
)
|
||||
|
||||
type pair struct{ from, to reflect.Type }
|
||||
type converter func(reflect.Value) (reflect.Value, error)
|
||||
type fields struct {
|
||||
names map[string]string
|
||||
ignoreSource, ignoreTarget map[string]bool
|
||||
}
|
||||
type settings struct {
|
||||
deep, skipZero, strict bool
|
||||
nilPolicy NilPolicy
|
||||
converters map[pair]converter
|
||||
fields map[pair]*fields
|
||||
}
|
||||
|
||||
// Option configures one immutable Copier. Options never change global behavior.
|
||||
type Option func(*settings) error
|
||||
|
||||
func WithDeepCopy() Option { return func(s *settings) error { s.deep = true; return nil } }
|
||||
|
||||
// WithSkipZero explicitly skips Go zero values, including false, 0 and "".
|
||||
func WithSkipZero() Option { return func(s *settings) error { s.skipZero = true; return nil } }
|
||||
|
||||
// WithStrictFields additionally rejects unconsumed exported source fields.
|
||||
// Target fields must have a source by default, unless explicitly ignored.
|
||||
func WithStrictFields() Option { return func(s *settings) error { s.strict = true; return nil } }
|
||||
func WithNilPolicy(p NilPolicy) Option {
|
||||
return func(s *settings) error {
|
||||
if p > NilSkip {
|
||||
return errors.New("copyx: invalid nil policy")
|
||||
}
|
||||
s.nilPolicy = p
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithConverter registers an exact source/target type pair. Typed nils are
|
||||
// delivered to converters before default nil handling. The function must not
|
||||
// mutate its input and must be safe for concurrent calls when Copier is shared.
|
||||
func WithConverter[S, T any](fn func(S) (T, error)) Option {
|
||||
return func(s *settings) error {
|
||||
if fn == nil {
|
||||
return errors.New("copyx: nil converter")
|
||||
}
|
||||
key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()}
|
||||
if _, exists := s.converters[key]; exists {
|
||||
return fmt.Errorf("copyx: duplicate converter %s -> %s", key.from, key.to)
|
||||
}
|
||||
s.converters[key] = func(v reflect.Value) (reflect.Value, error) {
|
||||
var input S
|
||||
reflect.ValueOf(&input).Elem().Set(v)
|
||||
out, err := fn(input)
|
||||
return reflect.ValueOf(&out).Elem(), err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// WithFieldNames declares source -> target Go field names for one struct pair.
|
||||
// It is not a JSON tag mapping. Only actual, unambiguous exported fields qualify.
|
||||
func WithFieldNames[S, T any](names map[string]string) Option {
|
||||
copyNames := make(map[string]string, len(names))
|
||||
for k, v := range names {
|
||||
copyNames[k] = v
|
||||
}
|
||||
return fieldOption[S, T](func(f *fields) error {
|
||||
for from, to := range copyNames {
|
||||
if _, exists := f.names[from]; exists {
|
||||
return fmt.Errorf("copyx: duplicate field rule %s", from)
|
||||
}
|
||||
f.names[from] = to
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
func WithIgnoreSource[S, T any](names ...string) Option {
|
||||
names = append([]string(nil), names...)
|
||||
return fieldOption[S, T](func(f *fields) error {
|
||||
for _, n := range names {
|
||||
f.ignoreSource[n] = true
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
|
||||
// WithIgnoreTarget preserves the corresponding existing target fields in Copy.
|
||||
// Deep mode rejects that preservation if it would retain a source reference.
|
||||
func WithIgnoreTarget[S, T any](names ...string) Option {
|
||||
names = append([]string(nil), names...)
|
||||
return fieldOption[S, T](func(f *fields) error {
|
||||
for _, n := range names {
|
||||
f.ignoreTarget[n] = true
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}
|
||||
func fieldOption[S, T any](apply func(*fields) error) Option {
|
||||
return func(s *settings) error {
|
||||
key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()}
|
||||
if key.from.Kind() != reflect.Struct || key.to.Kind() != reflect.Struct {
|
||||
return errors.New("copyx: field options require struct types")
|
||||
}
|
||||
f := s.fields[key]
|
||||
if f == nil {
|
||||
f = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}}
|
||||
s.fields[key] = f
|
||||
}
|
||||
return apply(f)
|
||||
}
|
||||
}
|
||||
|
||||
// Copier is immutable after construction. Each Copy owns its traversal state.
|
||||
type Copier struct{ options settings }
|
||||
|
||||
func New(options ...Option) (*Copier, error) {
|
||||
s := settings{converters: map[pair]converter{}, fields: map[pair]*fields{}}
|
||||
for _, option := range options {
|
||||
if option == nil {
|
||||
return nil, errors.New("copyx: nil option")
|
||||
}
|
||||
if err := option(&s); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
for key, f := range s.fields {
|
||||
from, to := exportedFields(key.from), exportedFields(key.to)
|
||||
used := map[string]bool{}
|
||||
for src, dst := range f.names {
|
||||
if _, ok := from[src]; !ok {
|
||||
return nil, fmt.Errorf("copyx: unknown source field %s.%s", key.from, src)
|
||||
}
|
||||
if _, ok := to[dst]; !ok {
|
||||
return nil, fmt.Errorf("copyx: unknown target field %s.%s", key.to, dst)
|
||||
}
|
||||
if used[dst] || f.ignoreSource[src] || f.ignoreTarget[dst] {
|
||||
return nil, fmt.Errorf("copyx: conflicting field rule %s", dst)
|
||||
}
|
||||
used[dst] = true
|
||||
}
|
||||
for n := range f.ignoreSource {
|
||||
if _, ok := from[n]; !ok {
|
||||
return nil, fmt.Errorf("copyx: unknown source field %s", n)
|
||||
}
|
||||
}
|
||||
for n := range f.ignoreTarget {
|
||||
if _, ok := to[n]; !ok {
|
||||
return nil, fmt.Errorf("copyx: unknown target field %s", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
return &Copier{options: s}, nil
|
||||
}
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
结构化 MySQL 配置、独立 TLS/mTLS、驱动日志和生成项目接入见 [官方 MySQL 适配器](mysql/README.md)。已有 Open/OpenConnector 接口继续支持。
|
||||
|
||||
使用 GORM 的项目可显式接入 [gormx 可选适配器](gormx/README.md),复用本包连接池,并将 SQL 日志接入 logx;普通 database 包不导入 GORM。
|
||||
|
||||
`database` 包提供基于标准库 `database/sql` 的连接池生命周期,不绑定 ORM 或驱动:
|
||||
|
||||
- `Config` / `ConfigFromEnv`:连接池参数和 `GO_OMNI_DB_*` 环境变量。
|
||||
|
||||
104
database/gormx/README.md
Normal file
104
database/gormx/README.md
Normal file
@@ -0,0 +1,104 @@
|
||||
# 可选 GORM 适配器
|
||||
|
||||
从 `v0.14.1-preview` 提供。`gormx` 把 GORM 接到 go-omni 已初始化的 `*sql.DB`,保留一份连接池和一处资源所有权。使用 GORM v2 API(本版验收版本为 `gorm.io/gorm v1.31.2`、MySQL dialect `v1.6.0`)。
|
||||
|
||||
核心 `omni`、`database`、`database/mysql` 不导入此包;普通项目的编译依赖链不包含 GORM。仓库同一 Go module 的 go.mod 会记录可选适配器依赖,使用方显式导入 `database/gormx` 才编译它。本版没有新增默认 ORM 或 `--orm` 脚手架开关。
|
||||
|
||||
## 生成项目接入
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
|
||||
go-omni new --database mysql --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
按 [MySQL 配置说明](../mysql/README.md) 设置连接信息、TLS 和池大小。在开发者维护的 `internal/svc/serviceContext.go` 中增加 `ORM *gorm.DB` 字段,并填写现有 Init:
|
||||
|
||||
```go
|
||||
import (
|
||||
"context"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database/gormx"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// 向现有 ServiceContext 添加 ORM *gorm.DB 字段。
|
||||
// 保留 Runtime/Config/Logger/DB 等现有字段。
|
||||
func (c *ServiceContext) Init(ctx context.Context) error {
|
||||
orm, err := gormx.MySQL(ctx, c.DB, gormx.WithLogger(c.Logger))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
c.ORM = orm
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
执行 `go mod tidy`。main、runtimeGen、Handler 和路由无需修改;重新生成保留 svc 的字段与 Init。已有项目先 `go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview`,同样接入;v0.14.0 生成项目的目录和生命周期契约不变。
|
||||
|
||||
Repository 接收 `*gorm.DB`,查询使用请求 context:
|
||||
|
||||
```go
|
||||
err := repo.db.WithContext(ctx).Where("id = ?", id).First(&item).Error
|
||||
|
||||
err = repo.db.WithContext(ctx).Transaction(func(tx *gorm.DB) error {
|
||||
// 同一事务内始终使用 tx。
|
||||
return tx.Create(&item).Error
|
||||
})
|
||||
```
|
||||
|
||||
现有模型、Hooks、Scopes、事务、命名策略和错误处理继续由项目维护。共享连接池不会把分别发起的 SQL/GORM 事务自动合并;不要在一个事务内切回根 ORM 或 c.DB 执行本应属于该事务的写入。
|
||||
|
||||
## 连接池与失败处理
|
||||
|
||||
- MySQL 配置、TLS、字符集、会话参数、池大小及健康检查由底层 `database/mysql` 管理,适配器不再设置另一套 DSN 或池参数。
|
||||
- GORM 方言的版本协商继续执行,自动 Ping 关闭以复用已完成的池初始化。初始化查询使用传入 context,并受池的 PingTimeout 限制;可用 `WithInitTimeout(3*time.Second)` 单独指定。
|
||||
- 返回成功后不保留启动 context,每次查询用 `WithContext(ctx)`。查询取消、错误、事务提交/回滚保持 GORM 原生语义。
|
||||
- 对方言初始化失败时 GORM 自动关闭连接池的路径提供保护;错误仍保留 errors.Is/As,原池继续归调用方管理。生成项目的 Init 返回错误时,再由 Runtime 统一回滚该项目资源。
|
||||
- 服务排空后由 go-omni 关闭原池;`orm.DB()` 返回的仍是那一个池。不要给它重复登记 Close,也不要让 Repository 关闭它。
|
||||
|
||||
适配器自身不创建后台 worker、不自动建表、不执行 AutoMigrate、不改写默认事务或全表更新保护,也不替业务重试写入。
|
||||
|
||||
## SQL 日志
|
||||
|
||||
默认借用当前 context 的 logx,也可显式 `WithLogger(c.Logger)`。每个 ORM 可以使用独立日志实例;不会改动进程全局 Logger。
|
||||
|
||||
```go
|
||||
sqlLog := gormx.DefaultLogConfig()
|
||||
sqlLog.Level = "info" // silent / error / warn / info
|
||||
sqlLog.SlowThreshold = 300*time.Millisecond
|
||||
|
||||
orm, err := gormx.MySQL(ctx, c.DB,
|
||||
gormx.WithLogger(c.Logger),
|
||||
gormx.WithLogConfig(sqlLog),
|
||||
gormx.WithConfig(gorm.Config{
|
||||
TranslateError: true,
|
||||
// NamingStrategy 等原生选项可按项目添加。
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
默认记录错误和超过 200 ms 的慢查询,忽略 ErrRecordNotFound 的错误事件;`SlowThreshold: 0` 关闭慢查询事件。事件为 `gorm.query`、`gorm.slowQuery`、`gorm.queryFailed`,携带 durationMs、rowsAffected、sql 和 context 中的 requestId/traceId。GORM 的 `Debug()`/`LogMode` 使用副本,不改变其他请求的日志级别;最终输出还受 logx 自身级别控制。
|
||||
|
||||
默认不插入绑定参数、不记录驱动完整错误文案,只记录错误类别;SQL 正文最多保留 4096 字节,截断后附加省略号。明确开启 `IncludeParameters` / `IncludeErrorDetails` 才输出这些内容。日志不会解析和脱敏直接写在原始 SQL 里的字面量,查询仍应使用占位符。仅需要日志适配的既有 GORM 实例,也可使用 `NewLogger(logger, config)` 返回的原生 `logger.Interface`。
|
||||
|
||||
`WithConfig` 中若显式设置 `gorm.Config.Logger`,则使用项目自己的 Logger,该 Logger 自行负责脱敏和上下文。配置和 Plugins/ClauseBuilders 容器会复制;用户自定义插件、NamingStrategy 等对象仍由用户保证并发安全。
|
||||
|
||||
## 其他方言与高级配置
|
||||
|
||||
MySQL 便捷入口封装了下面的通用入口:
|
||||
|
||||
```go
|
||||
orm, err := gormx.Open(ctx, pool,
|
||||
func(conn gorm.ConnPool) gorm.Dialector {
|
||||
return mysql.New(mysql.Config{Conn: conn})
|
||||
},
|
||||
gormx.WithLogger(logger),
|
||||
)
|
||||
```
|
||||
|
||||
其他方言可由项目提供相同形式的工厂,必须使用传入的 Conn,不能创建或替换连接池。本版真实数据库验收覆盖 MySQL 8.4,不代表其他数据库已经联调。
|
||||
|
||||
不能通过 WithConfig 的 ConnPool/Dialector 覆盖此约定。初始化期间不暴露裸 sql.DB;需要它的自定义插件应在 Open 成功后显式注册。创建额外池或替换 ConnPool 的路由插件不属于本适配器的一池所有权范围。
|
||||
|
||||
PrepareStmt 默认关闭;若显式开启,保留 GORM 的缓存行为,容量、TTL 和缓存清理由项目按 GORM 接口管理。当前固定的 GORM 版本,PreparedStmtDB.Close 只清理语句,不提供停止其缓存清理 goroutine 的公开接口;因此应在服务初始化时创建一次 ORM,避免按请求反复创建。适配器不修改第三方缓存实现,也不宣称解决其生命周期限制。
|
||||
|
||||
参考:[GORM 现有连接池](https://gorm.io/docs/connecting_to_the_database.html#Existing-database-connection)、[GORM Logger](https://gorm.io/docs/logger.html)。本版验收见 [GORM 验收报告](../../reports/gormAcceptance.md)。
|
||||
121
database/gormx/gorm.go
Normal file
121
database/gormx/gorm.go
Normal file
@@ -0,0 +1,121 @@
|
||||
// Package gormx optionally attaches GORM to a pool owned by GoOmni. It never
|
||||
// creates a pool, changes pool limits, migrates tables or closes the owner.
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"maps"
|
||||
"reflect"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
gormmysql "gorm.io/driver/mysql"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// DialectorFactory must use the supplied connection pool. It must not create a
|
||||
// pool or retain/use the ORM before Open succeeds. For example, return
|
||||
// mysql.New(mysql.Config{Conn: conn}). Other dialects can use the same contract.
|
||||
type DialectorFactory func(conn gorm.ConnPool) gorm.Dialector
|
||||
|
||||
type options struct {
|
||||
config gorm.Config
|
||||
logger *logx.Logger
|
||||
logConfig LogConfig
|
||||
timeout time.Duration
|
||||
}
|
||||
type Option func(*options)
|
||||
|
||||
// WithConfig preserves native GORM behavior, including naming, hooks, prepared
|
||||
// statements and transactions. ConnPool/Dialector are supplied by this adapter;
|
||||
// automatic Ping is disabled because the owning pool already passed startup Ping.
|
||||
// A custom Config.Logger takes precedence over the logx adapter.
|
||||
func WithConfig(config gorm.Config) Option { return func(o *options) { o.config = config } }
|
||||
func WithLogger(logger *logx.Logger) Option { return func(o *options) { o.logger = logger } }
|
||||
func WithLogConfig(config LogConfig) Option { return func(o *options) { o.logConfig = config } }
|
||||
func WithInitTimeout(timeout time.Duration) Option { return func(o *options) { o.timeout = timeout } }
|
||||
|
||||
// Error exposes a safe initialization summary while preserving errors.Is/As.
|
||||
type Error struct{ cause error }
|
||||
|
||||
func (e *Error) Error() string { return "gormx: initialization failed" }
|
||||
func (e *Error) Unwrap() error { return e.cause }
|
||||
|
||||
// MySQL attaches GORM's MySQL dialect, including version negotiation, to the
|
||||
// existing pool. Configure TLS, timeouts and connection limits on that pool.
|
||||
func MySQL(ctx context.Context, pool *database.DB, opts ...Option) (*gorm.DB, error) {
|
||||
return Open(ctx, pool, func(conn gorm.ConnPool) gorm.Dialector {
|
||||
return gormmysql.New(gormmysql.Config{Conn: conn})
|
||||
}, opts...)
|
||||
}
|
||||
|
||||
// Open borrows pool. On failure (including dialect initialization failure),
|
||||
// the original pool remains owned by the caller. No ORM Close is required.
|
||||
// The returned ORM does not retain the startup context: each request should use
|
||||
// WithContext(ctx). Never close the sql.DB returned by orm.DB() independently.
|
||||
func Open(ctx context.Context, pool *database.DB, factory DialectorFactory, opts ...Option) (*gorm.DB, error) {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, &Error{cause: err}
|
||||
}
|
||||
if pool == nil || pool.DB == nil || factory == nil {
|
||||
return nil, errors.New("gormx: pool and dialector factory are required")
|
||||
}
|
||||
timeout := pool.PoolConfig().PingTimeout
|
||||
if timeout == 0 {
|
||||
timeout = 5 * time.Second
|
||||
}
|
||||
o := options{timeout: timeout, logConfig: DefaultLogConfig(), logger: logx.FromContext(ctx)}
|
||||
for _, option := range opts {
|
||||
if option != nil {
|
||||
option(&o)
|
||||
}
|
||||
}
|
||||
if o.timeout <= 0 {
|
||||
return nil, errors.New("gormx: initialization timeout must be positive")
|
||||
}
|
||||
if o.config.ConnPool != nil || o.config.Dialector != nil {
|
||||
return nil, errors.New("gormx: configure the existing pool through the dialector factory")
|
||||
}
|
||||
logger, err := NewLogger(o.logger, o.logConfig)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
config := o.config
|
||||
config.Plugins = maps.Clone(config.Plugins)
|
||||
config.ClauseBuilders = maps.Clone(config.ClauseBuilders)
|
||||
if config.Logger == nil {
|
||||
config.Logger = logger
|
||||
}
|
||||
config.DisableAutomaticPing = true
|
||||
startup, cancel := context.WithTimeout(ctx, o.timeout)
|
||||
defer cancel()
|
||||
borrowed := &borrowedPool{db: pool.DB}
|
||||
borrowed.startup.Store(&startupContext{startup})
|
||||
config.ConnPool = borrowed
|
||||
dialect := factory(borrowed)
|
||||
if dialect == nil || (reflect.ValueOf(dialect).Kind() == reflect.Pointer && reflect.ValueOf(dialect).IsNil()) {
|
||||
return nil, errors.New("gormx: dialector factory returned nil")
|
||||
}
|
||||
orm, err := gorm.Open(dialect, &config)
|
||||
if err != nil {
|
||||
return nil, &Error{cause: err}
|
||||
}
|
||||
if err = startup.Err(); err != nil {
|
||||
return nil, &Error{cause: err}
|
||||
}
|
||||
// Prepared statements are GORM's own wrapper over the very same pool.
|
||||
conn := orm.ConnPool
|
||||
if prepared, ok := conn.(*gorm.PreparedStmtDB); ok {
|
||||
conn = prepared.ConnPool
|
||||
}
|
||||
if actual, ok := conn.(*borrowedPool); !ok || actual != borrowed {
|
||||
return nil, errors.New("gormx: dialector or plugin replaced the supplied pool")
|
||||
}
|
||||
borrowed.startup.Store(nil)
|
||||
return orm, nil
|
||||
}
|
||||
173
database/gormx/gorm_test.go
Normal file
173
database/gormx/gorm_test.go
Normal file
@@ -0,0 +1,173 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql/driver"
|
||||
"errors"
|
||||
"io"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
"gorm.io/gorm/clause"
|
||||
"gorm.io/gorm/schema"
|
||||
)
|
||||
|
||||
type testConnector struct {
|
||||
closed atomic.Int32
|
||||
query func(context.Context) error
|
||||
}
|
||||
|
||||
func (c *testConnector) Connect(context.Context) (driver.Conn, error) {
|
||||
return &testConn{owner: c}, nil
|
||||
}
|
||||
func (c *testConnector) Driver() driver.Driver { return testDriver{} }
|
||||
func (c *testConnector) Close() error { c.closed.Add(1); return nil }
|
||||
|
||||
type testDriver struct{}
|
||||
|
||||
func (testDriver) Open(string) (driver.Conn, error) { return nil, errors.New("unexpected new pool") }
|
||||
|
||||
type testConn struct{ owner *testConnector }
|
||||
|
||||
func (c *testConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("unused") }
|
||||
func (c *testConn) Close() error { return nil }
|
||||
func (c *testConn) Begin() (driver.Tx, error) { return nil, errors.New("unused") }
|
||||
func (c *testConn) Ping(context.Context) error { return nil }
|
||||
func (c *testConn) QueryContext(ctx context.Context, _ string, _ []driver.NamedValue) (driver.Rows, error) {
|
||||
if c.owner.query != nil {
|
||||
if err := c.owner.query(ctx); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return &testRows{}, nil
|
||||
}
|
||||
|
||||
type testRows struct{ done bool }
|
||||
|
||||
func (r *testRows) Columns() []string { return []string{"version"} }
|
||||
func (r *testRows) Close() error { return nil }
|
||||
func (r *testRows) Next(values []driver.Value) error {
|
||||
if r.done {
|
||||
return io.EOF
|
||||
}
|
||||
r.done = true
|
||||
values[0] = "8.4.0"
|
||||
return nil
|
||||
}
|
||||
func poolFixture(t *testing.T) (*database.DB, *testConnector) {
|
||||
t.Helper()
|
||||
connector := &testConnector{}
|
||||
pool, err := database.OpenConnector(context.Background(), connector, database.DefaultPoolConfig())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Cleanup(func() { _ = pool.Close() })
|
||||
return pool, connector
|
||||
}
|
||||
|
||||
func TestBorrowedPoolFailureAndStartupDeadline(t *testing.T) {
|
||||
for _, mode := range []string{"driverError", "deadline", "canceled"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
pool, connector := poolFixture(t)
|
||||
cause := errors.New("private-driver-error")
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
connector.query = func(ctx context.Context) error {
|
||||
if mode == "driverError" {
|
||||
return cause
|
||||
}
|
||||
<-ctx.Done()
|
||||
return ctx.Err()
|
||||
}
|
||||
if mode == "canceled" {
|
||||
cancel()
|
||||
cause = context.Canceled
|
||||
} else if mode == "deadline" {
|
||||
cause = context.DeadlineExceeded
|
||||
}
|
||||
start := time.Now()
|
||||
orm, err := MySQL(ctx, pool, WithInitTimeout(20*time.Millisecond))
|
||||
if orm != nil || !errors.Is(err, cause) {
|
||||
t.Fatal("lost cause", err)
|
||||
}
|
||||
if time.Since(start) > time.Second {
|
||||
t.Fatal("startup deadline ignored")
|
||||
}
|
||||
if connector.closed.Load() != 0 {
|
||||
t.Fatal("borrowed pool closed after GORM failure")
|
||||
}
|
||||
if healthErr := pool.Health(context.Background()); healthErr != nil {
|
||||
t.Fatal("owner pool unusable", healthErr)
|
||||
}
|
||||
if err.Error() == cause.Error() {
|
||||
t.Fatal("unsafe initialization error")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPoolIdentityNativeOptionsAndRequestContext(t *testing.T) {
|
||||
pool, connector := poolFixture(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cfg := gorm.Config{NamingStrategy: schema.NamingStrategy{TablePrefix: "business_"}, ClauseBuilders: map[string]clause.ClauseBuilder{}}
|
||||
logger, _ := logx.New(logx.Config{Output: "discard"})
|
||||
orm, err := MySQL(ctx, pool, WithConfig(cfg), WithLogger(logger))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cancel() // The startup context must not poison future queries.
|
||||
got, err := orm.DB()
|
||||
if err != nil || got != pool.DB {
|
||||
t.Fatal("pool identity changed", err)
|
||||
}
|
||||
if len(cfg.ClauseBuilders) != 0 || cfg.ConnPool != nil || cfg.Logger != nil {
|
||||
t.Fatal("native input config mutated")
|
||||
}
|
||||
if orm.NamingStrategy.TableName("Item") != "business_items" {
|
||||
t.Fatal("native option lost")
|
||||
}
|
||||
if orm.SkipDefaultTransaction || orm.AllowGlobalUpdate {
|
||||
t.Fatal("transaction safety defaults changed")
|
||||
}
|
||||
ctx, cancel = context.WithCancel(context.Background())
|
||||
cancel()
|
||||
var v string
|
||||
if err = orm.WithContext(ctx).Raw("SELECT VERSION()").Scan(&v).Error; !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = orm.WithContext(context.Background()).Raw("SELECT VERSION()").Scan(&v).Error; err != nil || v != "8.4.0" {
|
||||
t.Fatal(v, err)
|
||||
}
|
||||
if connector.closed.Load() != 0 {
|
||||
t.Fatal("pool unexpectedly closed")
|
||||
}
|
||||
if err = pool.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = orm.Raw("SELECT VERSION()").Scan(&v).Error; err == nil {
|
||||
t.Fatal("owner close did not affect GORM")
|
||||
}
|
||||
}
|
||||
|
||||
func TestRejectedOptionsDoNotTouchPool(t *testing.T) {
|
||||
for _, option := range []Option{WithInitTimeout(0), WithLogConfig(LogConfig{Level: "bad"}), WithConfig(gorm.Config{ConnPool: &borrowedPool{}})} {
|
||||
pool, connector := poolFixture(t)
|
||||
if _, err := MySQL(context.Background(), pool, option); err == nil {
|
||||
t.Fatal("invalid option accepted")
|
||||
}
|
||||
if connector.closed.Load() != 0 {
|
||||
t.Fatal("pool closed")
|
||||
}
|
||||
}
|
||||
if _, err := MySQL(nil, nil); err == nil {
|
||||
t.Fatal("nil pool accepted")
|
||||
}
|
||||
pool, _ := poolFixture(t)
|
||||
if _, err := Open(nil, pool, func(gorm.ConnPool) gorm.Dialector { return nil }); err == nil {
|
||||
t.Fatal("nil dialect accepted")
|
||||
}
|
||||
}
|
||||
157
database/gormx/integration_test.go
Normal file
157
database/gormx/integration_test.go
Normal file
@@ -0,0 +1,157 @@
|
||||
package gormx_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database/gormx"
|
||||
adapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
type record struct {
|
||||
ID int64 `gorm:"primaryKey;autoIncrement:false"`
|
||||
Name string
|
||||
}
|
||||
|
||||
func (record) TableName() string { return "gorm_adapter_records" }
|
||||
|
||||
func TestGORMMySQLTransactionsCancellationAndLogs(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" {
|
||||
t.Skip("use scripts/verifyConnector.py --gorm with isolated MySQL")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
|
||||
defer cancel()
|
||||
cfg := adapter.DefaultConfig()
|
||||
host, port, err := net.SplitHostPort(os.Getenv("GO_OMNI_CONNECTOR_ADDR"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg.Host = host
|
||||
cfg.Port, _ = strconv.Atoi(port)
|
||||
cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER")
|
||||
cfg.Password = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD")
|
||||
cfg.Database = os.Getenv("GO_OMNI_CONNECTOR_DATABASE")
|
||||
cfg.TLS.CAFile = os.Getenv("GO_OMNI_CONNECTOR_CA_FILE")
|
||||
cfg.TLS.ServerName = "localhost"
|
||||
cfg.Pool.MaxOpenConns = 2
|
||||
cfg.Pool.MaxIdleConns = 2
|
||||
pool, err := adapter.Open(ctx, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
var logs bytes.Buffer
|
||||
logger, err := logx.New(logx.Config{Writer: &logs})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer logger.Close()
|
||||
logConfig := gormx.DefaultLogConfig()
|
||||
logConfig.Level = "info"
|
||||
orm, err := gormx.MySQL(ctx, pool, gormx.WithLogger(logger), gormx.WithLogConfig(logConfig), gormx.WithConfig(gorm.Config{PrepareStmt: true, TranslateError: true}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
sqlDB, err := orm.DB()
|
||||
if err != nil || sqlDB != pool.DB {
|
||||
t.Fatal("pool not shared", err)
|
||||
}
|
||||
if sqlDB.Stats().MaxOpenConnections != 2 {
|
||||
t.Fatal("pool limits changed")
|
||||
}
|
||||
var tables int
|
||||
if err = pool.QueryRowContext(ctx, "SELECT COUNT(*) FROM information_schema.tables WHERE table_schema = DATABASE() AND table_name = 'gorm_adapter_records'").Scan(&tables); err != nil || tables != 0 {
|
||||
t.Fatal("adapter must not migrate schema", err)
|
||||
}
|
||||
if _, err = pool.ExecContext(ctx, "CREATE TABLE gorm_adapter_records (id BIGINT PRIMARY KEY, name VARCHAR(100) UNIQUE NOT NULL)"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.ExecContext(context.Background(), "DROP TABLE gorm_adapter_records")
|
||||
requestCtx := logx.WithRequestID(ctx, "gorm-request")
|
||||
orm = orm.WithContext(requestCtx)
|
||||
secret := "private-bound-name"
|
||||
if err = orm.Transaction(func(tx *gorm.DB) error { return tx.Create(&record{ID: 1, Name: secret}).Error }); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wanted := errors.New("rollback sentinel")
|
||||
if err = orm.Transaction(func(tx *gorm.DB) error {
|
||||
if e := tx.Create(&record{ID: 2, Name: "rollback"}).Error; e != nil {
|
||||
return e
|
||||
}
|
||||
return wanted
|
||||
}); !errors.Is(err, wanted) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
func() {
|
||||
defer func() {
|
||||
if recover() != "rollback panic" {
|
||||
t.Error("panic changed")
|
||||
}
|
||||
}()
|
||||
_ = orm.Transaction(func(tx *gorm.DB) error {
|
||||
if e := tx.Create(&record{ID: 3, Name: "panic"}).Error; e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
panic("rollback panic")
|
||||
})
|
||||
}()
|
||||
var count int64
|
||||
if err = orm.Model(&record{}).Count(&count).Error; err != nil || count != 1 {
|
||||
t.Fatal("commit/rollback differs", count, err)
|
||||
}
|
||||
if err = orm.Transaction(func(tx *gorm.DB) error {
|
||||
if e := tx.Create(&record{ID: 4, Name: "outer"}).Error; e != nil {
|
||||
return e
|
||||
}
|
||||
if e := tx.Transaction(func(inner *gorm.DB) error {
|
||||
if e := inner.Create(&record{ID: 5, Name: "inner"}).Error; e != nil {
|
||||
return e
|
||||
}
|
||||
return wanted
|
||||
}); !errors.Is(e, wanted) {
|
||||
return errors.New("nested rollback failed")
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = orm.Model(&record{}).Count(&count).Error; err != nil || count != 2 {
|
||||
t.Fatal(count, err)
|
||||
}
|
||||
if err = orm.Create(&record{ID: 6, Name: secret}).Error; !errors.Is(err, gorm.ErrDuplicatedKey) {
|
||||
t.Fatal("native error translation lost", err)
|
||||
}
|
||||
var missing record
|
||||
if err = orm.First(&missing, "id = ?", 999).Error; !errors.Is(err, gorm.ErrRecordNotFound) {
|
||||
t.Fatal("not-found semantics changed", err)
|
||||
}
|
||||
deadline, stop := context.WithTimeout(requestCtx, 30*time.Millisecond)
|
||||
start := time.Now()
|
||||
err = orm.WithContext(deadline).Exec("SELECT SLEEP(3)").Error
|
||||
stop()
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(start) > time.Second {
|
||||
t.Fatal("query cancellation lost", err)
|
||||
}
|
||||
if err = pool.Health(ctx); err != nil {
|
||||
t.Fatal("pool did not recover", err)
|
||||
}
|
||||
if strings.Contains(logs.String(), secret) || !strings.Contains(logs.String(), `"requestId":"gorm-request"`) || !strings.Contains(logs.String(), "gorm.queryFailed") {
|
||||
t.Fatal("SQL log privacy/correlation differs")
|
||||
}
|
||||
var cipher struct {
|
||||
VariableName string
|
||||
Value string
|
||||
}
|
||||
if err = orm.Raw("SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&cipher).Error; err != nil || cipher.Value == "" {
|
||||
t.Fatal("GORM bypassed TLS pool", err)
|
||||
}
|
||||
}
|
||||
171
database/gormx/logger.go
Normal file
171
database/gormx/logger.go
Normal file
@@ -0,0 +1,171 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
gormlog "gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
type LogConfig struct {
|
||||
Level string `json:"level"` // silent, error, warn (default), info
|
||||
SlowThreshold time.Duration `json:"slowThreshold"` // zero disables slow-query events
|
||||
IgnoreRecordNotFound bool `json:"ignoreRecordNotFound"`
|
||||
IncludeParameters bool `json:"includeParameters"`
|
||||
IncludeErrorDetails bool `json:"includeErrorDetails"`
|
||||
MaxSQLBytes int `json:"maxSqlBytes"` // zero uses 4096; bounds emitted SQL text
|
||||
}
|
||||
|
||||
func DefaultLogConfig() LogConfig {
|
||||
return LogConfig{Level: "warn", SlowThreshold: 200 * time.Millisecond, IgnoreRecordNotFound: true, MaxSQLBytes: 4096}
|
||||
}
|
||||
func (c LogConfig) Validate() error {
|
||||
switch c.Level {
|
||||
case "", "silent", "error", "warn", "info":
|
||||
default:
|
||||
return errors.New("gormx: invalid SQL log level")
|
||||
}
|
||||
if c.SlowThreshold < 0 || c.MaxSQLBytes < 0 {
|
||||
return errors.New("gormx: SQL log limits cannot be negative")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type sqlLogger struct {
|
||||
logger *logx.Logger
|
||||
config LogConfig
|
||||
level gormlog.LogLevel
|
||||
}
|
||||
|
||||
// NewLogger adapts logx without changing its level, globals or ownership.
|
||||
// Bound SQL arguments and raw driver error text are omitted by default. SQL
|
||||
// literals written directly into a query are not parsed or redacted.
|
||||
func NewLogger(logger *logx.Logger, config LogConfig) (gormlog.Interface, error) {
|
||||
if err := config.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if config.MaxSQLBytes == 0 {
|
||||
config.MaxSQLBytes = 4096
|
||||
}
|
||||
level := gormlog.Warn
|
||||
switch config.Level {
|
||||
case "silent":
|
||||
level = gormlog.Silent
|
||||
case "error":
|
||||
level = gormlog.Error
|
||||
case "info":
|
||||
level = gormlog.Info
|
||||
}
|
||||
return &sqlLogger{logger: logger, config: config, level: level}, nil
|
||||
}
|
||||
func (l *sqlLogger) LogMode(level gormlog.LogLevel) gormlog.Interface {
|
||||
copy := *l
|
||||
if level < gormlog.Silent || level > gormlog.Info {
|
||||
level = gormlog.Silent
|
||||
}
|
||||
copy.level = level
|
||||
return ©
|
||||
}
|
||||
func (l *sqlLogger) output(ctx context.Context) *logx.Logger {
|
||||
if l.logger != nil {
|
||||
return l.logger
|
||||
}
|
||||
return logx.FromContext(ctx)
|
||||
}
|
||||
func (l *sqlLogger) message(msg string, args []any) string {
|
||||
if !l.config.IncludeErrorDetails {
|
||||
copy := append([]any(nil), args...)
|
||||
for i, v := range copy {
|
||||
if e, ok := v.(error); ok {
|
||||
copy[i] = errorKind(e)
|
||||
}
|
||||
}
|
||||
args = copy
|
||||
}
|
||||
return clip(fmt.Sprintf(msg, args...), l.config.MaxSQLBytes)
|
||||
}
|
||||
func (l *sqlLogger) Info(ctx context.Context, msg string, args ...any) {
|
||||
if l.level >= gormlog.Info {
|
||||
l.output(ctx).InfoContext(ctx, "gorm.message", "message", l.message(msg, args))
|
||||
}
|
||||
}
|
||||
func (l *sqlLogger) Warn(ctx context.Context, msg string, args ...any) {
|
||||
if l.level >= gormlog.Warn {
|
||||
l.output(ctx).WarnContext(ctx, "gorm.message", "message", l.message(msg, args))
|
||||
}
|
||||
}
|
||||
func (l *sqlLogger) Error(ctx context.Context, msg string, args ...any) {
|
||||
if l.level >= gormlog.Error {
|
||||
l.output(ctx).ErrorContext(ctx, "gorm.message", "message", l.message(msg, args))
|
||||
}
|
||||
}
|
||||
|
||||
func (l *sqlLogger) ParamsFilter(_ context.Context, query string, args ...any) (string, []any) {
|
||||
if !l.config.IncludeParameters {
|
||||
return query, nil
|
||||
}
|
||||
return query, args
|
||||
}
|
||||
func (l *sqlLogger) Trace(ctx context.Context, begin time.Time, fc func() (string, int64), err error) {
|
||||
if l.level == gormlog.Silent {
|
||||
return
|
||||
}
|
||||
elapsed := time.Since(begin)
|
||||
failure := err != nil && !(l.config.IgnoreRecordNotFound && errors.Is(err, gorm.ErrRecordNotFound))
|
||||
slow := l.config.SlowThreshold > 0 && elapsed > l.config.SlowThreshold
|
||||
level := gormlog.Info
|
||||
if failure && l.level >= gormlog.Error {
|
||||
level = gormlog.Error
|
||||
} else if slow && l.level >= gormlog.Warn {
|
||||
level = gormlog.Warn
|
||||
} else if l.level < gormlog.Info {
|
||||
return
|
||||
}
|
||||
query, rows := fc()
|
||||
attrs := []any{"durationMs", float64(elapsed) / float64(time.Millisecond), "rowsAffected", rows, "sql", clip(query, l.config.MaxSQLBytes)}
|
||||
if level == gormlog.Error {
|
||||
attrs = append(attrs, "errorKind", errorKind(err))
|
||||
if l.config.IncludeErrorDetails {
|
||||
attrs = append(attrs, "error", clip(err.Error(), l.config.MaxSQLBytes))
|
||||
}
|
||||
}
|
||||
logger := l.output(ctx)
|
||||
switch level {
|
||||
case gormlog.Error:
|
||||
logger.ErrorContext(ctx, "gorm.queryFailed", attrs...)
|
||||
case gormlog.Warn:
|
||||
logger.WarnContext(ctx, "gorm.slowQuery", attrs...)
|
||||
default:
|
||||
logger.InfoContext(ctx, "gorm.query", attrs...)
|
||||
}
|
||||
}
|
||||
func errorKind(err error) string {
|
||||
switch {
|
||||
case errors.Is(err, context.Canceled):
|
||||
return "canceled"
|
||||
case errors.Is(err, context.DeadlineExceeded):
|
||||
return "deadlineExceeded"
|
||||
case errors.Is(err, gorm.ErrRecordNotFound):
|
||||
return "recordNotFound"
|
||||
default:
|
||||
return fmt.Sprintf("%T", err)
|
||||
}
|
||||
}
|
||||
func clip(value string, limit int) string {
|
||||
if len(value) <= limit {
|
||||
return value
|
||||
}
|
||||
value = value[:limit]
|
||||
for !utf8.ValidString(value) && len(value) > 0 {
|
||||
value = value[:len(value)-1]
|
||||
}
|
||||
return value + "…"
|
||||
}
|
||||
|
||||
var _ gormlog.Interface = (*sqlLogger)(nil)
|
||||
var _ gorm.ParamsFilter = (*sqlLogger)(nil)
|
||||
92
database/gormx/logger_test.go
Normal file
92
database/gormx/logger_test.go
Normal file
@@ -0,0 +1,92 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gorm.io/gorm"
|
||||
gormlog "gorm.io/gorm/logger"
|
||||
)
|
||||
|
||||
func TestSQLLoggerLevelsPrivacyAndCorrelation(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
base, _ := logx.New(logx.Config{Writer: &out})
|
||||
l, _ := NewLogger(base, DefaultLogConfig())
|
||||
ctx := logx.WithRequestID(logx.WithTraceID(context.Background(), "trace-1"), "request-1")
|
||||
count := 0
|
||||
fc := func() (string, int64) { count++; return "SELECT * FROM records WHERE token = ?", 1 }
|
||||
l.Trace(ctx, time.Now(), fc, nil)
|
||||
l.Trace(ctx, time.Now(), fc, fmt.Errorf("wrapped: %w", gorm.ErrRecordNotFound))
|
||||
l.LogMode(gormlog.Silent).Trace(ctx, time.Now(), fc, errors.New("password-value"))
|
||||
if count != 0 {
|
||||
t.Fatal("ignored event evaluated SQL")
|
||||
}
|
||||
l.Trace(ctx, time.Now().Add(-time.Second), fc, nil)
|
||||
l.Trace(ctx, time.Now(), fc, errors.New("password-value"))
|
||||
l.Error(ctx, "failed: %v", errors.New("password-value"))
|
||||
l.LogMode(gormlog.Info).Trace(ctx, time.Now(), fc, nil)
|
||||
if count != 3 {
|
||||
t.Fatal(count)
|
||||
}
|
||||
if strings.Contains(out.String(), "password-value") {
|
||||
t.Fatal("driver detail leaked", out.String())
|
||||
}
|
||||
for _, text := range []string{"gorm.slowQuery", "gorm.queryFailed", "gorm.query", `"requestId":"request-1"`, `"traceId":"trace-1"`} {
|
||||
if !strings.Contains(out.String(), text) {
|
||||
t.Fatal(text, out.String())
|
||||
}
|
||||
}
|
||||
q, args := l.(gorm.ParamsFilter).ParamsFilter(ctx, "SELECT ?", "private")
|
||||
if q != "SELECT ?" || len(args) != 0 {
|
||||
t.Fatal("default SQL arguments exposed")
|
||||
}
|
||||
// LogMode clones rather than mutating the shared instance.
|
||||
l.Trace(ctx, time.Now(), fc, nil)
|
||||
if count != 3 {
|
||||
t.Fatal("session mode changed root logger")
|
||||
}
|
||||
}
|
||||
|
||||
func TestSQLLoggerExplicitDetailsAndConcurrentModes(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
base, _ := logx.New(logx.Config{Writer: &out})
|
||||
cfg := DefaultLogConfig()
|
||||
cfg.IncludeParameters = true
|
||||
cfg.IncludeErrorDetails = true
|
||||
cfg.Level = "info"
|
||||
cfg.MaxSQLBytes = 8
|
||||
l, err := NewLogger(base, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, args := l.(gorm.ParamsFilter).ParamsFilter(context.Background(), "SELECT ?", "value")
|
||||
if len(args) != 1 {
|
||||
t.Fatal("explicit arguments dropped")
|
||||
}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
l.LogMode(gormlog.Error).Trace(context.Background(), time.Now(), func() (string, int64) { return "SELECT 中文标识符", -1 }, errors.New("failure"))
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
for _, line := range strings.Split(strings.TrimSpace(out.String()), "\n") {
|
||||
var event map[string]any
|
||||
if err = json.Unmarshal([]byte(line), &event); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if event["error"] != "failure" || strings.Contains(event["sql"].(string), "<22>") {
|
||||
t.Fatal(event)
|
||||
}
|
||||
}
|
||||
}
|
||||
52
database/gormx/pool.go
Normal file
52
database/gormx/pool.go
Normal file
@@ -0,0 +1,52 @@
|
||||
package gormx
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"sync/atomic"
|
||||
|
||||
"gorm.io/gorm"
|
||||
)
|
||||
|
||||
// GORM can call DB().Close on failed initialization. Do not expose the owner's
|
||||
// sql.DB until initialization has succeeded; SQL methods remain fully usable.
|
||||
// Dialects using context.Background during version probing are bounded by the
|
||||
// startup context, which stops affecting queries as soon as Open succeeds.
|
||||
type borrowedPool struct {
|
||||
db *sql.DB
|
||||
startup atomic.Pointer[startupContext]
|
||||
}
|
||||
|
||||
type startupContext struct{ context.Context }
|
||||
|
||||
func (p *borrowedPool) operationContext(ctx context.Context) context.Context {
|
||||
if startup := p.startup.Load(); startup != nil {
|
||||
return startup.Context
|
||||
}
|
||||
return ctx
|
||||
}
|
||||
func (p *borrowedPool) GetDBConn() (*sql.DB, error) {
|
||||
if p.startup.Load() != nil {
|
||||
return nil, gorm.ErrInvalidDB
|
||||
}
|
||||
return p.db, nil
|
||||
}
|
||||
func (p *borrowedPool) PrepareContext(ctx context.Context, query string) (*sql.Stmt, error) {
|
||||
return p.db.PrepareContext(p.operationContext(ctx), query)
|
||||
}
|
||||
func (p *borrowedPool) ExecContext(ctx context.Context, query string, args ...any) (sql.Result, error) {
|
||||
return p.db.ExecContext(p.operationContext(ctx), query, args...)
|
||||
}
|
||||
func (p *borrowedPool) QueryContext(ctx context.Context, query string, args ...any) (*sql.Rows, error) {
|
||||
return p.db.QueryContext(p.operationContext(ctx), query, args...)
|
||||
}
|
||||
func (p *borrowedPool) QueryRowContext(ctx context.Context, query string, args ...any) *sql.Row {
|
||||
return p.db.QueryRowContext(p.operationContext(ctx), query, args...)
|
||||
}
|
||||
func (p *borrowedPool) BeginTx(ctx context.Context, opts *sql.TxOptions) (*sql.Tx, error) {
|
||||
return p.db.BeginTx(p.operationContext(ctx), opts)
|
||||
}
|
||||
|
||||
var _ gorm.ConnPool = (*borrowedPool)(nil)
|
||||
var _ gorm.TxBeginner = (*borrowedPool)(nil)
|
||||
var _ gorm.GetDBConnector = (*borrowedPool)(nil)
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
本包可选,使用 `database/sql` 和 `go-sql-driver/mysql`,复用框架的唯一连接池、Ping、事务与健康检查。不自动建库/迁移、不重试写操作,也不强制某个 MySQL 版本或业务表。
|
||||
|
||||
项目使用 GORM 时,在 svc.Init 中调用 [gormx.MySQL](../gormx/README.md) 即可复用这个池、TLS 和清理流程。
|
||||
|
||||
生成项目选择 `--database mysql` 即接入:
|
||||
|
||||
```sh
|
||||
|
||||
23
examples/contracts/arrays.api
Normal file
23
examples/contracts/arrays.api
Normal file
@@ -0,0 +1,23 @@
|
||||
syntax = "v1"
|
||||
@app (defaultResponse: Common)
|
||||
response Common (
|
||||
codeType: int32
|
||||
successCode: 0
|
||||
errorCode: 9999
|
||||
messageField: msg
|
||||
dataMode: always
|
||||
)
|
||||
@server (group: arrays)
|
||||
service arrays {
|
||||
@handler strings
|
||||
get /strings returns ([]string)
|
||||
@handler integers
|
||||
get /integers returns ([]int64)
|
||||
@handler dynamic
|
||||
get /dynamic returns ([]any)
|
||||
@handler empty
|
||||
get /empty returns ([]any)
|
||||
@handler raw
|
||||
@response raw
|
||||
get /raw returns ([]bool)
|
||||
}
|
||||
23
examples/copying/README.md
Normal file
23
examples/copying/README.md
Normal file
@@ -0,0 +1,23 @@
|
||||
# 结构复制示例
|
||||
|
||||
在框架仓库根目录运行 `go run ./examples/copying`;不需要数据库、网络服务或 JSON 映射配置。
|
||||
|
||||
示例包含:可复用构造器、Go 字段改名、时间自定义转换、保留 false、深拷贝切片、同类型 Clone,以及拒绝自定义转换器返回输入的原切片。
|
||||
|
||||
预期输出:
|
||||
|
||||
```text
|
||||
name=sample enabled=false time=2026-01-01T00:00:00Z sourceTag=original
|
||||
clone detached: true
|
||||
shared converter rejected: true
|
||||
```
|
||||
|
||||
测试:`go test -race ./examples/copying`。完整策略、限制与旧生成映射迁移见 [COPY.md](../../COPY.md)。
|
||||
|
||||
独立工资三端的实际生成、HTTP 与金额用例见 [集成测试](../../cmd/go-omni/apigen/copyConsumer_test.go) 和 [公共契约](../../cmd/go-omni/apigen/testdata/composition/service.api)。框架根目录运行:
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1
|
||||
```
|
||||
|
||||
该测试在临时独立项目中执行,三个入口共用一个普通 Go copyx helper。公共字段新增时,只修改公共契约和独立业务结构,三个入口的转换规则不增加。
|
||||
61
examples/copying/main.go
Normal file
61
examples/copying/main.go
Normal file
@@ -0,0 +1,61 @@
|
||||
// copying demonstrates optional Go-to-Go conversion and checked deep copying.
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"log"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
)
|
||||
|
||||
type Input struct {
|
||||
Name string
|
||||
Enabled bool
|
||||
Tags []string
|
||||
CreatedAt time.Time
|
||||
}
|
||||
type View struct {
|
||||
DisplayName string
|
||||
Enabled bool
|
||||
Tags []string
|
||||
CreatedAt string
|
||||
}
|
||||
|
||||
func run() error {
|
||||
converter, err := copyx.New(
|
||||
copyx.WithDeepCopy(),
|
||||
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
|
||||
copyx.WithConverter(func(v time.Time) (string, error) { return v.UTC().Format(time.RFC3339), nil }),
|
||||
)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
source := Input{Name: "sample", Enabled: false, Tags: []string{"original"}, CreatedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)}
|
||||
var target View
|
||||
if err = converter.Copy(&target, source); err != nil {
|
||||
return err
|
||||
}
|
||||
target.Tags[0] = "changed"
|
||||
fmt.Printf("name=%s enabled=%t time=%s sourceTag=%s\n", target.DisplayName, target.Enabled, target.CreatedAt, source.Tags[0])
|
||||
|
||||
cloned, err := copyx.Clone(source)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cloned.Tags[0] = "separate"
|
||||
fmt.Println("clone detached:", source.Tags[0] == "original")
|
||||
|
||||
_, err = copyx.Clone(source.Tags, copyx.WithConverter(func(v []string) ([]string, error) { return v, nil }))
|
||||
fmt.Println("shared converter rejected:", errors.Is(err, copyx.ErrSharedReference))
|
||||
if !errors.Is(err, copyx.ErrSharedReference) {
|
||||
return errors.New("expected alias rejection")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
9
examples/copying/main_test.go
Normal file
9
examples/copying/main_test.go
Normal file
@@ -0,0 +1,9 @@
|
||||
package main
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestExample(t *testing.T) {
|
||||
if err := run(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -1,5 +1,6 @@
|
||||
{
|
||||
"templateVersion": "v0.14.0-preview",
|
||||
"generatorConfig": true,
|
||||
"templateVersion": "v0.19.0-preview",
|
||||
"profile": {
|
||||
"database": "none",
|
||||
"storage": "none",
|
||||
@@ -10,17 +11,17 @@
|
||||
"module": "example.com/managed",
|
||||
"entry": "api/managed.api",
|
||||
"generated": {
|
||||
"docs/openapi.json": "68c7e9e0294bf62937aa953a44732288c51571108c922a36a72faa72c4360b5f",
|
||||
"docs/openapi.json": "0370fd6bd6a387e2ef21f6b42b009ce66749846b8bea3d481009a34f3af039a5",
|
||||
"internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d",
|
||||
"internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375",
|
||||
"internal/handler/openapiGen.go": "ae9b794c63e0f3c8f9d200a5fc6977350139d4a6f00b42a26102c8630d907d03",
|
||||
"internal/handler/openapiGen.go": "f24cab3abbb8557a3fc4dd93fba23cc0b093f3e0c944d3ba1bd5b0eeff2d96f3",
|
||||
"internal/handler/responseGen.go": "84397a502bba4777ebfa6fae3d3bc418297acab4f26b04b42e3c31b36c735399",
|
||||
"internal/handler/routesGen.go": "76c1a5b2644e75e56553dc6e9e862a63128d902dd94807ce7c8aee5491cd1e3e",
|
||||
"internal/middleware/registryGen.go": "e5f47efa57a1432765ef3419bae59dbf3b4d55b3aac0a6c935ce7a52ba6b47f4",
|
||||
"internal/svc/runtimeGen.go": "78ffd0b2a196fbc29e29473919d1341e8a2f508f6594ad148c0b399fbcc40018",
|
||||
"internal/svc/runtimeGen.go": "29f6afb6614881935099c1845ecb3f7902ee773caa077940f566cb8dfdbcdbb1",
|
||||
"internal/types/dtoManagedd28645a2af1cGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344",
|
||||
"internal/types/typesGen.go": "372d67f23a313b3e77d2a3b7a236315db9721bc63350bf1f71db1293e57570d1",
|
||||
"main.go": "8a4051c0d51fc14ddf22c36d09de706f02499817accc937f0785e3d0fb5e2aa5"
|
||||
"main.go": "64e26d97de7e7c6c17f47186785edc73d43c23d6c20b50bfc7fff6ef2e914f17"
|
||||
},
|
||||
"contract": {
|
||||
"routes": {
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -2,7 +2,7 @@ module example.com/managed
|
||||
|
||||
go 1.22.0
|
||||
|
||||
require git.yuntuo.hk/k2safe/go-omni v0.14.0-preview
|
||||
require git.yuntuo.hk/k2safe/go-omni v0.19.0-preview
|
||||
|
||||
require (
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
@@ -38,16 +37,19 @@ func Load(path string) (Config, error) {
|
||||
return Config{}, err
|
||||
}
|
||||
}
|
||||
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_"})
|
||||
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_", Empty: "ignore", Aliases: config.ServerEnvAliases("GO_OMNI_", "server")})
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
|
||||
cfg.Server, err = config.ApplyEnv(cfg.Server, "")
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
output, outputSet := os.LookupEnv("GO_OMNI_SERVER_LOG_OUTPUT")
|
||||
if !outputSet {
|
||||
output = os.Getenv("GO_OMNI_LOG_OUTPUT")
|
||||
}
|
||||
if output := strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")); output != "" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE")) == "" {
|
||||
file, fileSet := os.LookupEnv("GO_OMNI_SERVER_LOG_FILE")
|
||||
if !fileSet {
|
||||
file = os.Getenv("GO_OMNI_LOG_FILE")
|
||||
}
|
||||
if output != "" && file == "" {
|
||||
cfg.Server.Log.File = ""
|
||||
cfg.Server.Log.Rotation = logx.RotationConfig{}
|
||||
}
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -37,7 +37,11 @@ func New(ctx context.Context, logger *logx.Logger, configs ...config.Config) (re
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := &ServiceContext{Config: cfg, Logger: logger, Health: checker, Middlewares: middleware.Defaults(), Runtime: lifecycle.NewRuntime(ctx)}
|
||||
runtime, err := lifecycle.NewRuntimeWithConfig(logx.WithLogger(ctx, logger), cfg.Server.Lifecycle)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
c := &ServiceContext{Config: cfg, Logger: logger, Health: checker, Middlewares: middleware.Defaults(), Runtime: runtime}
|
||||
ctx = logx.WithLogger(c.Context(), logger)
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
|
||||
@@ -8,6 +8,7 @@ import (
|
||||
"example.com/managed/internal/handler"
|
||||
"example.com/managed/internal/svc"
|
||||
"flag"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"log"
|
||||
"os"
|
||||
@@ -33,7 +34,13 @@ func run(ctx context.Context, path string) (runErr error) {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { runErr = errors.Join(runErr, logger.Close()) }()
|
||||
// Unjoined background work may still log. The owning process exits on this
|
||||
// error; do not close its borrowed logger before that work has finished.
|
||||
defer func() {
|
||||
if !errors.Is(runErr, lifecycle.ErrResourcesRetained) {
|
||||
runErr = errors.Join(runErr, logger.Close())
|
||||
}
|
||||
}()
|
||||
deps, err := svc.New(ctx, logger, cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
4
examples/managedService/omni.gen.json
Normal file
4
examples/managedService/omni.gen.json
Normal file
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"repository": "none",
|
||||
"middleware": "handlers"
|
||||
}
|
||||
4
go.mod
4
go.mod
@@ -19,6 +19,8 @@ require (
|
||||
golang.org/x/net v0.33.0
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1
|
||||
gopkg.in/yaml.v3 v3.0.1
|
||||
gorm.io/driver/mysql v1.6.0
|
||||
gorm.io/gorm v1.31.2
|
||||
)
|
||||
|
||||
require (
|
||||
@@ -36,6 +38,8 @@ require (
|
||||
github.com/google/uuid v1.6.0 // indirect
|
||||
github.com/hashicorp/errwrap v1.1.0 // indirect
|
||||
github.com/hashicorp/go-multierror v1.1.1 // indirect
|
||||
github.com/jinzhu/inflection v1.0.0 // indirect
|
||||
github.com/jinzhu/now v1.1.5 // indirect
|
||||
github.com/klauspost/compress v1.17.11 // indirect
|
||||
github.com/klauspost/cpuid/v2 v2.2.9 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
|
||||
12
go.sum
12
go.sum
@@ -67,6 +67,10 @@ github.com/hashicorp/errwrap v1.1.0 h1:OxrOeh75EUXMY8TBjag2fzXGZ40LB6IKw45YeGUDY
|
||||
github.com/hashicorp/errwrap v1.1.0/go.mod h1:YH+1FKiLXxHSkmPseP+kNlulaMuP3n2brvKWEqk/Jc4=
|
||||
github.com/hashicorp/go-multierror v1.1.1 h1:H5DkEtf6CXdFp0N0Em5UCwQpXMWke8IA0+lD48awMYo=
|
||||
github.com/hashicorp/go-multierror v1.1.1/go.mod h1:iw975J/qwKPdAO1clOe2L8331t/9/fmwbPZ6JB6eMoM=
|
||||
github.com/jinzhu/inflection v1.0.0 h1:K317FqzuhWc8YvSVlFMCCUb36O/S9MCKRDI7QkRKD/E=
|
||||
github.com/jinzhu/inflection v1.0.0/go.mod h1:h+uFLlag+Qp1Va5pdKtLDYj+kHp5pxUVkryuEj+Srlc=
|
||||
github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
|
||||
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
|
||||
github.com/klauspost/compress v1.17.11 h1:In6xLpyWOi1+C7tXUUWv2ot1QvBjxevKAaI6IXrJmUc=
|
||||
github.com/klauspost/compress v1.17.11/go.mod h1:pMDklpSncoRMuLFrf1W9Ss9KT+0rH90U12bZKk7uwG0=
|
||||
github.com/klauspost/cpuid/v2 v2.0.1/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg=
|
||||
@@ -76,6 +80,8 @@ github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||
github.com/lib/pq v1.10.9 h1:YXG7RB+JIjhP29X+OtkiDnYaXQwpS4JEWq7dtCCRUEw=
|
||||
github.com/lib/pq v1.10.9/go.mod h1:AlVN5x4E4T544tWzH6hKfbfQvm3HdbOxrmggDNAPY9o=
|
||||
github.com/mattn/go-sqlite3 v1.14.22 h1:2gZY6PC6kBnID23Tichd1K+Z0oS6nE/XwU+Vz/5o4kU=
|
||||
github.com/mattn/go-sqlite3 v1.14.22/go.mod h1:Uh1q+B4BYcTPb+yiD3kU8Ct7aC0hY9fxUwlHK0RXw+Y=
|
||||
github.com/minio/md5-simd v1.1.2 h1:Gdi1DZK69+ZVMoNHRXJyNcxrMA4dSxoYHZSQbirFg34=
|
||||
github.com/minio/md5-simd v1.1.2/go.mod h1:MzdKDxYpY2BT9XQFocsiZf/NKVtR7nkE4RoEpN+20RM=
|
||||
github.com/minio/minio-go/v7 v7.0.83 h1:W4Kokksvlz3OKf3OqIlzDNKd4MERlC2oN8YptwJ0+GA=
|
||||
@@ -134,3 +140,9 @@ gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
gorm.io/driver/mysql v1.6.0 h1:eNbLmNTpPpTOVZi8MMxCi2aaIm0ZpInbORNXDwyLGvg=
|
||||
gorm.io/driver/mysql v1.6.0/go.mod h1:D/oCC2GWK3M/dqoLxnOlaNKmXz8WNTfcS9y5ovaSqKo=
|
||||
gorm.io/driver/sqlite v1.6.0 h1:WHRRrIiulaPiPFmDcod6prc4l2VGVWHz80KspNsxSfQ=
|
||||
gorm.io/driver/sqlite v1.6.0/go.mod h1:AO9V1qIQddBESngQUKWL9yoH93HIeA1X6V633rBwyT8=
|
||||
gorm.io/gorm v1.31.2 h1:3o8FXNo9v9S858gil+3LlZA1LkCOzgb4g5BL64FgaCo=
|
||||
gorm.io/gorm v1.31.2/go.mod h1:XyQVbO2k6YkOis7C2437jSit3SsDK72s7n7rsSHd+Gs=
|
||||
|
||||
@@ -1,5 +1,67 @@
|
||||
# lifecycle
|
||||
|
||||
## 后台组件分阶段关闭
|
||||
|
||||
从 `v0.15.0-preview` 起,生成项目使用以下顺序:HTTP 停止接收并排空 → 全部 OnStop → 全部 OnDrain → 逆序 OnClose → 关闭日志。初始化、创建 App 或监听失败也经过后台收尾。Runtime 不导入 Orbit、数据库、ORM 或 Redis。
|
||||
|
||||
在项目维护的 svc.Init 中登记即可,main/runtimeGen 由生成器更新。例如使用可选的内存 task.Runner:
|
||||
|
||||
```go
|
||||
jobs, err := task.New(task.Config{Workers: 4, QueueSize: 128})
|
||||
if err != nil { return err }
|
||||
c.OnStop("jobs", jobs.StopAccepting)
|
||||
c.OnDrain("jobs", jobs.Wait)
|
||||
// 把 jobs 保存到项目的 svc 字段,供 Logic 使用。
|
||||
// 已接受任务使用 c.WorkContext(),保留到 HTTP 和后台正常排空结束。
|
||||
```
|
||||
|
||||
数据库、Redis 或第三方客户端仍用已有的 `c.OnClose(client.Close)`;框架创建的 DB 已登记,不要再次登记。这里的 jobs 不再额外登记 Close:StopAccepting 停止定时调度/提交,Wait 等待 worker 与回调全部退出,已完成其生命周期。
|
||||
|
||||
任何自定义组件都可以登记 `func(context.Context) error`,不必实现或依赖框架任务接口:
|
||||
|
||||
- OnStop 只停止接收新工作,不能关闭正在执行的工作所需的资源。每个 OnStop 必须有同名 OnDrain;缺失时拒绝释放共享资源。
|
||||
- OnDrain 返回 nil 表示已经收尾,包含最终写入、确认和监控协程退出。返回错误不作为已完成证明;运行中的业务错误可以先通过 RequestShutdown(err) 上报,再等完成后由 Drain 返回 nil。
|
||||
- 同一阶段的回调并发执行,不按登记顺序执行;框架等待整个阶段结束后才进入下一阶段。有组件之间的先后依赖时,组合为一个 Stop/Drain 登记项,由该项明确组织内部顺序。Stop 不能等待 Drain 才会完成的工作。
|
||||
- OnClose 保留原来的同步逆序执行;普通项目无后台组件时无需登记新钩子。生命周期钩子不能再次调用 Runtime.Close/Finish,否则会等待自身;可以调用 RequestShutdown。
|
||||
- 所有依赖初始化成功后立即登记,再启动使用它们的工作。构造器交付前失败的内部资源仍由构造器释放。
|
||||
|
||||
### 两种 context
|
||||
|
||||
`c.Context()` / `c.Lifecycle.Context()` 是早期退出通知,会随外部信号或后台故障取消。`c.WorkContext()` 保留初始化 context 的值但不继承其取消/截止时间,允许已接受工作在 HTTP 收尾期间继续使用依赖。
|
||||
|
||||
停止接收工作由 OnStop 负责,不能仅等待 WorkContext 取消。正常情况下 WorkContext 在后台排空完成后取消;阶段失败或超时会请求取消工作。框架不假设取消已经意味着 goroutine 结束,仍以 Drain 的完成为准。每个业务任务的执行时限继续由项目/任务组件设置。
|
||||
|
||||
### 时限与失败
|
||||
|
||||
```yaml
|
||||
server:
|
||||
shutdownTimeout: 10s # HTTP 排空时限,保持原语义
|
||||
lifecycle:
|
||||
stopTimeout: 5s
|
||||
drainTimeout: 10s
|
||||
cancelTimeout: 5s
|
||||
```
|
||||
|
||||
这些是各阶段的整体预算,不是每个回调分别等待这么久。零使用默认值,负值在启动前拒绝。可用 `GO_OMNI_LIFECYCLE_STOP_TIMEOUT`、`GO_OMNI_LIFECYCLE_DRAIN_TIMEOUT`、`GO_OMNI_LIFECYCLE_CANCEL_TIMEOUT` 覆盖;生成配置还支持完整的 `GO_OMNI_SERVER_LIFECYCLE_*` 路径,别名优先。配置应在 config 中设置,不放在创建 App 时才执行的 AppOptions 中。
|
||||
|
||||
阶段超时会取消阶段 context 和 WorkContext,再给尚未返回的钩子一个 cancelTimeout 收尾窗口。若所有钩子都返回 nil,可以释放依赖,但仍返回超时错误,进程非零退出。task.Wait 收到取消会取消其任务并返回错误;这不证明全部任务已退出,因此仍按以下保守规则处理。
|
||||
|
||||
Stop 超时且仍未返回的组件,不会再并发调用它同名的 Drain;其他组件仍可继续收尾。此时共享资源始终保留,避免未退出的 Stop 和 Drain 同时操作同一个组件。
|
||||
|
||||
钩子返回错误、panic、缺少匹配 Drain,或窗口结束后仍有钩子未返回时,Runtime 返回可由 `errors.Is(err, lifecycle.ErrResourcesRetained)` 判断的错误,**跳过全部 OnClose**;生成入口也保留借用的日志实例并非零退出,由进程退出释放剩余 OS 资源。不会悄悄重试 Close,也不会让晚到的后台结果触发自动关闭。错误链保留初始化/监听/后台故障及具体 HookError 的阶段、名称和原始原因。
|
||||
|
||||
手写入口必须处理这个错误并终止该服务,不能继续运行半关闭的 Runtime;嵌入式宿主需自行安排隔离/终止策略。Go 不能强杀不响应取消的 goroutine,有限等待与保证它已经结束不能同时无条件成立。OnClose 仍要求自身能返回;本版阶段时限不会强制中断资源 Close。HTTP 超时沿用现有传输层规则,业务处理器仍须响应请求取消。
|
||||
|
||||
手写项目使用 `lifecycle.NewRuntimeWithConfig(ctx, cfg)`;完成 HTTP 排空后调用 `runtime.Finish(runErr)`。库本身不执行 os.Exit。
|
||||
|
||||
### 升级
|
||||
|
||||
v0.14.x 受管理项目将 module 和 CLI 一起升级到 v0.15.0-preview,再运行 `go-omni api generate`、`go-omni api generate --check` 和业务测试。生成器更新 main/runtimeGen,保留 svc、Logic 和配置。已有配置不添加新字段也会使用默认时限,需要调整时再编辑 yaml。
|
||||
|
||||
要获得后台分阶段保证,项目需把原先 `OnClose(worker.Close)` 的后台组件改成对应 Stop/Drain,并使用适合的工作 context;框架不会猜测旧 Close 回调负责数据库还是 worker。仅在 OnClose 中登记的旧任务继续遵循旧的逆序清理规则。
|
||||
|
||||
## 退出通知
|
||||
|
||||
Controller 是服务退出通知入口,不管理具体数据库、租约或后台任务。生成项目已经将它接入 svc 与 main,业务在 svc 调用 `c.RequestShutdown(err)` 即可,见 [项目指南](../PROJECT.md#依赖主动申请退出)。
|
||||
|
||||
手写服务也可组合现有 App:
|
||||
|
||||
184
lifecycle/phases.go
Normal file
184
lifecycle/phases.go
Normal file
@@ -0,0 +1,184 @@
|
||||
package lifecycle
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// ErrResourcesRetained means background quiescence could not be established.
|
||||
// OnClose callbacks were deliberately skipped. Close is not retried; an owning
|
||||
// process should exit unsuccessfully instead of continuing with a stopped Runtime.
|
||||
var ErrResourcesRetained = errors.New("lifecycle: background shutdown incomplete; shared resources retained")
|
||||
|
||||
type ShutdownConfig struct {
|
||||
StopTimeout time.Duration `json:"stopTimeout"`
|
||||
DrainTimeout time.Duration `json:"drainTimeout"`
|
||||
CancelTimeout time.Duration `json:"cancelTimeout"`
|
||||
}
|
||||
|
||||
func DefaultShutdownConfig() ShutdownConfig {
|
||||
return ShutdownConfig{StopTimeout: 5 * time.Second, DrainTimeout: 10 * time.Second, CancelTimeout: 5 * time.Second}
|
||||
}
|
||||
func (c ShutdownConfig) Validate() error {
|
||||
if c.StopTimeout < 0 || c.DrainTimeout < 0 || c.CancelTimeout < 0 {
|
||||
return errors.New("lifecycle: shutdown timeouts must be nonnegative")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (c ShutdownConfig) defaults() ShutdownConfig {
|
||||
d := DefaultShutdownConfig()
|
||||
if c.StopTimeout == 0 {
|
||||
c.StopTimeout = d.StopTimeout
|
||||
}
|
||||
if c.DrainTimeout == 0 {
|
||||
c.DrainTimeout = d.DrainTimeout
|
||||
}
|
||||
if c.CancelTimeout == 0 {
|
||||
c.CancelTimeout = d.CancelTimeout
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
// HookError identifies the failed phase/component and preserves the original error.
|
||||
type HookError struct {
|
||||
Phase, Name string
|
||||
Err error
|
||||
}
|
||||
|
||||
func (e *HookError) Error() string { return fmt.Sprintf("lifecycle %s %q: %v", e.Phase, e.Name, e.Err) }
|
||||
func (e *HookError) Unwrap() error { return e.Err }
|
||||
|
||||
// phaseError keeps diagnostics separate from proof that every hook succeeded.
|
||||
// A deadline is still reported when all callbacks finish during cancellation grace.
|
||||
type phaseError struct {
|
||||
err error
|
||||
unsafe bool
|
||||
unjoined map[string]bool
|
||||
}
|
||||
|
||||
func (e *phaseError) Error() string { return e.err.Error() }
|
||||
func (e *phaseError) Unwrap() error { return e.err }
|
||||
func unsafePhase(err error) bool { var p *phaseError; return errors.As(err, &p) && p.unsafe }
|
||||
|
||||
type shutdownHook struct {
|
||||
name string
|
||||
fn func(context.Context) error
|
||||
}
|
||||
|
||||
// OnStop registers a prompt, idempotent stop-admission callback. It must not
|
||||
// close resources used by accepted work. All stop hooks precede all drain hooks.
|
||||
// Hooks within a phase run concurrently: combine dependent components in one hook.
|
||||
// Every OnStop requires a matching OnDrain with the same name.
|
||||
func (r *Runtime) OnStop(name string, fn func(context.Context) error) { r.addHook(&r.stops, name, fn) }
|
||||
|
||||
// OnDrain registers proof of quiescence. Return nil only after the component can
|
||||
// no longer use shared resources, including final acknowledgements/cleanup.
|
||||
// Deadline cancellation asks the component to stop; returning ctx.Err alone does
|
||||
// not prove completion. A callback may finish during the cancellation grace.
|
||||
func (r *Runtime) OnDrain(name string, fn func(context.Context) error) {
|
||||
r.addHook(&r.drains, name, fn)
|
||||
}
|
||||
|
||||
func (r *Runtime) addHook(list *[]shutdownHook, name string, fn func(context.Context) error) {
|
||||
if fn == nil || name == "" || len(name) > 128 || strings.TrimSpace(name) != name || strings.IndexFunc(name, unicode.IsControl) >= 0 {
|
||||
panic("lifecycle: valid hook name and function required")
|
||||
}
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed {
|
||||
panic("lifecycle: register hook after Close")
|
||||
}
|
||||
for _, h := range *list {
|
||||
if h.name == name {
|
||||
panic("lifecycle: duplicate phase hook name")
|
||||
}
|
||||
}
|
||||
*list = append(*list, shutdownHook{name, fn})
|
||||
}
|
||||
|
||||
func callHook(ctx context.Context, h shutdownHook) (err error) {
|
||||
defer func() {
|
||||
if p := recover(); p != nil {
|
||||
if cause, ok := p.(error); ok {
|
||||
err = fmt.Errorf("hook panic: %w", cause)
|
||||
} else {
|
||||
err = fmt.Errorf("hook panic: %v", p)
|
||||
}
|
||||
}
|
||||
}()
|
||||
return h.fn(ctx)
|
||||
}
|
||||
|
||||
func (r *Runtime) runPhase(phase string, hooks []shutdownHook, timeout time.Duration) error {
|
||||
if len(hooks) == 0 {
|
||||
return nil
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.WithoutCancel(r.work), timeout)
|
||||
defer cancel()
|
||||
type result struct {
|
||||
index int
|
||||
err error
|
||||
}
|
||||
results := make(chan result, len(hooks))
|
||||
pending := make([]bool, len(hooks))
|
||||
for i, h := range hooks {
|
||||
pending[i] = true
|
||||
go func(i int, h shutdownHook) { results <- result{i, callHook(ctx, h)} }(i, h)
|
||||
}
|
||||
remaining := len(hooks)
|
||||
var errs error
|
||||
unsafe := false
|
||||
accept := func(v result) {
|
||||
pending[v.index] = false
|
||||
remaining--
|
||||
if v.err != nil {
|
||||
unsafe = true
|
||||
errs = errors.Join(errs, &HookError{phase, hooks[v.index].name, v.err})
|
||||
r.cancelWork()
|
||||
}
|
||||
}
|
||||
for remaining > 0 {
|
||||
select {
|
||||
case v := <-results:
|
||||
accept(v)
|
||||
case <-ctx.Done():
|
||||
r.cancelWork()
|
||||
errs = errors.Join(errs, &HookError{phase, "deadline", ctx.Err()})
|
||||
grace := time.NewTimer(r.shutdown.CancelTimeout)
|
||||
defer grace.Stop()
|
||||
for remaining > 0 {
|
||||
select {
|
||||
case v := <-results:
|
||||
accept(v)
|
||||
case <-grace.C:
|
||||
// Collect callbacks already returned at the deadline before marking pending ones.
|
||||
for {
|
||||
select {
|
||||
case v := <-results:
|
||||
accept(v)
|
||||
default:
|
||||
goto collected
|
||||
}
|
||||
}
|
||||
collected:
|
||||
unjoined := make(map[string]bool)
|
||||
for i, active := range pending {
|
||||
if active {
|
||||
unjoined[hooks[i].name] = true
|
||||
errs = errors.Join(errs, &HookError{phase, hooks[i].name, context.DeadlineExceeded})
|
||||
}
|
||||
}
|
||||
return &phaseError{err: errs, unsafe: unsafe || remaining > 0, unjoined: unjoined}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if errs != nil {
|
||||
return &phaseError{err: errs, unsafe: unsafe}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
234
lifecycle/phases_test.go
Normal file
234
lifecycle/phases_test.go
Normal file
@@ -0,0 +1,234 @@
|
||||
package lifecycle
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"reflect"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func testRuntime(t *testing.T) *Runtime {
|
||||
t.Helper()
|
||||
r, err := NewRuntimeWithConfig(context.Background(), ShutdownConfig{StopTimeout: time.Second, DrainTimeout: time.Second, CancelTimeout: time.Second})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return r
|
||||
}
|
||||
|
||||
func TestPhasesBarrierAndWorkLifetime(t *testing.T) {
|
||||
parent, cancel := context.WithCancel(context.WithValue(context.Background(), struct{}{}, "value"))
|
||||
defer cancel()
|
||||
r := NewRuntime(parent)
|
||||
var stops, drains atomic.Int32
|
||||
var mu sync.Mutex
|
||||
var closed []int
|
||||
started := make(chan struct{}, 2)
|
||||
release := make(chan struct{})
|
||||
// Deliberately interleave registration. Phases take precedence over order.
|
||||
r.OnClose(func() error { mu.Lock(); defer mu.Unlock(); closed = append(closed, 1); return nil })
|
||||
for _, name := range []string{"producer", "consumer"} {
|
||||
r.OnStop(name, func(ctx context.Context) error {
|
||||
if r.WorkContext().Err() != nil || ctx.Err() != nil {
|
||||
return errors.New("work canceled before drain")
|
||||
}
|
||||
stops.Add(1)
|
||||
started <- struct{}{}
|
||||
<-release
|
||||
return nil
|
||||
})
|
||||
r.OnDrain(name, func(ctx context.Context) error {
|
||||
if stops.Load() != 2 || r.WorkContext().Err() != nil || ctx.Err() != nil {
|
||||
return errors.New("stage barrier failed")
|
||||
}
|
||||
drains.Add(1)
|
||||
return nil
|
||||
})
|
||||
}
|
||||
r.OnClose(func() error {
|
||||
if drains.Load() != 2 {
|
||||
return errors.New("dependency closed early")
|
||||
}
|
||||
mu.Lock()
|
||||
defer mu.Unlock()
|
||||
closed = append(closed, 2)
|
||||
return nil
|
||||
})
|
||||
cancel()
|
||||
if r.WorkContext().Err() != nil || r.WorkContext().Value(struct{}{}) != "value" {
|
||||
t.Fatal("work context inherited cancellation or lost values")
|
||||
}
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- r.Finish(nil) }()
|
||||
for i := 0; i < 2; i++ {
|
||||
select {
|
||||
case <-started:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("stop callbacks did not run concurrently")
|
||||
}
|
||||
}
|
||||
if drains.Load() != 0 {
|
||||
t.Fatal("drain ran before stop barrier")
|
||||
}
|
||||
close(release)
|
||||
if err := <-done; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !reflect.DeepEqual(closed, []int{2, 1}) || r.WorkContext().Err() == nil {
|
||||
t.Fatal(closed)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPhasesFailuresRetainDependenciesAndCauses(t *testing.T) {
|
||||
for _, phase := range []string{"stop", "drain"} {
|
||||
for _, panics := range []bool{false, true} {
|
||||
t.Run(phase+map[bool]string{false: "Error", true: "Panic"}[panics], func(t *testing.T) {
|
||||
r := testRuntime(t)
|
||||
cause := errors.New("component cause")
|
||||
fatal := errors.New("fatal monitor")
|
||||
cleanup := atomic.Int32{}
|
||||
other := atomic.Int32{}
|
||||
r.OnClose(func() error { cleanup.Add(1); return nil })
|
||||
hook := func(context.Context) error {
|
||||
r.RequestShutdown(fatal)
|
||||
if panics {
|
||||
panic(cause)
|
||||
}
|
||||
return cause
|
||||
}
|
||||
if phase == "stop" {
|
||||
r.OnStop("broken", hook)
|
||||
} else {
|
||||
r.OnDrain("broken", hook)
|
||||
}
|
||||
r.OnDrain("other", func(context.Context) error { other.Add(1); return nil })
|
||||
err := r.Finish(nil)
|
||||
var detail *HookError
|
||||
if !errors.Is(err, ErrResourcesRetained) || !errors.Is(err, cause) || !errors.Is(err, fatal) || !errors.As(err, &detail) || detail.Phase != phase {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cleanup.Load() != 0 || other.Load() != 1 || r.WorkContext().Err() == nil {
|
||||
t.Fatal("unsafe cleanup or omitted drain")
|
||||
}
|
||||
if !errors.Is(r.Close(), ErrResourcesRetained) || other.Load() != 1 {
|
||||
t.Fatal("close retried hooks")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPhasesDeadlineCancelsWorkAndJoinsBeforeClose(t *testing.T) {
|
||||
r, err := NewRuntimeWithConfig(context.Background(), ShutdownConfig{DrainTimeout: 20 * time.Millisecond, CancelTimeout: time.Second})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var joined atomic.Bool
|
||||
var closed atomic.Bool
|
||||
r.OnDrain("worker", func(ctx context.Context) error {
|
||||
<-ctx.Done()
|
||||
<-r.WorkContext().Done()
|
||||
// Cleanup can still use shared resources after forced cancellation.
|
||||
if closed.Load() {
|
||||
return errors.New("dependency closed during cancellation")
|
||||
}
|
||||
joined.Store(true)
|
||||
return nil
|
||||
})
|
||||
r.OnClose(func() error {
|
||||
if !joined.Load() {
|
||||
t.Error("worker not joined")
|
||||
}
|
||||
closed.Store(true)
|
||||
return nil
|
||||
})
|
||||
err = r.Close()
|
||||
if !errors.Is(err, context.DeadlineExceeded) || errors.Is(err, ErrResourcesRetained) || !closed.Load() {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPhasesUncooperativeHookIsBoundedAndNeverClosesDependencies(t *testing.T) {
|
||||
for _, phase := range []string{"stop", "drain"} {
|
||||
t.Run(phase, func(t *testing.T) {
|
||||
r, err := NewRuntimeWithConfig(context.Background(), ShutdownConfig{StopTimeout: 20 * time.Millisecond, DrainTimeout: 20 * time.Millisecond, CancelTimeout: 20 * time.Millisecond})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
release := make(chan struct{})
|
||||
finished := make(chan struct{})
|
||||
var closed atomic.Int32
|
||||
hook := func(context.Context) error { defer close(finished); <-release; return nil }
|
||||
if phase == "stop" {
|
||||
r.OnStop("stuck", hook)
|
||||
r.OnDrain("stuck", func(context.Context) error { t.Error("Drain overlapped unfinished Stop"); return nil })
|
||||
} else {
|
||||
r.OnDrain("stuck", hook)
|
||||
}
|
||||
r.OnClose(func() error { closed.Add(1); return nil })
|
||||
began := time.Now()
|
||||
err = r.Finish(errors.New("listen failed"))
|
||||
close(release)
|
||||
<-finished
|
||||
if time.Since(began) > time.Second || !errors.Is(err, ErrResourcesRetained) || !errors.Is(err, context.DeadlineExceeded) || closed.Load() != 0 {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_ = r.Close()
|
||||
if closed.Load() != 0 {
|
||||
t.Fatal("late cleanup raced work")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestPhasesConcurrentFinishAndValidation(t *testing.T) {
|
||||
if _, err := NewRuntimeWithConfig(nil, ShutdownConfig{DrainTimeout: -1}); err == nil {
|
||||
t.Fatal("negative timeout accepted")
|
||||
}
|
||||
r := testRuntime(t)
|
||||
var called atomic.Int32
|
||||
r.OnStop("worker", func(context.Context) error { called.Add(1); return nil })
|
||||
r.OnDrain("worker", func(context.Context) error { called.Add(1); return nil })
|
||||
r.OnClose(func() error { called.Add(1); return nil })
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := r.Finish(nil); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if called.Load() != 3 {
|
||||
t.Fatal(called.Load())
|
||||
}
|
||||
for _, fn := range []func(){func() { r.OnStop("late", func(context.Context) error { return nil }) }, func() { NewRuntime(nil).OnDrain("bad\nname", func(context.Context) error { return nil }) }, func() {
|
||||
x := NewRuntime(nil)
|
||||
x.OnStop("duplicate", func(context.Context) error { return nil })
|
||||
x.OnStop("duplicate", func(context.Context) error { return nil })
|
||||
}} {
|
||||
func() {
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Error("invalid registration accepted")
|
||||
}
|
||||
}()
|
||||
fn()
|
||||
}()
|
||||
}
|
||||
}
|
||||
|
||||
func TestMissingDrainRetainsResources(t *testing.T) {
|
||||
r := testRuntime(t)
|
||||
r.OnStop("worker", func(context.Context) error { return nil })
|
||||
closed := false
|
||||
r.OnClose(func() error { closed = true; return nil })
|
||||
if err := r.Close(); !errors.Is(err, ErrResourcesRetained) || closed {
|
||||
t.Fatal("missing completion proof accepted", err)
|
||||
}
|
||||
}
|
||||
@@ -11,17 +11,42 @@ import (
|
||||
// must finish draining before calling Close. Embed *Runtime in the project's svc.
|
||||
// The logger and any other borrowed resources must not be registered here.
|
||||
type Runtime struct {
|
||||
Lifecycle *Controller
|
||||
mu sync.Mutex
|
||||
cleanups []func() error
|
||||
closed bool
|
||||
once sync.Once
|
||||
closeErr error
|
||||
Lifecycle *Controller
|
||||
mu sync.Mutex
|
||||
cleanups []func() error
|
||||
closed bool
|
||||
once sync.Once
|
||||
closeErr error
|
||||
stops, drains []shutdownHook
|
||||
shutdown ShutdownConfig
|
||||
work context.Context
|
||||
cancelWork context.CancelFunc
|
||||
}
|
||||
|
||||
func NewRuntime(ctx context.Context) *Runtime { return &Runtime{Lifecycle: New(ctx)} }
|
||||
func (r *Runtime) Context() context.Context { return r.Lifecycle.Context() }
|
||||
func (r *Runtime) RequestShutdown(err error) { r.Lifecycle.RequestShutdown(err) }
|
||||
func NewRuntime(ctx context.Context) *Runtime {
|
||||
r, _ := NewRuntimeWithConfig(ctx, DefaultShutdownConfig())
|
||||
return r
|
||||
}
|
||||
|
||||
// NewRuntimeWithConfig separates the stop notification from the lifetime of
|
||||
// accepted background work. Zero timeout fields use documented defaults.
|
||||
func NewRuntimeWithConfig(ctx context.Context, cfg ShutdownConfig) (*Runtime, error) {
|
||||
if err := cfg.Validate(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
work, cancel := context.WithCancel(context.WithoutCancel(ctx))
|
||||
return &Runtime{Lifecycle: New(ctx), shutdown: cfg.defaults(), work: work, cancelWork: cancel}, nil
|
||||
}
|
||||
func (r *Runtime) Context() context.Context { return r.Lifecycle.Context() }
|
||||
func (r *Runtime) RequestShutdown(err error) { r.Lifecycle.RequestShutdown(err) }
|
||||
|
||||
// WorkContext stays alive during HTTP draining and graceful background draining.
|
||||
// It is canceled on a shutdown failure/deadline, or after successful draining.
|
||||
// Context() remains the early stop notification; do not use it for accepted work.
|
||||
func (r *Runtime) WorkContext() context.Context { return r.work }
|
||||
|
||||
// OnClose transfers cleanup ownership. Register immediately after creation,
|
||||
// before exposing resources to other goroutines. Registration after Close is a
|
||||
@@ -38,8 +63,10 @@ func (r *Runtime) OnClose(cleanup func() error) {
|
||||
r.cleanups = append(r.cleanups, cleanup)
|
||||
}
|
||||
|
||||
// Close releases all resources in reverse order, including after a failed
|
||||
// initialization. All callbacks are attempted even if one fails or panics.
|
||||
// Close runs stop and drain barriers before releasing resources in reverse
|
||||
// order, including after failed initialization. If quiescence is unproven it
|
||||
// skips resource callbacks and returns ErrResourcesRetained. Once release is
|
||||
// safe, all OnClose callbacks are attempted even if one fails or panics.
|
||||
func (r *Runtime) Close() error {
|
||||
if r == nil {
|
||||
return nil
|
||||
@@ -48,9 +75,43 @@ func (r *Runtime) Close() error {
|
||||
r.RequestShutdown(nil)
|
||||
r.mu.Lock()
|
||||
r.closed = true
|
||||
cleanups := r.cleanups
|
||||
cleanups, stops, drains := r.cleanups, r.stops, r.drains
|
||||
r.cleanups = nil
|
||||
r.stops, r.drains = nil, nil
|
||||
r.mu.Unlock()
|
||||
stopErr := r.runPhase("stop", stops, r.shutdown.StopTimeout)
|
||||
var stopped *phaseError
|
||||
errors.As(stopErr, &stopped)
|
||||
for _, stop := range stops {
|
||||
found := false
|
||||
for _, drain := range drains {
|
||||
if drain.name == stop.name {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
stopErr = &phaseError{err: errors.Join(stopErr, &HookError{"drain", stop.name, errors.New("matching OnDrain required")}), unsafe: true}
|
||||
}
|
||||
}
|
||||
if stopErr != nil {
|
||||
r.cancelWork()
|
||||
}
|
||||
// A timed-out Stop can still be executing. Never overlap its own Drain;
|
||||
// other components can still establish their completion independently.
|
||||
eligible := make([]shutdownHook, 0, len(drains))
|
||||
for _, drain := range drains {
|
||||
if stopped == nil || !stopped.unjoined[drain.name] {
|
||||
eligible = append(eligible, drain)
|
||||
}
|
||||
}
|
||||
drainErr := r.runPhase("drain", eligible, r.shutdown.DrainTimeout)
|
||||
r.cancelWork()
|
||||
r.closeErr = errors.Join(stopErr, drainErr)
|
||||
if unsafePhase(stopErr) || unsafePhase(drainErr) {
|
||||
r.closeErr = errors.Join(r.closeErr, ErrResourcesRetained)
|
||||
return
|
||||
}
|
||||
for i := len(cleanups) - 1; i >= 0; i-- {
|
||||
r.closeErr = errors.Join(r.closeErr, runCleanup(cleanups[i]))
|
||||
}
|
||||
|
||||
@@ -44,7 +44,13 @@ python3 scripts/verifyConnector.py --local --report reports/businessDevelopmentC
|
||||
|
||||
## 远程发布验收
|
||||
|
||||
代码发布标签后,使用以下命令在空 module 缓存验收同版远程 CLI/框架,不使用本地 replace;结果另行归档,发布标签不移动:
|
||||
远程验收已通过。代码标签固定在 `f5478cdc7662704b164637649305361e52f7c598`,报告另行提交,标签没有移动。两个脚本分别使用空 module 缓存,同版远程 CLI/框架均无本地 replace:
|
||||
|
||||
- [生成服务远程报告](businessDevelopmentManagedRemote.json):30.00 秒,覆盖实际生成、运行/构建、错误目录和模板参数、根数组/null、严格请求、声明式上传、依赖中间件、退出与清理。
|
||||
- [数据库远程报告](businessDevelopmentConnectorRemote.json):22.99 秒,真实 MySQL TLS/mTLS、Connector/池隔离、HTTP 限额与生成 MySQL 服务的配置和生命周期。
|
||||
- 两者 module 校验值一致:`h1:qG0Ipjt3enc8/Enu8ihJ6h8c8U6FCDedoNSdzuC3rBc=`。临时容器、凭据、证书和消费者目录已清理。
|
||||
|
||||
复验命令:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyManagedRelease.py --version v0.14.0-preview --report reports/businessDevelopmentManagedRemote.json
|
||||
|
||||
74
reports/businessDevelopmentConnectorRemote.json
Normal file
74
reports/businessDevelopmentConnectorRemote.json
Normal file
@@ -0,0 +1,74 @@
|
||||
{
|
||||
"passed": true,
|
||||
"frameworkVersion": "v0.14.0-preview",
|
||||
"localSource": false,
|
||||
"frameworkSum": "h1:qG0Ipjt3enc8/Enu8ihJ6h8c8U6FCDedoNSdzuC3rBc=",
|
||||
"freshModuleCache": true,
|
||||
"noLocalReplace": true,
|
||||
"mysqlImage": "sha256:1afd25fe334a1be64a3866495dd43a6bdba3c55dd1d6190ea12933163fe2843c",
|
||||
"mysqlLimits": {
|
||||
"cpus": 1,
|
||||
"memoryMiB": 768,
|
||||
"pids": 128
|
||||
},
|
||||
"durationSeconds": 22.99,
|
||||
"tests": [
|
||||
"TestHTTPConfiguredStandardServer",
|
||||
"TestHTTPInvalidStartupReleasesResources",
|
||||
"TestHTTPConfiguredNetworkLimits/nethttp/slowHeaders",
|
||||
"TestHTTPConfiguredNetworkLimits/nethttp/slowBody",
|
||||
"TestHTTPConfiguredNetworkLimits/nethttp/idle",
|
||||
"TestConnectorIndependentPoolsAndContext",
|
||||
"TestConnectorRejectsInvalidInputWithoutTakingOwnership",
|
||||
"TestConnectorStartupFailuresReleaseResources/connect",
|
||||
"TestConnectorStartupFailuresReleaseResources/ping",
|
||||
"TestConnectorStartupFailuresReleaseResources",
|
||||
"TestHTTPConfiguredNetworkLimits/nethttp/writeBackpressure",
|
||||
"TestConnectorTimeoutBoundsConnectAndPing/connect",
|
||||
"TestConnectorTimeoutBoundsConnectAndPing/ping",
|
||||
"TestConnectorTimeoutBoundsConnectAndPing",
|
||||
"TestConnectorHealthBoundsPoolWaitAndRecovers",
|
||||
"TestConnectorIdleZeroAndLifetime/noIdle",
|
||||
"TestConnectorIdleZeroAndLifetime/expired",
|
||||
"TestConnectorIdleZeroAndLifetime",
|
||||
"TestConnectorTransactionsKeepCommitAndRollbackSemantics",
|
||||
"TestMySQLConnectorDriverLoggerIsolation",
|
||||
"TestMySQLConnectorTLSAndPoolIsolation/untrustedCA",
|
||||
"TestMySQLConnectorTLSAndPoolIsolation/wrongHostname",
|
||||
"TestMySQLConnectorTLSAndPoolIsolation",
|
||||
"TestStructuredMySQLAdapterTLS",
|
||||
"TestHTTPFileAndEnvironment/server.json",
|
||||
"TestHTTPFileAndEnvironment/server.yaml",
|
||||
"TestHTTPFileAndEnvironment",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/READ_HEADER_TIMEOUTinvalid",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/READ_HEADER_TIMEOUT-1",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/READ_TIMEOUTinvalid",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/READ_TIMEOUT-1",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/WRITE_TIMEOUTinvalid",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/WRITE_TIMEOUT-1",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/IDLE_TIMEOUTinvalid",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/IDLE_TIMEOUT-1",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/MAX_HEADER_BYTESinvalid",
|
||||
"TestHTTPInvalidAndUnsupportedSettings/MAX_HEADER_BYTES-1",
|
||||
"TestHTTPInvalidAndUnsupportedSettings",
|
||||
"TestHTTPConfiguredNetworkLimits/nethttp/headerLimit",
|
||||
"TestHTTPConfiguredNetworkLimits/nethttp",
|
||||
"TestHTTPConfiguredNetworkLimits/native/slowHeaders",
|
||||
"TestHTTPConfiguredNetworkLimits/native/slowBody",
|
||||
"TestHTTPConfiguredNetworkLimits/native/idle",
|
||||
"TestHTTPConfiguredNetworkLimits/native/writeBackpressure",
|
||||
"TestHTTPConfiguredNetworkLimits/native/headerLimit",
|
||||
"TestHTTPConfiguredNetworkLimits/native",
|
||||
"TestHTTPConfiguredNetworkLimits",
|
||||
"TestGeneratedMySQLConfigurationAndLifecycle"
|
||||
],
|
||||
"sourceHashes": {
|
||||
"database/connector_test.go": "0663dfc9794ba24130225fa3b6418a6d5f75d57d0f01686151f70c7ea3f592c0",
|
||||
"database/mysqlConnector_test.go": "e1cdd11ef76b09f1332ba92bfdde5b1917d84c85cc42fac48f65a8b23f5e9720",
|
||||
"app/httpConfig_test.go": "b060dc0b76fb8e5feebae7e3b12cb7bca845693fcc097e2a0496a5ba937c837a",
|
||||
"config/http_test.go": "0cae3dcf213369cfaff5487bf630c2bc3ec097739ca1d40794124e69c3880db8",
|
||||
"database/mysql/integration_test.go": "05b0a73d50363eef760e8a3324a7e9a68c598f7519c6360ad530503178b56f43",
|
||||
"scripts/verifyConnector.py": "4d2b1da0a1fcf641e02600710789ced51a58cf6fa1726a00c194a01f8e557e1a",
|
||||
"scripts/testdata/mysqlGenerated_test.go": "5630123add5ed17f261c1a1bf63ba773f7fa6ea613fd573d5c5cb72da3e42645"
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user