Compare commits

...

14 Commits

Author SHA1 Message Date
96e7d9629b feat: simplify HTTP client proxy pools and authenticated JSON calls 2026-09-26 15:28:19 +08:00
068e85132f test: record remote configuration release acceptance 2026-09-26 14:49:28 +08:00
c60841f901 feat: unify generated configuration loading and extensible decoders 2026-09-26 14:47:33 +08:00
91a6beb2f3 test: verify remote copyx release and update usage reference 2026-09-25 17:31:48 +08:00
fd83235ce7 feat: replace mapping generation with checked copy utilities 2026-09-25 17:27:28 +08:00
ad9257d202 test: record remote nullable mapping release acceptance 2026-09-25 13:58:45 +08:00
4563a6354f feat: release nullable mappings and split mapping configuration 2026-09-25 13:56:44 +08:00
5fc82509a1 test: record remote generation experience release acceptance 2026-09-25 12:45:29 +08:00
8b588a24cd feat: release generic composition mapping and middleware factories 2026-09-25 12:43:34 +08:00
8493f9dae9 test: record remote contract tooling release acceptance 2026-09-25 02:23:58 +08:00
64e3bd6cfc feat: support primitive response arrays and shared catalog tooling 2026-09-25 02:22:24 +08:00
1f2d8b6a6a test: record remote phased lifecycle release acceptance 2026-09-25 00:37:21 +08:00
16deead70e feat: add generic phased background shutdown to managed services 2026-09-25 00:35:59 +08:00
adcb6f60b6 test: record remote optional GORM release acceptance 2026-09-24 22:27:21 +08:00
156 changed files with 11604 additions and 1717 deletions

8
AGENTS.md Normal file
View File

@@ -0,0 +1,8 @@
# 开发约定
- 框架保持通用,不内置接入项目的业务类型、字段、错误码或业务规则。
- 2026-09-25 用户要求:每次功能开发完成,必须同步维护根 README.md。README 需要覆盖当前全部公开功能,解释用途、基本用法,并提供可运行示例或链接到 examples 内的完整示例。
- 新增、调整和删除功能都必须同步文档、示例、测试与发布说明。删除功能要明确说明已移除内容、影响范围和迁移步骤,不能继续把历史用法当作当前能力推荐。
- examples 使用通用场景,避免依赖业务项目、私有数据库或线上服务。必要的外部基础设施配置应明确说明。
- 2026-09-25 用户决定:移除 mappings/mappingFiles 及结构映射代码生成机制,改用可选 copyx 工具包。不得恢复另一套 JSON 或 Go 映射声明语言。
- 深拷贝的自定义转换结果不得与输入共享可变引用;发现共享应报错,不能降级成浅拷贝或静默接受。

View File

@@ -1,5 +1,32 @@
# Changelog
## v0.21.0-preview(2026-09-26)
- httpc 可配置环境/直连/固定代理、常用连接阶段超时和独立空闲池参数。
- JSON 新增逐请求 Header 快照与响应大小选项,保持有界读取、关闭和认证隔离。
- 成功请求少复制和退避计算,丢弃响应流式处理,错误体有界回收以改善连接复用。
- 增加真实本地代理、超时、连接限制/回收、并发、日志脱敏、gzip、fuzz 和修改前后基准。
- JSON 自定义方法接口签名需同步 variadic 参数;借用 Transport 的参数混配现在报错。详见 httpc/README.md。
## v0.20.0-preview(2026-09-26)
- 生成项目配置加载入口持续由生成器维护;保留业务字段、默认值、校验与 NewLoader 扩展文件。
- config.LoadProject 统一 JSON/YAML/XML、环境覆盖和校验;提供实例级 Decoder 注册。
- 文件/字段/环境/行列的安全诊断,Invalid/AtField 与原始错误链。
- 增加格式等价、精度、null/空数组、并发、资源限制、默认值隔离、真实生成项目与性能验收。
- 旧手写 Load 必须显式迁移;低层 API 保留。固定数组严格长度、YAML alias/merge/custom tag 拒绝。用法及限制见 CONFIG.md。
## v0.19.0-preview(2026-09-25)
- 新增通用可选 copyx:构造器、浅/深拷贝、精确类型转换、字段规则、nil/零值策略、并发复用。
- 深拷贝拒绝自定义结果共享源可变引用;原子提交目标、保留错误链、限制遍历及估算分配。
- 移除 JSON 结构映射配置与生成;提供明确迁移诊断,保留公共字段组合等其他生成能力。
- 同步 README 全部能力索引、COPY.md、可运行示例和开发文档约定。
- 验收与不兼容迁移见 [发布说明](RELEASE.md)、[copyx 验收](reports/copyxAcceptance.md)。v0.14–v0.18 的完整变更保留在发布说明历史段落。
## v0.13.0-preview(2026-09-24)
- 新增通用 lifecycle.Controller,svc.RequestShutdown 支持依赖主动通知正常停止或致命故障,通知不阻塞,首个故障原因保留;生成入口排空 HTTP 后关闭依赖与日志,故障退出码为 1。

178
CONFIG.md Normal file
View File

@@ -0,0 +1,178 @@
# 统一配置加载
v0.20 起,新生成服务的 `main.go` 调用生成器维护的 `config.Load`。项目只需要定义业务字段、默认值和校验;日志、HTTP、数据库等配置使用同一流程,配置加载时不打开数据库、监听端口或创建日志文件。
```text
代码默认值 → envDefault(仅填零值)→ 可选配置文件
→ 完整结构体环境覆盖(含别名)→ 框架规范化 → 项目 Validate
→ 初始化日志及项目依赖 → 启动服务
```
## 生成项目要维护什么
| 文件 | 维护方 | 用途 |
| --- | --- | --- |
| `main.go` | 生成器 | 启动、信号、请求排空、依赖关闭 |
| `internal/config/loadGen.go` | 生成器 | 调用统一加载入口;纳入 `generate --check` |
| `internal/config/config.go` | 项目 | `Config`、`Default()`、`Validate()` |
| `internal/config/loader.go` | 项目 | `NewLoader()`,只有增加文件格式时才需修改 |
| `etc/config.yaml` | 项目 | 部署配置;可改用 JSON/XML 并传 `-config` |
| `internal/svc` | 项目与生成器分文件 | 初始化依赖、登记清理、AppOptions |
比如增加业务的批次数量:在 Config 增加一个字段,在 Validate 加范围校验;需要部署值时改配置文件或环境变量。不修改 main、Load 或 Handler。
```go
// 添加到项目 Config。
BatchSize int `json:"batchSize" envDefault:"100"`
// 添加到项目 Validate。
if c.BatchSize < 1 || c.BatchSize > 1000 {
return omniConfig.Invalid("batchSize", "must be between 1 and 1000")
}
```
自动支持 `GO_OMNI_BATCH_SIZE`。生成服务的空环境值策略为 `ignore`,0 和 false 是有效覆盖;范围由 Validate 决定。
重新生成保留 config.go、loader.go 和 svc 的手写扩展。
## 手写项目用法
```go
package main
import (
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
type Config struct {
Timeout time.Duration `json:"timeout" envDefault:"3s"`
Limit int `json:"limit" envDefault:"100"`
Token string `json:"-" env:"APP_TOKEN,secret"`
}
func (c Config) Validate() error {
if c.Timeout <= 0 {
return config.Invalid("timeout", "must be positive")
}
return nil
}
// cfg, err := config.LoadProject("etc/config.yaml", Config{},
// config.EnvOptions{Prefix: "APP_", Empty: "reject"})
```
`LoadProject` 接收结构体默认值,可实现值接收器或指针接收器的 `Validate() error`。空路径表示不读文件。
`DecodeFile` 只做文件覆盖,不做 envDefault、环境覆盖或业务 Validate;旧的两参数调用仍有效,可用结构体或结构体指针。低层 `ApplyDefaults`、`OverlayEnv`、`LoadFile` 继续存在。
嵌套校验用 `config.AtField("client", err)` 加路径前缀。公共原因通过 `config.Invalid` 提供,原因中不要包含密码、DSN、Token 或输入原值。
## JSON / YAML / XML
按文件扩展名选择解析器:`.json`、`.yaml`、`.yml`、`.xml`,大小写不敏感。所有格式使用 Go 结构体的 **json 字段名**,无需为 XML 复制一份结构或标签。
以下配置等价:
```json
{"client":{"timeout":"2s"},"enabled":false,"ports":[8080,8081],"optional":null}
```
```yaml
client:
timeout: 2s
enabled: false
ports: [8080, 8081]
optional: null
```
```xml
<config>
<client><timeout>2s</timeout></client>
<enabled>false</enabled>
<ports type="array"><item>8080</item><item>8081</item></ports>
<optional type="null"/>
</config>
```
XML 必须使用一个 `<config>` 根;普通叶节点按目标 Go 类型转换。数组显式 `type="array"` 加 `<item>`,不根据重复元素猜测类型。`type="array"` 空元素表示空数组,`type="object"` 表示空对象,`type="null"` 表示 null。`any` 中的 XML 叶节点默认是字符串,需要动态数字/布尔时标注 `type="number"` / `type="bool"`;也可用 `type="string"`。
类型和覆盖规则:
- 缺失字段保留默认值;显式 0、false、空字符串正常覆盖。
- null 清空指针、map、slice、interface;普通值字段保持默认值,沿用 Go JSON 解码语义。需要区分 null 时用指针。
- 数组整体替换;空 slice 保持非 nil;固定长度 Go 数组要求长度一致,避免丢弃多余配置。
- map 合并键,已提供的键整体覆盖对应值;map 中未提供的键保留。
- duration 支持 `250ms`、`3s`;整数保持精度,动态 JSON 数字使用 `json.Number`,不先转 float64。
- 自定义 `json.Unmarshaler` / `encoding.TextUnmarshaler` 仍可接入。这些回调是可信项目代码,必须自己保证并发和副作用安全;不要修改共享外部状态。
- JSON `,string` 字段、嵌入字段遵循 JSON 字段选择;未知字段、重复绑定、冲突均拒绝。
内置解析器限制:单文件最多 1 MiB,解析最多 64 层、100000 个节点;绑定和默认值复制也有深度限制。只接受普通文件,不读取设备或命名管道。不支持多文档、重复字段、无效 UTF-8;YAML 拒绝 alias/merge/custom tag,XML 拒绝 DTD/外部实体、命名空间、业务属性和混合内容。XML 仅支持配置所需的子集,不是通用 XML 文档映射器。
这些限制针对启动配置,避免意外超大文件、别名扩展和模糊覆盖。自定义值解码器的内部字段校验由该解码器负责;例如现有错误目录使用其专有解码规则。
## 环境覆盖
统一入口的 EnvOptions 控制前缀、空值策略、别名和 Lookup。完整 JSON 路径自动转换为 UPPER_SNAKE_CASE:`client.timeout` → `APP_CLIENT_TIMEOUT`。
```go
config.EnvOptions{
Prefix: "GO_OMNI_", Empty: "ignore",
Aliases: config.ServerEnvAliases("GO_OMNI_", "server"),
}
```
生成项目同时接受完整的 `GO_OMNI_SERVER_HTTP_READ_TIMEOUT` 与短别名 `GO_OMNI_HTTP_READ_TIMEOUT`。完整名存在时优先,即便其空值被 ignore,也不回退别名。不会再二次调用 ApplyEnv 覆盖业务字段。
显式 `env:"CUSTOM_NAME"` 是绝对名称;`env:"-"` 跳过子树。`json:"-"` 只在声明显式 env 名时参与,可让密钥只能从环境加载。`secret` 标签表达用途,解析错误对所有字段都不打印值。slice/map 的环境值采用 JSON。`required` 检查字符串非空、指针/集合非 nil;复杂范围由 Validate 负责。详见 [完整业务环境覆盖](DEVELOPMENT.md#完整业务配置的环境覆盖)。
显式环境选择日志 stdout/stderr 会清除文件 profile 的 File/Rotation,除非同时提供非空 File 覆盖。该兼容规则由框架处理。
## 扩展解析器
内置格式直接可用。新增格式只在 `internal/config/loader.go` 的 `NewLoader` 注册:
```go
func NewLoader() (*omniConfig.Loader, error) {
return omniConfig.NewLoader(
omniConfig.WithDecoder("custom", omniConfig.DecoderFunc(
func(data []byte) (omniConfig.Document, error) {
// 演示注册扩展名;实际可在这里调用项目选定的 TOML 等解析库。
return (omniConfig.JSONDecoder{}).Decode(data)
},
)),
)
}
```
Decoder 返回 Document:Value 是 map[string]any 根,子值为 string/bool/json.Number/nil/[]any/map[string]any;无自身类型的叶节点可返回 config.Text。Positions 可选,用 `client.timeout` 或 `servers[0].timeout` 标记 1-based 行列。不提供位置时仍有文件、字段、原因。
自己的语法错误可返回 `&config.Error{Field: "client.timeout", Line: 3, Column: 5, Reason: "expected a duration"}`。Reason 必须安全;普通第三方错误仅公开 `invalid configuration syntax`,内部 cause 通过 errors.Is/As 保留。
注册表属于 Loader 实例,无全局 init 注册;不能覆盖已有扩展名。Loader 可共享,但 Decoder 必须支持并发且不保留/修改输入。解析插件是可信 Go 代码,框架不是插件沙箱。新格式依赖由选择它的项目显式引入,核心不强制引入 TOML 等库。
## 定位配置错误
```text
config "etc/config.yaml":4:12 field "client.timeout": expected a duration such as 5s
config env "APP_CLIENT_TIMEOUT" field "client.timeout": expected a valid time.Duration
```
`errors.As(err, &diagnostic)` 可获取 `*config.Error` 的 File/Field/Env/Line/Column/Reason;原 `DecodeError` 名称保留为别名。`errors.Is` 可判断文件不存在等 I/O 原因。
解析失败显示文件和可获取的行列;Validate 使用 Invalid/AtField 才能定位业务字段。环境覆盖导致的错误优先标环境变量,不错误地归因到文件;数组元素验证也会追溯到覆盖它的集合环境变量。默认值未出现在文件里时没有行列。YAML 语法错误通常只有行号,列显示 1。
不打印底层错误原文或配置值;任意普通 `errors.New(...)` 校验失败的公开原因是 `validation failed`,内部原因仍在链中。项目应改用 Invalid 给出安全的原因。第三方子配置校验若没有字段元数据,只定位到对应子树,框架不会从任意错误文案猜字段。
不要把 Unwrap 后的原始错误、整个配置对象写入公开日志/HTTP;自定义解析器与校验器传入的 Reason 由项目负责脱敏。
## v0.19 及更早项目迁移
先提交现有代码,再升级匹配的 CLI 与框架。旧项目的 `internal/config/config.go` 含手写 Load,新生成器会明确阻止 generate/check/dry-run,不会直接覆盖它。
1. 保留 Config、Default、Validate、业务配置结构及辅助函数。
2. 删除旧 Load。普通的 defaults/file/env/Validate 流程已由 LoadProject 接管;移除由此不再使用的 import。将自定义默认值和校验分别迁入 Default / Validate,自定义文件格式迁入 NewLoader。
3. 框架只在 `loader.go` 不存在时创建 NewLoader;已有项目文件同名时需要保证它提供同签名函数。无需手写 loadGen.go。
4. 重新生成,执行 `generate --check`、`go test ./...`、`go build .`,再用真实配置验证启动。
保留业务需要的显式 env 标签/规则;如果原 Load 有解密或其他项目特殊步骤,不要直接删除,应先明确其归属,必要时在自定义 Decoder 中处理。生成器不自动搬动或猜测手写行为。`api upgrade` 仍可预览较早版本的基础设施差异,但已修改的配置必须按上述步骤迁移。
不会移除低层配置 API;本次移除的是**新项目里重复手写的加载流程**,不删除旧项目的业务文件。
完整可运行例子:[examples/configuration](examples/configuration/README.md)。

141
COPY.md Normal file
View File

@@ -0,0 +1,141 @@
# 可选拷贝工具 copyx
v0.19 用普通 Go 调用取代 JSON 结构映射生成。`copyx` 只依赖标准库,不需要注册到 App,不执行 JSON 往返,也不会自动暴露数据库模型。普通 Logic 直接使用 `.api` 生成的 DTO;只有存在独立业务类型或需要隔离可变数据时才用它。
完整可运行代码:[examples/copying](examples/copying)。在框架根目录运行:
```sh
go run ./examples/copying
go test -race ./copyx ./examples/copying
```
## 最简单的调用
```go
import "git.yuntuo.hk/k2safe/go-omni/copyx"
type Request struct { Name string; Enabled bool }
type Input struct { Name string; Enabled bool }
req := Request{Name: "sample", Enabled: false}
var input Input
if err := copyx.Copy(&input, req); err != nil { return err }
```
同名、同类型字段直接赋值。默认不会跳过 `false`、`0`、空字符串。目标字段没有来源时报错;多余源字段默认忽略,使用 `WithStrictFields()` 可要求源字段也全部使用或显式忽略。
默认是浅拷贝:同类型指针、map、slice 共享引用,符合普通 Go 赋值。JSON tag 不参与字段匹配。不会自动调用方法、SQL Scanner/Valuer,或把 `int32`、`int64`、string、命名标量互相强转。
新字段:普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时,两边各声明一次;同名同类型不增加拷贝配置。检查在运行时发生,实际类型对必须有测试,不能把运行时工具当作旧生成器的编译期字段覆盖检查。
## 深拷贝与构造器
```go
snapshot, err := copyx.Clone(source) // 保持源的静态类型,始终深拷贝
if err != nil { return err }
copier, err := copyx.New(copyx.WithDeepCopy(), copyx.WithStrictFields())
if err != nil { return err }
var result Output
if err := copier.Copy(&result, source); err != nil { return err }
```
`Copy` 的目标必须是非 nil 指针;`Clone[T]` 返回 `(T, error)`。重复执行时在初始化阶段创建 Copier 并复用;配置之后不可变,每次调用独立持有遍历状态,可以并发使用。源、目标的并发读写仍由调用方同步,自定义函数也须支持并发。
深拷贝会分配指针、map、slice,递归处理结构、数组、接口中的具体值。nil 集合与非 nil 空集合保持区别;slice 结果容量为长度,不复制长度之外的隐藏元素。普通循环和重复指针可处理。循环图应保留根指针,使用 `Clone(sourcePointer)` 或 `Copy(&targetPointer, sourcePointer)`;将循环指针根压成值结构会明确报错。
它保证结果不引用输入中的可变对象,不保证保留所有底层内存布局关系:不同重叠 slice 视图会分别复制,内部指针指向某个切片元素的关系也不重建。字符串和零尺寸值可安全共享;`time.Time` 按 Go 不可变值使用(Location 不做独立复制)。不要通过 unsafe 或 uintptr 隐藏可变引用后要求工具识别。
## 字段差异与自定义转换
```go
type Input struct { Name string; UpdatedAt time.Time; InternalID string }
type View struct { DisplayName string; UpdatedAt string; LocalNote string }
copier, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithStrictFields(),
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
copyx.WithIgnoreSource[Input, View]("InternalID"),
copyx.WithIgnoreTarget[Input, View]("LocalNote"),
copyx.WithConverter(func(v time.Time) (string, error) {
return v.UTC().Format(time.RFC3339), nil
}),
)
if err != nil { return err }
out := View{LocalNote: "preserve"}
if err := copier.Copy(&out, input); err != nil { return err }
```
字段规则的类型参数必须是结构类型,字段名使用导出的 Go 字段名。构造时检查不存在的字段、重名及冲突。不支持字符串路径表达式;复杂嵌套差异为该嵌套类型注册 converter,或直接写 Go。
`WithIgnoreSource` 只是声明该源字段不参与使用;目标同名字段若仍需保留,还需 `WithIgnoreTarget`。忽略目标会保留原目标值,Clone 中则为零值。深拷贝时如果保留值仍与源共享可变引用,也会拒绝。
转换器的签名是 `func(S) (T, error)`,按**精确类型对**匹配,不能重复注册。金额、编号、日期格式由项目明确转换。需要逐字段不同规则时,给整个源/目标结构注册函数;函数返回独立的新目标。无需另一份映射配置或全局注册。
精确 converter 优先于 nil/零值策略,能收到 typed nil;例如 `func(*Money) (Money, error)` 可自行决定 nil 是否允许。绑定到 `any` 字段的 converter 能接收 nil interface,但顶层未带类型的 `nil` 没有精确类型可匹配。浅拷贝中已经直接共享的同类型容器不会继续逐项转换其内部;需要遍历内部时选择深拷贝,或为该容器类型注册 converter。
## 深拷贝不能原样返回可变引用
```go
copier, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithConverter(func(v []string) ([]string, error) {
return v, nil // 会返回 ErrSharedReference
}),
)
```
检查覆盖完整源图与转换结果,包括新父对象引用旧子对象、原 map、重叠子切片、`source[:0]` 的可写容量、指向源数组元素的指针,以及未导出字段中的引用。会检查每个 converter 的结果和最终目标,不允许静默降级为浅拷贝。
返回相同字符串、整数并不会报错,它们不是可变引用。正确切片 converter 可以写:
```go
copyx.WithConverter(func(v []string) ([]string, error) {
if v == nil { return nil, nil }
out := make([]string, len(v))
copy(out, v)
return out, nil
})
```
第三方 Decimal/自定义 Money 如果含未导出的指针,默认深拷贝拒绝;注册该类型的 converter,使用它自己的构造/Clone API 创建独立值,避免转成 float64 损失精度。框架不内置某一金额库的规则。无法独立复制的类型直接返回错误。
## nil、零值和错误
| 选项 | 行为 |
| --- | --- |
| 默认 `NilError` | nil → 可空目标保留 nil;nil → 非可空目标报 `ErrNil` |
| `WithNilPolicy(NilZero)` | 显式允许 nil → 零值;可空目标仍为 nil |
| `WithNilPolicy(NilSkip)` | nil 保留已有目标;新集合元素没有旧目标,保留元素零值 |
| `WithSkipZero()` | 显式跳过 Go 零值,包括 `false`、`0`、`""`;非 nil 空集合不是 Go 零值 |
所有 nil 策略都不把不兼容类型自动变成兼容类型。非 nil 指针与值之间可以按对应元素转换;nil 到非可空目标仍需选择策略。局部更新不能默认使用 NilZero/SkipZero,它们可能改变清空字段或布尔值的业务语义。
工具只能复制绑定后的 Go 值:如果输入类型已经将“缺失”和“JSON null”合并为 nil,无法恢复两者区别。需要三态时,应先设计包含 presence 信息的类型和绑定逻辑。
赋值使用临时结果,成功后才替换目标;转换失败不会交付部分结果。`*copyx.Error` 包含 Go 字段/索引路径,`errors.Is/As` 保留内部错误链,工具不主动在错误中加入源值。converter 自己返回的错误文本仍由项目负责。
converter 必须只读输入;其外部副作用、输入修改无法回滚,panic 原样传播。框架只检查与本次输入的共享引用,无法证明 converter 未返回外部全局缓存的引用。它是受信 Go 代码扩展,不是可运行不可信代码的沙箱。
## 支持范围与资源限制
- 支持结构体、指针、interface、slice、同长度 array、map。map key 转换后冲突或不可比较则报错。
- 导出匿名结构字段可按提升后的无歧义名称访问;源嵌入指针为 nil 且需要其字段时明确报错。跨类型目标包含私有字段时拒绝;同类型私有纯值可作为整体保留。
- 不克隆文件、数据库连接、网络连接、锁、原子状态。已知标准库资源/同步结构按值复制也拒绝;深拷贝中的非 nil func、chan、unsafe.Pointer 拒绝。其他封装资源类型应由项目排除,工具不能识别所有自定义句柄。
- 单次遍历深度上限 256、节点上限 200,000;内置分配的估算预算为 64 MiB,超限返回 `ErrLimit`。预算不是进程堆内存硬限制,reflect 元数据和 converter 自行分配不计入。大对象流式读写应使用 IO/存储接口。
- 只缓存不可变类型元数据,不缓存请求值、用户 converter 或实例策略。没有 `unsafe` 写入、JSON 往返、隐式数值截断或方法调用。
- 这是运行时反射工具,会比直接 Go 赋值多检查与分配。热点应复用 Copier,并用真实数据测量;性能记录见 [本版验收](reports/copyxAcceptance.md)。
## 从旧 mapping 迁移
v0.19 **彻底移除**映射配置加载、Go 类型加载、映射代码生成及 `pointerResult` 等附属选项。公共 `.api` 字段组合、scalar、中间件工厂、Repository 开关继续存在。
1. 先检查每个转换是否必要。普通 Logic 可直接使用生成 DTO;独立核心保留自己的类型。
2. 将 `mapping.Xxx(...)` 调用改为普通 Go 或小型 `copyx` helper。字段改名、nil、金额规则写在该 helper 同一处。指针返回可直接 `var out *Response; err := copier.Copy(&out, input)`。
3. 用真实类型对测试缺失/null/零值/空数组、字段覆盖、金额精度和错误链。需要源字段新增时提醒遗漏,启用 `WithStrictFields()` 并测试。
4. 从 `omni.gen.json` 删除 `mappings` / `mappingFiles`,删除无其他用途的映射 JSON 文件。
5. 确认已无调用,再删除 `internal/mapping/mappingsGen.go`。如之前手改过文件,先迁出手写内容。生成器遇到该旧文件会阻止生成并提示迁移,**不会静默删除**;其余业务文件照常保留。
6. 升级框架与 CLI 到同版,执行 generate、generate --check、go test ./...;多个服务再执行 workspace generate/check。新 manifest 自动去掉已移除文件的所有权记录。
历史发布记录保留旧能力说明,仅用于理解旧版本,不代表 v0.19 继续支持。

View File

@@ -1,7 +1,9 @@
# 通用业务开发指南(v0.14.0-preview)
# 通用业务开发指南(v0.21.0-preview)
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
公共字段组合、自动中间件工厂和 Repository 开关,见 [生成能力指南](GENERATION.md)。必要的结构转换使用可选 [copyx](COPY.md),v0.19 已移除旧 JSON 映射生成。
## 返回数组、可空对象和 null
```text
@@ -21,12 +23,17 @@ delete /item
| 契约 | Logic 返回 | 成功数据 |
| --- | --- | --- |
| `returns (Item)` | `(*types.Item, error)` | 对象;nil 被视为实现错误 |
| `returns ([]Item)` | `([]types.Item, error)` | 数组;nil 统一输出 `[]` |
| `returns ([]Item)` | `([]types.Item, error)` | 对象数组;nil 统一输出 `[]` |
| `returns ([]string)` | `([]string, error)` | 字符串数组 |
| `returns ([]int64)` / `([]bool)` | `([]int64, error)` / `([]bool, error)` | 数值 / 布尔数组 |
| `returns ([]any)` | `([]any, error)` | 动态 JSON 数组,元素允许对象、数组、标量和 null |
| `returns (*Item)` | `(*types.Item, error)` | 对象或 `null` |
| `returns (null)` | `error` | HTTP 200 和空业务数据 |
| 不写 returns | `error` | HTTP 204,无响应体 |
统一响应中的空数据是否显示,仍遵循项目模板的 `dataMode`;需要 `data:null` 就设置 `dataMode: always`。`@response raw` 输出业务数据本身,包括 `[]`、对象或 `null`。可用 `@status 201/202`;204 不能声明响应类型。根类型当前支持以上四种形式,嵌套字段继续支持既有 slice/map/pointer/scalar/any。
统一响应中的空数据是否显示,仍遵循项目模板的 `dataMode`;需要 `data:null` 就设置 `dataMode: always`。`@response raw` 输出业务数据本身,包括 `[]`、对象或 `null`。可用 `@status 201/202`;204 不能声明响应类型。嵌套字段继续支持既有 slice/map/pointer/scalar/any。
基础类型数组还支持 int/int8/int16/int32、uint/uint16/uint32/uint64、float32/float64。nil 根数组都输出 `[]`;`[]any` 在 OpenAPI 中是 `type: array, items: {}`,不限制元素形状,但元素必须可被标准 JSON 编码器序列化。`[]uint8`(Go 的 byte slice)会编码为 base64,故明确拒绝;数字数组可用 `[]int32`,二进制用下载接口。当前不支持根嵌套数组或根自定义 scalar 数组。
## 契约驱动的请求策略
@@ -95,7 +102,9 @@ defer reader.Close()
## 完整业务配置的环境覆盖
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML 文件 → 全结构体环境覆盖 → 既有框架环境别名 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
生成器维护 `internal/config/loadGen.go`;项目维护 Config/Default/Validate 和可选 NewLoader。三种格式、可扩展解析器、字段错误定位及旧 Load 迁移见 [统一配置指南](CONFIG.md)。
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML/XML 文件 → 全结构体环境覆盖(含别名)→ 规范化 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
```go
type ClientConfig struct {
@@ -113,6 +122,23 @@ duration 使用 `250ms`、`3s`;slice/map 使用 JSON,如 `[1,3,5]`、`{"retr
手写项目可使用 `config.LoadProject(path, defaults, config.EnvOptions{Prefix:"MY_APP_"})`,或分别调用 `ApplyDefaults/DecodeFile/OverlayEnv`。读取失败不修改输入默认值;诊断不会打印输入值。不要主动打印含密钥的整个配置,或把底层解析错误展开到公开响应。Validate 不打开日志文件、数据库或存储连接;证书的可读性和握手信任在依赖初始化阶段验证。
### 一次覆盖完整配置与 HTTP/日志短名称
```go
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{
Prefix: "MY_APP_",
Empty: "reject", // allow / ignore / reject,项目自行选择
Aliases: config.ServerEnvAliases("MY_APP_", "server"),
// Lookup 可注入测试或项目配置源;默认 os.LookupEnv。
})
```
不要再紧接着调用 `ApplyEnv(cfg.Server, ...)`。上面的调用同时处理完整配置和 HTTP_ADDR、SERVICE、LOG_LEVEL 等框架短名称。例如 `MY_APP_SERVER_ADDRESS` 与 `MY_APP_HTTP_ADDR` 指向同一字段,**完整名称优先**;未设置时才读取短名称。完整名称显式设空仍优先:allow 交给该字段解析(string 可清空;数字等仍要求合法值),ignore 保留文件/默认值,reject 报错。不会因 ignore 再退回别名。字段 `env` 标签的 empty 规则优先于全局规则。
自定义别名用 `EnvOptions.Aliases`,键是完整环境名、值是按优先级排列的别名数组。两种名称都使用同一 Lookup、空值策略和类型转换;布尔 false、数值 0 与未设置不同。集合始终使用 JSON,包括 `MY_APP_TRUSTED_PROXIES='["127.0.0.1"]'`。
新项目模板一次调用 OverlayEnv,默认 `Empty: "ignore"`。旧 `ApplyEnv` 保留原有忽略空值、修剪空白及 TRUSTED_PROXIES 逗号列表行为。已有项目的 `internal/config` 属于开发者文件,升级不覆盖它;按上例合并原来的两次覆盖调用后,保留项目自己的 Validate。日志输出的显式完整名称也优先于短名称。
## 依赖、日志与中间件
`internal/svc/runtimeGen.go` 持续由生成器维护。项目的 ServiceContext 嵌入 `*lifecycle.Runtime`,只在 `Init(ctx)` 中装配额外依赖:
@@ -123,40 +149,15 @@ func (c *ServiceContext) Init(ctx context.Context) error {
if err != nil { return err }
c.Client = client
c.OnClose(client.Close)
c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(client)
return nil
}
```
每个资源初始化成功就登记清理;后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求,再逆序关闭依赖,最后关闭日志。关闭幂等,单个清理函数 panic/返回错误不会跳过其他清理;初始化 panic 清理后继续传播。项目清理函数仍须自行停止、等待后台任务并保证能返回,框架不能强制杀死任意 goroutine。
每个资源初始化成功就登记清理;后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求,再执行后台组件的 Stop/Drain 阶段,确认收尾后逆序关闭依赖,最后关闭日志。关闭幂等,单个清理函数 panic/返回错误不会跳过其他清理;初始化 panic 清理后继续传播。后台组件按 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭) 登记;无法确认收尾完成时保留共享依赖并异常退出,不能把取消等同于任务已经结束。
中间件用业务所需的小接口接收依赖,避免反向导入 svc:
新项目使用中间件工厂:项目维护 `middleware.Dependencies` 小接口与 `New<Name>(deps) (httpx.Middleware, error)`,svc 提供对应访问器。生成的 runtime 在 Init 成功后自动调用工厂并注册,不再按名字替换 map;工厂失败会释放已经登记的依赖。示例与现有项目迁移步骤见 [中间件工厂](GENERATION.md#中间件工厂自动注册)。
```go
// internal/middleware/accessFactory.go
package middleware
import (
"context"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
type AccessChecker interface {
CheckAccess(context.Context, string) error
}
func NewRequireAccess(checker AccessChecker) httpx.Middleware {
return func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
return err
}
return next(c)
}
}
}
```
`.api` 声明对应 middleware 后,保留生成的初始 hook,并另建上述工厂;在 Init 用工厂替换 registry 的同名项即可。该包只依赖接口和 HTTP 抽象,不导入 svc,也不把数据库藏到 context 里。
v0.16 项目及 `omni.gen.json` 中选择 `middleware: handlers` 的项目继续使用原有 `Defaults()`,可以在 Init 设置 `c.Middlewares["RequireAccess"]`。更新时不会重写这部分业务代码;只有显式切换到 factories 后才使用新流程。
生成入口为每个 App 注入独立 Logger,不修改进程默认 Logger。业务可用 `c.Logger.InfoContext(...)`,或请求链中的 `logx.InfoContext(ctx,...)`。标准库/第三方需要 `*log.Logger` 时使用 `c.Logger.Standard(slog.LevelWarn)`;适配器借用原 Logger,不负责关闭。手写多 App 用 `omni.WithLogInstance(logger)` 显式注入。
@@ -185,21 +186,9 @@ go-omni api generate --check
格式化保留注释、token 和多行字面量,规范四空格缩进、空行、行首尾空白及相邻单行字段的类型/标签/行尾注释对齐;结果幂等。先校验完整契约,check 不写文件;导入的共享目录不被某个服务的格式化命令修改。DTO 按源 `.api` 输出 `dto<Source><摘要>Gen.go`,都属于 `types` 包,跨文件引用无需改包名。
Go Catalog 是错误的唯一来源时,显式导出文档元数据:
Go Catalog 是错误的唯一来源时,优先使用 workspace 的显式导出配置,不再维护导出/复制脚本。AppOptions 仍通过 `omni.WithErrors(apierrors.Catalog())` 注册同一个 Go 函数,运行期只注册一次。单个项目也可继续使用 `errors.ExportCatalog` 导出入口目录的 `api/omni.errors.json`;此文件和下面的共享 source 二选一。
```go
// 在项目自己的导出命令中;appErrors.Catalog 与 AppOptions 注册同一函数。
if err := omnierrors.ExportCatalog(os.Stdout, appErrors.Catalog()); err != nil {
log.Fatal(err)
}
```
```sh
go run ./cmd/exportErrors > api/omni.errors.json
go-omni api generate
```
文件是派生产物,应纳入版本控制并由业务 CI 检查导出漂移。CLI 自动读取入口目录的 `omni.errors.json`,校验错误目录、合并状态/模板参数/具体示例到 OpenAPI;不会运行项目 Go,不创建另一份运行期注册器。带独立错误目录的路由继续使用自己的目录。没有路由级精确错误列表时,文档描述 App 目录的公共错误集合;特殊自定义 Encoder 的文档由项目显式扩展。
OpenAPI 自动把有收益的重复响应、schema 和示例移到 `components.responses/schemas/examples`,通过标准 `$ref` 引用。HTTP 200 同时含成功和业务失败时保留两种 schema、业务码、模板参数示例和 data:null;业务响应内容不会改变。消费 OpenAPI 的工具需要支持标准本地引用。
## 两个服务共享契约
@@ -241,9 +230,57 @@ go-omni workspace check --file omni.workspace.json
各服务保持独立 module;可显式填写各服务的 `api`(相对服务目录)和 `module`。所有服务预检通过后再写;每个服务独立提交文件事务,不承诺跨项目的崩溃原子性。check 汇总格式与生成漂移;业务编译/测试仍在各 module 执行。
### 同一份 Go 错误目录供多个服务生成
在 workspace 清单显式声明提供目录的 Go 包和函数:
```json
{
"errorCatalogs": [{
"dir": ".",
"package": "example.com/project/internal/apierrors",
"function": "Catalog",
"output": "contracts/common/errors.json"
}],
"services": [{"dir":"services/api"}, {"dir":"services/id"}]
}
```
`dir` 相对 workspace,必须包含 go.mod;函数签名为 `func Catalog() errors.Catalog`。允许 module 内的 internal 包。先准备 output 的父目录。生成器构建一个临时导出程序,使用 Go overlay、临时 go.mod/go.sum 和 `-mod=readonly`,不会往项目增加导出命令或修改 go.mod/go.sum。最多 16 个提供方,每个有 90 秒构建/执行总时限和 1 MiB stdout 限制。错误消息不回显程序输出。
只有显式运行 **workspace generate/check** 会执行清单指定的 Go 包及其依赖初始化;不要指向不信任的包。提供目录的函数应只构造目录,不连接数据库、启动服务、写 stdout 或修改项目。框架不扫描其他包,也不通过 shell 执行配置字符串。普通 api validate/export/generate 和 workspace format 只读取现有元数据。
各服务在自己的 `api/omni.imports.json` 引用共享 JSON:
```json
{
"roots": {"common":{"path":"../../../contracts/common","version":"1.0.0"}},
"errors": {
"source": "@common/errors.json",
"include": ["USERNAME_INVALID", "INTERNAL_ERROR"],
"routes": {"GET /permissions":["INTERNAL_ERROR"]}
}
}
```
`include` 省略或 null 表示全部目录,`[]` 表示不列业务错误;routes 按 **大写方法 + 空格 + .api 原路径**(如 `GET /users/:id`)覆盖服务级列表。错误标识、路由拼错或重复标识会明确报错。路由已经通过 `.api @errors` 声明独立运行期目录时,不能再给它设置文档 routes 列表。
**这些列表只决定文档展示,不修改 App 错误目录、路由继承或运行期可返回的错误。** `.api` 不需要重复定义 Go 错误表。状态、业务码、提示和模板参数仍来自同一 Catalog;特殊自定义 Encoder 的文档由项目显式扩展。
```sh
# 首次生成导出一次共享目录,所有服务预检通过才写入
# 共享 JSON 不复制到各服务;已有 api/omni.errors.json 请显式移除后改用 source。
go-omni workspace generate --file omni.workspace.json
go-omni workspace format --file omni.workspace.json
go-omni workspace generate --file omni.workspace.json
go-omni workspace check --file omni.workspace.json
```
共享 JSON 应纳入版本控制。check 在内存重新导出并比较 JSON,然后检查所有服务;缺失或过期时失败,不更新项目文件。Go 构建缓存仍可能更新;显式提供方的自有副作用不属于生成器可隔离范围。业务 CI 仍执行各 module 的编译与测试。目录函数本身有变化时重新 generate,受控共享根的固定 SHA256 如需改变仍由项目显式更新。
## v0.13 项目升级
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.14.0-preview(或显式引用当前源码)。
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.20.0-preview(或显式引用当前源码)。
```sh
go-omni api upgrade --dir . --dry-run

116
GENERATION.md Normal file
View File

@@ -0,0 +1,116 @@
# 公共字段、中间件工厂与生成选项
本页对应 `v0.19.0-preview`。CLI 与框架依赖固定到同一版本,安装见 [README](README.md#准备环境与安装)。
## 一份项目生成配置
新项目根目录的 `omni.gen.json` 只描述生成选项:
```json
{
"repository": "none",
"middleware": "factories"
}
```
- `repository: none` 默认不创建空 Repository 文件;`group` 按 group 补齐可编辑骨架。已有文件始终保留;切回 none 不会删除业务文件。SQL、ORM 和仓储划分由项目决定。
- `middleware: factories` 是新项目默认值,自动调用带依赖的工厂。`handlers` 保留原有 `Defaults()` 及手动注册方式,供既有项目迁移。
- 该文件与 `.api` 一起提交。generate/check/dry-run 和 workspace 采用相同规则;validate/export/format 只处理接口契约。
- 旧 manifest 没有配置时补 `middleware: handlers`、`repository: none`。已有配置不能直接删除,避免意外切换中间件模式。
- v0.19 已删除 `mappings` 和 `mappingFiles`;遇到旧配置会明确报错,不读取子映射文件或执行映射生成。处理步骤见 [copyx 迁移](COPY.md#从旧-mapping-迁移)。
## 公共字段组合
```text
type EmployeeFields {
// 员工姓名。
Name string `json:"name" binding:"max=80"`
Salary Money `json:"salary"`
}
type EnterpriseRequest {
EmployeeFields
}
type AdminRequest {
EmployeeFields
CompanyID string `path:"companyId"`
}
```
`Money` 由项目按既有 scalar 语法声明。组合行只有已声明的对象类型名,不加指针或字段 tag。公共对象可以通过已有相对 import 或声明过的共享根 `@common/...` 引入。
生成器展开为普通 Go 字段。JSON 保持平铺;字段的绑定来源、校验、注释、可空性和 OpenAPI 使用同一份展开结果。不会在 JSON 中增加 `EmployeeFields` 一层。重复 Go 字段、重复来源字段名、循环组合、未知对象和过深展开均失败;即使重复字段类型相同,也不能隐式覆盖。要改变某个字段的规则,应拆出更小的公共组合。
允许多层组合,上限 32 层、单类型 4096 字段、全契约展开 65536 字段。Go 匿名指针嵌入并非本功能;生成 DTO 中不会使用匿名嵌入。
## DTO 与业务类型
普通 Logic 直接接收生成的 Request、返回生成的 Response,不额外创建相同结构。只有独立核心或明确不同的输入/输出职责才保留业务类型。
必要转换直接写 Go,或用可选 `copyx.Copy(&target, source)`。同名同类型字段自动赋值,字段改名和类型变化通过 Go 参数、转换函数明确设置,见 [完整示例](examples/copying/main.go)。无需 JSON 映射配置、包路径字符串或额外生成文件。
`copyx` 在运行时检查类型和字段,**不提供旧映射生成阶段的静态字段覆盖保证**;CI 应给实际类型对添加测试。普通业务字段新增通常只改 `.api` 和 Logic;独立 DTO/领域类型都需要此字段时,两边各加一次,同名同类型无需追加拷贝配置。
## 中间件工厂自动注册
`.api` 沿用 `middleware: RequireAccess` 或 auth 对应的中间件声明。新项目会生成可编辑的 `internal/middleware/requireAccessFactory.go`,只需实现一次:
```go
// internal/middleware/dependencies.go,由项目维护。
type AccessChecker interface {
CheckAccess(context.Context, string) error
}
type Dependencies interface {
AccessChecker() AccessChecker
}
// internal/middleware/requireAccessFactory.go,由项目维护。
func NewRequireAccess(deps Dependencies) (httpx.Middleware, error) {
checker := deps.AccessChecker()
if checker == nil { return nil, errors.New("access checker is required") }
return func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil {
return err
}
return next(c)
}
}, nil
}
// internal/svc/access.go,由项目维护;ServiceContext.Init 初始化 c.Auth。
func (c *ServiceContext) AccessChecker() middleware.AccessChecker { return c.Auth }
```
编译器检查 ServiceContext 是否实现依赖接口,middleware 不导入 svc。多个工厂可以共用这些小接口。所需 imports 和具体鉴权服务仍属于项目。
启动顺序为:框架基础依赖 → `svc.Init` → 生成的 `middleware.New(c)` 调用各具名工厂 → 创建 App/注册路由 → 监听。中间件实例创建一次,由请求并发复用;不要把单次请求信息存在实例字段里。工厂之间没有初始化顺序依赖;路由执行顺序仍以 `.api` 声明为准。
工厂返回错误或 nil 中间件均阻止监听;错误链保留,已登记资源按现有生命周期清理。工厂 panic 也先清理依赖再传播。资源初始化和清理登记放在 svc.Init,工厂借用资源;HTTP 请求排空之前不会关闭这些依赖。
factory 模式不再在 Init 写 `Middlewares["..."]`;残留非空手写注册表会明确报错,防止生成注册静默覆盖它。未实现的工厂让启动失败;未实现的 Logic 仍返回 501。
### 既有项目迁移
1. 首次更新保持 `middleware: handlers`,现有 hook 和手写 Configure 不变。
2. 改为 factories 并重新生成,会补 `dependencies.go` 和 `*Factory.go`,保留原来的手写文件。
3. 将旧实现迁入工厂,提供 Dependencies 的访问器;移除 Init 中按名字覆盖 map 的代码。
4. 旧函数不再需要时由项目删除。若新工厂符号与旧文件中的同名函数冲突,整理旧文件;生成器不覆盖业务代码。
5. `generate --check` 与 `go test ./...` 检查后再运行。工厂没有全部实现前,服务不会启动。
## 文件维护边界
| 项目维护 | 生成器持续维护 |
| --- | --- |
| `.api` 公共字段与接口 | 展开后的 DTO、Handler、OpenAPI |
| `omni.gen.json` 的 Repository/中间件选项 | 对应骨架和注册代码;已有手写骨架不覆盖 |
| 独立 Go 业务类型、scalar、可选 copyx 调用和测试 | 不生成结构转换文件 |
| middleware Dependencies、具名工厂、svc 访问器 | 中间件 registry、启动调用 |
| Logic、Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 |
独立三端工资契约测试位于 `cmd/go-omni/apigen/testdata/composition`,不导入任何接入项目。它验证公共字段、Money scalar、三端 Handler、copyx、重新生成保留业务内容,以及新增字段不需要更新转换规则。
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1
```

View File

@@ -6,6 +6,24 @@
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
## 请求追踪与基础指标
`omni.New()` 的默认链包含 RequestID、访问日志(可配置开关)和 Recovery;无需重复挂载。业务通过 `c.Context()` 传递 requestId,`logx.FromContext(ctx)` 取得该 App 的 Logger。手动组装 httpx 链时,也可使用 `middleware.RequestID("X-Request-ID")`、`middleware.AccessLog()` 和 `middleware.Recovery()`。
轻量进程指标直接使用 `metrics`,无需 OTel collector:
```go
// import omni "git.yuntuo.hk/k2safe/go-omni"
// import "git.yuntuo.hk/k2safe/go-omni/metrics"
app := omni.New()
measurements := metrics.New()
app.Use(measurements.Middleware())
app.GET("/metrics", measurements.Handler())
// 也可以直接读取 measurements.Snapshot()。
```
记录请求数、失败数、处理中数量、累计耗时与 HTTP 状态分布;返回的是 JSON 快照,并非 Prometheus 文本格式。指标接口是否暴露以及访问权限由项目配置。需要 HTTP/SQL/Redis spans 和标准 OTel exporter 时,使用 [telemetry 示例](examples/platform/main.go)。CORS 由项目中间件或代理配置,当前没有内置 CORS 模块。
## 缓存:cache 与 cache/redis
```go
@@ -52,30 +70,26 @@ application.OnShutdown(client.Close)
```go
client, err := httpc.New(httpc.Config{
Timeout: 3*time.Second, MaxRetries: 2, MaxConnsPerHost: 32,
MaxResponseBytes: 1 << 20,
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
Timeout: 3*time.Second, DialTimeout: time.Second,
TLSHandshakeTimeout: time.Second, ResponseHeaderTimeout: time.Second,
MaxConnsPerHost: 32, MaxIdleConnsPerHost: 16, MaxIdleConns: 64,
IdleConnTimeout: 60*time.Second, MaxResponseBytes: 1<<20,
})
if err != nil { return err }
application.OnShutdown(client.Close)
err = client.JSON(ctx, "GET", serviceURL, nil, &result)
err = client.JSON(ctx, "GET", serviceURL, nil, &result,
httpc.WithHeader("Authorization", "Bearer "+token),
)
```
客户端应在服务启动时创建并复用。默认使用独立的 `net/http.Transport` 连接池,每个目标主机
有连接上限;该上限不是整个进程的全局连接数上限。也可以注入自有 `http.RoundTripper`,
其连接限制和生命周期由调用方负责。这样出站实现同样可以替换。
启动时创建并复用 Client。代理可选环境、直连、固定地址;不配置时保留标准库原代理策略。JSON 支持单次请求头和响应限额,不保存 Client 全局 Token。新阶段超时与空闲连接参数可选,0 沿用默认。
总超时覆盖建连、重试、等待和读取响应体。`Do` 返回后调用方必须读取/关闭 `response.Body`;
`JSON` 自动关闭,并限制解码前读取的字节数。`Do` 允许流式读取,本身不限制响应体大小。
`Close` 拒绝新请求、关闭自有池的空闲连接;在途请求应先排空。
成功响应读取有界;output=nil 流式丢弃,不整体缓存。小错误体只在大小/时间预算内尝试读完以复用连接;未知或大错误体及时关闭。Do 保留标准流式 Request/Response,由调用者限量读取并关闭 body。Close 拒绝新调用并关闭自有池空闲连接,项目先排空在途工作。
`MaxRetries` 默认 0,最多 5。封装只对可重放的 GET/HEAD/OPTIONS 重试网络错误及
429/502/503/504;POST/PUT/PATCH/DELETE 不做应用层自动重试。标准库 Transport 和借用的
RoundTripper 自身的重放规则仍生效,不能据此承诺“只执行一次”。退避带抖动,尊重
`Retry-After`;服务器要求等待超过 `MaxBackoff` 时直接返回该响应,不提前重试。
重定向交给业务处理,3xx 不会自动跟随。
自定义 Transport 借用、不修改、不关闭;不能同时指定池/代理/阶段选项。总超时持续覆盖 HTTP 等待、重试和读取 body,但不抢占 JSON 编解码或不遵守 context 的自定义代码。MaxRetries 默认 0,只对可重放 GET/HEAD/OPTIONS 重试;3xx 不跟随。标准库和借用 Transport 自身仍可能重放,不能据此承诺业务只执行一次。
默认从 context 传递 `X-Request-ID`,日志只包含方法、主机、状态、尝试次数和收到响应头的耗时,
不记录完整 URL、查询参数、凭证和响应体。非 2xx 的 JSON 调用返回只带状态码的 `StatusError`。
完整参数、NO_PROXY、TLS、安全日志、重试、签名兼容说明见 [httpc 指南](httpc/README.md),可运行 [HTTP Client 示例](examples/httpClient/README.md)。
## 请求保护:middleware

View File

@@ -1,6 +1,6 @@
# 生成项目与文件维护边界
v0.14.1-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
v0.21.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
## 创建和日常开发
@@ -38,27 +38,32 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
这是轻量源码分析,不运行项目 init、不执行 Go 命令、不下载依赖。项目自定义类型别名、嵌入方法或 build tags 等无法可靠判断时,会给 REVIEW 并交给 Go 编译器验证;不把这些情况强行判为签名错误。它不检查方法体是否引用已删除字段,也不证明业务逻辑正确。CI 中应同时执行 `go-omni api generate --check` 和 `go test ./...`,按项目实际构建参数测试。当前版本首次接手缺少快照的 manifest v2 时,会明确提示建立基线,不猜测旧契约;Logic 签名检查仍会执行。
生成选项、公共字段组合和可选 Go 拷贝工具,详见 [生成能力指南](GENERATION.md)。
## 哪些文件由用户维护
| 文件 | 首次生成后由项目维护 |
| --- | --- |
| `api/*.api` | 路由、DTO、具名中间件、响应模板、请求/响应模式 |
| `etc/config.yaml` | 运行参数;密钥通过环境变量提供 |
| `internal/config/config.go` | 自定义配置字段、加载、默认值与校验 |
| `internal/config/config.go` | 自定义配置字段、默认值与校验 |
| `internal/config/loader.go` | 需要额外文件格式时注册 Decoder |
| `internal/svc/serviceContext.go` | 依赖字段和初始化,持有加载后的 Config |
| `internal/svc/options.go` | AppOptions、GlobalMiddleware;注册 Go 错误目录和全局扩展 |
| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 |
| `internal/svc/storage.go` | 选用存储时生成,之后保留 |
| `internal/logic/<group>/*.go` | 业务方法,使用标准 context.Context |
| `internal/repository/<group>/*.go` | 数据访问,按需实现 |
| `internal/middleware/<name>.go` | 项目鉴权/权限等具名中间件实现 |
| `internal/repository/<group>/*.go` | 数据访问;默认不补空骨架,可选 group 模式 |
| `internal/middleware/*Factory.go`、`dependencies.go` | 中间件工厂与依赖小接口,svc 实现访问器 |
| `omni.gen.json` | Repository 和中间件生成选项;不再声明结构映射 |
| 可选的 Go 转换函数 / copyx 调用 | 只有独立类型确实需要转换时维护,见 [copyx](COPY.md) |
| 项目 Go 错误包 | 自行维护 Catalog 和常量,可跨服务共用 |
| 自定义标量文件、`*Bind.go`、`*Manual.go` | 只有显式选择相应扩展时才生成/使用 |
| 项目测试、迁移、部署脚本、go.mod/go.sum | 项目所有,生成器不重置其内容 |
以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。
生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
生成器持续管理:`internal/config/loadGen.go`、`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。
@@ -85,7 +90,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
}
```
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。静态 OpenAPI 不执行 Go;可从同一 Catalog 显式导出 `api/omni.errors.json`,详见 [元数据导出](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`,可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录;独立目录不与 App 合并。普通 `.api` 编译不执行 Go;可在 workspace 显式配置 Catalog 函数,统一导出共享文档元数据,也可沿用 `api/omni.errors.json`,详见 [元数据导出](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。
`GlobalMiddleware()` 提供全局中间件;`Middlewares` 映射提供 `.api` 声明的具名中间件。全部显式调用,不依赖项目 init 注册,也不要求 svc 导入 handler。框架请求链按原有次序运行。
@@ -93,7 +98,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
最终配置校验包含自定义响应编码器与错误目录的组合。码类型/映射冲突、编码器已绑定另一份目录等问题在创建 App 前返回错误,进入入口的清理流程,不等到构造 App 时才 panic。
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后按框架规则强制关闭连接。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,在 OnClose 中取消。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后按框架规则强制关闭连接。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,使用 WorkContext 并登记成对的 OnStop/OnDrain,详见 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭)。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
## 依赖主动申请退出
@@ -106,10 +111,12 @@ c.RequestShutdown(err)
这只是通知,立即返回,不等待 HTTP 排空,也不直接关闭数据库。生成入口停止接收新请求,等待在途请求完成;超过 `shutdownTimeout` 后取消请求并关闭连接,再执行 svc.Close,最后关闭日志。非空故障原因保留在返回错误链中,进程退出码为 1;`RequestShutdown(nil)` 或正常 SIGINT/SIGTERM 在没有其他故障时退出码为 0。
多个依赖可并发通知,关闭流程仅执行一次,首个非空故障原因保留。正常信号已开始排空时收到的故障也不会被清理动作覆盖。初始化期间的通知同样有效,不依赖 HTTP 监听已经启动。业务应在 Close/OnClose 中停止并等待自己启动的监控任务,再结束资源清理;关闭完成以后才报告的故障无法参与已经结束的进程结果。
多个依赖可并发通知,关闭流程仅执行一次,首个非空故障原因保留。正常信号已开始排空时收到的故障也不会被清理动作覆盖。初始化期间的通知同样有效,不依赖 HTTP 监听已经启动。后台监控任务应通过 OnStop/OnDrain 停止并等待,再进入资源清理;关闭完成以后才报告的故障无法参与已经结束的进程结果。
`Lifecycle *lifecycle.Controller` 是 svc 持有的通用控制器,初始化时创建,生成入口使用它的 Context。该 Context 表示服务正在停止,不代表数据库已经关闭。不能把它当作在途请求的 Context,也不要在它取消时立刻关闭请求仍依赖的资源。发号器失去租约后拒绝发号等业务安全规则仍由项目实现。
配置加载入口 `internal/config/loadGen.go` 也由生成器维护。业务扩展 Config/Default/Validate;只有新格式才需要修改 NewLoader。JSON/YAML/XML、错误定位和旧 Load 迁移见 [配置指南](CONFIG.md)。
## 配置内置接口
生成项目在 `etc/config.yaml` 明确列出以下默认配置;可以独立关闭、修改路径,或让健康接口返回原始 JSON:
@@ -208,7 +215,7 @@ get /download (DownloadReq)
## 已有服务接入步骤
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.14.1-preview:
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.21.0-preview:
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。

147
README.md
View File

@@ -2,16 +2,106 @@
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
**从这份 README 开始接入。** 当前版本 **`v0.14.1-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**从这份 README 开始接入。** 当前版本 **`v0.21.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。
普通接口的流程是:**修改 `.api` → 生成 Request/Response → 写 Logic → 测试**。有独立职责的业务类型才需要转换;可直接写 Go,也可以按需使用 [copyx](COPY.md)。公共字段组合、中间件工厂、Repository 开关继续支持,见 [生成能力指南](GENERATION.md)。
Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。
新增通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
契约支持 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录,以及统一环境变量别名覆盖。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。
已有通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
后台组件可在 svc 登记 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭),先停止接收工作并等待收尾,再关闭共享资源;无需绑定队列或 ORM。
使用 GORM 的项目可以选择 [gormx 适配器](database/gormx/README.md),复用数据库初始化和连接池,并将 SQL 日志接入 logx。
## 当前功能与使用入口
下表是当前公开能力的索引。每项给出调用入口和完整说明;模块按需使用,`omni.New()` 不会自动初始化数据库、Redis、ORM、队列或云存储。示例中的 import 统一以 `git.yuntuo.hk/k2safe/go-omni/` 开头。
| 功能与用途 | 怎么用 | 解释与示例 |
| --- | --- | --- |
| 项目、接口与文档生成 | `go-omni new`;改 `.api` 后 `api generate`;`--check` 检查漂移 | [下方快速开始](#新项目生成并运行)、[CLI](cmd/go-omni/README.md)、[完整生成项目](examples/managedService/README.md) |
| 公共字段与跨服务共享契约 | `.api` 组合对象字段;`omni.imports.json` 声明共享根;workspace 批量生成 | [公共字段示例](GENERATION.md#公共字段组合)、[共享契约与 workspace](DEVELOPMENT.md) |
| 中间件依赖工厂、可选 Repository | `omni.gen.json` 选择 factories / none;在 `New<Name>(deps)` 写一次实现 | [配置与完整工厂示例](GENERATION.md) |
| 路由、分组、请求上下文 | `omni.New()`、`app.GET` / `Group` / `Use`;业务使用 `context.Context` | [基本服务](examples/basic/main.go)、[已有项目接入](#已有项目按模块接入) |
| HTTP 网络与扩展适配 | 配置 transport 为 nethttp / native / http2;自定义 Router/Runner | [网络层配置与契约](transport/README.md)、[性能与限制](PERFORMANCE.md) |
| HTTP 老接口桥接 | 通过官方 Handler/Middleware 适配器接入标准 `net/http`,跟踪实际响应状态 | [接入说明](MIGRATION.md)、[桥接/上传/自定义 ID 示例](examples/compatibility/main.go) |
| 请求校验、动态 JSON、自定义标量 | `.api` 声明来源/校验/scalar;手写契约用 `contract.New[T]`;简单绑定可用 `validate.JSON/Query/URI` | [语法与例子](API.md)、[请求策略](DEVELOPMENT.md) |
| 数组、null、原始请求、上传下载 | `.api` 声明数组/可空返回、body/multipart 与大小限制;必要时使用 Bind/Manual 扩展 | [契约示例](examples/contracts)、[含上传/回调/下载的项目](examples/managedService/README.md) |
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
| 文件配置与环境覆盖 | 默认值→JSON/YAML/XML→环境→校验;完整业务结构用 `config.LoadProject[T]` / 环境别名 | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时;JSON 支持单次请求头/限额;`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
| 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
## 带认证的出站 HTTP
HTTP Client 在 svc 初始化一次,业务重复使用。普通内部请求可直接选择直连和连接参数,无需手写 Transport:
```go
client, err := httpc.New(httpc.Config{
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
Timeout: 3*time.Second, DialTimeout: time.Second,
TLSHandshakeTimeout: time.Second, ResponseHeaderTimeout: time.Second,
MaxConnsPerHost: 32, MaxIdleConnsPerHost: 16, MaxIdleConns: 64,
})
if err != nil { return err }
// 在 svc 生命周期登记 client.Close;关闭前先排空使用它的工作。
err = client.JSON(ctx, "GET", endpoint, nil, &result,
httpc.WithHeader("Authorization", "Bearer "+token),
httpc.WithMaxResponseBytes(64<<10),
)
```
请求头只作用于本次调用,不把用户 Token 留在共享 Client。默认代理/参数保持原行为;也可选择环境代理或固定代理。JSON 自动关闭并限制响应,Do 保留下载/流式读取扩展。完整配置、重试边界和已有自定义 Transport 的迁移见 [httpc](httpc/README.md)。
```sh
go run ./examples/httpClient
```
## 统一配置加载
新生成项目的加载流程由框架管理。业务在 `internal/config/config.go` 增加字段、默认值和 `Validate`,无需改 main 或复制环境变量解析。JSON、YAML、XML 共用字段和校验;扩展格式在 `loader.go` 注册。
```go
// 项目 Config 中增加字段,自动支持 GO_OMNI_BATCH_SIZE:
BatchSize int `json:"batchSize" envDefault:"100"`
// 项目 Validate 中增加业务范围检查:
if c.BatchSize < 1 {
return omniConfig.Invalid("batchSize", "must be positive")
}
```
错误包含文件、字段、可用的行列或环境变量名;输入原值不自动写入错误。
```sh
go run ./examples/configuration -format xml
DEMO_CLIENT_TIMEOUT=500ms go run ./examples/configuration
```
详见 [统一加载、解析器、迁移](CONFIG.md) 和 [完整示例](examples/configuration/README.md)。**旧项目须先迁移手写 Load,再重新生成**;生成器会保留业务配置并拒绝冲突,不静默覆盖。
## 选择接入方式
| 你的项目 | 从哪里开始 |
@@ -32,7 +122,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
```sh
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.21.0-preview
export PATH="$(go env GOPATH)/bin:$PATH"
go-omni --help
```
@@ -73,21 +163,22 @@ hello/
├── api/service.api # 维护接口定义,也可 import 其他 .api
├── main.go # 生成器维护:启动、路由注册、优雅退出
├── etc/config.yaml # 项目配置
├── omni.gen.json # 项目维护的生成选项与可选映射
├── internal/
│ ├── config/ # 配置读取
│ ├── svc/ # 数据库、日志、外部客户端等依赖
│ ├── types/*Gen.go # 按 .api 文件生成 DTO
│ ├── handler/ # 生成的绑定、响应策略、路由及文档
│ ├── logic/greet/ # 在这里实现业务逻辑
│ ├── repository/greet/ # 在这里实现数据访问
│ └── middleware/ # 声明鉴权等中间件后生成对应 hook
│ ├── repository/ # 按需创建,默认不生成空骨架
│ └── middleware/ # 声明鉴权等中间件后生成工厂
├── docs/openapi.json # 生成的 OpenAPI 3.0.3
└── .goOmni/apiManifest.json # 生成文件所有权与校验信息,一起提交 Git
```
修改 `api/service.api` 后重新生成,业务写在 `internal/logic`,数据访问写在 `internal/repository`,依赖在 `internal/svc` 装配。声明业务错误后还会生成 `internal/apierrors/errorsGen.go`。
`api new` 的问候接口可以直接运行;自定义接口新生成的 Logic 和鉴权 hook 返回 **501**,需要实现后才可用。生成器不会根据 HTTP 字段猜数据库表或业务规则。
`api new` 的问候接口可以直接运行;自定义接口未实现的 Logic 返回 **501**。新项目的具名中间件工厂未实现时直接阻止启动;v0.16 的原有 handler 模式仍返回 501。生成器不会根据 HTTP 字段猜数据库表或业务规则。
### 从自己的接口定义生成
@@ -188,7 +279,7 @@ go-omni new --api catalog.api \
在业务项目的 **go.mod 所在目录**执行:
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview
go get git.yuntuo.hk/k2safe/go-omni@v0.21.0-preview
```
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
@@ -312,7 +403,7 @@ auth UserAuth (
)
```
生成器会创建 `internal/middleware/auth.go`,由项目实现凭据校验和身份注入。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的 hook 返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。
新项目生成 `internal/middleware/authFactory.go`,由项目实现 `NewAuth(deps)`;生成器负责初始化和注册。v0.16 或 handler 模式继续使用 `internal/middleware/auth.go`。凭据校验和身份注入由项目实现。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的工厂阻止启动,原有 hook 模式返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。
业务错误可以集中定义,业务只选择发生了哪个错误。**HTTP 状态由项目配置,业务失败可以返回 HTTP 200**;框架不根据 HTTP 状态单独判断成功与否。
@@ -382,6 +473,38 @@ return apierrors.TextTooLong("企业名称", 100)
更详细的配置、作用域和升级步骤见 [统一错误目录](errors/README.md)。
### 可选结构转换与深拷贝
普通接口不需要任何拷贝配置,直接使用生成的 Request/Response。独立业务结构之间有转换需求时:
```go
// import "git.yuntuo.hk/k2safe/go-omni/copyx"
var input DomainInput
if err := copyx.Copy(&input, req); err != nil { return err }
// 源与目标之间的指针、map、slice 需要隔离时:
snapshot, err := copyx.Clone(source)
if err != nil { return err }
```
默认浅拷贝,同名同类型字段自动赋值,保留零值;默认不会把 nil 填成非可空目标的零值。不要为使用工具新增一套重复模型。
需要重复使用或自定义规则时创建 Copier:
```go
copier, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
copyx.WithConverter(func(v time.Time) (string, error) {
return v.UTC().Format(time.RFC3339), nil
}),
)
if err != nil { return err }
if err := copier.Copy(&view, input); err != nil { return err }
```
自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错;字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md),完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查,类型对需测试;深拷贝不适用于连接、锁等资源。
### 数据库、存储及其他模块
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
@@ -456,9 +579,9 @@ go-omni api generate --check
go test ./...
```
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
CLI 与业务 module 固定使用 `v0.14.1-preview`。v0.14.0 项目直接升级依赖与 CLI,GORM 按需导入;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
CLI 与业务 module 固定使用 `v0.21.0-preview`。v0.14–v0.19 项目先按 [配置入口迁移](CONFIG.md#v019-及更早项目迁移) 整理旧 Load,再升级依赖与 CLI 并重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
## 按需深入
@@ -474,3 +597,7 @@ CLI 与业务 module 固定使用 `v0.14.1-preview`。v0.14.0 项目直接升级
| 网络性能、测试条件与适用范围 | [性能说明](PERFORMANCE.md)、[HTTP/2 验收](transport/native/HTTP2.md) |
当前版本可用于固定版本接入与业务验证,仍标记为 preview。`.api` 借鉴 go-zero 的接口开发方式,支持范围以本仓库文档为准,不是完整 goctl 语法兼容层;HTTP/2 保持实验状态,OSS/R2 云账号实网验收不属于本次发布范围。
## 后续开发约定
每次功能开发完成,同步更新这份功能索引、用法说明、`examples/` 和发布记录;删除或改变功能必须注明影响与迁移。代码、示例和文档一起回归后再发布。约定已记录在 [AGENTS.md](AGENTS.md),后续开发继续遵循。

View File

@@ -1,4 +1,173 @@
# v0.14.1-preview
# v0.21.0-preview
让带认证的 HTTP 调用和内部直连可以直接使用 httpc,减少手写 Transport 与 JSON 响应处理。
- Proxy 支持环境、直连、固定代理;默认保留既有策略。
- 建连、TCP keepalive、TLS 握手、响应头、100-continue、空闲超时和独立空闲连接数量可选配置;不修改全局 Transport,不关闭借用实例。
- JSON 支持 WithHeader / WithHeaders / WithMaxResponseBytes;认证信息逐请求隔离,自动关闭和有界响应处理保留。
- 成功路径减少请求复制和无用退避;output=nil 按限额流式丢弃;小错误体在大小和时间预算内尝试复用连接。
- 不新增依赖、不增加业务规则;Do 保留标准流式扩展。CLI 最新版本与模板最低版本分开,本轮模板兼容门槛仍是 v0.20。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.21.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.21.0-preview
```
**兼容注意**:JSON 旧的直接调用仍有效,自定义 interface/函数值需要同步新增的 variadic 参数。自定义 Transport 与显式连接/代理/阶段参数混用现在报错,避免参数被忽略;迁移时在自己的 Transport 配置,或改用内置字段。没有删除原调用入口。
[参数与使用说明](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[场景与性能验收](reports/httpcAcceptance.md) 已同步到 README。48 个测试包全量 race、Go 1.22 兼容、vet/build、约 110 万次请求头 fuzz 通过。性能来自本机同条件基准,不能代替项目实际流量验证;仍为 preview。
---
# v0.20.0-preview(历史版本)
统一配置加载,减少生成项目重复维护的启动代码。
- 生成器维护 `internal/config/loadGen.go`,调用 `config.LoadProject`;项目只扩展 Config/Default/Validate,可选在 loader.go/NewLoader 注册新格式。入口、环境覆盖、日志输出规范化无需业务重复实现。
- JSON/YAML/XML 使用同一个加载与绑定流程;Decoder 通过实例注册,无全局 init 或额外强制依赖。内置格式、字段、文件大小与层数均有明确限制。
- `config.Error` 携带文件、字段、环境变量、可用行列及安全原因;`Invalid` / `AtField` 支持业务验证。保留内部错误链,不自动输出原值;大整数、空集合、null、默认值隔离和自定义反序列化有回归覆盖。
- 配置解析在日志、依赖和监听初始化之前;生成/check/dry-run 使用一致入口和文件所有权规则。
- **迁移要求:旧项目先整理手写 Load,再重新生成。** 不静默覆盖业务文件。低层配置 API 保留;新项目不再复制手写加载流程。固定 Go 数组严格检查长度,YAML alias/merge/custom tag 不再接受,避免模糊覆盖和扩展行为。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.20.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.20.0-preview
# 先按 CONFIG.md 迁移旧 Load,再执行:
go-omni api generate
go-omni api generate --check
go test ./...
go build .
```
完整用法、格式、脱敏边界和迁移见 [统一配置指南](CONFIG.md),可运行例子见 [examples/configuration](examples/configuration/README.md)。回归、模糊测试与启动加载性能见 [验收记录](reports/configurationAcceptance.md)。47 个测试包的全量 race、Go 1.22 兼容、vet/build、30 秒 fuzz 与 5 个基准场景三轮测量通过。空缓存远程安装和 8 组生成消费者回归通过,无本地 replace;标签固定在 `c60841f901a3bcc3e9582eece6cfb2cad008fb9e`,见 [远程记录](reports/configurationRemote.json)。仍为 preview;项目的实际配置和业务依赖应自行验收。
---
# v0.19.0-preview(历史版本)
将可选结构转换改为直接 Go 调用,减少业务项目需要维护的配置。普通接口继续使用 `.api` 生成 DTO,不要求增加独立模型或任何拷贝规则。
- 新增只依赖标准库的 `copyx`:Copy、Clone、可复用 Copier;浅/深拷贝、严格字段覆盖、nil/零值策略、字段改名、忽略字段、精确类型 converter。没有全局注册和隐式数字强转。
- 深拷贝检查整个输入和输出图。自定义 converter 原样返回指针、map、slice,或新对象仍嵌套旧引用时返回错误;保留普通循环图,拒绝资源对象及超出遍历/估算分配预算的输入。
- 转换失败不提交部分目标;保留错误路径和原始错误链。并发复用只共享不可变类型元数据;源/目标读写和 converter 副作用由调用方负责。
- **不兼容变更:删除 mappings / mappingFiles、Go 类型加载与映射代码生成。** 检出旧配置/旧生成文件时给出迁移提示;不会静默删除业务正在使用的代码。公共字段组合、scalar、中间件工厂、Repository 选项保留。
- README 补齐当前能力入口、解释和用法,新增 [完整 copyx 指南](COPY.md)、[可运行示例](examples/copying);[AGENTS.md](AGENTS.md) 记录每次修改同步文档/示例/移除说明的约定。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview
# 如用过旧 mapping,先按 COPY.md 完成迁移:
go-omni api generate
go-omni api generate --check
go test ./...
```
迁移步骤和语义边界见 [copyx](COPY.md#从旧-mapping-迁移)。新工具做运行时检查,不能声称具备旧生成器的编译期字段覆盖保证;实际类型对必须有测试。独立三端工资示例验证金额精度、缺失/null/零值、空集合、错误链和字段新增;不修改或代替接入项目的完整业务回归。
46 个测试包的全量 race(含实际生成项目)、vet/build、Go 1.22 兼容测试通过;30 秒 fuzz 无失败,14 种基准场景各测量三轮。验收条件、性能数据及本轮未覆盖的外部环境见 [v0.19 验收记录](reports/copyxAcceptance.md)。本版仍为 preview。 空缓存远程 CLI 与 7 组真实生成消费者验收通过,无本地 replace;标签固定在 `fd83235ce7fc15ce88a482d385d70e61f25d7b32`,详见 [远程记录](reports/copyxRemote.json)。
---
以下是历史版本记录,其中 mapping 功能已在 v0.19 移除,不应继续作为当前接入方式。
# v0.18.0-preview(历史版本)
补齐可空嵌套映射和按模块拆分配置,减少重复转换代码,业务空值规则仍由项目显式选择:
- 嵌套 mapping 支持 `*S → *T` 保留 nil、`S → *T`,以及指定 nil zero/error 后的 `*S → T`;同样适用于 slice/等长数组元素。错误保留字段路径、索引和内部错误链,失败不返回部分结果。
- `mappingFiles` 显式引用项目内的映射文件,支持跨文件向前引用;统一检查重名、循环、类型与字段覆盖。禁止多层导入、路径越界和隐式覆盖,生成期间核对全部配置输入。
- 同类型 `*T → T` 可显式配置 nil zero/error;现有 converter 保持完整控制,不默认放宽类型或混入领域规则。
- `pointerResult: true` 额外生成 `<Name>Ptr(S) (*T,error)`,原值函数保留;失败返回 nil 与原错误,辅助函数名冲突在生成前拒绝。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.18.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.18.0-preview
go-omni api generate
go-omni api generate --check
go test ./...
```
v0.17 的单文件 mappings、现有 converter 和业务文件继续保留。配置拆分时移动完整映射条目,主配置显式列出子文件;不保留重复定义。新能力均为可选生成配置,不增加运行时依赖。nil zero 表示 Go 零值并跳过子映射;业务必填、金额精度、编号格式和快照复制仍由项目负责。
使用与迁移见 [生成能力指南](GENERATION.md),回归和发布后验收见 [验收记录](reports/nullableMappingAcceptance.md)。本版仍为 preview。
全量 44 个测试包的 race、vet、build 和实际进程验收通过;空缓存远程 CLI 与 8 组实际生成消费者回归也通过,同版依赖且无本地 replace。标签固定在 `4563a6354fac19f22cd8d407fd76d98b8cc9c406`。
---
# v0.17.0-preview(历史版本)
减少项目里的重复声明、字段赋值和中间件装配代码,业务规则继续由项目控制:
- `.api` 公共字段组合:保持 JSON 平铺,DTO、绑定、校验和 OpenAPI 同步;字段冲突、循环和超限明确报错。
- 可选结构映射:在 `omni.gen.json` 显式声明源与目标,生成编译期检查的普通 Go 赋值。不同字段名、金额、时间和指针转换由项目指定函数;遗漏字段与类型错误在写文件前失败。不使用 JSON 往返或运行时反射。
- 带依赖的中间件工厂:svc 初始化依赖后自动调用 `New<Name>(deps)` 并注册;错误、nil 或 panic 阻止启动,已登记资源按生命周期清理。
- Repository 骨架默认关闭,可按 group 启用;已有业务文件始终保留。生成选项与映射纳入重新生成和 `--check`。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
go-omni api generate
go-omni api generate --check
go test ./...
```
新项目默认使用 factories;已有项目首次升级保留 handlers 模式,需要显式迁移后再切换。映射按需开启,先准备业务类型、转换函数和依赖,再声明映射;普通引用赋值保留 Go 语义,缺失/null 语义由项目类型与转换函数定义。
使用方法、文件归属和升级步骤见 [生成能力指南](GENERATION.md),回归与发布验收见 [验收记录](reports/generationExperienceAcceptance.md)。本版仍为 preview;框架验收不等于业务项目完成迁移。
全量 44 个测试包的 race、vet、build 和实际进程验收通过;发布后的空缓存远程 CLI 与 7 组生成消费者回归也通过,同版依赖且无本地 replace。标签固定在 `8b588a24cd653dac77dd68f6777e0db367981f19`。
---
# v0.16.0-preview(历史版本)
补齐通用契约与生成体验,业务响应协议继续由项目定义:
- `returns([]string/[]int64/[]bool/[]any)` 及其他支持的基础类型数组,贯通 Logic、Handler、OpenAPI 和生成检查。nil 根数组输出 `[]`;DTO 数组、nullable、null、raw 和 204 保持原语义。字节数组明确拒绝 base64/数组歧义。
- OpenAPI 以标准 components / `$ref` 共享重复响应、schema、示例;保留同一 HTTP 200 下的成功与失败、模板参数、数字业务码和 data:null。
- workspace 显式配置 Go Catalog 提供方,一次导出共享 JSON,多服务通过受控根引用。支持服务/路由的文档错误列表,运行期 App 目录继承不变。check 核对导出漂移并保持项目文件不变。
- EnvOptions.Aliases + ServerEnvAliases 统一完整配置与框架短名称的 Lookup、类型转换、空值规则。完整名称优先,显式空值不等于未设置;旧 ApplyEnv 的空值/逗号列表行为保留。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.16.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.16.0-preview
go-omni api generate
go-omni api generate --check
go test ./...
```
旧项目 config/Logic/svc 继续由开发者维护,不会被重新生成覆盖。已有环境配置需按 [一次覆盖完整配置](DEVELOPMENT.md#一次覆盖完整配置与-http日志短名称) 合并 OverlayEnv 与 ApplyEnv;共享错误目录接入见 [workspace 配置](DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。已有手写权限 Handler 切换自动生成前,先迁出实现并显式移除 Manual 文件,生成器不会静默删除。
全量 race(含实际生成消费者)、vet、build、根入口/退出/再生成验证通过。eTax 只读契约验收:API 文档 20,751,149 → 847,171 字节,ID 文档 1,008,210 → 392,481 字节;70 个操作展开引用后与原文档完全一致。空缓存远程 CLI / 固定版本消费者验收也已通过,没有本地 replace;标签固定在 `64e3bd6cfced225476d58e95037249cd0a4fe695`。详见 [验收记录](reports/contractFollowupAcceptance.md)。这不代表已完成业务项目迁移。
---
# v0.15.0-preview(历史版本)
增加通用后台分阶段关闭:生成入口先完成 HTTP 排空,再运行全部 OnStop、全部 OnDrain,确认组件收尾后逆序释放 OnClose 资源,最后关闭日志。适用于任意后台组件;不引入 Orbit,不改变业务事务或任务一致性策略。
- svc 登记成对、同名的 OnStop/OnDrain,Context 用于早期退出通知,WorkContext 供已接受工作在收尾期间使用。同一阶段并发执行,存在内部先后依赖的组件组合登记。
- server.lifecycle 配置 stopTimeout、drainTimeout、cancelTimeout。阶段超时取消工作并给未返回钩子收尾窗口;错误与阶段/组件名称可通过错误链追踪。
- 不能确认后台已经结束时,返回 ErrResourcesRetained,保留共享资源和借用日志,生成进程非零退出。未返回的 Stop 不会与自己的 Drain 重叠;错误、panic 不阻止其他组件尝试收尾。
- task.Runner 增加 StopAccepting/Wait,原 Shutdown/Close 继续可用。核心 lifecycle 不依赖 task 或其他适配器。
- 初始化和监听失败复用同一流程;重复关闭只执行一次。生成器继续保留配置、svc 和业务代码。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.15.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.15.0-preview
go-omni api generate
go-omni api generate --check
go test ./...
```
v0.14.x 无需重写 svc/config;生成器更新 main/runtimeGen,未配置时使用默认时限。也可 api upgrade --dry-run/--apply 更新依赖与生成文件,保留已有项目扩展。需要阶段保证的旧后台 OnClose 登记,应显式迁到 Stop/Drain;v0.13 及更早版本仍按原迁移指南处理。
全量 race(含实际生成项目)、vet、build 和隔离 MySQL/GORM 验收已通过。发布后的空缓存远程生成项目与真实 MySQL/GORM 验收也通过,CLI/框架同版且无本地 replace;标签固定在 `16deead70e860052c5b629511207578c03f9b2cb`。详见 [生命周期接入](lifecycle/README.md#后台组件分阶段关闭) 与 [验收记录](reports/phasedLifecycleAcceptance.md)。Go 无法强制结束任意 goroutine;OnClose 仍须能返回,HTTP 处理器仍须响应取消。此版不会将未知的第三方组件自动改造成优雅关闭组件。
---
# v0.14.1-preview(历史版本)
增加可选 `database/gormx`:GORM 复用 go-omni 已初始化的标准池、TLS、限额、健康检查和关闭流程,业务继续使用原生 GORM 模型与事务。核心 App/数据库包的编译依赖链不引入 GORM。
@@ -11,7 +180,7 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.14.1-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
```
全量 race(含生成消费者)、vet、build,以及隔离 MySQL 的事务/取消/日志/生成项目接入通过,详见 [GORM 验收](reports/gormAcceptance.md)。
全量 race(含生成消费者)、vet、build,以及隔离 MySQL 的事务/取消/日志/生成项目接入通过。发布后的空缓存远程 MySQL/GORM 和生成项目验收均通过,无本地 replace,CLI 与框架使用同版。标签固定在 `b8ab6b35219d53d6de392c920fe7ecbe9a76dade`,详见 [GORM 验收](reports/gormAcceptance.md)。
---

View File

@@ -1,5 +1,8 @@
# GoOmni 成熟度计划
当前发布基线为 v0.19.0-preview:普通接口通过 `.api` → DTO → Logic 开发;独立类型需要转换时可选 copyx。旧 JSON mapping 生成已移除,见 [迁移说明](COPY.md)。当前能力、使用入口和限制以 [README](README.md) 与 [发布记录](RELEASE.md) 为准。下文保留阶段性建设记录。
当前进展:`v0.14.0-preview` 已完成契约驱动的业务开发能力、通用 Runtime/配置和官方 MySQL 适配器,并通过全量 race、实际生成服务及空缓存远程验收。新项目/升级以 [README](README.md) 和 [开发指南](DEVELOPMENT.md) 为准;下方为历史阶段记录,完整本轮验收见 [报告](reports/businessDevelopmentAcceptance.md)。具体 eTax/ePay 业务迁移与业务回归仍由接入项目完成。
## 1. 新项目启动与发布流程(已完成,v0.1.1)

View File

@@ -1,5 +1,11 @@
# go-omni CLI
v0.19 移除结构映射生成:`omni.gen.json` 只保留 Repository 和中间件选项。普通接口直接使用生成 DTO;需要类型转换时用普通 Go 或可选 [copyx](../../COPY.md)。旧 `mappings` / `mappingFiles` 必须按迁移步骤清理,CLI 不会静默删除仍在调用的旧生成文件。
公共字段组合、中间件工厂和按需 Repository 继续支持,见 [生成能力指南](../../GENERATION.md)。
v0.16 增加基础类型/动态 JSON 数组返回、OpenAPI 公共引用和 workspace Go Catalog 统一导出。见 [接入与筛选规则](../../DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。
v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目录>|--apply]`,以及 `workspace format|generate|check --file omni.workspace.json`。详见 [通用开发指南](../../DEVELOPMENT.md)。
新项目采用根目录 `main.go` 与 `internal/config、svc、logic、handler、types`。完整职责、启动扩展、流式接口和生成文件边界见 [项目指南](../../PROJECT.md)。
@@ -8,14 +14,14 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目
```sh
# 仓库读取权限与 PATH 配置见根目录 README
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.1-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.21.0-preview
go-omni new --module example.com/hello ./hello
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
go-omni new --api service.api --module example.com/example ./example
```
`new` 与 `api new` 统一生成契约项目;不再提供另一套 CRUD 启动目录。未实现的 Logic / 中间件骨架返回 501。进入项目执行 `go mod tidy`、`go run .`、`go build .`。
`new` 与 `api new` 统一生成契约项目;不再提供另一套 CRUD 启动目录。未实现的 Logic 返回 501;中间件工厂未实现时阻止启动,既有 handler 模式仍返回 501。进入项目执行 `go mod tidy`、`go run .`、`go build .`。
参数:`--database none|memory|mysql`(默认 none;memory 等同 none),`--storage none|local|s3|r2|oss`,`--log-output stdout|stderr|file`,`--log-format json|text`。基础设施按需生成,连接凭据由环境提供,不自动创建数据库、表或云资源。选择 MySQL 不会凭空生成业务表或 Repository 查询。
@@ -43,14 +49,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
## 版本与迁移
CLI 与框架依赖固定使用 v0.14.1-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
CLI 与框架依赖固定使用 v0.20.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
```sh
# 在 go-omni 源码仓库中执行
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
```
模板最低需要 v0.14.0-preview 的通用 Runtime、配置和绑定能力,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
当前模板与依赖以 v0.20.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
## 数据库迁移命令
@@ -63,3 +69,7 @@ go-omni migrate --dir migrations --apply
```
SQL 命名示例 `0001_createUsers.up.sql`;预览只显示版本和 SHA-256,不连接数据库。失败保留 dirty 状态,不自动 down/force。细节见 [扩展模块](../../ADVANCED.md)。
### v0.20 配置入口
新生成 `internal/config/loadGen.go` 调用统一加载,纳入生成检查;项目 config.go 只维护字段、默认值和校验,loader.go 用于注册额外格式。JSON/YAML/XML 内置。旧手写 Load 会阻止生成,请先按 [迁移步骤](../../CONFIG.md#v019-及更早项目迁移) 整理,不会自动覆盖业务代码。

View File

@@ -170,6 +170,7 @@ func TestRegenerationAndConflicts(t *testing.T) {
servicePath := filepath.Join(dir, "internal/logic/greet/greet.go")
service := readFile(t, servicePath) + "\n// user edit must survive\n"
writeFile(t, servicePath, service)
writeFile(t, filepath.Join(dir, "internal/repository/greet/repository.go"), "package greet\n")
for _, file := range []string{"etc/config.yaml", "internal/svc/serviceContext.go", "internal/repository/greet/repository.go"} {
body := readFile(t, filepath.Join(dir, file))
if strings.HasSuffix(file, ".go") {
@@ -301,6 +302,7 @@ func TestGeneratedProjectsBuildAndRun(t *testing.T) {
if starter {
_, err = New(o)
} else {
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
o.API = filepath.Join(root, "examples/contracts/user.api")
_, err = Generate(o)
}

View File

@@ -0,0 +1,104 @@
package apigen
import (
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestPrimitiveArrayContracts(t *testing.T) {
for _, name := range []string{"string", "bool", "int", "int8", "int16", "int32", "int64", "uint", "uint16", "uint32", "uint64", "float32", "float64", "any"} {
t.Run(name, func(t *testing.T) {
spec, err := parseText(t, fmt.Sprintf("service arrays{\n@handler list\nget /list returns ([]%s)\n}", name))
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
schema := operation(t, document(t, c), "/list", "get")["responses"].(map[string]any)["200"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
data := schema["properties"].(map[string]any)["data"].(map[string]any)
if data["type"] != "array" || data["items"] == nil {
t.Fatal(data)
}
if name == "any" && len(data["items"].(map[string]any)) != 0 {
t.Fatal("dynamic JSON was constrained")
}
if responseGoType("[]"+name) != "[]"+name || responseNeedsTypes("[]"+name) {
t.Fatal("wrong Go signature/import")
}
})
}
for _, name := range []string{"uint8", "File", "Unknown", "[]string"} {
spec, err := parseText(t, fmt.Sprintf("service arrays{\n@handler list\nget /list returns ([]%s)\n}", name))
if err == nil {
_, err = Compile(spec)
}
if err == nil {
t.Fatal("unsupported root accepted", name)
}
}
}
func TestPrimitiveArrayConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "arrays")
o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/arrays.api"), Module: "example.com/arrays", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
for _, name := range []string{"strings", "integers", "dynamic", "empty", "raw"} {
writeFile(t, filepath.Join(dir, "internal/logic/arrays/"+name+".go"), "package arrays\n")
}
writeFile(t, filepath.Join(dir, "internal/logic/arrays/values.go"), readFile(t, "testdata/arrays/values.go"))
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/arrays/consumer_test.go"))
o.API = filepath.Join(dir, "api/arrays.api")
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
if !strings.Contains(readFile(t, filepath.Join(dir, "internal/logic/arrays/values.go")), "企业") {
t.Fatal("logic overwritten")
}
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-race", "-mod=mod", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("%v\n%s", err, out)
}
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
raw, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(raw))
}
var module struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(raw, &module); err != nil {
t.Fatal(err)
}
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
t.Fatal("not a fixed remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
}
}

View File

@@ -40,6 +40,7 @@ func TestBusinessGeneratedProject(t *testing.T) {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "business")
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/business.api"), Module: "example.com/business", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)

View File

@@ -2,41 +2,73 @@ package apigen
import (
"errors"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/response"
"io"
"os"
"path/filepath"
"strings"
)
const catalogDocumentFile = "omni.errors.json"
func readDocumentCatalog(root string, spec *Spec) error {
path := filepath.Join(root, catalogDocumentFile)
stat, err := os.Lstat(path)
if errors.Is(err, os.ErrNotExist) {
return nil
// DocumentErrors selects documentation only. Runtime App inheritance and
// route-local @errors declarations are never changed by this metadata.
// A nil Include lists all errors; [] lists none. Routes override Include.
type DocumentErrors struct {
Source string `json:"source"`
Include []string `json:"include"`
Routes map[string][]string `json:"routes,omitempty"`
}
func readDocumentCatalog(root string, spec *Spec, overrides map[string][]byte) error {
name := catalogDocumentFile
path := filepath.Join(root, name)
// Shared metadata uses the same explicitly authorized roots as .api imports.
if selection := spec.DocumentErrors; selection != nil {
if _, err := os.Lstat(path); !errors.Is(err, os.ErrNotExist) {
return errors.New("api: use either local omni.errors.json or shared errors.source, not both")
}
name = selection.Source
alias, sub, ok := strings.Cut(strings.TrimPrefix(name, "@"), "/")
declared, found := spec.SharedRoots[alias]
if !strings.HasPrefix(name, "@") || !ok || !found || sub == "" || filepath.Ext(sub) != ".json" {
return errors.New("api: errors.source must name a JSON file in a declared @root")
}
if err := safePath(declared.Path, sub); err != nil {
return err
}
path = filepath.Join(declared.Path, filepath.FromSlash(sub))
}
if err != nil {
return err
}
if !stat.Mode().IsRegular() {
return errors.New("api: error metadata must be a regular file")
}
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
raw, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
if err != nil {
return err
raw, ok := overrides[path]
if !ok {
stat, err := os.Lstat(path)
if errors.Is(err, os.ErrNotExist) && spec.DocumentErrors == nil {
return nil
}
if err != nil {
return err
}
if !stat.Mode().IsRegular() {
return errors.New("api: error metadata must be a regular file")
}
f, err := os.Open(path)
if err != nil {
return err
}
defer f.Close()
raw, err = io.ReadAll(io.LimitReader(f, (1<<20)+1))
if err != nil {
return err
}
}
if len(raw) > 1<<20 {
return errors.New("api: error metadata exceeds 1 MiB")
}
var catalog apierrors.Catalog
if err = apierrors.DecodeCatalogJSON(raw, &catalog); err != nil {
if err := apierrors.DecodeCatalogJSON(raw, &catalog); err != nil {
return err
}
registry, err := apierrors.NewRegistry(catalog)
@@ -45,10 +77,70 @@ func readDocumentCatalog(root string, spec *Spec) error {
}
normalized := registry.Catalog()
spec.DocumentCatalog = &normalized
spec.Sources[catalogDocumentFile] = raw
spec.SourcePaths[catalogDocumentFile] = path
spec.Sources[name] = raw
spec.SourcePaths[name] = path
return validateDocumentSelection(spec)
}
func documentRouteCatalog(spec *Spec, route Route) apierrors.Catalog {
catalog := spec.DocumentCatalog.Clone()
if spec.DocumentErrors == nil {
return catalog
}
include := spec.DocumentErrors.Include
if selected, ok := spec.DocumentErrors.Routes[strings.ToUpper(route.Method)+" "+route.Path]; ok {
include = selected
}
if include != nil {
selected := map[string]apierrors.Definition{}
for _, key := range include {
selected[key] = catalog.Definitions[key]
}
catalog.Definitions = selected
}
return catalog
}
func validateDocumentSelection(spec *Spec) error {
cfg := spec.DocumentErrors
if cfg == nil {
return nil
}
validate := func(keys []string) error {
seen := map[string]bool{}
for _, key := range keys {
if _, ok := spec.DocumentCatalog.Definitions[key]; !ok {
return fmt.Errorf("api: unknown documentation error %q", key)
}
if seen[key] {
return fmt.Errorf("api: duplicate documentation error %q", key)
}
seen[key] = true
}
return nil
}
if err := validate(cfg.Include); err != nil {
return err
}
routes := map[string]Route{}
for _, r := range spec.Routes {
routes[strings.ToUpper(r.Method)+" "+r.Path] = r
}
for key, keys := range cfg.Routes {
r, ok := routes[key]
if !ok {
return fmt.Errorf("api: unknown documentation route %q", key)
}
if len(r.Errors) > 0 {
return fmt.Errorf("api: %s already declares route-local errors", key)
}
if err := validate(keys); err != nil {
return err
}
}
return nil
}
func validateDocumentCatalog(spec *Spec) error {
if spec.DocumentCatalog == nil {
return nil
@@ -61,7 +153,7 @@ func validateDocumentCatalog(spec *Spec) error {
if err != nil {
return err
}
if err = policy.ValidateErrors(*spec.DocumentCatalog); err != nil {
if err = policy.ValidateErrors(documentRouteCatalog(spec, r)); err != nil {
return err
}
}

View File

@@ -40,6 +40,9 @@ func Compile(spec *Spec) (*Compiled, error) {
if prepareErr != nil {
return nil, prepareErr
}
if err := expandFields(spec); err != nil {
return nil, err
}
if err := validateDocumentCatalog(spec); err != nil {
return nil, err
}
@@ -70,7 +73,7 @@ func Compile(spec *Spec) (*Compiled, error) {
resolving := map[string]bool{}
var resolve func(string, int) (reflect.Type, error)
var expression func(ast.Expr, int) (reflect.Type, error)
primitives := map[string]reflect.Type{"File": reflect.TypeOf(contract.File{}), "any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
if depth > 32 {
return nil, errors.New("type nesting exceeds 32")
@@ -80,7 +83,7 @@ func Compile(spec *Spec) (*Compiled, error) {
if t, ok := scalarTypes[e.Name]; ok {
return t, nil
}
if t, ok := primitives[e.Name]; ok {
if t, ok := primitiveTypes[e.Name]; ok {
return t, nil
}
return resolve(e.Name, depth+1)
@@ -313,6 +316,7 @@ func Compile(spec *Spec) (*Compiled, error) {
return nil, err
}
decorateDocument(doc, spec, c.Bindings)
compactDocument(doc)
c.Document, err = json.MarshalIndent(doc, "", " ")
return c, err
}

View File

@@ -0,0 +1,102 @@
package apigen
import (
"fmt"
"reflect"
"strings"
)
// expandFields copies definitions rather than embedding Go structs. Binding,
// validation, DTOs, snapshots and OpenAPI all consume the same flattened fields.
func expandFields(spec *Spec) error {
definitions := map[string]Type{}
for _, t := range spec.Types {
if _, exists := definitions[t.Name]; exists {
return fmt.Errorf("%s: duplicate type %s", t.Position, t.Name)
}
definitions[t.Name] = t
}
expanded := map[string]Type{}
heights := map[string]int{}
visiting := map[string]bool{}
total := 0
var expand func(string, int) (Type, error)
expand = func(name string, depth int) (Type, error) {
if t, ok := expanded[name]; ok {
if depth+heights[name] > 32 {
return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name)
}
return t, nil
}
if visiting[name] {
return Type{}, fmt.Errorf("field composition cycle at %s", name)
}
if depth > 32 {
return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name)
}
t, ok := definitions[name]
if !ok {
return Type{}, fmt.Errorf("unknown composed object type %s", name)
}
visiting[name] = true
defer delete(visiting, name)
fields := []Field{}
height := 0
seen, wire := map[string]bool{}, map[string]bool{}
for _, f := range t.Fields {
add := []Field{f}
if f.Type == "" {
child, err := expand(f.Name, depth+1)
if err != nil {
return Type{}, fmt.Errorf("%s.%s: %w", name, f.Name, err)
}
add = child.Fields
if heights[f.Name]+1 > height {
height = heights[f.Name] + 1
}
if height > 32 {
return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name)
}
}
for _, field := range add {
if seen[field.Name] {
return Type{}, fmt.Errorf("%s: conflicting composed field %s", name, field.Name)
}
seen[field.Name] = true
for _, source := range []string{"json", "form", "path", "header", "multipart", "multipartJSON", "file"} {
tag := strings.Split(reflect.StructTag(field.Tag).Get(source), ",")[0]
if tag == "" || tag == "-" {
continue
}
if source == "header" {
tag = strings.ToLower(tag)
}
key := source + ":" + tag
if wire[key] {
return Type{}, fmt.Errorf("%s: conflicting composed field tag %s", name, key)
}
wire[key] = true
}
fields = append(fields, field)
total++
if len(fields) > 4096 || total > 65536 {
return Type{}, fmt.Errorf("field composition expansion limit exceeded")
}
}
}
t.Fields = fields
expanded[name] = t
heights[name] = height
return t, nil
}
result := make([]Type, 0, len(spec.Types))
for _, t := range spec.Types {
out, err := expand(t.Name, 0)
if err != nil {
return err
}
result = append(result, out)
}
spec.Types = result
return nil
}

View File

@@ -0,0 +1,163 @@
package apigen
import (
"bytes"
"encoding/json"
"fmt"
"os"
"path/filepath"
"strings"
"testing"
)
func TestFieldComposition(t *testing.T) {
source := readFile(t, "testdata/composition/service.api")
spec, err := parseText(t, source)
if err != nil {
t.Fatal(err)
}
first, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
second, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
if !bytes.Equal(first.Document, second.Document) || len(spec.Types[1].Fields) != 1 {
t.Fatal("composition mutates input")
}
for _, name := range []string{"EnterpriseRequest", "TenantRequest", "AdminRequest"} {
schema := first.Bindings[name].BodySchema()
raw, _ := json.Marshal(schema)
if !bytes.Contains(raw, []byte("Employee display name.")) || bytes.Contains(raw, []byte("SalaryFields")) {
t.Fatalf("flattened schema/comment missing: %s", raw)
}
}
cases := []struct{ name, decl, want string }{
{"unknown", "type Req {\n Missing\n}", "unknown composed"},
{"cycle", "type Req {\n Other\n}\ntype Other {\n Req\n}", "cycle"},
{"conflict", "type Base {\n Name string `json:\"name\"`\n}\ntype Req {\n Base\n Name string `json:\"other\"`\n}", "conflicting"},
{"wireConflict", "type Base {\n Name string `json:\"name\"`\n}\ntype Req {\n Base\n Other string `json:\"name\"`\n}", "conflicting"},
{"tagged", "type Req {\n Other `json:\"other\"`\n}", "cannot carry"},
{"diamond", "type Base {\n Name string `json:\"name\"`\n}\ntype Left {\n Base\n}\ntype Right {\n Base\n}\ntype Req {\n Left\n Right\n}", "conflicting"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
s, e := parseText(t, tc.decl+"\nservice demo{\n@handler post\npost / (Req)\n}")
if e == nil {
_, e = Compile(s)
}
if e == nil || !strings.Contains(e.Error(), tc.want) {
t.Fatal(e)
}
})
}
// Imports can contain reusable definitions and retain the existing root policy.
dir := t.TempDir()
writeFile(t, filepath.Join(dir, "base.api"), "type Base {\n Name string `json:\"name\"`\n}\n")
writeFile(t, filepath.Join(dir, "service.api"), "import \"base.api\"\ntype Req {\n Base\n}\nservice demo{\n@handler post\npost / (Req)\n}")
if _, err := FormatProject(filepath.Join(dir, "service.api"), false); err != nil {
t.Fatal(err)
}
if _, err := FormatProject(filepath.Join(dir, "service.api"), true); err != nil {
t.Fatal(err)
}
}
func TestRepositoryGenerationOptions(t *testing.T) {
dir := filepath.Join(t.TempDir(), "app")
if _, err := New(Options{Dir: dir, Module: "example.com/repository", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(dir, "internal/repository")); !os.IsNotExist(err) {
t.Fatal("default creates repository")
}
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"repository":"group","middleware":"factories"}`)
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
repo := filepath.Join(dir, "internal/repository/greet/repository.go")
original := readFile(t, repo) + "\n// business SQL belongs here\n"
writeFile(t, repo, original)
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"repository":"none","middleware":"factories"}`)
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
o.Check = true
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
if readFile(t, repo) != original {
t.Fatal("repository removed or overwritten")
}
for _, raw := range []string{`{"repository":"oops"}`, `{"middleware":"oops"}`, `{"unknown":true}`, `{"repository":"none","repository":"group"}`} {
writeFile(t, filepath.Join(dir, generatorConfigFile), raw)
before := snapshot(t, dir)
if _, err := Generate(o); err == nil {
t.Fatal("accepted", raw)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("invalid config wrote files")
}
}
}
func TestCompositionExpansionBound(t *testing.T) {
var source strings.Builder
source.WriteString("type Base {\n Value string `json:\"value\"`\n}\n")
last := "Base"
for i := 0; i < 35; i++ {
name := fmt.Sprintf("Level%d", i)
fmt.Fprintf(&source, "type %s {\n %s\n}\n", name, last)
last = name
}
// Reverse order forces traversal instead of hitting pre-expanded definitions.
spec, err := parseText(t, source.String()+"service demo{\n@handler post\npost / ("+last+")\n}")
if err != nil {
t.Fatal(err)
}
for i, j := 0, len(spec.Types)-1; i < j; i, j = i+1, j-1 {
spec.Types[i], spec.Types[j] = spec.Types[j], spec.Types[i]
}
if _, err = Compile(spec); err == nil {
t.Fatal("unbounded composition accepted")
}
}
func TestGenerationConfigMissingAndLegacy(t *testing.T) {
dir := filepath.Join(t.TempDir(), "app")
if _, err := New(Options{Dir: dir, Module: "example.com/config", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
if err := os.Remove(filepath.Join(dir, generatorConfigFile)); err != nil {
t.Fatal(err)
}
before := snapshot(t, dir)
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "restore") {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("missing config changed project")
}
// A genuine old manifest has never tracked a generator config.
var old manifest
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &old); err != nil {
t.Fatal(err)
}
old.GeneratorConfig = false
raw, _ := json.Marshal(old)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
cfg, exists, err := readGeneratorConfig(dir)
if err != nil || !exists || cfg.Middleware != "handlers" || cfg.Repository != "none" {
t.Fatal(cfg, exists, err)
}
if !strings.Contains(readFile(t, filepath.Join(dir, "internal/svc/runtimeGen.go")), "middleware.Defaults()") {
t.Fatal("legacy registration replaced")
}
}

View File

@@ -0,0 +1,113 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestGeneratedConfigurationConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "app")
api := filepath.Join(t.TempDir(), "service.api")
writeFile(t, api, "service configuration {\n@handler ping\nget /ping\n}\n")
o := Options{API: api, Dir: dir, Module: "example.com/configuration", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
main := readFile(t, filepath.Join(dir, "main.go"))
// The business only changes fields/defaults/validation, not the loading flow.
writeFile(t, filepath.Join(dir, "internal/config/config.go"), `package config
import("git.yuntuo.hk/k2safe/go-omni/config";"git.yuntuo.hk/k2safe/go-omni/logx")
type Config struct { Server config.Config `+"`json:\"server\"`"+`;Limit int `+"`json:\"limit\" envDefault:\"10\"`"+` }
func Default()Config{ c:=config.Default();c.Log.File="./var/log/app.log";c.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:10};return Config{Server:c} }
func(c Config)Validate()error{if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)};if c.Limit<0{return config.Invalid("limit","must not be negative")};return nil}
`)
writeFile(t, filepath.Join(dir, "internal/config/loader.go"), `package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
func NewLoader()(*omniConfig.Loader,error){return omniConfig.NewLoader(omniConfig.WithDecoder("custom",omniConfig.JSONDecoder{}))}
`)
svc := filepath.Join(dir, "internal/svc/serviceContext.go")
writeFile(t, svc, strings.Replace(readFile(t, svc), "return nil", "panic(\"dependencies must not initialize on invalid config\")", 1))
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/configuration/consumer_test.go"))
runConsumerGo(t, dir, "mod", "tidy")
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
runConsumerGo(t, dir, "build", ".")
o.API = filepath.Join(dir, "api/service.api")
before := snapshot(t, dir)
for _, mode := range []string{"dry", "generate", "check"} {
o.DryRun = mode == "dry"
o.Check = mode == "check"
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("business files overwritten")
}
if main != readFile(t, filepath.Join(dir, "main.go")) {
t.Fatal("main changed")
}
load := filepath.Join(dir, "internal/config/loadGen.go")
original := readFile(t, load)
writeFile(t, load, original+"\n// drift\n")
if err := generateManagedTestProject(t, o); err == nil {
t.Fatal("loadGen drift accepted")
}
writeFile(t, load, original)
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
data, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(data))
}
var dependency struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(data, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
}
func TestConfigLoadMigrationGuard(t *testing.T) {
dir := filepath.Join(t.TempDir(), "app")
o := Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}
if _, err := New(o); err != nil {
t.Fatal(err)
}
o.API = filepath.Join(dir, "api/service.api")
matches, err := filepath.Glob(filepath.Join(dir, "api/*.api"))
if err != nil || len(matches) != 1 {
t.Fatal(matches, err)
}
o.API = matches[0]
writeFile(t, filepath.Join(dir, "internal/config/legacy.go"), "package config\nfunc Load(path string)(Config,error){return Default(),nil}\n")
before := snapshot(t, dir)
for _, mode := range []string{"dry", "generate", "check"} {
o.DryRun = mode == "dry"
o.Check = mode == "check"
_, err := Generate(o)
if err == nil || !strings.Contains(err.Error(), "CONFIG.md") {
t.Fatal("missing migration diagnostic", err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("migration guard changed files")
}
}
}

View File

@@ -213,7 +213,7 @@ func decorateDocument(doc map[string]any, spec *Spec, bindings map[string]*contr
if len(r.Errors) == 0 && documentRegistry != nil {
registry = documentRegistry
op["x-omni-error-catalog-scope"] = "app"
for key, definition := range registry.Catalog().Definitions {
for key, definition := range documentRouteCatalog(spec, r).Definitions {
grouped[definition.Status] = append(grouped[definition.Status], ErrorDefinition{Name: key, Code: key, Status: definition.Status, Message: definition.Message, BusinessCode: definition.Code, Params: definition.Params})
}
}

View File

@@ -0,0 +1,118 @@
package apigen
import (
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func runConsumerGo(t *testing.T, dir string, args ...string) {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", args...)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("go %v: %v\n%s", args, err, out)
}
}
func TestCopyxThreeEndpointConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "app")
o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
for path, fixture := range map[string]string{
"internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go", "internal/convert/copy.go": "copy.go",
"internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go",
} {
writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture))
}
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1)
source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1)
writeFile(t, svcFile, source)
configFile := filepath.Join(dir, "internal/config/config.go")
writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1))
for _, group := range []string{"Enterprise", "Tenant", "Admin"} {
text := fmt.Sprintf(`package %s
import("context";"example.com/composition/internal/types";"example.com/composition/internal/convert";"example.com/composition/internal/domain")
func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){
input,err:=convert.Employee(*req);if err!=nil{return nil,err}
result,err:=convert.Response(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil
}
`, strings.ToLower(group), group)
writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text)
}
runConsumerGo(t, dir, "mod", "tidy")
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
o.API = filepath.Join(dir, "api/service.api")
before := snapshot(t, dir)
o.DryRun = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
o.DryRun = false
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("generation changed business files")
}
if _, err := os.Stat(filepath.Join(dir, "internal/mapping")); !os.IsNotExist(err) {
t.Fatal("mapping directory generated")
}
// Adding a common field requires one contract edit and one independent
// business field. Three endpoint converters need no new rules.
contract := readFile(t, o.API)
contract = strings.Replace(contract, "type SalaryFields {", "type SalaryFields {\n Note string `json:\"note,optional\"`", 1)
writeFile(t, o.API, contract)
domainPath := filepath.Join(dir, "internal/domain/employee.go")
writeFile(t, domainPath, strings.Replace(readFile(t, domainPath), "type Employee struct {", "type Employee struct {\n Note string", 1))
o.Check = false
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
runConsumerGo(t, dir, "build", ".")
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
data, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(data))
}
var dependency struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(data, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a fixed remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
}

View File

@@ -0,0 +1,86 @@
package apigen
import "encoding/json"
// compactDocument only replaces standard OpenAPI Reference Object positions.
// Payloads (including examples containing a literal "$ref") are never walked.
// Mixed HTTP 200 responses retain both success and error alternatives.
func compactDocument(doc map[string]any) {
type slot struct {
parent map[string]any
key string
}
components := doc["components"].(map[string]any)
var responses, schemas, examples []slot
for _, path := range doc["paths"].(map[string]any) {
for _, value := range path.(map[string]any) {
op, ok := value.(map[string]any)
if !ok {
continue
}
rs, _ := op["responses"].(map[string]any)
for status := range rs {
responses = append(responses, slot{rs, status})
}
}
}
factor := func(kind string, slots []slot) {
counts := map[string]int{}
for _, s := range slots {
raw, _ := json.Marshal(s.parent[s.key])
counts[string(raw)]++
}
shared, _ := components[kind].(map[string]any)
if shared == nil {
shared = map[string]any{}
}
for _, s := range slots {
raw, _ := json.Marshal(s.parent[s.key])
count := counts[string(raw)]
if count < 2 {
continue
}
name := "Omni_" + digest(raw)
ref := map[string]any{"$ref": "#/components/" + kind + "/" + name}
refJSON, _ := json.Marshal(ref)
// Keep small objects inline if components would increase size.
if (count-1)*len(raw) <= count*len(refJSON)+len(name)+4 {
continue
}
shared[name] = s.parent[s.key]
s.parent[s.key] = map[string]any{"$ref": "#/components/" + kind + "/" + name}
}
if len(shared) > 0 {
components[kind] = shared
}
}
// Share whole responses before counting inner objects. Otherwise a catalog
// copied across 100 routes can create hundreds of unnecessary components
// that only one shared response ultimately uses.
factor("responses", responses)
collectMedia := func(value any) {
resp := value.(map[string]any)
content, _ := resp["content"].(map[string]any)
for _, value := range content {
media := value.(map[string]any)
if media["schema"] != nil {
schemas = append(schemas, slot{media, "schema"})
}
es, _ := media["examples"].(map[string]any)
for name := range es {
examples = append(examples, slot{es, name})
}
}
}
for _, s := range responses {
collectMedia(s.parent[s.key])
}
sharedResponses, _ := components["responses"].(map[string]any)
for _, resp := range sharedResponses {
collectMedia(resp)
}
factor("examples", examples)
factor("schemas", schemas)
}

View File

@@ -0,0 +1,107 @@
package apigen
import (
"encoding/json"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"strings"
"testing"
)
func TestSharedErrorDocumentReferences(t *testing.T) {
source := `@app(defaultResponse: Common)
response Common(codeType:int32;successCode:0;errorCode:9999;messageField:msg;dataMode:always)
service demo {
`
for i := 0; i < 16; i++ {
kind := "string"
if i%2 == 0 {
kind = "int64"
}
source += fmt.Sprintf("@handler list%d\nget /list%d returns ([]%s)\n", i, i, kind)
}
source += "}\n"
spec, err := parseText(t, source)
if err != nil {
t.Fatal(err)
}
catalog := apierrors.Catalog{DefaultStatus: 400, Definitions: map[string]apierrors.Definition{}}
for i := 0; i < 80; i++ {
status := 400
if i%2 == 0 {
status = 200
}
catalog.Definitions[fmt.Sprintf("ERROR_%d", i)] = apierrors.Definition{Status: status, Code: 400000 + i, Message: strings.Repeat("说明", 40) + "{label}不能超过{max}个字符", Params: []apierrors.Parameter{{Name: "label", Type: "string", Example: "企业名称"}, {Name: "max", Type: "int", Example: 100}}}
}
spec.DocumentCatalog = &catalog
compiled, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
var raw map[string]any
if err = json.Unmarshal(compiled.Document, &raw); err != nil {
t.Fatal(err)
}
components := raw["components"].(map[string]any)
for _, kind := range []string{"responses", "examples"} {
if len(components[kind].(map[string]any)) == 0 {
t.Fatal("missing shared", kind)
}
}
expanded := document(t, compiled)
for i := 0; i < 16; i++ {
responses := operation(t, expanded, fmt.Sprintf("/list%d", i), "get")["responses"].(map[string]any)
for _, status := range []string{"200", "400"} {
media := responses[status].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
if len(media["examples"].(map[string]any)) != 40 {
t.Fatal("examples lost")
}
if status == "200" {
choices := media["schema"].(map[string]any)["anyOf"].([]any)
if len(choices) != 2 || choices[0].(map[string]any)["properties"].(map[string]any)["data"].(map[string]any)["type"] != "array" {
t.Fatal("200 success shape lost")
}
}
for _, example := range media["examples"].(map[string]any) {
value := example.(map[string]any)["value"].(map[string]any)
if _, ok := value["data"]; !ok || value["data"] != nil || !strings.Contains(value["msg"].(string), "企业名称不能超过100个字符") {
t.Fatal(value)
}
}
}
}
expandedBytes, _ := json.MarshalIndent(expanded, "", " ")
if len(compiled.Document)*5 > len(expandedBytes) {
t.Fatalf("poor dedup: %d vs %d", len(compiled.Document), len(expandedBytes))
}
again, err := Compile(spec)
if err != nil || string(again.Document) != string(compiled.Document) {
t.Fatal("non-deterministic document", err)
}
before, _ := json.Marshal(raw)
compactDocument(raw)
data, _ := json.Marshal(raw)
if string(data) != string(before) {
t.Fatal("compaction not idempotent")
}
}
func TestSharedSchemasAndLiteralExampleReference(t *testing.T) {
doc := map[string]any{"components": map[string]any{}, "paths": map[string]any{}}
paths := doc["paths"].(map[string]any)
for _, name := range []string{"first", "second", "third"} {
schema := map[string]any{"type": "object", "description": strings.Repeat("schema description ", 40)}
example := map[string]any{"summary": strings.Repeat("example ", 50), "value": map[string]any{"$ref": "literal business field"}}
paths["/"+name] = map[string]any{"get": map[string]any{"responses": map[string]any{"200": map[string]any{"description": name, "content": map[string]any{"application/json": map[string]any{"schema": schema, "examples": map[string]any{"same": example}}}}}}}
}
compactDocument(doc)
components := doc["components"].(map[string]any)
if len(components["schemas"].(map[string]any)) != 1 {
t.Fatal("schema not shared")
}
for _, example := range components["examples"].(map[string]any) {
if example.(map[string]any)["value"].(map[string]any)["$ref"] != "literal business field" {
t.Fatal("payload reference interpreted")
}
}
}

View File

@@ -19,11 +19,16 @@ import (
)
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
const minimumVersion = "v0.14.0-preview"
// Runtime-only releases need not raise the managed template compatibility floor.
const minimumVersion = "v0.20.0-preview"
const currentVersion = "v0.21.0-preview"
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
type Options struct {
generation GeneratorConfig
lockHeld bool
documentCatalogs map[string][]byte
Database, Storage string
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
Check bool
@@ -48,7 +53,7 @@ func Generate(o Options) (Output, error) {
if o.API == "" || o.Dir == "" {
return Output{}, errors.New("--api and --dir are required")
}
spec, err := Parse(o.API)
spec, err := parseWithCatalogs(o.API, o.documentCatalogs)
if err != nil {
return Output{}, err
}
@@ -96,9 +101,17 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
return Output{}, err
}
o.Dir = directory
genConfig, hasGenConfig, err := readGeneratorConfig(directory)
if err != nil {
return Output{}, err
}
o.generation = genConfig
if err = checkRuntimeExtension(directory); err != nil {
return Output{}, err
}
if err = checkConfigLoadExtension(directory); err != nil {
return Output{}, err
}
for _, r := range c.Spec.Routes {
if !r.Manual {
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
@@ -145,13 +158,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
if !filepath.IsAbs(path) {
path = filepath.Join(directory, path)
}
if _, err := os.Stat(filepath.Join(path, "lifecycle", "runtime.go")); err == nil {
if source, err := os.ReadFile(filepath.Join(path, "config", "loader.go")); err == nil && strings.Contains(string(source), "func NewLoader(") {
localCurrent = true
}
}
}
if !localCurrent {
return Output{}, fmt.Errorf("existing framework dependency predates the generated Runtime, request policy and structured MySQL support; use %s or a current local replacement", minimumVersion)
return Output{}, fmt.Errorf("existing framework dependency predates the generated configuration helpers; use %s or a current local replacement", minimumVersion)
}
}
if o.FrameworkDir != "" {
@@ -191,6 +204,10 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
for name, data := range base {
seeds[name] = data
}
if !hasGenConfig {
raw, _ := json.MarshalIndent(genConfig, "", " ")
seeds[generatorConfigFile] = append(raw, '\n')
}
// The API source remains developer-owned. A relocated project regenerates
// from its own api/ tree, including all imports.
entry := "api/" + c.Spec.Entry
@@ -241,8 +258,17 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
}
}
}
currentConfig, stillExists, err := readGeneratorConfig(directory)
if err != nil {
return Output{}, err
}
beforeConfig, _ := json.Marshal(genConfig)
afterConfig, _ := json.Marshal(currentConfig)
if hasGenConfig != stillExists || genConfig.inputDigest != currentConfig.inputDigest || string(beforeConfig) != string(afterConfig) {
return Output{}, errors.New("generation configuration changed during generation; retry")
}
out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld)
expectedVersion := minimumVersion
expectedVersion := currentVersion
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) {
expectedVersion = info.Main.Version
}
@@ -281,7 +307,7 @@ func moduleFile(o Options) ([]byte, error) {
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
version = info.Main.Version
} else {
version = minimumVersion
version = currentVersion
}
}
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {
@@ -454,7 +480,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
continue
}
hasAutomatic = true
needsTypes = needsTypes || r.Request != "" || responseGoType(r.Response) != ""
needsTypes = needsTypes || r.Request != "" || responseNeedsTypes(r.Response)
needsBinder = needsBinder || !r.CustomBinding
}
text += fmt.Sprintf("%q;%q;", frameworkModule+"/httpx", o.Module+"/internal/svc")
@@ -570,7 +596,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
}
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
if r.Request != "" || responseGoType(r.Response) != "" {
if r.Request != "" || responseNeedsTypes(r.Response) {
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
}
if streamingBody || r.Produces != "" {
@@ -601,7 +627,9 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
}
put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text)
put(seeds, "internal/logic/"+group+"/logic.go", fmt.Sprintf("package %s\nimport %q\ntype Logic struct {deps *svc.ServiceContext}\nfunc New(deps *svc.ServiceContext)*Logic{return &Logic{deps:deps}}\n", packageName, o.Module+"/internal/svc"))
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
if o.generation.Repository == "group" {
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
}
}
put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n")
registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n"
@@ -610,6 +638,9 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m)))
}
put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n")
if o.generation.Middleware == "factories" {
renderMiddlewareFactories(middlewareNames, generated, seeds)
}
for name, source := range baseTemplates {
content := strings.ReplaceAll(source, "MODULE", o.Module)
content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat)
@@ -629,6 +660,16 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map
return nil, nil, err
}
for name, data := range infra {
if o.generation.Middleware == "factories" && name == "internal/svc/runtimeGen.go" {
formatted, err := format.Source(data)
if err != nil {
return nil, nil, err
}
source := string(formatted)
source = strings.Replace(source, "Middlewares: middleware.Defaults(),", "", 1)
source = strings.Replace(source, "return c, nil", "if len(c.Middlewares) != 0 { return nil, errors.New(\"factory middleware is registered automatically; remove manual Middlewares assignments from Init\") }\nc.Middlewares, err = middleware.New(c)\nif err != nil { return nil, err }\nreturn c, nil", 1)
data = []byte(source)
}
if strings.HasSuffix(name, "Gen.go") {
generated[name] = append([]byte(generatedHeader), data...)
} else {

View File

@@ -0,0 +1,110 @@
package apigen
import (
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strings"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
)
const generatorConfigFile = "omni.gen.json"
// GeneratorConfig contains developer-owned scaffolding options only.
type GeneratorConfig struct {
Repository string `json:"repository,omitempty"`
Middleware string `json:"middleware,omitempty"`
inputDigest string
}
func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
cfg := GeneratorConfig{Repository: "none", Middleware: "factories"}
if err := safePath(root, generatorConfigFile); err != nil {
return cfg, false, err
}
stat, err := os.Lstat(filepath.Join(root, generatorConfigFile))
if errors.Is(err, os.ErrNotExist) {
// Existing projects keep their handwritten middleware signature until they
// explicitly select factories; do not rewrite or adopt developer files.
if raw, e := os.ReadFile(filepath.Join(root, manifestPath)); e == nil {
var previous manifest
if err := json.Unmarshal(raw, &previous); err != nil {
return cfg, false, err
}
if previous.GeneratorConfig {
return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options before regenerating")
}
cfg.Middleware = "handlers"
} else if !errors.Is(e, os.ErrNotExist) {
return cfg, false, e
}
return cfg, false, nil
}
if err != nil {
return cfg, false, err
}
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
return cfg, false, errors.New("omni.gen.json must be a regular file of at most 1 MiB")
}
raw, err := readGeneratorConfigFile(root, generatorConfigFile)
if err != nil {
return cfg, false, err
}
var keys map[string]json.RawMessage
if err = apierrors.DecodeCatalogJSON(raw, &keys); err != nil {
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
}
for _, key := range []string{"mappings", "mappingFiles"} {
if _, exists := keys[key]; exists {
return cfg, false, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go, remove mappings/mappingFiles from omni.gen.json, then remove internal/mapping/mappingsGen.go; see COPY.md")
}
}
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
}
digest := sha256.Sum256(raw)
cfg.inputDigest = fmt.Sprintf("%x", digest)
if cfg.Repository != "none" && cfg.Repository != "group" {
return cfg, false, errors.New("omni.gen.json: repository must be none or group")
}
if cfg.Middleware != "factories" && cfg.Middleware != "handlers" {
return cfg, false, errors.New("omni.gen.json: middleware must be factories or handlers")
}
return cfg, true, nil
}
// Read through a bounded reader as well as checking stat: a growing file cannot
// make configuration loading unbounded. safePath also rejects symlink escapes.
func readGeneratorConfigFile(root, name string) ([]byte, error) {
if err := safePath(root, name); err != nil {
return nil, err
}
f, err := os.Open(filepath.Join(root, filepath.FromSlash(name)))
if err != nil {
return nil, fmt.Errorf("%s: %w", name, err)
}
defer f.Close()
stat, err := f.Stat()
if err != nil {
return nil, err
}
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
return nil, fmt.Errorf("%s must be a regular file of at most 1 MiB", name)
}
raw, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
if err != nil {
return nil, err
}
if len(raw) > 1<<20 {
return nil, fmt.Errorf("%s exceeds 1 MiB", name)
}
if trimmed := strings.TrimSpace(string(raw)); !strings.HasPrefix(trimmed, "{") {
return nil, fmt.Errorf("%s must contain a JSON object", name)
}
return raw, nil
}

View File

@@ -0,0 +1,96 @@
package apigen
import (
"encoding/json"
"io"
"os"
"path/filepath"
"strings"
"testing"
)
func TestWorkspaceRejectsRemovedMappingWithoutWrites(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"one", "two"} {
if _, err := New(Options{Dir: filepath.Join(root, name), Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
}
firstAPI := filepath.Join(root, "one/api/service.api")
writeFile(t, firstAPI, strings.Replace(readFile(t, firstAPI), "Greet a visitor", "Changed description", 1))
writeFile(t, filepath.Join(root, "two", generatorConfigFile), `{"mappingFiles":["../../private.json"]}`)
workspace := filepath.Join(root, "omni.workspace.json")
writeFile(t, workspace, `{"services":[{"dir":"one"},{"dir":"two"}]}`)
before := snapshot(t, root)
for _, operation := range []string{"generate", "check"} {
if err := Workspace(workspace, operation, io.Discard); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
t.Fatal(operation, err)
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("workspace migration failure wrote files")
}
}
}
func TestRemovedMappingConfiguration(t *testing.T) {
dir := filepath.Join(t.TempDir(), "project")
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
for _, raw := range []string{`{"mappings":[]}`, `{"mappingFiles":[]}`, `{"mappings":null}`, `{"mappingFiles":["../../private.json"]}`} {
writeFile(t, filepath.Join(dir, generatorConfigFile), raw)
before := snapshot(t, dir)
for _, o := range []Options{{Dir: dir}, {Dir: dir, Check: true}, {Dir: dir, DryRun: true}} {
o.API = filepath.Join(dir, "api/service.api")
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("rejected config wrote project")
}
}
}
}
func TestOldMappingFileRequiresExplicitMigration(t *testing.T) {
dir := filepath.Join(t.TempDir(), "project")
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
name := "internal/mapping/mappingsGen.go"
body := generatedHeader + "package mapping\n"
writeFile(t, filepath.Join(dir, name), body)
var m manifest
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
t.Fatal(err)
}
m.Generated[name] = digest([]byte(body))
raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
before := snapshot(t, dir)
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
for _, mode := range []string{"generate", "check", "dry-run"} {
o.Check = mode == "check"
o.DryRun = mode == "dry-run"
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("legacy file deleted or project changed")
}
}
if err := os.Remove(filepath.Join(dir, name)); err != nil {
t.Fatal(err)
}
o.Check = false
o.DryRun = false
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
o.Check = true
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
if strings.Contains(readFile(t, filepath.Join(dir, manifestPath)), name) {
t.Fatal("old ownership retained")
}
}

View File

@@ -71,6 +71,7 @@ func TestManagedGeneratedProject(t *testing.T) {
files := map[string]string{
"consumer_test.go": "consumer_test.go", "internal/svc/options.go": "options.go", "internal/svc/client.go": "client.go", "internal/logic/demo/check.go": "operations.go",
"shutdown_test.go": "shutdown_test.go",
"phases_test.go": "phases_test.go", "internal/svc/background.go": "background.go",
}
for path, fixture := range files {
writeFile(t, filepath.Join(dir, path), readFile(t, filepath.Join("testdata/managed", fixture)))
@@ -79,7 +80,7 @@ func TestManagedGeneratedProject(t *testing.T) {
writeFile(t, filepath.Join(dir, "internal/logic/demo", name+".go"), "package demo\n")
}
configPath := filepath.Join(dir, "internal/config/config.go")
writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1))
writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n BackgroundMode string `json:\"backgroundMode\"`\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1))
svcPath := filepath.Join(dir, "internal/svc/serviceContext.go")
source := strings.Replace(readFile(t, svcPath), "type ServiceContext struct {", "type ServiceContext struct {\n Client *Client", 1)
source = strings.Replace(source, "return nil", "return initClient(ctx,c)", 1)
@@ -169,12 +170,15 @@ func generateManagedTestProject(t *testing.T, o Options) error {
_, err := Generate(o)
return err
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
args := []string{"api", "generate", "--api", o.API, "--dir", o.Dir, "--module", o.Module}
if o.Check {
args = append(args, "--check")
}
if o.DryRun {
args = append(args, "--dry-run")
}
cmd := exec.CommandContext(ctx, cli, args...)
cmd.Dir = filepath.Dir(o.Dir)
if data, err := cmd.CombinedOutput(); err != nil {

View File

@@ -74,7 +74,7 @@ func document(t *testing.T, c *Compiled) map[string]any {
if err := json.Unmarshal(c.Document, &doc); err != nil {
t.Fatal(err)
}
return doc
return expandDocumentRefs(t, doc, doc).(map[string]any)
}
func operation(t *testing.T, doc map[string]any, path, method string) map[string]any {
t.Helper()
@@ -238,6 +238,7 @@ func TestDeclaredContractConsumer(t *testing.T) {
source := filepath.Join(root, "input/service.api")
writeFile(t, source, declaredAPI)
dir := filepath.Join(root, "project")
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
if _, err := Generate(Options{API: source, Dir: dir, Module: "example.com/dx", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
@@ -303,3 +304,21 @@ func TestAuthMetadataWithParameterNamePrefixes(t *testing.T) {
t.Fatal("auth metadata lost")
}
}
// Existing semantic assertions resolve the standard local references emitted
// by the compiler. documentRefs_test additionally checks the actual references.
func expandDocumentRefs(t *testing.T, doc map[string]any, v any) any {
t.Helper()
switch v:=v.(type){
case map[string]any:
if ref,ok:=v["$ref"].(string);ok && strings.HasPrefix(ref,"#/components/") && len(v)==1 {
var target any=doc
for _,part:=range strings.Split(strings.TrimPrefix(ref,"#/"),"/"){m,ok:=target.(map[string]any);if !ok{t.Fatal("invalid reference",ref)};target=m[part]}
if target==nil{t.Fatal("dangling reference",ref)};return expandDocumentRefs(t,doc,target)
}
out:=map[string]any{};for k,x:=range v{out[k]=expandDocumentRefs(t,doc,x)};return out
case []any:
out:=make([]any,len(v));for i,x:=range v{out[i]=expandDocumentRefs(t,doc,x)};return out
default:return v
}
}

View File

@@ -0,0 +1,19 @@
package apigen
import "fmt"
func renderMiddlewareFactories(names []string, generated, seeds map[string][]byte) {
// All project-specific types live behind this project-owned interface. svc
// implements it; middleware never imports svc (and cannot form that cycle).
seeds["internal/middleware/dependencies.go"] = []byte("package middleware\n// Dependencies is implemented by *svc.ServiceContext. Add typed accessors here\n// and implement them on ServiceContext. Factories borrow dependencies; register\n// their cleanup in ServiceContext.Init, not in a middleware factory.\ntype Dependencies interface {}\n")
text := fmt.Sprintf("package middleware\nimport(\"fmt\";%q)\n// New constructs declared middleware after ServiceContext.Init succeeds.\nfunc New(deps Dependencies)(map[string]httpx.Middleware,error){\nresult:=map[string]httpx.Middleware{}\n", frameworkModule+"/httpx")
if len(names) == 0 {
text = fmt.Sprintf("package middleware\nimport %q\nfunc New(deps Dependencies)(map[string]httpx.Middleware,error){\nresult:=map[string]httpx.Middleware{}\n", frameworkModule+"/httpx")
}
for i, name := range names {
delete(seeds, "internal/middleware/"+lower(name)+".go")
text += fmt.Sprintf("m%d,err:=New%s(deps)\nif err!=nil{return nil,fmt.Errorf(%q,err)}\nif m%d==nil{return nil,fmt.Errorf(%q)}\nresult[%q]=m%d\n", i, upper(name), "middleware "+name+": %w", i, "middleware "+name+": factory returned nil", name, i)
seeds["internal/middleware/"+lower(name)+"Factory.go"] = []byte(fmt.Sprintf("package middleware\nimport(\"errors\";%q)\n// New%s is project-owned. Implement this factory once; registration is generated.\n// Dependencies have already been initialized. Do not acquire owned resources here.\nfunc New%s(deps Dependencies)(httpx.Middleware,error){\nreturn nil,errors.New(%q)\n}\n", frameworkModule+"/httpx", upper(name), upper(name), "implement "+name+" middleware factory"))
}
generated["internal/middleware/registryGen.go"] = []byte(generatedHeader + text + "return result,nil\n}\n")
}

View File

@@ -17,6 +17,7 @@ import (
type Spec struct {
DocumentCatalog *apierrors.Catalog
DocumentErrors *DocumentErrors
SharedRoots map[string]SharedRoot
SourcePaths map[string]string
RequestPolicy contract.RequestPolicy
@@ -38,6 +39,8 @@ type Type struct {
Fields []Field
Position string
}
// An empty Type denotes a named field composition, expanded before compilation.
type Field struct{ Name, Type, Tag, Description string }
type Route struct {
BoundMultipart bool
@@ -72,7 +75,9 @@ type parser struct {
imports []string
}
func Parse(path string) (*Spec, error) {
func Parse(path string) (*Spec, error) { return parseWithCatalogs(path, nil) }
func parseWithCatalogs(path string, catalogs map[string][]byte) (*Spec, error) {
entry, err := filepath.Abs(path)
if err != nil {
return nil, err
@@ -82,11 +87,12 @@ func Parse(path string) (*Spec, error) {
return nil, err
}
root := filepath.Dir(entry)
roots, err := readSharedRoots(root)
shared, err := readSharedConfig(root)
if err != nil {
return nil, err
}
spec := &Spec{Version: "1.0.0", SharedRoots: roots, SourcePaths: map[string]string{}, Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
roots := shared.Roots
spec := &Spec{Version: "1.0.0", DocumentErrors: shared.Errors, SharedRoots: roots, SourcePaths: map[string]string{}, Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
visiting, visited := map[string]bool{}, map[string]bool{}
total := 0
var visit func(string, string, int) error
@@ -169,7 +175,7 @@ func Parse(path string) (*Spec, error) {
if err = visit(entry, "", 0); err != nil {
return nil, err
}
if err = readDocumentCatalog(root, spec); err != nil {
if err = readDocumentCatalog(root, spec, catalogs); err != nil {
return nil, err
}
if err = validateSharedPins(spec); err != nil {
@@ -513,7 +519,11 @@ func (p *parser) parseType() error {
}
}
if expression == "" {
return p.err("field type required; embedded types are not supported")
if p.tokens[p.at].kind == scanner.RawString {
return p.err("composed types cannot carry field tags")
}
typ.Fields = append(typ.Fields, Field{Name: name, Description: description})
continue
}
if p.tokens[p.at].kind != scanner.RawString {
return p.err("field source tag required")

View File

@@ -73,6 +73,8 @@ func TestConfiguration(t *testing.T){
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
t.Setenv("GO_OMNI_SERVER_LOG_FILE","./explicit.log");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!="./explicit.log"{t.Fatalf("canonical log file: %v",err)}
t.Setenv("GO_OMNI_SERVER_LOG_OUTPUT","stderr");cfg,err=Load("");if err!=nil||cfg.Server.Log.Output!="stderr"{t.Fatalf("canonical output: %v",err)}
_=os.Getenv("unused")
}`
endpoint := ""

View File

@@ -4,11 +4,12 @@ import (
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/openapi"
"reflect"
"strings"
)
// Response roots deliberately allow named DTOs, []DTO, *DTO and null. Keeping
// these four shapes explicit makes nil/empty behavior uniform across encoders.
// Response roots allow named DTOs, []DTO, []primitive, *DTO and null.
// A nil root slice is always encoded as an empty JSON array.
func responseShape(value string) (kind, name string) {
if value == "null" {
return "null", ""
@@ -27,6 +28,9 @@ func responseGoType(value string) string {
return ""
}
if kind == "array" {
if primitiveTypes[name] != nil {
return "[]" + name
}
return "[]types." + name
}
return "*types." + name
@@ -36,9 +40,25 @@ func responseDataSchema(value string, bindings map[string]*contract.Binder) (*op
if kind == "null" {
return &openapi.Schema{Type: "object", Nullable: true, Enum: []any{nil}}, nil
}
if kind == "array" && primitiveTypes[name] != nil {
if name == "File" {
return nil, fmt.Errorf("File is request-only; use a response DTO")
}
// Reuse the contract compiler, including integer bounds and byte-slice rules.
t := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: reflect.SliceOf(primitiveTypes[name]), Tag: `json:"value"`}})
b, err := contract.Describe(t)
if err != nil {
return nil, err
}
s, err := b.ResponseSchema()
if err != nil {
return nil, err
}
return s.Properties["value"], nil
}
b, ok := bindings[name]
if !ok {
return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, *Name or null", name)
return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, []string/[]int64/[]bool/[]any, *Name or null", name)
}
s, err := b.ResponseSchema()
if err != nil {
@@ -52,3 +72,7 @@ func responseDataSchema(value string, bindings map[string]*contract.Binder) (*op
}
return s, nil
}
var primitiveTypes = map[string]reflect.Type{"File": reflect.TypeOf(contract.File{}), "any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
func responseNeedsTypes(value string) bool { return strings.Contains(responseGoType(value), "types.") }

View File

@@ -251,6 +251,7 @@ func TestResponseTemplateConsumer(t *testing.T) {
t.Fatal(err)
}
}
writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`)
generate(source, true)
writeFile(t, filepath.Join(dir, "response_test.go"), responseConsumerTest)
for _, method := range []string{"Item", "Callback", "Partner"} {

View File

@@ -23,48 +23,49 @@ type SharedRoot struct {
SHA256 string `json:"sha256,omitempty"`
}
type sharedConfig struct {
Roots map[string]SharedRoot `json:"roots"`
Roots map[string]SharedRoot `json:"roots"`
Errors *DocumentErrors `json:"errors,omitempty"`
}
func readSharedRoots(root string) (map[string]SharedRoot, error) {
func readSharedConfig(root string) (sharedConfig, error) {
stat, err := os.Lstat(filepath.Join(root, importsFile))
if errors.Is(err, os.ErrNotExist) {
return nil, nil
return sharedConfig{}, nil
}
if err != nil {
return nil, err
return sharedConfig{}, err
}
if !stat.Mode().IsRegular() {
return nil, errors.New("api: shared root configuration must be a regular file")
return sharedConfig{}, errors.New("api: shared root configuration must be a regular file")
}
f, err := os.Open(filepath.Join(root, importsFile))
if errors.Is(err, os.ErrNotExist) {
return nil, nil
return sharedConfig{}, nil
}
if err != nil {
return nil, err
return sharedConfig{}, err
}
defer f.Close()
raw, err := io.ReadAll(io.LimitReader(f, 65537))
if err != nil {
return nil, err
return sharedConfig{}, err
}
if len(raw) > 65536 {
return nil, errors.New("api: shared root config too large")
return sharedConfig{}, errors.New("api: shared root config too large")
}
var cfg sharedConfig
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
return nil, err
return sharedConfig{}, err
}
if len(cfg.Roots) > 16 {
return nil, errors.New("api: at most 16 shared roots")
return sharedConfig{}, errors.New("api: at most 16 shared roots")
}
for name, r := range cfg.Roots {
if !lowerName.MatchString(name) || r.Path == "" || len(r.Version) > 128 {
return nil, errors.New("api: invalid shared root")
return sharedConfig{}, errors.New("api: invalid shared root")
}
if r.SHA256 != "" && (len(r.SHA256) != 64 || strings.Trim(r.SHA256, "0123456789abcdef") != "") {
return nil, errors.New("api: shared sha256 must be 64 lowercase hex characters")
return sharedConfig{}, errors.New("api: shared sha256 must be 64 lowercase hex characters")
}
path := r.Path
if !filepath.IsAbs(path) {
@@ -72,22 +73,22 @@ func readSharedRoots(root string) (map[string]SharedRoot, error) {
}
r.Path, err = filepath.EvalSymlinks(path)
if err != nil {
return nil, err
return sharedConfig{}, err
}
r.Path, err = filepath.Abs(r.Path)
if err != nil {
return nil, err
return sharedConfig{}, err
}
stat, err := os.Stat(r.Path)
if err != nil {
return nil, err
return sharedConfig{}, err
}
if !stat.IsDir() {
return nil, errors.New("api: shared root must be a directory")
return sharedConfig{}, errors.New("api: shared root must be a directory")
}
cfg.Roots[name] = r
}
return cfg.Roots, nil
return cfg, nil
}
func sourceWithin(root, name string) (string, error) {
real, err := filepath.EvalSymlinks(name)
@@ -122,7 +123,7 @@ func sourceMetadata(spec *Spec) map[string]any {
return map[string]any{"files": hashes, "shared": shared}
}
func sharedSeeds(spec *Spec, directory string) ([]byte, error) {
cfg := sharedConfig{Roots: map[string]SharedRoot{}}
cfg := sharedConfig{Roots: map[string]SharedRoot{}, Errors: spec.DocumentErrors}
for name, r := range spec.SharedRoots {
rel, err := filepath.Rel(filepath.Join(directory, "api"), r.Path)
if err != nil {

View File

@@ -13,6 +13,7 @@ import (
"os/signal"
"syscall"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"MODULE/internal/config"
"MODULE/internal/handler"
"MODULE/internal/svc"
@@ -27,7 +28,9 @@ func main(){
func run(ctx context.Context,path string)(runErr error){
cfg,err:=config.Load(path);if err!=nil{return err}
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,logger.Close())}()
// Unjoined background work may still log. The owning process exits on this
// error; do not close its borrowed logger before that work has finished.
defer func(){if !errors.Is(runErr,lifecycle.ErrResourcesRetained){runErr=errors.Join(runErr,logger.Close())}}()
deps,err:=svc.New(ctx,logger,cfg);if err!=nil{return err}
// RunContext drains HTTP requests before returning. This scope owns dependencies
// and closes them exactly once, before closing the borrowed logger.
@@ -48,6 +51,19 @@ This project is generated from an editable API contract. Business code uses
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
main.go and HTTP composition are continuously generator-owned.
omni.gen.json selects repository: none (default) or group, and middleware:
factories for new projects. Named middleware factories run after svc.Init;
implement Dependencies accessors on ServiceContext and New<Name>(deps) once.
Unimplemented/nil factories fail startup and clean registered dependencies.
Existing projects retain middleware: handlers until explicitly migrated.
Ordinary Logic uses the generated Request/Response types directly. Where an
independent business type is needed, use ordinary Go or the optional copyx package.
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use one pipeline. See the framework CONFIG.md.
There are no mappings/mappingFiles settings or generated mapping files in v0.19.
See COPY.md in the framework repository for copy policies and migration steps,
and GENERATION.md for field composition and middleware factories.
## Run
`+"```sh"+`
@@ -130,8 +146,9 @@ internal/svc/runtimeGen.go # GENERATED lifecycle and infrastructure startup
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
internal/handler/<group>/ # generated binding and Logic adapters
internal/logic/<group>/ # developer-owned business operations
internal/repository/<group>/ # developer-owned persistence contracts
internal/middleware/ # named hooks and generated registry
internal/repository/<group>/ # optional; repository: group creates editable seeds
internal/middleware/ # typed dependency interface, factories and registry
omni.gen.json # developer-owned repository/middleware options
docs/openapi.json # generated OpenAPI 3.0.3
.goOmni/apiManifest.json # ownership and content hashes
`+"```"+`
@@ -182,7 +199,16 @@ then immediately register each cleanup with OnClose. A later initialization fail
runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware
provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated
once before route registration; logging is configured earlier through Config.
The generated main runs HTTP draining before closing svc resources, then the logger.
The generated main runs HTTP draining, all OnStop hooks, all OnDrain hooks, resource
OnClose callbacks, then logger close. Stop/drain hooks within a phase run concurrently;
combine components with ordering dependencies into one pair. Pair OnStop/OnDrain
with the same name. OnDrain returns nil only after final work and acknowledgements
have finished. Use WorkContext for accepted background work; Context is an early
shutdown notification. server.lifecycle configures stopTimeout (5s), drainTimeout
(10s), and cancelTimeout (5s grace per timed-out phase). Zero uses defaults.
Timeout/failure cancels WorkContext. If hooks cannot prove completion, resources
and the borrowed logger are retained and the generated process exits unsuccessfully.
Errors retain their causes. No automatic retry of close or force killing of goroutines.
Do not close borrowed resources or attach duplicate cleanup hooks to App. The startup
context cancels on signals; long-lived clients/workers that requests depend on need
their own lifetime, canceled by OnClose after HTTP draining.

View File

@@ -0,0 +1,54 @@
package main
import (
"context"
"encoding/json"
"example.com/arrays/internal/config"
"example.com/arrays/internal/handler"
"example.com/arrays/internal/svc"
"git.yuntuo.hk/k2safe/go-omni/logx"
"io"
"net/http/httptest"
"strings"
"testing"
)
func TestArrayWire(t *testing.T) {
cfg := config.Default()
cfg.Server.Log.Writer = io.Discard
logger, err := logx.New(cfg.Server.Log)
if err != nil {
t.Fatal(err)
}
defer logger.Close()
deps, err := svc.New(context.Background(), logger, cfg)
if err != nil {
t.Fatal(err)
}
defer deps.Close()
app, err := handler.NewApp(deps)
if err != nil {
t.Fatal(err)
}
defer app.Close()
for path, want := range map[string]string{
"/strings": `["企业","备注"]`, "/integers": `[9223372036854775807,-3]`, "/dynamic": `["text",7,true,null,{"name":"企业"},[false,2]]`, "/empty": `[]`, "/raw": `[true,false]`,
} {
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("GET", path, nil))
got := strings.TrimSpace(w.Body.String())
if w.Code != 200 {
t.Fatal(got)
}
if path != "/raw" {
var obj map[string]json.RawMessage
if err := json.Unmarshal(w.Body.Bytes(), &obj); err != nil {
t.Fatal(err)
}
got = string(obj["data"])
}
if got != want {
t.Fatalf("%s got %s want %s", path, got, want)
}
}
}

View File

@@ -0,0 +1,13 @@
package arrays
import "context"
func (s *Logic) Strings(context.Context) ([]string, error) { return []string{"企业", "备注"}, nil }
func (s *Logic) Integers(context.Context) ([]int64, error) {
return []int64{9223372036854775807, -3}, nil
}
func (s *Logic) Dynamic(context.Context) ([]any, error) {
return []any{"text", 7, true, nil, map[string]any{"name": "企业"}, []any{false, 2}}, nil
}
func (s *Logic) Empty(context.Context) ([]any, error) { return nil, nil }
func (s *Logic) Raw(context.Context) ([]bool, error) { return []bool{true, false}, nil }

View File

@@ -0,0 +1,30 @@
package middleware
import (
"errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
var ErrFactory = errors.New("factory fault")
func NewAccess(deps Dependencies) (httpx.Middleware, error) {
if deps.Access() == nil {
return nil, errors.New("dependencies not initialized")
}
switch deps.FactoryMode() {
case "fail":
return nil, ErrFactory
case "nil":
return nil, nil
case "panic":
panic(ErrFactory)
}
return func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
if err := deps.Access().Check(c.Context()); err != nil {
return err
}
return next(c)
}
}, nil
}

View File

@@ -0,0 +1,52 @@
package amount
import (
"encoding/json"
"errors"
"fmt"
"math/big"
"regexp"
)
// This fixture's business policy is exact decimal HALF_UP. The framework does
// not choose its precision, rounding mode or currency.
type Money string
var decimalPattern = regexp.MustCompile(`^-?[0-9]+(\.[0-9]+)?$`)
func (m Money) String() string {
if m == "" {
return "0.00"
}
return string(m)
}
func (m Money) MarshalJSON() ([]byte, error) { return json.Marshal(m.String()) }
func (m *Money) UnmarshalJSON(data []byte) error {
var text string
if err := json.Unmarshal(data, &text); err != nil {
return err
}
if !decimalPattern.MatchString(text) {
return errors.New("invalid decimal")
}
value, ok := new(big.Rat).SetString(text)
if !ok {
return errors.New("invalid decimal")
}
value.Mul(value, big.NewRat(100, 1))
sign := value.Sign()
value.Abs(value)
cents, rem := new(big.Int), new(big.Int)
cents.QuoRem(value.Num(), value.Denom(), rem)
if rem.Mul(rem, big.NewInt(2)).Cmp(value.Denom()) >= 0 {
cents.Add(cents, big.NewInt(1))
}
whole, fraction := new(big.Int), new(big.Int)
whole.QuoRem(cents, big.NewInt(100), fraction)
prefix := ""
if sign < 0 && cents.Sign() != 0 {
prefix = "-"
}
*m = Money(fmt.Sprintf("%s%s.%02d", prefix, whole.String(), fraction.Int64()))
return nil
}

View File

@@ -0,0 +1,36 @@
package svc
import (
"context"
"errors"
"example.com/composition/internal/middleware"
"sync/atomic"
)
var Closes atomic.Int64
var ErrInit = errors.New("init fault")
type Client struct{ closed atomic.Bool }
func (c *Client) Check(ctx context.Context) error {
if c.closed.Load() {
return errors.New("client closed before HTTP drained")
}
return ctx.Err()
}
func (c *ServiceContext) initClient(ctx context.Context) error {
c.client = &Client{}
c.OnClose(func() error {
if !c.client.closed.CompareAndSwap(false, true) {
return errors.New("double close")
}
Closes.Add(1)
return nil
})
if c.Config.FactoryMode == "initFail" {
return ErrInit
}
return nil
}
func (c *ServiceContext) Access() middleware.AccessChecker { return c.client }
func (c *ServiceContext) FactoryMode() string { return c.Config.FactoryMode }

View File

@@ -0,0 +1,217 @@
package main
import (
"context"
"encoding/json"
"errors"
"io"
"net"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"sync"
"testing"
"example.com/composition/internal/amount"
"example.com/composition/internal/config"
"example.com/composition/internal/convert"
"example.com/composition/internal/domain"
"example.com/composition/internal/handler"
"example.com/composition/internal/middleware"
"example.com/composition/internal/svc"
"example.com/composition/internal/types"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
func logger(t *testing.T) *logx.Logger {
t.Helper()
cfg := logx.Default()
cfg.Writer = io.Discard
l, e := logx.New(cfg)
if e != nil {
t.Fatal(e)
}
t.Cleanup(func() { _ = l.Close() })
return l
}
func TestHTTPComposition(t *testing.T) {
deps, e := svc.New(context.Background(), logger(t))
if e != nil {
t.Fatal(e)
}
defer deps.Close()
app, e := handler.NewApp(deps)
if e != nil {
t.Fatal(e)
}
defer app.Close()
for _, path := range []string{"/enterprise", "/tenant/123", "/admin/123"} {
for _, tc := range []struct {
name, body string
status int
salary, nameValue string
employer any
}{
{"exact", `{"name":"employee","salary":"90071992547409.91","employer":null,"enabled":false}`, 200, "90071992547409.91", "employee", nil},
{"round", `{"salary":"1.005","employer":"0"}`, 200, "1.01", "", "0.00"},
{"missing", `{}`, 200, "0.00", "", nil},
{"null", `{"name":null,"salary":null,"employer":null,"enabled":null}`, 200, "0.00", "", nil},
{"zero", `{"salary":"0","enabled":false}`, 200, "0.00", "", nil},
{"invalid", `{"salary":"abc"}`, 400, "", "", nil},
{"empty", `{"salary":""}`, 400, "", "", nil},
{"numeric", `{"salary":100}`, 400, "", "", nil},
{"flat", `{"SalaryFields":{"salary":"1"}}`, 400, "", "", nil},
{"validation", `{"name":"` + strings.Repeat("x", 81) + `"}`, 400, "", "", nil},
} {
t.Run(path+tc.name, func(t *testing.T) {
req := httptest.NewRequest("POST", path, strings.NewReader(tc.body))
req.Header.Set("Content-Type", "application/json")
w := httptest.NewRecorder()
app.Engine().ServeHTTP(w, req)
if w.Code != tc.status {
t.Fatalf("status=%d body=%s", w.Code, w.Body.String())
}
if w.Code == 200 {
var out map[string]any
if e := json.Unmarshal(w.Body.Bytes(), &out); e != nil {
t.Fatal(e)
}
if out["salary"] != tc.salary || out["employeeName"] != tc.nameValue || out["employer"] != tc.employer || out["id"] != "9007199254740993" {
t.Fatal(out)
}
}
})
}
}
}
func TestFullFormAndCopySemantics(t *testing.T) {
name := "employee"
money := amount.Money("2.00")
yes := true
req := types.EnterpriseRequest{Name: &name, Salary: &money, Enabled: &yes, Labels: []string{"one"}}
first, e := convert.Employee(req)
if e != nil {
t.Fatal(e)
}
second, e := convert.Employee(types.EnterpriseRequest{})
if e != nil {
t.Fatal(e)
}
if first.EmployeeName != "employee" || !first.Enabled || second.EmployeeName != "" || second.Enabled || second.Salary.String() != "0.00" || second.Employer != nil {
t.Fatal("full-form update became patch behavior")
}
// Direct assignments retain standard Go aliasing, including scalar pointers.
first.Labels[0] = "shared"
if req.Labels[0] != "shared" {
t.Fatal("unexpected implicit deep copy")
}
req.Employer = &money
first, e = convert.Employee(req)
if e != nil || first.Employer != req.Employer {
t.Fatal("nullable pointer changed", e)
}
bad := "reject"
failed, e := convert.Employee(types.EnterpriseRequest{Name: &bad})
if !errors.Is(e, convert.ErrName) || failed.EmployeeName != "" || !strings.Contains(e.Error(), "EmployeeName") {
t.Fatal("converter cause/zero result lost", e)
}
empty, e := convert.ToBatch(domain.Batch{})
if e != nil || empty.Rows != nil {
t.Fatal("nil slice changed", e)
}
src := domain.Batch{Rows: []domain.Row{}, Fixed: [2]domain.Row{{Value: "a"}, {Value: "b"}}, Optional: &name}
out, e := convert.ToBatch(src)
if e != nil || out.Rows == nil || len(out.Rows) != 0 || out.Fixed[1].Value != "b" || out.Optional != src.Optional {
t.Fatal(out, e)
}
src.Rows = []domain.Row{{Value: "one"}}
out, e = convert.ToBatch(src)
if e != nil || out.Rows[0].Value != "one" {
t.Fatal(out, e)
}
out.Rows[0].Value = "changed"
if src.Rows[0].Value != "one" {
t.Fatal("element mapping reused input slice")
}
var wg sync.WaitGroup
for i := 0; i < 32; i++ {
wg.Add(1)
go func() {
defer wg.Done()
result, err := convert.Employee(types.EnterpriseRequest{Name: &name, Salary: &money})
if err != nil || result.EmployeeName != name || result.Salary != money {
t.Error(result, err)
}
}()
}
wg.Wait()
}
func TestFactoryLifecycle(t *testing.T) {
for _, mode := range []string{"fail", "nil", "initFail", "panic"} {
t.Run(mode, func(t *testing.T) {
before := svc.Closes.Load()
cfg := config.Default()
cfg.FactoryMode = mode
if mode == "panic" {
func() {
defer func() {
if recover() != middleware.ErrFactory {
t.Error("panic cause changed")
}
}()
_, _ = svc.New(context.Background(), logger(t), cfg)
}()
} else {
deps, err := svc.New(context.Background(), logger(t), cfg)
if deps != nil || err == nil {
t.Fatal("factory did not fail closed")
}
if mode == "fail" && !errors.Is(err, middleware.ErrFactory) {
t.Fatal("factory cause lost", err)
}
if mode == "initFail" && !errors.Is(err, svc.ErrInit) {
t.Fatal("init cause lost", err)
}
}
if svc.Closes.Load() != before+1 {
t.Fatal("dependency not closed exactly once")
}
})
}
deps, e := svc.New(context.Background(), logger(t))
if e != nil {
t.Fatal(e)
}
if e = deps.Access().Check(context.Background()); e != nil {
t.Fatal(e)
}
before := svc.Closes.Load()
if e = deps.Close(); e != nil {
t.Fatal(e)
}
if e = deps.Close(); e != nil {
t.Fatal(e)
}
if svc.Closes.Load() != before+1 {
t.Fatal("normal close repeated")
}
}
func TestGeneratedEntryListenFailure(t *testing.T) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer listener.Close()
path := filepath.Join(t.TempDir(), "config.yaml")
if err = os.WriteFile(path, []byte("server:\n address: "+listener.Addr().String()+"\n"), 0600); err != nil {
t.Fatal(err)
}
before := svc.Closes.Load()
if err = run(context.Background(), path); err == nil {
t.Fatal("listen should fail")
}
if svc.Closes.Load() != before+1 {
t.Fatal("listen failure did not close dependencies")
}
}

View File

@@ -0,0 +1,21 @@
package convert
import (
"errors"
"strconv"
)
var ErrName = errors.New("rejected name")
// Explicit project rules: full-form omission clears strings/bools and uses zero
// for required money. Nullable employer amounts remain nil by direct assignment.
func Name(value *string) (string, error) {
if value == nil {
return "", nil
}
if *value == "reject" {
return "", ErrName
}
return *value, nil
}
func ID(value int64) string { return strconv.FormatInt(value, 10) }

View File

@@ -0,0 +1,29 @@
package convert
import (
"example.com/composition/internal/domain"
"example.com/composition/internal/types"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
// The three endpoints reuse one ordinary Go helper and their existing scalar.
// There is no field list or package/type name JSON to keep in sync.
func Employee[S any](source S) (domain.Employee, error) {
var target domain.Employee
err := copyx.Copy(&target, source,
copyx.WithFieldNames[S, domain.Employee](map[string]string{"Name": "EmployeeName"}),
copyx.WithNilPolicy(copyx.NilZero),
copyx.WithConverter(Name),
)
return target, err
}
func Response(source domain.Result) (types.EmployeeResponse, error) {
var target types.EmployeeResponse
err := copyx.Copy(&target, source, copyx.WithConverter(func(v int64) (string, error) { return ID(v), nil }))
return target, err
}
func ToBatch(source domain.Batch) (domain.BatchDTO, error) {
var target domain.BatchDTO
err := copyx.Copy(&target, source)
return target, err
}

View File

@@ -0,0 +1,9 @@
package middleware
import "context"
type AccessChecker interface{ Check(context.Context) error }
type Dependencies interface {
Access() AccessChecker
FactoryMode() string
}

View File

@@ -0,0 +1,29 @@
package domain
import "example.com/composition/internal/amount"
type Employee struct {
EmployeeName string
Salary amount.Money
Employer *amount.Money
Enabled bool
Labels []string
}
type Result struct {
Employee
ID int64
}
// Distinct container item types exercise explicit Go collection copying.
type Row struct{ Value string }
type RowDTO struct{ Value string }
type Batch struct {
Rows []Row
Fixed [2]Row
Optional *string
}
type BatchDTO struct {
Rows []RowDTO
Fixed [2]RowDTO
Optional *string
}

View File

@@ -0,0 +1,63 @@
scalar Money (
goImport: "example.com/composition/internal/amount"
goType: Money
jsonInput: string
jsonOutput: string
)
type SalaryFields {
// Employee display name.
Name *string `json:"name,optional" binding:"max=80"`
Salary *Money `json:"salary,optional"`
Employer *Money `json:"employer,optional"`
Enabled *bool `json:"enabled,optional"`
Labels []string `json:"labels,optional"`
}
type EnterpriseRequest {
SalaryFields
}
type TenantRequest {
SalaryFields
CompanyID string `path:"companyId"`
}
type AdminRequest {
SalaryFields
CompanyID string `path:"companyId"`
}
type EmployeeResponse {
ID string `json:"id"`
EmployeeName string `json:"employeeName"`
Salary Money `json:"salary"`
Employer *Money `json:"employer"`
Enabled bool `json:"enabled"`
Labels []string `json:"labels"`
}
@server (
group: enterprise
middleware: Access
response: raw
)
service composition {
@handler save
post /enterprise (EnterpriseRequest) returns (EmployeeResponse)
}
@server (
group: tenant
middleware: Access
response: raw
)
service composition {
@handler save
post /tenant/:companyId (TenantRequest) returns (EmployeeResponse)
}
@server (
group: admin
middleware: Access
response: raw
)
service composition {
@handler save
post /admin/:companyId (AdminRequest) returns (EmployeeResponse)
}

View File

@@ -0,0 +1,59 @@
package main
import (
"context"
"errors"
"example.com/configuration/internal/config"
omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
"os"
"path/filepath"
"strings"
"testing"
)
func TestGeneratedConfiguration(t *testing.T) {
for ext, source := range map[string]string{
"json": `{"limit":7}`, "yaml": "limit: 7\n", "xml": "<config><limit>7</limit></config>", "custom": `{"limit":7}`,
} {
t.Run(ext, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "app."+ext)
if err := os.WriteFile(path, []byte(source), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.Load(path)
if err != nil {
t.Fatal(err)
}
if cfg.Limit != 7 {
t.Fatal(cfg.Limit)
}
t.Setenv("GO_OMNI_LIMIT", "0")
cfg, err = config.Load(path)
if err != nil || cfg.Limit != 0 {
t.Fatal(cfg.Limit, err)
}
})
}
t.Setenv("GO_OMNI_LOG_OUTPUT", "stderr")
cfg, err := config.Load("")
if err != nil {
t.Fatal(err)
}
if cfg.Limit != 10 || cfg.Server.Log.File != "" || cfg.Server.Log.Rotation.Enabled {
t.Fatal("defaults/log normalization", cfg, err)
}
path := filepath.Join(t.TempDir(), "bad.yaml")
os.WriteFile(path, []byte("limit: -1\n"), 0600)
// Validation must run before svc.Initialize; the project initializer panics
// below if reached, so this also detects accidentally starting dependencies.
err = run(context.Background(), path)
var diagnostic *omniConfig.Error
if !errors.As(err, &diagnostic) || diagnostic.Field != "limit" || diagnostic.File != path || diagnostic.Line != 1 {
t.Fatal(err)
}
t.Setenv("GO_OMNI_LIMIT", "private-not-a-number")
_, err = config.Load(path)
if !errors.As(err, &diagnostic) || diagnostic.Env != "GO_OMNI_LIMIT" || strings.Contains(err.Error(), "private-not-a-number") {
t.Fatal(err)
}
}

View File

@@ -0,0 +1,49 @@
package svc
import (
"context"
"errors"
"git.yuntuo.hk/k2safe/go-omni/task"
"sync/atomic"
"time"
)
type BackgroundState struct {
Stop, Drain, Release, Started chan struct{}
Wrote atomic.Bool
Future *task.Future
}
var Background atomic.Pointer[BackgroundState]
func initBackground(c *ServiceContext) error {
state := &BackgroundState{Stop: make(chan struct{}), Drain: make(chan struct{}), Release: make(chan struct{}), Started: make(chan struct{})}
baseline := Closed.Load()
runner, err := task.New(task.Config{Workers: 1, QueueSize: 1, TaskTimeout: time.Minute, OnError: func(string, error) {}})
if err != nil {
return err
}
c.OnStop("jobs", func(ctx context.Context) error { close(state.Stop); return runner.StopAccepting(ctx) })
c.OnDrain("jobs", func(ctx context.Context) error { close(state.Drain); return runner.Wait(ctx) })
state.Future, err = runner.Submit(c.WorkContext(), "finish", func(ctx context.Context) error {
close(state.Started)
if c.Config.BackgroundMode == "auto" {
<-state.Stop
} else {
<-state.Release
}
if ctx.Err() != nil {
return ctx.Err()
}
if Closed.Load() != baseline {
return errors.New("shared client closed before background write")
}
state.Wrote.Store(true)
return nil
})
if err == nil {
<-state.Started
}
Background.Store(state)
return err
}

View File

@@ -20,6 +20,11 @@ func initClient(ctx context.Context, c *ServiceContext) error {
Opened.Add(1)
c.Client = &Client{}
c.OnClose(func() error { Closed.Add(1); c.Logger.Info("test.client.closed"); return nil })
if c.Config.BackgroundMode != "" {
if err := initBackground(c); err != nil {
return err
}
}
if c.Config.FailInit {
return errors.New("client initialization failed")
}

View File

@@ -0,0 +1,155 @@
package main
import (
"context"
"encoding/json"
"errors"
"example.com/managed/internal/svc"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"io"
"net"
"net/http"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func phasedConfig(t *testing.T, address, mode string, fail bool) string {
t.Helper()
path := filepath.Join(t.TempDir(), "config.json")
drainTimeout := "1s"
if mode == "stuck" {
drainTimeout = "50ms"
}
data, _ := json.Marshal(map[string]any{"backgroundMode": mode, "failInit": fail, "server": map[string]any{"address": address, "lifecycle": map[string]string{"stopTimeout": "1s", "drainTimeout": drainTimeout, "cancelTimeout": "50ms"}}})
if err := os.WriteFile(path, data, 0600); err != nil {
t.Fatal(err)
}
return path
}
func await(t *testing.T, ch <-chan struct{}) {
t.Helper()
select {
case <-ch:
case <-time.After(3 * time.Second):
t.Fatal("phase did not arrive")
}
}
func TestGeneratedBackgroundPhases(t *testing.T) {
for _, transport := range []string{"nethttp", "native"} {
t.Run(transport, func(t *testing.T) {
t.Setenv("GO_OMNI_TRANSPORT", transport)
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
addr := listener.Addr().String()
_ = listener.Close()
svc.Started = make(chan struct{}, 1)
svc.Release = make(chan struct{})
before := svc.Closed.Load()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
done := make(chan error, 1)
path := phasedConfig(t, addr, "hold", false)
go func() { done <- run(ctx, path) }()
waitReady(t, "http://"+addr, done)
state := svc.Background.Load()
defer func() {
select {
case <-state.Release:
default:
close(state.Release)
}
}()
requestDone := make(chan error, 1)
go func() {
client := http.Client{Timeout: 3 * time.Second}
resp, e := client.Get("http://" + addr + "/slow")
if e == nil {
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
if resp.StatusCode != 200 {
e = errors.New("HTTP drain failed")
}
}
requestDone <- e
}()
await(t, svc.Started)
deps := svc.Current.Load()
deps.RequestShutdown(svc.FatalDependency)
if deps.WorkContext().Err() != nil {
t.Fatal("accepted work canceled at signal")
}
select {
case <-state.Stop:
t.Fatal("stopped background before HTTP finished")
default:
}
close(svc.Release)
if err := <-requestDone; err != nil {
t.Fatal(err)
}
await(t, state.Drain)
if svc.Closed.Load() != before {
t.Fatal("client closed during background drain")
}
close(state.Release)
if err := <-done; !errors.Is(err, svc.FatalDependency) || errors.Is(err, lifecycle.ErrResourcesRetained) {
t.Fatal(err)
}
if !state.Wrote.Load() || svc.Closed.Load() != before+1 {
t.Fatal("missing final write or cleanup")
}
if err := state.Future.Wait(context.Background()); err != nil {
t.Fatal(err)
}
})
}
}
func TestGeneratedBackgroundFailureCleanup(t *testing.T) {
for _, failInit := range []bool{false, true} {
t.Run(map[bool]string{true: "initialization", false: "listen"}[failInit], func(t *testing.T) {
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer l.Close()
before := svc.Closed.Load()
err = run(context.Background(), phasedConfig(t, l.Addr().String(), "auto", failInit))
if err == nil || errors.Is(err, lifecycle.ErrResourcesRetained) {
t.Fatal(err)
}
state := svc.Background.Load()
if !state.Wrote.Load() || svc.Closed.Load() != before+1 {
t.Fatal("failure cleanup did not join background first")
}
})
}
}
func TestGeneratedBackgroundTimeoutProcess(t *testing.T) {
if path := os.Getenv("OMNI_PHASE_TIMEOUT_CONFIG"); path != "" {
os.Args = []string{os.Args[0], "-config", path}
main()
return
}
path := phasedConfig(t, "127.0.0.1:0", "stuck", true)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestGeneratedBackgroundTimeoutProcess$")
cmd.Env = append(os.Environ(), "OMNI_PHASE_TIMEOUT_CONFIG="+path)
output, err := cmd.CombinedOutput()
var exit *exec.ExitError
if !errors.As(err, &exit) || exit.ExitCode() != 1 || ctx.Err() != nil {
t.Fatal(err, string(output))
}
if strings.Contains(string(output), "test.client.closed") || !strings.Contains(string(output), "shared resources retained") || !strings.Contains(string(output), "client initialization failed") {
t.Fatal(string(output))
}
}

View File

@@ -30,6 +30,39 @@ type UpgradePlan struct {
Module, Entry string
}
// Handwritten old loaders must be migrated explicitly, never silently replaced
// or left competing with the generated entry point.
func checkConfigLoadExtension(root string) error {
dir := filepath.Join(root, "internal/config")
entries, err := os.ReadDir(dir)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return err
}
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") || name == "loadGen.go" {
continue
}
rel := "internal/config/" + name
if err := safePath(root, rel); err != nil {
return err
}
f, err := goparser.ParseFile(gotoken.NewFileSet(), filepath.Join(root, rel), nil, 0)
if err != nil {
return fmt.Errorf("parse %s: %w", rel, err)
}
for _, decl := range f.Decls {
if fn, ok := decl.(*ast.FuncDecl); ok && fn.Recv == nil && fn.Name.Name == "Load" {
return fmt.Errorf("%s: migrate the project-owned Load function before generating internal/config/loadGen.go; retain Config/Default/Validate and move decoder options to NewLoader; see CONFIG.md", rel)
}
}
}
return nil
}
// PlanUpgrade compares infrastructure against the exact previous release seeds.
// A custom file is never classified as safe to replace. Logic, middleware,
// options and repositories outside the changed templates are left untouched.
@@ -78,6 +111,12 @@ func PlanUpgrade(dir string) (UpgradePlan, error) {
names[name] = true
}
}
// v0.14 already owns config/svc extensions. Only bump the module here;
// Generate updates generator-owned main/runtime files without replaying the
// v0.13 seed migration over developer configuration and dependency code.
if semver.IsValid(m.TemplateVersion) && semver.Compare(m.TemplateVersion, "v0.14.0-preview") >= 0 {
names = map[string]bool{}
}
sorted := make([]string, 0, len(names))
for name := range names {
sorted = append(sorted, name)

View File

@@ -39,7 +39,7 @@ func oldInfrastructureProject(t *testing.T) string {
raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
path := filepath.Join(dir, "go.mod")
writeFile(t, path, strings.ReplaceAll(readFile(t, path), minimumVersion, "v0.13.0-preview"))
writeFile(t, path, strings.ReplaceAll(readFile(t, path), currentVersion, "v0.13.0-preview"))
return dir
}
@@ -60,6 +60,49 @@ func TestUpgradeRejectsEscapingEntry(t *testing.T) {
t.Fatal("invalid upgrade wrote project")
}
}
func TestUpgradeV014PreservesExtensions(t *testing.T) {
dir := filepath.Join(t.TempDir(), "service")
if _, err := New(Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
var m manifest
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
t.Fatal(err)
}
m.TemplateVersion = "v0.14.0-preview"
raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
mod := filepath.Join(dir, "go.mod")
writeFile(t, mod, strings.ReplaceAll(readFile(t, mod), currentVersion, "v0.14.1-preview"))
owned := map[string]string{}
for _, name := range []string{"internal/svc/serviceContext.go", "internal/config/config.go", "etc/config.yaml"} {
path := filepath.Join(dir, name)
value := readFile(t, path)
if strings.HasSuffix(name, ".go") {
value += "\n// project extension kept\n"
} else {
value += "\n# project configuration kept\n"
}
writeFile(t, path, value)
owned[path] = value
}
plan, err := PlanUpgrade(dir)
if err != nil || len(plan.Changes) != 1 || plan.Changes[0].Path != "go.mod" {
t.Fatal(plan, err)
}
if _, err := ApplyUpgrade(dir); err != nil {
t.Fatal(err)
}
for path, want := range owned {
if readFile(t, path) != want {
t.Fatal("developer file modified", path)
}
}
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
t.Fatal(err)
}
}
func TestSafeInfrastructureUpgrade(t *testing.T) {
dir := oldInfrastructureProject(t)
before := snapshot(t, dir)

View File

@@ -16,7 +16,8 @@ type WorkspaceService struct {
Module string `json:"module,omitempty"`
}
type WorkspaceConfig struct {
Services []WorkspaceService `json:"services"`
Services []WorkspaceService `json:"services"`
ErrorCatalogs []WorkspaceCatalog `json:"errorCatalogs,omitempty"`
}
// Workspace operates independent Go modules listed in an explicit local file.
@@ -83,6 +84,16 @@ func Workspace(path, command string, w io.Writer) error {
}
options = append(options, Options{API: entry, Dir: dir, Module: service.Module, DryRun: true})
}
var catalogs map[string][]byte
if command != "format" {
catalogs, err = prepareWorkspaceCatalogs(root, cfg.ErrorCatalogs, command == "check")
if err != nil {
return err
}
for i := range options {
options[i].documentCatalogs = catalogs
}
}
var failures error
for _, o := range options {
if command == "format" {
@@ -101,6 +112,13 @@ func Workspace(path, command string, w io.Writer) error {
if failures != nil {
return failures
}
if command == "generate" {
for path, data := range catalogs {
if err = writeWorkspaceCatalog(path, data); err != nil {
return err
}
}
}
for _, o := range options {
switch command {
case "format":

View File

@@ -0,0 +1,217 @@
package apigen
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
gotoken "go/token"
"io"
"os"
"os/exec"
"path/filepath"
"strings"
"time"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"golang.org/x/mod/modfile"
"golang.org/x/mod/module"
)
// WorkspaceCatalog is an explicitly trusted Go provider: func() errors.Catalog.
// Only workspace generate/check execute providers. Parsing an .api never does.
type WorkspaceCatalog struct {
Dir string `json:"dir"`
Package string `json:"package"`
Function string `json:"function"`
Output string `json:"output"`
}
type limitedCatalogBuffer struct {
bytes.Buffer
limit int
}
func (b *limitedCatalogBuffer) Write(p []byte) (int, error) {
if len(p) > b.limit-b.Len() {
return 0, errors.New("catalog provider output exceeds limit")
}
return b.Buffer.Write(p)
}
func exportWorkspaceCatalog(root string, cfg WorkspaceCatalog) ([]byte, error) {
if cfg.Dir == "" || cfg.Output == "" || !gotoken.IsIdentifier(cfg.Function) || !gotoken.IsExported(cfg.Function) || module.CheckImportPath(cfg.Package) != nil {
return nil, errors.New("workspace catalog requires dir, an importable package, an exported function and output")
}
if err := safePath(root, filepath.ToSlash(filepath.Join(cfg.Dir, ".goOmniExportCheck"))); err != nil {
return nil, err
}
if err := safePath(root, cfg.Output); err != nil {
return nil, err
}
if filepath.Ext(cfg.Output) != ".json" {
return nil, errors.New("workspace catalog output must be .json")
}
dir := filepath.Join(root, filepath.FromSlash(cfg.Dir))
if stat, err := os.Stat(filepath.Join(dir, "go.mod")); err != nil || !stat.Mode().IsRegular() {
return nil, errors.New("workspace catalog dir must contain go.mod")
}
temp, err := os.MkdirTemp("", "go-omni-catalog-*")
if err != nil {
return nil, err
}
defer os.RemoveAll(temp)
// Overlay a virtual Go file under the provider module. This allows internal
// packages without creating files or changing go.mod/go.sum in that module.
virtual := filepath.Join(dir, ".goOmniCatalogExport", "main.go")
if _, err := os.Lstat(filepath.Dir(virtual)); !errors.Is(err, os.ErrNotExist) {
return nil, errors.New("reserved catalog export path already exists")
}
source := fmt.Sprintf("package main\nimport (\"os\"; provider %q; omnierrors %q)\nfunc main(){if err:=omnierrors.ExportCatalog(os.Stdout,provider.%s());err!=nil{os.Exit(1)}}\n", cfg.Package, frameworkModule+"/errors", cfg.Function)
backing := filepath.Join(temp, "main.go")
if err = os.WriteFile(backing, []byte(source), 0600); err != nil {
return nil, err
}
overlay, _ := json.Marshal(map[string]any{"Replace": map[string]string{virtual: backing}})
overlayPath := filepath.Join(temp, "overlay.json")
if err = os.WriteFile(overlayPath, overlay, 0600); err != nil {
return nil, err
}
// Keep all module metadata writes in the temporary directory as well.
// -mod=readonly alone must not be relied on to protect every go.sum update.
moduleData, err := os.ReadFile(filepath.Join(dir, "go.mod"))
if err != nil {
return nil, err
}
mf, err := modfile.Parse("go.mod", moduleData, nil)
if err != nil {
return nil, errors.New("workspace catalog has invalid go.mod")
}
replacements := append([]*modfile.Replace(nil), mf.Replace...)
for _, r := range replacements {
if r.New.Version == "" && !filepath.IsAbs(r.New.Path) {
if err = mf.AddReplace(r.Old.Path, r.Old.Version, filepath.Join(dir, r.New.Path), ""); err != nil {
return nil, err
}
}
}
moduleData, err = mf.Format()
if err != nil {
return nil, err
}
modPath := filepath.Join(temp, "go.mod")
if err = os.WriteFile(modPath, moduleData, 0600); err != nil {
return nil, err
}
if sums, e := os.ReadFile(filepath.Join(dir, "go.sum")); e == nil {
if err = os.WriteFile(filepath.Join(temp, "go.sum"), sums, 0600); err != nil {
return nil, err
}
} else if !errors.Is(e, os.ErrNotExist) {
return nil, e
}
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
executable := filepath.Join(temp, "catalogExport")
// Build first, then execute the binary directly: cancellation kills the
// provider itself instead of only terminating a go-run wrapper process.
build := exec.CommandContext(ctx, "go", "build", "-mod=readonly", "-modfile", modPath, "-overlay", overlayPath, "-o", executable, virtual)
build.Dir = dir
build.Env = append(os.Environ(), "GOWORK=off")
build.WaitDelay = 2 * time.Second
build.Stdout = &limitedCatalogBuffer{limit: 64 << 10}
build.Stderr = &limitedCatalogBuffer{limit: 64 << 10}
if err = build.Run(); err != nil {
return nil, fmt.Errorf("workspace catalog %s.%s: build failed (%v); check module dependencies and func() errors.Catalog signature", cfg.Package, cfg.Function, err)
}
cmd := exec.CommandContext(ctx, executable)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
cmd.WaitDelay = 2 * time.Second
stdout := &limitedCatalogBuffer{limit: 1 << 20}
stderr := &limitedCatalogBuffer{limit: 64 << 10}
cmd.Stdout = stdout
cmd.Stderr = stderr
if err = cmd.Run(); err != nil {
if ctx.Err() != nil {
return nil, fmt.Errorf("workspace catalog %s: %w", cfg.Package, ctx.Err())
}
// Provider output can contain arbitrary application data. Do not echo it.
return nil, fmt.Errorf("workspace catalog %s.%s: execution failed (%v); provider must return errors.Catalog without writing stdout", cfg.Package, cfg.Function, err)
}
var catalog apierrors.Catalog
if err = apierrors.DecodeCatalogJSON(stdout.Bytes(), &catalog); err != nil {
return nil, errors.New("workspace catalog provider did not emit a valid catalog")
}
var normalized bytes.Buffer
if err = apierrors.ExportCatalog(&normalized, catalog); err != nil {
return nil, err
}
return normalized.Bytes(), nil
}
func prepareWorkspaceCatalogs(root string, configs []WorkspaceCatalog, check bool) (map[string][]byte, error) {
if len(configs) > 16 {
return nil, errors.New("workspace supports at most 16 catalog providers")
}
result := map[string][]byte{}
for _, cfg := range configs {
if err := safePath(root, cfg.Output); err != nil {
return nil, err
}
path := filepath.Join(root, filepath.FromSlash(cfg.Output))
for prior := range result {
if strings.EqualFold(prior, path) {
return nil, errors.New("workspace catalog outputs overlap")
}
}
data, err := exportWorkspaceCatalog(root, cfg)
if err != nil {
return nil, err
}
if check {
old, err := os.ReadFile(path)
if err != nil || !bytes.Equal(old, data) {
return nil, fmt.Errorf("workspace catalog drift: %s; run workspace generate", cfg.Output)
}
}
result[path] = data
}
return result, nil
}
func writeWorkspaceCatalog(path string, data []byte) error {
mode := os.FileMode(0644)
if stat, err := os.Lstat(path); err == nil {
if !stat.Mode().IsRegular() {
return errors.New("workspace catalog output must be a regular file")
}
mode = stat.Mode().Perm()
} else if !errors.Is(err, os.ErrNotExist) {
return err
}
old, err := os.ReadFile(path)
if err == nil && bytes.Equal(old, data) {
return nil
}
if err != nil && !errors.Is(err, os.ErrNotExist) {
return err
}
f, err := os.CreateTemp(filepath.Dir(path), ".omni-catalog-*")
if err != nil {
return err
}
defer os.Remove(f.Name())
if err = f.Chmod(mode); err != nil {
_ = f.Close()
return err
}
_, writeErr := io.Copy(f, bytes.NewReader(data))
err = errors.Join(writeErr, f.Close())
if err != nil {
return err
}
return os.Rename(f.Name(), path)
}

View File

@@ -0,0 +1,168 @@
package apigen
import (
"bytes"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"strings"
"testing"
)
func TestWorkspaceSharedGoCatalog(t *testing.T) {
root := t.TempDir()
writeFile(t, filepath.Join(root, "go.mod"), fmt.Sprintf("module example.com/catalogs\n\ngo 1.24.0\nrequire %s v0.15.0-preview\nreplace %s => %s\n", frameworkModule, frameworkModule, strconvQuote(frameworkRoot(t))))
writeFile(t, filepath.Join(root, "internal/apierrors/catalog.go"), `package apierrors
import errors "git.yuntuo.hk/k2safe/go-omni/errors"
func Catalog() errors.Catalog {return errors.Catalog{DefaultStatus:400,Definitions:map[string]errors.Definition{
"INVALID":{Code:"INVALID",Message:"invalid input"},"MISSING":{Status:404,Code:"MISSING",Message:"missing"},
}}}
`)
if err := os.MkdirAll(filepath.Join(root, "contracts"), 0755); err != nil {
t.Fatal(err)
}
for _, name := range []string{"api", "id"} {
dir := filepath.Join(root, "services", name)
if _, err := New(Options{Dir: dir, Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), `{"roots":{"common":{"path":"../../../contracts"}},"errors":{"source":"@common/errors.json","include":["INVALID"],"routes":{"GET /api/greet/:name":["MISSING"]}}}`)
}
workspace := filepath.Join(root, "omni.workspace.json")
writeFile(t, workspace, `{"errorCatalogs":[{"dir":".","package":"example.com/catalogs/internal/apierrors","function":"Catalog","output":"contracts/errors.json"}],"services":[{"dir":"services/api"},{"dir":"services/id"}]}`)
if err := Workspace(workspace, "generate", io.Discard); err != nil {
t.Fatal(err)
}
if err := Workspace(workspace, "format", io.Discard); err != nil {
t.Fatal(err)
}
if err := Workspace(workspace, "generate", io.Discard); err != nil {
t.Fatal(err)
}
before := snapshot(t, root)
if err := Workspace(workspace, "check", io.Discard); err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("check wrote project files")
}
for _, name := range []string{"api", "id"} {
entry := filepath.Join(root, "services", name, "api/service.api")
spec, err := Parse(entry)
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
op := operation(t, document(t, c), "/api/greet/{name}", "get")
if op["responses"].(map[string]any)["404"] == nil || len(spec.Errors) != 0 || len(spec.Routes[0].Errors) != 0 {
t.Fatal("selection changed runtime catalog")
}
if _, err = os.Stat(filepath.Join(filepath.Dir(entry), catalogDocumentFile)); !os.IsNotExist(err) {
t.Fatal("shared catalog was copied")
}
}
shared := filepath.Join(root, "contracts/errors.json")
data := readFile(t, shared)
writeFile(t, shared, strings.Replace(data, "invalid input", "stale", 1))
before = snapshot(t, root)
if err := Workspace(workspace, "check", io.Discard); err == nil || !strings.Contains(err.Error(), "catalog drift") {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("drift check wrote files")
}
if err := Workspace(workspace, "generate", io.Discard); err != nil {
t.Fatal(err)
}
if got := readFile(t, shared); got != data {
t.Fatal("catalog not refreshed")
}
// Failed provider/preflight may not update even the shared metadata.
source := filepath.Join(root, "internal/apierrors/catalog.go")
writeFile(t, source, readFile(t, source)+"\ninvalid go source")
before = snapshot(t, root)
if err := Workspace(workspace, "generate", io.Discard); err == nil {
t.Fatal("provider failure ignored")
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("provider failure mutated project")
}
}
func TestDocumentSelectionBoundaries(t *testing.T) {
root := t.TempDir()
local := filepath.Join(root, "api")
shared := filepath.Join(root, "shared")
entry := filepath.Join(local, "service.api")
writeFile(t, entry, "service demo {\n@handler list\nget /list returns ([]string)\n}\n")
writeFile(t, filepath.Join(shared, "errors.json"), `{"defaultStatus":400,"definitions":{"INVALID":{"code":400002,"message":"invalid"}}}`)
for _, cfg := range []string{
`{"source":"@common/errors.json","include":["UNKNOWN"]}`,
`{"source":"@common/errors.json","routes":{"GET /absent":["INVALID"]}}`,
`{"source":"@common/../api/service.api"}`,
`{"source":"../shared/errors.json"}`,
} {
writeFile(t, filepath.Join(local, importsFile), `{"roots":{"common":{"path":"../shared"}},"errors":`+cfg+`}`)
if _, err := Parse(entry); err == nil {
t.Fatal("bad selection accepted", cfg)
}
}
writeFile(t, filepath.Join(local, importsFile), `{"roots":{"common":{"path":"../shared"}},"errors":{"source":"@common/errors.json","include":[]}}`)
spec, err := Parse(entry)
if err != nil {
t.Fatal(err)
}
if len(documentRouteCatalog(spec, spec.Routes[0]).Definitions) != 0 {
t.Fatal("empty include ignored")
}
seeded, err := sharedSeeds(spec, filepath.Join(root, "generated"))
if err != nil {
t.Fatal(err)
}
var cfg sharedConfig
if err = json.Unmarshal(seeded, &cfg); err != nil {
t.Fatal(err)
}
if cfg.Errors.Include == nil || !bytes.Contains(seeded, []byte(`"include": []`)) {
t.Fatal("empty selection lost during relocation")
}
}
func TestWorkspaceCatalogPathsAndProviderValidation(t *testing.T) {
root := t.TempDir()
writeFile(t, filepath.Join(root, "go.mod"), "module example.com/test\ngo 1.24.0\n")
good := WorkspaceCatalog{Dir: ".", Package: "example.com/test/internal/errors", Function: "Catalog", Output: "errors.json"}
cases := []WorkspaceCatalog{}
for _, change := range []func(*WorkspaceCatalog){
func(c *WorkspaceCatalog) { c.Dir = "../outside" },
func(c *WorkspaceCatalog) { c.Output = "../outside.json" },
func(c *WorkspaceCatalog) { c.Package = "example.com/pkg;bad" },
func(c *WorkspaceCatalog) { c.Function = "Catalog()" },
func(c *WorkspaceCatalog) { c.Output = "go.mod" },
} {
c := good
change(&c)
cases = append(cases, c)
}
for _, c := range cases {
if _, err := exportWorkspaceCatalog(root, c); err == nil {
t.Fatal("bad provider accepted", c)
}
}
outside := filepath.Join(t.TempDir(), "errors.json")
writeFile(t, outside, `{}`)
if err := os.Symlink(outside, filepath.Join(root, "errors.json")); err != nil {
t.Fatal(err)
}
if _, err := prepareWorkspaceCatalogs(root, []WorkspaceCatalog{good}, false); err == nil {
t.Fatal("symlink output accepted")
}
if readFile(t, outside) != `{}` {
t.Fatal("outside file modified")
}
}

View File

@@ -16,6 +16,7 @@ import (
const manifestPath = ".goOmni/apiManifest.json"
type manifest struct {
GeneratorConfig bool `json:"generatorConfig,omitempty"`
TemplateVersion string `json:"templateVersion,omitempty"`
Profile *projectProfile `json:"profile,omitempty"`
Version int `json:"version"`
@@ -109,13 +110,24 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
return result, err
}
}
next := manifest{TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
next := manifest{GeneratorConfig: true, TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract}
for name, data := range generated {
next.Generated[name] = digest(data)
}
var changes []change
// Check ALL previous outputs, even stale ones, before writing any new file.
for name, hash := range old.Generated {
// Migration guard only; old mappings are never regenerated or silently
// deleted. Once the project removes them, forget the stale ownership.
if name == "internal/mapping/mappingsGen.go" {
if err = safePath(root, name); err != nil {
return result, err
}
if _, e := os.Lstat(filepath.Join(root, name)); !errors.Is(e, os.ErrNotExist) {
return result, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go and remove internal/mapping/mappingsGen.go before generating; see COPY.md")
}
continue
}
if !ownedPath(name) {
return result, fmt.Errorf("api: invalid owned path %q", name)
}
@@ -272,6 +284,9 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
}
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
func ownedPath(name string) bool {
if name == "internal/config/loadGen.go" {
return true
}
if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) {
return true
}

View File

@@ -8,7 +8,7 @@ import (
)
func TestNewWithRemoteFrameworkVersion(t *testing.T) {
for _, flag := range [][]string{{"--framework-version", "v0.14.0-preview"}, {"--framework-version=v0.14.0-preview"}} {
for _, flag := range [][]string{{"--framework-version", "v0.21.0-preview"}, {"--framework-version=v0.21.0-preview"}} {
project := filepath.Join(t.TempDir(), "billing")
args := append([]string{"--module", "example.com/billing"}, flag...)
if err := runNew(append(args, project)); err != nil {
@@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(content), "go-omni v0.14.0-preview") || strings.Contains(string(content), "replace ") {
if !strings.Contains(string(content), "go-omni v0.21.0-preview") || strings.Contains(string(content), "replace ") {
t.Fatalf("CLI did not pin remote dependency: %s", content)
}
}

View File

@@ -21,6 +21,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
"internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl",
"internal/svc/options.go": "templates/api-options.go.tmpl",
"internal/config/config.go": "templates/api-config.go.tmpl",
"internal/config/loadGen.go": "templates/api-load.go.tmpl",
"internal/config/loader.go": "templates/api-loader.go.tmpl",
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
"internal/svc/serviceContext.go": "templates/api-context.go.tmpl",
}
@@ -31,6 +33,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl"
}
if legacy {
delete(files, "internal/config/loadGen.go")
delete(files, "internal/config/loader.go")
delete(files, "internal/svc/runtimeGen.go")
delete(files, "internal/svc/databaseOptions.go")
files["internal/svc/close.go"] = "templates/api-close.go.tmpl"

View File

@@ -1,10 +1,9 @@
package settings
import (
"os"
"strings"
{{if and (not .API) (ne .Storage "none")}}"strings"{{end}}
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
{{if eq .LogOutput "file"}}"git.yuntuo.hk/k2safe/go-omni/logx"{{end}}
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
{{if ne .Storage "none"}}"errors"
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
@@ -35,22 +34,10 @@ func Default() Config {
}{{end}}}
}
// Load applies defaults -> file -> typed environment -> validation.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string)(Config,error){
cfg:=Default();var err error
cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err}
if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}}
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_"});if err!=nil{return Config{},err}
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
cfg.Server,err=config.ApplyEnv(cfg.Server,"");if err!=nil{return Config{},err}
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
return cfg,cfg.Validate()
}
func(c Config)Validate()error{
if err:=c.Server.Validate();err!=nil{return err}
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return err}{{end}}
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return err}{{end}}
if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)}
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return config.AtField("database",err)}{{end}}
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return config.AtField("storage",err)}{{end}}
return nil
}
{{if ne .Storage "none"}}

View File

@@ -28,6 +28,8 @@ type Context struct {
{{end}}
}
// Init is project-owned. Add dependencies here and immediately register each
// cleanup with c.OnClose. Returning an error automatically rolls everything back.
// cleanup with c.OnClose. Background components use paired OnStop/OnDrain hooks
// and WorkContext. Failed initialization stops/drains before releasing resources;
// unjoined background work returns ErrResourcesRetained instead of closing its dependencies.
// Generated infrastructure is ready here. Never create another App or logger.
func(c *Context)Init(ctx context.Context)error {return nil}

View File

@@ -0,0 +1,13 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// Load is maintained by the generator. Extend Config/Default/Validate in
// config.go, or NewLoader in loader.go; do not duplicate the loading pipeline.
func Load(path string) (Config, error) {
loader,err:=NewLoader()
if err!=nil{return Config{},err}
return omniConfig.LoadProject(path,Default(),omniConfig.EnvOptions{
Prefix:"GO_OMNI_",Empty:"ignore",Aliases:omniConfig.ServerEnvAliases("GO_OMNI_","server"),
},loader)
}

View File

@@ -0,0 +1,7 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// NewLoader is project-owned. JSON/YAML/XML work without changes. Register
// additional format decoders here only when the project needs them.
func NewLoader() (*omniConfig.Loader,error) { return omniConfig.NewLoader() }

View File

@@ -18,7 +18,8 @@ func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(res
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
if err=cfg.Validate();err!=nil{return nil,err}
checker,err:=health.New();if err!=nil{return nil,err}
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Runtime:lifecycle.NewRuntime(ctx)}
runtime,err:=lifecycle.NewRuntimeWithConfig(logx.WithLogger(ctx,logger),cfg.Server.Lifecycle);if err!=nil{return nil,err}
c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Runtime:runtime}
ctx=logx.WithLogger(c.Context(),logger)
defer func(){if p:=recover();p!=nil{_ = c.Close();panic(p)};if err!=nil{err=c.Finish(err)}}()
{{if eq .Database "mysql"}}

View File

@@ -4,7 +4,11 @@ server:
enableLogger: true
slowRequestThreshold: 500ms
shutdownTimeout: 10s
{{if .API}} endpoints:
{{if .API}} lifecycle:
stopTimeout: 5s
drainTimeout: 10s
cancelTimeout: 5s
endpoints:
liveness:
enabled: true
path: /health/live

View File

@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Source builds can carry an unpublished VCS version (including +dirty).
// Keep this fallback in sync with the next release; installed CLIs use their
// downloaded module version, which also has a module checksum.
const defaultFrameworkVersion = "v0.14.1-preview"
const defaultFrameworkVersion = "v0.21.0-preview"
// This target is not a claim that the release already exists. Source users
// select a local framework checkout until the module templates are published.

66
config/aliases_test.go Normal file
View File

@@ -0,0 +1,66 @@
package config
import (
"strings"
"testing"
"time"
)
func TestUnifiedEnvAliases(t *testing.T) {
type project struct {
Server Config `json:"server"`
Label string `json:"label"`
}
base := project{Server: Default(), Label: "base"}
for _, policy := range []string{"allow", "ignore", "reject"} {
t.Run(policy, func(t *testing.T) {
env := map[string]string{"TEST_HTTP_ADDR": "127.0.0.1:8099", "TEST_LOG_SERVICE": "", "TEST_SERVICE": "", "TEST_LABEL": ""}
opts := EnvOptions{Prefix: "TEST_", Empty: policy, Aliases: ServerEnvAliases("TEST_", "server"), Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
got, err := OverlayEnv(base, opts)
if policy == "reject" {
if err == nil || !strings.Contains(err.Error(), "empty") {
t.Fatal(err)
}
return
}
if err != nil {
t.Fatal(err)
}
if got.Server.Address != "127.0.0.1:8099" {
t.Fatal("alias ignored")
}
if policy == "allow" && got.Label != "" || policy == "ignore" && got.Label != "base" {
t.Fatal("empty policy differs")
}
env["TEST_SERVER_ADDRESS"] = "127.0.0.1:9001"
env["TEST_HTTP_READ_TIMEOUT"] = "3s"
env["TEST_ENABLE_LOGGER"] = "false"
env["TEST_TRUSTED_PROXIES"] = `["127.0.0.1"]`
got, err = OverlayEnv(base, opts)
if err != nil || got.Server.Address != "127.0.0.1:9001" || got.Server.HTTP.ReadTimeout != 3*time.Second || got.Server.EnableLogger || len(got.Server.TrustedProxies) != 1 {
t.Fatal(got, err)
}
env["TEST_SERVER_ADDRESS"] = ""
got, err = OverlayEnv(base, opts)
if err != nil {
t.Fatal(err)
}
expected := ""
if policy == "ignore" {
expected = base.Server.Address
}
if got.Server.Address != expected {
t.Fatal("empty canonical unexpectedly fell through to alias")
}
})
}
if base.Server.Address != ":8080" || base.Label != "base" {
t.Fatal("defaults mutated")
}
}
func TestAliasTypeErrorsDoNotExposeValue(t *testing.T) {
opts := EnvOptions{Prefix: "X_", Empty: "reject", Aliases: ServerEnvAliases("X_", ""), Lookup: func(k string) (string, bool) { return "private-invalid", k == "X_HTTP_READ_TIMEOUT" }}
if _, err := OverlayEnv(Default(), opts); err == nil || strings.Contains(err.Error(), "private-invalid") {
t.Fatal(err)
}
}

320
config/bind.go Normal file
View File

@@ -0,0 +1,320 @@
package config
import (
"encoding/json"
"fmt"
"reflect"
"sort"
"strings"
)
var jsonUnmarshalerType = reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
type documentBinder struct {
doc Document
nodes int
}
func (b *documentBinder) step(path string, depth int) error {
b.nodes++
if depth > maxConfigDepth || b.nodes > maxConfigNodes {
return b.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
}
return nil
}
func bindDocument[T any](doc Document, defaults T) (T, error) {
var zero T
base := reflect.ValueOf(&defaults).Elem()
if base.Kind() != reflect.Struct && (base.Kind() != reflect.Pointer || base.Type().Elem().Kind() != reflect.Struct) {
return zero, &Error{Reason: "project configuration must be a struct"}
}
result, err := cloneConfiguration(base, 0)
if err != nil {
return zero, &Error{Reason: "cannot clone configuration defaults", cause: err}
}
b := documentBinder{doc: doc}
if err = b.bind(doc.Value, result, "", 0, false); err != nil {
return zero, err
}
return result.Interface().(T), nil
}
func (b *documentBinder) decode(v any, dst reflect.Value, path string) (err error) {
defer func() {
if recover() != nil {
err = b.doc.diagnostic(path, "custom value decoder panicked", nil)
}
}()
raw, err := json.Marshal(v)
if err != nil {
return b.doc.diagnostic(path, "unsupported configuration value", err)
}
if err = json.Unmarshal(raw, dst.Addr().Interface()); err != nil {
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
}
return nil
}
func (b *documentBinder) bind(v any, dst reflect.Value, path string, depth int, quoted bool) error {
if err := b.step(path, depth); err != nil {
return err
}
if v == nil {
return b.decode(nil, dst, path)
} // Preserve encoding/json null semantics.
if dst.Kind() == reflect.Pointer {
if dst.IsNil() {
dst.Set(reflect.New(dst.Type().Elem()))
}
return b.bind(v, dst.Elem(), path, depth+1, quoted)
}
if dst.Type() == durationType {
var raw string
switch x := v.(type) {
case string:
raw = x
case Text:
raw = string(x)
default:
return b.decode(v, dst, path)
}
if err := parseEnvValue(dst, strings.TrimSpace(raw)); err != nil {
return b.doc.diagnostic(path, "expected a duration such as 5s", err)
}
return nil
}
if dst.CanAddr() && dst.Addr().Type().Implements(jsonUnmarshalerType) {
normalized, err := b.dynamic(v, path, depth+1)
if err != nil {
return err
}
return b.decode(normalized, dst, path)
}
if dst.CanAddr() && dst.Addr().Type().Implements(textType) {
if text, ok := v.(Text); ok {
v = string(text)
}
return b.decode(v, dst, path)
}
if text, ok := v.(Text); ok {
if dst.Kind() == reflect.String || dst.Kind() == reflect.Interface || (dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8) {
return b.decode(string(text), dst, path)
}
switch dst.Kind() {
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64:
if err := parseEnvValue(dst, strings.TrimSpace(string(text))); err != nil {
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
}
return nil
default:
return b.doc.diagnostic(path, "expected an object or array, not scalar text", nil)
}
}
if quoted {
s, ok := v.(string)
if !ok {
return b.doc.diagnostic(path, "expected a JSON-encoded string", nil)
}
if err := json.Unmarshal([]byte(s), dst.Addr().Interface()); err != nil {
return b.doc.diagnostic(path, "invalid string-encoded value", err)
}
return nil
}
switch dst.Kind() {
case reflect.Struct:
m, ok := v.(map[string]any)
if !ok {
return b.doc.diagnostic(path, "expected an object", nil)
}
fields := configurationFields(dst.Type())
used := map[string]bool{}
for _, name := range sortedKeys(m) {
child := joinField(path, name)
f, ok := fields[name]
canonical := name
if !ok {
for n, field := range fields {
if strings.EqualFold(n, name) {
if ok {
return b.doc.diagnostic(child, "ambiguous field name", nil)
}
f, ok, canonical = field, true, n
}
}
}
if !ok {
return b.doc.diagnostic(child, "unknown field", nil)
}
if used[canonical] {
return b.doc.diagnostic(child, "duplicate field binding", nil)
}
used[canonical] = true
target := dst
for _, index := range f.Index {
if target.Kind() == reflect.Pointer {
if target.IsNil() {
if !target.CanSet() {
return b.doc.diagnostic(child, "field cannot be assigned", nil)
}
target.Set(reflect.New(target.Type().Elem()))
}
target = target.Elem()
}
target = target.Field(index)
}
if !target.CanSet() {
return b.doc.diagnostic(child, "field cannot be assigned", nil)
}
if err := b.bind(m[name], target, child, depth+1, strings.Contains(","+f.Tag.Get("json")+",", ",string,")); err != nil {
return err
}
}
return nil
case reflect.Map:
m, ok := v.(map[string]any)
if !ok || dst.Type().Key().Kind() != reflect.String {
return b.doc.diagnostic(path, "expected an object with string keys", nil)
}
if dst.IsNil() {
dst.Set(reflect.MakeMap(dst.Type()))
}
for _, key := range sortedKeys(m) {
value := reflect.New(dst.Type().Elem()).Elem()
if err := b.bind(m[key], value, joinField(path, key), depth+1, false); err != nil {
return err
}
k := reflect.New(dst.Type().Key()).Elem()
k.SetString(key)
dst.SetMapIndex(k, value)
}
return nil
case reflect.Slice, reflect.Array:
if _, ok := v.(string); ok && dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8 {
return b.decode(v, dst, path)
}
items, ok := v.([]any)
if !ok {
return b.doc.diagnostic(path, "expected an array", nil)
}
if dst.Kind() == reflect.Array && len(items) != dst.Len() {
return b.doc.diagnostic(path, "array length does not match its declared type", nil)
}
if dst.Kind() == reflect.Slice {
dst.Set(reflect.MakeSlice(dst.Type(), len(items), len(items)))
}
for i, value := range items {
if err := b.bind(value, dst.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1, false); err != nil {
return err
}
}
return nil
case reflect.Interface:
v, err := b.dynamic(v, path, depth+1)
if err != nil {
return err
}
if dst.Type().NumMethod() == 0 {
dst.Set(reflect.ValueOf(v))
return nil
}
return b.decode(v, dst, path)
default:
return b.decode(v, dst, path)
}
}
func sortedKeys(m map[string]any) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
}
func (b *documentBinder) dynamic(v any, path string, depth int) (any, error) {
if err := b.step(path, depth); err != nil {
return nil, err
}
switch x := v.(type) {
case Text:
return string(x), nil
case map[string]any:
out := make(map[string]any, len(x))
for _, k := range sortedKeys(x) {
c, err := b.dynamic(x[k], joinField(path, k), depth+1)
if err != nil {
return nil, err
}
out[k] = c
}
return out, nil
case []any:
out := make([]any, len(x))
for i, v := range x {
c, err := b.dynamic(v, fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return nil, err
}
out[i] = c
}
return out, nil
default:
return v, nil
}
}
// Match encoding/json's field promotion: the shallowest declaration wins;
// explicit tags win ties, and ambiguous fields remain unavailable.
func configurationFields(t reflect.Type) map[string]reflect.StructField {
type candidate struct {
field reflect.StructField
tagged bool
}
candidates := map[string][]candidate{}
var visit func(reflect.Type, []int, map[reflect.Type]bool)
visit = func(current reflect.Type, prefix []int, visiting map[reflect.Type]bool) {
if visiting[current] {
return
}
visiting[current] = true
defer delete(visiting, current)
for i := 0; i < current.NumField(); i++ {
f := current.Field(i)
name := strings.Split(f.Tag.Get("json"), ",")[0]
if name == "-" {
continue
}
child := f.Type
if child.Kind() == reflect.Pointer {
child = child.Elem()
}
if f.PkgPath != "" && (!f.Anonymous || child.Kind() != reflect.Struct) {
continue
}
f.Index = append(append([]int(nil), prefix...), i)
if name == "" && f.Anonymous && child.Kind() == reflect.Struct {
visit(child, f.Index, visiting)
continue
}
tagged := name != ""
if name == "" {
name = f.Name
}
candidates[name] = append(candidates[name], candidate{f, tagged})
}
}
visit(t, nil, map[reflect.Type]bool{})
fields := map[string]reflect.StructField{}
for name, list := range candidates {
sort.Slice(list, func(i, j int) bool {
if len(list[i].field.Index) != len(list[j].field.Index) {
return len(list[i].field.Index) < len(list[j].field.Index)
}
return list[i].tagged && !list[j].tagged
})
if len(list) > 1 && len(list[0].field.Index) == len(list[1].field.Index) && list[0].tagged == list[1].tagged {
continue
}
fields[name] = list[0].field
}
return fields
}

View File

@@ -7,6 +7,7 @@ import (
"git.yuntuo.hk/k2safe/go-omni/contract"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
)
@@ -27,22 +28,23 @@ type TLSConfig struct {
// Config controls the default runtime behavior of an App.
type Config struct {
Request contract.RequestPolicy `json:"request"`
Address string `json:"address"`
Transport string `json:"transport"`
TLS TLSConfig `json:"tls"`
HTTP HTTPConfig `json:"http"`
Endpoints EndpointsConfig `json:"endpoints"`
RequestIDHeader string `json:"requestIdHeader"`
EnableLogger bool `json:"enableLogger"`
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
TrustedProxies []string `json:"trustedProxies"`
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
Log logx.Config `json:"log"`
Errors apierrors.Catalog `json:"errors"`
Response response.Config `json:"response"`
ResponseEncoder response.Encoder `json:"-"`
LogInstance *logx.Logger `json:"-"`
Request contract.RequestPolicy `json:"request"`
Address string `json:"address"`
Transport string `json:"transport"`
TLS TLSConfig `json:"tls"`
HTTP HTTPConfig `json:"http"`
Endpoints EndpointsConfig `json:"endpoints"`
RequestIDHeader string `json:"requestIdHeader"`
EnableLogger bool `json:"enableLogger"`
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
TrustedProxies []string `json:"trustedProxies"`
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
Lifecycle lifecycle.ShutdownConfig `json:"lifecycle"`
Log logx.Config `json:"log"`
Errors apierrors.Catalog `json:"errors"`
Response response.Config `json:"response"`
ResponseEncoder response.Encoder `json:"-"`
LogInstance *logx.Logger `json:"-"`
}
func Default() Config {
@@ -51,6 +53,7 @@ func Default() Config {
Transport: TransportNetHTTP,
RequestIDHeader: defaultRequestIDHeader,
ShutdownTimeout: 10 * time.Second,
Lifecycle: lifecycle.DefaultShutdownConfig(),
Endpoints: DefaultEndpoints(),
Log: logx.Default(),
}
@@ -58,17 +61,20 @@ func Default() Config {
// Validate checks configuration values before an application starts.
func (c Config) Validate() error {
if err := c.Lifecycle.Validate(); err != nil {
return AtField("lifecycle", err)
}
if err := c.Request.Validate(); err != nil {
return err
return AtField("request", err)
}
if err := c.Log.Validate(); err != nil {
return err
return AtField("log", err)
}
if err := c.Endpoints.Validate(); err != nil {
return err
}
if err := c.Errors.Validate(); err != nil {
return err
return AtField("errors", err)
}
encoder := c.ResponseEncoder
if encoder == nil {
@@ -84,7 +90,7 @@ func (c Config) Validate() error {
return err
}
if err := c.Response.Validate(); err != nil {
return err
return AtField("response", err)
}
if c.SlowRequestThreshold < 0 {
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
@@ -93,13 +99,13 @@ func (c Config) Validate() error {
return &ConfigError{Field: "address", Message: "is required"}
}
if strings.TrimSpace(c.RequestIDHeader) == "" {
return &ConfigError{Field: "request_id_header", Message: "is required"}
return &ConfigError{Field: "requestIdHeader", Message: "is required"}
}
if c.ShutdownTimeout <= 0 {
return &ConfigError{Field: "shutdown_timeout", Message: "must be positive"}
return &ConfigError{Field: "shutdownTimeout", Message: "must be positive"}
}
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
return &ConfigError{Field: "trusted_proxies", Message: err.Error()}
return &Error{Field: "trustedProxies", Reason: "expected IP addresses or CIDRs", cause: err}
}
return c.ValidateTransport()
}

View File

@@ -112,7 +112,7 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s")
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com")
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trusted_proxies") {
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trustedProxies") {
t.Fatalf("expected invalid trusted proxy error, got %v", err)
}
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "")
@@ -123,7 +123,7 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
}
cfg = Default()
cfg.RequestIDHeader = " "
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "request_id_header") {
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "requestIdHeader") {
t.Fatalf("expected invalid request header error, got %v", err)
}
}

398
config/decoders.go Normal file
View File

@@ -0,0 +1,398 @@
package config
import (
"bytes"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"regexp"
"sort"
"strconv"
"strings"
"unicode/utf8"
"gopkg.in/yaml.v3"
)
type JSONDecoder struct{}
type YAMLDecoder struct{}
type XMLDecoder struct{}
type parseState struct {
doc Document
nodes int
}
func newParseState() *parseState { return &parseState{doc: Document{Positions: map[string]Position{}}} }
func (p *parseState) enter(path string, pos Position, depth int) error {
p.nodes++
p.doc.Positions[path] = pos
if depth > maxConfigDepth || p.nodes > maxConfigNodes {
return p.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
}
return nil
}
type sourcePositions struct{ starts []int }
func newSourcePositions(data []byte) sourcePositions {
p := sourcePositions{starts: []int{0}}
for i, c := range data {
if c == '\n' {
p.starts = append(p.starts, i+1)
}
}
return p
}
func (p sourcePositions) at(offset int64) Position {
if offset < 0 {
offset = 0
}
i := sort.Search(len(p.starts), func(i int) bool { return int64(p.starts[i]) > offset }) - 1
return Position{Line: i + 1, Column: int(offset) - p.starts[i] + 1}
}
var yamlLine = regexp.MustCompile(`line ([0-9]+)`)
func yamlSyntax(err error) *Error {
d := &Error{Reason: "invalid YAML syntax", cause: err}
if m := yamlLine.FindStringSubmatch(err.Error()); len(m) == 2 {
d.Line, _ = strconv.Atoi(m[1])
d.Column = 1
}
return d
}
func (JSONDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
positions := newSourcePositions(data)
d := json.NewDecoder(bytes.NewReader(data))
d.UseNumber()
var parse func(string, int) (any, error)
parse = func(path string, depth int) (any, error) {
offset := d.InputOffset()
for offset < int64(len(data)) && strings.ContainsRune(" \t\r\n:,", rune(data[offset])) {
offset++
}
if err := p.enter(path, positions.at(offset), depth); err != nil {
return nil, err
}
token, err := d.Token()
if err != nil {
pos := positions.at(d.InputOffset())
var syntax *json.SyntaxError
if errors.As(err, &syntax) {
pos = positions.at(syntax.Offset - 1)
}
return nil, &Error{Field: path, Line: pos.Line, Column: pos.Column, Reason: "invalid JSON syntax", cause: err}
}
if delim, ok := token.(json.Delim); ok {
switch delim {
case '{':
out := map[string]any{}
for d.More() {
k, e := d.Token()
if e != nil {
return nil, p.doc.diagnostic(path, "invalid object key", e)
}
key, ok := k.(string)
if !ok {
return nil, p.doc.diagnostic(path, "object key must be a string", nil)
}
child := joinField(path, key)
if _, exists := out[key]; exists {
return nil, p.doc.diagnostic(child, "duplicate field", nil)
}
out[key], e = parse(child, depth+1)
if e != nil {
return nil, e
}
}
if _, err = d.Token(); err != nil {
return nil, p.doc.diagnostic(path, "unterminated object", err)
}
return out, nil
case '[':
out := []any{}
for d.More() {
v, e := parse(fmt.Sprintf("%s[%d]", path, len(out)), depth+1)
if e != nil {
return nil, e
}
out = append(out, v)
}
if _, err = d.Token(); err != nil {
return nil, p.doc.diagnostic(path, "unterminated array", err)
}
return out, nil
default:
return nil, p.doc.diagnostic(path, "unexpected JSON delimiter", nil)
}
}
return token, nil
}
var err error
p.doc.Value, err = parse("", 0)
if err == nil {
if _, e := d.Token(); e != io.EOF {
err = p.doc.diagnostic("", "exactly one JSON document required", e)
}
}
return p.doc, err
}
func (YAMLDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
d := yaml.NewDecoder(bytes.NewReader(data))
var root yaml.Node
if err := d.Decode(&root); err != nil {
return p.doc, yamlSyntax(err)
}
var extra yaml.Node
if err := d.Decode(&extra); err != io.EOF {
return p.doc, &Error{Reason: "exactly one YAML document required", cause: err}
}
var walk func(*yaml.Node, string, int) (any, error)
walk = func(n *yaml.Node, path string, depth int) (any, error) {
if err := p.enter(path, Position{n.Line, n.Column}, depth); err != nil {
return nil, err
}
switch n.Kind {
case yaml.MappingNode:
out := map[string]any{}
for i := 0; i < len(n.Content); i += 2 {
k, v := n.Content[i], n.Content[i+1]
if k.Tag != "!!str" {
return nil, p.doc.diagnostic(path, "mapping keys must be strings; merge keys are not supported", nil)
}
child := joinField(path, k.Value)
if _, ok := out[k.Value]; ok {
return nil, &Error{Field: child, Line: k.Line, Column: k.Column, Reason: "duplicate field"}
}
x, err := walk(v, child, depth+1)
if err != nil {
return nil, err
}
out[k.Value] = x
}
return out, nil
case yaml.SequenceNode:
out := make([]any, len(n.Content))
for i, c := range n.Content {
var err error
out[i], err = walk(c, fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return nil, err
}
}
return out, nil
case yaml.AliasNode:
return nil, p.doc.diagnostic(path, "YAML aliases are not supported", nil)
case yaml.ScalarNode:
switch n.Tag {
case "!!str", "!!timestamp":
return n.Value, nil
case "!!null":
return nil, nil
case "!!bool", "!!int", "!!float":
var v any
if err := n.Decode(&v); err != nil {
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
}
raw, err := json.Marshal(v)
if err != nil {
return nil, p.doc.diagnostic(path, "number must be finite", err)
}
var out any
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
if err = dec.Decode(&out); err != nil {
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
}
return out, nil
default:
return nil, p.doc.diagnostic(path, "unsupported YAML tag", nil)
}
}
return nil, p.doc.diagnostic(path, "unsupported YAML node", nil)
}
if len(root.Content) != 1 {
return p.doc, &Error{Reason: "one nonempty YAML document required"}
}
var err error
p.doc.Value, err = walk(root.Content[0], "", 0)
return p.doc, err
}
// XML uses a config root, element field names, type="array" with item children,
// type="object" for empty objects, and type="null" for explicit null. Scalar
// text is bound using the destination type; optional number/bool/string types
// let dynamic any values preserve their intended JSON type.
func (XMLDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
d := xml.NewDecoder(bytes.NewReader(data))
var parse func(xml.StartElement, string, int, Position) (any, error)
parse = func(start xml.StartElement, path string, depth int, pos Position) (any, error) {
if err := p.enter(path, pos, depth); err != nil {
return nil, err
}
if start.Name.Space != "" {
return nil, p.doc.diagnostic(path, "XML namespaces are not supported", nil)
}
kind := ""
attrSeen := false
for _, a := range start.Attr {
if a.Name.Space != "" || a.Name.Local != "type" || attrSeen {
return nil, p.doc.diagnostic(path, "only one XML type attribute is supported", nil)
}
kind = a.Value
attrSeen = true
}
if depth == 0 && kind == "" {
kind = "object"
}
switch kind {
case "", "object", "array", "null", "string", "number", "bool":
default:
return nil, p.doc.diagnostic(path, "unsupported XML type", nil)
}
children := map[string]any{}
array := []any{}
var text strings.Builder
for {
line, column := d.InputPos()
token, err := d.Token()
if err != nil {
return nil, &Error{Field: path, Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
}
switch x := token.(type) {
case xml.StartElement:
if kind != "" && kind != "object" && kind != "array" {
return nil, p.doc.diagnostic(path, "scalar XML element cannot contain children", nil)
}
child := joinField(path, x.Name.Local)
if kind == "array" {
if x.Name.Local != "item" {
return nil, p.doc.diagnostic(path, "XML array children must be item elements", nil)
}
child = fmt.Sprintf("%s[%d]", path, len(array))
} else if _, ok := children[x.Name.Local]; ok {
return nil, p.doc.diagnostic(child, "duplicate field; use type=\"array\" for lists", nil)
}
v, e := parse(x, child, depth+1, Position{line, column})
if e != nil {
return nil, e
}
if kind == "array" {
array = append(array, v)
} else {
children[x.Name.Local] = v
}
case xml.CharData:
text.Write([]byte(x))
case xml.Comment:
case xml.EndElement:
raw := text.String()
if len(children) > 0 || kind == "array" || kind == "object" {
if strings.TrimSpace(raw) != "" {
return nil, p.doc.diagnostic(path, "mixed XML text and elements are not supported", nil)
}
if kind == "array" {
return array, nil
}
return children, nil
}
switch kind {
case "null":
if strings.TrimSpace(raw) != "" {
return nil, p.doc.diagnostic(path, "null element must be empty", nil)
}
return nil, nil
case "string":
return raw, nil
case "number":
var n any
dec := json.NewDecoder(strings.NewReader(strings.TrimSpace(raw)))
dec.UseNumber()
if err := dec.Decode(&n); err != nil {
return nil, p.doc.diagnostic(path, "invalid number", err)
}
if _, ok := n.(json.Number); !ok {
return nil, p.doc.diagnostic(path, "expected number", nil)
}
if _, e := dec.Token(); e != io.EOF {
return nil, p.doc.diagnostic(path, "invalid number", e)
}
return n, nil
case "bool":
b, e := strconv.ParseBool(strings.TrimSpace(raw))
if e != nil {
return nil, p.doc.diagnostic(path, "expected boolean", e)
}
return b, nil
}
return Text(raw), nil
default:
return nil, p.doc.diagnostic(path, "XML directives and processing instructions are not supported", nil)
}
}
}
found := false
prolog := false
for {
line, column := d.InputPos()
token, err := d.Token()
if err == io.EOF {
break
}
if err != nil {
return p.doc, &Error{Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
}
switch x := token.(type) {
case xml.StartElement:
if found || x.Name.Local != "config" {
return p.doc, &Error{Line: line, Column: column, Reason: "exactly one config root required"}
}
found = true
p.doc.Value, err = parse(x, "", 0, Position{line, column})
if err != nil {
return p.doc, err
}
case xml.CharData:
if strings.TrimSpace(string(x)) != "" {
return p.doc, &Error{Reason: "unexpected text outside XML root"}
}
case xml.Comment:
case xml.ProcInst:
if found || prolog || x.Target != "xml" {
return p.doc, &Error{Reason: "XML processing instructions are not supported"}
}
prolog = true
default:
return p.doc, &Error{Reason: "XML directives are not supported"}
}
}
if !found {
return p.doc, &Error{Reason: "config root required"}
}
return p.doc, nil
}

94
config/diagnostic.go Normal file
View File

@@ -0,0 +1,94 @@
package config
import (
"errors"
"fmt"
"strings"
)
// Error is a value-free diagnostic. Cause may contain private parser details;
// errors.Is/As remain available, but Error never formats Cause automatically.
type Error struct {
File, Field, Env, Reason string
Line, Column int
cause error
}
func (e *Error) Error() string {
where := "config"
if e.File != "" {
where = fmt.Sprintf("config %q", e.File)
}
if e.Line > 0 {
where += fmt.Sprintf(":%d:%d", e.Line, e.Column)
}
if e.Env != "" {
where += fmt.Sprintf(" env %q", e.Env)
}
if e.Field != "" {
where += fmt.Sprintf(" field %q", e.Field)
}
return where + ": " + e.Reason
}
func (e *Error) Unwrap() error { return e.cause }
// DecodeError is retained for callers of errors.As from earlier releases.
type DecodeError = Error
// Invalid reports a public, value-free validation reason for a configuration field.
func Invalid(field, reason string) error { return &ConfigError{Field: field, Message: reason} }
// AtField prefixes a nested validation error without losing its original cause.
// Unstructured validation errors are deliberately not echoed; use Invalid when
// a custom validator needs a public reason rather than a private error message.
func AtField(field string, err error) error {
if err == nil {
return nil
}
d := validationError(err)
d.Field = joinField(field, d.Field)
return d
}
func validationError(err error) *Error {
var d *Error
if errors.As(err, &d) {
out := *d
out.cause = err
return &out
}
var c *ConfigError
if errors.As(err, &c) {
return &Error{Field: c.Field, Reason: c.Message, cause: err}
}
return &Error{Reason: "validation failed", cause: err}
}
func joinField(parent, child string) string {
if parent == "" {
return child
}
if child == "" {
return parent
}
if strings.HasPrefix(child, "[") {
return parent + child
}
return parent + "." + child
}
// Position uses one-based source line/column numbers. Zero means unavailable.
type Position struct{ Line, Column int }
// Document is the common decoder result. Values use JSON-compatible data:
// string, bool, json.Number, nil, []any, map[string]any. Text is available for
// formats such as XML whose scalar tokens have no inherent JSON type.
type Document struct {
Value any
Positions map[string]Position
}
type Text string
func (d Document) diagnostic(field, reason string, cause error) *Error {
p := d.Positions[field]
return &Error{Field: field, Reason: reason, Line: p.Line, Column: p.Column, cause: cause}
}

View File

@@ -1,135 +1,83 @@
package config
import (
"fmt"
"encoding/json"
"os"
"strconv"
"strings"
"time"
)
const defaultEnvPrefix = "GO_OMNI_"
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
// database settings remain owned by database.ConfigFromEnv.
func LoadFromEnv(prefix string) (Config, error) {
return applyEnv(Default(), prefix)
// ServerEnvAliases returns the documented short names for Config fields.
// field is the JSON path of Config in a project (usually "server"), or ""
// for Config itself. Canonical names win over aliases. Both use the same
// EnvOptions.Empty/Lookup and typed parser; collections use JSON syntax.
func ServerEnvAliases(prefix, field string) map[string][]string {
root := prefix
if field != "" {
for _, part := range strings.Split(field, ".") {
root += envName(part) + "_"
}
}
names := map[string]string{
"ADDRESS": "HTTP_ADDR", "TRANSPORT": "TRANSPORT", "TLS_CERT_FILE": "TLS_CERT_FILE", "TLS_KEY_FILE": "TLS_KEY_FILE",
"REQUEST_ID_HEADER": "REQUEST_ID_HEADER", "ENABLE_LOGGER": "ENABLE_LOGGER", "SLOW_REQUEST_THRESHOLD": "SLOW_REQUEST_THRESHOLD",
"TRUSTED_PROXIES": "TRUSTED_PROXIES", "SHUTDOWN_TIMEOUT": "SHUTDOWN_TIMEOUT",
"LOG_LEVEL": "LOG_LEVEL", "LOG_FORMAT": "LOG_FORMAT", "LOG_OUTPUT": "LOG_OUTPUT", "LOG_FILE": "LOG_FILE", "LOG_SERVICE": "SERVICE", "LOG_ENVIRONMENT": "ENVIRONMENT", "LOG_ADD_SOURCE": "LOG_ADD_SOURCE",
"HTTP_READ_HEADER_TIMEOUT": "HTTP_READ_HEADER_TIMEOUT", "HTTP_READ_TIMEOUT": "HTTP_READ_TIMEOUT", "HTTP_WRITE_TIMEOUT": "HTTP_WRITE_TIMEOUT", "HTTP_IDLE_TIMEOUT": "HTTP_IDLE_TIMEOUT", "HTTP_MAX_HEADER_BYTES": "HTTP_MAX_HEADER_BYTES",
"LIFECYCLE_STOP_TIMEOUT": "LIFECYCLE_STOP_TIMEOUT", "LIFECYCLE_DRAIN_TIMEOUT": "LIFECYCLE_DRAIN_TIMEOUT", "LIFECYCLE_CANCEL_TIMEOUT": "LIFECYCLE_CANCEL_TIMEOUT",
}
for _, endpoint := range []string{"LIVENESS", "READINESS", "OPENAPI"} {
for _, part := range []string{"ENABLED", "PATH", "RESPONSE_MODE"} {
if endpoint == "OPENAPI" && part == "RESPONSE_MODE" {
continue
}
name := "ENDPOINTS_" + endpoint + "_" + part
names[name] = name
}
}
aliases := map[string][]string{}
for key, alias := range names {
if root+key != prefix+alias {
aliases[root+key] = []string{prefix + alias}
}
}
return aliases
}
// ApplyEnv overlays nonempty environment values on a caller's defaults or file
// settings, then validates the result. Caller-owned slices are not retained.
// LoadFromEnv loads defaults using the legacy short-name conventions.
func LoadFromEnv(prefix string) (Config, error) { return ApplyEnv(Default(), prefix) }
// ApplyEnv retains legacy whitespace trimming, ignored empty variables and
// comma-separated TRUSTED_PROXIES. New projects should use OverlayEnv with
// ServerEnvAliases for one explicit empty-value policy across the whole config.
func ApplyEnv(defaults Config, prefix string) (Config, error) {
defaults.Errors = defaults.Errors.Clone()
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
return applyEnv(defaults, prefix)
}
func applyEnv(cfg Config, prefix string) (Config, error) {
prefix = strings.TrimSpace(prefix)
if prefix == "" {
prefix = defaultEnvPrefix
}
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
cfg.TrustedProxies = strings.Split(value, ",")
for i := range cfg.TrustedProxies {
cfg.TrustedProxies[i] = strings.TrimSpace(cfg.TrustedProxies[i])
}
}
cfg.Log.Level = envString(prefix+"LOG_LEVEL", cfg.Log.Level)
cfg.Log.Format = envString(prefix+"LOG_FORMAT", cfg.Log.Format)
cfg.Log.Output = envString(prefix+"LOG_OUTPUT", cfg.Log.Output)
cfg.Log.File = envString(prefix+"LOG_FILE", cfg.Log.File)
cfg.Log.Service = envString(prefix+"SERVICE", cfg.Log.Service)
cfg.Log.Environment = envString(prefix+"ENVIRONMENT", cfg.Log.Environment)
var err error
for _, endpoint := range []struct {
name string
enabled *bool
path *string
mode *string
}{
{"LIVENESS", &cfg.Endpoints.Liveness.Enabled, &cfg.Endpoints.Liveness.Path, &cfg.Endpoints.Liveness.ResponseMode},
{"READINESS", &cfg.Endpoints.Readiness.Enabled, &cfg.Endpoints.Readiness.Path, &cfg.Endpoints.Readiness.ResponseMode},
{"OPENAPI", &cfg.Endpoints.OpenAPI.Enabled, &cfg.Endpoints.OpenAPI.Path, nil},
} {
key := prefix + "ENDPOINTS_" + endpoint.name
*endpoint.enabled, err = envBool(key+"_ENABLED", *endpoint.enabled)
if err != nil {
return Config{}, err
}
*endpoint.path = envString(key+"_PATH", *endpoint.path)
if endpoint.mode != nil {
*endpoint.mode = envString(key+"_RESPONSE_MODE", *endpoint.mode)
}
}
for _, field := range []struct {
name string
value *time.Duration
}{
{"HTTP_READ_HEADER_TIMEOUT", &cfg.HTTP.ReadHeaderTimeout},
{"HTTP_READ_TIMEOUT", &cfg.HTTP.ReadTimeout},
{"HTTP_WRITE_TIMEOUT", &cfg.HTTP.WriteTimeout},
{"HTTP_IDLE_TIMEOUT", &cfg.HTTP.IdleTimeout},
} {
if value := strings.TrimSpace(os.Getenv(prefix + field.name)); value != "" {
*field.value, err = time.ParseDuration(value)
if err != nil {
return Config{}, fmt.Errorf("%s%s must be a duration: %w", prefix, field.name, err)
lookup := func(key string) (string, bool) {
raw, ok := os.LookupEnv(key)
raw = strings.TrimSpace(raw)
if ok && raw != "" && key == prefix+"TRUSTED_PROXIES" && !strings.HasPrefix(raw, "[") {
values := strings.Split(raw, ",")
for i := range values {
values[i] = strings.TrimSpace(values[i])
}
data, _ := json.Marshal(values)
raw = string(data)
}
return raw, ok
}
if value := strings.TrimSpace(os.Getenv(prefix + "HTTP_MAX_HEADER_BYTES")); value != "" {
cfg.HTTP.MaxHeaderBytes, err = strconv.Atoi(value)
if err != nil {
return Config{}, fmt.Errorf("%sHTTP_MAX_HEADER_BYTES must be an integer: %w", prefix, err)
}
}
if cfg.EnableLogger, err = envBool(prefix+"ENABLE_LOGGER", cfg.EnableLogger); err != nil {
defaults.Errors = defaults.Errors.Clone()
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
cfg, err := OverlayEnv(defaults, EnvOptions{Prefix: prefix, Empty: "ignore", Lookup: lookup, Aliases: ServerEnvAliases(prefix, "")})
if err != nil {
return Config{}, err
}
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
return Config{}, err
}
if value := strings.TrimSpace(os.Getenv(prefix + "SLOW_REQUEST_THRESHOLD")); value != "" {
cfg.SlowRequestThreshold, err = time.ParseDuration(value)
if err != nil {
return Config{}, fmt.Errorf("%sSLOW_REQUEST_THRESHOLD must be a duration: %w", prefix, err)
}
}
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
cfg.ShutdownTimeout, err = time.ParseDuration(value)
if err != nil {
return Config{}, fmt.Errorf("%sSHUTDOWN_TIMEOUT must be a duration: %w", prefix, err)
}
}
if err := cfg.Validate(); err != nil {
if err = cfg.Validate(); err != nil {
return Config{}, err
}
return cfg, nil
}
func envString(key, fallback string) string {
if value := strings.TrimSpace(os.Getenv(key)); value != "" {
return value
}
return fallback
}
func envBool(key string, fallback bool) (bool, error) {
value := strings.TrimSpace(os.Getenv(key))
if value == "" {
return fallback, nil
}
parsed, err := strconv.ParseBool(value)
if err != nil {
return false, fmt.Errorf("%s must be a boolean: %w", key, err)
}
return parsed, nil
}

View File

@@ -1,100 +1,16 @@
package config
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"os"
"path/filepath"
"reflect"
"strings"
"sync"
"time"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/logx"
"gopkg.in/yaml.v3"
)
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
// field names for both formats, rejects unknown fields and accepts durations
// such as "3s". Application-specific validation remains with the caller.
func DecodeFile[T any](path string, defaults T) (T, error) {
result, err := decodeFile(path, defaults)
if err != nil {
var zero T
return zero, &DecodeError{cause: err}
}
return result, nil
}
// DecodeError keeps input values out of public diagnostics while preserving
// errors.Is/As for explicitly handled underlying I/O and syntax errors.
type DecodeError struct{ cause error }
func (e *DecodeError) Error() string {
return "config: cannot decode file; check its path, format, field names and value types"
}
func (e *DecodeError) Unwrap() error { return e.cause }
func decodeFile[T any](path string, defaults T) (T, error) {
var zero T
f, err := os.Open(path)
if err != nil {
return zero, err
}
defer f.Close()
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
if err != nil {
return zero, err
}
if len(data) > 1<<20 {
return zero, errors.New("config: file exceeds 1 MiB")
}
var value any
switch strings.ToLower(filepath.Ext(path)) {
case ".yaml", ".yml":
dec := yaml.NewDecoder(bytes.NewReader(data))
if err := dec.Decode(&value); err != nil {
return zero, err
}
var extra any
if err := dec.Decode(&extra); err != io.EOF {
return zero, errors.New("config: one YAML document required")
}
case ".json":
if err := apierrors.DecodeCatalogJSON(data, &value); err != nil {
return zero, err
}
default:
return zero, errors.New("config: expected .json, .yaml or .yml")
}
if _, ok := value.(map[string]any); !ok {
return zero, errors.New("config: root must be an object")
}
value, err = durations(value, reflect.TypeOf(defaults))
if err != nil {
return zero, err
}
encoded, err := json.Marshal(value)
if err != nil {
return zero, err
}
// Clone configuration without serializing borrowed json:"-" capabilities.
var result T
original, err := cloneConfiguration(reflect.ValueOf(&defaults).Elem(), 0)
if err != nil {
return zero, err
}
reflect.ValueOf(&result).Elem().Set(original)
dec := json.NewDecoder(bytes.NewReader(encoded))
dec.DisallowUnknownFields()
if err = dec.Decode(&result); err != nil {
return zero, err
}
return result, nil
}
func durations(v any, t reflect.Type) (any, error) {
if t == nil {
return v, nil
@@ -159,7 +75,7 @@ func LoadFile(path, prefix string) (Config, error) {
if err != nil {
return Config{}, err
}
return applyEnv(cfg, prefix)
return ApplyEnv(cfg, prefix)
}
// Reloader only accepts log.level changes. Static changes and invalid files

37
config/lifecycle_test.go Normal file
View File

@@ -0,0 +1,37 @@
package config_test
import (
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"os"
"path/filepath"
"testing"
"time"
)
func TestLifecycleConfiguration(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.yaml")
if err := os.WriteFile(path, []byte("lifecycle:\n stopTimeout: 2s\n drainTimeout: 3s\n cancelTimeout: 4s\n"), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.LoadFile(path, "PHASE_TEST_")
if err != nil {
t.Fatal(err)
}
if cfg.Lifecycle != (lifecycle.ShutdownConfig{StopTimeout: 2 * time.Second, DrainTimeout: 3 * time.Second, CancelTimeout: 4 * time.Second}) {
t.Fatal(cfg.Lifecycle)
}
for _, key := range []string{"STOP_TIMEOUT", "DRAIN_TIMEOUT", "CANCEL_TIMEOUT"} {
t.Setenv("PHASE_TEST_LIFECYCLE_"+key, "6s")
}
cfg, err = config.ApplyEnv(cfg, "PHASE_TEST_")
if err != nil || cfg.Lifecycle.StopTimeout != 6*time.Second || cfg.Lifecycle.DrainTimeout != 6*time.Second || cfg.Lifecycle.CancelTimeout != 6*time.Second {
t.Fatal(cfg.Lifecycle, err)
}
for _, v := range []string{"bad", "-1s"} {
t.Setenv("PHASE_TEST_LIFECYCLE_DRAIN_TIMEOUT", v)
if _, err := config.ApplyEnv(cfg, "PHASE_TEST_"); err == nil {
t.Fatal("invalid duration accepted")
}
}
}

175
config/loader.go Normal file
View File

@@ -0,0 +1,175 @@
package config
import (
"errors"
"io"
"os"
"path/filepath"
"reflect"
"strings"
)
const maxConfigBytes = 1 << 20
const maxConfigDepth = 64
const maxConfigNodes = 100000
// Decoder must not retain or mutate input and must support concurrent calls
// when its Loader is shared. It is trusted application code, not a sandbox.
type Decoder interface {
Decode([]byte) (Document, error)
}
type DecoderFunc func([]byte) (Document, error)
func (f DecoderFunc) Decode(data []byte) (Document, error) { return f(data) }
// Loader owns an immutable per-instance decoder registry. No global registration.
type Loader struct {
decoders map[string]Decoder
sealed bool
}
type LoaderOption func(*Loader) error
// WithDecoder adds a new extension; builtin/duplicate extensions cannot be replaced.
func WithDecoder(extension string, decoder Decoder) LoaderOption {
return func(l *Loader) error {
if l == nil || l.decoders == nil || l.sealed {
return errors.New("config: decoder registry cannot be changed after construction")
}
ext := strings.ToLower(strings.TrimPrefix(extension, "."))
if ext == "" || strings.ContainsAny(ext, "./\\ \t\r\n") {
return errors.New("config: invalid decoder extension")
}
if decoder == nil {
return errors.New("config: nil decoder")
}
v := reflect.ValueOf(decoder)
if (v.Kind() == reflect.Pointer || v.Kind() == reflect.Func || v.Kind() == reflect.Map || v.Kind() == reflect.Slice || v.Kind() == reflect.Interface) && v.IsNil() {
return errors.New("config: nil decoder")
}
if _, exists := l.decoders[ext]; exists {
return errors.New("config: decoder extension already registered")
}
l.decoders[ext] = decoder
return nil
}
}
func NewLoader(options ...LoaderOption) (*Loader, error) {
l := &Loader{decoders: map[string]Decoder{"json": JSONDecoder{}, "yaml": YAMLDecoder{}, "yml": YAMLDecoder{}, "xml": XMLDecoder{}}}
for _, o := range options {
if o == nil {
return nil, errors.New("config: nil loader option")
}
if err := o(l); err != nil {
return nil, err
}
}
l.sealed = true
return l, nil
}
var defaultLoader, _ = NewLoader()
func selectLoader(loaders []*Loader) (*Loader, error) {
if len(loaders) == 0 {
return defaultLoader, nil
}
if len(loaders) != 1 || loaders[0] == nil {
return nil, errors.New("config: expected one non-nil loader")
}
return loaders[0], nil
}
func (l *Loader) read(path string) (doc Document, err error) {
defer func() {
if err != nil {
var d *Error
if errors.As(err, &d) {
c := *d
c.File = path
err = &c
} else {
reason := "cannot read configuration file"
if errors.Is(err, os.ErrNotExist) {
reason = "configuration file does not exist"
} else if errors.Is(err, os.ErrPermission) {
reason = "permission denied reading configuration file"
}
err = &Error{File: path, Reason: reason, cause: err}
}
}
}()
decoder := l.decoders[strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")]
if decoder == nil {
return doc, &Error{Reason: "unsupported configuration format"}
}
info, e := os.Stat(path)
if e != nil {
return doc, e
}
if !info.Mode().IsRegular() {
return doc, &Error{Reason: "configuration must be a regular file"}
}
f, e := os.Open(path)
if e != nil {
return doc, e
}
defer f.Close()
info, e = f.Stat()
if e != nil {
return doc, e
}
if !info.Mode().IsRegular() {
return doc, &Error{Reason: "configuration must be a regular file"}
}
data, e := io.ReadAll(io.LimitReader(f, maxConfigBytes+1))
if e != nil {
return doc, e
}
if len(data) > maxConfigBytes {
return doc, &Error{Reason: "file exceeds 1 MiB"}
}
doc, err = decodeSafely(decoder, data)
if err != nil {
return doc, err
}
if _, ok := doc.Value.(map[string]any); !ok {
return doc, doc.diagnostic("", "root must be an object", nil)
}
return doc, nil
}
func decodeSafely(decoder Decoder, data []byte) (doc Document, err error) {
defer func() {
if recover() != nil {
err = &Error{Reason: "decoder panicked"}
}
}()
doc, err = decoder.Decode(data)
if err != nil {
var d *Error
if !errors.As(err, &d) {
err = &Error{Reason: "invalid configuration syntax", cause: err}
}
}
return
}
// DecodeFile overlays a bounded document onto independent defaults. With no
// loader argument it accepts JSON/YAML/XML. It never opens dependencies.
func DecodeFile[T any](path string, defaults T, loaders ...*Loader) (T, error) {
var zero T
l, err := selectLoader(loaders)
if err != nil {
return zero, err
}
doc, err := l.read(path)
if err != nil {
return zero, err
}
result, err := bindDocument(doc, defaults)
if err != nil {
d := validationError(err)
d.File = path
return zero, d
}
return result, nil
}

397
config/loader_test.go Normal file
View File

@@ -0,0 +1,397 @@
package config
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
"sync"
"testing"
"time"
)
type loaderExample struct {
Server Config `json:"server"`
Upload struct {
MaxBytes int64 `json:"maxBytes" envDefault:"1024"`
} `json:"upload"`
Values []int64 `json:"values"`
Optional *string `json:"optional"`
Enabled bool `json:"enabled"`
Dynamic any `json:"dynamic"`
}
func (c loaderExample) Validate() error {
if err := c.Server.Validate(); err != nil {
return AtField("server", err)
}
if c.Upload.MaxBytes < 0 {
return Invalid("upload.maxBytes", "must be nonnegative")
}
return nil
}
func noEnvironment(string) (string, bool) { return "", false }
func configFile(t *testing.T, ext, raw string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "service."+ext)
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
t.Fatal(err)
}
return path
}
func TestUnifiedFormatAndPrecedence(t *testing.T) {
formats := map[string]string{
"json": `{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":7},"values":[9007199254740993],"optional":null,"enabled":true,"dynamic":{"id":9007199254740993}}`,
"yaml": "server:\n http:\n readTimeout: 2s\nupload:\n maxBytes: 7\nvalues: [9007199254740993]\noptional: null\nenabled: true\ndynamic:\n id: 9007199254740993\n",
"xml": `<config><server><http><readTimeout>2s</readTimeout></http></server><upload><maxBytes>7</maxBytes></upload><values type="array"><item>9007199254740993</item></values><optional type="null"/><enabled>true</enabled><dynamic><id type="number">9007199254740993</id></dynamic></config>`,
}
var reference loaderExample
for _, ext := range []string{"json", "yaml", "xml"} {
t.Run(ext, func(t *testing.T) {
old := "old"
base := loaderExample{Server: Default(), Optional: &old}
cfg, err := LoadProject(configFile(t, ext, formats[ext]), base, EnvOptions{Prefix: "APP_", Lookup: func(key string) (string, bool) {
v, ok := map[string]string{"APP_UPLOAD_MAX_BYTES": "0", "APP_ENABLED": "false"}[key]
return v, ok
}})
if err != nil {
t.Fatal(err)
}
if cfg.Upload.MaxBytes != 0 || cfg.Enabled || cfg.Optional != nil || cfg.Values[0] != 9007199254740993 || cfg.Server.HTTP.ReadTimeout != 2*time.Second {
t.Fatal("values changed", cfg)
}
if cfg.Dynamic.(map[string]any)["id"] != json.Number("9007199254740993") {
t.Fatal("dynamic integer precision lost")
}
if ext == "json" {
reference = cfg
} else if !reflect.DeepEqual(reference, cfg) {
t.Fatal("format-dependent semantics")
}
if base.Optional == nil || *base.Optional != "old" {
t.Fatal("defaults mutated")
}
})
}
cfg, err := LoadProject("", loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
if err != nil || cfg.Upload.MaxBytes != 1024 {
t.Fatal(cfg, err)
}
}
func TestConfigDiagnosticsAndCauses(t *testing.T) {
for _, tc := range []struct {
ext, body, field string
line bool
}{
{"json", "{\n\"upload\": {\"maxBytes\":\"private-value\"}}", "upload.maxBytes", true},
{"yaml", "server:\n http:\n readTimeout: private-value", "server.http.readTimeout", true},
{"xml", "<config>\n<upload><maxBytes>private-value</maxBytes></upload></config>", "upload.maxBytes", true},
{"json", `{"upload":{"maxBytes":-1}}`, "upload.maxBytes", true},
{"yaml", "server:\n http:\n readTimeout: -1s", "server.http.readTimeout", true},
{"json", `{"upload":{"typo":3}}`, "upload.typo", true},
} {
t.Run(tc.ext+tc.field, func(t *testing.T) {
path := configFile(t, tc.ext, tc.body)
_, err := LoadProject(path, loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
var d *Error
if !errors.As(err, &d) || d.File != path || d.Field != tc.field || d.Reason == "" || tc.line && d.Line < 1 {
t.Fatalf("missing location: %#v %v", d, err)
}
if strings.Contains(err.Error(), "private-value") {
t.Fatal("secret leaked", err)
}
})
}
path := filepath.Join(t.TempDir(), "missing.json")
_, err := LoadProject(path, loaderExample{}, EnvOptions{})
if !errors.Is(err, os.ErrNotExist) {
t.Fatal("I/O cause discarded", err)
}
_, err = DecodeFile(configFile(t, "json", `{"count":"private-value"}`), struct {
Count int `json:"count"`
}{})
var typed *json.UnmarshalTypeError
if !errors.As(err, &typed) {
t.Fatal("type cause discarded", err)
}
_, err = LoadProject(configFile(t, "yaml", "upload:\n maxBytes: 7"), loaderExample{Server: Default()}, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { return "-1", k == "APP_UPLOAD_MAX_BYTES" }})
var d *Error
if !errors.As(err, &d) || d.Env != "APP_UPLOAD_MAX_BYTES" || d.Field != "upload.maxBytes" || d.File != "" || d.Line != 0 {
t.Fatal("environment validation wrongly blamed file", err)
}
}
func TestMalformedFormatsAndLimits(t *testing.T) {
for ext, bodies := range map[string][]string{
"json": {`{"enabled":true,"enabled":false}`, `{} {}`, `{"upload":`, `{"enabled":true,"Enabled":false}`, strings.Repeat(`{"a":`, 70) + `0` + strings.Repeat(`}`, 70)},
"yaml": {"enabled: true\nenabled: false", "---\n{}\n---\n{}", "a: &x [1]\nb: *x", "a:\n <<: {}", "upload: [\n"},
"xml": {`<!DOCTYPE config><config/>`, `<config>&private-value;</config>`, `<config/><config/>`, `<config xmlns="x"/>`, `<config type="" type=""/>`, `<config><enabled>true</enabled><enabled>false</enabled></config>`, `<config>text<enabled>true</enabled></config>`, `<config><values type="array"><bad>1</bad></values></config>`},
} {
for i, raw := range bodies {
t.Run(fmt.Sprintf("%s%d", ext, i), func(t *testing.T) {
_, err := LoadProject(configFile(t, ext, raw), loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
if err == nil || strings.Contains(err.Error(), "private-value") {
t.Fatal(err)
}
})
}
}
if _, err := DecodeFile(configFile(t, "json", strings.Repeat(" ", maxConfigBytes+1)), loaderExample{}); err == nil {
t.Fatal("unbounded file")
}
if _, err := DecodeFile(configFile(t, "json", string([]byte{'{', '"', 0xff, '"', ':', '0', '}'})), loaderExample{}); err == nil {
t.Fatal("invalid UTF-8")
}
}
func TestDecoderRegistryAndConcurrentUse(t *testing.T) {
decoder := DecoderFunc(func([]byte) (Document, error) {
return Document{Value: map[string]any{"upload": map[string]any{"maxBytes": json.Number("42")}}}, nil
})
l, err := NewLoader(WithDecoder("custom", decoder))
if err != nil {
t.Fatal(err)
}
path := configFile(t, "custom", "ignored")
var wg sync.WaitGroup
for i := 0; i < 24; i++ {
wg.Add(1)
go func() {
defer wg.Done()
cfg, e := LoadProject(path, loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment}, l)
if e != nil || cfg.Upload.MaxBytes != 42 {
t.Error(e)
}
}()
}
wg.Wait()
if _, err := DecodeFile(path, loaderExample{}); err == nil {
t.Fatal("decoder registration leaked globally")
}
for _, options := range [][]LoaderOption{{WithDecoder("json", decoder)}, {WithDecoder("x", decoder), WithDecoder("x", decoder)}, {WithDecoder("x", DecoderFunc(nil))}} {
if _, err = NewLoader(options...); err == nil {
t.Fatal("invalid registration")
}
}
l, _ = NewLoader(WithDecoder("panic", DecoderFunc(func([]byte) (Document, error) { panic("private-value") })))
_, err = DecodeFile(configFile(t, "panic", ""), loaderExample{}, l)
if err == nil || strings.Contains(err.Error(), "private-value") {
t.Fatal(err)
}
cycle := map[string]any{}
cycle["self"] = cycle
l, _ = NewLoader(WithDecoder("cycle", DecoderFunc(func([]byte) (Document, error) { return Document{Value: map[string]any{"dynamic": cycle}}, nil })))
if _, err = DecodeFile(configFile(t, "cycle", ""), loaderExample{}, l); err == nil {
t.Fatal("decoder bypassed traversal limit")
}
}
func TestUnifiedLoggerNormalization(t *testing.T) {
type project struct {
Server Config `json:"server"`
}
base := project{Server: Default()}
base.Server.Log.File = "old.log"
base.Server.Log.Rotation.Enabled = true
for _, full := range []bool{false, true} {
name := "GO_OMNI_LOG_OUTPUT"
if full {
name = "GO_OMNI_SERVER_LOG_OUTPUT"
}
cfg, err := LoadProject("", base, EnvOptions{Prefix: "GO_OMNI_", Empty: "ignore", Aliases: ServerEnvAliases("GO_OMNI_", "server"), Lookup: func(k string) (string, bool) { return "stdout", k == name }})
if err != nil || cfg.Server.Log.File != "" || cfg.Server.Log.Rotation.Enabled || base.Server.Log.File != "old.log" {
t.Fatal(cfg, err)
}
}
}
func FuzzConfigurationDecoders(f *testing.F) {
f.Add(uint8(0), []byte(`{"count":1}`))
f.Add(uint8(1), []byte("count: 1"))
f.Add(uint8(2), []byte("<config><count>1</count></config>"))
f.Fuzz(func(t *testing.T, format uint8, data []byte) {
if len(data) > 8192 {
data = data[:8192]
}
ds := []Decoder{JSONDecoder{}, YAMLDecoder{}, XMLDecoder{}}
doc, err := ds[int(format)%len(ds)].Decode(data)
if err == nil {
_, _ = bindDocument(doc, struct {
Count int `json:"count"`
}{})
}
})
}
func BenchmarkConfiguration(b *testing.B) {
for name, tc := range map[string]struct {
d Decoder
raw string
}{"json": {JSONDecoder{}, `{"count":9007199254740993,"timeout":"5s","values":[1,2,3]}`}, "yaml": {YAMLDecoder{}, "count: 9007199254740993\ntimeout: 5s\nvalues: [1,2,3]"}, "xml": {XMLDecoder{}, `<config><count>9007199254740993</count><timeout>5s</timeout><values type="array"><item>1</item><item>2</item><item>3</item></values></config>`}} {
b.Run(name, func(b *testing.B) {
b.ReportAllocs()
raw := []byte(tc.raw)
for i := 0; i < b.N; i++ {
doc, err := tc.d.Decode(raw)
if err != nil {
b.Fatal(err)
}
_, err = bindDocument(doc, struct {
Count int64 `json:"count"`
Timeout time.Duration `json:"timeout"`
Values []int `json:"values"`
}{})
if err != nil {
b.Fatal(err)
}
}
})
}
}
// Text-only scalars and embedded config fields use the same behavior in each format.
type testText struct{ Value string }
func (v *testText) UnmarshalText(data []byte) error { v.Value = string(data); return nil }
func TestConfigScalarAndEmbedding(t *testing.T) {
type Fields struct {
Name string `json:"name"`
}
type C struct {
Fields
Nested Fields `json:"nested"`
Scalar testText `json:"scalar"`
Bytes []byte `json:"bytes"`
}
cases := map[string]string{
"json": `{"name":"outer","nested":{"name":"inner"},"scalar":"text","bytes":"aGk="}`,
"yaml": "name: outer\nnested:\n name: inner\nscalar: text\nbytes: aGk=\n",
"xml": `<config><name>outer</name><nested><name>inner</name></nested><scalar>text</scalar><bytes>aGk=</bytes></config>`,
}
for ext, data := range cases {
t.Run(ext, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "config."+ext)
os.WriteFile(path, []byte(data), 0600)
got, err := DecodeFile(path, &C{})
if err != nil {
t.Fatal(err)
}
if got.Name != "outer" || got.Nested.Name != "inner" || got.Scalar.Value != "text" || string(got.Bytes) != "hi" {
t.Fatal(got)
}
})
}
type Tagged struct {
Fields `json:"nested"`
}
doc, _ := (JSONDecoder{}).Decode([]byte(`{"name":"wrong"}`))
if _, err := bindDocument(doc, Tagged{}); err == nil {
t.Fatal("tagged embed incorrectly flattened")
}
}
func TestConfigDefaultsIsolationAndEmptyValues(t *testing.T) {
type Child struct {
Name string `json:"name"`
}
type C struct {
Child *Child `json:"child"`
Items []int `json:"items"`
Labels map[string]string `json:"labels"`
Enabled bool `json:"enabled"`
}
base := C{Child: &Child{Name: "old"}, Items: []int{1}, Labels: map[string]string{"old": "yes"}, Enabled: true}
doc, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"new"},"labels":{"new":"yes"},"items":[],"enabled":false}`))
got, err := bindDocument(doc, base)
if err != nil {
t.Fatal(err)
}
if got.Child.Name != "new" || got.Items == nil || len(got.Items) != 0 || got.Enabled {
t.Fatal(got)
}
if base.Child.Name != "old" || len(base.Items) != 1 || len(base.Labels) != 1 {
t.Fatal("mutated defaults")
}
bad, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"changed"},"items":"bad"}`))
if _, err := bindDocument(bad, base); err == nil {
t.Fatal("invalid input accepted")
}
if base.Child.Name != "old" {
t.Fatal("failed load changed default pointer")
}
null, _ := (JSONDecoder{}).Decode([]byte(`{"child":null,"items":null,"labels":null,"enabled":null}`))
got, err = bindDocument(null, base)
if err != nil || got.Child != nil || got.Items != nil || got.Labels != nil || !got.Enabled {
t.Fatal(got, err)
}
}
func TestDefaultStageDoesNotReadEnvironment(t *testing.T) {
reads := 0
type C struct {
Value int `json:"value" envDefault:"3"`
}
got, err := LoadProject("", C{}, EnvOptions{Lookup: func(string) (string, bool) { reads++; return "", false }})
if err != nil || got.Value != 3 || reads != 1 {
t.Fatal(got, err, reads)
}
d := locateError(Invalid("servers[0].timeout", "must be positive"), "a.json", Document{}, map[string]string{"servers": "APP_SERVERS"})
if d.Env != "APP_SERVERS" || d.File != "" {
t.Fatal(d)
}
}
func TestDecoderRegistryFrozen(t *testing.T) {
l, err := NewLoader()
if err != nil {
t.Fatal(err)
}
if err = WithDecoder("extra", JSONDecoder{})(l); err == nil {
t.Fatal("mutable registry")
}
}
func BenchmarkConfigurationStartup(b *testing.B) {
path := filepath.Join(b.TempDir(), "app.json")
if err := os.WriteFile(path, []byte(`{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":4096}}`), 0600); err != nil {
b.Fatal(err)
}
options := EnvOptions{Prefix: "BENCH_", Lookup: func(name string) (string, bool) {
if name == "BENCH_UPLOAD_MAX_BYTES" {
return "8192", true
}
return "", false
}}
defaults := loaderExample{Server: Default()}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := LoadProject(path, defaults, options); err != nil {
b.Fatal(err)
}
}
}
func BenchmarkConfigurationLarge(b *testing.B) {
type Entry struct {
Timeout time.Duration `json:"timeout"`
Count int `json:"count"`
}
type Large struct {
Entries []Entry `json:"entries"`
}
raw := []byte(`{"entries":[` + strings.TrimSuffix(strings.Repeat(`{"timeout":"3s","count":10},`, 1000), ",") + `]}`)
b.SetBytes(int64(len(raw)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
doc, err := (JSONDecoder{}).Decode(raw)
if err != nil {
b.Fatal(err)
}
if _, err = bindDocument(doc, Large{}); err != nil {
b.Fatal(err)
}
}
}

View File

@@ -7,6 +7,7 @@ import (
"errors"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/logx"
"math"
"os"
"reflect"
@@ -23,6 +24,9 @@ type EnvOptions struct {
Prefix string
Empty string
Lookup func(string) (string, bool)
// Aliases maps canonical absolute environment names to ordered fallback names.
// A present canonical value wins, including an explicit empty value.
Aliases map[string][]string
}
// ApplyDefaults applies envDefault tags to zero fields before file overlays.
@@ -31,30 +35,114 @@ func ApplyDefaults[T any](base T) (T, error) { return projectOverlay(base, EnvOp
// LoadProject applies code/tag defaults, a strict optional file, environment,
// then the project's Validate method, before any logger or dependency is opened.
// envDefault is only applied to zero-valued defaults before reading the file.
func LoadProject[T any](path string, defaults T, options EnvOptions) (T, error) {
func LoadProject[T any](path string, defaults T, options EnvOptions, loaders ...*Loader) (T, error) {
var zero T
loader, err := selectLoader(loaders)
if err != nil {
return zero, err
}
cfg, err := projectOverlay(defaults, options, true)
if err != nil {
return zero, err
}
var doc Document
if path != "" {
cfg, err = DecodeFile(path, cfg)
doc, err = loader.read(path)
if err != nil {
return zero, errors.New("config: cannot decode file; check its path, format, field names and value types")
return zero, err
}
cfg, err = bindDocument(doc, cfg)
if err != nil {
return zero, locateError(err, path, doc, nil)
}
} else {
value, e := cloneConfiguration(reflect.ValueOf(&cfg).Elem(), 0)
if e != nil {
return zero, &Error{Reason: "cannot clone defaults", cause: e}
}
cfg = value.Interface().(T)
}
cfg, err = OverlayEnv(cfg, options)
origins := map[string]string{}
cfg, err = projectOverlay(cfg, options, false, origins)
if err != nil {
return zero, err
}
normalizeProject(reflect.ValueOf(&cfg).Elem(), "", origins, 0)
if valid, ok := any(&cfg).(interface{ Validate() error }); ok {
if err = valid.Validate(); err != nil {
return zero, err
return zero, locateError(err, path, doc, origins)
}
}
return cfg, nil
}
func locateError(err error, path string, doc Document, origins map[string]string) *Error {
d := validationError(err)
field := d.Field
for field != "" && origins[field] == "" {
index := strings.LastIndexAny(field, ".[")
if index < 0 {
field = ""
} else {
field = field[:index]
}
}
if env := origins[field]; env != "" {
d.Env = env
d.File = ""
d.Line = 0
d.Column = 0
return d
}
d.File = path
if d.Line == 0 {
p := doc.Positions[d.Field]
d.Line = p.Line
d.Column = p.Column
}
return d
}
// Framework-owned normalization stays inside the common pipeline. Explicit
// environment stream selection clears a file profile unless a file override
// was also provided; project code never needs to repeat this rule.
func normalizeProject(v reflect.Value, path string, origins map[string]string, depth int) {
if depth > 32 {
return
}
if v.Kind() == reflect.Pointer {
if v.IsNil() {
return
}
v = v.Elem()
}
if v.Kind() != reflect.Struct {
return
}
if v.Type() == reflect.TypeOf(Config{}) && v.CanAddr() {
c := v.Addr().Interface().(*Config)
if origins[joinField(path, "log.output")] != "" && c.Log.Output != "" && (origins[joinField(path, "log.file")] == "" || c.Log.File == "") {
c.Log.File = ""
c.Log.Rotation = logx.RotationConfig{}
}
return
}
for i := 0; i < v.NumField(); i++ {
f := v.Type().Field(i)
if f.PkgPath != "" {
continue
}
name := strings.Split(f.Tag.Get("json"), ",")[0]
if name == "-" {
continue
}
if name == "" {
name = f.Name
}
normalizeProject(v.Field(i), joinField(path, name), origins, depth+1)
}
}
// OverlayEnv overlays the entire project struct without mutating the input.
// Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a
// subtree. required means nonempty string/non-nil pointer or collection; false
@@ -67,7 +155,7 @@ func OverlayEnv[T any](base T, options EnvOptions) (T, error) {
var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
var durationType = reflect.TypeOf(time.Duration(0))
func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error) {
func projectOverlay[T any](base T, options EnvOptions, defaults bool, sourceMaps ...map[string]string) (T, error) {
var zero T
if options.Lookup == nil {
options.Lookup = os.LookupEnv
@@ -85,8 +173,8 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
result := reflect.New(v.Type()).Elem()
result.Set(v)
seen := map[string]bool{}
var walk func(reflect.Value, string, int) error
walk = func(value reflect.Value, prefix string, depth int) error {
var walk func(reflect.Value, string, string, int) error
walk = func(value reflect.Value, prefix, path string, depth int) error {
if depth > 32 {
return errors.New("config: nesting exceeds 32")
}
@@ -107,6 +195,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if jsonName == "" {
jsonName = field.Name
}
fieldPath := joinField(path, jsonName)
key := prefix + envName(jsonName)
if tag[0] != "" {
key = tag[0]
@@ -128,7 +217,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
}
}
if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) {
if err := walk(dst, key+"_", depth+1); err != nil {
if err := walk(dst, key+"_", fieldPath, depth+1); err != nil {
return err
}
continue
@@ -139,14 +228,14 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if !dst.IsNil() {
child.Elem().Set(dst.Elem())
}
if err := walk(child.Elem(), key+"_", depth+1); err != nil {
if err := walk(child.Elem(), key+"_", fieldPath, depth+1); err != nil {
return err
}
if !dst.IsNil() || !child.Elem().IsZero() {
dst.Set(child)
}
if required && !defaults && dst.IsNil() {
return fmt.Errorf("config: %s is required", key)
return &Error{Env: key, Field: fieldPath, Reason: "is required"}
}
continue
}
@@ -161,14 +250,27 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
return fmt.Errorf("config: duplicate environment key %s", key)
}
seen[key] = true
raw, present := options.Lookup(key)
var raw string
var present bool
if !defaults {
raw, present = options.Lookup(key)
}
selectedKey := key
if !present && !defaults {
for _, alias := range options.Aliases[key] {
if raw, present = options.Lookup(alias); present {
selectedKey = alias
break
}
}
}
if defaults {
raw, present = field.Tag.Lookup("envDefault")
present = present && dst.IsZero()
}
if present && raw == "" && !defaults {
if empty == "reject" {
return fmt.Errorf("config: %s must not be empty", key)
return &Error{Env: selectedKey, Field: fieldPath, Reason: "must not be empty"}
}
if empty == "ignore" {
present = false
@@ -177,17 +279,24 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if present {
parsed := reflect.New(dst.Type()).Elem()
if err := parseEnvValue(parsed, raw); err != nil {
return fmt.Errorf("config: %s must be a valid %s", key, dst.Type())
origin := selectedKey
if defaults {
origin = ""
}
return &Error{Env: origin, Field: fieldPath, Reason: "expected a valid " + dst.Type().String(), cause: err}
}
dst.Set(parsed)
if !defaults && len(sourceMaps) > 0 {
sourceMaps[0][fieldPath] = selectedKey
}
}
if required && !defaults && emptyRequired(dst) {
return fmt.Errorf("config: %s is required", key)
return &Error{Env: key, Field: fieldPath, Reason: "is required"}
}
}
return nil
}
if err := walk(result, options.Prefix, 0); err != nil {
if err := walk(result, options.Prefix, "", 0); err != nil {
return zero, err
}
return result.Interface().(T), nil

117
copyx/benchmark_test.go Normal file
View File

@@ -0,0 +1,117 @@
package copyx_test
import (
"errors"
"fmt"
"testing"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
func BenchmarkScenarios(b *testing.B) {
type request struct {
Name string
Count int64
Active bool
}
type result struct {
Label string
Count int64
Active bool
}
source := request{"sample", 9007199254740993, true}
option := copyx.WithFieldNames[request, result](map[string]string{"Name": "Label"})
copier, err := copyx.New(option)
if err != nil {
b.Fatal(err)
}
for _, reuse := range []bool{false, true} {
b.Run(fmt.Sprintf("struct/reuse%t", reuse), func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var target result
var err error
if reuse {
err = copier.Copy(&target, source)
} else {
err = copyx.Copy(&target, source, option)
}
if err != nil {
b.Fatal(err)
}
}
})
}
deep, _ := copyx.New(copyx.WithDeepCopy())
b.Run("bytes64KiB", func(b *testing.B) {
src := make([]byte, 64<<10)
b.SetBytes(int64(len(src)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var dst []byte
if err := deep.Copy(&dst, src); err != nil {
b.Fatal(err)
}
}
})
src := map[string][]int{}
for i := 0; i < 100; i++ {
src[fmt.Sprint(i)] = []int{1, 2, 3}
}
b.Run("map100", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var dst map[string][]int
if err := deep.Copy(&dst, src); err != nil {
b.Fatal(err)
}
}
})
cycle := &detail{Values: []int{1, 2, 3}}
cycle.Next = cycle
b.Run("cycle", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var dst *detail
if err := deep.Copy(&dst, cycle); err != nil {
b.Fatal(err)
}
}
})
b.Run("parallelCycle", func(b *testing.B) {
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
var dst *detail
if err := deep.Copy(&dst, cycle); err != nil {
b.Error(err)
return
}
}
})
})
for _, fresh := range []bool{false, true} {
b.Run(fmt.Sprintf("converter/fresh%t", fresh), func(b *testing.B) {
c, err := copyx.New(copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) ([]int, error) {
if fresh {
return append([]int(nil), v...), nil
}
return v, nil
}))
if err != nil {
b.Fatal(err)
}
src := []int{1, 2, 3}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var dst []int
err := c.Copy(&dst, src)
if fresh && err != nil || !fresh && !errors.Is(err, copyx.ErrSharedReference) {
b.Fatal(err)
}
}
})
}
}

537
copyx/copy.go Normal file
View File

@@ -0,0 +1,537 @@
package copyx
import (
"errors"
"fmt"
"reflect"
"runtime"
"sort"
"sync"
)
var (
ErrType = errors.New("copyx: incompatible types")
ErrNil = errors.New("copyx: nil cannot fill a non-nullable target")
ErrField = errors.New("copyx: unmatched field")
ErrUnsupported = errors.New("copyx: unsupported value; provide an explicit data converter")
ErrSharedReference = errors.New("copyx: deep copy retains a mutable source reference")
ErrLimit = errors.New("copyx: traversal limit exceeded")
)
// Error describes a field/index path without recording the source value.
type Error struct {
Path string
Cause error
}
func (e *Error) Error() string { return e.Path + ": " + e.Cause.Error() }
func (e *Error) Unwrap() error { return e.Cause }
func at(path string, err error) error { return &Error{Path: path, Cause: err} }
func Copy(target, source any, options ...Option) error {
c, err := New(options...)
if err != nil {
return err
}
return c.Copy(target, source)
}
// Clone always makes a deep copy. Nil and empty collections stay distinct.
func Clone[T any](source T, options ...Option) (T, error) {
var target T
options = append(append([]Option(nil), options...), WithDeepCopy())
err := Copy(&target, source, options...)
return target, err
}
// Copy stages the complete result and only assigns target on success. User
// converter side effects (including mutation of source) cannot be rolled back.
// Caller must synchronize concurrent access to source and target themselves.
func (c *Copier) Copy(target, source any) error {
// Alias indexes contain addresses only. Keep the original graph alive until
// validation and commit finish so the GC cannot recycle any indexed address.
defer func() { runtime.KeepAlive(source) }()
if c == nil {
return errors.New("copyx: nil Copier")
}
dst := reflect.ValueOf(target)
if !dst.IsValid() || dst.Kind() != reflect.Pointer || dst.IsNil() {
return errors.New("copyx: target must be a non-nil pointer")
}
src := reflect.ValueOf(source)
s := state{options: &c.options, seen: map[visit]reflect.Value{}}
if c.options.deep {
var err error
s.graph, err = sourceGraph(src)
if err != nil {
return err
}
if src.IsValid() && src.Kind() == reflect.Pointer && !src.IsNil() && dst.Elem().Kind() == reflect.Struct {
id := identity(src)
s.valueRoot = &id
}
}
out, err := s.walk(src, dst.Elem(), dst.Elem().Type(), "$", 0)
if err != nil {
return err
}
if s.graph != nil {
if err = s.graph.audit(out); err != nil {
return err
}
}
dst.Elem().Set(out)
return nil
}
type visit struct {
reference
target reflect.Type
}
type state struct {
options *settings
seen map[visit]reflect.Value
graph *graph
nodes int
allocated uint64
valueRoot *reference
}
func (s *state) reserve(count int, size uintptr, path string) error {
if count < 0 || (count > 0 && uint64(size) > maxCopyBytes/uint64(count)) {
return at(path, ErrLimit)
}
s.allocated += uint64(count) * uint64(size)
if s.allocated > maxCopyBytes {
return at(path, ErrLimit)
}
return nil
}
func nullable(k reflect.Kind) bool {
switch k {
case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan:
return true
}
return false
}
func baseValue(base reflect.Value, t reflect.Type) reflect.Value {
if base.IsValid() && base.Type() == t {
return base
}
return reflect.Zero(t)
}
func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) {
s.nodes++
if depth > maxDepth || s.nodes > maxNodes {
return reflect.Value{}, at(path, ErrLimit)
}
base = baseValue(base, t)
if !shallowSafe(t) {
return reflect.Value{}, at(path, ErrUnsupported)
}
if src.IsValid() && !src.CanInterface() {
return reflect.Value{}, at(path, ErrUnsupported)
}
// Exact converters also receive typed nils and zeros. No method calls,
// Scanner/Valuer invocation or implicit numeric conversion is attempted.
if src.IsValid() {
if convert, ok := s.options.converters[pair{src.Type(), t}]; ok {
out, err := convert(src)
if err != nil {
return reflect.Value{}, at(path, err)
}
if s.graph != nil {
if err = s.graph.audit(out); err != nil {
return reflect.Value{}, at(path, err)
}
}
return out, nil
}
}
if !src.IsValid() || (nullable(src.Kind()) && src.IsNil()) {
if src.IsValid() && !s.nilCompatible(src.Type(), t, map[pair]bool{}) {
return reflect.Value{}, at(path, ErrType)
}
if s.options.nilPolicy == NilSkip {
return base, nil
}
if nullable(t.Kind()) || s.options.nilPolicy == NilZero {
return reflect.Zero(t), nil
}
return reflect.Value{}, at(path, ErrNil)
}
if s.options.skipZero && src.IsZero() {
if !s.nilCompatible(src.Type(), t, map[pair]bool{}) {
return reflect.Value{}, at(path, ErrType)
}
return base, nil
}
if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && !s.options.skipZero && s.options.nilPolicy != NilSkip {
if !s.options.deep || simpleValue(t) {
return src, nil
}
}
if src.Kind() == reflect.Interface {
return s.walk(src.Elem(), base, t, path, depth+1)
}
if t.Kind() == reflect.Interface {
if !src.Type().Implements(t) {
return reflect.Value{}, at(path, ErrType)
}
out, err := s.walk(src, reflect.Value{}, src.Type(), path, depth+1)
if err != nil {
return reflect.Value{}, err
}
v := reflect.New(t).Elem()
v.Set(out)
return v, nil
}
if s.options.deep && forbidden(t) {
return reflect.Value{}, at(path, ErrUnsupported)
}
if src.Kind() == reflect.Pointer && t.Kind() != reflect.Pointer {
return s.walk(src.Elem(), base, t, path, depth+1)
}
if t.Kind() == reflect.Pointer {
if !s.options.deep && src.Type() == t {
return src, nil
}
if src.Kind() == reflect.Pointer {
key := visit{identity(src), t}
if s.valueRoot != nil && key.reference == *s.valueRoot {
return reflect.Value{}, at(path, fmt.Errorf("%w: cyclic root must retain its pointer; use Clone(source) or Copy(&targetPointer, source)", ErrUnsupported))
}
if prior, ok := s.seen[key]; ok {
return prior, nil
}
}
if err := s.reserve(1, t.Elem().Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.New(t.Elem())
if out.Type() != t {
out = out.Convert(t)
}
if src.Kind() == reflect.Pointer {
key := visit{identity(src), t}
s.seen[key] = out
src = src.Elem()
}
var childBase reflect.Value
if !base.IsNil() {
childBase = base.Elem()
}
value, err := s.walk(src, childBase, t.Elem(), path, depth+1)
if err != nil {
return reflect.Value{}, err
}
out.Elem().Set(value)
return out, nil
}
if src.Kind() != t.Kind() {
return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t))
}
if t == timeType && src.Type() == t {
return src, nil
}
switch t.Kind() {
case reflect.Struct:
return s.structure(src, base, t, path, depth)
case reflect.Slice:
if !s.options.deep && src.Type() == t {
return src, nil
}
key := visit{identity(src), t}
if prior, ok := s.seen[key]; ok {
return prior, nil
}
if err := s.reserve(src.Len(), t.Elem().Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.MakeSlice(t, src.Len(), src.Len())
s.seen[key] = out
if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && simpleValue(t.Elem()) {
reflect.Copy(out, src)
return out, nil
}
for i := 0; i < src.Len(); i++ {
v, err := s.walk(src.Index(i), reflect.Value{}, t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return reflect.Value{}, err
}
out.Index(i).Set(v)
}
return out, nil
case reflect.Array:
if src.Len() != t.Len() {
return reflect.Value{}, at(path, ErrType)
}
if err := s.reserve(1, t.Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.New(t).Elem()
for i := 0; i < src.Len(); i++ {
v, err := s.walk(src.Index(i), base.Index(i), t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return reflect.Value{}, err
}
out.Index(i).Set(v)
}
return out, nil
case reflect.Map:
if !s.options.deep && src.Type() == t {
return src, nil
}
key := visit{identity(src), t}
if prior, ok := s.seen[key]; ok {
return prior, nil
}
if err := s.reserve(src.Len(), t.Key().Size()+t.Elem().Size()+32, path); err != nil {
return reflect.Value{}, err
}
out := reflect.MakeMapWithSize(t, src.Len())
s.seen[key] = out
it := src.MapRange()
for it.Next() {
k, err := s.walk(it.Key(), reflect.Value{}, t.Key(), path+"[key]", depth+1)
if err != nil {
return reflect.Value{}, err
}
if !k.Comparable() {
return reflect.Value{}, at(path+"[key]", ErrType)
}
if out.MapIndex(k).IsValid() {
return reflect.Value{}, at(path+"[key]", errors.New("copyx: key conversion collision"))
}
v, err := s.walk(it.Value(), reflect.Value{}, t.Elem(), path+"[value]", depth+1)
if err != nil {
return reflect.Value{}, err
}
out.SetMapIndex(k, v)
}
return out, nil
case reflect.Func, reflect.Chan, reflect.UnsafePointer:
if s.options.deep {
return reflect.Value{}, at(path, ErrUnsupported)
}
}
if src.Type() != t {
return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t))
}
return src, nil
}
var fieldTypes sync.Map
func exportedFields(t reflect.Type) map[string]reflect.StructField {
if known, ok := fieldTypes.Load(t); ok {
return known.(map[string]reflect.StructField)
}
out := map[string]reflect.StructField{}
for _, f := range reflect.VisibleFields(t) {
if f.PkgPath != "" {
continue
}
kind := f.Type.Kind()
if kind == reflect.Pointer {
kind = f.Type.Elem().Kind()
}
if f.Anonymous && kind == reflect.Struct {
continue
}
out[f.Name] = f
}
actual, _ := fieldTypes.LoadOrStore(t, out)
return actual.(map[string]reflect.StructField)
}
func (s *state) nilCompatible(from, to reflect.Type, seen map[pair]bool) bool {
key := pair{from, to}
if seen[key] {
return true
}
seen[key] = true
if from == to {
return true
}
if _, ok := s.options.converters[key]; ok {
return true
}
if to.Kind() == reflect.Interface {
return from.Implements(to)
}
if from.Kind() == reflect.Pointer {
return s.nilCompatible(from.Elem(), to, seen)
}
if to.Kind() == reflect.Pointer {
return s.nilCompatible(from, to.Elem(), seen)
}
if from.Kind() != to.Kind() {
return false
}
switch from.Kind() {
case reflect.Struct:
// Nil must not hide a structurally incompatible type pair. There are no
// field values to visit, but coverage and conversion types still apply.
for i := 0; i < to.NumField(); i++ {
if to.Field(i).PkgPath != "" {
return false
}
}
sources, targets := exportedFields(from), exportedFields(to)
rules := s.options.fields[key]
if rules == nil {
rules = &fields{}
}
reverse, used := map[string]string{}, map[string]bool{}
for a, b := range rules.names {
reverse[b] = a
}
for name, target := range targets {
if rules.ignoreTarget[name] {
continue
}
srcName := name
if renamed, ok := reverse[name]; ok {
srcName = renamed
}
source, ok := sources[srcName]
if !ok || rules.ignoreSource[srcName] || !s.nilCompatible(source.Type, target.Type, seen) {
return false
}
used[srcName] = true
}
if s.options.strict {
for name := range sources {
if !used[name] && !rules.ignoreSource[name] {
return false
}
}
}
return true
case reflect.Slice:
return s.nilCompatible(from.Elem(), to.Elem(), seen)
case reflect.Map:
return s.nilCompatible(from.Key(), to.Key(), seen) && s.nilCompatible(from.Elem(), to.Elem(), seen)
case reflect.Array:
return from.Len() == to.Len() && s.nilCompatible(from.Elem(), to.Elem(), seen)
}
return false
}
func fieldValue(v reflect.Value, index []int) (reflect.Value, bool) {
for _, i := range index {
if v.Kind() == reflect.Pointer {
if v.IsNil() {
return reflect.Value{}, false
}
v = v.Elem()
}
v = v.Field(i)
}
return v, true
}
func setField(v reflect.Value, index []int, value reflect.Value) error {
for j, i := range index {
field := v.Field(i)
if !field.CanSet() {
return ErrUnsupported
}
if j == len(index)-1 {
field.Set(value)
return nil
}
if field.Kind() == reflect.Pointer {
// Detach embedded destination pointers before editing promoted fields.
fresh := reflect.New(field.Type().Elem())
if !field.IsNil() {
fresh.Elem().Set(field.Elem())
}
field.Set(fresh)
v = fresh.Elem()
} else {
v = field
}
}
return nil
}
func (s *state) structure(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) {
if err := s.reserve(1, t.Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.New(t).Elem()
out.Set(base)
if src.Type() == t {
// Private scalar state may be assigned as part of its whole value; mutable
// private state requires a converter, never unsafe writes or silent loss.
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if f.PkgPath != "" && s.options.deep && privateMutable(f.Type) {
return reflect.Value{}, at(path+"."+f.Name, ErrUnsupported)
}
}
out.Set(src)
} else {
for i := 0; i < t.NumField(); i++ {
if t.Field(i).PkgPath != "" {
return reflect.Value{}, at(path, ErrUnsupported)
}
}
}
from, to := exportedFields(src.Type()), exportedFields(t)
rules := s.options.fields[pair{src.Type(), t}]
if rules == nil {
rules = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}}
}
reverse := map[string]string{}
for a, b := range rules.names {
reverse[b] = a
}
names := make([]string, 0, len(to))
for n := range to {
names = append(names, n)
}
sort.Strings(names)
used := map[string]bool{}
for _, name := range names {
target := to[name]
old, _ := fieldValue(base, target.Index)
if rules.ignoreTarget[name] {
if !old.IsValid() {
old = reflect.Zero(target.Type)
}
if err := setField(out, target.Index, old); err != nil {
return reflect.Value{}, at(path+"."+name, err)
}
continue
}
srcName := name
if renamed, ok := reverse[name]; ok {
srcName = renamed
}
source, ok := from[srcName]
if !ok || rules.ignoreSource[srcName] {
return reflect.Value{}, at(path+"."+name, ErrField)
}
used[srcName] = true
v, ok := fieldValue(src, source.Index)
if !ok {
return reflect.Value{}, at(path+"."+name, ErrNil)
}
value, err := s.walk(v, old, target.Type, path+"."+name, depth+1)
if err != nil {
return reflect.Value{}, err
}
if err = setField(out, target.Index, value); err != nil {
return reflect.Value{}, at(path+"."+name, err)
}
}
if s.options.strict {
for name := range from {
if !used[name] && !rules.ignoreSource[name] {
return reflect.Value{}, at(path+"."+name, ErrField)
}
}
}
return out, nil
}

448
copyx/copy_test.go Normal file
View File

@@ -0,0 +1,448 @@
package copyx_test
import (
"errors"
"fmt"
"reflect"
"strings"
"sync"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
type detail struct {
Values []int
Index map[string][]int
Next *detail
}
func TestDeepGraph(t *testing.T) {
src := &detail{Values: []int{1, 2}, Index: map[string][]int{"a": {3}}}
src.Next = src
dst, err := copyx.Clone(src)
if err != nil {
t.Fatal(err)
}
if dst == src || dst.Next != dst {
t.Fatal("pointer cycle not preserved")
}
dst.Values[0] = 9
dst.Index["a"][0] = 8
if src.Values[0] != 1 || src.Index["a"][0] != 3 {
t.Fatal("deep alias")
}
m := map[string]any{}
m["self"] = m
n, err := copyx.Clone(m)
if err != nil {
t.Fatal(err)
}
n["self"].(map[string]any)["new"] = 1
if n["new"] != 1 || m["new"] != nil {
t.Fatal("map cycle changed")
}
x := []any{nil}
x[0] = x
y, err := copyx.Clone(x)
if err != nil {
t.Fatal(err)
}
y[0].([]any)[0] = "changed"
if y[0] != "changed" {
t.Fatal("slice cycle changed")
}
if _, ok := x[0].([]any); !ok {
t.Fatal("source slice changed")
}
}
func TestShallowAndNil(t *testing.T) {
src := detail{Values: []int{1}}
var dst detail
if err := copyx.Copy(&dst, src); err != nil {
t.Fatal(err)
}
dst.Values[0] = 2
if src.Values[0] != 2 {
t.Fatal("shallow unexpectedly detached")
}
for _, slice := range [][]int{nil, {}} {
out, err := copyx.Clone(slice)
if err != nil || (out == nil) != (slice == nil) {
t.Fatal(out, err)
}
}
for _, m := range []map[string]int{nil, {}} {
out, err := copyx.Clone(m)
if err != nil || (out == nil) != (m == nil) {
t.Fatal(out, err)
}
}
var p *int
n := 7
if err := copyx.Copy(&n, p); !errors.Is(err, copyx.ErrNil) || n != 7 {
t.Fatal(n, err)
}
if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilSkip)); err != nil || n != 7 {
t.Fatal(n, err)
}
if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilZero)); err != nil || n != 0 {
t.Fatal(n, err)
}
x := 3
out := &x
if err := copyx.Copy(&out, p); err != nil || out != nil {
t.Fatal(out, err)
}
flag := true
if err := copyx.Copy(&flag, false); err != nil || flag {
t.Fatal(flag, err)
}
flag = true
if err := copyx.Copy(&flag, false, copyx.WithSkipZero()); err != nil || !flag {
t.Fatal(flag, err)
}
}
func TestConverterAliasesRejected(t *testing.T) {
for _, kind := range []string{"same", "nested", "subslice", "capacity", "fieldPointer", "private", "map", "sibling"} {
t.Run(kind, func(t *testing.T) {
src := &detail{Values: []int{1, 2, 3}, Index: map[string][]int{"x": {1}}}
var err error
switch kind {
case "same":
_, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return v, nil }))
case "nested":
_, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return &detail{Values: v.Values}, nil }))
case "subslice":
_, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[1:], nil }))
case "capacity":
_, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[:0], nil }))
case "map":
_, err = copyx.Clone(src.Index, copyx.WithConverter(func(v map[string][]int) (map[string][]int, error) { return v, nil }))
case "fieldPointer":
var dst *int
err = copyx.Copy(&dst, src.Values, copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) (*int, error) { return &v[1], nil }))
case "private":
type opaque struct{ data []int }
v := opaque{src.Values}
_, err = copyx.Clone(v, copyx.WithConverter(func(v opaque) (opaque, error) { return v, nil }))
case "sibling":
type object struct{ A, B []int }
v := object{[]int{1}, []int{2}}
_, err = copyx.Clone(v, copyx.WithConverter(func(_ []int) ([]int, error) { return v.B, nil }))
}
if !errors.Is(err, copyx.ErrSharedReference) {
t.Fatalf("got %v", err)
}
})
}
_, err := copyx.Clone(4, copyx.WithConverter(func(v int) (int, error) { return v, nil }))
if err != nil {
t.Fatal("equal primitive rejected", err)
}
_, err = copyx.Clone("same", copyx.WithConverter(func(v string) (string, error) { return v, nil }))
if err != nil {
t.Fatal(err)
}
good, err := copyx.Clone([]int{1}, copyx.WithConverter(func(v []int) ([]int, error) { return append([]int(nil), v...), nil }))
if err != nil || good[0] != 1 {
t.Fatal(good, err)
}
}
func TestPrivateAndUnsupported(t *testing.T) {
type money struct{ cents *int64 }
n := int64(9007199254740991)
src := money{&n}
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
out, err := copyx.Clone(src, copyx.WithConverter(func(v money) (money, error) { n := *v.cents; return money{&n}, nil }))
if err != nil || *out.cents != n || out.cents == src.cents {
t.Fatal(out, err)
}
tm := time.Now()
cloned, err := copyx.Clone(tm)
if err != nil || !cloned.Equal(tm) {
t.Fatal(cloned, err)
}
for _, v := range []any{make(chan int), func() {}, &sync.Mutex{}, sync.Mutex{}} {
if _, err := copyx.Clone(v); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatalf("%T: %v", v, err)
}
}
}
func TestNilTypeMismatchAndInterfaceConverter(t *testing.T) {
var source *string
target := 7
if err := copyx.Copy(&target, source, copyx.WithNilPolicy(copyx.NilZero)); !errors.Is(err, copyx.ErrType) || target != 7 {
t.Fatal(target, err)
}
var list []int
var targetMap map[string]int
if err := copyx.Copy(&targetMap, list); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
type input struct{ Value any }
type output struct{ Value string }
var out output
err := copyx.Copy(&out, input{}, copyx.WithConverter(func(v any) (string, error) {
if v == nil {
return "nil handled", nil
}
return "value", nil
}))
if err != nil || out.Value != "nil handled" {
t.Fatal(out, err)
}
var lock sync.Mutex
if err := copyx.Copy(&lock, sync.Mutex{}); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
type locked struct {
mu sync.Mutex
Value int
}
var l locked
if err := copyx.Copy(&l, locked{}); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
}
func TestAllocationLimitAndPointerRoot(t *testing.T) {
src := make([]byte, (64<<20)+1)
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) {
t.Fatal(err)
}
n := &detail{}
n.Next = n
var value detail
if err := copyx.Copy(&value, n, copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
var p *detail
if err := copyx.Copy(&p, n, copyx.WithDeepCopy()); err != nil || p.Next != p || p == n {
t.Fatal(p, err)
}
}
func TestFieldsConvertersAndAtomicity(t *testing.T) {
type input struct {
Name string
Age int
Extra string
}
type output struct {
Label string
Age int
Local string
}
rename := map[string]string{"Name": "Label"}
c, err := copyx.New(copyx.WithFieldNames[input, output](rename), copyx.WithIgnoreTarget[input, output]("Local"))
if err != nil {
t.Fatal(err)
}
rename["Name"] = "Missing"
dst := output{Local: "preserved"}
if err = c.Copy(&dst, input{Name: "hello", Age: 2}); err != nil || dst.Label != "hello" || dst.Local != "preserved" {
t.Fatal(dst, err)
}
if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local")); !errors.Is(err, copyx.ErrField) {
t.Fatal(err)
}
if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local"), copyx.WithIgnoreSource[input, output]("Extra")); err != nil {
t.Fatal(err)
}
if err = copyx.Copy(&dst, input{}); !errors.Is(err, copyx.ErrField) {
t.Fatal(err)
}
if _, err = copyx.New(copyx.WithFieldNames[input, output](map[string]string{"Nope": "Age"})); err == nil {
t.Fatal("invalid rule accepted")
}
if _, err = copyx.New(copyx.WithConverter(func(v int) (string, error) { return "", nil }), copyx.WithConverter(func(v int) (string, error) { return "", nil })); err == nil {
t.Fatal("duplicate converter accepted")
}
sentinel := errors.New("conversion failed")
type node struct {
A []int
Z string
}
old := node{[]int{8}, "old"}
before := old.A
err = copyx.Copy(&old, node{[]int{1}, "bad"}, copyx.WithDeepCopy(), copyx.WithConverter(func(v string) (string, error) { return "", sentinel }))
if !errors.Is(err, sentinel) || !strings.Contains(err.Error(), "Z") || old.Z != "old" || old.A[0] != 8 || &old.A[0] != &before[0] {
t.Fatal(old, err)
}
// Exact typed nil converter must get control before the global nil policy.
var ptr *int
var value int
err = copyx.Copy(&value, ptr, copyx.WithConverter(func(v *int) (int, error) {
if v == nil {
return 0, sentinel
}
return *v, nil
}))
if !errors.Is(err, sentinel) {
t.Fatal(err)
}
var small int8 = 3
if err = copyx.Copy(&small, int64(300)); !errors.Is(err, copyx.ErrType) || small != 3 {
t.Fatal(small, err)
}
}
type Embedded struct {
A int
Z string
}
func TestEmbeddedDestinationDetached(t *testing.T) {
type target struct{ *Embedded }
type source struct {
A int
Z int
}
ptr := &Embedded{5, "keep"}
dst := target{ptr}
if err := copyx.Copy(&dst, source{6, 7}); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
if ptr.A != 5 || dst.Embedded != ptr {
t.Fatal("failure mutated embedded target")
}
if err := copyx.Copy(&dst, Embedded{6, "new"}); err != nil {
t.Fatal(err)
}
if ptr.A != 5 || dst.A != 6 {
t.Fatal("success mutated old embedded object")
}
}
func TestConcurrentCopier(t *testing.T) {
c, err := copyx.New(copyx.WithDeepCopy())
if err != nil {
t.Fatal(err)
}
src := &detail{Values: []int{7}, Index: map[string][]int{"a": {9}}}
var wg sync.WaitGroup
for i := 0; i < 64; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for n := 0; n < 20; n++ {
var dst *detail
if err := c.Copy(&dst, src); err != nil {
t.Error(err)
return
}
dst.Values[0] = n
if src.Values[0] != 7 {
t.Error("shared")
}
}
}()
}
wg.Wait()
}
func TestBoundAndCollections(t *testing.T) {
src := &detail{}
p := src
for i := 0; i < 300; i++ {
p.Next = &detail{}
p = p.Next
}
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) {
t.Fatal(err)
}
type from struct{ Value string }
type to struct{ Value string }
var out []to
if err := copyx.Copy(&out, []from{{"a"}, {"b"}}); err != nil || len(out) != 2 || out[1].Value != "b" {
t.Fatal(out, err)
}
var array [2]to
if err := copyx.Copy(&array, [2]from{{"a"}, {"b"}}); err != nil {
t.Fatal(err)
}
if err := copyx.Copy(&array, [1]from{{"a"}}); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
var outMap map[string]int
err := copyx.Copy(&outMap, map[int]int{1: 1, 2: 2}, copyx.WithConverter(func(int) (string, error) { return "same", nil }))
if err == nil || outMap != nil {
t.Fatal(outMap, err)
}
}
func FuzzClone(f *testing.F) {
f.Add([]byte{1, 2, 3})
f.Add([]byte{})
f.Fuzz(func(t *testing.T, data []byte) {
if len(data) > 1024 {
data = data[:1024]
}
src := map[string]any{"bytes": data, "other": []string{"x"}}
out, err := copyx.Clone(src)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(src, out) {
t.Fatal("changed value")
}
if len(data) > 0 {
out["bytes"].([]byte)[0] ^= 255
if data[0] == out["bytes"].([]byte)[0] {
t.Fatal("alias")
}
}
})
}
func TestTraversalNodeLimit(t *testing.T) {
source := make([]any, 200001)
if _, err := copyx.Clone(source); !errors.Is(err, copyx.ErrLimit) {
t.Fatal("expected bounded traversal", err)
}
}
func TestNilCannotHideFieldMismatch(t *testing.T) {
type source struct{ Value string }
type target struct{ Value int }
var src *source
var dst *target
for _, policy := range []copyx.NilPolicy{copyx.NilError, copyx.NilZero, copyx.NilSkip} {
if err := copyx.Copy(&dst, src, copyx.WithNilPolicy(policy)); !errors.Is(err, copyx.ErrType) {
t.Fatal("nil concealed incompatible fields", err)
}
}
if err := copyx.Copy(&dst, src, copyx.WithConverter(func(string) (int, error) { return 0, nil })); err != nil {
t.Fatal(err)
}
text := "keep"
if err := copyx.Copy(&text, 0, copyx.WithSkipZero()); !errors.Is(err, copyx.ErrType) || text != "keep" {
t.Fatal("zero concealed incompatible types", text, err)
}
}
func BenchmarkCopy(b *testing.B) {
for _, n := range []int{1, 100, 1000} {
for _, deep := range []bool{false, true} {
b.Run(fmt.Sprintf("fields%d/deep%t", n, deep), func(b *testing.B) {
src := detail{Values: make([]int, n), Index: map[string][]int{"one": {1, 2}}}
var opts []copyx.Option
if deep {
opts = append(opts, copyx.WithDeepCopy())
}
c, _ := copyx.New(opts...)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var dst detail
if err := c.Copy(&dst, src); err != nil {
b.Fatal(err)
}
}
})
}
}
}

270
copyx/graph.go Normal file
View File

@@ -0,0 +1,270 @@
package copyx
import (
"fmt"
"reflect"
"sort"
"sync"
"time"
)
const maxDepth = 256
const maxNodes = 200000
const maxCopyBytes = 64 << 20
var timeType = reflect.TypeFor[time.Time]()
// Immutable type metadata is shared, never values or converter registrations.
var valueTypes sync.Map
func simpleValue(t reflect.Type) bool {
if known, ok := valueTypes.Load(t); ok {
return known.(bool)
}
result := t == timeType
if !result {
switch t.Kind() {
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr,
reflect.Float32, reflect.Float64, reflect.Complex64, reflect.Complex128, reflect.String:
result = true
case reflect.Array:
result = simpleValue(t.Elem())
case reflect.Struct:
result = !forbidden(t)
for i := 0; result && i < t.NumField(); i++ {
result = simpleValue(t.Field(i).Type)
}
}
}
valueTypes.Store(t, result)
return result
}
type reference struct {
kind reflect.Kind
typ reflect.Type
ptr uintptr
length, capacity int
}
func identity(v reflect.Value) reference {
r := reference{kind: v.Kind(), typ: v.Type(), ptr: uintptr(v.UnsafePointer())}
if v.Kind() == reflect.Slice {
r.length = v.Len()
r.capacity = v.Cap()
}
return r
}
type span struct{ start, end uintptr }
type graph struct {
maps map[uintptr]bool
spans []span
maxEnd []uintptr
}
// References are inspected without reading private values through Interface or
// using unsafe writes. Slice capacity matters: returning source[:0] is an alias.
func scanGraph(v reflect.Value, visit func(reflect.Value, string) error) error {
seen := map[reference]bool{}
nodes := 0
var walk func(reflect.Value, string, int) error
walk = func(v reflect.Value, path string, depth int) error {
nodes++
if depth > maxDepth || nodes > maxNodes {
return at(path, ErrLimit)
}
if !v.IsValid() {
return nil
}
if err := visit(v, path); err != nil {
return err
}
if simpleValue(v.Type()) {
return nil
} // time.Time is a documented immutable value.
switch v.Kind() {
case reflect.Interface:
if !v.IsNil() {
return walk(v.Elem(), path, depth+1)
}
case reflect.Pointer, reflect.Map, reflect.Slice:
if v.IsNil() {
return nil
}
id := identity(v)
if seen[id] {
return nil
}
seen[id] = true
switch v.Kind() {
case reflect.Pointer:
return walk(v.Elem(), path, depth+1)
case reflect.Map:
it := v.MapRange()
for it.Next() {
if err := walk(it.Key(), path+"[key]", depth+1); err != nil {
return err
}
if err := walk(it.Value(), path+"[value]", depth+1); err != nil {
return err
}
}
case reflect.Slice:
if simpleValue(v.Type().Elem()) {
return nil
}
all := v.Slice(0, v.Cap())
for i := 0; i < all.Len(); i++ {
if err := walk(all.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil {
return err
}
}
}
case reflect.Struct:
for i := 0; i < v.NumField(); i++ {
if err := walk(v.Field(i), path+"."+v.Type().Field(i).Name, depth+1); err != nil {
return err
}
}
case reflect.Array:
for i := 0; i < v.Len(); i++ {
if err := walk(v.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil {
return err
}
}
}
return nil
}
return walk(v, "$", 0)
}
func mutableSpan(v reflect.Value) (span, bool) {
var n uintptr
switch v.Kind() {
case reflect.Pointer:
if v.IsNil() {
return span{}, false
}
n = v.Type().Elem().Size()
case reflect.Slice:
if v.IsNil() {
return span{}, false
}
n = uintptr(v.Cap()) * v.Type().Elem().Size()
default:
return span{}, false
}
if n == 0 {
return span{}, false
}
start := uintptr(v.UnsafePointer())
return span{start, start + n}, true
}
func sourceGraph(v reflect.Value) (*graph, error) {
g := &graph{maps: map[uintptr]bool{}}
err := scanGraph(v, func(v reflect.Value, _ string) error {
if r, ok := mutableSpan(v); ok {
g.spans = append(g.spans, r)
}
if v.Kind() == reflect.Map && !v.IsNil() {
g.maps[uintptr(v.UnsafePointer())] = true
}
return nil
})
sort.Slice(g.spans, func(i, j int) bool { return g.spans[i].start < g.spans[j].start })
var end uintptr
for _, r := range g.spans {
if r.end > end {
end = r.end
}
g.maxEnd = append(g.maxEnd, end)
}
return g, err
}
func (g *graph) audit(v reflect.Value) error {
return scanGraph(v, func(v reflect.Value, path string) error {
if forbidden(v.Type()) {
return at(path, ErrUnsupported)
}
switch v.Kind() {
case reflect.Func, reflect.Chan:
if !v.IsNil() {
return at(path, ErrUnsupported)
}
case reflect.UnsafePointer:
if v.Pointer() != 0 {
return at(path, ErrUnsupported)
}
}
if v.Kind() == reflect.Map && !v.IsNil() && g.maps[uintptr(v.UnsafePointer())] {
return at(path, ErrSharedReference)
}
if r, ok := mutableSpan(v); ok {
i := sort.Search(len(g.spans), func(i int) bool { return g.spans[i].start >= r.end })
if i > 0 && g.maxEnd[i-1] > r.start {
return at(path, ErrSharedReference)
}
}
return nil
})
}
func forbidden(t reflect.Type) bool {
// Lock/resource implementations are not data. A custom converter may turn
// them into a different data type but may not manufacture cloned handles.
switch t.PkgPath() {
case "sync", "sync/atomic", "os", "net", "database/sql":
return t.Kind() == reflect.Struct
}
return false
}
// Shallow copies may share handles through pointers, but must not copy lock or
// resource state by value, including state inside a containing struct/array.
var safeValueTypes sync.Map
func shallowSafe(t reflect.Type) bool {
if known, ok := safeValueTypes.Load(t); ok {
return known.(bool)
}
result := true
if forbidden(t) {
safeValueTypes.Store(t, false)
return false
}
switch t.Kind() {
case reflect.Struct:
for i := 0; i < t.NumField(); i++ {
if !shallowSafe(t.Field(i).Type) {
result = false
break
}
}
case reflect.Array:
result = shallowSafe(t.Elem())
}
safeValueTypes.Store(t, result)
return result
}
func privateMutable(t reflect.Type) bool {
if t == timeType {
return false
}
switch t.Kind() {
case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan, reflect.UnsafePointer:
return true
case reflect.Array:
return privateMutable(t.Elem())
case reflect.Struct:
if forbidden(t) {
return true
}
for i := 0; i < t.NumField(); i++ {
if privateMutable(t.Field(i).Type) {
return true
}
}
}
return false
}

169
copyx/options.go Normal file
View File

@@ -0,0 +1,169 @@
// Package copyx copies Go values without JSON, generated code or global registration.
package copyx
import (
"errors"
"fmt"
"reflect"
)
// NilPolicy controls nil input. NilError preserves nil for nullable targets and
// rejects nil to non-nullable targets. NilSkip retains the existing target.
type NilPolicy uint8
const (
NilError NilPolicy = iota
NilZero
NilSkip
)
type pair struct{ from, to reflect.Type }
type converter func(reflect.Value) (reflect.Value, error)
type fields struct {
names map[string]string
ignoreSource, ignoreTarget map[string]bool
}
type settings struct {
deep, skipZero, strict bool
nilPolicy NilPolicy
converters map[pair]converter
fields map[pair]*fields
}
// Option configures one immutable Copier. Options never change global behavior.
type Option func(*settings) error
func WithDeepCopy() Option { return func(s *settings) error { s.deep = true; return nil } }
// WithSkipZero explicitly skips Go zero values, including false, 0 and "".
func WithSkipZero() Option { return func(s *settings) error { s.skipZero = true; return nil } }
// WithStrictFields additionally rejects unconsumed exported source fields.
// Target fields must have a source by default, unless explicitly ignored.
func WithStrictFields() Option { return func(s *settings) error { s.strict = true; return nil } }
func WithNilPolicy(p NilPolicy) Option {
return func(s *settings) error {
if p > NilSkip {
return errors.New("copyx: invalid nil policy")
}
s.nilPolicy = p
return nil
}
}
// WithConverter registers an exact source/target type pair. Typed nils are
// delivered to converters before default nil handling. The function must not
// mutate its input and must be safe for concurrent calls when Copier is shared.
func WithConverter[S, T any](fn func(S) (T, error)) Option {
return func(s *settings) error {
if fn == nil {
return errors.New("copyx: nil converter")
}
key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()}
if _, exists := s.converters[key]; exists {
return fmt.Errorf("copyx: duplicate converter %s -> %s", key.from, key.to)
}
s.converters[key] = func(v reflect.Value) (reflect.Value, error) {
var input S
reflect.ValueOf(&input).Elem().Set(v)
out, err := fn(input)
return reflect.ValueOf(&out).Elem(), err
}
return nil
}
}
// WithFieldNames declares source -> target Go field names for one struct pair.
// It is not a JSON tag mapping. Only actual, unambiguous exported fields qualify.
func WithFieldNames[S, T any](names map[string]string) Option {
copyNames := make(map[string]string, len(names))
for k, v := range names {
copyNames[k] = v
}
return fieldOption[S, T](func(f *fields) error {
for from, to := range copyNames {
if _, exists := f.names[from]; exists {
return fmt.Errorf("copyx: duplicate field rule %s", from)
}
f.names[from] = to
}
return nil
})
}
func WithIgnoreSource[S, T any](names ...string) Option {
names = append([]string(nil), names...)
return fieldOption[S, T](func(f *fields) error {
for _, n := range names {
f.ignoreSource[n] = true
}
return nil
})
}
// WithIgnoreTarget preserves the corresponding existing target fields in Copy.
// Deep mode rejects that preservation if it would retain a source reference.
func WithIgnoreTarget[S, T any](names ...string) Option {
names = append([]string(nil), names...)
return fieldOption[S, T](func(f *fields) error {
for _, n := range names {
f.ignoreTarget[n] = true
}
return nil
})
}
func fieldOption[S, T any](apply func(*fields) error) Option {
return func(s *settings) error {
key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()}
if key.from.Kind() != reflect.Struct || key.to.Kind() != reflect.Struct {
return errors.New("copyx: field options require struct types")
}
f := s.fields[key]
if f == nil {
f = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}}
s.fields[key] = f
}
return apply(f)
}
}
// Copier is immutable after construction. Each Copy owns its traversal state.
type Copier struct{ options settings }
func New(options ...Option) (*Copier, error) {
s := settings{converters: map[pair]converter{}, fields: map[pair]*fields{}}
for _, option := range options {
if option == nil {
return nil, errors.New("copyx: nil option")
}
if err := option(&s); err != nil {
return nil, err
}
}
for key, f := range s.fields {
from, to := exportedFields(key.from), exportedFields(key.to)
used := map[string]bool{}
for src, dst := range f.names {
if _, ok := from[src]; !ok {
return nil, fmt.Errorf("copyx: unknown source field %s.%s", key.from, src)
}
if _, ok := to[dst]; !ok {
return nil, fmt.Errorf("copyx: unknown target field %s.%s", key.to, dst)
}
if used[dst] || f.ignoreSource[src] || f.ignoreTarget[dst] {
return nil, fmt.Errorf("copyx: conflicting field rule %s", dst)
}
used[dst] = true
}
for n := range f.ignoreSource {
if _, ok := from[n]; !ok {
return nil, fmt.Errorf("copyx: unknown source field %s", n)
}
}
for n := range f.ignoreTarget {
if _, ok := to[n]; !ok {
return nil, fmt.Errorf("copyx: unknown target field %s", n)
}
}
}
return &Copier{options: s}, nil
}

View File

@@ -0,0 +1,21 @@
# 统一配置加载示例
从框架根目录运行,无需数据库或外部服务:
```sh
go run ./examples/configuration
go run ./examples/configuration -format json
go run ./examples/configuration -format xml
DEMO_CLIENT_TIMEOUT=500ms DEMO_MAX_ITEMS=0 go run ./examples/configuration
DEMO_MAX_ITEMS=wrong go run ./examples/configuration
```
前三次都输出 `timeout=2s maxItems=100 allowed=[json xml]`。环境覆盖允许显式零值。
最后一条输出定位 `maxItems` / `DEMO_MAX_ITEMS` 的安全错误,不打印输入原值。
也可 `-config /path/to/app.yaml` 读取自己的文件。示例没有建立网络连接。
`Config` 定义字段,`envDefault` 定义缺省值,`Validate` 用 `config.Invalid` 返回字段与公开原因。
`newLoader` 额外注册 `.custom`(演示复用 JSON decoder);同一个 `load` 可读取所有格式。
JSON/YAML/XML 直接可用,无需注册。新增格式只修改解析器注册,不改业务结构或校验。
只打印选定非敏感字段,不输出整个配置。完整格式、错误定位和旧项目迁移见 [CONFIG.md](../../CONFIG.md)。

View File

@@ -0,0 +1,5 @@
{
"client": {"endpoint": "https://example.com", "timeout": "2s"},
"maxItems": 100,
"allowed": ["json", "xml"]
}

View File

@@ -0,0 +1,8 @@
<config>
<client>
<endpoint>https://example.com</endpoint>
<timeout>2s</timeout>
</client>
<maxItems>100</maxItems>
<allowed type="array"><item>json</item><item>xml</item></allowed>
</config>

View File

@@ -0,0 +1,5 @@
client:
endpoint: https://example.com
timeout: 2s
maxItems: 100
allowed: [json, xml]

View File

@@ -0,0 +1,85 @@
// Run with go run ./examples/configuration -format xml, or -config /path/to/app.yaml.
package main
import (
"embed"
"flag"
"fmt"
"os"
"path/filepath"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
//go:embed config.json config.yaml config.xml
var examples embed.FS
type ClientConfig struct {
Endpoint string `json:"endpoint"`
Timeout time.Duration `json:"timeout" envDefault:"3s"`
// This field is deliberately unavailable to configuration files.
Token string `json:"-" env:"DEMO_CLIENT_TOKEN,secret"`
}
type Config struct {
Client ClientConfig `json:"client"`
MaxItems int `json:"maxItems" envDefault:"20"`
Allowed []string `json:"allowed"`
}
func (c Config) Validate() error {
if c.Client.Timeout <= 0 {
return config.Invalid("client.timeout", "must be positive")
}
if c.MaxItems < 0 {
return config.Invalid("maxItems", "must not be negative")
}
return nil
}
func newLoader() (*config.Loader, error) {
// Example of an instance-local extension. A TOML/etc. adapter can implement
// Decoder and return Document using that parser instead; no global registry.
return config.NewLoader(config.WithDecoder("custom", config.DecoderFunc(func(data []byte) (config.Document, error) {
return (config.JSONDecoder{}).Decode(data)
})))
}
func load(path string) (Config, error) {
loader, err := newLoader()
if err != nil {
return Config{}, err
}
return config.LoadProject(path, Config{}, config.EnvOptions{Prefix: "DEMO_", Empty: "reject"}, loader)
}
func run() error {
path := flag.String("config", "", "external configuration file")
format := flag.String("format", "yaml", "embedded example: json, yaml, xml")
flag.Parse()
if *path == "" {
data, err := examples.ReadFile("config." + *format)
if err != nil {
return fmt.Errorf("choose json, yaml or xml")
}
dir, err := os.MkdirTemp("", "goOmniConfiguration-")
if err != nil {
return err
}
defer os.RemoveAll(dir)
*path = filepath.Join(dir, "config."+*format)
if err = os.WriteFile(*path, data, 0600); err != nil {
return err
}
}
cfg, err := load(*path)
if err != nil {
return err
}
// Print selected non-sensitive fields, never the whole configuration.
fmt.Printf("timeout=%s maxItems=%d allowed=%v\n", cfg.Client.Timeout, cfg.MaxItems, cfg.Allowed)
return nil
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

View File

@@ -0,0 +1,34 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
func TestExamples(t *testing.T) {
var previous Config
for i, ext := range []string{"json", "yaml", "xml", "custom"} {
source := ext
if source == "custom" {
source = "json"
}
data, err := examples.ReadFile("config." + source)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "app."+ext)
if err = os.WriteFile(path, data, 0600); err != nil {
t.Fatal(err)
}
got, err := load(path)
if err != nil {
t.Fatal(err)
}
if i > 0 && !reflect.DeepEqual(previous, got) {
t.Fatal("format difference")
}
previous = got
}
}

View File

@@ -0,0 +1,23 @@
syntax = "v1"
@app (defaultResponse: Common)
response Common (
codeType: int32
successCode: 0
errorCode: 9999
messageField: msg
dataMode: always
)
@server (group: arrays)
service arrays {
@handler strings
get /strings returns ([]string)
@handler integers
get /integers returns ([]int64)
@handler dynamic
get /dynamic returns ([]any)
@handler empty
get /empty returns ([]any)
@handler raw
@response raw
get /raw returns ([]bool)
}

View File

@@ -0,0 +1,23 @@
# 结构复制示例
在框架仓库根目录运行 `go run ./examples/copying`;不需要数据库、网络服务或 JSON 映射配置。
示例包含:可复用构造器、Go 字段改名、时间自定义转换、保留 false、深拷贝切片、同类型 Clone,以及拒绝自定义转换器返回输入的原切片。
预期输出:
```text
name=sample enabled=false time=2026-01-01T00:00:00Z sourceTag=original
clone detached: true
shared converter rejected: true
```
测试:`go test -race ./examples/copying`。完整策略、限制与旧生成映射迁移见 [COPY.md](../../COPY.md)。
独立工资三端的实际生成、HTTP 与金额用例见 [集成测试](../../cmd/go-omni/apigen/copyConsumer_test.go) 和 [公共契约](../../cmd/go-omni/apigen/testdata/composition/service.api)。框架根目录运行:
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1
```
该测试在临时独立项目中执行,三个入口共用一个普通 Go copyx helper。公共字段新增时,只修改公共契约和独立业务结构,三个入口的转换规则不增加。

61
examples/copying/main.go Normal file
View File

@@ -0,0 +1,61 @@
// copying demonstrates optional Go-to-Go conversion and checked deep copying.
package main
import (
"errors"
"fmt"
"log"
"time"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
type Input struct {
Name string
Enabled bool
Tags []string
CreatedAt time.Time
}
type View struct {
DisplayName string
Enabled bool
Tags []string
CreatedAt string
}
func run() error {
converter, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
copyx.WithConverter(func(v time.Time) (string, error) { return v.UTC().Format(time.RFC3339), nil }),
)
if err != nil {
return err
}
source := Input{Name: "sample", Enabled: false, Tags: []string{"original"}, CreatedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)}
var target View
if err = converter.Copy(&target, source); err != nil {
return err
}
target.Tags[0] = "changed"
fmt.Printf("name=%s enabled=%t time=%s sourceTag=%s\n", target.DisplayName, target.Enabled, target.CreatedAt, source.Tags[0])
cloned, err := copyx.Clone(source)
if err != nil {
return err
}
cloned.Tags[0] = "separate"
fmt.Println("clone detached:", source.Tags[0] == "original")
_, err = copyx.Clone(source.Tags, copyx.WithConverter(func(v []string) ([]string, error) { return v, nil }))
fmt.Println("shared converter rejected:", errors.Is(err, copyx.ErrSharedReference))
if !errors.Is(err, copyx.ErrSharedReference) {
return errors.New("expected alias rejection")
}
return nil
}
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}

View File

@@ -0,0 +1,9 @@
package main
import "testing"
func TestExample(t *testing.T) {
if err := run(); err != nil {
t.Fatal(err)
}
}

View File

@@ -0,0 +1,12 @@
# HTTP Client 示例
```sh
go run ./examples/httpClient
```
只创建本机临时 HTTP 服务,不访问业务系统、数据库或公网;结束后关闭连接和监听器。
示例创建并复用一个 Client,显式直连,配置连接池/阶段超时,通过 JSON 选项传入 Authorization 和 Accept,并按单次请求收紧响应上限。
输出 `authenticated request succeeded`。示例 token 是固定演示字符串,实际项目从配置或请求上下文取得认证信息。
完整参数、代理规则、流式调用、生命周期、重试边界和兼容说明见 [httpc](../../httpc/README.md)。

View File

@@ -0,0 +1,57 @@
// Run with go run ./examples/httpClient. All HTTP traffic stays on loopback.
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"os"
"time"
"git.yuntuo.hk/k2safe/go-omni/httpc"
)
type result struct {
Message string `json:"message"`
}
func run() error {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer example-token" {
w.WriteHeader(http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(result{Message: "authenticated request succeeded"})
}))
defer upstream.Close()
client, err := httpc.New(httpc.Config{
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
Timeout: 3 * time.Second, DialTimeout: time.Second, TLSHandshakeTimeout: time.Second,
ResponseHeaderTimeout: time.Second, IdleConnTimeout: 30 * time.Second,
MaxConnsPerHost: 16, MaxIdleConnsPerHost: 8, MaxIdleConns: 32,
MaxResponseBytes: 1 << 20, DisableLogging: true,
})
if err != nil {
return err
}
defer client.Close()
var output result
if err = client.JSON(context.Background(), http.MethodGet, upstream.URL, nil, &output,
httpc.WithHeader("Authorization", "Bearer example-token"),
httpc.WithHeaders(http.Header{"Accept": []string{"application/json"}}),
httpc.WithMaxResponseBytes(64<<10),
); err != nil {
return err
}
fmt.Println(output.Message)
return nil
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

View File

@@ -0,0 +1,9 @@
package main
import "testing"
func TestExample(t *testing.T) {
if err := run(); err != nil {
t.Fatal(err)
}
}

View File

@@ -1,5 +1,6 @@
{
"templateVersion": "v0.14.0-preview",
"generatorConfig": true,
"templateVersion": "v0.20.0-preview",
"profile": {
"database": "none",
"storage": "none",
@@ -10,17 +11,18 @@
"module": "example.com/managed",
"entry": "api/managed.api",
"generated": {
"docs/openapi.json": "68c7e9e0294bf62937aa953a44732288c51571108c922a36a72faa72c4360b5f",
"docs/openapi.json": "0370fd6bd6a387e2ef21f6b42b009ce66749846b8bea3d481009a34f3af039a5",
"internal/config/loadGen.go": "5d630b894abdbbd66ddfa7d7d30f7573fc4ff995020a32a9f9157cd4af3a5655",
"internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d",
"internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375",
"internal/handler/openapiGen.go": "ae9b794c63e0f3c8f9d200a5fc6977350139d4a6f00b42a26102c8630d907d03",
"internal/handler/openapiGen.go": "f24cab3abbb8557a3fc4dd93fba23cc0b093f3e0c944d3ba1bd5b0eeff2d96f3",
"internal/handler/responseGen.go": "84397a502bba4777ebfa6fae3d3bc418297acab4f26b04b42e3c31b36c735399",
"internal/handler/routesGen.go": "76c1a5b2644e75e56553dc6e9e862a63128d902dd94807ce7c8aee5491cd1e3e",
"internal/middleware/registryGen.go": "e5f47efa57a1432765ef3419bae59dbf3b4d55b3aac0a6c935ce7a52ba6b47f4",
"internal/svc/runtimeGen.go": "78ffd0b2a196fbc29e29473919d1341e8a2f508f6594ad148c0b399fbcc40018",
"internal/svc/runtimeGen.go": "29f6afb6614881935099c1845ecb3f7902ee773caa077940f566cb8dfdbcdbb1",
"internal/types/dtoManagedd28645a2af1cGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344",
"internal/types/typesGen.go": "372d67f23a313b3e77d2a3b7a236315db9721bc63350bf1f71db1293e57570d1",
"main.go": "8a4051c0d51fc14ddf22c36d09de706f02499817accc937f0785e3d0fb5e2aa5"
"main.go": "64e26d97de7e7c6c17f47186785edc73d43c23d6c20b50bfc7fff6ef2e914f17"
},
"contract": {
"routes": {

File diff suppressed because it is too large Load Diff

View File

@@ -2,7 +2,7 @@ module example.com/managed
go 1.22.0
require git.yuntuo.hk/k2safe/go-omni v0.14.0-preview
require git.yuntuo.hk/k2safe/go-omni v0.21.0-preview
require (
github.com/gabriel-vasile/mimetype v1.4.3 // indirect

View File

@@ -1,11 +1,9 @@
package config
import (
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
"os"
"strings"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
type Config struct {
@@ -23,39 +21,10 @@ func Default() Config {
return Config{Server: server}
}
// Load applies defaults -> file -> typed environment -> validation.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string) (Config, error) {
cfg := Default()
var err error
cfg, err = config.ApplyDefaults(cfg)
if err != nil {
return Config{}, err
}
if path != "" {
cfg, err = config.DecodeFile(path, cfg)
if err != nil {
return Config{}, err
}
}
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_"})
if err != nil {
return Config{}, err
}
// Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc).
cfg.Server, err = config.ApplyEnv(cfg.Server, "")
if err != nil {
return Config{}, err
}
if output := strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")); output != "" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE")) == "" {
cfg.Server.Log.File = ""
cfg.Server.Log.Rotation = logx.RotationConfig{}
}
return cfg, cfg.Validate()
}
// Validate checks configuration before logs or dependencies open.
func (c Config) Validate() error {
if err := c.Server.Validate(); err != nil {
return err
return config.AtField("server", err)
}
return nil

Some files were not shown because too many files have changed in this diff Show More