Compare commits

..

6 Commits

78 changed files with 5355 additions and 1227 deletions

8
AGENTS.md Normal file
View File

@@ -0,0 +1,8 @@
# 开发约定
- 框架保持通用,不内置接入项目的业务类型、字段、错误码或业务规则。
- 2026-09-25 用户要求:每次功能开发完成,必须同步维护根 README.md。README 需要覆盖当前全部公开功能,解释用途、基本用法,并提供可运行示例或链接到 examples 内的完整示例。
- 新增、调整和删除功能都必须同步文档、示例、测试与发布说明。删除功能要明确说明已移除内容、影响范围和迁移步骤,不能继续把历史用法当作当前能力推荐。
- examples 使用通用场景,避免依赖业务项目、私有数据库或线上服务。必要的外部基础设施配置应明确说明。
- 2026-09-25 用户决定:移除 mappings/mappingFiles 及结构映射代码生成机制,改用可选 copyx 工具包。不得恢复另一套 JSON 或 Go 映射声明语言。
- 深拷贝的自定义转换结果不得与输入共享可变引用;发现共享应报错,不能降级成浅拷贝或静默接受。

View File

@@ -1,5 +1,23 @@
# Changelog
## v0.20.0-preview(2026-09-26)
- 生成项目配置加载入口持续由生成器维护;保留业务字段、默认值、校验与 NewLoader 扩展文件。
- config.LoadProject 统一 JSON/YAML/XML、环境覆盖和校验;提供实例级 Decoder 注册。
- 文件/字段/环境/行列的安全诊断,Invalid/AtField 与原始错误链。
- 增加格式等价、精度、null/空数组、并发、资源限制、默认值隔离、真实生成项目与性能验收。
- 旧手写 Load 必须显式迁移;低层 API 保留。固定数组严格长度、YAML alias/merge/custom tag 拒绝。用法及限制见 CONFIG.md。
## v0.19.0-preview(2026-09-25)
- 新增通用可选 copyx:构造器、浅/深拷贝、精确类型转换、字段规则、nil/零值策略、并发复用。
- 深拷贝拒绝自定义结果共享源可变引用;原子提交目标、保留错误链、限制遍历及估算分配。
- 移除 JSON 结构映射配置与生成;提供明确迁移诊断,保留公共字段组合等其他生成能力。
- 同步 README 全部能力索引、COPY.md、可运行示例和开发文档约定。
- 验收与不兼容迁移见 [发布说明](RELEASE.md)、[copyx 验收](reports/copyxAcceptance.md)。v0.14–v0.18 的完整变更保留在发布说明历史段落。
## v0.13.0-preview(2026-09-24)
- 新增通用 lifecycle.Controller,svc.RequestShutdown 支持依赖主动通知正常停止或致命故障,通知不阻塞,首个故障原因保留;生成入口排空 HTTP 后关闭依赖与日志,故障退出码为 1。

178
CONFIG.md Normal file
View File

@@ -0,0 +1,178 @@
# 统一配置加载
v0.20 起,新生成服务的 `main.go` 调用生成器维护的 `config.Load`。项目只需要定义业务字段、默认值和校验;日志、HTTP、数据库等配置使用同一流程,配置加载时不打开数据库、监听端口或创建日志文件。
```text
代码默认值 → envDefault(仅填零值)→ 可选配置文件
→ 完整结构体环境覆盖(含别名)→ 框架规范化 → 项目 Validate
→ 初始化日志及项目依赖 → 启动服务
```
## 生成项目要维护什么
| 文件 | 维护方 | 用途 |
| --- | --- | --- |
| `main.go` | 生成器 | 启动、信号、请求排空、依赖关闭 |
| `internal/config/loadGen.go` | 生成器 | 调用统一加载入口;纳入 `generate --check` |
| `internal/config/config.go` | 项目 | `Config`、`Default()`、`Validate()` |
| `internal/config/loader.go` | 项目 | `NewLoader()`,只有增加文件格式时才需修改 |
| `etc/config.yaml` | 项目 | 部署配置;可改用 JSON/XML 并传 `-config` |
| `internal/svc` | 项目与生成器分文件 | 初始化依赖、登记清理、AppOptions |
比如增加业务的批次数量:在 Config 增加一个字段,在 Validate 加范围校验;需要部署值时改配置文件或环境变量。不修改 main、Load 或 Handler。
```go
// 添加到项目 Config。
BatchSize int `json:"batchSize" envDefault:"100"`
// 添加到项目 Validate。
if c.BatchSize < 1 || c.BatchSize > 1000 {
return omniConfig.Invalid("batchSize", "must be between 1 and 1000")
}
```
自动支持 `GO_OMNI_BATCH_SIZE`。生成服务的空环境值策略为 `ignore`,0 和 false 是有效覆盖;范围由 Validate 决定。
重新生成保留 config.go、loader.go 和 svc 的手写扩展。
## 手写项目用法
```go
package main
import (
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
type Config struct {
Timeout time.Duration `json:"timeout" envDefault:"3s"`
Limit int `json:"limit" envDefault:"100"`
Token string `json:"-" env:"APP_TOKEN,secret"`
}
func (c Config) Validate() error {
if c.Timeout <= 0 {
return config.Invalid("timeout", "must be positive")
}
return nil
}
// cfg, err := config.LoadProject("etc/config.yaml", Config{},
// config.EnvOptions{Prefix: "APP_", Empty: "reject"})
```
`LoadProject` 接收结构体默认值,可实现值接收器或指针接收器的 `Validate() error`。空路径表示不读文件。
`DecodeFile` 只做文件覆盖,不做 envDefault、环境覆盖或业务 Validate;旧的两参数调用仍有效,可用结构体或结构体指针。低层 `ApplyDefaults`、`OverlayEnv`、`LoadFile` 继续存在。
嵌套校验用 `config.AtField("client", err)` 加路径前缀。公共原因通过 `config.Invalid` 提供,原因中不要包含密码、DSN、Token 或输入原值。
## JSON / YAML / XML
按文件扩展名选择解析器:`.json`、`.yaml`、`.yml`、`.xml`,大小写不敏感。所有格式使用 Go 结构体的 **json 字段名**,无需为 XML 复制一份结构或标签。
以下配置等价:
```json
{"client":{"timeout":"2s"},"enabled":false,"ports":[8080,8081],"optional":null}
```
```yaml
client:
timeout: 2s
enabled: false
ports: [8080, 8081]
optional: null
```
```xml
<config>
<client><timeout>2s</timeout></client>
<enabled>false</enabled>
<ports type="array"><item>8080</item><item>8081</item></ports>
<optional type="null"/>
</config>
```
XML 必须使用一个 `<config>` 根;普通叶节点按目标 Go 类型转换。数组显式 `type="array"` 加 `<item>`,不根据重复元素猜测类型。`type="array"` 空元素表示空数组,`type="object"` 表示空对象,`type="null"` 表示 null。`any` 中的 XML 叶节点默认是字符串,需要动态数字/布尔时标注 `type="number"` / `type="bool"`;也可用 `type="string"`。
类型和覆盖规则:
- 缺失字段保留默认值;显式 0、false、空字符串正常覆盖。
- null 清空指针、map、slice、interface;普通值字段保持默认值,沿用 Go JSON 解码语义。需要区分 null 时用指针。
- 数组整体替换;空 slice 保持非 nil;固定长度 Go 数组要求长度一致,避免丢弃多余配置。
- map 合并键,已提供的键整体覆盖对应值;map 中未提供的键保留。
- duration 支持 `250ms`、`3s`;整数保持精度,动态 JSON 数字使用 `json.Number`,不先转 float64。
- 自定义 `json.Unmarshaler` / `encoding.TextUnmarshaler` 仍可接入。这些回调是可信项目代码,必须自己保证并发和副作用安全;不要修改共享外部状态。
- JSON `,string` 字段、嵌入字段遵循 JSON 字段选择;未知字段、重复绑定、冲突均拒绝。
内置解析器限制:单文件最多 1 MiB,解析最多 64 层、100000 个节点;绑定和默认值复制也有深度限制。只接受普通文件,不读取设备或命名管道。不支持多文档、重复字段、无效 UTF-8;YAML 拒绝 alias/merge/custom tag,XML 拒绝 DTD/外部实体、命名空间、业务属性和混合内容。XML 仅支持配置所需的子集,不是通用 XML 文档映射器。
这些限制针对启动配置,避免意外超大文件、别名扩展和模糊覆盖。自定义值解码器的内部字段校验由该解码器负责;例如现有错误目录使用其专有解码规则。
## 环境覆盖
统一入口的 EnvOptions 控制前缀、空值策略、别名和 Lookup。完整 JSON 路径自动转换为 UPPER_SNAKE_CASE:`client.timeout` → `APP_CLIENT_TIMEOUT`。
```go
config.EnvOptions{
Prefix: "GO_OMNI_", Empty: "ignore",
Aliases: config.ServerEnvAliases("GO_OMNI_", "server"),
}
```
生成项目同时接受完整的 `GO_OMNI_SERVER_HTTP_READ_TIMEOUT` 与短别名 `GO_OMNI_HTTP_READ_TIMEOUT`。完整名存在时优先,即便其空值被 ignore,也不回退别名。不会再二次调用 ApplyEnv 覆盖业务字段。
显式 `env:"CUSTOM_NAME"` 是绝对名称;`env:"-"` 跳过子树。`json:"-"` 只在声明显式 env 名时参与,可让密钥只能从环境加载。`secret` 标签表达用途,解析错误对所有字段都不打印值。slice/map 的环境值采用 JSON。`required` 检查字符串非空、指针/集合非 nil;复杂范围由 Validate 负责。详见 [完整业务环境覆盖](DEVELOPMENT.md#完整业务配置的环境覆盖)。
显式环境选择日志 stdout/stderr 会清除文件 profile 的 File/Rotation,除非同时提供非空 File 覆盖。该兼容规则由框架处理。
## 扩展解析器
内置格式直接可用。新增格式只在 `internal/config/loader.go` 的 `NewLoader` 注册:
```go
func NewLoader() (*omniConfig.Loader, error) {
return omniConfig.NewLoader(
omniConfig.WithDecoder("custom", omniConfig.DecoderFunc(
func(data []byte) (omniConfig.Document, error) {
// 演示注册扩展名;实际可在这里调用项目选定的 TOML 等解析库。
return (omniConfig.JSONDecoder{}).Decode(data)
},
)),
)
}
```
Decoder 返回 Document:Value 是 map[string]any 根,子值为 string/bool/json.Number/nil/[]any/map[string]any;无自身类型的叶节点可返回 config.Text。Positions 可选,用 `client.timeout` 或 `servers[0].timeout` 标记 1-based 行列。不提供位置时仍有文件、字段、原因。
自己的语法错误可返回 `&config.Error{Field: "client.timeout", Line: 3, Column: 5, Reason: "expected a duration"}`。Reason 必须安全;普通第三方错误仅公开 `invalid configuration syntax`,内部 cause 通过 errors.Is/As 保留。
注册表属于 Loader 实例,无全局 init 注册;不能覆盖已有扩展名。Loader 可共享,但 Decoder 必须支持并发且不保留/修改输入。解析插件是可信 Go 代码,框架不是插件沙箱。新格式依赖由选择它的项目显式引入,核心不强制引入 TOML 等库。
## 定位配置错误
```text
config "etc/config.yaml":4:12 field "client.timeout": expected a duration such as 5s
config env "APP_CLIENT_TIMEOUT" field "client.timeout": expected a valid time.Duration
```
`errors.As(err, &diagnostic)` 可获取 `*config.Error` 的 File/Field/Env/Line/Column/Reason;原 `DecodeError` 名称保留为别名。`errors.Is` 可判断文件不存在等 I/O 原因。
解析失败显示文件和可获取的行列;Validate 使用 Invalid/AtField 才能定位业务字段。环境覆盖导致的错误优先标环境变量,不错误地归因到文件;数组元素验证也会追溯到覆盖它的集合环境变量。默认值未出现在文件里时没有行列。YAML 语法错误通常只有行号,列显示 1。
不打印底层错误原文或配置值;任意普通 `errors.New(...)` 校验失败的公开原因是 `validation failed`,内部原因仍在链中。项目应改用 Invalid 给出安全的原因。第三方子配置校验若没有字段元数据,只定位到对应子树,框架不会从任意错误文案猜字段。
不要把 Unwrap 后的原始错误、整个配置对象写入公开日志/HTTP;自定义解析器与校验器传入的 Reason 由项目负责脱敏。
## v0.19 及更早项目迁移
先提交现有代码,再升级匹配的 CLI 与框架。旧项目的 `internal/config/config.go` 含手写 Load,新生成器会明确阻止 generate/check/dry-run,不会直接覆盖它。
1. 保留 Config、Default、Validate、业务配置结构及辅助函数。
2. 删除旧 Load。普通的 defaults/file/env/Validate 流程已由 LoadProject 接管;移除由此不再使用的 import。将自定义默认值和校验分别迁入 Default / Validate,自定义文件格式迁入 NewLoader。
3. 框架只在 `loader.go` 不存在时创建 NewLoader;已有项目文件同名时需要保证它提供同签名函数。无需手写 loadGen.go。
4. 重新生成,执行 `generate --check`、`go test ./...`、`go build .`,再用真实配置验证启动。
保留业务需要的显式 env 标签/规则;如果原 Load 有解密或其他项目特殊步骤,不要直接删除,应先明确其归属,必要时在自定义 Decoder 中处理。生成器不自动搬动或猜测手写行为。`api upgrade` 仍可预览较早版本的基础设施差异,但已修改的配置必须按上述步骤迁移。
不会移除低层配置 API;本次移除的是**新项目里重复手写的加载流程**,不删除旧项目的业务文件。
完整可运行例子:[examples/configuration](examples/configuration/README.md)。

141
COPY.md Normal file
View File

@@ -0,0 +1,141 @@
# 可选拷贝工具 copyx
v0.19 用普通 Go 调用取代 JSON 结构映射生成。`copyx` 只依赖标准库,不需要注册到 App,不执行 JSON 往返,也不会自动暴露数据库模型。普通 Logic 直接使用 `.api` 生成的 DTO;只有存在独立业务类型或需要隔离可变数据时才用它。
完整可运行代码:[examples/copying](examples/copying)。在框架根目录运行:
```sh
go run ./examples/copying
go test -race ./copyx ./examples/copying
```
## 最简单的调用
```go
import "git.yuntuo.hk/k2safe/go-omni/copyx"
type Request struct { Name string; Enabled bool }
type Input struct { Name string; Enabled bool }
req := Request{Name: "sample", Enabled: false}
var input Input
if err := copyx.Copy(&input, req); err != nil { return err }
```
同名、同类型字段直接赋值。默认不会跳过 `false`、`0`、空字符串。目标字段没有来源时报错;多余源字段默认忽略,使用 `WithStrictFields()` 可要求源字段也全部使用或显式忽略。
默认是浅拷贝:同类型指针、map、slice 共享引用,符合普通 Go 赋值。JSON tag 不参与字段匹配。不会自动调用方法、SQL Scanner/Valuer,或把 `int32`、`int64`、string、命名标量互相强转。
新字段:普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时,两边各声明一次;同名同类型不增加拷贝配置。检查在运行时发生,实际类型对必须有测试,不能把运行时工具当作旧生成器的编译期字段覆盖检查。
## 深拷贝与构造器
```go
snapshot, err := copyx.Clone(source) // 保持源的静态类型,始终深拷贝
if err != nil { return err }
copier, err := copyx.New(copyx.WithDeepCopy(), copyx.WithStrictFields())
if err != nil { return err }
var result Output
if err := copier.Copy(&result, source); err != nil { return err }
```
`Copy` 的目标必须是非 nil 指针;`Clone[T]` 返回 `(T, error)`。重复执行时在初始化阶段创建 Copier 并复用;配置之后不可变,每次调用独立持有遍历状态,可以并发使用。源、目标的并发读写仍由调用方同步,自定义函数也须支持并发。
深拷贝会分配指针、map、slice,递归处理结构、数组、接口中的具体值。nil 集合与非 nil 空集合保持区别;slice 结果容量为长度,不复制长度之外的隐藏元素。普通循环和重复指针可处理。循环图应保留根指针,使用 `Clone(sourcePointer)` 或 `Copy(&targetPointer, sourcePointer)`;将循环指针根压成值结构会明确报错。
它保证结果不引用输入中的可变对象,不保证保留所有底层内存布局关系:不同重叠 slice 视图会分别复制,内部指针指向某个切片元素的关系也不重建。字符串和零尺寸值可安全共享;`time.Time` 按 Go 不可变值使用(Location 不做独立复制)。不要通过 unsafe 或 uintptr 隐藏可变引用后要求工具识别。
## 字段差异与自定义转换
```go
type Input struct { Name string; UpdatedAt time.Time; InternalID string }
type View struct { DisplayName string; UpdatedAt string; LocalNote string }
copier, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithStrictFields(),
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
copyx.WithIgnoreSource[Input, View]("InternalID"),
copyx.WithIgnoreTarget[Input, View]("LocalNote"),
copyx.WithConverter(func(v time.Time) (string, error) {
return v.UTC().Format(time.RFC3339), nil
}),
)
if err != nil { return err }
out := View{LocalNote: "preserve"}
if err := copier.Copy(&out, input); err != nil { return err }
```
字段规则的类型参数必须是结构类型,字段名使用导出的 Go 字段名。构造时检查不存在的字段、重名及冲突。不支持字符串路径表达式;复杂嵌套差异为该嵌套类型注册 converter,或直接写 Go。
`WithIgnoreSource` 只是声明该源字段不参与使用;目标同名字段若仍需保留,还需 `WithIgnoreTarget`。忽略目标会保留原目标值,Clone 中则为零值。深拷贝时如果保留值仍与源共享可变引用,也会拒绝。
转换器的签名是 `func(S) (T, error)`,按**精确类型对**匹配,不能重复注册。金额、编号、日期格式由项目明确转换。需要逐字段不同规则时,给整个源/目标结构注册函数;函数返回独立的新目标。无需另一份映射配置或全局注册。
精确 converter 优先于 nil/零值策略,能收到 typed nil;例如 `func(*Money) (Money, error)` 可自行决定 nil 是否允许。绑定到 `any` 字段的 converter 能接收 nil interface,但顶层未带类型的 `nil` 没有精确类型可匹配。浅拷贝中已经直接共享的同类型容器不会继续逐项转换其内部;需要遍历内部时选择深拷贝,或为该容器类型注册 converter。
## 深拷贝不能原样返回可变引用
```go
copier, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithConverter(func(v []string) ([]string, error) {
return v, nil // 会返回 ErrSharedReference
}),
)
```
检查覆盖完整源图与转换结果,包括新父对象引用旧子对象、原 map、重叠子切片、`source[:0]` 的可写容量、指向源数组元素的指针,以及未导出字段中的引用。会检查每个 converter 的结果和最终目标,不允许静默降级为浅拷贝。
返回相同字符串、整数并不会报错,它们不是可变引用。正确切片 converter 可以写:
```go
copyx.WithConverter(func(v []string) ([]string, error) {
if v == nil { return nil, nil }
out := make([]string, len(v))
copy(out, v)
return out, nil
})
```
第三方 Decimal/自定义 Money 如果含未导出的指针,默认深拷贝拒绝;注册该类型的 converter,使用它自己的构造/Clone API 创建独立值,避免转成 float64 损失精度。框架不内置某一金额库的规则。无法独立复制的类型直接返回错误。
## nil、零值和错误
| 选项 | 行为 |
| --- | --- |
| 默认 `NilError` | nil → 可空目标保留 nil;nil → 非可空目标报 `ErrNil` |
| `WithNilPolicy(NilZero)` | 显式允许 nil → 零值;可空目标仍为 nil |
| `WithNilPolicy(NilSkip)` | nil 保留已有目标;新集合元素没有旧目标,保留元素零值 |
| `WithSkipZero()` | 显式跳过 Go 零值,包括 `false`、`0`、`""`;非 nil 空集合不是 Go 零值 |
所有 nil 策略都不把不兼容类型自动变成兼容类型。非 nil 指针与值之间可以按对应元素转换;nil 到非可空目标仍需选择策略。局部更新不能默认使用 NilZero/SkipZero,它们可能改变清空字段或布尔值的业务语义。
工具只能复制绑定后的 Go 值:如果输入类型已经将“缺失”和“JSON null”合并为 nil,无法恢复两者区别。需要三态时,应先设计包含 presence 信息的类型和绑定逻辑。
赋值使用临时结果,成功后才替换目标;转换失败不会交付部分结果。`*copyx.Error` 包含 Go 字段/索引路径,`errors.Is/As` 保留内部错误链,工具不主动在错误中加入源值。converter 自己返回的错误文本仍由项目负责。
converter 必须只读输入;其外部副作用、输入修改无法回滚,panic 原样传播。框架只检查与本次输入的共享引用,无法证明 converter 未返回外部全局缓存的引用。它是受信 Go 代码扩展,不是可运行不可信代码的沙箱。
## 支持范围与资源限制
- 支持结构体、指针、interface、slice、同长度 array、map。map key 转换后冲突或不可比较则报错。
- 导出匿名结构字段可按提升后的无歧义名称访问;源嵌入指针为 nil 且需要其字段时明确报错。跨类型目标包含私有字段时拒绝;同类型私有纯值可作为整体保留。
- 不克隆文件、数据库连接、网络连接、锁、原子状态。已知标准库资源/同步结构按值复制也拒绝;深拷贝中的非 nil func、chan、unsafe.Pointer 拒绝。其他封装资源类型应由项目排除,工具不能识别所有自定义句柄。
- 单次遍历深度上限 256、节点上限 200,000;内置分配的估算预算为 64 MiB,超限返回 `ErrLimit`。预算不是进程堆内存硬限制,reflect 元数据和 converter 自行分配不计入。大对象流式读写应使用 IO/存储接口。
- 只缓存不可变类型元数据,不缓存请求值、用户 converter 或实例策略。没有 `unsafe` 写入、JSON 往返、隐式数值截断或方法调用。
- 这是运行时反射工具,会比直接 Go 赋值多检查与分配。热点应复用 Copier,并用真实数据测量;性能记录见 [本版验收](reports/copyxAcceptance.md)。
## 从旧 mapping 迁移
v0.19 **彻底移除**映射配置加载、Go 类型加载、映射代码生成及 `pointerResult` 等附属选项。公共 `.api` 字段组合、scalar、中间件工厂、Repository 开关继续存在。
1. 先检查每个转换是否必要。普通 Logic 可直接使用生成 DTO;独立核心保留自己的类型。
2. 将 `mapping.Xxx(...)` 调用改为普通 Go 或小型 `copyx` helper。字段改名、nil、金额规则写在该 helper 同一处。指针返回可直接 `var out *Response; err := copier.Copy(&out, input)`。
3. 用真实类型对测试缺失/null/零值/空数组、字段覆盖、金额精度和错误链。需要源字段新增时提醒遗漏,启用 `WithStrictFields()` 并测试。
4. 从 `omni.gen.json` 删除 `mappings` / `mappingFiles`,删除无其他用途的映射 JSON 文件。
5. 确认已无调用,再删除 `internal/mapping/mappingsGen.go`。如之前手改过文件,先迁出手写内容。生成器遇到该旧文件会阻止生成并提示迁移,**不会静默删除**;其余业务文件照常保留。
6. 升级框架与 CLI 到同版,执行 generate、generate --check、go test ./...;多个服务再执行 workspace generate/check。新 manifest 自动去掉已移除文件的所有权记录。
历史发布记录保留旧能力说明,仅用于理解旧版本,不代表 v0.19 继续支持。

View File

@@ -1,8 +1,8 @@
# 通用业务开发指南(v0.17.0-preview)
# 通用业务开发指南(v0.20.0-preview)
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
本版新增的公共字段组合、可选结构映射、自动中间件工厂和 Repository 开关,见 [生成能力指南](GENERATION.md)。
公共字段组合、自动中间件工厂和 Repository 开关,见 [生成能力指南](GENERATION.md)。必要的结构转换使用可选 [copyx](COPY.md),v0.19 已移除旧 JSON 映射生成。
## 返回数组、可空对象和 null
@@ -102,7 +102,9 @@ defer reader.Close()
## 完整业务配置的环境覆盖
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML 文件 → 全结构体环境覆盖 → 既有框架环境别名 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
生成器维护 `internal/config/loadGen.go`;项目维护 Config/Default/Validate 和可选 NewLoader。三种格式、可扩展解析器、字段错误定位及旧 Load 迁移见 [统一配置指南](CONFIG.md)。
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML/XML 文件 → 全结构体环境覆盖(含别名)→ 规范化 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
```go
type ClientConfig struct {
@@ -278,7 +280,7 @@ go-omni workspace check --file omni.workspace.json
## v0.13 项目升级
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.17.0-preview(或显式引用当前源码)。
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.20.0-preview(或显式引用当前源码)。
```sh
go-omni api upgrade --dir . --dry-run

View File

@@ -1,18 +1,10 @@
# 公共字段、结构映射与生成选项
# 公共字段、中间件工厂与生成选项
本页对应 `v0.17.0-preview`。CLI 与框架依赖固定到同一版本:
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
# 已有业务项目内执行:
go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
```
私有仓库访问设置见 [安装说明](README.md#准备环境与安装)。升级不会覆盖业务文件;已有中间件保持 handlers 模式,切换工厂模式按本页迁移步骤完成。
本页对应 `v0.19.0-preview`。CLI 与框架依赖固定到同一版本,安装见 [README](README.md#准备环境与安装)。
## 一份项目生成配置
新项目根目录生成开发者维护的 `omni.gen.json`:
新项目根目录的 `omni.gen.json` 只描述生成选项:
```json
{
@@ -21,12 +13,11 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
}
```
该文件与 `.api` 一起提交。每次 `api generate`、`--check`、`--dry-run` 和 `workspace generate/check` 都会读取;`api validate/export/format` 只处理契约,不加载业务 Go 类型。
- `repository: none` 是默认值,不创建空 Repository 文件。`group` 按 `.api` 的 group 补齐 `internal/repository/<group>/repository.go`;文件始终属于业务。切回 none 后不再补骨架,已有文件不删除、不覆盖。实际 SQL、ORM 和仓储划分由项目决定。
- `middleware: factories` 是新项目默认值,按下文自动初始化。`handlers` 保留原有 `Defaults()` 与 `func(next Handler) Handler` 接口,供既有项目逐步迁移。
- `mappings` 可省略;只有明确列出的类型对才生成映射。未启用时不会运行 Go 类型加载命令,也不增加运行时依赖。
- 升级首次遇到没有配置的旧 manifest 时,补一份 `middleware: handlers` 的配置,保留原有手写注册;Repository 默认 none。生成配置建立后不能直接删除,否则生成失败,避免误关闭映射或改变中间件模式。
- `repository: none` 默认不创建空 Repository 文件;`group` 按 group 补齐可编辑骨架。已有文件始终保留;切回 none 不会删除业务文件。SQL、ORM 和仓储划分由项目决定。
- `middleware: factories` 是新项目默认值,自动调用带依赖的工厂。`handlers` 保留原有 `Defaults()` 及手动注册方式,供既有项目迁移。
- 该文件与 `.api` 一起提交。generate/check/dry-run 和 workspace 采用相同规则;validate/export/format 只处理接口契约。
- 旧 manifest 没有配置时补 `middleware: handlers`、`repository: none`。已有配置不能直接删除,避免意外切换中间件模式。
- v0.19 已删除 `mappings` 和 `mappingFiles`;遇到旧配置会明确报错,不读取子映射文件或执行映射生成。处理步骤见 [copyx 迁移](COPY.md#从旧-mapping-迁移)。
## 公共字段组合
@@ -53,104 +44,13 @@ type AdminRequest {
允许多层组合,上限 32 层、单类型 4096 字段、全契约展开 65536 字段。Go 匿名指针嵌入并非本功能;生成 DTO 中不会使用匿名嵌入。
## 可选结构映射
## DTO 与业务类型
`.api` 生成 DTO,业务独立维护输入/结果类型。项目明确选择需要自动赋值的类型对;不从数据库反向生成公开接口,不自动推断反向映射。
普通 Logic 直接接收生成的 Request、返回生成的 Response,不额外创建相同结构。只有独立核心或明确不同的输入/输出职责才保留业务类型。
假设 `types.AdminRequest` 有 `Name string`、`Salary amount.Money`、`CompanyID string`,业务的 `domain.EmployeeInput` 有 `EmployeeName string` 和同类型 `Salary`:
必要转换直接写 Go,或用可选 `copyx.Copy(&target, source)`。同名同类型字段自动赋值,字段改名和类型变化通过 Go 参数、转换函数明确设置,见 [完整示例](examples/copying/main.go)。无需 JSON 映射配置、包路径字符串或额外生成文件。
```json
{
"repository": "none",
"middleware": "factories",
"mappings": [
{
"name": "ToEmployeeInput",
"source": {"package": "example.com/payroll/internal/types", "type": "AdminRequest"},
"target": {"package": "example.com/payroll/internal/domain", "type": "EmployeeInput"},
"fields": {
"EmployeeName": {"source": "Name"}
},
"ignoreSource": {
"CompanyID": "由 Logic 单独做企业范围授权,不属于请求体输入"
}
}
]
}
```
执行 `go-omni api generate` 后,产生独立的生成文件 `internal/mapping/mappingsGen.go`:
```go
func ToEmployeeInput(source types.AdminRequest) (domain.EmployeeInput, error) {
var result domain.EmployeeInput
result.EmployeeName = source.Name
result.Salary = source.Salary
return result, nil
}
```
包别名由生成器决定。业务调用 `mapping.ToEmployeeInput(*req)`;反向响应映射按另一条 source/target 规则声明。两个类型完全相同时通常无需映射。
### 类型与覆盖检查
自动赋值要求同名且 **Go 类型一致**(类型别名按 Go 类型系统处理),字段名使用 Go 名称,不按 JSON 标签猜对应关系。int32 → int64、`*Money` → Money、数值 ID → string 都必须显式转换。不会暗中补零、解引用或执行数字强转。
每个源字段必须被使用或出现在 `ignoreSource`;每个目标字段必须被赋值或出现在 `ignoreTarget`。忽略规则必须给出非空原因,目标忽略明确意味着保留零值;规则引用不存在的字段也会失败。新字段若无法自动匹配,会在生成阶段报错。两边同时添加的同名同类型字段会自动加入生成代码,`--check` 检出未重新生成的差异。
根类型是项目可导入的具名结构;支持值类型匿名嵌入,规则可用 `Employee.EmployeeName` 这样的实际 Go 字段路径。同名提升歧义、未导出字段和匿名指针嵌入拒绝处理。普通具名嵌套字段使用独立映射或转换函数,不猜测内部结构。泛型声明不能直接作为根;具体字段类型按 Go 类型系统检查。
配置变化、遗漏映射、类型错误会在写入前失败。映射生成与其他生成产物使用同一套文件所有权机制;手改映射生成文件会被拒绝覆盖。
### 显式转换函数
```json
"fields": {
"Salary": {
"source": "Salary",
"convert": {"package": "example.com/payroll/internal/convert", "name": "SalaryValue"}
}
}
```
支持这两种精确签名:
```go
func SalaryValue(source *amount.Money) amount.Money
func SalaryValue(source *amount.Money) (amount.Money, error)
```
项目选择一种实现。nil、默认值、金额精度、日期格式和错误码由函数负责;若两边已经是同一个 Money 类型,无需再次解析或序列化。转换函数应仅负责值转换,不执行 SQL 或改变授权范围。
错误通过 `%w` 加上映射名/目标字段路径,保留 `errors.Is/As` 和框架错误链;返回零目标值,不交付部分完成的结果。包装不主动添加源值或转换后的值。错误是否公开仍由项目错误目录和响应策略决定。
### 嵌套对象、数组与引用
`mapping` 指向同配置中的另一条映射,`elements` 明确要求逐项映射:
```json
"fields": {
"Summary": {"source": "Summary", "mapping": "ToSummary"},
"Items": {"source": "Items", "mapping": "ToItem", "elements": true}
}
```
`mapping` 与 `convert` 互斥。`elements` 可以配合其中任意一种,也可以对类型相同的元素直接赋值;支持 slice→slice、等长 array→array,禁止隐式互转。nil slice 保持 nil,非 nil 空 slice 保持非 nil,逐项失败的错误包含索引,循环映射声明被拒绝。
未选择 elements 的同类型指针、map、slice 使用普通 Go 赋值,**共享引用**,不做隐式深拷贝。elements 会分配新的目标 slice,但元素内的引用仍遵循其映射规则。需要深拷贝时显式提供转换函数。
映射不保存原 JSON 的字段存在信息。普通 `*T` 在绑定后可能已把缺失和 null 合并为 nil;无法在转换阶段恢复区分。有三态需要的项目必须先使用能携带该信息的输入类型及绑定扩展;没有声明三态时,保持当前输入类型的语义。完整表单更新和局部更新由业务决定,框架不把 nil 自动解释成“保留旧值”。
### 生成顺序与 CI
1. 先从 `.api` 生成项目,添加业务结构、scalar 和转换函数,执行 `go mod tidy`。
2. 在 `omni.gen.json` 列出映射,执行 `go-omni api generate`。
3. Logic 调用生成函数。后续改字段后重复生成,不修改 `*Gen.go`。
4. CI 同时运行 `go-omni api generate --check` 与 `go test ./...`。
启用映射后,CLI 通过 `go list -deps -export` 加载明确声明的包与依赖,按当前构建环境做类型检查,不执行业务代码、init 或转换函数。采用待生成 DTO 的临时 overlay,避免“新增字段还看旧 DTO”。go.mod/go.sum 的副本及 overlay 都在临时目录,check 不修改项目文件;Go 自己的构建/模块缓存可能更新。使用当前 GOOS/GOARCH/GOFLAGS、`GOWORK=off`;依赖用 go.mod 与 replace 表达,生成超时为 90 秒。
类型包/转换函数包不能依赖输出包 `internal/mapping`;把转换函数放在独立包中,避免导入环。若包本身无法编译,先运行 `go list -deps -export <相关包>` 排查;CLI 不回显可能含私有仓库凭据的 Go 下载输出。纯值字段转换只生成标准 Go 代码,不引入运行时反射搬运、JSON 往返、ORM 或队列依赖。
`copyx` 在运行时检查类型和字段,**不提供旧映射生成阶段的静态字段覆盖保证**;CI 应给实际类型对添加测试。普通业务字段新增通常只改 `.api` 和 Logic;独立 DTO/领域类型都需要此字段时,两边各加一次,同名同类型无需追加拷贝配置。
## 中间件工厂自动注册
@@ -204,9 +104,13 @@ factory 模式不再在 Init 写 `Middlewares["..."]`;残留非空手写注册
| 项目维护 | 生成器持续维护 |
| --- | --- |
| `.api` 公共字段与接口 | 展开后的 DTO、Handler、OpenAPI |
| `omni.gen.json` 的映射与选项 | `internal/mapping/mappingsGen.go` |
| Go 业务类型、scalar、转换函数 | 编译期可检查的赋值与调用代码 |
| `omni.gen.json` 的 Repository/中间件选项 | 对应骨架和注册代码;已有手写骨架不覆盖 |
| 独立 Go 业务类型、scalar、可选 copyx 调用和测试 | 不生成结构转换文件 |
| middleware Dependencies、具名工厂、svc 访问器 | 中间件 registry、启动调用 |
| Logic、按需 Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 |
| Logic、Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 |
回归样例位于 `cmd/go-omni/apigen/testdata/composition`,为独立三端服务,不导入任何 eTax 代码。
独立三端工资契约测试位于 `cmd/go-omni/apigen/testdata/composition`,不导入任何接入项目。它验证公共字段、Money scalar、三端 Handler、copyx、重新生成保留业务内容,以及新增字段不需要更新转换规则。
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1
```

View File

@@ -6,6 +6,24 @@
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
## 请求追踪与基础指标
`omni.New()` 的默认链包含 RequestID、访问日志(可配置开关)和 Recovery;无需重复挂载。业务通过 `c.Context()` 传递 requestId,`logx.FromContext(ctx)` 取得该 App 的 Logger。手动组装 httpx 链时,也可使用 `middleware.RequestID("X-Request-ID")`、`middleware.AccessLog()` 和 `middleware.Recovery()`。
轻量进程指标直接使用 `metrics`,无需 OTel collector:
```go
// import omni "git.yuntuo.hk/k2safe/go-omni"
// import "git.yuntuo.hk/k2safe/go-omni/metrics"
app := omni.New()
measurements := metrics.New()
app.Use(measurements.Middleware())
app.GET("/metrics", measurements.Handler())
// 也可以直接读取 measurements.Snapshot()。
```
记录请求数、失败数、处理中数量、累计耗时与 HTTP 状态分布;返回的是 JSON 快照,并非 Prometheus 文本格式。指标接口是否暴露以及访问权限由项目配置。需要 HTTP/SQL/Redis spans 和标准 OTel exporter 时,使用 [telemetry 示例](examples/platform/main.go)。CORS 由项目中间件或代理配置,当前没有内置 CORS 模块。
## 缓存:cache 与 cache/redis
```go

View File

@@ -1,6 +1,6 @@
# 生成项目与文件维护边界
v0.17.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
v0.20.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
## 创建和日常开发
@@ -38,7 +38,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
这是轻量源码分析,不运行项目 init、不执行 Go 命令、不下载依赖。项目自定义类型别名、嵌入方法或 build tags 等无法可靠判断时,会给 REVIEW 并交给 Go 编译器验证;不把这些情况强行判为签名错误。它不检查方法体是否引用已删除字段,也不证明业务逻辑正确。CI 中应同时执行 `go-omni api generate --check` 和 `go test ./...`,按项目实际构建参数测试。当前版本首次接手缺少快照的 manifest v2 时,会明确提示建立基线,不猜测旧契约;Logic 签名检查仍会执行。
本版新增生成选项、公共字段组合和映射,详见 [生成能力指南](GENERATION.md)。
生成选项、公共字段组合和可选 Go 拷贝工具,详见 [生成能力指南](GENERATION.md)。
## 哪些文件由用户维护
@@ -46,7 +46,8 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
| --- | --- |
| `api/*.api` | 路由、DTO、具名中间件、响应模板、请求/响应模式 |
| `etc/config.yaml` | 运行参数;密钥通过环境变量提供 |
| `internal/config/config.go` | 自定义配置字段、加载、默认值与校验 |
| `internal/config/config.go` | 自定义配置字段、默认值与校验 |
| `internal/config/loader.go` | 需要额外文件格式时注册 Decoder |
| `internal/svc/serviceContext.go` | 依赖字段和初始化,持有加载后的 Config |
| `internal/svc/options.go` | AppOptions、GlobalMiddleware;注册 Go 错误目录和全局扩展 |
| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 |
@@ -54,14 +55,15 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
| `internal/logic/<group>/*.go` | 业务方法,使用标准 context.Context |
| `internal/repository/<group>/*.go` | 数据访问;默认不补空骨架,可选 group 模式 |
| `internal/middleware/*Factory.go`、`dependencies.go` | 中间件工厂与依赖小接口,svc 实现访问器 |
| `omni.gen.json`、映射转换函数 | 生成选项、显式类型对与差异规则 |
| `omni.gen.json` | Repository 和中间件生成选项;不再声明结构映射 |
| 可选的 Go 转换函数 / copyx 调用 | 只有独立类型确实需要转换时维护,见 [copyx](COPY.md) |
| 项目 Go 错误包 | 自行维护 Catalog 和常量,可跨服务共用 |
| 自定义标量文件、`*Bind.go`、`*Manual.go` | 只有显式选择相应扩展时才生成/使用 |
| 项目测试、迁移、部署脚本、go.mod/go.sum | 项目所有,生成器不重置其内容 |
以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。
生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、可选映射 `internal/mapping/mappingsGen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
生成器持续管理:`internal/config/loadGen.go`、`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。
@@ -113,6 +115,8 @@ c.RequestShutdown(err)
`Lifecycle *lifecycle.Controller` 是 svc 持有的通用控制器,初始化时创建,生成入口使用它的 Context。该 Context 表示服务正在停止,不代表数据库已经关闭。不能把它当作在途请求的 Context,也不要在它取消时立刻关闭请求仍依赖的资源。发号器失去租约后拒绝发号等业务安全规则仍由项目实现。
配置加载入口 `internal/config/loadGen.go` 也由生成器维护。业务扩展 Config/Default/Validate;只有新格式才需要修改 NewLoader。JSON/YAML/XML、错误定位和旧 Load 迁移见 [配置指南](CONFIG.md)。
## 配置内置接口
生成项目在 `etc/config.yaml` 明确列出以下默认配置;可以独立关闭、修改路径,或让健康接口返回原始 JSON:
@@ -211,7 +215,7 @@ get /download (DownloadReq)
## 已有服务接入步骤
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.17.0-preview:
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.20.0-preview:
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。

107
README.md
View File

@@ -2,15 +2,17 @@
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
**从这份 README 开始接入。** 当前版本 **`v0.17.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**从这份 README 开始接入。** 当前版本 **`v0.20.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**本版新增**:公共字段组合、可选结构映射、带依赖的中间件工厂、按需 Repository 骨架,使用与迁移见 [生成能力指南](GENERATION.md)。
**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。
普通接口的流程是:**修改 `.api` → 生成 Request/Response → 写 Logic → 测试**。有独立职责的业务类型才需要转换;可直接写 Go,也可以按需使用 [copyx](COPY.md)。公共字段组合、中间件工厂、Repository 开关继续支持,见 [生成能力指南](GENERATION.md)。
Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。
本版补齐 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录,以及统一环境变量别名覆盖。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。
契约支持 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录,以及统一环境变量别名覆盖。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。
已有通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
@@ -18,6 +20,63 @@ Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可
使用 GORM 的项目可以选择 [gormx 适配器](database/gormx/README.md),复用数据库初始化和连接池,并将 SQL 日志接入 logx。
## 当前功能与使用入口
下表是当前公开能力的索引。每项给出调用入口和完整说明;模块按需使用,`omni.New()` 不会自动初始化数据库、Redis、ORM、队列或云存储。示例中的 import 统一以 `git.yuntuo.hk/k2safe/go-omni/` 开头。
| 功能与用途 | 怎么用 | 解释与示例 |
| --- | --- | --- |
| 项目、接口与文档生成 | `go-omni new`;改 `.api` 后 `api generate`;`--check` 检查漂移 | [下方快速开始](#新项目生成并运行)、[CLI](cmd/go-omni/README.md)、[完整生成项目](examples/managedService/README.md) |
| 公共字段与跨服务共享契约 | `.api` 组合对象字段;`omni.imports.json` 声明共享根;workspace 批量生成 | [公共字段示例](GENERATION.md#公共字段组合)、[共享契约与 workspace](DEVELOPMENT.md) |
| 中间件依赖工厂、可选 Repository | `omni.gen.json` 选择 factories / none;在 `New<Name>(deps)` 写一次实现 | [配置与完整工厂示例](GENERATION.md) |
| 路由、分组、请求上下文 | `omni.New()`、`app.GET` / `Group` / `Use`;业务使用 `context.Context` | [基本服务](examples/basic/main.go)、[已有项目接入](#已有项目按模块接入) |
| HTTP 网络与扩展适配 | 配置 transport 为 nethttp / native / http2;自定义 Router/Runner | [网络层配置与契约](transport/README.md)、[性能与限制](PERFORMANCE.md) |
| HTTP 老接口桥接 | 通过官方 Handler/Middleware 适配器接入标准 `net/http`,跟踪实际响应状态 | [接入说明](MIGRATION.md)、[桥接/上传/自定义 ID 示例](examples/compatibility/main.go) |
| 请求校验、动态 JSON、自定义标量 | `.api` 声明来源/校验/scalar;手写契约用 `contract.New[T]`;简单绑定可用 `validate.JSON/Query/URI` | [语法与例子](API.md)、[请求策略](DEVELOPMENT.md) |
| 数组、null、原始请求、上传下载 | `.api` 声明数组/可空返回、body/multipart 与大小限制;必要时使用 Bind/Manual 扩展 | [契约示例](examples/contracts)、[含上传/回调/下载的项目](examples/managedService/README.md) |
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
| 文件配置与环境覆盖 | 默认值→JSON/YAML/XML→环境→校验;完整业务结构用 `config.LoadProject[T]` / 环境别名 | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
| 出站 HTTP、熔断 | `httpc.New` 复用连接池与截止时间;`resilience.New` 隔离故障依赖 | [HTTP 用法](MODULES.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
| 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
## 统一配置加载
新生成项目的加载流程由框架管理。业务在 `internal/config/config.go` 增加字段、默认值和 `Validate`,无需改 main 或复制环境变量解析。JSON、YAML、XML 共用字段和校验;扩展格式在 `loader.go` 注册。
```go
// 项目 Config 中增加字段,自动支持 GO_OMNI_BATCH_SIZE:
BatchSize int `json:"batchSize" envDefault:"100"`
// 项目 Validate 中增加业务范围检查:
if c.BatchSize < 1 {
return omniConfig.Invalid("batchSize", "must be positive")
}
```
错误包含文件、字段、可用的行列或环境变量名;输入原值不自动写入错误。
```sh
go run ./examples/configuration -format xml
DEMO_CLIENT_TIMEOUT=500ms go run ./examples/configuration
```
详见 [统一加载、解析器、迁移](CONFIG.md) 和 [完整示例](examples/configuration/README.md)。**旧项目须先迁移手写 Load,再重新生成**;生成器会保留业务配置并拒绝冲突,不静默覆盖。
## 选择接入方式
| 你的项目 | 从哪里开始 |
@@ -38,7 +97,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
```sh
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.20.0-preview
export PATH="$(go env GOPATH)/bin:$PATH"
go-omni --help
```
@@ -195,7 +254,7 @@ go-omni new --api catalog.api \
在业务项目的 **go.mod 所在目录**执行:
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
go get git.yuntuo.hk/k2safe/go-omni@v0.20.0-preview
```
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
@@ -389,6 +448,38 @@ return apierrors.TextTooLong("企业名称", 100)
更详细的配置、作用域和升级步骤见 [统一错误目录](errors/README.md)。
### 可选结构转换与深拷贝
普通接口不需要任何拷贝配置,直接使用生成的 Request/Response。独立业务结构之间有转换需求时:
```go
// import "git.yuntuo.hk/k2safe/go-omni/copyx"
var input DomainInput
if err := copyx.Copy(&input, req); err != nil { return err }
// 源与目标之间的指针、map、slice 需要隔离时:
snapshot, err := copyx.Clone(source)
if err != nil { return err }
```
默认浅拷贝,同名同类型字段自动赋值,保留零值;默认不会把 nil 填成非可空目标的零值。不要为使用工具新增一套重复模型。
需要重复使用或自定义规则时创建 Copier:
```go
copier, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
copyx.WithConverter(func(v time.Time) (string, error) {
return v.UTC().Format(time.RFC3339), nil
}),
)
if err != nil { return err }
if err := copier.Copy(&view, input); err != nil { return err }
```
自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错;字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md),完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查,类型对需测试;深拷贝不适用于连接、锁等资源。
### 数据库、存储及其他模块
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
@@ -465,7 +556,7 @@ go test ./...
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
CLI 与业务 module 固定使用 `v0.17.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
CLI 与业务 module 固定使用 `v0.20.0-preview`。v0.14–v0.19 项目先按 [配置入口迁移](CONFIG.md#v019-及更早项目迁移) 整理旧 Load,再升级依赖与 CLI 并重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
## 按需深入
@@ -481,3 +572,7 @@ CLI 与业务 module 固定使用 `v0.17.0-preview`。v0.14.x 项目升级依赖
| 网络性能、测试条件与适用范围 | [性能说明](PERFORMANCE.md)、[HTTP/2 验收](transport/native/HTTP2.md) |
当前版本可用于固定版本接入与业务验证,仍标记为 preview。`.api` 借鉴 go-zero 的接口开发方式,支持范围以本仓库文档为准,不是完整 goctl 语法兼容层;HTTP/2 保持实验状态,OSS/R2 云账号实网验收不属于本次发布范围。
## 后续开发约定
每次功能开发完成,同步更新这份功能索引、用法说明、`examples/` 和发布记录;删除或改变功能必须注明影响与迁移。代码、示例和文档一起回归后再发布。约定已记录在 [AGENTS.md](AGENTS.md),后续开发继续遵循。

View File

@@ -1,4 +1,80 @@
# v0.17.0-preview
# v0.20.0-preview
统一配置加载,减少生成项目重复维护的启动代码。
- 生成器维护 `internal/config/loadGen.go`,调用 `config.LoadProject`;项目只扩展 Config/Default/Validate,可选在 loader.go/NewLoader 注册新格式。入口、环境覆盖、日志输出规范化无需业务重复实现。
- JSON/YAML/XML 使用同一个加载与绑定流程;Decoder 通过实例注册,无全局 init 或额外强制依赖。内置格式、字段、文件大小与层数均有明确限制。
- `config.Error` 携带文件、字段、环境变量、可用行列及安全原因;`Invalid` / `AtField` 支持业务验证。保留内部错误链,不自动输出原值;大整数、空集合、null、默认值隔离和自定义反序列化有回归覆盖。
- 配置解析在日志、依赖和监听初始化之前;生成/check/dry-run 使用一致入口和文件所有权规则。
- **迁移要求:旧项目先整理手写 Load,再重新生成。** 不静默覆盖业务文件。低层配置 API 保留;新项目不再复制手写加载流程。固定 Go 数组严格检查长度,YAML alias/merge/custom tag 不再接受,避免模糊覆盖和扩展行为。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.20.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.20.0-preview
# 先按 CONFIG.md 迁移旧 Load,再执行:
go-omni api generate
go-omni api generate --check
go test ./...
go build .
```
完整用法、格式、脱敏边界和迁移见 [统一配置指南](CONFIG.md),可运行例子见 [examples/configuration](examples/configuration/README.md)。回归、模糊测试与启动加载性能见 [验收记录](reports/configurationAcceptance.md)。仍为 preview;项目的实际配置和业务依赖应自行验收。
---
# v0.19.0-preview(历史版本)
将可选结构转换改为直接 Go 调用,减少业务项目需要维护的配置。普通接口继续使用 `.api` 生成 DTO,不要求增加独立模型或任何拷贝规则。
- 新增只依赖标准库的 `copyx`:Copy、Clone、可复用 Copier;浅/深拷贝、严格字段覆盖、nil/零值策略、字段改名、忽略字段、精确类型 converter。没有全局注册和隐式数字强转。
- 深拷贝检查整个输入和输出图。自定义 converter 原样返回指针、map、slice,或新对象仍嵌套旧引用时返回错误;保留普通循环图,拒绝资源对象及超出遍历/估算分配预算的输入。
- 转换失败不提交部分目标;保留错误路径和原始错误链。并发复用只共享不可变类型元数据;源/目标读写和 converter 副作用由调用方负责。
- **不兼容变更:删除 mappings / mappingFiles、Go 类型加载与映射代码生成。** 检出旧配置/旧生成文件时给出迁移提示;不会静默删除业务正在使用的代码。公共字段组合、scalar、中间件工厂、Repository 选项保留。
- README 补齐当前能力入口、解释和用法,新增 [完整 copyx 指南](COPY.md)、[可运行示例](examples/copying);[AGENTS.md](AGENTS.md) 记录每次修改同步文档/示例/移除说明的约定。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview
# 如用过旧 mapping,先按 COPY.md 完成迁移:
go-omni api generate
go-omni api generate --check
go test ./...
```
迁移步骤和语义边界见 [copyx](COPY.md#从旧-mapping-迁移)。新工具做运行时检查,不能声称具备旧生成器的编译期字段覆盖保证;实际类型对必须有测试。独立三端工资示例验证金额精度、缺失/null/零值、空集合、错误链和字段新增;不修改或代替接入项目的完整业务回归。
46 个测试包的全量 race(含实际生成项目)、vet/build、Go 1.22 兼容测试通过;30 秒 fuzz 无失败,14 种基准场景各测量三轮。验收条件、性能数据及本轮未覆盖的外部环境见 [v0.19 验收记录](reports/copyxAcceptance.md)。本版仍为 preview。 空缓存远程 CLI 与 7 组真实生成消费者验收通过,无本地 replace;标签固定在 `fd83235ce7fc15ce88a482d385d70e61f25d7b32`,详见 [远程记录](reports/copyxRemote.json)。
---
以下是历史版本记录,其中 mapping 功能已在 v0.19 移除,不应继续作为当前接入方式。
# v0.18.0-preview(历史版本)
补齐可空嵌套映射和按模块拆分配置,减少重复转换代码,业务空值规则仍由项目显式选择:
- 嵌套 mapping 支持 `*S → *T` 保留 nil、`S → *T`,以及指定 nil zero/error 后的 `*S → T`;同样适用于 slice/等长数组元素。错误保留字段路径、索引和内部错误链,失败不返回部分结果。
- `mappingFiles` 显式引用项目内的映射文件,支持跨文件向前引用;统一检查重名、循环、类型与字段覆盖。禁止多层导入、路径越界和隐式覆盖,生成期间核对全部配置输入。
- 同类型 `*T → T` 可显式配置 nil zero/error;现有 converter 保持完整控制,不默认放宽类型或混入领域规则。
- `pointerResult: true` 额外生成 `<Name>Ptr(S) (*T,error)`,原值函数保留;失败返回 nil 与原错误,辅助函数名冲突在生成前拒绝。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.18.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.18.0-preview
go-omni api generate
go-omni api generate --check
go test ./...
```
v0.17 的单文件 mappings、现有 converter 和业务文件继续保留。配置拆分时移动完整映射条目,主配置显式列出子文件;不保留重复定义。新能力均为可选生成配置,不增加运行时依赖。nil zero 表示 Go 零值并跳过子映射;业务必填、金额精度、编号格式和快照复制仍由项目负责。
使用与迁移见 [生成能力指南](GENERATION.md),回归和发布后验收见 [验收记录](reports/nullableMappingAcceptance.md)。本版仍为 preview。
全量 44 个测试包的 race、vet、build 和实际进程验收通过;空缓存远程 CLI 与 8 组实际生成消费者回归也通过,同版依赖且无本地 replace。标签固定在 `4563a6354fac19f22cd8d407fd76d98b8cc9c406`。
---
# v0.17.0-preview(历史版本)
减少项目里的重复声明、字段赋值和中间件装配代码,业务规则继续由项目控制:
@@ -19,6 +95,8 @@ go test ./...
使用方法、文件归属和升级步骤见 [生成能力指南](GENERATION.md),回归与发布验收见 [验收记录](reports/generationExperienceAcceptance.md)。本版仍为 preview;框架验收不等于业务项目完成迁移。
全量 44 个测试包的 race、vet、build 和实际进程验收通过;发布后的空缓存远程 CLI 与 7 组生成消费者回归也通过,同版依赖且无本地 replace。标签固定在 `8b588a24cd653dac77dd68f6777e0db367981f19`。
---
# v0.16.0-preview(历史版本)

View File

@@ -1,5 +1,8 @@
# GoOmni 成熟度计划
当前发布基线为 v0.19.0-preview:普通接口通过 `.api` → DTO → Logic 开发;独立类型需要转换时可选 copyx。旧 JSON mapping 生成已移除,见 [迁移说明](COPY.md)。当前能力、使用入口和限制以 [README](README.md) 与 [发布记录](RELEASE.md) 为准。下文保留阶段性建设记录。
当前进展:`v0.14.0-preview` 已完成契约驱动的业务开发能力、通用 Runtime/配置和官方 MySQL 适配器,并通过全量 race、实际生成服务及空缓存远程验收。新项目/升级以 [README](README.md) 和 [开发指南](DEVELOPMENT.md) 为准;下方为历史阶段记录,完整本轮验收见 [报告](reports/businessDevelopmentAcceptance.md)。具体 eTax/ePay 业务迁移与业务回归仍由接入项目完成。
## 1. 新项目启动与发布流程(已完成,v0.1.1)

View File

@@ -1,6 +1,8 @@
# go-omni CLI
v0.17 增加公共字段组合、可选结构映射、自动中间件工厂和按需 Repository。配置和迁移见 [生成能力指南](../../GENERATION.md)。
v0.19 移除结构映射生成:`omni.gen.json` 只保留 Repository 和中间件选项。普通接口直接使用生成 DTO;需要类型转换时用普通 Go 或可选 [copyx](../../COPY.md)。旧 `mappings` / `mappingFiles` 必须按迁移步骤清理,CLI 不会静默删除仍在调用的旧生成文件。
公共字段组合、中间件工厂和按需 Repository 继续支持,见 [生成能力指南](../../GENERATION.md)。
v0.16 增加基础类型/动态 JSON 数组返回、OpenAPI 公共引用和 workspace Go Catalog 统一导出。见 [接入与筛选规则](../../DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。
@@ -12,7 +14,7 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目
```sh
# 仓库读取权限与 PATH 配置见根目录 README
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.20.0-preview
go-omni new --module example.com/hello ./hello
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
@@ -47,14 +49,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
## 版本与迁移
CLI 与框架依赖固定使用 v0.17.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
CLI 与框架依赖固定使用 v0.20.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
```sh
# 在 go-omni 源码仓库中执行
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
```
当前模板与依赖以 v0.17.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
当前模板与依赖以 v0.20.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
## 数据库迁移命令
@@ -67,3 +69,7 @@ go-omni migrate --dir migrations --apply
```
SQL 命名示例 `0001_createUsers.up.sql`;预览只显示版本和 SHA-256,不连接数据库。失败保留 dirty 状态,不自动 down/force。细节见 [扩展模块](../../ADVANCED.md)。
### v0.20 配置入口
新生成 `internal/config/loadGen.go` 调用统一加载,纳入生成检查;项目 config.go 只维护字段、默认值和校验,loader.go 用于注册额外格式。JSON/YAML/XML 内置。旧手写 Load 会阻止生成,请先按 [迁移步骤](../../CONFIG.md#v019-及更早项目迁移) 整理,不会自动覆盖业务代码。

View File

@@ -0,0 +1,113 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestGeneratedConfigurationConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "app")
api := filepath.Join(t.TempDir(), "service.api")
writeFile(t, api, "service configuration {\n@handler ping\nget /ping\n}\n")
o := Options{API: api, Dir: dir, Module: "example.com/configuration", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
main := readFile(t, filepath.Join(dir, "main.go"))
// The business only changes fields/defaults/validation, not the loading flow.
writeFile(t, filepath.Join(dir, "internal/config/config.go"), `package config
import("git.yuntuo.hk/k2safe/go-omni/config";"git.yuntuo.hk/k2safe/go-omni/logx")
type Config struct { Server config.Config `+"`json:\"server\"`"+`;Limit int `+"`json:\"limit\" envDefault:\"10\"`"+` }
func Default()Config{ c:=config.Default();c.Log.File="./var/log/app.log";c.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:10};return Config{Server:c} }
func(c Config)Validate()error{if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)};if c.Limit<0{return config.Invalid("limit","must not be negative")};return nil}
`)
writeFile(t, filepath.Join(dir, "internal/config/loader.go"), `package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
func NewLoader()(*omniConfig.Loader,error){return omniConfig.NewLoader(omniConfig.WithDecoder("custom",omniConfig.JSONDecoder{}))}
`)
svc := filepath.Join(dir, "internal/svc/serviceContext.go")
writeFile(t, svc, strings.Replace(readFile(t, svc), "return nil", "panic(\"dependencies must not initialize on invalid config\")", 1))
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/configuration/consumer_test.go"))
runConsumerGo(t, dir, "mod", "tidy")
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
runConsumerGo(t, dir, "build", ".")
o.API = filepath.Join(dir, "api/service.api")
before := snapshot(t, dir)
for _, mode := range []string{"dry", "generate", "check"} {
o.DryRun = mode == "dry"
o.Check = mode == "check"
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("business files overwritten")
}
if main != readFile(t, filepath.Join(dir, "main.go")) {
t.Fatal("main changed")
}
load := filepath.Join(dir, "internal/config/loadGen.go")
original := readFile(t, load)
writeFile(t, load, original+"\n// drift\n")
if err := generateManagedTestProject(t, o); err == nil {
t.Fatal("loadGen drift accepted")
}
writeFile(t, load, original)
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
data, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(data))
}
var dependency struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(data, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
}
func TestConfigLoadMigrationGuard(t *testing.T) {
dir := filepath.Join(t.TempDir(), "app")
o := Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}
if _, err := New(o); err != nil {
t.Fatal(err)
}
o.API = filepath.Join(dir, "api/service.api")
matches, err := filepath.Glob(filepath.Join(dir, "api/*.api"))
if err != nil || len(matches) != 1 {
t.Fatal(matches, err)
}
o.API = matches[0]
writeFile(t, filepath.Join(dir, "internal/config/legacy.go"), "package config\nfunc Load(path string)(Config,error){return Default(),nil}\n")
before := snapshot(t, dir)
for _, mode := range []string{"dry", "generate", "check"} {
o.DryRun = mode == "dry"
o.Check = mode == "check"
_, err := Generate(o)
if err == nil || !strings.Contains(err.Error(), "CONFIG.md") {
t.Fatal("missing migration diagnostic", err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("migration guard changed files")
}
}
}

View File

@@ -0,0 +1,118 @@
package apigen
import (
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func runConsumerGo(t *testing.T, dir string, args ...string) {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", args...)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("go %v: %v\n%s", args, err, out)
}
}
func TestCopyxThreeEndpointConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "app")
o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
for path, fixture := range map[string]string{
"internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go", "internal/convert/copy.go": "copy.go",
"internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go",
} {
writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture))
}
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1)
source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1)
writeFile(t, svcFile, source)
configFile := filepath.Join(dir, "internal/config/config.go")
writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1))
for _, group := range []string{"Enterprise", "Tenant", "Admin"} {
text := fmt.Sprintf(`package %s
import("context";"example.com/composition/internal/types";"example.com/composition/internal/convert";"example.com/composition/internal/domain")
func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){
input,err:=convert.Employee(*req);if err!=nil{return nil,err}
result,err:=convert.Response(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil
}
`, strings.ToLower(group), group)
writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text)
}
runConsumerGo(t, dir, "mod", "tidy")
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
o.API = filepath.Join(dir, "api/service.api")
before := snapshot(t, dir)
o.DryRun = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
o.DryRun = false
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("generation changed business files")
}
if _, err := os.Stat(filepath.Join(dir, "internal/mapping")); !os.IsNotExist(err) {
t.Fatal("mapping directory generated")
}
// Adding a common field requires one contract edit and one independent
// business field. Three endpoint converters need no new rules.
contract := readFile(t, o.API)
contract = strings.Replace(contract, "type SalaryFields {", "type SalaryFields {\n Note string `json:\"note,optional\"`", 1)
writeFile(t, o.API, contract)
domainPath := filepath.Join(dir, "internal/domain/employee.go")
writeFile(t, domainPath, strings.Replace(readFile(t, domainPath), "type Employee struct {", "type Employee struct {\n Note string", 1))
o.Check = false
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
runConsumerGo(t, dir, "build", ".")
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
data, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(data))
}
var dependency struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(data, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a fixed remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
}

View File

@@ -19,7 +19,7 @@ import (
)
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
const minimumVersion = "v0.17.0-preview"
const minimumVersion = "v0.20.0-preview"
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
type Options struct {
@@ -106,6 +106,9 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
if err = checkRuntimeExtension(directory); err != nil {
return Output{}, err
}
if err = checkConfigLoadExtension(directory); err != nil {
return Output{}, err
}
for _, r := range c.Spec.Routes {
if !r.Manual {
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
@@ -152,7 +155,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
if !filepath.IsAbs(path) {
path = filepath.Join(directory, path)
}
if source, err := os.ReadFile(filepath.Join(path, "config", "env.go")); err == nil && strings.Contains(string(source), "func ServerEnvAliases(") {
if source, err := os.ReadFile(filepath.Join(path, "config", "loader.go")); err == nil && strings.Contains(string(source), "func NewLoader(") {
localCurrent = true
}
}
@@ -252,17 +255,14 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
}
}
}
if err := renderMappings(o, generated, seeds); err != nil {
return Output{}, err
}
currentConfig, stillExists, err := readGeneratorConfig(directory)
if err != nil {
return Output{}, err
}
beforeConfig, _ := json.Marshal(genConfig)
afterConfig, _ := json.Marshal(currentConfig)
if hasGenConfig != stillExists || string(beforeConfig) != string(afterConfig) {
return Output{}, errors.New("omni.gen.json changed during generation; retry")
if hasGenConfig != stillExists || genConfig.inputDigest != currentConfig.inputDigest || string(beforeConfig) != string(afterConfig) {
return Output{}, errors.New("generation configuration changed during generation; retry")
}
out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld)
expectedVersion := minimumVersion

View File

@@ -1,23 +1,25 @@
package apigen
import (
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strings"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
)
const generatorConfigFile = "omni.gen.json"
// GeneratorConfig is developer-owned and checked on every generation. Mapping
// rules never change the public API or infer a persistence model.
// GeneratorConfig contains developer-owned scaffolding options only.
type GeneratorConfig struct {
Repository string `json:"repository,omitempty"`
Middleware string `json:"middleware,omitempty"`
Mappings []Mapping `json:"mappings,omitempty"`
Repository string `json:"repository,omitempty"`
Middleware string `json:"middleware,omitempty"`
inputDigest string
}
func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
@@ -35,7 +37,7 @@ func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
return cfg, false, err
}
if previous.GeneratorConfig {
return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options and mappings before regenerating")
return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options before regenerating")
}
cfg.Middleware = "handlers"
} else if !errors.Is(e, os.ErrNotExist) {
@@ -49,21 +51,60 @@ func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
return cfg, false, errors.New("omni.gen.json must be a regular file of at most 1 MiB")
}
raw, err := os.ReadFile(filepath.Join(root, generatorConfigFile))
raw, err := readGeneratorConfigFile(root, generatorConfigFile)
if err != nil {
return cfg, false, err
}
var keys map[string]json.RawMessage
if err = apierrors.DecodeCatalogJSON(raw, &keys); err != nil {
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
}
for _, key := range []string{"mappings", "mappingFiles"} {
if _, exists := keys[key]; exists {
return cfg, false, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go, remove mappings/mappingFiles from omni.gen.json, then remove internal/mapping/mappingsGen.go; see COPY.md")
}
}
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
}
digest := sha256.Sum256(raw)
cfg.inputDigest = fmt.Sprintf("%x", digest)
if cfg.Repository != "none" && cfg.Repository != "group" {
return cfg, false, errors.New("omni.gen.json: repository must be none or group")
}
if cfg.Middleware != "factories" && cfg.Middleware != "handlers" {
return cfg, false, errors.New("omni.gen.json: middleware must be factories or handlers")
}
if len(cfg.Mappings) > 128 {
return cfg, false, errors.New("omni.gen.json: at most 128 mappings supported")
}
return cfg, true, nil
}
// Read through a bounded reader as well as checking stat: a growing file cannot
// make configuration loading unbounded. safePath also rejects symlink escapes.
func readGeneratorConfigFile(root, name string) ([]byte, error) {
if err := safePath(root, name); err != nil {
return nil, err
}
f, err := os.Open(filepath.Join(root, filepath.FromSlash(name)))
if err != nil {
return nil, fmt.Errorf("%s: %w", name, err)
}
defer f.Close()
stat, err := f.Stat()
if err != nil {
return nil, err
}
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
return nil, fmt.Errorf("%s must be a regular file of at most 1 MiB", name)
}
raw, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
if err != nil {
return nil, err
}
if len(raw) > 1<<20 {
return nil, fmt.Errorf("%s exceeds 1 MiB", name)
}
if trimmed := strings.TrimSpace(string(raw)); !strings.HasPrefix(trimmed, "{") {
return nil, fmt.Errorf("%s must contain a JSON object", name)
}
return raw, nil
}

View File

@@ -0,0 +1,96 @@
package apigen
import (
"encoding/json"
"io"
"os"
"path/filepath"
"strings"
"testing"
)
func TestWorkspaceRejectsRemovedMappingWithoutWrites(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"one", "two"} {
if _, err := New(Options{Dir: filepath.Join(root, name), Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
}
firstAPI := filepath.Join(root, "one/api/service.api")
writeFile(t, firstAPI, strings.Replace(readFile(t, firstAPI), "Greet a visitor", "Changed description", 1))
writeFile(t, filepath.Join(root, "two", generatorConfigFile), `{"mappingFiles":["../../private.json"]}`)
workspace := filepath.Join(root, "omni.workspace.json")
writeFile(t, workspace, `{"services":[{"dir":"one"},{"dir":"two"}]}`)
before := snapshot(t, root)
for _, operation := range []string{"generate", "check"} {
if err := Workspace(workspace, operation, io.Discard); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
t.Fatal(operation, err)
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("workspace migration failure wrote files")
}
}
}
func TestRemovedMappingConfiguration(t *testing.T) {
dir := filepath.Join(t.TempDir(), "project")
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
for _, raw := range []string{`{"mappings":[]}`, `{"mappingFiles":[]}`, `{"mappings":null}`, `{"mappingFiles":["../../private.json"]}`} {
writeFile(t, filepath.Join(dir, generatorConfigFile), raw)
before := snapshot(t, dir)
for _, o := range []Options{{Dir: dir}, {Dir: dir, Check: true}, {Dir: dir, DryRun: true}} {
o.API = filepath.Join(dir, "api/service.api")
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("rejected config wrote project")
}
}
}
}
func TestOldMappingFileRequiresExplicitMigration(t *testing.T) {
dir := filepath.Join(t.TempDir(), "project")
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
name := "internal/mapping/mappingsGen.go"
body := generatedHeader + "package mapping\n"
writeFile(t, filepath.Join(dir, name), body)
var m manifest
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
t.Fatal(err)
}
m.Generated[name] = digest([]byte(body))
raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
before := snapshot(t, dir)
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
for _, mode := range []string{"generate", "check", "dry-run"} {
o.Check = mode == "check"
o.DryRun = mode == "dry-run"
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("legacy file deleted or project changed")
}
}
if err := os.Remove(filepath.Join(dir, name)); err != nil {
t.Fatal(err)
}
o.Check = false
o.DryRun = false
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
o.Check = true
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
if strings.Contains(readFile(t, filepath.Join(dir, manifestPath)), name) {
t.Fatal("old ownership retained")
}
}

View File

@@ -176,6 +176,9 @@ func generateManagedTestProject(t *testing.T, o Options) error {
if o.Check {
args = append(args, "--check")
}
if o.DryRun {
args = append(args, "--dry-run")
}
cmd := exec.CommandContext(ctx, cli, args...)
cmd.Dir = filepath.Dir(o.Dir)
if data, err := cmd.CombinedOutput(); err != nil {

View File

@@ -1,433 +0,0 @@
package apigen
import (
"fmt"
"go/ast"
"go/format"
goparser "go/parser"
gotoken "go/token"
"go/types"
"sort"
"strconv"
"strings"
"golang.org/x/mod/module"
)
// GoTypeRef names an existing, exported, non-generic Go struct.
type GoTypeRef struct {
Package string `json:"package"`
Type string `json:"type"`
}
type GoFunctionRef struct {
Package string `json:"package"`
Name string `json:"name"`
}
type MappingField struct {
Source string `json:"source"`
Convert *GoFunctionRef `json:"convert,omitempty"`
Mapping string `json:"mapping,omitempty"`
Elements bool `json:"elements,omitempty"`
}
type Mapping struct {
Name string `json:"name"`
Source GoTypeRef `json:"source"`
Target GoTypeRef `json:"target"`
Fields map[string]MappingField `json:"fields,omitempty"`
IgnoreSource map[string]string `json:"ignoreSource,omitempty"`
IgnoreTarget map[string]string `json:"ignoreTarget,omitempty"`
}
type mappingField struct {
path, name string
typ types.Type
}
type mappingTypes struct{ source, target types.Type }
type mappingRenderer struct {
importer types.Importer
imports map[string]string
definitions map[string]Mapping
types map[string]mappingTypes
}
func renderMappings(o Options, generated, seeds map[string][]byte) error {
if len(o.generation.Mappings) == 0 {
return nil
}
refs, err := validateMappings(o.generation.Mappings, o.Module)
if err != nil {
return err
}
importer, close, err := loadMappingTypes(o, refs, generated, seeds)
if err != nil {
return err
}
defer close()
r := mappingRenderer{importer: importer, imports: map[string]string{}, definitions: map[string]Mapping{}, types: map[string]mappingTypes{}}
for _, m := range o.generation.Mappings {
source, err := r.lookup(m.Source)
if err != nil {
return err
}
target, err := r.lookup(m.Target)
if err != nil {
return err
}
r.definitions[m.Name] = m
r.types[m.Name] = mappingTypes{source, target}
}
names := make([]string, 0, len(r.definitions))
for name := range r.definitions {
names = append(names, name)
}
sort.Strings(names)
var body strings.Builder
for _, name := range names {
source, err := r.render(r.definitions[name])
if err != nil {
return fmt.Errorf("mapping %s: %w", name, err)
}
body.WriteString(source)
}
imports := make([]string, 0, len(r.imports))
for path := range r.imports {
imports = append(imports, path)
}
sort.Strings(imports)
text := generatedHeader + "package mapping\nimport(\n"
for _, path := range imports {
text += r.imports[path] + " " + strconv.Quote(path) + "\n"
}
text += ")\n" + body.String()
code, err := format.Source([]byte(text))
if err != nil {
return fmt.Errorf("format mappings: %w", err)
}
// The generated package is type-checked before it is written. No application
// code (including init or converters) is executed during generation.
fset := gotoken.NewFileSet()
file, err := goparser.ParseFile(fset, "mappingsGen.go", code, 0)
if err != nil {
return err
}
check := types.Config{Importer: importer}
if _, err = check.Check(o.Module+"/internal/mapping", fset, []*ast.File{file}, nil); err != nil {
return fmt.Errorf("generated mapping type check: %w", err)
}
generated["internal/mapping/mappingsGen.go"] = code
return nil
}
func validateMappings(defs []Mapping, project string) ([]string, error) {
refs := map[string]bool{}
names := map[string]bool{}
validPackage := func(path string) bool {
if module.CheckImportPath(path) != nil || path == project+"/internal/mapping" {
return false
}
// go/types does not enforce the compiler's internal import visibility.
if strings.HasPrefix(path, "internal/") {
return false
}
if i := strings.LastIndex(path, "/internal/"); i >= 0 {
return strings.HasPrefix(project+"/internal/mapping", path[:i]+"/")
}
return true
}
for _, m := range defs {
if !gotoken.IsIdentifier(m.Name) || !gotoken.IsExported(m.Name) || names[m.Name] {
return nil, fmt.Errorf("mapping name %q must be unique and exported", m.Name)
}
names[m.Name] = true
for _, ref := range []GoTypeRef{m.Source, m.Target} {
if !validPackage(ref.Package) || !gotoken.IsIdentifier(ref.Type) || !gotoken.IsExported(ref.Type) {
return nil, fmt.Errorf("mapping %s: invalid Go type reference", m.Name)
}
refs[ref.Package] = true
}
if len(m.Fields) > 4096 || len(m.IgnoreSource) > 4096 || len(m.IgnoreTarget) > 4096 {
return nil, fmt.Errorf("mapping %s: field rule limit exceeded", m.Name)
}
for target, f := range m.Fields {
if target == "" || f.Source == "" {
return nil, fmt.Errorf("mapping %s: explicit field rule requires target and source paths", m.Name)
}
if f.Convert != nil {
if f.Mapping != "" || !validPackage(f.Convert.Package) || !gotoken.IsIdentifier(f.Convert.Name) || !gotoken.IsExported(f.Convert.Name) {
return nil, fmt.Errorf("mapping %s.%s: invalid or conflicting converter", m.Name, target)
}
refs[f.Convert.Package] = true
}
}
}
visiting, done := map[string]bool{}, map[string]bool{}
byName := map[string]Mapping{}
for _, m := range defs {
byName[m.Name] = m
}
var visit func(string) error
visit = func(name string) error {
if done[name] {
return nil
}
if visiting[name] {
return fmt.Errorf("mapping cycle at %s", name)
}
m, ok := byName[name]
if !ok {
return fmt.Errorf("unknown mapping %s", name)
}
visiting[name] = true
for _, f := range m.Fields {
if f.Mapping != "" {
if err := visit(f.Mapping); err != nil {
return err
}
}
}
delete(visiting, name)
done[name] = true
return nil
}
for _, m := range defs {
if err := visit(m.Name); err != nil {
return nil, err
}
}
out := make([]string, 0, len(refs))
for path := range refs {
out = append(out, path)
}
sort.Strings(out)
return out, nil
}
func (r *mappingRenderer) qualifier(p *types.Package) string {
if alias, ok := r.imports[p.Path()]; ok {
return alias
}
alias := fmt.Sprintf("pkg%d", len(r.imports))
r.imports[p.Path()] = alias
return alias
}
func (r *mappingRenderer) typeName(t types.Type) string { return types.TypeString(t, r.qualifier) }
func (r *mappingRenderer) lookup(ref GoTypeRef) (types.Type, error) {
pkg, err := r.importer.Import(ref.Package)
if err != nil {
return nil, fmt.Errorf("mapping type %s.%s: %w", ref.Package, ref.Type, err)
}
obj, ok := pkg.Scope().Lookup(ref.Type).(*types.TypeName)
if !ok {
return nil, fmt.Errorf("mapping type %s.%s not found", ref.Package, ref.Type)
}
typ := obj.Type()
if n, ok := typ.(*types.Named); ok && n.TypeParams() != nil && n.TypeParams().Len() > 0 {
return nil, fmt.Errorf("mapping root %s must be a non-generic struct", ref.Type)
}
if _, ok := typ.Underlying().(*types.Struct); !ok {
return nil, fmt.Errorf("mapping root %s must be a struct", ref.Type)
}
return typ, nil
}
func mappingFields(typ types.Type) (map[string]mappingField, error) {
out := map[string]mappingField{}
leaves := map[string]string{}
var walk func(types.Type, string, int) error
walk = func(typ types.Type, prefix string, depth int) error {
if depth > 32 {
return fmt.Errorf("embedded field nesting exceeds 32")
}
st, ok := typ.Underlying().(*types.Struct)
if !ok {
return fmt.Errorf("embedded field %s must be a value struct", prefix)
}
for i := 0; i < st.NumFields(); i++ {
f := st.Field(i)
path := prefix + f.Name()
if !f.Exported() {
return fmt.Errorf("unexported field %s cannot be mapped; use an explicit converter on the containing value", path)
}
if f.Embedded() {
if err := walk(f.Type(), path+".", depth+1); err != nil {
return err
}
continue
}
if previous, exists := leaves[f.Name()]; exists {
return fmt.Errorf("ambiguous embedded fields %s and %s", previous, path)
}
leaves[f.Name()] = path
out[path] = mappingField{path, f.Name(), f.Type()}
if len(out) > 4096 {
return fmt.Errorf("mapping field limit exceeded")
}
}
return nil
}
if err := walk(typ, "", 0); err != nil {
return nil, err
}
return out, nil
}
func (r *mappingRenderer) render(m Mapping) (string, error) {
mt := r.types[m.Name]
source, err := mappingFields(mt.source)
if err != nil {
return "", err
}
target, err := mappingFields(mt.target)
if err != nil {
return "", err
}
for key, reason := range m.IgnoreSource {
if _, ok := source[key]; !ok || strings.TrimSpace(reason) == "" {
return "", fmt.Errorf("ignoreSource %s requires an existing field and reason", key)
}
}
for key, reason := range m.IgnoreTarget {
if _, ok := target[key]; !ok || strings.TrimSpace(reason) == "" {
return "", fmt.Errorf("ignoreTarget %s requires an existing field and reason", key)
}
}
for key := range m.Fields {
if _, ok := target[key]; !ok {
return "", fmt.Errorf("unknown target field %s", key)
}
if _, ignored := m.IgnoreTarget[key]; ignored {
return "", fmt.Errorf("target %s is both mapped and ignored", key)
}
}
names := make([]string, 0, len(target))
for name := range target {
names = append(names, name)
}
sort.Strings(names)
used := map[string]bool{}
text := fmt.Sprintf("// %s maps one value. Pointer/map/slice assignments retain Go reference semantics.\nfunc %s(source %s)(%s,error){\nvar result %s\n", m.Name, m.Name, r.typeName(mt.source), r.typeName(mt.target), r.typeName(mt.target))
for _, path := range names {
dst := target[path]
if reason, ignored := m.IgnoreTarget[path]; ignored {
text += "// Target " + path + " is intentionally left at zero; see omni.gen.json.\n"
_ = reason
continue
}
rule, explicit := m.Fields[path]
if !explicit {
for key, f := range source {
if f.name == dst.name {
rule.Source = key
break
}
}
}
src, ok := source[rule.Source]
if !ok {
return "", fmt.Errorf("target %s has no source; add a field rule or ignoreTarget reason", path)
}
if _, ignored := m.IgnoreSource[src.path]; ignored {
return "", fmt.Errorf("source %s is both mapped and ignored", src.path)
}
used[src.path] = true
from, to := src.typ, dst.typ
srcExpr, dstExpr := "source."+src.path, "result."+path
if rule.Elements {
switch d := to.Underlying().(type) {
case *types.Slice:
s, ok := from.Underlying().(*types.Slice)
if !ok {
return "", fmt.Errorf("%s: elements requires slice to slice or equal-length array to array", path)
}
text += fmt.Sprintf("if %s!=nil {\n%s=make(%s,len(%s))\nfor index:=range %s {\n", srcExpr, dstExpr, r.typeName(to), srcExpr, srcExpr)
from, to = s.Elem(), d.Elem()
case *types.Array:
s, ok := from.Underlying().(*types.Array)
if !ok || s.Len() != d.Len() {
return "", fmt.Errorf("%s: array lengths must match", path)
}
text += fmt.Sprintf("for index:=range %s {\n", srcExpr)
from, to = s.Elem(), d.Elem()
default:
return "", fmt.Errorf("%s: elements requires a slice or array", path)
}
srcExpr += "[index]"
dstExpr += "[index]"
}
expr, fallible, err := r.expression(rule, from, to, srcExpr)
if err != nil {
return "", fmt.Errorf("%s -> %s: %w", src.path, path, err)
}
if fallible {
fmtPkg, err := r.importer.Import("fmt")
if err != nil {
return "", err
}
alias := r.qualifier(fmtPkg)
message := m.Name + "." + path + ": %w"
args := "err"
if rule.Elements {
message = m.Name + "." + path + "[%d]: %w"
args = "index,err"
}
text += fmt.Sprintf("{ value,err:=%s\nif err!=nil{var zero %s;return zero,%s.Errorf(%q,%s)}\n%s=value\n}\n", expr, r.typeName(mt.target), alias, message, args, dstExpr)
} else {
text += dstExpr + "=" + expr + "\n"
}
if rule.Elements {
text += "}\n"
if _, ok := dst.typ.Underlying().(*types.Slice); ok {
text += "}\n"
}
}
}
keys := make([]string, 0, len(source))
for key := range source {
keys = append(keys, key)
}
sort.Strings(keys)
for _, key := range keys {
if !used[key] {
if _, ignored := m.IgnoreSource[key]; !ignored {
return "", fmt.Errorf("unmapped source field %s; map it or provide an ignoreSource reason", key)
}
}
}
text += "return result,nil\n}\n"
return text, nil
}
func (r *mappingRenderer) expression(rule MappingField, from, to types.Type, source string) (string, bool, error) {
if rule.Mapping != "" {
m := r.types[rule.Mapping]
if !types.Identical(from, m.source) || !types.Identical(to, m.target) {
return "", false, fmt.Errorf("nested mapping %s has incompatible types", rule.Mapping)
}
return rule.Mapping + "(" + source + ")", true, nil
}
if rule.Convert != nil {
pkg, err := r.importer.Import(rule.Convert.Package)
if err != nil {
return "", false, err
}
fn, ok := pkg.Scope().Lookup(rule.Convert.Name).(*types.Func)
if !ok {
return "", false, fmt.Errorf("converter function %s not found", rule.Convert.Name)
}
sig := fn.Type().(*types.Signature)
if sig.Variadic() || (sig.TypeParams() != nil && sig.TypeParams().Len() > 0) || sig.Params().Len() != 1 || sig.Results().Len() < 1 || sig.Results().Len() > 2 {
return "", false, fmt.Errorf("converter must have signature func(Source) Target or func(Source) (Target,error)")
}
if !types.Identical(sig.Params().At(0).Type(), from) || !types.Identical(sig.Results().At(0).Type(), to) {
return "", false, fmt.Errorf("converter signature does not match field types")
}
fallible := sig.Results().Len() == 2
if fallible && !types.Identical(sig.Results().At(1).Type(), types.Universe.Lookup("error").Type()) {
return "", false, fmt.Errorf("converter second result must be error")
}
return r.qualifier(pkg) + "." + rule.Convert.Name + "(" + source + ")", fallible, nil
}
if !types.Identical(from, to) {
return "", false, fmt.Errorf("types %s and %s differ; provide an explicit converter", types.TypeString(from, nil), types.TypeString(to, nil))
}
return source, false, nil
}

View File

@@ -1,175 +0,0 @@
package apigen
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"go/importer"
gotoken "go/token"
"go/types"
"io"
"os"
"os/exec"
"path/filepath"
"sort"
"strings"
"time"
"golang.org/x/mod/modfile"
)
// loadMappingTypes uses the Go compiler's export data, including the pending
// DTOs via an overlay. Module files and sum updates remain in a temporary dir.
// It compiles only explicitly selected packages and their dependencies; never
// runs project code. Build tags/GOOS follow the caller; GOWORK is disabled.
func loadMappingTypes(o Options, refs []string, generated, seeds map[string][]byte) (types.Importer, func(), error) {
// go list resolves its working directory (notably /var -> /private/var on
// macOS). Overlay keys must name that same physical path or stale DTOs win.
o.Dir = canonical(o.Dir)
temp, err := os.MkdirTemp("", "go-omni-mapping-*")
if err != nil {
return nil, nil, err
}
cleanup := func() { _ = os.RemoveAll(temp) }
fail := func(err error) (types.Importer, func(), error) { cleanup(); return nil, nil, err }
raw, err := os.ReadFile(filepath.Join(o.Dir, "go.mod"))
if err != nil {
return fail(errors.New("mapping generation requires an existing Go module; scaffold first and resolve dependencies"))
}
mf, err := modfile.Parse("go.mod", raw, nil)
if err != nil {
return fail(err)
}
replacements := append([]*modfile.Replace(nil), mf.Replace...)
for _, r := range replacements {
if r.New.Version == "" && !filepath.IsAbs(r.New.Path) {
if err = mf.AddReplace(r.Old.Path, r.Old.Version, filepath.Join(o.Dir, r.New.Path), ""); err != nil {
return fail(err)
}
}
}
raw, err = mf.Format()
if err != nil {
return fail(err)
}
modPath := filepath.Join(temp, "go.mod")
if err = os.WriteFile(modPath, raw, 0600); err != nil {
return fail(err)
}
if sums, e := os.ReadFile(filepath.Join(o.Dir, "go.sum")); e == nil {
if err = os.WriteFile(filepath.Join(temp, "go.sum"), sums, 0600); err != nil {
return fail(err)
}
} else if !errors.Is(e, os.ErrNotExist) {
return fail(e)
}
overlay := map[string]string{}
// Hide stale owned Go files, which may otherwise redeclare a renamed DTO.
var previous manifest
if data, e := os.ReadFile(filepath.Join(o.Dir, manifestPath)); e == nil {
if err = json.Unmarshal(data, &previous); err != nil {
return fail(err)
}
for name := range previous.Generated {
if strings.HasSuffix(name, ".go") {
if !ownedPath(name) {
return fail(fmt.Errorf("invalid owned path %s", name))
}
if err = safePath(o.Dir, name); err != nil {
return fail(err)
}
if _, ok := generated[name]; !ok {
overlay[filepath.Join(o.Dir, filepath.FromSlash(name))] = ""
}
}
}
} else if !errors.Is(e, os.ErrNotExist) {
return fail(e)
}
for i, files := range []map[string][]byte{generated, seeds} {
names := make([]string, 0, len(files))
for name := range files {
if strings.HasSuffix(name, ".go") {
names = append(names, name)
}
}
sort.Strings(names)
for _, name := range names {
if err = safePath(o.Dir, name); err != nil {
return fail(err)
}
path := filepath.Join(o.Dir, filepath.FromSlash(name))
if i == 1 {
if _, e := os.Stat(path); e == nil {
continue
} else if !errors.Is(e, os.ErrNotExist) {
return fail(e)
}
}
backing := filepath.Join(temp, fmt.Sprintf("overlay%d.go", len(overlay)))
if err = os.WriteFile(backing, files[name], 0600); err != nil {
return fail(err)
}
overlay[path] = backing
}
}
raw, err = json.Marshal(map[string]any{"Replace": overlay})
if err != nil {
return fail(err)
}
overlayPath := filepath.Join(temp, "overlay.json")
if err = os.WriteFile(overlayPath, raw, 0600); err != nil {
return fail(err)
}
args := []string{"list", "-deps", "-export", "-json", "-mod=readonly", "-modfile", modPath, "-overlay", overlayPath}
// fmt is used for fallible conversions even if neither struct imports it.
args = append(args, "fmt")
args = append(args, refs...)
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", args...)
cmd.Dir = o.Dir
cmd.Env = append(os.Environ(), "GOWORK=off")
cmd.WaitDelay = 2 * time.Second
stdout := &limitedCatalogBuffer{limit: 32 << 20}
stderr := &limitedCatalogBuffer{limit: 64 << 10}
cmd.Stdout = stdout
cmd.Stderr = stderr
if err = cmd.Run(); err != nil {
if ctx.Err() != nil {
return fail(fmt.Errorf("mapping type loading: %w", ctx.Err()))
}
// Go download errors may contain authenticated remote URLs. Keep arbitrary
// command output out of diagnostics; exact mapping errors follow type loading.
return fail(fmt.Errorf("mapping type loading failed (%v); run go list -deps -export for the configured source/target/converter packages; check dependencies, build tags and import cycles", err))
}
exports := map[string]string{}
dec := json.NewDecoder(bytes.NewReader(stdout.Bytes()))
for {
var p struct {
ImportPath, Export string
Error *struct{ Err string }
}
if err = dec.Decode(&p); err == io.EOF {
break
} else if err != nil {
return fail(err)
}
if p.Error != nil {
return fail(fmt.Errorf("mapping package %s could not be compiled", p.ImportPath))
}
if p.Export != "" {
exports[p.ImportPath] = p.Export
}
}
imp := importer.ForCompiler(gotoken.NewFileSet(), "gc", func(path string) (io.ReadCloser, error) {
file, ok := exports[path]
if !ok {
return nil, fmt.Errorf("no export data for mapping dependency %s", path)
}
return os.Open(file)
})
return imp, cleanup, nil
}

View File

@@ -1,266 +0,0 @@
package apigen
import (
"context"
"encoding/json"
"fmt"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func compositionConfig() GeneratorConfig {
pkg := "example.com/composition/internal/"
cfg := GeneratorConfig{Repository: "none", Middleware: "factories"}
for _, name := range []string{"Enterprise", "Tenant", "Admin"} {
fields := map[string]MappingField{
"EmployeeName": {Source: "Name", Convert: &GoFunctionRef{pkg + "convert", "Name"}},
"Salary": {Source: "Salary", Convert: &GoFunctionRef{pkg + "convert", "Money"}},
"Enabled": {Source: "Enabled", Convert: &GoFunctionRef{pkg + "convert", "Bool"}},
}
m := Mapping{Name: "From" + name, Source: GoTypeRef{pkg + "types", name + "Request"}, Target: GoTypeRef{pkg + "domain", "Employee"}, Fields: fields}
if name != "Enterprise" {
m.IgnoreSource = map[string]string{"CompanyID": "Authorized separately by Logic; never copy path scope into body"}
}
cfg.Mappings = append(cfg.Mappings, m)
}
cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToResponse", Source: GoTypeRef{pkg + "domain", "Result"}, Target: GoTypeRef{pkg + "types", "EmployeeResponse"}, Fields: map[string]MappingField{"ID": {Source: "ID", Convert: &GoFunctionRef{pkg + "convert", "ID"}}}})
cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToRow", Source: GoTypeRef{pkg + "domain", "Row"}, Target: GoTypeRef{pkg + "domain", "RowDTO"}})
cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToBatch", Source: GoTypeRef{pkg + "domain", "Batch"}, Target: GoTypeRef{pkg + "domain", "BatchDTO"}, Fields: map[string]MappingField{
"Rows": {Source: "Rows", Mapping: "ToRow", Elements: true}, "Fixed": {Source: "Fixed", Mapping: "ToRow", Elements: true},
}})
return cfg
}
func writeGeneratorConfig(t *testing.T, dir string, cfg GeneratorConfig) {
t.Helper()
raw, err := json.MarshalIndent(cfg, "", " ")
if err != nil {
t.Fatal(err)
}
writeFile(t, filepath.Join(dir, generatorConfigFile), string(raw)+"\n")
}
func runCompositionGo(t *testing.T, dir string, args ...string) {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", args...)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("go %v: %v\n%s", args, err, out)
}
}
func TestCompositionMappingConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "app")
o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
fixtures := map[string]string{
"internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go",
"internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go",
}
for path, fixture := range fixtures {
writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture))
}
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1)
source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1)
writeFile(t, svcFile, source)
configFile := filepath.Join(dir, "internal/config/config.go")
writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1))
// Resolve dependencies before opting into compile-time mapping. The first
// generation creates DTOs; the second sees current DTOs through an overlay.
// Logic does not yet import the generated mapping package.
testSource := readFile(t, filepath.Join(dir, "consumer_test.go"))
if err := os.Remove(filepath.Join(dir, "consumer_test.go")); err != nil {
t.Fatal(err)
}
runCompositionGo(t, dir, "mod", "tidy")
cfg := compositionConfig()
writeGeneratorConfig(t, dir, cfg)
o.API = filepath.Join(dir, "api/service.api")
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
writeFile(t, filepath.Join(dir, "consumer_test.go"), testSource)
for _, group := range []string{"Enterprise", "Tenant", "Admin"} {
text := fmt.Sprintf(`package %s
import("context";"example.com/composition/internal/types";"example.com/composition/internal/mapping";"example.com/composition/internal/domain")
func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){
input,err:=mapping.From%s(*req);if err!=nil{return nil,err}
result,err:=mapping.ToResponse(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil
}
`, strings.ToLower(group), group, group)
writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text)
}
before := snapshot(t, dir)
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("regeneration changed project")
}
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("check changed project")
}
runCompositionGo(t, dir, "test", "-race", "-count=1", "./...")
runCompositionGo(t, dir, "build", ".")
generated := readFile(t, filepath.Join(dir, "internal/mapping/mappingsGen.go"))
if strings.Contains(generated, "encoding/json") || strings.Contains(generated, "reflect") || !strings.Contains(generated, "result.EmployeeName = value") {
t.Fatal("mapping does not emit typed assignments")
}
// Adding a source or target field is detected before any files are written.
for _, tc := range []struct{ name, path, before, after, want string }{
{"source", o.API, "Name *string", "NewField *string `json:\"newField,optional\"`\n Name *string", "unmapped source field NewField"},
{"target", filepath.Join(dir, "internal/domain/employee.go"), "EmployeeName string", "Unmapped string\n EmployeeName string", "target Unmapped has no source"},
} {
t.Run(tc.name, func(t *testing.T) {
original := readFile(t, tc.path)
writeFile(t, tc.path, strings.Replace(original, tc.before, tc.after, 1))
defer writeFile(t, tc.path, original)
snap := snapshot(t, dir)
if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatal(err)
}
if !sameFiles(snap, snapshot(t, dir)) {
t.Fatal("failed check wrote files")
}
o.Check = false
defer func() { o.Check = true }()
if err := generateManagedTestProject(t, o); err == nil {
t.Fatal("generate accepted missing mapping")
}
if !sameFiles(snap, snapshot(t, dir)) {
t.Fatal("failed generate wrote files")
}
})
}
for _, tc := range []struct {
name string
edit func(*GeneratorConfig)
want string
}{
{"pointerPolicy", func(c *GeneratorConfig) { delete(c.Mappings[0].Fields, "Salary") }, "provide an explicit converter"},
{"unknownSource", func(c *GeneratorConfig) { c.Mappings[0].Fields["Salary"] = MappingField{Source: "Missing"} }, "has no source"},
{"wrongConverter", func(c *GeneratorConfig) {
f := c.Mappings[0].Fields["Salary"]
f.Convert.Name = "ID"
c.Mappings[0].Fields["Salary"] = f
}, "signature does not match"},
{"unhandledScope", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource = nil }, "unmapped source field CompanyID"},
{"badIgnore", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource["Missing"] = "stale rule" }, "ignoreSource Missing"},
{"arrayLength", func(c *GeneratorConfig) {
f := c.Mappings[5].Fields["Fixed"]
f.Source = "Rows"
c.Mappings[5].Fields["Fixed"] = f
}, "array lengths must match"},
} {
t.Run(tc.name, func(t *testing.T) {
changed := compositionConfig()
tc.edit(&changed)
writeGeneratorConfig(t, dir, changed)
defer writeGeneratorConfig(t, dir, cfg)
snap := snapshot(t, dir)
if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatal(err)
}
if !sameFiles(snap, snapshot(t, dir)) {
t.Fatal("invalid rule wrote files")
}
})
}
// All diagnostics exercise current DTOs, not stale exported generated types.
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
// Exercise the repository switch with the same actual CLI as the mapper.
if _, err := os.Stat(filepath.Join(dir, "internal/repository")); !os.IsNotExist(err) {
t.Fatalf("default repository skeleton: %v", err)
}
o.Check = false
cfg.Repository = "group"
writeGeneratorConfig(t, dir, cfg)
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
repo := filepath.Join(dir, "internal/repository/enterprise/repository.go")
writeFile(t, repo, readFile(t, repo)+"\n// Project-owned persistence.\n")
cfg.Repository = "none"
writeGeneratorConfig(t, dir, cfg)
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
if !strings.Contains(readFile(t, repo), "Project-owned persistence.") {
t.Fatal("disabling skeleton overwrote business code")
}
o.Check = true
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
data, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(data))
}
var dependency struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(data, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a fixed remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
}
func TestMappingConfigurationRejection(t *testing.T) {
cases := []struct {
name string
edit func(*GeneratorConfig)
want string
}{
{"cycle", func(c *GeneratorConfig) {
c.Mappings[4].Fields = map[string]MappingField{"Value": {Source: "Value", Mapping: "ToRow"}}
}, "cycle"},
{"unknown", func(c *GeneratorConfig) {
c.Mappings[5].Fields["Rows"] = MappingField{Source: "Rows", Mapping: "Missing", Elements: true}
}, "unknown mapping"},
{"conflict", func(c *GeneratorConfig) {
f := c.Mappings[0].Fields["Salary"]
f.Mapping = "ToRow"
c.Mappings[0].Fields["Salary"] = f
}, "conflicting converter"},
{"internal", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.net/private/internal/types" }, "invalid Go type"},
{"outputCycle", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.com/composition/internal/mapping" }, "invalid Go type"},
{"duplicates", func(c *GeneratorConfig) { c.Mappings[1].Name = c.Mappings[0].Name }, "unique and exported"},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
cfg := compositionConfig()
tc.edit(&cfg)
if _, err := validateMappings(cfg.Mappings, "example.com/composition"); err == nil || !strings.Contains(err.Error(), tc.want) {
t.Fatal(err)
}
})
}
}

View File

@@ -56,10 +56,13 @@ factories for new projects. Named middleware factories run after svc.Init;
implement Dependencies accessors on ServiceContext and New<Name>(deps) once.
Unimplemented/nil factories fail startup and clean registered dependencies.
Existing projects retain middleware: handlers until explicitly migrated.
Optional mappings generate typed assignments in internal/mapping/mappingsGen.go;
configure source/target types and explicit conversion rules after resolving module
dependencies. Missing fields fail generation and --check. See GENERATION.md in
the framework repository for composition, null/reference semantics and migration.
Ordinary Logic uses the generated Request/Response types directly. Where an
independent business type is needed, use ordinary Go or the optional copyx package.
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use one pipeline. See the framework CONFIG.md.
There are no mappings/mappingFiles settings or generated mapping files in v0.19.
See COPY.md in the framework repository for copy policies and migration steps,
and GENERATION.md for field composition and middleware factories.
## Run
@@ -145,8 +148,7 @@ internal/handler/<group>/ # generated binding and Logic adapters
internal/logic/<group>/ # developer-owned business operations
internal/repository/<group>/ # optional; repository: group creates editable seeds
internal/middleware/ # typed dependency interface, factories and registry
omni.gen.json # developer-owned generator options and mappings
internal/mapping/*Gen.go # optional compile-checked struct mappings
omni.gen.json # developer-owned repository/middleware options
docs/openapi.json # generated OpenAPI 3.0.3
.goOmni/apiManifest.json # ownership and content hashes
`+"```"+`

View File

@@ -18,7 +18,6 @@ import (
"example.com/composition/internal/convert"
"example.com/composition/internal/domain"
"example.com/composition/internal/handler"
"example.com/composition/internal/mapping"
"example.com/composition/internal/middleware"
"example.com/composition/internal/svc"
"example.com/composition/internal/types"
@@ -86,16 +85,16 @@ func TestHTTPComposition(t *testing.T) {
}
}
}
func TestFullFormAndMappingSemantics(t *testing.T) {
func TestFullFormAndCopySemantics(t *testing.T) {
name := "employee"
money := amount.Money("2.00")
yes := true
req := types.EnterpriseRequest{Name: &name, Salary: &money, Enabled: &yes, Labels: []string{"one"}}
first, e := mapping.FromEnterprise(req)
first, e := convert.Employee(req)
if e != nil {
t.Fatal(e)
}
second, e := mapping.FromEnterprise(types.EnterpriseRequest{})
second, e := convert.Employee(types.EnterpriseRequest{})
if e != nil {
t.Fatal(e)
}
@@ -108,26 +107,26 @@ func TestFullFormAndMappingSemantics(t *testing.T) {
t.Fatal("unexpected implicit deep copy")
}
req.Employer = &money
first, e = mapping.FromEnterprise(req)
first, e = convert.Employee(req)
if e != nil || first.Employer != req.Employer {
t.Fatal("nullable pointer changed", e)
}
bad := "reject"
failed, e := mapping.FromEnterprise(types.EnterpriseRequest{Name: &bad})
failed, e := convert.Employee(types.EnterpriseRequest{Name: &bad})
if !errors.Is(e, convert.ErrName) || failed.EmployeeName != "" || !strings.Contains(e.Error(), "EmployeeName") {
t.Fatal("converter cause/zero result lost", e)
}
empty, e := mapping.ToBatch(domain.Batch{})
empty, e := convert.ToBatch(domain.Batch{})
if e != nil || empty.Rows != nil {
t.Fatal("nil slice changed", e)
}
src := domain.Batch{Rows: []domain.Row{}, Fixed: [2]domain.Row{{Value: "a"}, {Value: "b"}}, Optional: &name}
out, e := mapping.ToBatch(src)
out, e := convert.ToBatch(src)
if e != nil || out.Rows == nil || len(out.Rows) != 0 || out.Fixed[1].Value != "b" || out.Optional != src.Optional {
t.Fatal(out, e)
}
src.Rows = []domain.Row{{Value: "one"}}
out, e = mapping.ToBatch(src)
out, e = convert.ToBatch(src)
if e != nil || out.Rows[0].Value != "one" {
t.Fatal(out, e)
}
@@ -140,7 +139,7 @@ func TestFullFormAndMappingSemantics(t *testing.T) {
wg.Add(1)
go func() {
defer wg.Done()
result, err := mapping.FromEnterprise(types.EnterpriseRequest{Name: &name, Salary: &money})
result, err := convert.Employee(types.EnterpriseRequest{Name: &name, Salary: &money})
if err != nil || result.EmployeeName != name || result.Salary != money {
t.Error(result, err)
}

View File

@@ -2,7 +2,6 @@ package convert
import (
"errors"
"example.com/composition/internal/amount"
"strconv"
)
@@ -19,11 +18,4 @@ func Name(value *string) (string, error) {
}
return *value, nil
}
func Money(value *amount.Money) amount.Money {
if value == nil {
return ""
}
return *value
}
func Bool(value *bool) bool { return value != nil && *value }
func ID(value int64) string { return strconv.FormatInt(value, 10) }

View File

@@ -0,0 +1,29 @@
package convert
import (
"example.com/composition/internal/domain"
"example.com/composition/internal/types"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
// The three endpoints reuse one ordinary Go helper and their existing scalar.
// There is no field list or package/type name JSON to keep in sync.
func Employee[S any](source S) (domain.Employee, error) {
var target domain.Employee
err := copyx.Copy(&target, source,
copyx.WithFieldNames[S, domain.Employee](map[string]string{"Name": "EmployeeName"}),
copyx.WithNilPolicy(copyx.NilZero),
copyx.WithConverter(Name),
)
return target, err
}
func Response(source domain.Result) (types.EmployeeResponse, error) {
var target types.EmployeeResponse
err := copyx.Copy(&target, source, copyx.WithConverter(func(v int64) (string, error) { return ID(v), nil }))
return target, err
}
func ToBatch(source domain.Batch) (domain.BatchDTO, error) {
var target domain.BatchDTO
err := copyx.Copy(&target, source)
return target, err
}

View File

@@ -14,7 +14,7 @@ type Result struct {
ID int64
}
// Distinct container item types exercise explicitly selected element mappings.
// Distinct container item types exercise explicit Go collection copying.
type Row struct{ Value string }
type RowDTO struct{ Value string }
type Batch struct {

View File

@@ -0,0 +1,59 @@
package main
import (
"context"
"errors"
"example.com/configuration/internal/config"
omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
"os"
"path/filepath"
"strings"
"testing"
)
func TestGeneratedConfiguration(t *testing.T) {
for ext, source := range map[string]string{
"json": `{"limit":7}`, "yaml": "limit: 7\n", "xml": "<config><limit>7</limit></config>", "custom": `{"limit":7}`,
} {
t.Run(ext, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "app."+ext)
if err := os.WriteFile(path, []byte(source), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.Load(path)
if err != nil {
t.Fatal(err)
}
if cfg.Limit != 7 {
t.Fatal(cfg.Limit)
}
t.Setenv("GO_OMNI_LIMIT", "0")
cfg, err = config.Load(path)
if err != nil || cfg.Limit != 0 {
t.Fatal(cfg.Limit, err)
}
})
}
t.Setenv("GO_OMNI_LOG_OUTPUT", "stderr")
cfg, err := config.Load("")
if err != nil {
t.Fatal(err)
}
if cfg.Limit != 10 || cfg.Server.Log.File != "" || cfg.Server.Log.Rotation.Enabled {
t.Fatal("defaults/log normalization", cfg, err)
}
path := filepath.Join(t.TempDir(), "bad.yaml")
os.WriteFile(path, []byte("limit: -1\n"), 0600)
// Validation must run before svc.Initialize; the project initializer panics
// below if reached, so this also detects accidentally starting dependencies.
err = run(context.Background(), path)
var diagnostic *omniConfig.Error
if !errors.As(err, &diagnostic) || diagnostic.Field != "limit" || diagnostic.File != path || diagnostic.Line != 1 {
t.Fatal(err)
}
t.Setenv("GO_OMNI_LIMIT", "private-not-a-number")
_, err = config.Load(path)
if !errors.As(err, &diagnostic) || diagnostic.Env != "GO_OMNI_LIMIT" || strings.Contains(err.Error(), "private-not-a-number") {
t.Fatal(err)
}
}

View File

@@ -30,6 +30,39 @@ type UpgradePlan struct {
Module, Entry string
}
// Handwritten old loaders must be migrated explicitly, never silently replaced
// or left competing with the generated entry point.
func checkConfigLoadExtension(root string) error {
dir := filepath.Join(root, "internal/config")
entries, err := os.ReadDir(dir)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return err
}
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") || name == "loadGen.go" {
continue
}
rel := "internal/config/" + name
if err := safePath(root, rel); err != nil {
return err
}
f, err := goparser.ParseFile(gotoken.NewFileSet(), filepath.Join(root, rel), nil, 0)
if err != nil {
return fmt.Errorf("parse %s: %w", rel, err)
}
for _, decl := range f.Decls {
if fn, ok := decl.(*ast.FuncDecl); ok && fn.Recv == nil && fn.Name.Name == "Load" {
return fmt.Errorf("%s: migrate the project-owned Load function before generating internal/config/loadGen.go; retain Config/Default/Validate and move decoder options to NewLoader; see CONFIG.md", rel)
}
}
}
return nil
}
// PlanUpgrade compares infrastructure against the exact previous release seeds.
// A custom file is never classified as safe to replace. Logic, middleware,
// options and repositories outside the changed templates are left untouched.

View File

@@ -117,6 +117,17 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
var changes []change
// Check ALL previous outputs, even stale ones, before writing any new file.
for name, hash := range old.Generated {
// Migration guard only; old mappings are never regenerated or silently
// deleted. Once the project removes them, forget the stale ownership.
if name == "internal/mapping/mappingsGen.go" {
if err = safePath(root, name); err != nil {
return result, err
}
if _, e := os.Lstat(filepath.Join(root, name)); !errors.Is(e, os.ErrNotExist) {
return result, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go and remove internal/mapping/mappingsGen.go before generating; see COPY.md")
}
continue
}
if !ownedPath(name) {
return result, fmt.Errorf("api: invalid owned path %q", name)
}
@@ -273,7 +284,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
}
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
func ownedPath(name string) bool {
if name == "internal/mapping/mappingsGen.go" {
if name == "internal/config/loadGen.go" {
return true
}
if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) {

View File

@@ -8,7 +8,7 @@ import (
)
func TestNewWithRemoteFrameworkVersion(t *testing.T) {
for _, flag := range [][]string{{"--framework-version", "v0.17.0-preview"}, {"--framework-version=v0.17.0-preview"}} {
for _, flag := range [][]string{{"--framework-version", "v0.20.0-preview"}, {"--framework-version=v0.20.0-preview"}} {
project := filepath.Join(t.TempDir(), "billing")
args := append([]string{"--module", "example.com/billing"}, flag...)
if err := runNew(append(args, project)); err != nil {
@@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) {
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(content), "go-omni v0.17.0-preview") || strings.Contains(string(content), "replace ") {
if !strings.Contains(string(content), "go-omni v0.20.0-preview") || strings.Contains(string(content), "replace ") {
t.Fatalf("CLI did not pin remote dependency: %s", content)
}
}

View File

@@ -21,6 +21,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
"internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl",
"internal/svc/options.go": "templates/api-options.go.tmpl",
"internal/config/config.go": "templates/api-config.go.tmpl",
"internal/config/loadGen.go": "templates/api-load.go.tmpl",
"internal/config/loader.go": "templates/api-loader.go.tmpl",
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
"internal/svc/serviceContext.go": "templates/api-context.go.tmpl",
}
@@ -31,6 +33,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl"
}
if legacy {
delete(files, "internal/config/loadGen.go")
delete(files, "internal/config/loader.go")
delete(files, "internal/svc/runtimeGen.go")
delete(files, "internal/svc/databaseOptions.go")
files["internal/svc/close.go"] = "templates/api-close.go.tmpl"

View File

@@ -1,10 +1,9 @@
package settings
import (
"os"
{{if and (not .API) (ne .Storage "none")}}"strings"{{end}}
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
{{if eq .LogOutput "file"}}"git.yuntuo.hk/k2safe/go-omni/logx"{{end}}
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
{{if ne .Storage "none"}}"errors"
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
@@ -35,22 +34,10 @@ func Default() Config {
}{{end}}}
}
// Load applies defaults -> file -> typed environment -> validation.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string)(Config,error){
cfg:=Default();var err error
cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err}
if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}}
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_",Empty:"ignore",Aliases:config.ServerEnvAliases("GO_OMNI_","server")});if err!=nil{return Config{},err}
output,outputSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_OUTPUT");if !outputSet {output=os.Getenv("GO_OMNI_LOG_OUTPUT")}
file,fileSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_FILE");if !fileSet {file=os.Getenv("GO_OMNI_LOG_FILE")}
if output!="" && file=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
return cfg,cfg.Validate()
}
func(c Config)Validate()error{
if err:=c.Server.Validate();err!=nil{return err}
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return err}{{end}}
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return err}{{end}}
if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)}
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return config.AtField("database",err)}{{end}}
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return config.AtField("storage",err)}{{end}}
return nil
}
{{if ne .Storage "none"}}

View File

@@ -0,0 +1,13 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// Load is maintained by the generator. Extend Config/Default/Validate in
// config.go, or NewLoader in loader.go; do not duplicate the loading pipeline.
func Load(path string) (Config, error) {
loader,err:=NewLoader()
if err!=nil{return Config{},err}
return omniConfig.LoadProject(path,Default(),omniConfig.EnvOptions{
Prefix:"GO_OMNI_",Empty:"ignore",Aliases:omniConfig.ServerEnvAliases("GO_OMNI_","server"),
},loader)
}

View File

@@ -0,0 +1,7 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// NewLoader is project-owned. JSON/YAML/XML work without changes. Register
// additional format decoders here only when the project needs them.
func NewLoader() (*omniConfig.Loader,error) { return omniConfig.NewLoader() }

View File

@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Source builds can carry an unpublished VCS version (including +dirty).
// Keep this fallback in sync with the next release; installed CLIs use their
// downloaded module version, which also has a module checksum.
const defaultFrameworkVersion = "v0.17.0-preview"
const defaultFrameworkVersion = "v0.20.0-preview"
// This target is not a claim that the release already exists. Source users
// select a local framework checkout until the module templates are published.

320
config/bind.go Normal file
View File

@@ -0,0 +1,320 @@
package config
import (
"encoding/json"
"fmt"
"reflect"
"sort"
"strings"
)
var jsonUnmarshalerType = reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
type documentBinder struct {
doc Document
nodes int
}
func (b *documentBinder) step(path string, depth int) error {
b.nodes++
if depth > maxConfigDepth || b.nodes > maxConfigNodes {
return b.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
}
return nil
}
func bindDocument[T any](doc Document, defaults T) (T, error) {
var zero T
base := reflect.ValueOf(&defaults).Elem()
if base.Kind() != reflect.Struct && (base.Kind() != reflect.Pointer || base.Type().Elem().Kind() != reflect.Struct) {
return zero, &Error{Reason: "project configuration must be a struct"}
}
result, err := cloneConfiguration(base, 0)
if err != nil {
return zero, &Error{Reason: "cannot clone configuration defaults", cause: err}
}
b := documentBinder{doc: doc}
if err = b.bind(doc.Value, result, "", 0, false); err != nil {
return zero, err
}
return result.Interface().(T), nil
}
func (b *documentBinder) decode(v any, dst reflect.Value, path string) (err error) {
defer func() {
if recover() != nil {
err = b.doc.diagnostic(path, "custom value decoder panicked", nil)
}
}()
raw, err := json.Marshal(v)
if err != nil {
return b.doc.diagnostic(path, "unsupported configuration value", err)
}
if err = json.Unmarshal(raw, dst.Addr().Interface()); err != nil {
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
}
return nil
}
func (b *documentBinder) bind(v any, dst reflect.Value, path string, depth int, quoted bool) error {
if err := b.step(path, depth); err != nil {
return err
}
if v == nil {
return b.decode(nil, dst, path)
} // Preserve encoding/json null semantics.
if dst.Kind() == reflect.Pointer {
if dst.IsNil() {
dst.Set(reflect.New(dst.Type().Elem()))
}
return b.bind(v, dst.Elem(), path, depth+1, quoted)
}
if dst.Type() == durationType {
var raw string
switch x := v.(type) {
case string:
raw = x
case Text:
raw = string(x)
default:
return b.decode(v, dst, path)
}
if err := parseEnvValue(dst, strings.TrimSpace(raw)); err != nil {
return b.doc.diagnostic(path, "expected a duration such as 5s", err)
}
return nil
}
if dst.CanAddr() && dst.Addr().Type().Implements(jsonUnmarshalerType) {
normalized, err := b.dynamic(v, path, depth+1)
if err != nil {
return err
}
return b.decode(normalized, dst, path)
}
if dst.CanAddr() && dst.Addr().Type().Implements(textType) {
if text, ok := v.(Text); ok {
v = string(text)
}
return b.decode(v, dst, path)
}
if text, ok := v.(Text); ok {
if dst.Kind() == reflect.String || dst.Kind() == reflect.Interface || (dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8) {
return b.decode(string(text), dst, path)
}
switch dst.Kind() {
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64:
if err := parseEnvValue(dst, strings.TrimSpace(string(text))); err != nil {
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
}
return nil
default:
return b.doc.diagnostic(path, "expected an object or array, not scalar text", nil)
}
}
if quoted {
s, ok := v.(string)
if !ok {
return b.doc.diagnostic(path, "expected a JSON-encoded string", nil)
}
if err := json.Unmarshal([]byte(s), dst.Addr().Interface()); err != nil {
return b.doc.diagnostic(path, "invalid string-encoded value", err)
}
return nil
}
switch dst.Kind() {
case reflect.Struct:
m, ok := v.(map[string]any)
if !ok {
return b.doc.diagnostic(path, "expected an object", nil)
}
fields := configurationFields(dst.Type())
used := map[string]bool{}
for _, name := range sortedKeys(m) {
child := joinField(path, name)
f, ok := fields[name]
canonical := name
if !ok {
for n, field := range fields {
if strings.EqualFold(n, name) {
if ok {
return b.doc.diagnostic(child, "ambiguous field name", nil)
}
f, ok, canonical = field, true, n
}
}
}
if !ok {
return b.doc.diagnostic(child, "unknown field", nil)
}
if used[canonical] {
return b.doc.diagnostic(child, "duplicate field binding", nil)
}
used[canonical] = true
target := dst
for _, index := range f.Index {
if target.Kind() == reflect.Pointer {
if target.IsNil() {
if !target.CanSet() {
return b.doc.diagnostic(child, "field cannot be assigned", nil)
}
target.Set(reflect.New(target.Type().Elem()))
}
target = target.Elem()
}
target = target.Field(index)
}
if !target.CanSet() {
return b.doc.diagnostic(child, "field cannot be assigned", nil)
}
if err := b.bind(m[name], target, child, depth+1, strings.Contains(","+f.Tag.Get("json")+",", ",string,")); err != nil {
return err
}
}
return nil
case reflect.Map:
m, ok := v.(map[string]any)
if !ok || dst.Type().Key().Kind() != reflect.String {
return b.doc.diagnostic(path, "expected an object with string keys", nil)
}
if dst.IsNil() {
dst.Set(reflect.MakeMap(dst.Type()))
}
for _, key := range sortedKeys(m) {
value := reflect.New(dst.Type().Elem()).Elem()
if err := b.bind(m[key], value, joinField(path, key), depth+1, false); err != nil {
return err
}
k := reflect.New(dst.Type().Key()).Elem()
k.SetString(key)
dst.SetMapIndex(k, value)
}
return nil
case reflect.Slice, reflect.Array:
if _, ok := v.(string); ok && dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8 {
return b.decode(v, dst, path)
}
items, ok := v.([]any)
if !ok {
return b.doc.diagnostic(path, "expected an array", nil)
}
if dst.Kind() == reflect.Array && len(items) != dst.Len() {
return b.doc.diagnostic(path, "array length does not match its declared type", nil)
}
if dst.Kind() == reflect.Slice {
dst.Set(reflect.MakeSlice(dst.Type(), len(items), len(items)))
}
for i, value := range items {
if err := b.bind(value, dst.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1, false); err != nil {
return err
}
}
return nil
case reflect.Interface:
v, err := b.dynamic(v, path, depth+1)
if err != nil {
return err
}
if dst.Type().NumMethod() == 0 {
dst.Set(reflect.ValueOf(v))
return nil
}
return b.decode(v, dst, path)
default:
return b.decode(v, dst, path)
}
}
func sortedKeys(m map[string]any) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
}
func (b *documentBinder) dynamic(v any, path string, depth int) (any, error) {
if err := b.step(path, depth); err != nil {
return nil, err
}
switch x := v.(type) {
case Text:
return string(x), nil
case map[string]any:
out := make(map[string]any, len(x))
for _, k := range sortedKeys(x) {
c, err := b.dynamic(x[k], joinField(path, k), depth+1)
if err != nil {
return nil, err
}
out[k] = c
}
return out, nil
case []any:
out := make([]any, len(x))
for i, v := range x {
c, err := b.dynamic(v, fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return nil, err
}
out[i] = c
}
return out, nil
default:
return v, nil
}
}
// Match encoding/json's field promotion: the shallowest declaration wins;
// explicit tags win ties, and ambiguous fields remain unavailable.
func configurationFields(t reflect.Type) map[string]reflect.StructField {
type candidate struct {
field reflect.StructField
tagged bool
}
candidates := map[string][]candidate{}
var visit func(reflect.Type, []int, map[reflect.Type]bool)
visit = func(current reflect.Type, prefix []int, visiting map[reflect.Type]bool) {
if visiting[current] {
return
}
visiting[current] = true
defer delete(visiting, current)
for i := 0; i < current.NumField(); i++ {
f := current.Field(i)
name := strings.Split(f.Tag.Get("json"), ",")[0]
if name == "-" {
continue
}
child := f.Type
if child.Kind() == reflect.Pointer {
child = child.Elem()
}
if f.PkgPath != "" && (!f.Anonymous || child.Kind() != reflect.Struct) {
continue
}
f.Index = append(append([]int(nil), prefix...), i)
if name == "" && f.Anonymous && child.Kind() == reflect.Struct {
visit(child, f.Index, visiting)
continue
}
tagged := name != ""
if name == "" {
name = f.Name
}
candidates[name] = append(candidates[name], candidate{f, tagged})
}
}
visit(t, nil, map[reflect.Type]bool{})
fields := map[string]reflect.StructField{}
for name, list := range candidates {
sort.Slice(list, func(i, j int) bool {
if len(list[i].field.Index) != len(list[j].field.Index) {
return len(list[i].field.Index) < len(list[j].field.Index)
}
return list[i].tagged && !list[j].tagged
})
if len(list) > 1 && len(list[0].field.Index) == len(list[1].field.Index) && list[0].tagged == list[1].tagged {
continue
}
fields[name] = list[0].field
}
return fields
}

View File

@@ -62,19 +62,19 @@ func Default() Config {
// Validate checks configuration values before an application starts.
func (c Config) Validate() error {
if err := c.Lifecycle.Validate(); err != nil {
return err
return AtField("lifecycle", err)
}
if err := c.Request.Validate(); err != nil {
return err
return AtField("request", err)
}
if err := c.Log.Validate(); err != nil {
return err
return AtField("log", err)
}
if err := c.Endpoints.Validate(); err != nil {
return err
}
if err := c.Errors.Validate(); err != nil {
return err
return AtField("errors", err)
}
encoder := c.ResponseEncoder
if encoder == nil {
@@ -90,7 +90,7 @@ func (c Config) Validate() error {
return err
}
if err := c.Response.Validate(); err != nil {
return err
return AtField("response", err)
}
if c.SlowRequestThreshold < 0 {
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
@@ -99,13 +99,13 @@ func (c Config) Validate() error {
return &ConfigError{Field: "address", Message: "is required"}
}
if strings.TrimSpace(c.RequestIDHeader) == "" {
return &ConfigError{Field: "request_id_header", Message: "is required"}
return &ConfigError{Field: "requestIdHeader", Message: "is required"}
}
if c.ShutdownTimeout <= 0 {
return &ConfigError{Field: "shutdown_timeout", Message: "must be positive"}
return &ConfigError{Field: "shutdownTimeout", Message: "must be positive"}
}
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
return &ConfigError{Field: "trusted_proxies", Message: err.Error()}
return &Error{Field: "trustedProxies", Reason: "expected IP addresses or CIDRs", cause: err}
}
return c.ValidateTransport()
}

View File

@@ -112,7 +112,7 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s")
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com")
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trusted_proxies") {
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trustedProxies") {
t.Fatalf("expected invalid trusted proxy error, got %v", err)
}
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "")
@@ -123,7 +123,7 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
}
cfg = Default()
cfg.RequestIDHeader = " "
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "request_id_header") {
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "requestIdHeader") {
t.Fatalf("expected invalid request header error, got %v", err)
}
}

398
config/decoders.go Normal file
View File

@@ -0,0 +1,398 @@
package config
import (
"bytes"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"regexp"
"sort"
"strconv"
"strings"
"unicode/utf8"
"gopkg.in/yaml.v3"
)
type JSONDecoder struct{}
type YAMLDecoder struct{}
type XMLDecoder struct{}
type parseState struct {
doc Document
nodes int
}
func newParseState() *parseState { return &parseState{doc: Document{Positions: map[string]Position{}}} }
func (p *parseState) enter(path string, pos Position, depth int) error {
p.nodes++
p.doc.Positions[path] = pos
if depth > maxConfigDepth || p.nodes > maxConfigNodes {
return p.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
}
return nil
}
type sourcePositions struct{ starts []int }
func newSourcePositions(data []byte) sourcePositions {
p := sourcePositions{starts: []int{0}}
for i, c := range data {
if c == '\n' {
p.starts = append(p.starts, i+1)
}
}
return p
}
func (p sourcePositions) at(offset int64) Position {
if offset < 0 {
offset = 0
}
i := sort.Search(len(p.starts), func(i int) bool { return int64(p.starts[i]) > offset }) - 1
return Position{Line: i + 1, Column: int(offset) - p.starts[i] + 1}
}
var yamlLine = regexp.MustCompile(`line ([0-9]+)`)
func yamlSyntax(err error) *Error {
d := &Error{Reason: "invalid YAML syntax", cause: err}
if m := yamlLine.FindStringSubmatch(err.Error()); len(m) == 2 {
d.Line, _ = strconv.Atoi(m[1])
d.Column = 1
}
return d
}
func (JSONDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
positions := newSourcePositions(data)
d := json.NewDecoder(bytes.NewReader(data))
d.UseNumber()
var parse func(string, int) (any, error)
parse = func(path string, depth int) (any, error) {
offset := d.InputOffset()
for offset < int64(len(data)) && strings.ContainsRune(" \t\r\n:,", rune(data[offset])) {
offset++
}
if err := p.enter(path, positions.at(offset), depth); err != nil {
return nil, err
}
token, err := d.Token()
if err != nil {
pos := positions.at(d.InputOffset())
var syntax *json.SyntaxError
if errors.As(err, &syntax) {
pos = positions.at(syntax.Offset - 1)
}
return nil, &Error{Field: path, Line: pos.Line, Column: pos.Column, Reason: "invalid JSON syntax", cause: err}
}
if delim, ok := token.(json.Delim); ok {
switch delim {
case '{':
out := map[string]any{}
for d.More() {
k, e := d.Token()
if e != nil {
return nil, p.doc.diagnostic(path, "invalid object key", e)
}
key, ok := k.(string)
if !ok {
return nil, p.doc.diagnostic(path, "object key must be a string", nil)
}
child := joinField(path, key)
if _, exists := out[key]; exists {
return nil, p.doc.diagnostic(child, "duplicate field", nil)
}
out[key], e = parse(child, depth+1)
if e != nil {
return nil, e
}
}
if _, err = d.Token(); err != nil {
return nil, p.doc.diagnostic(path, "unterminated object", err)
}
return out, nil
case '[':
out := []any{}
for d.More() {
v, e := parse(fmt.Sprintf("%s[%d]", path, len(out)), depth+1)
if e != nil {
return nil, e
}
out = append(out, v)
}
if _, err = d.Token(); err != nil {
return nil, p.doc.diagnostic(path, "unterminated array", err)
}
return out, nil
default:
return nil, p.doc.diagnostic(path, "unexpected JSON delimiter", nil)
}
}
return token, nil
}
var err error
p.doc.Value, err = parse("", 0)
if err == nil {
if _, e := d.Token(); e != io.EOF {
err = p.doc.diagnostic("", "exactly one JSON document required", e)
}
}
return p.doc, err
}
func (YAMLDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
d := yaml.NewDecoder(bytes.NewReader(data))
var root yaml.Node
if err := d.Decode(&root); err != nil {
return p.doc, yamlSyntax(err)
}
var extra yaml.Node
if err := d.Decode(&extra); err != io.EOF {
return p.doc, &Error{Reason: "exactly one YAML document required", cause: err}
}
var walk func(*yaml.Node, string, int) (any, error)
walk = func(n *yaml.Node, path string, depth int) (any, error) {
if err := p.enter(path, Position{n.Line, n.Column}, depth); err != nil {
return nil, err
}
switch n.Kind {
case yaml.MappingNode:
out := map[string]any{}
for i := 0; i < len(n.Content); i += 2 {
k, v := n.Content[i], n.Content[i+1]
if k.Tag != "!!str" {
return nil, p.doc.diagnostic(path, "mapping keys must be strings; merge keys are not supported", nil)
}
child := joinField(path, k.Value)
if _, ok := out[k.Value]; ok {
return nil, &Error{Field: child, Line: k.Line, Column: k.Column, Reason: "duplicate field"}
}
x, err := walk(v, child, depth+1)
if err != nil {
return nil, err
}
out[k.Value] = x
}
return out, nil
case yaml.SequenceNode:
out := make([]any, len(n.Content))
for i, c := range n.Content {
var err error
out[i], err = walk(c, fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return nil, err
}
}
return out, nil
case yaml.AliasNode:
return nil, p.doc.diagnostic(path, "YAML aliases are not supported", nil)
case yaml.ScalarNode:
switch n.Tag {
case "!!str", "!!timestamp":
return n.Value, nil
case "!!null":
return nil, nil
case "!!bool", "!!int", "!!float":
var v any
if err := n.Decode(&v); err != nil {
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
}
raw, err := json.Marshal(v)
if err != nil {
return nil, p.doc.diagnostic(path, "number must be finite", err)
}
var out any
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
if err = dec.Decode(&out); err != nil {
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
}
return out, nil
default:
return nil, p.doc.diagnostic(path, "unsupported YAML tag", nil)
}
}
return nil, p.doc.diagnostic(path, "unsupported YAML node", nil)
}
if len(root.Content) != 1 {
return p.doc, &Error{Reason: "one nonempty YAML document required"}
}
var err error
p.doc.Value, err = walk(root.Content[0], "", 0)
return p.doc, err
}
// XML uses a config root, element field names, type="array" with item children,
// type="object" for empty objects, and type="null" for explicit null. Scalar
// text is bound using the destination type; optional number/bool/string types
// let dynamic any values preserve their intended JSON type.
func (XMLDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
d := xml.NewDecoder(bytes.NewReader(data))
var parse func(xml.StartElement, string, int, Position) (any, error)
parse = func(start xml.StartElement, path string, depth int, pos Position) (any, error) {
if err := p.enter(path, pos, depth); err != nil {
return nil, err
}
if start.Name.Space != "" {
return nil, p.doc.diagnostic(path, "XML namespaces are not supported", nil)
}
kind := ""
attrSeen := false
for _, a := range start.Attr {
if a.Name.Space != "" || a.Name.Local != "type" || attrSeen {
return nil, p.doc.diagnostic(path, "only one XML type attribute is supported", nil)
}
kind = a.Value
attrSeen = true
}
if depth == 0 && kind == "" {
kind = "object"
}
switch kind {
case "", "object", "array", "null", "string", "number", "bool":
default:
return nil, p.doc.diagnostic(path, "unsupported XML type", nil)
}
children := map[string]any{}
array := []any{}
var text strings.Builder
for {
line, column := d.InputPos()
token, err := d.Token()
if err != nil {
return nil, &Error{Field: path, Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
}
switch x := token.(type) {
case xml.StartElement:
if kind != "" && kind != "object" && kind != "array" {
return nil, p.doc.diagnostic(path, "scalar XML element cannot contain children", nil)
}
child := joinField(path, x.Name.Local)
if kind == "array" {
if x.Name.Local != "item" {
return nil, p.doc.diagnostic(path, "XML array children must be item elements", nil)
}
child = fmt.Sprintf("%s[%d]", path, len(array))
} else if _, ok := children[x.Name.Local]; ok {
return nil, p.doc.diagnostic(child, "duplicate field; use type=\"array\" for lists", nil)
}
v, e := parse(x, child, depth+1, Position{line, column})
if e != nil {
return nil, e
}
if kind == "array" {
array = append(array, v)
} else {
children[x.Name.Local] = v
}
case xml.CharData:
text.Write([]byte(x))
case xml.Comment:
case xml.EndElement:
raw := text.String()
if len(children) > 0 || kind == "array" || kind == "object" {
if strings.TrimSpace(raw) != "" {
return nil, p.doc.diagnostic(path, "mixed XML text and elements are not supported", nil)
}
if kind == "array" {
return array, nil
}
return children, nil
}
switch kind {
case "null":
if strings.TrimSpace(raw) != "" {
return nil, p.doc.diagnostic(path, "null element must be empty", nil)
}
return nil, nil
case "string":
return raw, nil
case "number":
var n any
dec := json.NewDecoder(strings.NewReader(strings.TrimSpace(raw)))
dec.UseNumber()
if err := dec.Decode(&n); err != nil {
return nil, p.doc.diagnostic(path, "invalid number", err)
}
if _, ok := n.(json.Number); !ok {
return nil, p.doc.diagnostic(path, "expected number", nil)
}
if _, e := dec.Token(); e != io.EOF {
return nil, p.doc.diagnostic(path, "invalid number", e)
}
return n, nil
case "bool":
b, e := strconv.ParseBool(strings.TrimSpace(raw))
if e != nil {
return nil, p.doc.diagnostic(path, "expected boolean", e)
}
return b, nil
}
return Text(raw), nil
default:
return nil, p.doc.diagnostic(path, "XML directives and processing instructions are not supported", nil)
}
}
}
found := false
prolog := false
for {
line, column := d.InputPos()
token, err := d.Token()
if err == io.EOF {
break
}
if err != nil {
return p.doc, &Error{Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
}
switch x := token.(type) {
case xml.StartElement:
if found || x.Name.Local != "config" {
return p.doc, &Error{Line: line, Column: column, Reason: "exactly one config root required"}
}
found = true
p.doc.Value, err = parse(x, "", 0, Position{line, column})
if err != nil {
return p.doc, err
}
case xml.CharData:
if strings.TrimSpace(string(x)) != "" {
return p.doc, &Error{Reason: "unexpected text outside XML root"}
}
case xml.Comment:
case xml.ProcInst:
if found || prolog || x.Target != "xml" {
return p.doc, &Error{Reason: "XML processing instructions are not supported"}
}
prolog = true
default:
return p.doc, &Error{Reason: "XML directives are not supported"}
}
}
if !found {
return p.doc, &Error{Reason: "config root required"}
}
return p.doc, nil
}

94
config/diagnostic.go Normal file
View File

@@ -0,0 +1,94 @@
package config
import (
"errors"
"fmt"
"strings"
)
// Error is a value-free diagnostic. Cause may contain private parser details;
// errors.Is/As remain available, but Error never formats Cause automatically.
type Error struct {
File, Field, Env, Reason string
Line, Column int
cause error
}
func (e *Error) Error() string {
where := "config"
if e.File != "" {
where = fmt.Sprintf("config %q", e.File)
}
if e.Line > 0 {
where += fmt.Sprintf(":%d:%d", e.Line, e.Column)
}
if e.Env != "" {
where += fmt.Sprintf(" env %q", e.Env)
}
if e.Field != "" {
where += fmt.Sprintf(" field %q", e.Field)
}
return where + ": " + e.Reason
}
func (e *Error) Unwrap() error { return e.cause }
// DecodeError is retained for callers of errors.As from earlier releases.
type DecodeError = Error
// Invalid reports a public, value-free validation reason for a configuration field.
func Invalid(field, reason string) error { return &ConfigError{Field: field, Message: reason} }
// AtField prefixes a nested validation error without losing its original cause.
// Unstructured validation errors are deliberately not echoed; use Invalid when
// a custom validator needs a public reason rather than a private error message.
func AtField(field string, err error) error {
if err == nil {
return nil
}
d := validationError(err)
d.Field = joinField(field, d.Field)
return d
}
func validationError(err error) *Error {
var d *Error
if errors.As(err, &d) {
out := *d
out.cause = err
return &out
}
var c *ConfigError
if errors.As(err, &c) {
return &Error{Field: c.Field, Reason: c.Message, cause: err}
}
return &Error{Reason: "validation failed", cause: err}
}
func joinField(parent, child string) string {
if parent == "" {
return child
}
if child == "" {
return parent
}
if strings.HasPrefix(child, "[") {
return parent + child
}
return parent + "." + child
}
// Position uses one-based source line/column numbers. Zero means unavailable.
type Position struct{ Line, Column int }
// Document is the common decoder result. Values use JSON-compatible data:
// string, bool, json.Number, nil, []any, map[string]any. Text is available for
// formats such as XML whose scalar tokens have no inherent JSON type.
type Document struct {
Value any
Positions map[string]Position
}
type Text string
func (d Document) diagnostic(field, reason string, cause error) *Error {
p := d.Positions[field]
return &Error{Field: field, Reason: reason, Line: p.Line, Column: p.Column, cause: cause}
}

View File

@@ -1,100 +1,16 @@
package config
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"os"
"path/filepath"
"reflect"
"strings"
"sync"
"time"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/logx"
"gopkg.in/yaml.v3"
)
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
// field names for both formats, rejects unknown fields and accepts durations
// such as "3s". Application-specific validation remains with the caller.
func DecodeFile[T any](path string, defaults T) (T, error) {
result, err := decodeFile(path, defaults)
if err != nil {
var zero T
return zero, &DecodeError{cause: err}
}
return result, nil
}
// DecodeError keeps input values out of public diagnostics while preserving
// errors.Is/As for explicitly handled underlying I/O and syntax errors.
type DecodeError struct{ cause error }
func (e *DecodeError) Error() string {
return "config: cannot decode file; check its path, format, field names and value types"
}
func (e *DecodeError) Unwrap() error { return e.cause }
func decodeFile[T any](path string, defaults T) (T, error) {
var zero T
f, err := os.Open(path)
if err != nil {
return zero, err
}
defer f.Close()
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
if err != nil {
return zero, err
}
if len(data) > 1<<20 {
return zero, errors.New("config: file exceeds 1 MiB")
}
var value any
switch strings.ToLower(filepath.Ext(path)) {
case ".yaml", ".yml":
dec := yaml.NewDecoder(bytes.NewReader(data))
if err := dec.Decode(&value); err != nil {
return zero, err
}
var extra any
if err := dec.Decode(&extra); err != io.EOF {
return zero, errors.New("config: one YAML document required")
}
case ".json":
if err := apierrors.DecodeCatalogJSON(data, &value); err != nil {
return zero, err
}
default:
return zero, errors.New("config: expected .json, .yaml or .yml")
}
if _, ok := value.(map[string]any); !ok {
return zero, errors.New("config: root must be an object")
}
value, err = durations(value, reflect.TypeOf(defaults))
if err != nil {
return zero, err
}
encoded, err := json.Marshal(value)
if err != nil {
return zero, err
}
// Clone configuration without serializing borrowed json:"-" capabilities.
var result T
original, err := cloneConfiguration(reflect.ValueOf(&defaults).Elem(), 0)
if err != nil {
return zero, err
}
reflect.ValueOf(&result).Elem().Set(original)
dec := json.NewDecoder(bytes.NewReader(encoded))
dec.DisallowUnknownFields()
if err = dec.Decode(&result); err != nil {
return zero, err
}
return result, nil
}
func durations(v any, t reflect.Type) (any, error) {
if t == nil {
return v, nil

175
config/loader.go Normal file
View File

@@ -0,0 +1,175 @@
package config
import (
"errors"
"io"
"os"
"path/filepath"
"reflect"
"strings"
)
const maxConfigBytes = 1 << 20
const maxConfigDepth = 64
const maxConfigNodes = 100000
// Decoder must not retain or mutate input and must support concurrent calls
// when its Loader is shared. It is trusted application code, not a sandbox.
type Decoder interface {
Decode([]byte) (Document, error)
}
type DecoderFunc func([]byte) (Document, error)
func (f DecoderFunc) Decode(data []byte) (Document, error) { return f(data) }
// Loader owns an immutable per-instance decoder registry. No global registration.
type Loader struct {
decoders map[string]Decoder
sealed bool
}
type LoaderOption func(*Loader) error
// WithDecoder adds a new extension; builtin/duplicate extensions cannot be replaced.
func WithDecoder(extension string, decoder Decoder) LoaderOption {
return func(l *Loader) error {
if l == nil || l.decoders == nil || l.sealed {
return errors.New("config: decoder registry cannot be changed after construction")
}
ext := strings.ToLower(strings.TrimPrefix(extension, "."))
if ext == "" || strings.ContainsAny(ext, "./\\ \t\r\n") {
return errors.New("config: invalid decoder extension")
}
if decoder == nil {
return errors.New("config: nil decoder")
}
v := reflect.ValueOf(decoder)
if (v.Kind() == reflect.Pointer || v.Kind() == reflect.Func || v.Kind() == reflect.Map || v.Kind() == reflect.Slice || v.Kind() == reflect.Interface) && v.IsNil() {
return errors.New("config: nil decoder")
}
if _, exists := l.decoders[ext]; exists {
return errors.New("config: decoder extension already registered")
}
l.decoders[ext] = decoder
return nil
}
}
func NewLoader(options ...LoaderOption) (*Loader, error) {
l := &Loader{decoders: map[string]Decoder{"json": JSONDecoder{}, "yaml": YAMLDecoder{}, "yml": YAMLDecoder{}, "xml": XMLDecoder{}}}
for _, o := range options {
if o == nil {
return nil, errors.New("config: nil loader option")
}
if err := o(l); err != nil {
return nil, err
}
}
l.sealed = true
return l, nil
}
var defaultLoader, _ = NewLoader()
func selectLoader(loaders []*Loader) (*Loader, error) {
if len(loaders) == 0 {
return defaultLoader, nil
}
if len(loaders) != 1 || loaders[0] == nil {
return nil, errors.New("config: expected one non-nil loader")
}
return loaders[0], nil
}
func (l *Loader) read(path string) (doc Document, err error) {
defer func() {
if err != nil {
var d *Error
if errors.As(err, &d) {
c := *d
c.File = path
err = &c
} else {
reason := "cannot read configuration file"
if errors.Is(err, os.ErrNotExist) {
reason = "configuration file does not exist"
} else if errors.Is(err, os.ErrPermission) {
reason = "permission denied reading configuration file"
}
err = &Error{File: path, Reason: reason, cause: err}
}
}
}()
decoder := l.decoders[strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")]
if decoder == nil {
return doc, &Error{Reason: "unsupported configuration format"}
}
info, e := os.Stat(path)
if e != nil {
return doc, e
}
if !info.Mode().IsRegular() {
return doc, &Error{Reason: "configuration must be a regular file"}
}
f, e := os.Open(path)
if e != nil {
return doc, e
}
defer f.Close()
info, e = f.Stat()
if e != nil {
return doc, e
}
if !info.Mode().IsRegular() {
return doc, &Error{Reason: "configuration must be a regular file"}
}
data, e := io.ReadAll(io.LimitReader(f, maxConfigBytes+1))
if e != nil {
return doc, e
}
if len(data) > maxConfigBytes {
return doc, &Error{Reason: "file exceeds 1 MiB"}
}
doc, err = decodeSafely(decoder, data)
if err != nil {
return doc, err
}
if _, ok := doc.Value.(map[string]any); !ok {
return doc, doc.diagnostic("", "root must be an object", nil)
}
return doc, nil
}
func decodeSafely(decoder Decoder, data []byte) (doc Document, err error) {
defer func() {
if recover() != nil {
err = &Error{Reason: "decoder panicked"}
}
}()
doc, err = decoder.Decode(data)
if err != nil {
var d *Error
if !errors.As(err, &d) {
err = &Error{Reason: "invalid configuration syntax", cause: err}
}
}
return
}
// DecodeFile overlays a bounded document onto independent defaults. With no
// loader argument it accepts JSON/YAML/XML. It never opens dependencies.
func DecodeFile[T any](path string, defaults T, loaders ...*Loader) (T, error) {
var zero T
l, err := selectLoader(loaders)
if err != nil {
return zero, err
}
doc, err := l.read(path)
if err != nil {
return zero, err
}
result, err := bindDocument(doc, defaults)
if err != nil {
d := validationError(err)
d.File = path
return zero, d
}
return result, nil
}

397
config/loader_test.go Normal file
View File

@@ -0,0 +1,397 @@
package config
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
"sync"
"testing"
"time"
)
type loaderExample struct {
Server Config `json:"server"`
Upload struct {
MaxBytes int64 `json:"maxBytes" envDefault:"1024"`
} `json:"upload"`
Values []int64 `json:"values"`
Optional *string `json:"optional"`
Enabled bool `json:"enabled"`
Dynamic any `json:"dynamic"`
}
func (c loaderExample) Validate() error {
if err := c.Server.Validate(); err != nil {
return AtField("server", err)
}
if c.Upload.MaxBytes < 0 {
return Invalid("upload.maxBytes", "must be nonnegative")
}
return nil
}
func noEnvironment(string) (string, bool) { return "", false }
func configFile(t *testing.T, ext, raw string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "service."+ext)
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
t.Fatal(err)
}
return path
}
func TestUnifiedFormatAndPrecedence(t *testing.T) {
formats := map[string]string{
"json": `{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":7},"values":[9007199254740993],"optional":null,"enabled":true,"dynamic":{"id":9007199254740993}}`,
"yaml": "server:\n http:\n readTimeout: 2s\nupload:\n maxBytes: 7\nvalues: [9007199254740993]\noptional: null\nenabled: true\ndynamic:\n id: 9007199254740993\n",
"xml": `<config><server><http><readTimeout>2s</readTimeout></http></server><upload><maxBytes>7</maxBytes></upload><values type="array"><item>9007199254740993</item></values><optional type="null"/><enabled>true</enabled><dynamic><id type="number">9007199254740993</id></dynamic></config>`,
}
var reference loaderExample
for _, ext := range []string{"json", "yaml", "xml"} {
t.Run(ext, func(t *testing.T) {
old := "old"
base := loaderExample{Server: Default(), Optional: &old}
cfg, err := LoadProject(configFile(t, ext, formats[ext]), base, EnvOptions{Prefix: "APP_", Lookup: func(key string) (string, bool) {
v, ok := map[string]string{"APP_UPLOAD_MAX_BYTES": "0", "APP_ENABLED": "false"}[key]
return v, ok
}})
if err != nil {
t.Fatal(err)
}
if cfg.Upload.MaxBytes != 0 || cfg.Enabled || cfg.Optional != nil || cfg.Values[0] != 9007199254740993 || cfg.Server.HTTP.ReadTimeout != 2*time.Second {
t.Fatal("values changed", cfg)
}
if cfg.Dynamic.(map[string]any)["id"] != json.Number("9007199254740993") {
t.Fatal("dynamic integer precision lost")
}
if ext == "json" {
reference = cfg
} else if !reflect.DeepEqual(reference, cfg) {
t.Fatal("format-dependent semantics")
}
if base.Optional == nil || *base.Optional != "old" {
t.Fatal("defaults mutated")
}
})
}
cfg, err := LoadProject("", loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
if err != nil || cfg.Upload.MaxBytes != 1024 {
t.Fatal(cfg, err)
}
}
func TestConfigDiagnosticsAndCauses(t *testing.T) {
for _, tc := range []struct {
ext, body, field string
line bool
}{
{"json", "{\n\"upload\": {\"maxBytes\":\"private-value\"}}", "upload.maxBytes", true},
{"yaml", "server:\n http:\n readTimeout: private-value", "server.http.readTimeout", true},
{"xml", "<config>\n<upload><maxBytes>private-value</maxBytes></upload></config>", "upload.maxBytes", true},
{"json", `{"upload":{"maxBytes":-1}}`, "upload.maxBytes", true},
{"yaml", "server:\n http:\n readTimeout: -1s", "server.http.readTimeout", true},
{"json", `{"upload":{"typo":3}}`, "upload.typo", true},
} {
t.Run(tc.ext+tc.field, func(t *testing.T) {
path := configFile(t, tc.ext, tc.body)
_, err := LoadProject(path, loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
var d *Error
if !errors.As(err, &d) || d.File != path || d.Field != tc.field || d.Reason == "" || tc.line && d.Line < 1 {
t.Fatalf("missing location: %#v %v", d, err)
}
if strings.Contains(err.Error(), "private-value") {
t.Fatal("secret leaked", err)
}
})
}
path := filepath.Join(t.TempDir(), "missing.json")
_, err := LoadProject(path, loaderExample{}, EnvOptions{})
if !errors.Is(err, os.ErrNotExist) {
t.Fatal("I/O cause discarded", err)
}
_, err = DecodeFile(configFile(t, "json", `{"count":"private-value"}`), struct {
Count int `json:"count"`
}{})
var typed *json.UnmarshalTypeError
if !errors.As(err, &typed) {
t.Fatal("type cause discarded", err)
}
_, err = LoadProject(configFile(t, "yaml", "upload:\n maxBytes: 7"), loaderExample{Server: Default()}, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { return "-1", k == "APP_UPLOAD_MAX_BYTES" }})
var d *Error
if !errors.As(err, &d) || d.Env != "APP_UPLOAD_MAX_BYTES" || d.Field != "upload.maxBytes" || d.File != "" || d.Line != 0 {
t.Fatal("environment validation wrongly blamed file", err)
}
}
func TestMalformedFormatsAndLimits(t *testing.T) {
for ext, bodies := range map[string][]string{
"json": {`{"enabled":true,"enabled":false}`, `{} {}`, `{"upload":`, `{"enabled":true,"Enabled":false}`, strings.Repeat(`{"a":`, 70) + `0` + strings.Repeat(`}`, 70)},
"yaml": {"enabled: true\nenabled: false", "---\n{}\n---\n{}", "a: &x [1]\nb: *x", "a:\n <<: {}", "upload: [\n"},
"xml": {`<!DOCTYPE config><config/>`, `<config>&private-value;</config>`, `<config/><config/>`, `<config xmlns="x"/>`, `<config type="" type=""/>`, `<config><enabled>true</enabled><enabled>false</enabled></config>`, `<config>text<enabled>true</enabled></config>`, `<config><values type="array"><bad>1</bad></values></config>`},
} {
for i, raw := range bodies {
t.Run(fmt.Sprintf("%s%d", ext, i), func(t *testing.T) {
_, err := LoadProject(configFile(t, ext, raw), loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
if err == nil || strings.Contains(err.Error(), "private-value") {
t.Fatal(err)
}
})
}
}
if _, err := DecodeFile(configFile(t, "json", strings.Repeat(" ", maxConfigBytes+1)), loaderExample{}); err == nil {
t.Fatal("unbounded file")
}
if _, err := DecodeFile(configFile(t, "json", string([]byte{'{', '"', 0xff, '"', ':', '0', '}'})), loaderExample{}); err == nil {
t.Fatal("invalid UTF-8")
}
}
func TestDecoderRegistryAndConcurrentUse(t *testing.T) {
decoder := DecoderFunc(func([]byte) (Document, error) {
return Document{Value: map[string]any{"upload": map[string]any{"maxBytes": json.Number("42")}}}, nil
})
l, err := NewLoader(WithDecoder("custom", decoder))
if err != nil {
t.Fatal(err)
}
path := configFile(t, "custom", "ignored")
var wg sync.WaitGroup
for i := 0; i < 24; i++ {
wg.Add(1)
go func() {
defer wg.Done()
cfg, e := LoadProject(path, loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment}, l)
if e != nil || cfg.Upload.MaxBytes != 42 {
t.Error(e)
}
}()
}
wg.Wait()
if _, err := DecodeFile(path, loaderExample{}); err == nil {
t.Fatal("decoder registration leaked globally")
}
for _, options := range [][]LoaderOption{{WithDecoder("json", decoder)}, {WithDecoder("x", decoder), WithDecoder("x", decoder)}, {WithDecoder("x", DecoderFunc(nil))}} {
if _, err = NewLoader(options...); err == nil {
t.Fatal("invalid registration")
}
}
l, _ = NewLoader(WithDecoder("panic", DecoderFunc(func([]byte) (Document, error) { panic("private-value") })))
_, err = DecodeFile(configFile(t, "panic", ""), loaderExample{}, l)
if err == nil || strings.Contains(err.Error(), "private-value") {
t.Fatal(err)
}
cycle := map[string]any{}
cycle["self"] = cycle
l, _ = NewLoader(WithDecoder("cycle", DecoderFunc(func([]byte) (Document, error) { return Document{Value: map[string]any{"dynamic": cycle}}, nil })))
if _, err = DecodeFile(configFile(t, "cycle", ""), loaderExample{}, l); err == nil {
t.Fatal("decoder bypassed traversal limit")
}
}
func TestUnifiedLoggerNormalization(t *testing.T) {
type project struct {
Server Config `json:"server"`
}
base := project{Server: Default()}
base.Server.Log.File = "old.log"
base.Server.Log.Rotation.Enabled = true
for _, full := range []bool{false, true} {
name := "GO_OMNI_LOG_OUTPUT"
if full {
name = "GO_OMNI_SERVER_LOG_OUTPUT"
}
cfg, err := LoadProject("", base, EnvOptions{Prefix: "GO_OMNI_", Empty: "ignore", Aliases: ServerEnvAliases("GO_OMNI_", "server"), Lookup: func(k string) (string, bool) { return "stdout", k == name }})
if err != nil || cfg.Server.Log.File != "" || cfg.Server.Log.Rotation.Enabled || base.Server.Log.File != "old.log" {
t.Fatal(cfg, err)
}
}
}
func FuzzConfigurationDecoders(f *testing.F) {
f.Add(uint8(0), []byte(`{"count":1}`))
f.Add(uint8(1), []byte("count: 1"))
f.Add(uint8(2), []byte("<config><count>1</count></config>"))
f.Fuzz(func(t *testing.T, format uint8, data []byte) {
if len(data) > 8192 {
data = data[:8192]
}
ds := []Decoder{JSONDecoder{}, YAMLDecoder{}, XMLDecoder{}}
doc, err := ds[int(format)%len(ds)].Decode(data)
if err == nil {
_, _ = bindDocument(doc, struct {
Count int `json:"count"`
}{})
}
})
}
func BenchmarkConfiguration(b *testing.B) {
for name, tc := range map[string]struct {
d Decoder
raw string
}{"json": {JSONDecoder{}, `{"count":9007199254740993,"timeout":"5s","values":[1,2,3]}`}, "yaml": {YAMLDecoder{}, "count: 9007199254740993\ntimeout: 5s\nvalues: [1,2,3]"}, "xml": {XMLDecoder{}, `<config><count>9007199254740993</count><timeout>5s</timeout><values type="array"><item>1</item><item>2</item><item>3</item></values></config>`}} {
b.Run(name, func(b *testing.B) {
b.ReportAllocs()
raw := []byte(tc.raw)
for i := 0; i < b.N; i++ {
doc, err := tc.d.Decode(raw)
if err != nil {
b.Fatal(err)
}
_, err = bindDocument(doc, struct {
Count int64 `json:"count"`
Timeout time.Duration `json:"timeout"`
Values []int `json:"values"`
}{})
if err != nil {
b.Fatal(err)
}
}
})
}
}
// Text-only scalars and embedded config fields use the same behavior in each format.
type testText struct{ Value string }
func (v *testText) UnmarshalText(data []byte) error { v.Value = string(data); return nil }
func TestConfigScalarAndEmbedding(t *testing.T) {
type Fields struct {
Name string `json:"name"`
}
type C struct {
Fields
Nested Fields `json:"nested"`
Scalar testText `json:"scalar"`
Bytes []byte `json:"bytes"`
}
cases := map[string]string{
"json": `{"name":"outer","nested":{"name":"inner"},"scalar":"text","bytes":"aGk="}`,
"yaml": "name: outer\nnested:\n name: inner\nscalar: text\nbytes: aGk=\n",
"xml": `<config><name>outer</name><nested><name>inner</name></nested><scalar>text</scalar><bytes>aGk=</bytes></config>`,
}
for ext, data := range cases {
t.Run(ext, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "config."+ext)
os.WriteFile(path, []byte(data), 0600)
got, err := DecodeFile(path, &C{})
if err != nil {
t.Fatal(err)
}
if got.Name != "outer" || got.Nested.Name != "inner" || got.Scalar.Value != "text" || string(got.Bytes) != "hi" {
t.Fatal(got)
}
})
}
type Tagged struct {
Fields `json:"nested"`
}
doc, _ := (JSONDecoder{}).Decode([]byte(`{"name":"wrong"}`))
if _, err := bindDocument(doc, Tagged{}); err == nil {
t.Fatal("tagged embed incorrectly flattened")
}
}
func TestConfigDefaultsIsolationAndEmptyValues(t *testing.T) {
type Child struct {
Name string `json:"name"`
}
type C struct {
Child *Child `json:"child"`
Items []int `json:"items"`
Labels map[string]string `json:"labels"`
Enabled bool `json:"enabled"`
}
base := C{Child: &Child{Name: "old"}, Items: []int{1}, Labels: map[string]string{"old": "yes"}, Enabled: true}
doc, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"new"},"labels":{"new":"yes"},"items":[],"enabled":false}`))
got, err := bindDocument(doc, base)
if err != nil {
t.Fatal(err)
}
if got.Child.Name != "new" || got.Items == nil || len(got.Items) != 0 || got.Enabled {
t.Fatal(got)
}
if base.Child.Name != "old" || len(base.Items) != 1 || len(base.Labels) != 1 {
t.Fatal("mutated defaults")
}
bad, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"changed"},"items":"bad"}`))
if _, err := bindDocument(bad, base); err == nil {
t.Fatal("invalid input accepted")
}
if base.Child.Name != "old" {
t.Fatal("failed load changed default pointer")
}
null, _ := (JSONDecoder{}).Decode([]byte(`{"child":null,"items":null,"labels":null,"enabled":null}`))
got, err = bindDocument(null, base)
if err != nil || got.Child != nil || got.Items != nil || got.Labels != nil || !got.Enabled {
t.Fatal(got, err)
}
}
func TestDefaultStageDoesNotReadEnvironment(t *testing.T) {
reads := 0
type C struct {
Value int `json:"value" envDefault:"3"`
}
got, err := LoadProject("", C{}, EnvOptions{Lookup: func(string) (string, bool) { reads++; return "", false }})
if err != nil || got.Value != 3 || reads != 1 {
t.Fatal(got, err, reads)
}
d := locateError(Invalid("servers[0].timeout", "must be positive"), "a.json", Document{}, map[string]string{"servers": "APP_SERVERS"})
if d.Env != "APP_SERVERS" || d.File != "" {
t.Fatal(d)
}
}
func TestDecoderRegistryFrozen(t *testing.T) {
l, err := NewLoader()
if err != nil {
t.Fatal(err)
}
if err = WithDecoder("extra", JSONDecoder{})(l); err == nil {
t.Fatal("mutable registry")
}
}
func BenchmarkConfigurationStartup(b *testing.B) {
path := filepath.Join(b.TempDir(), "app.json")
if err := os.WriteFile(path, []byte(`{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":4096}}`), 0600); err != nil {
b.Fatal(err)
}
options := EnvOptions{Prefix: "BENCH_", Lookup: func(name string) (string, bool) {
if name == "BENCH_UPLOAD_MAX_BYTES" {
return "8192", true
}
return "", false
}}
defaults := loaderExample{Server: Default()}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := LoadProject(path, defaults, options); err != nil {
b.Fatal(err)
}
}
}
func BenchmarkConfigurationLarge(b *testing.B) {
type Entry struct {
Timeout time.Duration `json:"timeout"`
Count int `json:"count"`
}
type Large struct {
Entries []Entry `json:"entries"`
}
raw := []byte(`{"entries":[` + strings.TrimSuffix(strings.Repeat(`{"timeout":"3s","count":10},`, 1000), ",") + `]}`)
b.SetBytes(int64(len(raw)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
doc, err := (JSONDecoder{}).Decode(raw)
if err != nil {
b.Fatal(err)
}
if _, err = bindDocument(doc, Large{}); err != nil {
b.Fatal(err)
}
}
}

View File

@@ -7,6 +7,7 @@ import (
"errors"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/logx"
"math"
"os"
"reflect"
@@ -34,30 +35,114 @@ func ApplyDefaults[T any](base T) (T, error) { return projectOverlay(base, EnvOp
// LoadProject applies code/tag defaults, a strict optional file, environment,
// then the project's Validate method, before any logger or dependency is opened.
// envDefault is only applied to zero-valued defaults before reading the file.
func LoadProject[T any](path string, defaults T, options EnvOptions) (T, error) {
func LoadProject[T any](path string, defaults T, options EnvOptions, loaders ...*Loader) (T, error) {
var zero T
loader, err := selectLoader(loaders)
if err != nil {
return zero, err
}
cfg, err := projectOverlay(defaults, options, true)
if err != nil {
return zero, err
}
var doc Document
if path != "" {
cfg, err = DecodeFile(path, cfg)
doc, err = loader.read(path)
if err != nil {
return zero, errors.New("config: cannot decode file; check its path, format, field names and value types")
return zero, err
}
cfg, err = bindDocument(doc, cfg)
if err != nil {
return zero, locateError(err, path, doc, nil)
}
} else {
value, e := cloneConfiguration(reflect.ValueOf(&cfg).Elem(), 0)
if e != nil {
return zero, &Error{Reason: "cannot clone defaults", cause: e}
}
cfg = value.Interface().(T)
}
cfg, err = OverlayEnv(cfg, options)
origins := map[string]string{}
cfg, err = projectOverlay(cfg, options, false, origins)
if err != nil {
return zero, err
}
normalizeProject(reflect.ValueOf(&cfg).Elem(), "", origins, 0)
if valid, ok := any(&cfg).(interface{ Validate() error }); ok {
if err = valid.Validate(); err != nil {
return zero, err
return zero, locateError(err, path, doc, origins)
}
}
return cfg, nil
}
func locateError(err error, path string, doc Document, origins map[string]string) *Error {
d := validationError(err)
field := d.Field
for field != "" && origins[field] == "" {
index := strings.LastIndexAny(field, ".[")
if index < 0 {
field = ""
} else {
field = field[:index]
}
}
if env := origins[field]; env != "" {
d.Env = env
d.File = ""
d.Line = 0
d.Column = 0
return d
}
d.File = path
if d.Line == 0 {
p := doc.Positions[d.Field]
d.Line = p.Line
d.Column = p.Column
}
return d
}
// Framework-owned normalization stays inside the common pipeline. Explicit
// environment stream selection clears a file profile unless a file override
// was also provided; project code never needs to repeat this rule.
func normalizeProject(v reflect.Value, path string, origins map[string]string, depth int) {
if depth > 32 {
return
}
if v.Kind() == reflect.Pointer {
if v.IsNil() {
return
}
v = v.Elem()
}
if v.Kind() != reflect.Struct {
return
}
if v.Type() == reflect.TypeOf(Config{}) && v.CanAddr() {
c := v.Addr().Interface().(*Config)
if origins[joinField(path, "log.output")] != "" && c.Log.Output != "" && (origins[joinField(path, "log.file")] == "" || c.Log.File == "") {
c.Log.File = ""
c.Log.Rotation = logx.RotationConfig{}
}
return
}
for i := 0; i < v.NumField(); i++ {
f := v.Type().Field(i)
if f.PkgPath != "" {
continue
}
name := strings.Split(f.Tag.Get("json"), ",")[0]
if name == "-" {
continue
}
if name == "" {
name = f.Name
}
normalizeProject(v.Field(i), joinField(path, name), origins, depth+1)
}
}
// OverlayEnv overlays the entire project struct without mutating the input.
// Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a
// subtree. required means nonempty string/non-nil pointer or collection; false
@@ -70,7 +155,7 @@ func OverlayEnv[T any](base T, options EnvOptions) (T, error) {
var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
var durationType = reflect.TypeOf(time.Duration(0))
func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error) {
func projectOverlay[T any](base T, options EnvOptions, defaults bool, sourceMaps ...map[string]string) (T, error) {
var zero T
if options.Lookup == nil {
options.Lookup = os.LookupEnv
@@ -88,8 +173,8 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
result := reflect.New(v.Type()).Elem()
result.Set(v)
seen := map[string]bool{}
var walk func(reflect.Value, string, int) error
walk = func(value reflect.Value, prefix string, depth int) error {
var walk func(reflect.Value, string, string, int) error
walk = func(value reflect.Value, prefix, path string, depth int) error {
if depth > 32 {
return errors.New("config: nesting exceeds 32")
}
@@ -110,6 +195,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if jsonName == "" {
jsonName = field.Name
}
fieldPath := joinField(path, jsonName)
key := prefix + envName(jsonName)
if tag[0] != "" {
key = tag[0]
@@ -131,7 +217,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
}
}
if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) {
if err := walk(dst, key+"_", depth+1); err != nil {
if err := walk(dst, key+"_", fieldPath, depth+1); err != nil {
return err
}
continue
@@ -142,14 +228,14 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if !dst.IsNil() {
child.Elem().Set(dst.Elem())
}
if err := walk(child.Elem(), key+"_", depth+1); err != nil {
if err := walk(child.Elem(), key+"_", fieldPath, depth+1); err != nil {
return err
}
if !dst.IsNil() || !child.Elem().IsZero() {
dst.Set(child)
}
if required && !defaults && dst.IsNil() {
return fmt.Errorf("config: %s is required", key)
return &Error{Env: key, Field: fieldPath, Reason: "is required"}
}
continue
}
@@ -164,7 +250,11 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
return fmt.Errorf("config: duplicate environment key %s", key)
}
seen[key] = true
raw, present := options.Lookup(key)
var raw string
var present bool
if !defaults {
raw, present = options.Lookup(key)
}
selectedKey := key
if !present && !defaults {
for _, alias := range options.Aliases[key] {
@@ -180,7 +270,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
}
if present && raw == "" && !defaults {
if empty == "reject" {
return fmt.Errorf("config: %s must not be empty", selectedKey)
return &Error{Env: selectedKey, Field: fieldPath, Reason: "must not be empty"}
}
if empty == "ignore" {
present = false
@@ -189,17 +279,24 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if present {
parsed := reflect.New(dst.Type()).Elem()
if err := parseEnvValue(parsed, raw); err != nil {
return fmt.Errorf("config: %s must be a valid %s", selectedKey, dst.Type())
origin := selectedKey
if defaults {
origin = ""
}
return &Error{Env: origin, Field: fieldPath, Reason: "expected a valid " + dst.Type().String(), cause: err}
}
dst.Set(parsed)
if !defaults && len(sourceMaps) > 0 {
sourceMaps[0][fieldPath] = selectedKey
}
}
if required && !defaults && emptyRequired(dst) {
return fmt.Errorf("config: %s is required", key)
return &Error{Env: key, Field: fieldPath, Reason: "is required"}
}
}
return nil
}
if err := walk(result, options.Prefix, 0); err != nil {
if err := walk(result, options.Prefix, "", 0); err != nil {
return zero, err
}
return result.Interface().(T), nil

117
copyx/benchmark_test.go Normal file
View File

@@ -0,0 +1,117 @@
package copyx_test
import (
"errors"
"fmt"
"testing"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
func BenchmarkScenarios(b *testing.B) {
type request struct {
Name string
Count int64
Active bool
}
type result struct {
Label string
Count int64
Active bool
}
source := request{"sample", 9007199254740993, true}
option := copyx.WithFieldNames[request, result](map[string]string{"Name": "Label"})
copier, err := copyx.New(option)
if err != nil {
b.Fatal(err)
}
for _, reuse := range []bool{false, true} {
b.Run(fmt.Sprintf("struct/reuse%t", reuse), func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var target result
var err error
if reuse {
err = copier.Copy(&target, source)
} else {
err = copyx.Copy(&target, source, option)
}
if err != nil {
b.Fatal(err)
}
}
})
}
deep, _ := copyx.New(copyx.WithDeepCopy())
b.Run("bytes64KiB", func(b *testing.B) {
src := make([]byte, 64<<10)
b.SetBytes(int64(len(src)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var dst []byte
if err := deep.Copy(&dst, src); err != nil {
b.Fatal(err)
}
}
})
src := map[string][]int{}
for i := 0; i < 100; i++ {
src[fmt.Sprint(i)] = []int{1, 2, 3}
}
b.Run("map100", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var dst map[string][]int
if err := deep.Copy(&dst, src); err != nil {
b.Fatal(err)
}
}
})
cycle := &detail{Values: []int{1, 2, 3}}
cycle.Next = cycle
b.Run("cycle", func(b *testing.B) {
b.ReportAllocs()
for i := 0; i < b.N; i++ {
var dst *detail
if err := deep.Copy(&dst, cycle); err != nil {
b.Fatal(err)
}
}
})
b.Run("parallelCycle", func(b *testing.B) {
b.ReportAllocs()
b.RunParallel(func(pb *testing.PB) {
for pb.Next() {
var dst *detail
if err := deep.Copy(&dst, cycle); err != nil {
b.Error(err)
return
}
}
})
})
for _, fresh := range []bool{false, true} {
b.Run(fmt.Sprintf("converter/fresh%t", fresh), func(b *testing.B) {
c, err := copyx.New(copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) ([]int, error) {
if fresh {
return append([]int(nil), v...), nil
}
return v, nil
}))
if err != nil {
b.Fatal(err)
}
src := []int{1, 2, 3}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var dst []int
err := c.Copy(&dst, src)
if fresh && err != nil || !fresh && !errors.Is(err, copyx.ErrSharedReference) {
b.Fatal(err)
}
}
})
}
}

537
copyx/copy.go Normal file
View File

@@ -0,0 +1,537 @@
package copyx
import (
"errors"
"fmt"
"reflect"
"runtime"
"sort"
"sync"
)
var (
ErrType = errors.New("copyx: incompatible types")
ErrNil = errors.New("copyx: nil cannot fill a non-nullable target")
ErrField = errors.New("copyx: unmatched field")
ErrUnsupported = errors.New("copyx: unsupported value; provide an explicit data converter")
ErrSharedReference = errors.New("copyx: deep copy retains a mutable source reference")
ErrLimit = errors.New("copyx: traversal limit exceeded")
)
// Error describes a field/index path without recording the source value.
type Error struct {
Path string
Cause error
}
func (e *Error) Error() string { return e.Path + ": " + e.Cause.Error() }
func (e *Error) Unwrap() error { return e.Cause }
func at(path string, err error) error { return &Error{Path: path, Cause: err} }
func Copy(target, source any, options ...Option) error {
c, err := New(options...)
if err != nil {
return err
}
return c.Copy(target, source)
}
// Clone always makes a deep copy. Nil and empty collections stay distinct.
func Clone[T any](source T, options ...Option) (T, error) {
var target T
options = append(append([]Option(nil), options...), WithDeepCopy())
err := Copy(&target, source, options...)
return target, err
}
// Copy stages the complete result and only assigns target on success. User
// converter side effects (including mutation of source) cannot be rolled back.
// Caller must synchronize concurrent access to source and target themselves.
func (c *Copier) Copy(target, source any) error {
// Alias indexes contain addresses only. Keep the original graph alive until
// validation and commit finish so the GC cannot recycle any indexed address.
defer func() { runtime.KeepAlive(source) }()
if c == nil {
return errors.New("copyx: nil Copier")
}
dst := reflect.ValueOf(target)
if !dst.IsValid() || dst.Kind() != reflect.Pointer || dst.IsNil() {
return errors.New("copyx: target must be a non-nil pointer")
}
src := reflect.ValueOf(source)
s := state{options: &c.options, seen: map[visit]reflect.Value{}}
if c.options.deep {
var err error
s.graph, err = sourceGraph(src)
if err != nil {
return err
}
if src.IsValid() && src.Kind() == reflect.Pointer && !src.IsNil() && dst.Elem().Kind() == reflect.Struct {
id := identity(src)
s.valueRoot = &id
}
}
out, err := s.walk(src, dst.Elem(), dst.Elem().Type(), "$", 0)
if err != nil {
return err
}
if s.graph != nil {
if err = s.graph.audit(out); err != nil {
return err
}
}
dst.Elem().Set(out)
return nil
}
type visit struct {
reference
target reflect.Type
}
type state struct {
options *settings
seen map[visit]reflect.Value
graph *graph
nodes int
allocated uint64
valueRoot *reference
}
func (s *state) reserve(count int, size uintptr, path string) error {
if count < 0 || (count > 0 && uint64(size) > maxCopyBytes/uint64(count)) {
return at(path, ErrLimit)
}
s.allocated += uint64(count) * uint64(size)
if s.allocated > maxCopyBytes {
return at(path, ErrLimit)
}
return nil
}
func nullable(k reflect.Kind) bool {
switch k {
case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan:
return true
}
return false
}
func baseValue(base reflect.Value, t reflect.Type) reflect.Value {
if base.IsValid() && base.Type() == t {
return base
}
return reflect.Zero(t)
}
func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) {
s.nodes++
if depth > maxDepth || s.nodes > maxNodes {
return reflect.Value{}, at(path, ErrLimit)
}
base = baseValue(base, t)
if !shallowSafe(t) {
return reflect.Value{}, at(path, ErrUnsupported)
}
if src.IsValid() && !src.CanInterface() {
return reflect.Value{}, at(path, ErrUnsupported)
}
// Exact converters also receive typed nils and zeros. No method calls,
// Scanner/Valuer invocation or implicit numeric conversion is attempted.
if src.IsValid() {
if convert, ok := s.options.converters[pair{src.Type(), t}]; ok {
out, err := convert(src)
if err != nil {
return reflect.Value{}, at(path, err)
}
if s.graph != nil {
if err = s.graph.audit(out); err != nil {
return reflect.Value{}, at(path, err)
}
}
return out, nil
}
}
if !src.IsValid() || (nullable(src.Kind()) && src.IsNil()) {
if src.IsValid() && !s.nilCompatible(src.Type(), t, map[pair]bool{}) {
return reflect.Value{}, at(path, ErrType)
}
if s.options.nilPolicy == NilSkip {
return base, nil
}
if nullable(t.Kind()) || s.options.nilPolicy == NilZero {
return reflect.Zero(t), nil
}
return reflect.Value{}, at(path, ErrNil)
}
if s.options.skipZero && src.IsZero() {
if !s.nilCompatible(src.Type(), t, map[pair]bool{}) {
return reflect.Value{}, at(path, ErrType)
}
return base, nil
}
if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && !s.options.skipZero && s.options.nilPolicy != NilSkip {
if !s.options.deep || simpleValue(t) {
return src, nil
}
}
if src.Kind() == reflect.Interface {
return s.walk(src.Elem(), base, t, path, depth+1)
}
if t.Kind() == reflect.Interface {
if !src.Type().Implements(t) {
return reflect.Value{}, at(path, ErrType)
}
out, err := s.walk(src, reflect.Value{}, src.Type(), path, depth+1)
if err != nil {
return reflect.Value{}, err
}
v := reflect.New(t).Elem()
v.Set(out)
return v, nil
}
if s.options.deep && forbidden(t) {
return reflect.Value{}, at(path, ErrUnsupported)
}
if src.Kind() == reflect.Pointer && t.Kind() != reflect.Pointer {
return s.walk(src.Elem(), base, t, path, depth+1)
}
if t.Kind() == reflect.Pointer {
if !s.options.deep && src.Type() == t {
return src, nil
}
if src.Kind() == reflect.Pointer {
key := visit{identity(src), t}
if s.valueRoot != nil && key.reference == *s.valueRoot {
return reflect.Value{}, at(path, fmt.Errorf("%w: cyclic root must retain its pointer; use Clone(source) or Copy(&targetPointer, source)", ErrUnsupported))
}
if prior, ok := s.seen[key]; ok {
return prior, nil
}
}
if err := s.reserve(1, t.Elem().Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.New(t.Elem())
if out.Type() != t {
out = out.Convert(t)
}
if src.Kind() == reflect.Pointer {
key := visit{identity(src), t}
s.seen[key] = out
src = src.Elem()
}
var childBase reflect.Value
if !base.IsNil() {
childBase = base.Elem()
}
value, err := s.walk(src, childBase, t.Elem(), path, depth+1)
if err != nil {
return reflect.Value{}, err
}
out.Elem().Set(value)
return out, nil
}
if src.Kind() != t.Kind() {
return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t))
}
if t == timeType && src.Type() == t {
return src, nil
}
switch t.Kind() {
case reflect.Struct:
return s.structure(src, base, t, path, depth)
case reflect.Slice:
if !s.options.deep && src.Type() == t {
return src, nil
}
key := visit{identity(src), t}
if prior, ok := s.seen[key]; ok {
return prior, nil
}
if err := s.reserve(src.Len(), t.Elem().Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.MakeSlice(t, src.Len(), src.Len())
s.seen[key] = out
if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && simpleValue(t.Elem()) {
reflect.Copy(out, src)
return out, nil
}
for i := 0; i < src.Len(); i++ {
v, err := s.walk(src.Index(i), reflect.Value{}, t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return reflect.Value{}, err
}
out.Index(i).Set(v)
}
return out, nil
case reflect.Array:
if src.Len() != t.Len() {
return reflect.Value{}, at(path, ErrType)
}
if err := s.reserve(1, t.Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.New(t).Elem()
for i := 0; i < src.Len(); i++ {
v, err := s.walk(src.Index(i), base.Index(i), t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return reflect.Value{}, err
}
out.Index(i).Set(v)
}
return out, nil
case reflect.Map:
if !s.options.deep && src.Type() == t {
return src, nil
}
key := visit{identity(src), t}
if prior, ok := s.seen[key]; ok {
return prior, nil
}
if err := s.reserve(src.Len(), t.Key().Size()+t.Elem().Size()+32, path); err != nil {
return reflect.Value{}, err
}
out := reflect.MakeMapWithSize(t, src.Len())
s.seen[key] = out
it := src.MapRange()
for it.Next() {
k, err := s.walk(it.Key(), reflect.Value{}, t.Key(), path+"[key]", depth+1)
if err != nil {
return reflect.Value{}, err
}
if !k.Comparable() {
return reflect.Value{}, at(path+"[key]", ErrType)
}
if out.MapIndex(k).IsValid() {
return reflect.Value{}, at(path+"[key]", errors.New("copyx: key conversion collision"))
}
v, err := s.walk(it.Value(), reflect.Value{}, t.Elem(), path+"[value]", depth+1)
if err != nil {
return reflect.Value{}, err
}
out.SetMapIndex(k, v)
}
return out, nil
case reflect.Func, reflect.Chan, reflect.UnsafePointer:
if s.options.deep {
return reflect.Value{}, at(path, ErrUnsupported)
}
}
if src.Type() != t {
return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t))
}
return src, nil
}
var fieldTypes sync.Map
func exportedFields(t reflect.Type) map[string]reflect.StructField {
if known, ok := fieldTypes.Load(t); ok {
return known.(map[string]reflect.StructField)
}
out := map[string]reflect.StructField{}
for _, f := range reflect.VisibleFields(t) {
if f.PkgPath != "" {
continue
}
kind := f.Type.Kind()
if kind == reflect.Pointer {
kind = f.Type.Elem().Kind()
}
if f.Anonymous && kind == reflect.Struct {
continue
}
out[f.Name] = f
}
actual, _ := fieldTypes.LoadOrStore(t, out)
return actual.(map[string]reflect.StructField)
}
func (s *state) nilCompatible(from, to reflect.Type, seen map[pair]bool) bool {
key := pair{from, to}
if seen[key] {
return true
}
seen[key] = true
if from == to {
return true
}
if _, ok := s.options.converters[key]; ok {
return true
}
if to.Kind() == reflect.Interface {
return from.Implements(to)
}
if from.Kind() == reflect.Pointer {
return s.nilCompatible(from.Elem(), to, seen)
}
if to.Kind() == reflect.Pointer {
return s.nilCompatible(from, to.Elem(), seen)
}
if from.Kind() != to.Kind() {
return false
}
switch from.Kind() {
case reflect.Struct:
// Nil must not hide a structurally incompatible type pair. There are no
// field values to visit, but coverage and conversion types still apply.
for i := 0; i < to.NumField(); i++ {
if to.Field(i).PkgPath != "" {
return false
}
}
sources, targets := exportedFields(from), exportedFields(to)
rules := s.options.fields[key]
if rules == nil {
rules = &fields{}
}
reverse, used := map[string]string{}, map[string]bool{}
for a, b := range rules.names {
reverse[b] = a
}
for name, target := range targets {
if rules.ignoreTarget[name] {
continue
}
srcName := name
if renamed, ok := reverse[name]; ok {
srcName = renamed
}
source, ok := sources[srcName]
if !ok || rules.ignoreSource[srcName] || !s.nilCompatible(source.Type, target.Type, seen) {
return false
}
used[srcName] = true
}
if s.options.strict {
for name := range sources {
if !used[name] && !rules.ignoreSource[name] {
return false
}
}
}
return true
case reflect.Slice:
return s.nilCompatible(from.Elem(), to.Elem(), seen)
case reflect.Map:
return s.nilCompatible(from.Key(), to.Key(), seen) && s.nilCompatible(from.Elem(), to.Elem(), seen)
case reflect.Array:
return from.Len() == to.Len() && s.nilCompatible(from.Elem(), to.Elem(), seen)
}
return false
}
func fieldValue(v reflect.Value, index []int) (reflect.Value, bool) {
for _, i := range index {
if v.Kind() == reflect.Pointer {
if v.IsNil() {
return reflect.Value{}, false
}
v = v.Elem()
}
v = v.Field(i)
}
return v, true
}
func setField(v reflect.Value, index []int, value reflect.Value) error {
for j, i := range index {
field := v.Field(i)
if !field.CanSet() {
return ErrUnsupported
}
if j == len(index)-1 {
field.Set(value)
return nil
}
if field.Kind() == reflect.Pointer {
// Detach embedded destination pointers before editing promoted fields.
fresh := reflect.New(field.Type().Elem())
if !field.IsNil() {
fresh.Elem().Set(field.Elem())
}
field.Set(fresh)
v = fresh.Elem()
} else {
v = field
}
}
return nil
}
func (s *state) structure(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) {
if err := s.reserve(1, t.Size(), path); err != nil {
return reflect.Value{}, err
}
out := reflect.New(t).Elem()
out.Set(base)
if src.Type() == t {
// Private scalar state may be assigned as part of its whole value; mutable
// private state requires a converter, never unsafe writes or silent loss.
for i := 0; i < t.NumField(); i++ {
f := t.Field(i)
if f.PkgPath != "" && s.options.deep && privateMutable(f.Type) {
return reflect.Value{}, at(path+"."+f.Name, ErrUnsupported)
}
}
out.Set(src)
} else {
for i := 0; i < t.NumField(); i++ {
if t.Field(i).PkgPath != "" {
return reflect.Value{}, at(path, ErrUnsupported)
}
}
}
from, to := exportedFields(src.Type()), exportedFields(t)
rules := s.options.fields[pair{src.Type(), t}]
if rules == nil {
rules = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}}
}
reverse := map[string]string{}
for a, b := range rules.names {
reverse[b] = a
}
names := make([]string, 0, len(to))
for n := range to {
names = append(names, n)
}
sort.Strings(names)
used := map[string]bool{}
for _, name := range names {
target := to[name]
old, _ := fieldValue(base, target.Index)
if rules.ignoreTarget[name] {
if !old.IsValid() {
old = reflect.Zero(target.Type)
}
if err := setField(out, target.Index, old); err != nil {
return reflect.Value{}, at(path+"."+name, err)
}
continue
}
srcName := name
if renamed, ok := reverse[name]; ok {
srcName = renamed
}
source, ok := from[srcName]
if !ok || rules.ignoreSource[srcName] {
return reflect.Value{}, at(path+"."+name, ErrField)
}
used[srcName] = true
v, ok := fieldValue(src, source.Index)
if !ok {
return reflect.Value{}, at(path+"."+name, ErrNil)
}
value, err := s.walk(v, old, target.Type, path+"."+name, depth+1)
if err != nil {
return reflect.Value{}, err
}
if err = setField(out, target.Index, value); err != nil {
return reflect.Value{}, at(path+"."+name, err)
}
}
if s.options.strict {
for name := range from {
if !used[name] && !rules.ignoreSource[name] {
return reflect.Value{}, at(path+"."+name, ErrField)
}
}
}
return out, nil
}

448
copyx/copy_test.go Normal file
View File

@@ -0,0 +1,448 @@
package copyx_test
import (
"errors"
"fmt"
"reflect"
"strings"
"sync"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
type detail struct {
Values []int
Index map[string][]int
Next *detail
}
func TestDeepGraph(t *testing.T) {
src := &detail{Values: []int{1, 2}, Index: map[string][]int{"a": {3}}}
src.Next = src
dst, err := copyx.Clone(src)
if err != nil {
t.Fatal(err)
}
if dst == src || dst.Next != dst {
t.Fatal("pointer cycle not preserved")
}
dst.Values[0] = 9
dst.Index["a"][0] = 8
if src.Values[0] != 1 || src.Index["a"][0] != 3 {
t.Fatal("deep alias")
}
m := map[string]any{}
m["self"] = m
n, err := copyx.Clone(m)
if err != nil {
t.Fatal(err)
}
n["self"].(map[string]any)["new"] = 1
if n["new"] != 1 || m["new"] != nil {
t.Fatal("map cycle changed")
}
x := []any{nil}
x[0] = x
y, err := copyx.Clone(x)
if err != nil {
t.Fatal(err)
}
y[0].([]any)[0] = "changed"
if y[0] != "changed" {
t.Fatal("slice cycle changed")
}
if _, ok := x[0].([]any); !ok {
t.Fatal("source slice changed")
}
}
func TestShallowAndNil(t *testing.T) {
src := detail{Values: []int{1}}
var dst detail
if err := copyx.Copy(&dst, src); err != nil {
t.Fatal(err)
}
dst.Values[0] = 2
if src.Values[0] != 2 {
t.Fatal("shallow unexpectedly detached")
}
for _, slice := range [][]int{nil, {}} {
out, err := copyx.Clone(slice)
if err != nil || (out == nil) != (slice == nil) {
t.Fatal(out, err)
}
}
for _, m := range []map[string]int{nil, {}} {
out, err := copyx.Clone(m)
if err != nil || (out == nil) != (m == nil) {
t.Fatal(out, err)
}
}
var p *int
n := 7
if err := copyx.Copy(&n, p); !errors.Is(err, copyx.ErrNil) || n != 7 {
t.Fatal(n, err)
}
if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilSkip)); err != nil || n != 7 {
t.Fatal(n, err)
}
if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilZero)); err != nil || n != 0 {
t.Fatal(n, err)
}
x := 3
out := &x
if err := copyx.Copy(&out, p); err != nil || out != nil {
t.Fatal(out, err)
}
flag := true
if err := copyx.Copy(&flag, false); err != nil || flag {
t.Fatal(flag, err)
}
flag = true
if err := copyx.Copy(&flag, false, copyx.WithSkipZero()); err != nil || !flag {
t.Fatal(flag, err)
}
}
func TestConverterAliasesRejected(t *testing.T) {
for _, kind := range []string{"same", "nested", "subslice", "capacity", "fieldPointer", "private", "map", "sibling"} {
t.Run(kind, func(t *testing.T) {
src := &detail{Values: []int{1, 2, 3}, Index: map[string][]int{"x": {1}}}
var err error
switch kind {
case "same":
_, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return v, nil }))
case "nested":
_, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return &detail{Values: v.Values}, nil }))
case "subslice":
_, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[1:], nil }))
case "capacity":
_, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[:0], nil }))
case "map":
_, err = copyx.Clone(src.Index, copyx.WithConverter(func(v map[string][]int) (map[string][]int, error) { return v, nil }))
case "fieldPointer":
var dst *int
err = copyx.Copy(&dst, src.Values, copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) (*int, error) { return &v[1], nil }))
case "private":
type opaque struct{ data []int }
v := opaque{src.Values}
_, err = copyx.Clone(v, copyx.WithConverter(func(v opaque) (opaque, error) { return v, nil }))
case "sibling":
type object struct{ A, B []int }
v := object{[]int{1}, []int{2}}
_, err = copyx.Clone(v, copyx.WithConverter(func(_ []int) ([]int, error) { return v.B, nil }))
}
if !errors.Is(err, copyx.ErrSharedReference) {
t.Fatalf("got %v", err)
}
})
}
_, err := copyx.Clone(4, copyx.WithConverter(func(v int) (int, error) { return v, nil }))
if err != nil {
t.Fatal("equal primitive rejected", err)
}
_, err = copyx.Clone("same", copyx.WithConverter(func(v string) (string, error) { return v, nil }))
if err != nil {
t.Fatal(err)
}
good, err := copyx.Clone([]int{1}, copyx.WithConverter(func(v []int) ([]int, error) { return append([]int(nil), v...), nil }))
if err != nil || good[0] != 1 {
t.Fatal(good, err)
}
}
func TestPrivateAndUnsupported(t *testing.T) {
type money struct{ cents *int64 }
n := int64(9007199254740991)
src := money{&n}
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
out, err := copyx.Clone(src, copyx.WithConverter(func(v money) (money, error) { n := *v.cents; return money{&n}, nil }))
if err != nil || *out.cents != n || out.cents == src.cents {
t.Fatal(out, err)
}
tm := time.Now()
cloned, err := copyx.Clone(tm)
if err != nil || !cloned.Equal(tm) {
t.Fatal(cloned, err)
}
for _, v := range []any{make(chan int), func() {}, &sync.Mutex{}, sync.Mutex{}} {
if _, err := copyx.Clone(v); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatalf("%T: %v", v, err)
}
}
}
func TestNilTypeMismatchAndInterfaceConverter(t *testing.T) {
var source *string
target := 7
if err := copyx.Copy(&target, source, copyx.WithNilPolicy(copyx.NilZero)); !errors.Is(err, copyx.ErrType) || target != 7 {
t.Fatal(target, err)
}
var list []int
var targetMap map[string]int
if err := copyx.Copy(&targetMap, list); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
type input struct{ Value any }
type output struct{ Value string }
var out output
err := copyx.Copy(&out, input{}, copyx.WithConverter(func(v any) (string, error) {
if v == nil {
return "nil handled", nil
}
return "value", nil
}))
if err != nil || out.Value != "nil handled" {
t.Fatal(out, err)
}
var lock sync.Mutex
if err := copyx.Copy(&lock, sync.Mutex{}); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
type locked struct {
mu sync.Mutex
Value int
}
var l locked
if err := copyx.Copy(&l, locked{}); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
}
func TestAllocationLimitAndPointerRoot(t *testing.T) {
src := make([]byte, (64<<20)+1)
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) {
t.Fatal(err)
}
n := &detail{}
n.Next = n
var value detail
if err := copyx.Copy(&value, n, copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
var p *detail
if err := copyx.Copy(&p, n, copyx.WithDeepCopy()); err != nil || p.Next != p || p == n {
t.Fatal(p, err)
}
}
func TestFieldsConvertersAndAtomicity(t *testing.T) {
type input struct {
Name string
Age int
Extra string
}
type output struct {
Label string
Age int
Local string
}
rename := map[string]string{"Name": "Label"}
c, err := copyx.New(copyx.WithFieldNames[input, output](rename), copyx.WithIgnoreTarget[input, output]("Local"))
if err != nil {
t.Fatal(err)
}
rename["Name"] = "Missing"
dst := output{Local: "preserved"}
if err = c.Copy(&dst, input{Name: "hello", Age: 2}); err != nil || dst.Label != "hello" || dst.Local != "preserved" {
t.Fatal(dst, err)
}
if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local")); !errors.Is(err, copyx.ErrField) {
t.Fatal(err)
}
if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local"), copyx.WithIgnoreSource[input, output]("Extra")); err != nil {
t.Fatal(err)
}
if err = copyx.Copy(&dst, input{}); !errors.Is(err, copyx.ErrField) {
t.Fatal(err)
}
if _, err = copyx.New(copyx.WithFieldNames[input, output](map[string]string{"Nope": "Age"})); err == nil {
t.Fatal("invalid rule accepted")
}
if _, err = copyx.New(copyx.WithConverter(func(v int) (string, error) { return "", nil }), copyx.WithConverter(func(v int) (string, error) { return "", nil })); err == nil {
t.Fatal("duplicate converter accepted")
}
sentinel := errors.New("conversion failed")
type node struct {
A []int
Z string
}
old := node{[]int{8}, "old"}
before := old.A
err = copyx.Copy(&old, node{[]int{1}, "bad"}, copyx.WithDeepCopy(), copyx.WithConverter(func(v string) (string, error) { return "", sentinel }))
if !errors.Is(err, sentinel) || !strings.Contains(err.Error(), "Z") || old.Z != "old" || old.A[0] != 8 || &old.A[0] != &before[0] {
t.Fatal(old, err)
}
// Exact typed nil converter must get control before the global nil policy.
var ptr *int
var value int
err = copyx.Copy(&value, ptr, copyx.WithConverter(func(v *int) (int, error) {
if v == nil {
return 0, sentinel
}
return *v, nil
}))
if !errors.Is(err, sentinel) {
t.Fatal(err)
}
var small int8 = 3
if err = copyx.Copy(&small, int64(300)); !errors.Is(err, copyx.ErrType) || small != 3 {
t.Fatal(small, err)
}
}
type Embedded struct {
A int
Z string
}
func TestEmbeddedDestinationDetached(t *testing.T) {
type target struct{ *Embedded }
type source struct {
A int
Z int
}
ptr := &Embedded{5, "keep"}
dst := target{ptr}
if err := copyx.Copy(&dst, source{6, 7}); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
if ptr.A != 5 || dst.Embedded != ptr {
t.Fatal("failure mutated embedded target")
}
if err := copyx.Copy(&dst, Embedded{6, "new"}); err != nil {
t.Fatal(err)
}
if ptr.A != 5 || dst.A != 6 {
t.Fatal("success mutated old embedded object")
}
}
func TestConcurrentCopier(t *testing.T) {
c, err := copyx.New(copyx.WithDeepCopy())
if err != nil {
t.Fatal(err)
}
src := &detail{Values: []int{7}, Index: map[string][]int{"a": {9}}}
var wg sync.WaitGroup
for i := 0; i < 64; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for n := 0; n < 20; n++ {
var dst *detail
if err := c.Copy(&dst, src); err != nil {
t.Error(err)
return
}
dst.Values[0] = n
if src.Values[0] != 7 {
t.Error("shared")
}
}
}()
}
wg.Wait()
}
func TestBoundAndCollections(t *testing.T) {
src := &detail{}
p := src
for i := 0; i < 300; i++ {
p.Next = &detail{}
p = p.Next
}
if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) {
t.Fatal(err)
}
type from struct{ Value string }
type to struct{ Value string }
var out []to
if err := copyx.Copy(&out, []from{{"a"}, {"b"}}); err != nil || len(out) != 2 || out[1].Value != "b" {
t.Fatal(out, err)
}
var array [2]to
if err := copyx.Copy(&array, [2]from{{"a"}, {"b"}}); err != nil {
t.Fatal(err)
}
if err := copyx.Copy(&array, [1]from{{"a"}}); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
var outMap map[string]int
err := copyx.Copy(&outMap, map[int]int{1: 1, 2: 2}, copyx.WithConverter(func(int) (string, error) { return "same", nil }))
if err == nil || outMap != nil {
t.Fatal(outMap, err)
}
}
func FuzzClone(f *testing.F) {
f.Add([]byte{1, 2, 3})
f.Add([]byte{})
f.Fuzz(func(t *testing.T, data []byte) {
if len(data) > 1024 {
data = data[:1024]
}
src := map[string]any{"bytes": data, "other": []string{"x"}}
out, err := copyx.Clone(src)
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(src, out) {
t.Fatal("changed value")
}
if len(data) > 0 {
out["bytes"].([]byte)[0] ^= 255
if data[0] == out["bytes"].([]byte)[0] {
t.Fatal("alias")
}
}
})
}
func TestTraversalNodeLimit(t *testing.T) {
source := make([]any, 200001)
if _, err := copyx.Clone(source); !errors.Is(err, copyx.ErrLimit) {
t.Fatal("expected bounded traversal", err)
}
}
func TestNilCannotHideFieldMismatch(t *testing.T) {
type source struct{ Value string }
type target struct{ Value int }
var src *source
var dst *target
for _, policy := range []copyx.NilPolicy{copyx.NilError, copyx.NilZero, copyx.NilSkip} {
if err := copyx.Copy(&dst, src, copyx.WithNilPolicy(policy)); !errors.Is(err, copyx.ErrType) {
t.Fatal("nil concealed incompatible fields", err)
}
}
if err := copyx.Copy(&dst, src, copyx.WithConverter(func(string) (int, error) { return 0, nil })); err != nil {
t.Fatal(err)
}
text := "keep"
if err := copyx.Copy(&text, 0, copyx.WithSkipZero()); !errors.Is(err, copyx.ErrType) || text != "keep" {
t.Fatal("zero concealed incompatible types", text, err)
}
}
func BenchmarkCopy(b *testing.B) {
for _, n := range []int{1, 100, 1000} {
for _, deep := range []bool{false, true} {
b.Run(fmt.Sprintf("fields%d/deep%t", n, deep), func(b *testing.B) {
src := detail{Values: make([]int, n), Index: map[string][]int{"one": {1, 2}}}
var opts []copyx.Option
if deep {
opts = append(opts, copyx.WithDeepCopy())
}
c, _ := copyx.New(opts...)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
var dst detail
if err := c.Copy(&dst, src); err != nil {
b.Fatal(err)
}
}
})
}
}
}

270
copyx/graph.go Normal file
View File

@@ -0,0 +1,270 @@
package copyx
import (
"fmt"
"reflect"
"sort"
"sync"
"time"
)
const maxDepth = 256
const maxNodes = 200000
const maxCopyBytes = 64 << 20
var timeType = reflect.TypeFor[time.Time]()
// Immutable type metadata is shared, never values or converter registrations.
var valueTypes sync.Map
func simpleValue(t reflect.Type) bool {
if known, ok := valueTypes.Load(t); ok {
return known.(bool)
}
result := t == timeType
if !result {
switch t.Kind() {
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64,
reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr,
reflect.Float32, reflect.Float64, reflect.Complex64, reflect.Complex128, reflect.String:
result = true
case reflect.Array:
result = simpleValue(t.Elem())
case reflect.Struct:
result = !forbidden(t)
for i := 0; result && i < t.NumField(); i++ {
result = simpleValue(t.Field(i).Type)
}
}
}
valueTypes.Store(t, result)
return result
}
type reference struct {
kind reflect.Kind
typ reflect.Type
ptr uintptr
length, capacity int
}
func identity(v reflect.Value) reference {
r := reference{kind: v.Kind(), typ: v.Type(), ptr: uintptr(v.UnsafePointer())}
if v.Kind() == reflect.Slice {
r.length = v.Len()
r.capacity = v.Cap()
}
return r
}
type span struct{ start, end uintptr }
type graph struct {
maps map[uintptr]bool
spans []span
maxEnd []uintptr
}
// References are inspected without reading private values through Interface or
// using unsafe writes. Slice capacity matters: returning source[:0] is an alias.
func scanGraph(v reflect.Value, visit func(reflect.Value, string) error) error {
seen := map[reference]bool{}
nodes := 0
var walk func(reflect.Value, string, int) error
walk = func(v reflect.Value, path string, depth int) error {
nodes++
if depth > maxDepth || nodes > maxNodes {
return at(path, ErrLimit)
}
if !v.IsValid() {
return nil
}
if err := visit(v, path); err != nil {
return err
}
if simpleValue(v.Type()) {
return nil
} // time.Time is a documented immutable value.
switch v.Kind() {
case reflect.Interface:
if !v.IsNil() {
return walk(v.Elem(), path, depth+1)
}
case reflect.Pointer, reflect.Map, reflect.Slice:
if v.IsNil() {
return nil
}
id := identity(v)
if seen[id] {
return nil
}
seen[id] = true
switch v.Kind() {
case reflect.Pointer:
return walk(v.Elem(), path, depth+1)
case reflect.Map:
it := v.MapRange()
for it.Next() {
if err := walk(it.Key(), path+"[key]", depth+1); err != nil {
return err
}
if err := walk(it.Value(), path+"[value]", depth+1); err != nil {
return err
}
}
case reflect.Slice:
if simpleValue(v.Type().Elem()) {
return nil
}
all := v.Slice(0, v.Cap())
for i := 0; i < all.Len(); i++ {
if err := walk(all.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil {
return err
}
}
}
case reflect.Struct:
for i := 0; i < v.NumField(); i++ {
if err := walk(v.Field(i), path+"."+v.Type().Field(i).Name, depth+1); err != nil {
return err
}
}
case reflect.Array:
for i := 0; i < v.Len(); i++ {
if err := walk(v.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil {
return err
}
}
}
return nil
}
return walk(v, "$", 0)
}
func mutableSpan(v reflect.Value) (span, bool) {
var n uintptr
switch v.Kind() {
case reflect.Pointer:
if v.IsNil() {
return span{}, false
}
n = v.Type().Elem().Size()
case reflect.Slice:
if v.IsNil() {
return span{}, false
}
n = uintptr(v.Cap()) * v.Type().Elem().Size()
default:
return span{}, false
}
if n == 0 {
return span{}, false
}
start := uintptr(v.UnsafePointer())
return span{start, start + n}, true
}
func sourceGraph(v reflect.Value) (*graph, error) {
g := &graph{maps: map[uintptr]bool{}}
err := scanGraph(v, func(v reflect.Value, _ string) error {
if r, ok := mutableSpan(v); ok {
g.spans = append(g.spans, r)
}
if v.Kind() == reflect.Map && !v.IsNil() {
g.maps[uintptr(v.UnsafePointer())] = true
}
return nil
})
sort.Slice(g.spans, func(i, j int) bool { return g.spans[i].start < g.spans[j].start })
var end uintptr
for _, r := range g.spans {
if r.end > end {
end = r.end
}
g.maxEnd = append(g.maxEnd, end)
}
return g, err
}
func (g *graph) audit(v reflect.Value) error {
return scanGraph(v, func(v reflect.Value, path string) error {
if forbidden(v.Type()) {
return at(path, ErrUnsupported)
}
switch v.Kind() {
case reflect.Func, reflect.Chan:
if !v.IsNil() {
return at(path, ErrUnsupported)
}
case reflect.UnsafePointer:
if v.Pointer() != 0 {
return at(path, ErrUnsupported)
}
}
if v.Kind() == reflect.Map && !v.IsNil() && g.maps[uintptr(v.UnsafePointer())] {
return at(path, ErrSharedReference)
}
if r, ok := mutableSpan(v); ok {
i := sort.Search(len(g.spans), func(i int) bool { return g.spans[i].start >= r.end })
if i > 0 && g.maxEnd[i-1] > r.start {
return at(path, ErrSharedReference)
}
}
return nil
})
}
func forbidden(t reflect.Type) bool {
// Lock/resource implementations are not data. A custom converter may turn
// them into a different data type but may not manufacture cloned handles.
switch t.PkgPath() {
case "sync", "sync/atomic", "os", "net", "database/sql":
return t.Kind() == reflect.Struct
}
return false
}
// Shallow copies may share handles through pointers, but must not copy lock or
// resource state by value, including state inside a containing struct/array.
var safeValueTypes sync.Map
func shallowSafe(t reflect.Type) bool {
if known, ok := safeValueTypes.Load(t); ok {
return known.(bool)
}
result := true
if forbidden(t) {
safeValueTypes.Store(t, false)
return false
}
switch t.Kind() {
case reflect.Struct:
for i := 0; i < t.NumField(); i++ {
if !shallowSafe(t.Field(i).Type) {
result = false
break
}
}
case reflect.Array:
result = shallowSafe(t.Elem())
}
safeValueTypes.Store(t, result)
return result
}
func privateMutable(t reflect.Type) bool {
if t == timeType {
return false
}
switch t.Kind() {
case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan, reflect.UnsafePointer:
return true
case reflect.Array:
return privateMutable(t.Elem())
case reflect.Struct:
if forbidden(t) {
return true
}
for i := 0; i < t.NumField(); i++ {
if privateMutable(t.Field(i).Type) {
return true
}
}
}
return false
}

169
copyx/options.go Normal file
View File

@@ -0,0 +1,169 @@
// Package copyx copies Go values without JSON, generated code or global registration.
package copyx
import (
"errors"
"fmt"
"reflect"
)
// NilPolicy controls nil input. NilError preserves nil for nullable targets and
// rejects nil to non-nullable targets. NilSkip retains the existing target.
type NilPolicy uint8
const (
NilError NilPolicy = iota
NilZero
NilSkip
)
type pair struct{ from, to reflect.Type }
type converter func(reflect.Value) (reflect.Value, error)
type fields struct {
names map[string]string
ignoreSource, ignoreTarget map[string]bool
}
type settings struct {
deep, skipZero, strict bool
nilPolicy NilPolicy
converters map[pair]converter
fields map[pair]*fields
}
// Option configures one immutable Copier. Options never change global behavior.
type Option func(*settings) error
func WithDeepCopy() Option { return func(s *settings) error { s.deep = true; return nil } }
// WithSkipZero explicitly skips Go zero values, including false, 0 and "".
func WithSkipZero() Option { return func(s *settings) error { s.skipZero = true; return nil } }
// WithStrictFields additionally rejects unconsumed exported source fields.
// Target fields must have a source by default, unless explicitly ignored.
func WithStrictFields() Option { return func(s *settings) error { s.strict = true; return nil } }
func WithNilPolicy(p NilPolicy) Option {
return func(s *settings) error {
if p > NilSkip {
return errors.New("copyx: invalid nil policy")
}
s.nilPolicy = p
return nil
}
}
// WithConverter registers an exact source/target type pair. Typed nils are
// delivered to converters before default nil handling. The function must not
// mutate its input and must be safe for concurrent calls when Copier is shared.
func WithConverter[S, T any](fn func(S) (T, error)) Option {
return func(s *settings) error {
if fn == nil {
return errors.New("copyx: nil converter")
}
key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()}
if _, exists := s.converters[key]; exists {
return fmt.Errorf("copyx: duplicate converter %s -> %s", key.from, key.to)
}
s.converters[key] = func(v reflect.Value) (reflect.Value, error) {
var input S
reflect.ValueOf(&input).Elem().Set(v)
out, err := fn(input)
return reflect.ValueOf(&out).Elem(), err
}
return nil
}
}
// WithFieldNames declares source -> target Go field names for one struct pair.
// It is not a JSON tag mapping. Only actual, unambiguous exported fields qualify.
func WithFieldNames[S, T any](names map[string]string) Option {
copyNames := make(map[string]string, len(names))
for k, v := range names {
copyNames[k] = v
}
return fieldOption[S, T](func(f *fields) error {
for from, to := range copyNames {
if _, exists := f.names[from]; exists {
return fmt.Errorf("copyx: duplicate field rule %s", from)
}
f.names[from] = to
}
return nil
})
}
func WithIgnoreSource[S, T any](names ...string) Option {
names = append([]string(nil), names...)
return fieldOption[S, T](func(f *fields) error {
for _, n := range names {
f.ignoreSource[n] = true
}
return nil
})
}
// WithIgnoreTarget preserves the corresponding existing target fields in Copy.
// Deep mode rejects that preservation if it would retain a source reference.
func WithIgnoreTarget[S, T any](names ...string) Option {
names = append([]string(nil), names...)
return fieldOption[S, T](func(f *fields) error {
for _, n := range names {
f.ignoreTarget[n] = true
}
return nil
})
}
func fieldOption[S, T any](apply func(*fields) error) Option {
return func(s *settings) error {
key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()}
if key.from.Kind() != reflect.Struct || key.to.Kind() != reflect.Struct {
return errors.New("copyx: field options require struct types")
}
f := s.fields[key]
if f == nil {
f = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}}
s.fields[key] = f
}
return apply(f)
}
}
// Copier is immutable after construction. Each Copy owns its traversal state.
type Copier struct{ options settings }
func New(options ...Option) (*Copier, error) {
s := settings{converters: map[pair]converter{}, fields: map[pair]*fields{}}
for _, option := range options {
if option == nil {
return nil, errors.New("copyx: nil option")
}
if err := option(&s); err != nil {
return nil, err
}
}
for key, f := range s.fields {
from, to := exportedFields(key.from), exportedFields(key.to)
used := map[string]bool{}
for src, dst := range f.names {
if _, ok := from[src]; !ok {
return nil, fmt.Errorf("copyx: unknown source field %s.%s", key.from, src)
}
if _, ok := to[dst]; !ok {
return nil, fmt.Errorf("copyx: unknown target field %s.%s", key.to, dst)
}
if used[dst] || f.ignoreSource[src] || f.ignoreTarget[dst] {
return nil, fmt.Errorf("copyx: conflicting field rule %s", dst)
}
used[dst] = true
}
for n := range f.ignoreSource {
if _, ok := from[n]; !ok {
return nil, fmt.Errorf("copyx: unknown source field %s", n)
}
}
for n := range f.ignoreTarget {
if _, ok := to[n]; !ok {
return nil, fmt.Errorf("copyx: unknown target field %s", n)
}
}
}
return &Copier{options: s}, nil
}

View File

@@ -0,0 +1,21 @@
# 统一配置加载示例
从框架根目录运行,无需数据库或外部服务:
```sh
go run ./examples/configuration
go run ./examples/configuration -format json
go run ./examples/configuration -format xml
DEMO_CLIENT_TIMEOUT=500ms DEMO_MAX_ITEMS=0 go run ./examples/configuration
DEMO_MAX_ITEMS=wrong go run ./examples/configuration
```
前三次都输出 `timeout=2s maxItems=100 allowed=[json xml]`。环境覆盖允许显式零值。
最后一条输出定位 `maxItems` / `DEMO_MAX_ITEMS` 的安全错误,不打印输入原值。
也可 `-config /path/to/app.yaml` 读取自己的文件。示例没有建立网络连接。
`Config` 定义字段,`envDefault` 定义缺省值,`Validate` 用 `config.Invalid` 返回字段与公开原因。
`newLoader` 额外注册 `.custom`(演示复用 JSON decoder);同一个 `load` 可读取所有格式。
JSON/YAML/XML 直接可用,无需注册。新增格式只修改解析器注册,不改业务结构或校验。
只打印选定非敏感字段,不输出整个配置。完整格式、错误定位和旧项目迁移见 [CONFIG.md](../../CONFIG.md)。

View File

@@ -0,0 +1,5 @@
{
"client": {"endpoint": "https://example.com", "timeout": "2s"},
"maxItems": 100,
"allowed": ["json", "xml"]
}

View File

@@ -0,0 +1,8 @@
<config>
<client>
<endpoint>https://example.com</endpoint>
<timeout>2s</timeout>
</client>
<maxItems>100</maxItems>
<allowed type="array"><item>json</item><item>xml</item></allowed>
</config>

View File

@@ -0,0 +1,5 @@
client:
endpoint: https://example.com
timeout: 2s
maxItems: 100
allowed: [json, xml]

View File

@@ -0,0 +1,85 @@
// Run with go run ./examples/configuration -format xml, or -config /path/to/app.yaml.
package main
import (
"embed"
"flag"
"fmt"
"os"
"path/filepath"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
//go:embed config.json config.yaml config.xml
var examples embed.FS
type ClientConfig struct {
Endpoint string `json:"endpoint"`
Timeout time.Duration `json:"timeout" envDefault:"3s"`
// This field is deliberately unavailable to configuration files.
Token string `json:"-" env:"DEMO_CLIENT_TOKEN,secret"`
}
type Config struct {
Client ClientConfig `json:"client"`
MaxItems int `json:"maxItems" envDefault:"20"`
Allowed []string `json:"allowed"`
}
func (c Config) Validate() error {
if c.Client.Timeout <= 0 {
return config.Invalid("client.timeout", "must be positive")
}
if c.MaxItems < 0 {
return config.Invalid("maxItems", "must not be negative")
}
return nil
}
func newLoader() (*config.Loader, error) {
// Example of an instance-local extension. A TOML/etc. adapter can implement
// Decoder and return Document using that parser instead; no global registry.
return config.NewLoader(config.WithDecoder("custom", config.DecoderFunc(func(data []byte) (config.Document, error) {
return (config.JSONDecoder{}).Decode(data)
})))
}
func load(path string) (Config, error) {
loader, err := newLoader()
if err != nil {
return Config{}, err
}
return config.LoadProject(path, Config{}, config.EnvOptions{Prefix: "DEMO_", Empty: "reject"}, loader)
}
func run() error {
path := flag.String("config", "", "external configuration file")
format := flag.String("format", "yaml", "embedded example: json, yaml, xml")
flag.Parse()
if *path == "" {
data, err := examples.ReadFile("config." + *format)
if err != nil {
return fmt.Errorf("choose json, yaml or xml")
}
dir, err := os.MkdirTemp("", "goOmniConfiguration-")
if err != nil {
return err
}
defer os.RemoveAll(dir)
*path = filepath.Join(dir, "config."+*format)
if err = os.WriteFile(*path, data, 0600); err != nil {
return err
}
}
cfg, err := load(*path)
if err != nil {
return err
}
// Print selected non-sensitive fields, never the whole configuration.
fmt.Printf("timeout=%s maxItems=%d allowed=%v\n", cfg.Client.Timeout, cfg.MaxItems, cfg.Allowed)
return nil
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

View File

@@ -0,0 +1,34 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
func TestExamples(t *testing.T) {
var previous Config
for i, ext := range []string{"json", "yaml", "xml", "custom"} {
source := ext
if source == "custom" {
source = "json"
}
data, err := examples.ReadFile("config." + source)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "app."+ext)
if err = os.WriteFile(path, data, 0600); err != nil {
t.Fatal(err)
}
got, err := load(path)
if err != nil {
t.Fatal(err)
}
if i > 0 && !reflect.DeepEqual(previous, got) {
t.Fatal("format difference")
}
previous = got
}
}

View File

@@ -0,0 +1,23 @@
# 结构复制示例
在框架仓库根目录运行 `go run ./examples/copying`;不需要数据库、网络服务或 JSON 映射配置。
示例包含:可复用构造器、Go 字段改名、时间自定义转换、保留 false、深拷贝切片、同类型 Clone,以及拒绝自定义转换器返回输入的原切片。
预期输出:
```text
name=sample enabled=false time=2026-01-01T00:00:00Z sourceTag=original
clone detached: true
shared converter rejected: true
```
测试:`go test -race ./examples/copying`。完整策略、限制与旧生成映射迁移见 [COPY.md](../../COPY.md)。
独立工资三端的实际生成、HTTP 与金额用例见 [集成测试](../../cmd/go-omni/apigen/copyConsumer_test.go) 和 [公共契约](../../cmd/go-omni/apigen/testdata/composition/service.api)。框架根目录运行:
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1
```
该测试在临时独立项目中执行,三个入口共用一个普通 Go copyx helper。公共字段新增时,只修改公共契约和独立业务结构,三个入口的转换规则不增加。

61
examples/copying/main.go Normal file
View File

@@ -0,0 +1,61 @@
// copying demonstrates optional Go-to-Go conversion and checked deep copying.
package main
import (
"errors"
"fmt"
"log"
"time"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
type Input struct {
Name string
Enabled bool
Tags []string
CreatedAt time.Time
}
type View struct {
DisplayName string
Enabled bool
Tags []string
CreatedAt string
}
func run() error {
converter, err := copyx.New(
copyx.WithDeepCopy(),
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
copyx.WithConverter(func(v time.Time) (string, error) { return v.UTC().Format(time.RFC3339), nil }),
)
if err != nil {
return err
}
source := Input{Name: "sample", Enabled: false, Tags: []string{"original"}, CreatedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)}
var target View
if err = converter.Copy(&target, source); err != nil {
return err
}
target.Tags[0] = "changed"
fmt.Printf("name=%s enabled=%t time=%s sourceTag=%s\n", target.DisplayName, target.Enabled, target.CreatedAt, source.Tags[0])
cloned, err := copyx.Clone(source)
if err != nil {
return err
}
cloned.Tags[0] = "separate"
fmt.Println("clone detached:", source.Tags[0] == "original")
_, err = copyx.Clone(source.Tags, copyx.WithConverter(func(v []string) ([]string, error) { return v, nil }))
fmt.Println("shared converter rejected:", errors.Is(err, copyx.ErrSharedReference))
if !errors.Is(err, copyx.ErrSharedReference) {
return errors.New("expected alias rejection")
}
return nil
}
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}

View File

@@ -0,0 +1,9 @@
package main
import "testing"
func TestExample(t *testing.T) {
if err := run(); err != nil {
t.Fatal(err)
}
}

View File

@@ -1,6 +1,6 @@
{
"generatorConfig": true,
"templateVersion": "v0.17.0-preview",
"templateVersion": "v0.20.0-preview",
"profile": {
"database": "none",
"storage": "none",
@@ -12,6 +12,7 @@
"entry": "api/managed.api",
"generated": {
"docs/openapi.json": "0370fd6bd6a387e2ef21f6b42b009ce66749846b8bea3d481009a34f3af039a5",
"internal/config/loadGen.go": "5d630b894abdbbd66ddfa7d7d30f7573fc4ff995020a32a9f9157cd4af3a5655",
"internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d",
"internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375",
"internal/handler/openapiGen.go": "f24cab3abbb8557a3fc4dd93fba23cc0b093f3e0c944d3ba1bd5b0eeff2d96f3",

View File

@@ -2,7 +2,7 @@ module example.com/managed
go 1.22.0
require git.yuntuo.hk/k2safe/go-omni v0.17.0-preview
require git.yuntuo.hk/k2safe/go-omni v0.20.0-preview
require (
github.com/gabriel-vasile/mimetype v1.4.3 // indirect

View File

@@ -1,10 +1,9 @@
package config
import (
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
"os"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
type Config struct {
@@ -22,42 +21,10 @@ func Default() Config {
return Config{Server: server}
}
// Load applies defaults -> file -> typed environment -> validation.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string) (Config, error) {
cfg := Default()
var err error
cfg, err = config.ApplyDefaults(cfg)
if err != nil {
return Config{}, err
}
if path != "" {
cfg, err = config.DecodeFile(path, cfg)
if err != nil {
return Config{}, err
}
}
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_", Empty: "ignore", Aliases: config.ServerEnvAliases("GO_OMNI_", "server")})
if err != nil {
return Config{}, err
}
output, outputSet := os.LookupEnv("GO_OMNI_SERVER_LOG_OUTPUT")
if !outputSet {
output = os.Getenv("GO_OMNI_LOG_OUTPUT")
}
file, fileSet := os.LookupEnv("GO_OMNI_SERVER_LOG_FILE")
if !fileSet {
file = os.Getenv("GO_OMNI_LOG_FILE")
}
if output != "" && file == "" {
cfg.Server.Log.File = ""
cfg.Server.Log.Rotation = logx.RotationConfig{}
}
return cfg, cfg.Validate()
}
// Validate checks configuration before logs or dependencies open.
func (c Config) Validate() error {
if err := c.Server.Validate(); err != nil {
return err
return config.AtField("server", err)
}
return nil

View File

@@ -0,0 +1,16 @@
// Code generated by go-omni api. DO NOT EDIT.
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// Load is maintained by the generator. Extend Config/Default/Validate in
// config.go, or NewLoader in loader.go; do not duplicate the loading pipeline.
func Load(path string) (Config, error) {
loader, err := NewLoader()
if err != nil {
return Config{}, err
}
return omniConfig.LoadProject(path, Default(), omniConfig.EnvOptions{
Prefix: "GO_OMNI_", Empty: "ignore", Aliases: omniConfig.ServerEnvAliases("GO_OMNI_", "server"),
}, loader)
}

View File

@@ -0,0 +1,7 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// NewLoader is project-owned. JSON/YAML/XML work without changes. Register
// additional format decoders here only when the project needs them.
func NewLoader() (*omniConfig.Loader, error) { return omniConfig.NewLoader() }

View File

@@ -0,0 +1,53 @@
# v0.20 统一配置加载验收(2026-09-26)
本轮修改 go-omni 配置包、生成模板与文档。没有修改接入项目的业务配置/数据库,也没有使用线上服务器做压力测试。
## 回归结果
- 全量 `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...`:47 个有测试的包通过,包含实际生成项目、生命周期失败清理、错误目录/响应继承、上传和拷贝工具回归。
- `go vet ./...`、`go build ./...` 通过。
- `GOTOOLCHAIN=go1.22.12 go test ./config ./examples/configuration` 通过。
- 新生成消费者的 `go test -race ./...`、`go build .`、generate/check/dry-run 通过。
- checked-in managedService 已显式迁移旧 Load,生成检查、race 和构建通过;三种格式示例与环境覆盖命令实际运行通过。
针对性验证:
| 场景 | 结果 |
| --- | --- |
| JSON / YAML / XML 的 defaults→file→env→Validate | 同值,0/false 可覆盖,envDefault 不覆盖文件 |
| 大于 2^53 的整数、动态 json.Number | 精度保留 |
| null、空数组、嵌套指针、map、duration、TextUnmarshaler、JSON tag / 嵌入字段 | 语义回归通过 |
| 文件字段类型错、未知字段、重复字段、业务 Validate 失败 | 文件/字段/可用行列与安全原因 |
| 环境类型或校验失败、集合子字段校验 | 定位环境来源,不错误归因于文件 |
| 内部错误链和默认值隔离 | errors.Is/As 可用,失败不提交半成品/修改默认值 |
| 24 个并发加载、自定义 Decoder、禁止事后注册 | race 通过,实例间不泄漏 |
| 超大文件、超深输入、重复 XML 根、DTD/entity、YAML alias/merge、UTF-8 | 拒绝;解析器 panic 不公开 panic 内容 |
| 业务新增字段与自定义格式 | 不改 main/loadGen,生成保留手写文件 |
| 配置校验失败 | 依赖初始化之前返回;未触发测试中的 Init panic |
| 旧手写 Load、loadGen 漂移 | generate/check/dry-run 拒绝冲突,不改业务文件 |
30 秒、4 worker 的 `FuzzConfigurationDecoders`:383586 次执行,无失败(总运行 31.309 秒,包含停止收尾)。模糊输入上限 8 KiB;另有确定性 1 MiB/深度上限和恶意格式回归,不把 fuzz 次数当作形式证明。
## 性能
Go 1.27.1、darwin/arm64、Apple M2 Max;无 race 的本地基准,每个场景三轮。原始数据:[configurationBenchmark.txt](configurationBenchmark.txt)。
| 场景 | 中位数耗时 | 典型分配 |
| --- | ---: | ---: |
| 小配置 JSON 解析+绑定 | 5.068 µs | 3781 B / 75 allocs |
| 小配置 YAML 解析+绑定 | 14.855 µs | 15602 B / 181 allocs |
| 小配置 XML 解析+绑定 | 7.137 µs | 5605 B / 122 allocs |
| 含完整 Server 默认值:文件读取+环境覆盖+校验 | 107.660 µs | 约 67.4 KB / 1288 allocs |
| 1000 个条目,约 28 KB JSON 解析+绑定 | 2.548 ms | 约 2.40 MB / 31585 allocs |
配置在启动阶段加载,不经过业务请求路径。这些是暖文件缓存、本机测试结果,不能等同冷磁盘启动耗时、请求 QPS 或跨机器性能承诺。没有为此引入全局反射缓存、解析器池或后台线程。
## 边界与迁移
- 旧手写 Load 必须先明确迁移,生成器不会猜测其解密/其他特殊行为。
- 内置 XML 是明确的配置子集;YAML alias/merge/custom tag 拒绝;固定 Go 数组严格长度。
- 自定义 Decoder/反序列化/Validate 是可信 Go 回调,其并发、副作用与公开 Reason 的脱敏由项目负责。
- 第三方非结构化验证错误仅定位子树,并保留内部原因;业务用 Invalid/AtField 提供准确的公开字段与原因。
- 未进行业务数据库/第三方真实依赖验收,未修改或替代各接入项目的业务回归。
完整使用与迁移:[CONFIG.md](../CONFIG.md)。远程固定版本安装验收在发布后追加。

View File

@@ -0,0 +1,21 @@
goos: darwin
goarch: arm64
pkg: git.yuntuo.hk/k2safe/go-omni/config
cpu: Apple M2 Max
BenchmarkConfiguration/json-12 239197 5068 ns/op 3781 B/op 75 allocs/op
BenchmarkConfiguration/json-12 234421 5058 ns/op 3781 B/op 75 allocs/op
BenchmarkConfiguration/json-12 233330 5564 ns/op 3781 B/op 75 allocs/op
BenchmarkConfiguration/yaml-12 83022 14827 ns/op 15602 B/op 181 allocs/op
BenchmarkConfiguration/yaml-12 83197 14855 ns/op 15604 B/op 181 allocs/op
BenchmarkConfiguration/yaml-12 74828 16070 ns/op 15602 B/op 181 allocs/op
BenchmarkConfiguration/xml-12 167509 7144 ns/op 5606 B/op 122 allocs/op
BenchmarkConfiguration/xml-12 171202 7086 ns/op 5605 B/op 122 allocs/op
BenchmarkConfiguration/xml-12 167727 7137 ns/op 5605 B/op 122 allocs/op
BenchmarkConfigurationStartup-12 10000 107660 ns/op 67429 B/op 1288 allocs/op
BenchmarkConfigurationStartup-12 10000 107420 ns/op 67427 B/op 1288 allocs/op
BenchmarkConfigurationStartup-12 10000 107828 ns/op 67430 B/op 1288 allocs/op
BenchmarkConfigurationLarge-12 481 2548066 ns/op 10.99 MB/s 2403822 B/op 31585 allocs/op
BenchmarkConfigurationLarge-12 471 2532248 ns/op 11.06 MB/s 2403851 B/op 31586 allocs/op
BenchmarkConfigurationLarge-12 477 2641417 ns/op 10.61 MB/s 2403402 B/op 31583 allocs/op
PASS
ok git.yuntuo.hk/k2safe/go-omni/config 20.682s

View File

@@ -0,0 +1,82 @@
# v0.19 copyx 与映射移除验收
日期:2026-09-25。环境:Apple M2 Max、darwin/arm64、Go 1.27.1,基准默认 GOMAXPROCS=12。最低支持版本另用 Go 1.22.12 验证 copyx 与可运行示例。
## 范围与行为
本轮新增可选 Go 工具包并删除原映射配置/生成机制,没有修改接入项目代码、数据库或线上服务。没有新增第三方依赖,没有内置工资、金额或权限规则;工资只是生成器的独立通用验收 fixture。
| 场景 | 验证内容 |
| --- | --- |
| 浅/深复制 | 值、结构、指针、数组、map、slice、interface;浅共享与深隔离分别验证 |
| 空值与精度 | nil 与空集合、默认保留 false/0、显式 nil/skip 策略;不兼容类型不能借 nil/零值绕过检查 |
| 可变引用 | 原指针/map、子切片、零长度但有容量的视图、源元素指针、私有子引用、其他字段的引用均拒绝 |
| 自定义函数 | 字段改名、精确类型对、金额私有状态复制、nil interface;重复/冲突配置拒绝 |
| 失败语义 | errors.Is 保留 cause、目标不提交部分结果、嵌入目标指针不提前修改 |
| 对象图 | 指针/map/slice 循环,重复引用;循环指针根压成值结构拒绝 |
| 并发与资源 | 同一 Copier 64 goroutine 各复制 20 次,race;深度/节点/估算分配超限拒绝;锁/资源/函数等明确限制 |
| 真正生成的消费者 | 三端工资公共字段,Money scalar,大数精度/舍入、非法金额、缺失/null/零值、完整表单、错误码与错误链 |
| 维护成本 | 公共字段新增:改一处契约和一处独立业务结构;三个接口共用 helper,不增加拷贝规则 |
| 生成与生命周期 | 重复 generate/check/dry-run 保留业务文件;工厂异常、初始化/监听失败及资源关闭由既有回归验证 |
| 移除与 workspace | 旧配置/旧生成文件阻断且不写入,明确迁移后所有权清理;workspace 失败不部分更新服务 |
FuzzClone 使用动态 JSON 容器、随机 bytes 和嵌套 slice,验证值一致与引用隔离。输入超过 1 KiB 时截取继续验证,运行 30 秒、4 workers:**4,237,917 次执行,未发现失败**。这不等于所有形状都已 fuzz;循环、类型转换及资源拒绝另外用定向用例覆盖。
全量 46 个测试包通过 race 回归(含实际生成项目),vet/build、Go 1.22.12、示例运行与生成漂移检查均通过。
## 可复现命令
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -count=1 ./...
go vet ./...
go build ./...
GOTOOLCHAIN=go1.22.12 go test ./copyx ./examples/copying
go test ./copyx -run '^$' -fuzz FuzzClone -fuzztime 30s -parallel 4
go test ./copyx -run '^$' -bench . -benchmem -benchtime 300ms -count 3
go run ./examples/copying
go run ./cmd/go-omni api generate --dir examples/managedService --check
# examples/managedService 中另执行 go test -race ./... 与 go build .
```
全量回归包含真实临时生成项目,集成测试的子进程使用 go test -race / go build,并不只比较生成字符串。基准测量时没有同时运行本轮 fuzz 或全量测试;原始三轮结果见 [copyxBenchmark.txt](copyxBenchmark.txt)。
## 性能与分配
下表取三轮中位数。构造器复用与一次性调用都包含真实复制;`fields1000` 指整数 slice 的长度,不是 1,000 个结构字段。所有深拷贝保留引用检查。
| 场景 | 中位耗时 | B/op | allocs/op |
| --- | ---: | ---: | ---: |
| 相同结构浅拷贝(含引用字段) | 85.8 ns | 96 | 2 |
| 3 字段跨结构改名,复用 Copier | 697.5 ns | 168 | 7 |
| 相同跨结构,每次构建 Copier | 1,181 ns | 1,120 | 17 |
| 嵌套结构 + 1 个整数 + map,深拷贝 | 2,758 ns | 1,368 | 42 |
| 嵌套结构 + 100 个整数 + map,深拷贝 | 2,787 ns | 2,256 | 42 |
| 嵌套结构 + 1,000 个整数 + map,深拷贝 | 3,806 ns | 9,552 | 42 |
| 64 KiB byte slice 深拷贝 | 5,780 ns | 65,768 | 9 |
| 100 项 map,每项含整数 slice | 88,859 ns | 87,704 | 1,554 |
| 自引用循环对象深拷贝 | 2,128 ns | 640 | 26 |
| converter 新建 slice 并通过检查 | 539.7 ns | 280 | 10 |
| converter 返回原 slice 被拒绝 | 383.8 ns | 320 | 11 |
并行循环对象基准中位 515.2 ns/op 是多核聚合吞吐均摊,**不是单请求延迟**。复杂 map 的多次图扫描、字段路径和反射分配明显更贵;不据此宣称运行时工具快于手写赋值。热点复用 Copier;大对象和高频映射应按实际业务结构衡量是否改用直接 Go。
优化只包括不可变类型元数据缓存、纯值快速路径、纯值 slice 批量复制;没有去掉深拷贝别名检查。内置估算分配上限 64 MiB 不是硬堆内存隔离,converter 的外部分配及副作用不受该预算控制。
## 边界
- 新方式在运行时做类型与覆盖检查,不再有旧映射生成时的静态保证;字段变更要运行实际类型对测试。
- converter 是受信项目代码,必须只读输入;不能自动回滚其副作用、证明其未共享外部全局对象,或强杀无限循环。
- 当前记录是本机库/生成器验收,不是线上服务吞吐、Linux soak、真实业务数据库回归或 OSS/R2 云账号验收。本轮没有对线上服务器施压。
- 旧消费者仍需按 [COPY.md](../COPY.md#从旧-mapping-迁移) 迁移,不会被这次框架发布自动修改。
## 发布后远程验收
标签 `v0.19.0-preview` 固定在 `fd83235ce7fc15ce88a482d385d70e61f25d7b32`。空 module 缓存安装远程 CLI,消费者固定同版依赖、具有校验和、无本地 replace,7 组实际生成消费者 race 回归通过;包含三端 copyx 示例。
根目录 `go run .` / 构建、nethttp/native 的 SIGINT/SIGTERM、404/405、Go 错误目录、原始回调、上传/下载、workspace 共用错误目录及资源收尾均通过。远程单独执行 `go run git.yuntuo.hk/k2safe/go-omni/examples/copying@v0.19.0-preview` 也验证深拷贝隔离与共享 converter 拒绝。
```sh
python3 scripts/verifyManagedRelease.py --version v0.19.0-preview --report reports/copyxRemote.json
```
发布后验收工具同步移除了旧 mapping 用例,改为 copyx 消费者并核对实际远程依赖校验和。详细检查项、源摘要见 [copyxRemote.json](copyxRemote.json)。标签不随验收记录的后续文档提交移动。

View File

@@ -0,0 +1,48 @@
goos: darwin
goarch: arm64
pkg: git.yuntuo.hk/k2safe/go-omni/copyx
cpu: Apple M2 Max
BenchmarkScenarios/struct/reusefalse-12 251924 1209 ns/op 1120 B/op 17 allocs/op
BenchmarkScenarios/struct/reusefalse-12 316450 1181 ns/op 1120 B/op 17 allocs/op
BenchmarkScenarios/struct/reusefalse-12 314466 1140 ns/op 1120 B/op 17 allocs/op
BenchmarkScenarios/struct/reusetrue-12 517544 703.0 ns/op 168 B/op 7 allocs/op
BenchmarkScenarios/struct/reusetrue-12 517842 691.9 ns/op 168 B/op 7 allocs/op
BenchmarkScenarios/struct/reusetrue-12 522170 697.5 ns/op 168 B/op 7 allocs/op
BenchmarkScenarios/bytes64KiB-12 67933 5857 ns/op 11189.69 MB/s 65768 B/op 9 allocs/op
BenchmarkScenarios/bytes64KiB-12 61680 5780 ns/op 11337.56 MB/s 65768 B/op 9 allocs/op
BenchmarkScenarios/bytes64KiB-12 61843 5771 ns/op 11355.66 MB/s 65768 B/op 9 allocs/op
BenchmarkScenarios/map100-12 3808 88598 ns/op 87704 B/op 1554 allocs/op
BenchmarkScenarios/map100-12 4136 88859 ns/op 87704 B/op 1554 allocs/op
BenchmarkScenarios/map100-12 4051 89614 ns/op 87704 B/op 1554 allocs/op
BenchmarkScenarios/cycle-12 168308 2128 ns/op 640 B/op 26 allocs/op
BenchmarkScenarios/cycle-12 169057 2163 ns/op 640 B/op 26 allocs/op
BenchmarkScenarios/cycle-12 169892 2120 ns/op 640 B/op 26 allocs/op
BenchmarkScenarios/parallelCycle-12 963274 660.6 ns/op 640 B/op 26 allocs/op
BenchmarkScenarios/parallelCycle-12 582906 515.1 ns/op 640 B/op 26 allocs/op
BenchmarkScenarios/parallelCycle-12 748388 515.2 ns/op 640 B/op 26 allocs/op
BenchmarkScenarios/converter/freshfalse-12 828752 382.8 ns/op 320 B/op 11 allocs/op
BenchmarkScenarios/converter/freshfalse-12 897363 383.8 ns/op 320 B/op 11 allocs/op
BenchmarkScenarios/converter/freshfalse-12 1000000 387.5 ns/op 320 B/op 11 allocs/op
BenchmarkScenarios/converter/freshtrue-12 665454 541.2 ns/op 280 B/op 10 allocs/op
BenchmarkScenarios/converter/freshtrue-12 647089 535.0 ns/op 280 B/op 10 allocs/op
BenchmarkScenarios/converter/freshtrue-12 704719 539.7 ns/op 280 B/op 10 allocs/op
BenchmarkCopy/fields1/deepfalse-12 4058166 87.96 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields1/deepfalse-12 4137807 85.79 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields1/deepfalse-12 4201804 85.07 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields1/deeptrue-12 131413 2739 ns/op 1368 B/op 42 allocs/op
BenchmarkCopy/fields1/deeptrue-12 131658 2771 ns/op 1368 B/op 42 allocs/op
BenchmarkCopy/fields1/deeptrue-12 131880 2758 ns/op 1368 B/op 42 allocs/op
BenchmarkCopy/fields100/deepfalse-12 4209133 85.61 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields100/deepfalse-12 4239586 84.30 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields100/deepfalse-12 4248972 84.59 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields100/deeptrue-12 127617 2787 ns/op 2256 B/op 42 allocs/op
BenchmarkCopy/fields100/deeptrue-12 129816 2826 ns/op 2256 B/op 42 allocs/op
BenchmarkCopy/fields100/deeptrue-12 124041 2780 ns/op 2256 B/op 42 allocs/op
BenchmarkCopy/fields1000/deepfalse-12 4214600 85.59 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields1000/deepfalse-12 4250719 85.20 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields1000/deepfalse-12 4247498 85.05 ns/op 96 B/op 2 allocs/op
BenchmarkCopy/fields1000/deeptrue-12 92329 3818 ns/op 9552 B/op 42 allocs/op
BenchmarkCopy/fields1000/deeptrue-12 93945 3806 ns/op 9552 B/op 42 allocs/op
BenchmarkCopy/fields1000/deeptrue-12 96010 3782 ns/op 9552 B/op 42 allocs/op
PASS
ok git.yuntuo.hk/k2safe/go-omni/copyx 17.513s

15
reports/copyxFuzz.txt Normal file
View File

@@ -0,0 +1,15 @@
fuzz: elapsed: 0s, gathering baseline coverage: 0/10 completed
fuzz: elapsed: 0s, gathering baseline coverage: 10/10 completed, now fuzzing with 4 workers
fuzz: elapsed: 3s, execs: 426851 (142240/sec), new interesting: 5 (total: 15)
fuzz: elapsed: 6s, execs: 837791 (137019/sec), new interesting: 5 (total: 15)
fuzz: elapsed: 9s, execs: 1281087 (147767/sec), new interesting: 5 (total: 15)
fuzz: elapsed: 12s, execs: 1705531 (141463/sec), new interesting: 5 (total: 15)
fuzz: elapsed: 15s, execs: 2125995 (140169/sec), new interesting: 6 (total: 16)
fuzz: elapsed: 18s, execs: 2547285 (140432/sec), new interesting: 6 (total: 16)
fuzz: elapsed: 21s, execs: 2954940 (135854/sec), new interesting: 6 (total: 16)
fuzz: elapsed: 24s, execs: 3389762 (144941/sec), new interesting: 6 (total: 16)
fuzz: elapsed: 27s, execs: 3808273 (139491/sec), new interesting: 6 (total: 16)
fuzz: elapsed: 30s, execs: 4237917 (143257/sec), new interesting: 6 (total: 16)
fuzz: elapsed: 30s, execs: 4237917 (0/sec), new interesting: 6 (total: 16)
PASS
ok git.yuntuo.hk/k2safe/go-omni/copyx 30.363s

View File

@@ -0,0 +1,59 @@
ok git.yuntuo.hk/k2safe/go-omni 1.519s
ok git.yuntuo.hk/k2safe/go-omni/app 3.503s
ok git.yuntuo.hk/k2safe/go-omni/audit 1.970s
ok git.yuntuo.hk/k2safe/go-omni/auth 2.265s
ok git.yuntuo.hk/k2safe/go-omni/cache 2.566s
ok git.yuntuo.hk/k2safe/go-omni/cache/redis 2.712s
ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni 5.321s
ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen 76.921s
ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold 47.757s
ok git.yuntuo.hk/k2safe/go-omni/config 4.374s
ok git.yuntuo.hk/k2safe/go-omni/contract 4.702s
ok git.yuntuo.hk/k2safe/go-omni/copyx 5.171s
ok git.yuntuo.hk/k2safe/go-omni/database 4.141s
ok git.yuntuo.hk/k2safe/go-omni/database/gormx 3.916s
ok git.yuntuo.hk/k2safe/go-omni/database/migrate 3.856s
ok git.yuntuo.hk/k2safe/go-omni/database/mysql 3.816s
ok git.yuntuo.hk/k2safe/go-omni/errors 3.930s
? git.yuntuo.hk/k2safe/go-omni/examples/basic [no test files]
? git.yuntuo.hk/k2safe/go-omni/examples/compatibility [no test files]
? git.yuntuo.hk/k2safe/go-omni/examples/compatibility/ids [no test files]
ok git.yuntuo.hk/k2safe/go-omni/examples/copying 3.198s
ok git.yuntuo.hk/k2safe/go-omni/examples/modules 2.930s
ok git.yuntuo.hk/k2safe/go-omni/examples/platform 2.766s
ok git.yuntuo.hk/k2safe/go-omni/examples/reference 2.794s
? git.yuntuo.hk/k2safe/go-omni/examples/reference/handler [no test files]
? git.yuntuo.hk/k2safe/go-omni/examples/reference/model [no test files]
ok git.yuntuo.hk/k2safe/go-omni/examples/reference/repository 2.880s
? git.yuntuo.hk/k2safe/go-omni/examples/reference/service [no test files]
ok git.yuntuo.hk/k2safe/go-omni/health 3.091s
ok git.yuntuo.hk/k2safe/go-omni/httpc 3.345s
ok git.yuntuo.hk/k2safe/go-omni/httpx 3.870s
ok git.yuntuo.hk/k2safe/go-omni/internal/perfcheck 3.284s
? git.yuntuo.hk/k2safe/go-omni/internal/storagefixture [no test files]
? git.yuntuo.hk/k2safe/go-omni/internal/testutil [no test files]
ok git.yuntuo.hk/k2safe/go-omni/lifecycle 3.390s
ok git.yuntuo.hk/k2safe/go-omni/logx 3.459s
ok git.yuntuo.hk/k2safe/go-omni/metrics 2.732s
ok git.yuntuo.hk/k2safe/go-omni/middleware 2.718s
ok git.yuntuo.hk/k2safe/go-omni/middleware/redis 3.086s
ok git.yuntuo.hk/k2safe/go-omni/openapi 3.319s
ok git.yuntuo.hk/k2safe/go-omni/pagination 3.356s
? git.yuntuo.hk/k2safe/go-omni/queue [no test files]
ok git.yuntuo.hk/k2safe/go-omni/queue/redis 3.060s
ok git.yuntuo.hk/k2safe/go-omni/resilience 2.898s
ok git.yuntuo.hk/k2safe/go-omni/response 3.157s
ok git.yuntuo.hk/k2safe/go-omni/storage 3.414s
ok git.yuntuo.hk/k2safe/go-omni/storage/local 3.617s
ok git.yuntuo.hk/k2safe/go-omni/storage/oss 3.572s
ok git.yuntuo.hk/k2safe/go-omni/storage/s3 3.752s
? git.yuntuo.hk/k2safe/go-omni/storage/storagetest [no test files]
ok git.yuntuo.hk/k2safe/go-omni/task 3.094s
ok git.yuntuo.hk/k2safe/go-omni/telemetry 3.089s
ok git.yuntuo.hk/k2safe/go-omni/transport 3.311s
ok git.yuntuo.hk/k2safe/go-omni/transport/http2 14.938s
? git.yuntuo.hk/k2safe/go-omni/transport/internal/routing [no test files]
ok git.yuntuo.hk/k2safe/go-omni/transport/native 4.422s
ok git.yuntuo.hk/k2safe/go-omni/transport/nethttp 3.401s
? git.yuntuo.hk/k2safe/go-omni/transport/transporttest [no test files]
? git.yuntuo.hk/k2safe/go-omni/validate [no test files]

117
reports/copyxRemote.json Normal file
View File

@@ -0,0 +1,117 @@
{
"passed": true,
"frameworkVersion": "v0.19.0-preview",
"frameworkSum": "h1:pMMAyBOgKpGvU8WDxKIwvMU4jWf72cx0aKHVO+snFE4=",
"freshModuleCache": true,
"noLocalReplace": true,
"durationSeconds": 62.46,
"checks": [
"remoteCLIInstall",
"freshModuleCache",
"fixedVersionNoReplace",
"contractChangePreview",
"persistentLogicSignatureFailure",
"businessSourcePreserved",
"fixedConsumerBuild",
"rootGoRun",
"nethttpAndNativeSignals",
"listenerReleased",
"default404And405",
"goOwnedErrorCatalog",
"unwrappedDownloads",
"rawCallback",
"multipart400And413",
"nethttpAndNativeTransfers",
"sharedGoCatalogExport",
"documentOnlyErrorSelection",
"workspaceCheckNoWrites",
"staleCatalogRejected",
"twoIndependentServiceBuilds",
"initializationAndListenFailureCleanup",
"drainBeforeDependencyClose",
"generatedConsumerRaceTests",
"errorParamsAndInheritance",
"manualHooksAndScalars",
"dependencyFatalShutdown",
"fatalProcessExitCode",
"initializationShutdownNotification",
"configuredBuiltinEndpoints",
"projectHealthEnvelope",
"endpointConflictCleanup",
"responseArrayNullableNull",
"primitiveAndDynamicArrays",
"strictRequestPolicies",
"declarativeMultipartBinding",
"backgroundStopDrainResourceOrder",
"backgroundInitAndListenFailureCleanup",
"backgroundTimeoutRetainsResourcesAndExits",
"flatFieldComposition",
"copyxThreeEndpointConsumer",
"explicitNullAndMoneyPolicies",
"fieldAdditionWithoutCopyRuleChanges",
"middlewareFactoryInjectionAndCleanup",
"optionalRepositoryPreservesBusiness"
],
"sourceHashes": {
"lifecycle/runtime.go": "7d890150b931cdc6c5d03f3706ea09b1d220a958d91fbd6ddc6715a9ec13a5d2",
"lifecycle/phases.go": "1436856c708d004ebb28704a867316bf7a44553de56b3e29152d41475a2f793a",
"config/lifecycle_test.go": "8e5fccf0f7432c64d7fa8a52687002307d3ebb1fd039adb6ae09da0c9a2891bf",
"task/task.go": "acb1a2cc0e26029331065545582c30e0c547e3b6869f7d8b16c7d25ab0077545",
"cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "e8575c04b0923d924d96e4334731313076284fd068c4990715b7a07a2724f969",
"cmd/go-omni/apigen/templates.go": "980856557a247784a3d8e63212bf7fcc3965bcbf7689119f8ee999ccd7867f26",
"config/project.go": "0917888f4e471761b897ddd59426eb098bf586edb5c5dc856320b84b3bdb5f7f",
"contract/bind.go": "7590c3100a9bf90236aba7fda0a021d065d9dacd8c432c8eed1066f2ed8d37c8",
"contract/policy.go": "73a3a55fb88195b32f22b2bc066d9b13dccd7fd520e98c5a1b2979a3fd2447eb",
"contract/multipart.go": "db0377599b1b8b39246d41129513e8187109e44c1892971244d581c22ccc8ecd",
"contract/file.go": "20dd2de17e4854d452afc3c65f0293d504fb8fc25e91583ef45207fdab30948d",
"cmd/go-omni/apigen/business_test.go": "3b484352282d945f5a58047682079e3d64d4b0bf6ec6bafc23151b1197bb3942",
"cmd/go-omni/apigen/generate.go": "c4bc6652784714f7cb0fd4815094a1668f230b244827fc33cb8f4aa90bbc76fc",
"lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31",
"config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d",
"health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866",
"httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb",
"app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0",
"app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73",
"scripts/verifyManagedRelease.py": "fdc33d4357d501a9bf37d3612fe4fc51aa2a3a5686a475328ad62f1663eab022",
"scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18",
"cmd/go-omni/apigen/managed_test.go": "e0b2b5aa6c4d6383590b467d57d7b4e00505ca35d6b6cde2475e4ad574041e94",
"cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247",
"cmd/go-omni/apigen/testdata/managed/background.go": "fb357b5deefd5187f2df406be5189fc4d57da857bd55b474f4726bf065c77167",
"cmd/go-omni/apigen/testdata/managed/client.go": "cdd7d394ddad4bcdc10224890a884aa513ed2ffd8275ecdad8de9963dd10986a",
"cmd/go-omni/apigen/testdata/managed/phases_test.go": "4d826d7133027ac2399f8209f03efa46ed54aa45dcd0a35274a9009314941dca",
"cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319",
"cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f",
"cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1",
"cmd/go-omni/apigen/testdata/business/accessFactory.go": "2f982cf39b5e1a4be98596ce5d0a296af0fbf8c94616543a268e67ec376f31b4",
"cmd/go-omni/apigen/testdata/business/client.go": "d4cbaeec18dfb3a0e2821217e8eb0515000e351a745a93ec811dabca44e58c8a",
"cmd/go-omni/apigen/testdata/business/consumer_test.go": "e4159862a729ca2c5c9699840cc16a56350fc2914c43841f08946e9a6e9c8091",
"cmd/go-omni/apigen/testdata/business/operations.go": "11ded7921a0753f22975c525e334e3d45366143625e41d24fe500504e785c64c",
"cmd/go-omni/apigen/responseShape.go": "5b705130b66a3d4aaed842025b75abe54e6427acac8366366ca88ae9862429e1",
"cmd/go-omni/apigen/documentRefs.go": "7b841a668154d872493104eda0f9049340c67d9323eb99f1621a1ef6ba343891",
"cmd/go-omni/apigen/catalogDocument.go": "94df60328b2f105c35b73d9c6362ea14abadce6cb18e4d9921193d2d46854252",
"cmd/go-omni/apigen/workspaceCatalog.go": "c00784220b379ed80deb07c060532b756cbc47434ecb8b1ca10bc12b12159328",
"config/env.go": "ea84bccdcd39e3f6b4958a0c8ac8e7d6a0be516b24930f93039248165361bf35",
"cmd/go-omni/apigen/arrayResponse_test.go": "9e3e4475f76683d3346d349cdfa94e9e75cf913ac6979bd7a4ed86628466d879",
"cmd/go-omni/apigen/testdata/arrays/values.go": "d5e7f3adf12aabb301f46c296e8c894a7b2f6f70a6b6e7e02c74da7f1fd17bc5",
"cmd/go-omni/apigen/testdata/arrays/consumer_test.go": "435932f2952ad897943e2cda83b04eccb2950579d8d88772661541e1e2a2333d",
"cmd/go-omni/apigen/composition.go": "7ed3761b27d9c932d8ddc43e63c2aca798bdbe38f2b40571af8916c8401ea650",
"cmd/go-omni/apigen/generatorConfig.go": "78db0c7808cf6ec997c5b64a4a5fbea11b74049709bc5900ccfbc5d8dabebbc8",
"cmd/go-omni/apigen/copyConsumer_test.go": "e08ae4daed5dd25ab89d8872f653caf19499984dcaddc9835c13884c410fc0c3",
"cmd/go-omni/apigen/generatorConfig_test.go": "a6d8c8ebea5f1accf0e37dd044e57b48e422821fe75086d8d2eb3abe3acb147c",
"cmd/go-omni/apigen/middlewareFactory.go": "cc0220edb3eff25d7f66b2c37e206455f8730a2b909613713322bee7caf72303",
"cmd/go-omni/apigen/testdata/composition/accessFactory.go": "e1e5057ff79f806b3218e93249e8191d097b5c4844977204a8c32848443cba33",
"cmd/go-omni/apigen/testdata/composition/domain.go": "ab27ebd7ad5e3ed5505a2aa268b9b5ceedbb01477b59011bec63bb270114078b",
"cmd/go-omni/apigen/testdata/composition/amount.go": "f2d1834502d71d9ecf72ebe3b1077e5ef41f44fd816abd4617ac19f9bb962fdd",
"cmd/go-omni/apigen/testdata/composition/copy.go": "32d37d6f1f0e9aa2409aeb90b052801d1cdca70e504c2dd294cdae01422c9cb7",
"cmd/go-omni/apigen/testdata/composition/client.go": "19102e191c0471cae9e7405623906d0d804865c40b80ff1d5b240a41ceab6a4e",
"cmd/go-omni/apigen/testdata/composition/service.api": "01d230d5409756e2f3eb602a7fe065efa7d9599f7a4089da4bf8f07a2474444e",
"cmd/go-omni/apigen/testdata/composition/consumer_test.go": "74aaf60318cd0de67495fc9142b3f67c3b6e232bb48b46b3e134de5aadd225ad",
"cmd/go-omni/apigen/testdata/composition/dependencies.go": "edc153204b06da00c0e45c52db43a4a22f8ffe22c4a6d3f3ebec0137b861f84d",
"cmd/go-omni/apigen/testdata/composition/converters.go": "ecdcfe31c75ca815dab8d1a82533b12cbbd40e12b3a5c444e815097f6c4df08e",
"copyx/options.go": "edea2f3d2043bf813e56dedf0aea319aaa52301d3b589f41fef986e33a77dc32",
"copyx/copy.go": "a752a81acc0e847158043daaf0fdaaecec43abaf27d5561725a47b3e120ffbdc",
"copyx/graph.go": "ae6a2a11a8175033d327b580cd570c97fe9d2ddb814bb50884cc19a90766badd",
"copyx/benchmark_test.go": "b1bf408957cadea47b47215ffb099d140d5d1a2eb061024a6d0c1d769f469a79",
"copyx/copy_test.go": "89b65a6281591d7cd5823cc1c7d2c384c6bd446ef06aa1763ddc3acaa84dfb4a"
}
}

View File

@@ -23,7 +23,21 @@
[实际进程验收](generationExperienceLocal.json):根目录 go run/go build、重复生成、生成检查、业务保留、nethttp/native 的 SIGINT/SIGTERM、监听端口释放、下载/原始回调/multipart、错误目录与默认 404/405 通过,耗时 4.99 秒;临时进程和目录均由脚本清理。
现有 `examples/managedService` 更新生成配置基线后 `generate --check` 与模块编译测试通过。远程发布验收将在固定标签发布后补充;本轮不涉及 eTax 业务迁移。
现有 `examples/managedService` 更新生成配置基线后 `generate --check` 与模块编译测试通过。本轮不涉及 eTax 业务迁移。
## 远程发布验收
`v0.17.0-preview` 标签固定在 `8b588a24cd653dac77dd68f6777e0db367981f19`,已推送 main 与标签。使用空模块缓存从远程安装 CLI;框架校验和为 `h1:CH1axmZEcf8sMTHp3dRVAN0DMmkg9nURFd22H9LXASQ=`,所有消费者固定同版且没有本地 replace。
[远程验收报告](generationExperienceRemote.json):46 项检查通过,耗时 44.56 秒。真实进程验证根目录启动、nethttp/native 信号退出、下载、multipart、原始回调、默认错误响应和共享错误目录;7 组实际生成消费者通过 race,包括公共字段/结构映射/工厂依赖与失败清理/Repository 开关,以及原有参数、错误目录和生命周期行为。参数不兼容或映射遗漏时写入前失败;重新生成保留业务实现,`--check` 保持只读。
复验命令:
```sh
python3 scripts/verifyManagedRelease.py --version v0.17.0-preview --report /tmp/goOmniRemote.json
```
验收临时目录和服务进程已清理。远程报告以独立提交补充,发布标签不移动。
## 使用边界

View File

@@ -0,0 +1,114 @@
{
"passed": true,
"frameworkVersion": "v0.17.0-preview",
"frameworkSum": "h1:CH1axmZEcf8sMTHp3dRVAN0DMmkg9nURFd22H9LXASQ=",
"freshModuleCache": true,
"noLocalReplace": true,
"durationSeconds": 44.56,
"checks": [
"remoteCLIInstall",
"freshModuleCache",
"fixedVersionNoReplace",
"contractChangePreview",
"persistentLogicSignatureFailure",
"businessSourcePreserved",
"fixedConsumerBuild",
"rootGoRun",
"nethttpAndNativeSignals",
"listenerReleased",
"default404And405",
"goOwnedErrorCatalog",
"unwrappedDownloads",
"rawCallback",
"multipart400And413",
"nethttpAndNativeTransfers",
"sharedGoCatalogExport",
"documentOnlyErrorSelection",
"workspaceCheckNoWrites",
"staleCatalogRejected",
"twoIndependentServiceBuilds",
"initializationAndListenFailureCleanup",
"drainBeforeDependencyClose",
"generatedConsumerRaceTests",
"errorParamsAndInheritance",
"manualHooksAndScalars",
"dependencyFatalShutdown",
"fatalProcessExitCode",
"initializationShutdownNotification",
"configuredBuiltinEndpoints",
"projectHealthEnvelope",
"endpointConflictCleanup",
"responseArrayNullableNull",
"primitiveAndDynamicArrays",
"strictRequestPolicies",
"declarativeMultipartBinding",
"backgroundStopDrainResourceOrder",
"backgroundInitAndListenFailureCleanup",
"backgroundTimeoutRetainsResourcesAndExits",
"flatFieldComposition",
"typedStructureMapping",
"explicitNullAndMoneyPolicies",
"mappingFailureBeforeWrites",
"mappingConsumerRaceTests",
"middlewareFactoryInjectionAndCleanup",
"optionalRepositoryPreservesBusiness"
],
"sourceHashes": {
"lifecycle/runtime.go": "7d890150b931cdc6c5d03f3706ea09b1d220a958d91fbd6ddc6715a9ec13a5d2",
"lifecycle/phases.go": "1436856c708d004ebb28704a867316bf7a44553de56b3e29152d41475a2f793a",
"config/lifecycle_test.go": "8e5fccf0f7432c64d7fa8a52687002307d3ebb1fd039adb6ae09da0c9a2891bf",
"task/task.go": "acb1a2cc0e26029331065545582c30e0c547e3b6869f7d8b16c7d25ab0077545",
"cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "e8575c04b0923d924d96e4334731313076284fd068c4990715b7a07a2724f969",
"cmd/go-omni/apigen/templates.go": "e3177491934aef8cec783bed4cbeb2cfa3cb9b0815c9f7f9c63b85270bb07d24",
"config/project.go": "0917888f4e471761b897ddd59426eb098bf586edb5c5dc856320b84b3bdb5f7f",
"contract/bind.go": "7590c3100a9bf90236aba7fda0a021d065d9dacd8c432c8eed1066f2ed8d37c8",
"contract/policy.go": "73a3a55fb88195b32f22b2bc066d9b13dccd7fd520e98c5a1b2979a3fd2447eb",
"contract/multipart.go": "db0377599b1b8b39246d41129513e8187109e44c1892971244d581c22ccc8ecd",
"contract/file.go": "20dd2de17e4854d452afc3c65f0293d504fb8fc25e91583ef45207fdab30948d",
"cmd/go-omni/apigen/business_test.go": "3b484352282d945f5a58047682079e3d64d4b0bf6ec6bafc23151b1197bb3942",
"cmd/go-omni/apigen/generate.go": "5a25134afc402c62ca6afe04320b5ef14141a5ff4caedb9a5eeca4952b302f03",
"lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31",
"config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d",
"health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866",
"httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb",
"app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0",
"app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73",
"scripts/verifyManagedRelease.py": "3a736a308d9dbe3a51555e2864dbcb39dc38cd01bd4e103095f4305a41bde958",
"scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18",
"cmd/go-omni/apigen/managed_test.go": "52c8ab8a0919c6503cb8835eb258f91f237204c9fc13d9dc735bb913944168b0",
"cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247",
"cmd/go-omni/apigen/testdata/managed/background.go": "fb357b5deefd5187f2df406be5189fc4d57da857bd55b474f4726bf065c77167",
"cmd/go-omni/apigen/testdata/managed/client.go": "cdd7d394ddad4bcdc10224890a884aa513ed2ffd8275ecdad8de9963dd10986a",
"cmd/go-omni/apigen/testdata/managed/phases_test.go": "4d826d7133027ac2399f8209f03efa46ed54aa45dcd0a35274a9009314941dca",
"cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319",
"cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f",
"cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1",
"cmd/go-omni/apigen/testdata/business/accessFactory.go": "2f982cf39b5e1a4be98596ce5d0a296af0fbf8c94616543a268e67ec376f31b4",
"cmd/go-omni/apigen/testdata/business/client.go": "d4cbaeec18dfb3a0e2821217e8eb0515000e351a745a93ec811dabca44e58c8a",
"cmd/go-omni/apigen/testdata/business/consumer_test.go": "e4159862a729ca2c5c9699840cc16a56350fc2914c43841f08946e9a6e9c8091",
"cmd/go-omni/apigen/testdata/business/operations.go": "11ded7921a0753f22975c525e334e3d45366143625e41d24fe500504e785c64c",
"cmd/go-omni/apigen/responseShape.go": "5b705130b66a3d4aaed842025b75abe54e6427acac8366366ca88ae9862429e1",
"cmd/go-omni/apigen/documentRefs.go": "7b841a668154d872493104eda0f9049340c67d9323eb99f1621a1ef6ba343891",
"cmd/go-omni/apigen/catalogDocument.go": "94df60328b2f105c35b73d9c6362ea14abadce6cb18e4d9921193d2d46854252",
"cmd/go-omni/apigen/workspaceCatalog.go": "c00784220b379ed80deb07c060532b756cbc47434ecb8b1ca10bc12b12159328",
"config/env.go": "ea84bccdcd39e3f6b4958a0c8ac8e7d6a0be516b24930f93039248165361bf35",
"cmd/go-omni/apigen/arrayResponse_test.go": "9e3e4475f76683d3346d349cdfa94e9e75cf913ac6979bd7a4ed86628466d879",
"cmd/go-omni/apigen/testdata/arrays/values.go": "d5e7f3adf12aabb301f46c296e8c894a7b2f6f70a6b6e7e02c74da7f1fd17bc5",
"cmd/go-omni/apigen/testdata/arrays/consumer_test.go": "435932f2952ad897943e2cda83b04eccb2950579d8d88772661541e1e2a2333d",
"cmd/go-omni/apigen/composition.go": "7ed3761b27d9c932d8ddc43e63c2aca798bdbe38f2b40571af8916c8401ea650",
"cmd/go-omni/apigen/generatorConfig.go": "467112eeb5180fc1571e811e38e90e6dee3f128fe39e17bb978591cee0fa1b22",
"cmd/go-omni/apigen/mapping.go": "1d28f3b1300f50654b392735be546ff9c588664bfc0563e4ff5f838f495e091e",
"cmd/go-omni/apigen/mappingLoad.go": "b301bd49e4b67160ba70be4163c8ab1f3620f5963ec26f4b28af5df185a75170",
"cmd/go-omni/apigen/mapping_test.go": "6660f3183e6078ae5d6baecc654b0d2bcdfc243b6bc1da28811af8d25fa73efa",
"cmd/go-omni/apigen/middlewareFactory.go": "cc0220edb3eff25d7f66b2c37e206455f8730a2b909613713322bee7caf72303",
"cmd/go-omni/apigen/testdata/composition/accessFactory.go": "e1e5057ff79f806b3218e93249e8191d097b5c4844977204a8c32848443cba33",
"cmd/go-omni/apigen/testdata/composition/domain.go": "19bc978d173081f18efda643ffad3510287c03b4e47436c7019ea91973f32b72",
"cmd/go-omni/apigen/testdata/composition/amount.go": "f2d1834502d71d9ecf72ebe3b1077e5ef41f44fd816abd4617ac19f9bb962fdd",
"cmd/go-omni/apigen/testdata/composition/client.go": "19102e191c0471cae9e7405623906d0d804865c40b80ff1d5b240a41ceab6a4e",
"cmd/go-omni/apigen/testdata/composition/service.api": "01d230d5409756e2f3eb602a7fe065efa7d9599f7a4089da4bf8f07a2474444e",
"cmd/go-omni/apigen/testdata/composition/consumer_test.go": "9021c47c8af7d0c338c005d9f0a1f97ecdaaffc0b17b01c09d58023cc8c69fd7",
"cmd/go-omni/apigen/testdata/composition/dependencies.go": "edc153204b06da00c0e45c52db43a4a22f8ffe22c4a6d3f3ebec0137b861f84d",
"cmd/go-omni/apigen/testdata/composition/converters.go": "3ded306c68182d0c2184e44eecb6307c366f0ae43a4d23a6a3ef70fc14f45833"
},
"releaseCommit": "8b588a24cd653dac77dd68f6777e0db367981f19"
}

View File

@@ -0,0 +1,52 @@
# 可空映射与配置拆分验收(v0.18.0-preview)
基于 v0.17.0-preview 补齐通用生成能力,未添加业务模型、权限、金额默认值或领域校验。运行时和依赖不变,旧单文件配置与已有映射函数保留。
## 本轮完成
- 显式嵌套 mapping 支持 `S → T`、`*S → *T`、`S → *T`、显式 nil 策略下的 `*S → T`,也适用于 elements 下的 slice/等长数组。默认保留可空对象;非空目标必须明确选 zero/error。
- 同类型 `*T → T` 可显式选择 nil zero/error,省去重复解引用 converter。自定义 converter 继续独立控制行为,禁止同时配置 nil 策略。
- `pointerResult: true` 额外生成 `<Name>Ptr(S) (*T,error)`,原值函数不变。转换失败返回 nil 与原错误链;辅助函数名统一查重。
- 主配置以 mappingFiles 列出项目内子文件。子文件只含 mappings,统一解析跨文件引用、严格类型和字段覆盖;不做递归导入或隐式覆盖。
- 配置均为项目文件,生成不会改写。check、dry-run、workspace generate/check 共享加载过程;对子文件变化重新核对,保留边界与大小限制。
## 回归证据
[回归记录](nullableMappingRegression.json):全仓库 44 个有测试的包通过 `-race`,包含实际生成消费者;go vet、go build、diff 检查通过。
`TestNullableMappingConsumer` 生成独立项目,配置跨文件向前引用与项目自定义转换函数,验证:
- nil、缺失、非 nil 零值对象、nil/非 nil 空 slice、nil 元素,以及 slice/array 内的嵌套转换。
- zero/error 空值策略、值到指针、可选指针返回;未显式映射的同类型指针仍共享引用。
- 子转换失败时父结果归零,指针辅助返回 nil,errors.Is 保留原因,错误含子字段与数组索引;内部原因和映射路径不进入公开 HTTP 提示。
- 32 个并发调用;通过实际生成 Handler 和 Logic 测试 JSON 请求与响应;生成模块执行 race 测试和根目录 build。
- 主配置与子文件、业务 Logic 在重新生成后逐字保留;check/preview 只读。子文件修改触发漂移,workspace generate 后 workspace check 通过。
- 未声明 nil 行为、错误类型/策略、辅助函数重名、未知映射、跨文件循环和字段遗漏均在写入前拒绝。
`TestMappingFileConfiguration/Rejection/Bounds` 覆盖文件加载顺序、来源诊断、配置变化摘要、重复文件/定义/JSON key、路径越界、符号链接、嵌套导入拒绝,以及单文件/总大小/文件数/合并映射数限制。
[原始最小复现](nullableMappingRepro.json):在隔离副本中,用源码构建的真实 CLI 成功生成原来报 `nested mapping Child has incompatible types` 的项目;生成检查、模块编译测试和 build 通过。原项目文件快照一致,临时目录已清理。
[实际进程回归](nullableMappingLocal.json):原有根目录启动、nethttp/native 信号退出、监听端口释放、上传下载、原始回调、错误响应与生成文件保护通过。
## 使用与边界
使用及迁移见 [生成能力指南](../GENERATION.md)。子文件仅拆分配置,不生成需要人工补齐的赋值代码;普通映射仍为编译器检查的 Go 代码,无运行时反射或 JSON 往返。
zero 表示目标 Go 零值,并跳过子映射;领域必填、金额精度、时间/编号格式和快照深拷贝仍由项目负责。普通指针输入已合并的 missing/null 不能在映射阶段恢复。nil error 是普通转换错误,特定业务错误码用自定义 converter 表达。
本版已完成固定标签的远程发布验收。框架验收不代表完整业务项目已迁移,本轮没有修改 eTax 业务代码。
## 远程发布验收
`v0.18.0-preview` 标签固定在 `4563a6354fac19f22cd8d407fd76d98b8cc9c406`,main 与标签已推送。空模块缓存从远程安装 CLI,所有实际生成消费者固定同版框架且没有本地 replace;校验和为 `h1:VV1T2i+TG5SmXfAO8ii3y0QunNXbKA8x8YR319CWsHg=`。
[远程报告](nullableMappingRemote.json):53 项检查通过,耗时 51.27 秒。8 组生成消费者通过 race,包括可空映射、配置拆分、显式 nil 策略、指针辅助、错误链、检查/预览只读、workspace 再生成及旧能力回归。配置拆分验收的 generate、check、dry-run 和 workspace 操作均使用远程安装的 CLI。临时目录与服务进程已清理。
复验命令:
```sh
python3 scripts/verifyManagedRelease.py --version v0.18.0-preview --report /tmp/goOmniRemote.json
```
验收报告以独立提交补充,发布标签保持不变。

View File

@@ -0,0 +1,59 @@
{
"passed": true,
"mode": "localSource",
"durationSeconds": 5.28,
"checks": [
"rootGoRun",
"rootGoBuild",
"stableRegeneration",
"checkMainOwnership",
"contractChangePreview",
"logicSignatureCheckPersists",
"businessSourcePreserved",
"fixedConsumerBuild",
"nethttpSIGINT",
"nethttpSIGTERM",
"nativeSIGINT",
"nativeSIGTERM",
"listenerReleased",
"temporaryProcessesRemoved",
"nethttpGeneratedTransfers",
"nativeGeneratedTransfers",
"malformedMultipart400",
"goOwnedErrors",
"default404And405"
],
"sourceHashes": {
"cmd/go-omni/apigen/generate.go": "a977803a6f8f4c64a30bc8156decfb072301f4084a5cf8821f4fe98da80cd7af",
"cmd/go-omni/apigen/templates.go": "e3177491934aef8cec783bed4cbeb2cfa3cb9b0815c9f7f9c63b85270bb07d24",
"cmd/go-omni/apigen/startup.go": "e3322fa3e2aff9f45d66427f44187535c7c0960edc0f5fdfd5d7b4943b349334",
"cmd/go-omni/apigen/managed_test.go": "e0b2b5aa6c4d6383590b467d57d7b4e00505ca35d6b6cde2475e4ad574041e94",
"cmd/go-omni/apigen/contractChanges.go": "aeb83990e908738faaa3cdfa20c3b8938023db05b6e55f2d24352c44144056a8",
"cmd/go-omni/apigen/logicCheck.go": "ca1af4f876a3a0db7b7b9ba9b82c1d6399fd89d29f667dad39cfd3de526eb1cb",
"cmd/go-omni/apigen/write.go": "eab29555906c1d6675ffdf07a30f00536065ec8d406cc1a4e53a05d8db720ff8",
"cmd/go-omni/api.go": "5203a17c5f66977a9eba838d61cf5a7356de53d5f72a1196fd4f2046aa43ac85",
"config/config.go": "b4935ebb452c9df34b733842687ba11eec7e006a54ce42ff767273dd74d6e672",
"httpx/transfer.go": "c9fe4e98c0e599d367eaf19ebecf367db914edfca9ad3a65527496d1dea80a3d",
"httpx/multipart.go": "b942ede7a810b268cd43b395c3bddcfaaacad92812bbb60e653466b8d86bc875",
"scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18",
"lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31",
"health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866",
"config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d",
"httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb",
"app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0",
"app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73",
"cmd/go-omni/scaffold/templates/api-config.go.tmpl": "715d29a9be33ba232eac339547cc0a13ae09c8bcb38a65a69a4f32f09fea5346",
"cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "e8575c04b0923d924d96e4334731313076284fd068c4990715b7a07a2724f969",
"cmd/go-omni/scaffold/templates/api-close.go.tmpl": "3f22d272e92f3b64b62123b3d80ec1a6ce8834257d64752fd75facf12623902b",
"cmd/go-omni/scaffold/templates/api-databaseOptions.go.tmpl": "6e5395b4a787c77a6cabfd85f16049a39578d6ec751a9b78203f3a03baabdf28",
"cmd/go-omni/scaffold/templates/api-options.go.tmpl": "11fe34ce45e8c98f7b5c9c0a2c512eb0b3abf86f95b4b576e083bcb6f5d0f824",
"cmd/go-omni/scaffold/templates/api-context.go.tmpl": "6cf67d37ef64f388b23a98781c3ee3abb71a2e0b9e71eeb70f1e5492ff7e32b4",
"cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247",
"cmd/go-omni/apigen/testdata/managed/background.go": "fb357b5deefd5187f2df406be5189fc4d57da857bd55b474f4726bf065c77167",
"cmd/go-omni/apigen/testdata/managed/client.go": "cdd7d394ddad4bcdc10224890a884aa513ed2ffd8275ecdad8de9963dd10986a",
"cmd/go-omni/apigen/testdata/managed/phases_test.go": "4d826d7133027ac2399f8209f03efa46ed54aa45dcd0a35274a9009314941dca",
"cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319",
"cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f",
"cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1"
}
}

View File

@@ -0,0 +1,79 @@
{
"passed": true,
"status": "releaseCandidateValidated",
"baseRelease": "v0.17.0-preview",
"testedPackages": 44,
"goVersion": "go version go1.27.1 darwin/arm64",
"commands": [
"GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./... -count=1 -timeout=300s",
"go vet ./...",
"go build ./...",
"python3 scripts/verifyManagedProject.py --report reports/nullableMappingLocal.json",
"source CLI: api generate / api generate --check / go mod tidy / go test ./... / go build . on isolated original reproduction",
"git diff --check",
"go run ./cmd/go-omni api generate --dir examples/managedService --check",
"cd examples/managedService && go test ./..."
],
"checks": [
"nullableNestedObject",
"nilPointerPreserved",
"nilAndEmptySlices",
"nullableArrayElements",
"explicitNilZeroAndError",
"nonNilZeroPreserved",
"nestedFailureReturnsZero",
"errorChainAndFieldIndex",
"privateCauseNotInHTTP",
"pointerResultHelper",
"ordinaryPointerAliasingUnchanged",
"concurrentCalls",
"splitConfigForwardReferences",
"readOnlyGenerateCheckAndPreview",
"workspaceGenerateCheck",
"childFileDriftDetection",
"businessFilesPreserved",
"duplicateMappingsAndHelpersRejected",
"cyclesAndUnknownMappingsRejected",
"invalidPoliciesRejectedBeforeWrites",
"symlinksAndTraversalRejected",
"configSizeAndCountLimits",
"configChangeDigest",
"originalFeedbackReproducedAndFixed"
],
"logs": {
"fullRace": {
"path": "/tmp/go-omni-v018-race.log",
"sha256": "1f0f9a55354c041c0035ae82e0f21a2a913544d46ae40e05795a9ae5b707250f"
},
"vet": {
"path": "/tmp/go-omni-v018-vet.log",
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
},
"build": {
"path": "/tmp/go-omni-v018-build.log",
"sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
},
"process": {
"path": "/tmp/go-omni-v018-process.log",
"sha256": "748bf31b88405f7f1d550b1d74bbbc5543ea09c5d09b59ec98da6ad040694b12"
}
},
"sourceHashes": {
"cmd/go-omni/apigen/generate.go": "a977803a6f8f4c64a30bc8156decfb072301f4084a5cf8821f4fe98da80cd7af",
"cmd/go-omni/apigen/generatorConfig.go": "6eaa70821312ce670e5f8067f99f5133d1789e8558140203520230179da083d0",
"cmd/go-omni/apigen/generatorConfig_test.go": "f22a55dfa5c8188cf7bce56a2b53a778cbe3feb581cd551f16ebfc6a10932e21",
"cmd/go-omni/apigen/managed_test.go": "e0b2b5aa6c4d6383590b467d57d7b4e00505ca35d6b6cde2475e4ad574041e94",
"cmd/go-omni/apigen/mapping.go": "fb4f4bf39137921c3fc8f793c636c8c29f68e962060a56c82b10869d6a200243",
"cmd/go-omni/apigen/mappingNullable_test.go": "cc22182e5a68138b97817f7dda7a8a5a10ca3aab6c690f8926b6a6cc2ba2a8cb",
"cmd/go-omni/apigen/mappingPointer.go": "5b4aab84c2ea9371d2a295f8d8cc9a7f82a88b412daa07970c9ac9ea32e504e6",
"cmd/go-omni/apigen/mapping_test.go": "94a7927e2237f60c157a7dfc4cc793e17537559ed65c1468d3b8b8b8e3f39394",
"cmd/go-omni/apigen/testdata/nullable/consumer_test.go": "5b6353c09fe61c39602071cc7072b6d640cf1a6cab1ae55f9490199ca62102b2",
"cmd/go-omni/apigen/testdata/nullable/convert.go": "4465a51d0eb7efde66eca08ab7e0e8fd6565e04a8447c2d6d8c65cea47c5a87f",
"cmd/go-omni/apigen/testdata/nullable/domain.go": "cb31f4939efbc9ddbb9b68a9e0c72302b93c2c365b1023f65bf7079862936a76",
"cmd/go-omni/apigen/testdata/nullable/service.api": "6a5e76d1b56efc5e89d89579710b06465a7c18accbd08ed58d88ccb9c59828d9",
"cmd/go-omni/main_test.go": "0c709d4f8c81df91e3ad318afc02f1534a1062218dbbb10adbaa5a28c6cb2021",
"cmd/go-omni/scaffold/version.go": "dc875b37247ebea3890cd3c6ecf614875301ca2d3c92036abd313dea6d29418c",
"scripts/verifyManagedRelease.py": "28d883e933a131e8df8e798b00e4b4a6af097a9b880ef1aa7b753456a90c6137"
},
"frameworkVersion": "v0.18.0-preview"
}

View File

@@ -0,0 +1,128 @@
{
"passed": true,
"frameworkVersion": "v0.18.0-preview",
"frameworkSum": "h1:VV1T2i+TG5SmXfAO8ii3y0QunNXbKA8x8YR319CWsHg=",
"freshModuleCache": true,
"noLocalReplace": true,
"durationSeconds": 51.27,
"checks": [
"remoteCLIInstall",
"freshModuleCache",
"fixedVersionNoReplace",
"contractChangePreview",
"persistentLogicSignatureFailure",
"businessSourcePreserved",
"fixedConsumerBuild",
"rootGoRun",
"nethttpAndNativeSignals",
"listenerReleased",
"default404And405",
"goOwnedErrorCatalog",
"unwrappedDownloads",
"rawCallback",
"multipart400And413",
"nethttpAndNativeTransfers",
"sharedGoCatalogExport",
"documentOnlyErrorSelection",
"workspaceCheckNoWrites",
"staleCatalogRejected",
"twoIndependentServiceBuilds",
"initializationAndListenFailureCleanup",
"drainBeforeDependencyClose",
"generatedConsumerRaceTests",
"errorParamsAndInheritance",
"manualHooksAndScalars",
"dependencyFatalShutdown",
"fatalProcessExitCode",
"initializationShutdownNotification",
"configuredBuiltinEndpoints",
"projectHealthEnvelope",
"endpointConflictCleanup",
"responseArrayNullableNull",
"primitiveAndDynamicArrays",
"strictRequestPolicies",
"declarativeMultipartBinding",
"backgroundStopDrainResourceOrder",
"backgroundInitAndListenFailureCleanup",
"backgroundTimeoutRetainsResourcesAndExits",
"flatFieldComposition",
"typedStructureMapping",
"explicitNullAndMoneyPolicies",
"mappingFailureBeforeWrites",
"mappingConsumerRaceTests",
"middlewareFactoryInjectionAndCleanup",
"optionalRepositoryPreservesBusiness",
"nullableNestedMappings",
"splitMappingFiles",
"explicitNilPolicies",
"pointerResultHelpers",
"mappingWorkspaceRegeneration",
"nestedErrorChainAndPrivateCause",
"mappingPreviewNoWrites"
],
"sourceHashes": {
"lifecycle/runtime.go": "7d890150b931cdc6c5d03f3706ea09b1d220a958d91fbd6ddc6715a9ec13a5d2",
"lifecycle/phases.go": "1436856c708d004ebb28704a867316bf7a44553de56b3e29152d41475a2f793a",
"config/lifecycle_test.go": "8e5fccf0f7432c64d7fa8a52687002307d3ebb1fd039adb6ae09da0c9a2891bf",
"task/task.go": "acb1a2cc0e26029331065545582c30e0c547e3b6869f7d8b16c7d25ab0077545",
"cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "e8575c04b0923d924d96e4334731313076284fd068c4990715b7a07a2724f969",
"cmd/go-omni/apigen/templates.go": "e3177491934aef8cec783bed4cbeb2cfa3cb9b0815c9f7f9c63b85270bb07d24",
"config/project.go": "0917888f4e471761b897ddd59426eb098bf586edb5c5dc856320b84b3bdb5f7f",
"contract/bind.go": "7590c3100a9bf90236aba7fda0a021d065d9dacd8c432c8eed1066f2ed8d37c8",
"contract/policy.go": "73a3a55fb88195b32f22b2bc066d9b13dccd7fd520e98c5a1b2979a3fd2447eb",
"contract/multipart.go": "db0377599b1b8b39246d41129513e8187109e44c1892971244d581c22ccc8ecd",
"contract/file.go": "20dd2de17e4854d452afc3c65f0293d504fb8fc25e91583ef45207fdab30948d",
"cmd/go-omni/apigen/business_test.go": "3b484352282d945f5a58047682079e3d64d4b0bf6ec6bafc23151b1197bb3942",
"cmd/go-omni/apigen/generate.go": "a977803a6f8f4c64a30bc8156decfb072301f4084a5cf8821f4fe98da80cd7af",
"lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31",
"config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d",
"health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866",
"httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb",
"app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0",
"app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73",
"scripts/verifyManagedRelease.py": "28d883e933a131e8df8e798b00e4b4a6af097a9b880ef1aa7b753456a90c6137",
"scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18",
"cmd/go-omni/apigen/managed_test.go": "e0b2b5aa6c4d6383590b467d57d7b4e00505ca35d6b6cde2475e4ad574041e94",
"cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247",
"cmd/go-omni/apigen/testdata/managed/background.go": "fb357b5deefd5187f2df406be5189fc4d57da857bd55b474f4726bf065c77167",
"cmd/go-omni/apigen/testdata/managed/client.go": "cdd7d394ddad4bcdc10224890a884aa513ed2ffd8275ecdad8de9963dd10986a",
"cmd/go-omni/apigen/testdata/managed/phases_test.go": "4d826d7133027ac2399f8209f03efa46ed54aa45dcd0a35274a9009314941dca",
"cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319",
"cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f",
"cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1",
"cmd/go-omni/apigen/testdata/business/accessFactory.go": "2f982cf39b5e1a4be98596ce5d0a296af0fbf8c94616543a268e67ec376f31b4",
"cmd/go-omni/apigen/testdata/business/client.go": "d4cbaeec18dfb3a0e2821217e8eb0515000e351a745a93ec811dabca44e58c8a",
"cmd/go-omni/apigen/testdata/business/consumer_test.go": "e4159862a729ca2c5c9699840cc16a56350fc2914c43841f08946e9a6e9c8091",
"cmd/go-omni/apigen/testdata/business/operations.go": "11ded7921a0753f22975c525e334e3d45366143625e41d24fe500504e785c64c",
"cmd/go-omni/apigen/responseShape.go": "5b705130b66a3d4aaed842025b75abe54e6427acac8366366ca88ae9862429e1",
"cmd/go-omni/apigen/documentRefs.go": "7b841a668154d872493104eda0f9049340c67d9323eb99f1621a1ef6ba343891",
"cmd/go-omni/apigen/catalogDocument.go": "94df60328b2f105c35b73d9c6362ea14abadce6cb18e4d9921193d2d46854252",
"cmd/go-omni/apigen/workspaceCatalog.go": "c00784220b379ed80deb07c060532b756cbc47434ecb8b1ca10bc12b12159328",
"config/env.go": "ea84bccdcd39e3f6b4958a0c8ac8e7d6a0be516b24930f93039248165361bf35",
"cmd/go-omni/apigen/arrayResponse_test.go": "9e3e4475f76683d3346d349cdfa94e9e75cf913ac6979bd7a4ed86628466d879",
"cmd/go-omni/apigen/testdata/arrays/values.go": "d5e7f3adf12aabb301f46c296e8c894a7b2f6f70a6b6e7e02c74da7f1fd17bc5",
"cmd/go-omni/apigen/testdata/arrays/consumer_test.go": "435932f2952ad897943e2cda83b04eccb2950579d8d88772661541e1e2a2333d",
"cmd/go-omni/apigen/composition.go": "7ed3761b27d9c932d8ddc43e63c2aca798bdbe38f2b40571af8916c8401ea650",
"cmd/go-omni/apigen/generatorConfig.go": "6eaa70821312ce670e5f8067f99f5133d1789e8558140203520230179da083d0",
"cmd/go-omni/apigen/mapping.go": "fb4f4bf39137921c3fc8f793c636c8c29f68e962060a56c82b10869d6a200243",
"cmd/go-omni/apigen/mappingLoad.go": "b301bd49e4b67160ba70be4163c8ab1f3620f5963ec26f4b28af5df185a75170",
"cmd/go-omni/apigen/mapping_test.go": "94a7927e2237f60c157a7dfc4cc793e17537559ed65c1468d3b8b8b8e3f39394",
"cmd/go-omni/apigen/middlewareFactory.go": "cc0220edb3eff25d7f66b2c37e206455f8730a2b909613713322bee7caf72303",
"cmd/go-omni/apigen/testdata/composition/accessFactory.go": "e1e5057ff79f806b3218e93249e8191d097b5c4844977204a8c32848443cba33",
"cmd/go-omni/apigen/testdata/composition/domain.go": "19bc978d173081f18efda643ffad3510287c03b4e47436c7019ea91973f32b72",
"cmd/go-omni/apigen/testdata/composition/amount.go": "f2d1834502d71d9ecf72ebe3b1077e5ef41f44fd816abd4617ac19f9bb962fdd",
"cmd/go-omni/apigen/testdata/composition/client.go": "19102e191c0471cae9e7405623906d0d804865c40b80ff1d5b240a41ceab6a4e",
"cmd/go-omni/apigen/testdata/composition/service.api": "01d230d5409756e2f3eb602a7fe065efa7d9599f7a4089da4bf8f07a2474444e",
"cmd/go-omni/apigen/testdata/composition/consumer_test.go": "9021c47c8af7d0c338c005d9f0a1f97ecdaaffc0b17b01c09d58023cc8c69fd7",
"cmd/go-omni/apigen/testdata/composition/dependencies.go": "edc153204b06da00c0e45c52db43a4a22f8ffe22c4a6d3f3ebec0137b861f84d",
"cmd/go-omni/apigen/testdata/composition/converters.go": "3ded306c68182d0c2184e44eecb6307c366f0ae43a4d23a6a3ef70fc14f45833",
"cmd/go-omni/apigen/mappingPointer.go": "5b4aab84c2ea9371d2a295f8d8cc9a7f82a88b412daa07970c9ac9ea32e504e6",
"cmd/go-omni/apigen/mappingNullable_test.go": "cc22182e5a68138b97817f7dda7a8a5a10ca3aab6c690f8926b6a6cc2ba2a8cb",
"cmd/go-omni/apigen/generatorConfig_test.go": "f22a55dfa5c8188cf7bce56a2b53a778cbe3feb581cd551f16ebfc6a10932e21",
"cmd/go-omni/apigen/testdata/nullable/domain.go": "cb31f4939efbc9ddbb9b68a9e0c72302b93c2c365b1023f65bf7079862936a76",
"cmd/go-omni/apigen/testdata/nullable/convert.go": "4465a51d0eb7efde66eca08ab7e0e8fd6565e04a8447c2d6d8c65cea47c5a87f",
"cmd/go-omni/apigen/testdata/nullable/service.api": "6a5e76d1b56efc5e89d89579710b06465a7c18accbd08ed58d88ccb9c59828d9",
"cmd/go-omni/apigen/testdata/nullable/consumer_test.go": "5b6353c09fe61c39602071cc7072b6d640cf1a6cab1ae55f9490199ca62102b2"
},
"releaseCommit": "4563a6354fac19f22cd8d407fd76d98b8cc9c406"
}

View File

@@ -0,0 +1,14 @@
{
"passed": true,
"status": "localSourceUnreleased",
"durationSeconds": 2.8,
"checks": [
"originalNullablePointerRepro",
"actualCLIGenerate",
"actualCLICheck",
"consumerCompile",
"consumerBuild",
"originalProjectUnchanged",
"temporaryResourcesRemoved"
]
}

View File

@@ -124,7 +124,7 @@ def main():
checks += ["sharedGoCatalogExport", "documentOnlyErrorSelection", "workspaceCheckNoWrites", "staleCatalogRejected", "twoIndependentServiceBuilds"]
env.update(GO_OMNI_SCAFFOLD_INTEGRATION="1", GO_OMNI_RELEASE_CLI=str(cli), GO_OMNI_RELEASE_VERSION=args.version)
tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer", "TestCompositionMappingConsumer"]
tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer", "TestCopyxThreeEndpointConsumer", "TestGeneratedConfigurationConsumer"]
print("Running generated consumers against the remote framework, including lifecycle failure cleanup", flush=True)
output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=240s", "./cmd/go-omni/apigen", "-run", "^(" + "|".join(tests) + ")$"], ROOT, env)
events = [json.loads(line) for line in output.splitlines() if line.startswith("{")]
@@ -135,15 +135,20 @@ def main():
assert len(sums) == len(tests) and all(s == dependency["Sum"] for s in sums), "remote consumer checksum mismatch"
checks += ["initializationAndListenFailureCleanup", "drainBeforeDependencyClose", "generatedConsumerRaceTests", "errorParamsAndInheritance", "manualHooksAndScalars", "dependencyFatalShutdown", "fatalProcessExitCode", "initializationShutdownNotification", "configuredBuiltinEndpoints", "projectHealthEnvelope", "endpointConflictCleanup", "responseArrayNullableNull", "primitiveAndDynamicArrays", "strictRequestPolicies", "declarativeMultipartBinding", "backgroundStopDrainResourceOrder", "backgroundInitAndListenFailureCleanup", "backgroundTimeoutRetainsResourcesAndExits"]
checks += ["flatFieldComposition", "typedStructureMapping", "explicitNullAndMoneyPolicies", "mappingFailureBeforeWrites", "mappingConsumerRaceTests", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"]
checks += ["flatFieldComposition", "copyxThreeEndpointConsumer", "explicitNullAndMoneyPolicies", "fieldAdditionWithoutCopyRuleChanges", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"]
checks += ["generatedUnifiedConfig", "jsonYamlXmlAndCustomDecoder", "configValidationBeforeDependencies", "configEnvProvenance", "configExtensionPreserved", "configLoaderDriftCheck"]
sources = ["lifecycle/runtime.go", "lifecycle/phases.go", "config/lifecycle_test.go", "task/task.go", "cmd/go-omni/scaffold/templates/api-runtime.go.tmpl", "cmd/go-omni/apigen/templates.go", "config/project.go", "contract/bind.go", "contract/policy.go", "contract/multipart.go", "contract/file.go", "cmd/go-omni/apigen/business_test.go", "cmd/go-omni/apigen/generate.go", "lifecycle/controller.go", "config/endpoints.go", "health/health.go", "httpx/response.go", "app/app.go", "app/transport.go", "scripts/verifyManagedRelease.py", "scripts/verifyManagedProject.py", "cmd/go-omni/apigen/managed_test.go"]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/managed").glob("*.go")]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/business").glob("*.go")]
sources += ["cmd/go-omni/apigen/responseShape.go", "cmd/go-omni/apigen/documentRefs.go", "cmd/go-omni/apigen/catalogDocument.go", "cmd/go-omni/apigen/workspaceCatalog.go", "config/env.go", "cmd/go-omni/apigen/arrayResponse_test.go"]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/arrays").glob("*.go")]
sources += ["cmd/go-omni/apigen/" + name for name in ("composition.go", "generatorConfig.go", "mapping.go", "mappingLoad.go", "mapping_test.go", "middlewareFactory.go")]
sources += ["cmd/go-omni/apigen/" + name for name in ("composition.go", "generatorConfig.go", "copyConsumer_test.go", "generatorConfig_test.go", "middlewareFactory.go")]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/composition").iterdir() if p.is_file()]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "copyx").glob("*.go")]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "config").glob("*.go")]
sources += ["cmd/go-omni/apigen/configLoader_test.go", "cmd/go-omni/apigen/testdata/configuration/consumer_test.go", "cmd/go-omni/scaffold/templates/api-load.go.tmpl", "cmd/go-omni/scaffold/templates/api-loader.go.tmpl"]
report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"],
"freshModuleCache": True, "noLocalReplace": True,
"durationSeconds": round(time.monotonic() - started, 2), "checks": checks,