diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..c905727 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,8 @@ +# 开发约定 + +- 框架保持通用,不内置接入项目的业务类型、字段、错误码或业务规则。 +- 2026-09-25 用户要求:每次功能开发完成,必须同步维护根 README.md。README 需要覆盖当前全部公开功能,解释用途、基本用法,并提供可运行示例或链接到 examples 内的完整示例。 +- 新增、调整和删除功能都必须同步文档、示例、测试与发布说明。删除功能要明确说明已移除内容、影响范围和迁移步骤,不能继续把历史用法当作当前能力推荐。 +- examples 使用通用场景,避免依赖业务项目、私有数据库或线上服务。必要的外部基础设施配置应明确说明。 +- 2026-09-25 用户决定:移除 mappings/mappingFiles 及结构映射代码生成机制,改用可选 copyx 工具包。不得恢复另一套 JSON 或 Go 映射声明语言。 +- 深拷贝的自定义转换结果不得与输入共享可变引用;发现共享应报错,不能降级成浅拷贝或静默接受。 diff --git a/CHANGELOG.md b/CHANGELOG.md index 6ef56c4..b928528 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,14 @@ # Changelog +## v0.19.0-preview(2026-09-25) + +- 新增通用可选 copyx:构造器、浅/深拷贝、精确类型转换、字段规则、nil/零值策略、并发复用。 +- 深拷贝拒绝自定义结果共享源可变引用;原子提交目标、保留错误链、限制遍历及估算分配。 +- 移除 JSON 结构映射配置与生成;提供明确迁移诊断,保留公共字段组合等其他生成能力。 +- 同步 README 全部能力索引、COPY.md、可运行示例和开发文档约定。 +- 验收与不兼容迁移见 [发布说明](RELEASE.md)、[copyx 验收](reports/copyxAcceptance.md)。v0.14–v0.18 的完整变更保留在发布说明历史段落。 + + ## v0.13.0-preview(2026-09-24) - 新增通用 lifecycle.Controller,svc.RequestShutdown 支持依赖主动通知正常停止或致命故障,通知不阻塞,首个故障原因保留;生成入口排空 HTTP 后关闭依赖与日志,故障退出码为 1。 diff --git a/COPY.md b/COPY.md new file mode 100644 index 0000000..43e0c75 --- /dev/null +++ b/COPY.md @@ -0,0 +1,141 @@ +# 可选拷贝工具 copyx + +v0.19 用普通 Go 调用取代 JSON 结构映射生成。`copyx` 只依赖标准库,不需要注册到 App,不执行 JSON 往返,也不会自动暴露数据库模型。普通 Logic 直接使用 `.api` 生成的 DTO;只有存在独立业务类型或需要隔离可变数据时才用它。 + +完整可运行代码:[examples/copying](examples/copying)。在框架根目录运行: + +```sh +go run ./examples/copying +go test -race ./copyx ./examples/copying +``` + +## 最简单的调用 + +```go +import "git.yuntuo.hk/k2safe/go-omni/copyx" + +type Request struct { Name string; Enabled bool } +type Input struct { Name string; Enabled bool } + +req := Request{Name: "sample", Enabled: false} +var input Input +if err := copyx.Copy(&input, req); err != nil { return err } +``` + +同名、同类型字段直接赋值。默认不会跳过 `false`、`0`、空字符串。目标字段没有来源时报错;多余源字段默认忽略,使用 `WithStrictFields()` 可要求源字段也全部使用或显式忽略。 + +默认是浅拷贝:同类型指针、map、slice 共享引用,符合普通 Go 赋值。JSON tag 不参与字段匹配。不会自动调用方法、SQL Scanner/Valuer,或把 `int32`、`int64`、string、命名标量互相强转。 + +新字段:普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时,两边各声明一次;同名同类型不增加拷贝配置。检查在运行时发生,实际类型对必须有测试,不能把运行时工具当作旧生成器的编译期字段覆盖检查。 + +## 深拷贝与构造器 + +```go +snapshot, err := copyx.Clone(source) // 保持源的静态类型,始终深拷贝 +if err != nil { return err } + +copier, err := copyx.New(copyx.WithDeepCopy(), copyx.WithStrictFields()) +if err != nil { return err } +var result Output +if err := copier.Copy(&result, source); err != nil { return err } +``` + +`Copy` 的目标必须是非 nil 指针;`Clone[T]` 返回 `(T, error)`。重复执行时在初始化阶段创建 Copier 并复用;配置之后不可变,每次调用独立持有遍历状态,可以并发使用。源、目标的并发读写仍由调用方同步,自定义函数也须支持并发。 + +深拷贝会分配指针、map、slice,递归处理结构、数组、接口中的具体值。nil 集合与非 nil 空集合保持区别;slice 结果容量为长度,不复制长度之外的隐藏元素。普通循环和重复指针可处理。循环图应保留根指针,使用 `Clone(sourcePointer)` 或 `Copy(&targetPointer, sourcePointer)`;将循环指针根压成值结构会明确报错。 + +它保证结果不引用输入中的可变对象,不保证保留所有底层内存布局关系:不同重叠 slice 视图会分别复制,内部指针指向某个切片元素的关系也不重建。字符串和零尺寸值可安全共享;`time.Time` 按 Go 不可变值使用(Location 不做独立复制)。不要通过 unsafe 或 uintptr 隐藏可变引用后要求工具识别。 + +## 字段差异与自定义转换 + +```go +type Input struct { Name string; UpdatedAt time.Time; InternalID string } +type View struct { DisplayName string; UpdatedAt string; LocalNote string } + +copier, err := copyx.New( + copyx.WithDeepCopy(), + copyx.WithStrictFields(), + copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}), + copyx.WithIgnoreSource[Input, View]("InternalID"), + copyx.WithIgnoreTarget[Input, View]("LocalNote"), + copyx.WithConverter(func(v time.Time) (string, error) { + return v.UTC().Format(time.RFC3339), nil + }), +) +if err != nil { return err } +out := View{LocalNote: "preserve"} +if err := copier.Copy(&out, input); err != nil { return err } +``` + +字段规则的类型参数必须是结构类型,字段名使用导出的 Go 字段名。构造时检查不存在的字段、重名及冲突。不支持字符串路径表达式;复杂嵌套差异为该嵌套类型注册 converter,或直接写 Go。 + +`WithIgnoreSource` 只是声明该源字段不参与使用;目标同名字段若仍需保留,还需 `WithIgnoreTarget`。忽略目标会保留原目标值,Clone 中则为零值。深拷贝时如果保留值仍与源共享可变引用,也会拒绝。 + +转换器的签名是 `func(S) (T, error)`,按**精确类型对**匹配,不能重复注册。金额、编号、日期格式由项目明确转换。需要逐字段不同规则时,给整个源/目标结构注册函数;函数返回独立的新目标。无需另一份映射配置或全局注册。 + +精确 converter 优先于 nil/零值策略,能收到 typed nil;例如 `func(*Money) (Money, error)` 可自行决定 nil 是否允许。绑定到 `any` 字段的 converter 能接收 nil interface,但顶层未带类型的 `nil` 没有精确类型可匹配。浅拷贝中已经直接共享的同类型容器不会继续逐项转换其内部;需要遍历内部时选择深拷贝,或为该容器类型注册 converter。 + +## 深拷贝不能原样返回可变引用 + +```go +copier, err := copyx.New( + copyx.WithDeepCopy(), + copyx.WithConverter(func(v []string) ([]string, error) { + return v, nil // 会返回 ErrSharedReference + }), +) +``` + +检查覆盖完整源图与转换结果,包括新父对象引用旧子对象、原 map、重叠子切片、`source[:0]` 的可写容量、指向源数组元素的指针,以及未导出字段中的引用。会检查每个 converter 的结果和最终目标,不允许静默降级为浅拷贝。 + +返回相同字符串、整数并不会报错,它们不是可变引用。正确切片 converter 可以写: + +```go +copyx.WithConverter(func(v []string) ([]string, error) { + if v == nil { return nil, nil } + out := make([]string, len(v)) + copy(out, v) + return out, nil +}) +``` + +第三方 Decimal/自定义 Money 如果含未导出的指针,默认深拷贝拒绝;注册该类型的 converter,使用它自己的构造/Clone API 创建独立值,避免转成 float64 损失精度。框架不内置某一金额库的规则。无法独立复制的类型直接返回错误。 + +## nil、零值和错误 + +| 选项 | 行为 | +| --- | --- | +| 默认 `NilError` | nil → 可空目标保留 nil;nil → 非可空目标报 `ErrNil` | +| `WithNilPolicy(NilZero)` | 显式允许 nil → 零值;可空目标仍为 nil | +| `WithNilPolicy(NilSkip)` | nil 保留已有目标;新集合元素没有旧目标,保留元素零值 | +| `WithSkipZero()` | 显式跳过 Go 零值,包括 `false`、`0`、`""`;非 nil 空集合不是 Go 零值 | + +所有 nil 策略都不把不兼容类型自动变成兼容类型。非 nil 指针与值之间可以按对应元素转换;nil 到非可空目标仍需选择策略。局部更新不能默认使用 NilZero/SkipZero,它们可能改变清空字段或布尔值的业务语义。 + +工具只能复制绑定后的 Go 值:如果输入类型已经将“缺失”和“JSON null”合并为 nil,无法恢复两者区别。需要三态时,应先设计包含 presence 信息的类型和绑定逻辑。 + +赋值使用临时结果,成功后才替换目标;转换失败不会交付部分结果。`*copyx.Error` 包含 Go 字段/索引路径,`errors.Is/As` 保留内部错误链,工具不主动在错误中加入源值。converter 自己返回的错误文本仍由项目负责。 + +converter 必须只读输入;其外部副作用、输入修改无法回滚,panic 原样传播。框架只检查与本次输入的共享引用,无法证明 converter 未返回外部全局缓存的引用。它是受信 Go 代码扩展,不是可运行不可信代码的沙箱。 + +## 支持范围与资源限制 + +- 支持结构体、指针、interface、slice、同长度 array、map。map key 转换后冲突或不可比较则报错。 +- 导出匿名结构字段可按提升后的无歧义名称访问;源嵌入指针为 nil 且需要其字段时明确报错。跨类型目标包含私有字段时拒绝;同类型私有纯值可作为整体保留。 +- 不克隆文件、数据库连接、网络连接、锁、原子状态。已知标准库资源/同步结构按值复制也拒绝;深拷贝中的非 nil func、chan、unsafe.Pointer 拒绝。其他封装资源类型应由项目排除,工具不能识别所有自定义句柄。 +- 单次遍历深度上限 256、节点上限 200,000;内置分配的估算预算为 64 MiB,超限返回 `ErrLimit`。预算不是进程堆内存硬限制,reflect 元数据和 converter 自行分配不计入。大对象流式读写应使用 IO/存储接口。 +- 只缓存不可变类型元数据,不缓存请求值、用户 converter 或实例策略。没有 `unsafe` 写入、JSON 往返、隐式数值截断或方法调用。 +- 这是运行时反射工具,会比直接 Go 赋值多检查与分配。热点应复用 Copier,并用真实数据测量;性能记录见 [本版验收](reports/copyxAcceptance.md)。 + +## 从旧 mapping 迁移 + +v0.19 **彻底移除**映射配置加载、Go 类型加载、映射代码生成及 `pointerResult` 等附属选项。公共 `.api` 字段组合、scalar、中间件工厂、Repository 开关继续存在。 + +1. 先检查每个转换是否必要。普通 Logic 可直接使用生成 DTO;独立核心保留自己的类型。 +2. 将 `mapping.Xxx(...)` 调用改为普通 Go 或小型 `copyx` helper。字段改名、nil、金额规则写在该 helper 同一处。指针返回可直接 `var out *Response; err := copier.Copy(&out, input)`。 +3. 用真实类型对测试缺失/null/零值/空数组、字段覆盖、金额精度和错误链。需要源字段新增时提醒遗漏,启用 `WithStrictFields()` 并测试。 +4. 从 `omni.gen.json` 删除 `mappings` / `mappingFiles`,删除无其他用途的映射 JSON 文件。 +5. 确认已无调用,再删除 `internal/mapping/mappingsGen.go`。如之前手改过文件,先迁出手写内容。生成器遇到该旧文件会阻止生成并提示迁移,**不会静默删除**;其余业务文件照常保留。 +6. 升级框架与 CLI 到同版,执行 generate、generate --check、go test ./...;多个服务再执行 workspace generate/check。新 manifest 自动去掉已移除文件的所有权记录。 + +历史发布记录保留旧能力说明,仅用于理解旧版本,不代表 v0.19 继续支持。 diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 594d3da..f4419e4 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,8 +1,8 @@ -# 通用业务开发指南(v0.18.0-preview) +# 通用业务开发指南(v0.19.0-preview) 从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。 -本版新增的公共字段组合、可选结构映射、自动中间件工厂和 Repository 开关,见 [生成能力指南](GENERATION.md)。 +公共字段组合、自动中间件工厂和 Repository 开关,见 [生成能力指南](GENERATION.md)。必要的结构转换使用可选 [copyx](COPY.md),v0.19 已移除旧 JSON 映射生成。 ## 返回数组、可空对象和 null @@ -278,7 +278,7 @@ go-omni workspace check --file omni.workspace.json ## v0.13 项目升级 -先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.18.0-preview(或显式引用当前源码)。 +先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.19.0-preview(或显式引用当前源码)。 ```sh go-omni api upgrade --dir . --dry-run diff --git a/GENERATION.md b/GENERATION.md index 02b59c2..65d1366 100644 --- a/GENERATION.md +++ b/GENERATION.md @@ -1,18 +1,10 @@ -# 公共字段、结构映射与生成选项 +# 公共字段、中间件工厂与生成选项 -本页对应 `v0.18.0-preview`。CLI 与框架依赖固定到同一版本: - -```sh -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.18.0-preview -# 已有业务项目内执行: -go get git.yuntuo.hk/k2safe/go-omni@v0.18.0-preview -``` - -私有仓库访问设置见 [安装说明](README.md#准备环境与安装)。升级不会覆盖业务文件;已有中间件保持 handlers 模式,切换工厂模式按本页迁移步骤完成。 +本页对应 `v0.19.0-preview`。CLI 与框架依赖固定到同一版本,安装见 [README](README.md#准备环境与安装)。 ## 一份项目生成配置 -新项目根目录生成开发者维护的 `omni.gen.json`: +新项目根目录的 `omni.gen.json` 只描述生成选项: ```json { @@ -21,32 +13,11 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.18.0-preview } ``` -该文件与 `.api` 一起提交。每次 `api generate`、`--check`、`--dry-run` 和 `workspace generate/check` 都会读取;`api validate/export/format` 只处理契约,不加载业务 Go 类型。 - -- `repository: none` 是默认值,不创建空 Repository 文件。`group` 按 `.api` 的 group 补齐 `internal/repository//repository.go`;文件始终属于业务。切回 none 后不再补骨架,已有文件不删除、不覆盖。实际 SQL、ORM 和仓储划分由项目决定。 -- `middleware: factories` 是新项目默认值,按下文自动初始化。`handlers` 保留原有 `Defaults()` 与 `func(next Handler) Handler` 接口,供既有项目逐步迁移。 -- `mappings` 可省略;只有明确列出的类型对才生成映射。未启用时不会运行 Go 类型加载命令,也不增加运行时依赖。 -- 升级首次遇到没有配置的旧 manifest 时,补一份 `middleware: handlers` 的配置,保留原有手写注册;Repository 默认 none。生成配置建立后不能直接删除,否则生成失败,避免误关闭映射或改变中间件模式。 - -### 按模块拆分映射(v0.18 新增) - -主配置可以保留少量内联 `mappings`,也可以只列出文件: - -```json -{ - "repository": "none", - "middleware": "factories", - "mappingFiles": ["mappings/requests.json", "mappings/responses.json"] -} -``` - -每个子文件只包含 `{"mappings": [...]}`,其中每条映射的写法与原来一致。先加载主配置内联定义,再按文件列表顺序加载;所有定义合并后统一校验,可以跨文件、向前引用 `mapping`。输出按映射名排序,文件拆分不改变函数名或语义。 - -- 路径相对项目根目录,必须是明确列出的 `.json` 文件。不展开通配符,不允许越界路径、符号链接、重复文件或多层导入;子文件不能覆盖 repository/middleware 等项目选项。 -- 重名映射、指针辅助函数名冲突、未知映射与循环引用明确报错;文件之间不能用覆盖顺序隐藏冲突。重复定义会指出两个来源文件,字段生成错误也带来源文件。 -- 文件均由项目维护,重新生成不改写它们。原来的单文件 `mappings` 继续有效。迁移时将完整条目移到子文件并删除原位置,不能复制后保留两份同名定义。 -- 最多 64 个子文件、单文件 1 MiB、全部配置 4 MiB;合计仍最多 128 个映射。主配置和子文件一起提交 Git。 -- `api generate`、`--check`、`--dry-run` 与 `workspace generate/check` 使用同一加载逻辑。缺少子文件或不完整配置立即失败,check/预览不写入;生成期间重新核对所有配置输入,发生变化时要求重试。 +- `repository: none` 默认不创建空 Repository 文件;`group` 按 group 补齐可编辑骨架。已有文件始终保留;切回 none 不会删除业务文件。SQL、ORM 和仓储划分由项目决定。 +- `middleware: factories` 是新项目默认值,自动调用带依赖的工厂。`handlers` 保留原有 `Defaults()` 及手动注册方式,供既有项目迁移。 +- 该文件与 `.api` 一起提交。generate/check/dry-run 和 workspace 采用相同规则;validate/export/format 只处理接口契约。 +- 旧 manifest 没有配置时补 `middleware: handlers`、`repository: none`。已有配置不能直接删除,避免意外切换中间件模式。 +- v0.19 已删除 `mappings` 和 `mappingFiles`;遇到旧配置会明确报错,不读取子映射文件或执行映射生成。处理步骤见 [copyx 迁移](COPY.md#从旧-mapping-迁移)。 ## 公共字段组合 @@ -73,151 +44,13 @@ type AdminRequest { 允许多层组合,上限 32 层、单类型 4096 字段、全契约展开 65536 字段。Go 匿名指针嵌入并非本功能;生成 DTO 中不会使用匿名嵌入。 -## 可选结构映射 +## DTO 与业务类型 -`.api` 生成 DTO,业务独立维护输入/结果类型。项目明确选择需要自动赋值的类型对;不从数据库反向生成公开接口,不自动推断反向映射。 +普通 Logic 直接接收生成的 Request、返回生成的 Response,不额外创建相同结构。只有独立核心或明确不同的输入/输出职责才保留业务类型。 -假设 `types.AdminRequest` 有 `Name string`、`Salary amount.Money`、`CompanyID string`,业务的 `domain.EmployeeInput` 有 `EmployeeName string` 和同类型 `Salary`: +必要转换直接写 Go,或用可选 `copyx.Copy(&target, source)`。同名同类型字段自动赋值,字段改名和类型变化通过 Go 参数、转换函数明确设置,见 [完整示例](examples/copying/main.go)。无需 JSON 映射配置、包路径字符串或额外生成文件。 -```json -{ - "repository": "none", - "middleware": "factories", - "mappings": [ - { - "name": "ToEmployeeInput", - "source": {"package": "example.com/payroll/internal/types", "type": "AdminRequest"}, - "target": {"package": "example.com/payroll/internal/domain", "type": "EmployeeInput"}, - "fields": { - "EmployeeName": {"source": "Name"} - }, - "ignoreSource": { - "CompanyID": "由 Logic 单独做企业范围授权,不属于请求体输入" - } - } - ] -} -``` - -执行 `go-omni api generate` 后,产生独立的生成文件 `internal/mapping/mappingsGen.go`: - -```go -func ToEmployeeInput(source types.AdminRequest) (domain.EmployeeInput, error) { - var result domain.EmployeeInput - result.EmployeeName = source.Name - result.Salary = source.Salary - return result, nil -} -``` - -包别名由生成器决定。业务调用 `mapping.ToEmployeeInput(*req)`;反向响应映射按另一条 source/target 规则声明。两个类型完全相同时通常无需映射。 - -### 类型与覆盖检查 - -自动赋值要求同名且 **Go 类型一致**(类型别名按 Go 类型系统处理),字段名使用 Go 名称,不按 JSON 标签猜对应关系。int32 → int64、`*Money` → Money、数值 ID → string 都必须显式转换。不会暗中补零、解引用或执行数字强转。 - -每个源字段必须被使用或出现在 `ignoreSource`;每个目标字段必须被赋值或出现在 `ignoreTarget`。忽略规则必须给出非空原因,目标忽略明确意味着保留零值;规则引用不存在的字段也会失败。新字段若无法自动匹配,会在生成阶段报错。两边同时添加的同名同类型字段会自动加入生成代码,`--check` 检出未重新生成的差异。 - -根类型是项目可导入的具名结构;支持值类型匿名嵌入,规则可用 `Employee.EmployeeName` 这样的实际 Go 字段路径。同名提升歧义、未导出字段和匿名指针嵌入拒绝处理。普通具名嵌套字段使用独立映射或转换函数,不猜测内部结构。泛型声明不能直接作为根;具体字段类型按 Go 类型系统检查。 - -配置变化、遗漏映射、类型错误会在写入前失败。映射生成与其他生成产物使用同一套文件所有权机制;手改映射生成文件会被拒绝覆盖。 - -### 显式转换函数 - -```json -"fields": { - "Salary": { - "source": "Salary", - "convert": {"package": "example.com/payroll/internal/convert", "name": "SalaryValue"} - } -} -``` - -支持这两种精确签名: - -```go -func SalaryValue(source *amount.Money) amount.Money -func SalaryValue(source *amount.Money) (amount.Money, error) -``` - -项目选择一种实现。nil、默认值、金额精度、日期格式和错误码由函数负责;若两边已经是同一个 Money 类型,无需再次解析或序列化。转换函数应仅负责值转换,不执行 SQL 或改变授权范围。 - -错误通过 `%w` 加上映射名/目标字段路径,保留 `errors.Is/As` 和框架错误链;返回零目标值,不交付部分完成的结果。包装不主动添加源值或转换后的值。错误是否公开仍由项目错误目录和响应策略决定。 - -### 嵌套对象、数组与引用 - -`mapping` 指向同配置中的另一条映射,`elements` 明确要求逐项映射: - -```json -"fields": { - "Summary": {"source": "Summary", "mapping": "ToSummary"}, - "Items": {"source": "Items", "mapping": "ToItem", "elements": true} -} -``` - -`mapping` 与 `convert` 互斥。`elements` 可以配合其中任意一种,也可以对类型相同的元素直接赋值;支持 slice→slice、等长 array→array,禁止隐式互转。nil slice 保持 nil,非 nil 空 slice 保持非 nil,逐项失败的错误包含索引,循环映射声明被拒绝。 - -未选择 elements 的同类型指针、map、slice 使用普通 Go 赋值,**共享引用**,不做隐式深拷贝。elements 会分配新的目标 slice,但元素内的引用仍遵循其映射规则。需要深拷贝时显式提供转换函数。 - -映射不保存原 JSON 的字段存在信息。普通 `*T` 在绑定后可能已把缺失和 null 合并为 nil;无法在转换阶段恢复区分。有三态需要的项目必须先使用能携带该信息的输入类型及绑定扩展;没有声明三态时,保持当前输入类型的语义。完整表单更新和局部更新由业务决定,框架不把 nil 自动解释成“保留旧值”。 - -### 可空嵌套映射与显式 nil 策略(v0.18 新增) - -假设已有 `ToChild` 定义 `types.Child → domain.Child`。父对象的字段规则可以直接复用它: - -```json -"fields": { - "Child": {"source": "Child", "mapping": "ToChild"}, - "Children": {"source": "Children", "mapping": "ToChild", "elements": true} -} -``` - -| 源字段 → 目标字段 | 明确选择 mapping 后的行为 | -| --- | --- | -| `S → T` | 调用原值映射 | -| `*S → *T` | 默认保留 nil;非 nil 调用值映射,并分配新的目标对象;也可配置 `nil: "error"` 拒绝 nil | -| `S → *T` | 调用值映射,成功后返回目标对象指针 | -| `*S → T` | 必须选择 `nil: "zero"` 或 `nil: "error"`,不猜默认值 | - -`nil: "preserve"` 只适用于可空源和可空目标。`nil: "zero"` 只适用于可空源和非空目标:nil 直接产生目标 Go 零值,**不调用子映射或转换函数**。`nil: "error"` 在 nil 时返回带映射/字段路径的普通转换错误,HTTP 公开响应仍由项目的错误处理策略决定;需要领域错误码时使用项目 converter。 - -同一规则适用于显式 `elements: true` 下的 slice/等长数组元素,包括 `[]*S → []*T`。nil slice、非 nil 空 slice、nil 元素分别保留;元素失败保留索引与内部错误链,整个映射返回零目标值,不交付部分结果。只支持这里列出的单层指针,不自动处理多层指针或递归对象图。 - -普通同类型指针字段未指定 mapping 时仍直接赋值、共享引用;上述显式映射会创建目标子对象,但不承诺对其中所有引用深拷贝。业务不可变快照继续由项目设计。 - -简单的 `*T → T` 也可以单独指定 nil 策略,减少重复解引用函数: - -```json -"fields": { - "Title": {"source": "Title", "nil": "zero"}, - "Count": {"source": "Count", "nil": "error"} -} -``` - -这里必须是完全相同的 Go 元素类型,`*int32 → int64`、枚举 → string、数字编号 → string 仍需显式 converter。非 nil 的零值正常保留。`convert` 与 `nil` 不能同时设置,现有 converter 继续完整控制自己的空值、格式和错误语义。 - -### 可选指针返回辅助(v0.18 新增) - -在映射条目上增加 `"pointerResult": true`,生成器保留 `ToResponse(source S) (T, error)`,另加: - -```go -func ToResponsePtr(source S) (*T, error) -``` - -Logic 在业务调用成功后可直接写 `return mapping.ToResponsePtr(result)`。转换失败时返回 `nil, err`,错误链不变;成功才返回非 nil 指针。业务调用本身的错误仍由 Logic 处理,辅助函数不执行任何业务操作。 - -辅助函数名称固定为映射名加 `Ptr`,与其他映射/辅助函数同名会在生成前报错。输入仍是原来的值类型 S,不用 nil 输入表达业务错误;关闭该选项会移除辅助函数,使用它的业务代码需同时调整。 - -### 生成顺序与 CI - -1. 先从 `.api` 生成项目,添加业务结构、scalar 和转换函数,执行 `go mod tidy`。 -2. 在 `omni.gen.json` 列出映射,执行 `go-omni api generate`。 -3. Logic 调用生成函数。后续改字段后重复生成,不修改 `*Gen.go`。 -4. CI 同时运行 `go-omni api generate --check` 与 `go test ./...`。 - -启用映射后,CLI 通过 `go list -deps -export` 加载明确声明的包与依赖,按当前构建环境做类型检查,不执行业务代码、init 或转换函数。采用待生成 DTO 的临时 overlay,避免“新增字段还看旧 DTO”。go.mod/go.sum 的副本及 overlay 都在临时目录,check 不修改项目文件;Go 自己的构建/模块缓存可能更新。使用当前 GOOS/GOARCH/GOFLAGS、`GOWORK=off`;依赖用 go.mod 与 replace 表达,生成超时为 90 秒。 - -类型包/转换函数包不能依赖输出包 `internal/mapping`;把转换函数放在独立包中,避免导入环。若包本身无法编译,先运行 `go list -deps -export <相关包>` 排查;CLI 不回显可能含私有仓库凭据的 Go 下载输出。纯值字段转换只生成标准 Go 代码,不引入运行时反射搬运、JSON 往返、ORM 或队列依赖。 +`copyx` 在运行时检查类型和字段,**不提供旧映射生成阶段的静态字段覆盖保证**;CI 应给实际类型对添加测试。普通业务字段新增通常只改 `.api` 和 Logic;独立 DTO/领域类型都需要此字段时,两边各加一次,同名同类型无需追加拷贝配置。 ## 中间件工厂自动注册 @@ -271,9 +104,13 @@ factory 模式不再在 Init 写 `Middlewares["..."]`;残留非空手写注册 | 项目维护 | 生成器持续维护 | | --- | --- | | `.api` 公共字段与接口 | 展开后的 DTO、Handler、OpenAPI | -| `omni.gen.json` 的映射与选项 | `internal/mapping/mappingsGen.go` | -| Go 业务类型、scalar、转换函数 | 编译期可检查的赋值与调用代码 | +| `omni.gen.json` 的 Repository/中间件选项 | 对应骨架和注册代码;已有手写骨架不覆盖 | +| 独立 Go 业务类型、scalar、可选 copyx 调用和测试 | 不生成结构转换文件 | | middleware Dependencies、具名工厂、svc 访问器 | 中间件 registry、启动调用 | -| Logic、按需 Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 | +| Logic、Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 | -回归样例位于 `cmd/go-omni/apigen/testdata/composition`,为独立三端服务,不导入任何 eTax 代码。 +独立三端工资契约测试位于 `cmd/go-omni/apigen/testdata/composition`,不导入任何接入项目。它验证公共字段、Money scalar、三端 Handler、copyx、重新生成保留业务内容,以及新增字段不需要更新转换规则。 + +```sh +GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1 +``` diff --git a/PROJECT.md b/PROJECT.md index f58bd4a..b8958b8 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -1,6 +1,6 @@ # 生成项目与文件维护边界 -v0.18.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 +v0.19.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 ## 创建和日常开发 @@ -54,14 +54,15 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name): | `internal/logic//*.go` | 业务方法,使用标准 context.Context | | `internal/repository//*.go` | 数据访问;默认不补空骨架,可选 group 模式 | | `internal/middleware/*Factory.go`、`dependencies.go` | 中间件工厂与依赖小接口,svc 实现访问器 | -| `omni.gen.json`、映射转换函数(支持 `mappingFiles` 引用子文件) | 生成选项、显式类型对与差异规则 | +| `omni.gen.json` | Repository 和中间件生成选项;不再声明结构映射 | +| 可选的 Go 转换函数 / copyx 调用 | 只有独立类型确实需要转换时维护,见 [copyx](COPY.md) | | 项目 Go 错误包 | 自行维护 Catalog 和常量,可跨服务共用 | | 自定义标量文件、`*Bind.go`、`*Manual.go` | 只有显式选择相应扩展时才生成/使用 | | 项目测试、迁移、部署脚本、go.mod/go.sum | 项目所有,生成器不重置其内容 | 以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。 -生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、可选映射 `internal/mapping/mappingsGen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。 +生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。 Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。 @@ -211,7 +212,7 @@ get /download (DownloadReq) ## 已有服务接入步骤 -每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.18.0-preview: +每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.19.0-preview: 1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。 2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。 diff --git a/README.md b/README.md index 1d23b48..9a91868 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,17 @@ GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。 -**从这份 README 开始接入。** 当前版本 **`v0.18.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。 +**从这份 README 开始接入。** 当前版本 **`v0.19.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。 -**v0.17 提供**:公共字段组合、可选结构映射、带依赖的中间件工厂、按需 Repository 骨架,使用与迁移见 [生成能力指南](GENERATION.md)。 +**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。 -**本版新增**:可空嵌套映射、映射配置按文件拆分、显式 nil 策略和可选指针返回辅助,见 [生成能力指南](GENERATION.md)。 +普通接口的流程是:**修改 `.api` → 生成 Request/Response → 写 Logic → 测试**。有独立职责的业务类型才需要转换;可直接写 Go,也可以按需使用 [copyx](COPY.md)。公共字段组合、中间件工厂、Repository 开关继续支持,见 [生成能力指南](GENERATION.md)。 Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。 响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。 -本版补齐 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录,以及统一环境变量别名覆盖。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。 +契约支持 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录,以及统一环境变量别名覆盖。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。 已有通用开发能力见 [业务开发指南](DEVELOPMENT.md):数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。 @@ -20,6 +20,40 @@ Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可 使用 GORM 的项目可以选择 [gormx 适配器](database/gormx/README.md),复用数据库初始化和连接池,并将 SQL 日志接入 logx。 +## 当前功能与使用入口 + +下表是当前公开能力的索引。每项给出调用入口和完整说明;模块按需使用,`omni.New()` 不会自动初始化数据库、Redis、ORM、队列或云存储。示例中的 import 统一以 `git.yuntuo.hk/k2safe/go-omni/` 开头。 + +| 功能与用途 | 怎么用 | 解释与示例 | +| --- | --- | --- | +| 项目、接口与文档生成 | `go-omni new`;改 `.api` 后 `api generate`;`--check` 检查漂移 | [下方快速开始](#新项目生成并运行)、[CLI](cmd/go-omni/README.md)、[完整生成项目](examples/managedService/README.md) | +| 公共字段与跨服务共享契约 | `.api` 组合对象字段;`omni.imports.json` 声明共享根;workspace 批量生成 | [公共字段示例](GENERATION.md#公共字段组合)、[共享契约与 workspace](DEVELOPMENT.md) | +| 中间件依赖工厂、可选 Repository | `omni.gen.json` 选择 factories / none;在 `New(deps)` 写一次实现 | [配置与完整工厂示例](GENERATION.md) | +| 路由、分组、请求上下文 | `omni.New()`、`app.GET` / `Group` / `Use`;业务使用 `context.Context` | [基本服务](examples/basic/main.go)、[已有项目接入](#已有项目按模块接入) | +| HTTP 网络与扩展适配 | 配置 transport 为 nethttp / native / http2;自定义 Router/Runner | [网络层配置与契约](transport/README.md)、[性能与限制](PERFORMANCE.md) | +| HTTP 老接口桥接 | 通过官方 Handler/Middleware 适配器接入标准 `net/http`,跟踪实际响应状态 | [接入说明](MIGRATION.md)、[桥接/上传/自定义 ID 示例](examples/compatibility/main.go) | +| 请求校验、动态 JSON、自定义标量 | `.api` 声明来源/校验/scalar;手写契约用 `contract.New[T]`;简单绑定可用 `validate.JSON/Query/URI` | [语法与例子](API.md)、[请求策略](DEVELOPMENT.md) | +| 数组、null、原始请求、上传下载 | `.api` 声明数组/可空返回、body/multipart 与大小限制;必要时使用 Bind/Manual 扩展 | [契约示例](examples/contracts)、[含上传/回调/下载的项目](examples/managedService/README.md) | +| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) | +| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) | +| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) | +| 结构化业务日志 | `logx.Info/Error`、`logx.From(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) | +| 文件配置与环境覆盖 | 默认值→JSON/YAML→环境;完整业务结构用 `config.LoadProject[T]` / 环境别名 | [配置详解与例子](DEVELOPMENT.md)、[动态级别](ADVANCED.md#配置与日志热更新) | +| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) | +| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) | +| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) | +| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) | +| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) | +| 出站 HTTP、熔断 | `httpc.New` 复用连接池与截止时间;`resilience.New` 隔离故障依赖 | [HTTP 用法](MODULES.md)、[熔断/观测](ADVANCED.md#熔断与观测) | +| CORS、日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md)、[分布式实现](ADVANCED.md#分布式限流与幂等) | +| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) | +| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) | +| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) | +| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [观测用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) | +| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) | +| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) | +| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) | + ## 选择接入方式 | 你的项目 | 从哪里开始 | @@ -40,7 +74,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可 ```sh export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk" -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.18.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview export PATH="$(go env GOPATH)/bin:$PATH" go-omni --help ``` @@ -197,7 +231,7 @@ go-omni new --api catalog.api \ 在业务项目的 **go.mod 所在目录**执行: ```sh -go get git.yuntuo.hk/k2safe/go-omni@v0.18.0-preview +go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview ``` 已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。 @@ -391,6 +425,38 @@ return apierrors.TextTooLong("企业名称", 100) 更详细的配置、作用域和升级步骤见 [统一错误目录](errors/README.md)。 +### 可选结构转换与深拷贝 + +普通接口不需要任何拷贝配置,直接使用生成的 Request/Response。独立业务结构之间有转换需求时: + +```go +// import "git.yuntuo.hk/k2safe/go-omni/copyx" +var input DomainInput +if err := copyx.Copy(&input, req); err != nil { return err } + +// 源与目标之间的指针、map、slice 需要隔离时: +snapshot, err := copyx.Clone(source) +if err != nil { return err } +``` + +默认浅拷贝,同名同类型字段自动赋值,保留零值;默认不会把 nil 填成非可空目标的零值。不要为使用工具新增一套重复模型。 + +需要重复使用或自定义规则时创建 Copier: + +```go +copier, err := copyx.New( + copyx.WithDeepCopy(), + copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}), + copyx.WithConverter(func(v time.Time) (string, error) { + return v.UTC().Format(time.RFC3339), nil + }), +) +if err != nil { return err } +if err := copier.Copy(&view, input); err != nil { return err } +``` + +自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错;字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md),完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查,类型对需测试;深拷贝不适用于连接、锁等资源。 + ### 数据库、存储及其他模块 生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。 @@ -467,7 +533,7 @@ go test ./... `main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。 -CLI 与业务 module 固定使用 `v0.18.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 +CLI 与业务 module 固定使用 `v0.19.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 ## 按需深入 @@ -483,3 +549,7 @@ CLI 与业务 module 固定使用 `v0.18.0-preview`。v0.14.x 项目升级依赖 | 网络性能、测试条件与适用范围 | [性能说明](PERFORMANCE.md)、[HTTP/2 验收](transport/native/HTTP2.md) | 当前版本可用于固定版本接入与业务验证,仍标记为 preview。`.api` 借鉴 go-zero 的接口开发方式,支持范围以本仓库文档为准,不是完整 goctl 语法兼容层;HTTP/2 保持实验状态,OSS/R2 云账号实网验收不属于本次发布范围。 + +## 后续开发约定 + +每次功能开发完成,同步更新这份功能索引、用法说明、`examples/` 和发布记录;删除或改变功能必须注明影响与迁移。代码、示例和文档一起回归后再发布。约定已记录在 [AGENTS.md](AGENTS.md),后续开发继续遵循。 diff --git a/RELEASE.md b/RELEASE.md index f8e9e28..18248ae 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,4 +1,31 @@ -# v0.18.0-preview +# v0.19.0-preview + +将可选结构转换改为直接 Go 调用,减少业务项目需要维护的配置。普通接口继续使用 `.api` 生成 DTO,不要求增加独立模型或任何拷贝规则。 + +- 新增只依赖标准库的 `copyx`:Copy、Clone、可复用 Copier;浅/深拷贝、严格字段覆盖、nil/零值策略、字段改名、忽略字段、精确类型 converter。没有全局注册和隐式数字强转。 +- 深拷贝检查整个输入和输出图。自定义 converter 原样返回指针、map、slice,或新对象仍嵌套旧引用时返回错误;保留普通循环图,拒绝资源对象及超出遍历/估算分配预算的输入。 +- 转换失败不提交部分目标;保留错误路径和原始错误链。并发复用只共享不可变类型元数据;源/目标读写和 converter 副作用由调用方负责。 +- **不兼容变更:删除 mappings / mappingFiles、Go 类型加载与映射代码生成。** 检出旧配置/旧生成文件时给出迁移提示;不会静默删除业务正在使用的代码。公共字段组合、scalar、中间件工厂、Repository 选项保留。 +- README 补齐当前能力入口、解释和用法,新增 [完整 copyx 指南](COPY.md)、[可运行示例](examples/copying);[AGENTS.md](AGENTS.md) 记录每次修改同步文档/示例/移除说明的约定。 + +```sh +go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview +# 如用过旧 mapping,先按 COPY.md 完成迁移: +go-omni api generate +go-omni api generate --check +go test ./... +``` + +迁移步骤和语义边界见 [copyx](COPY.md#从旧-mapping-迁移)。新工具做运行时检查,不能声称具备旧生成器的编译期字段覆盖保证;实际类型对必须有测试。独立三端工资示例验证金额精度、缺失/null/零值、空集合、错误链和字段新增;不修改或代替接入项目的完整业务回归。 + +46 个测试包的全量 race(含实际生成项目)、vet/build、Go 1.22 兼容测试通过;30 秒 fuzz 无失败,14 种基准场景各测量三轮。验收条件、性能数据及本轮未覆盖的外部环境见 [v0.19 验收记录](reports/copyxAcceptance.md)。本版仍为 preview。 + +--- + +以下是历史版本记录,其中 mapping 功能已在 v0.19 移除,不应继续作为当前接入方式。 + +# v0.18.0-preview(历史版本) 补齐可空嵌套映射和按模块拆分配置,减少重复转换代码,业务空值规则仍由项目显式选择: diff --git a/ROADMAP.md b/ROADMAP.md index 8c55fe7..4e8bf70 100644 --- a/ROADMAP.md +++ b/ROADMAP.md @@ -1,5 +1,8 @@ # GoOmni 成熟度计划 +当前发布基线为 v0.19.0-preview:普通接口通过 `.api` → DTO → Logic 开发;独立类型需要转换时可选 copyx。旧 JSON mapping 生成已移除,见 [迁移说明](COPY.md)。当前能力、使用入口和限制以 [README](README.md) 与 [发布记录](RELEASE.md) 为准。下文保留阶段性建设记录。 + + 当前进展:`v0.14.0-preview` 已完成契约驱动的业务开发能力、通用 Runtime/配置和官方 MySQL 适配器,并通过全量 race、实际生成服务及空缓存远程验收。新项目/升级以 [README](README.md) 和 [开发指南](DEVELOPMENT.md) 为准;下方为历史阶段记录,完整本轮验收见 [报告](reports/businessDevelopmentAcceptance.md)。具体 eTax/ePay 业务迁移与业务回归仍由接入项目完成。 ## 1. 新项目启动与发布流程(已完成,v0.1.1) diff --git a/cmd/go-omni/README.md b/cmd/go-omni/README.md index 7ea2973..ea60944 100644 --- a/cmd/go-omni/README.md +++ b/cmd/go-omni/README.md @@ -1,8 +1,8 @@ # go-omni CLI -v0.18 新增:可空嵌套映射、`mappingFiles` 配置拆分、显式 nil 策略和可选 `pointerResult`。原 v0.17 配置继续有效,见 [生成能力指南](../../GENERATION.md)。 +v0.19 移除结构映射生成:`omni.gen.json` 只保留 Repository 和中间件选项。普通接口直接使用生成 DTO;需要类型转换时用普通 Go 或可选 [copyx](../../COPY.md)。旧 `mappings` / `mappingFiles` 必须按迁移步骤清理,CLI 不会静默删除仍在调用的旧生成文件。 -v0.17 增加公共字段组合、可选结构映射、自动中间件工厂和按需 Repository。配置和迁移见 [生成能力指南](../../GENERATION.md)。 +公共字段组合、中间件工厂和按需 Repository 继续支持,见 [生成能力指南](../../GENERATION.md)。 v0.16 增加基础类型/动态 JSON 数组返回、OpenAPI 公共引用和 workspace Go Catalog 统一导出。见 [接入与筛选规则](../../DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。 @@ -14,7 +14,7 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目 ```sh # 仓库读取权限与 PATH 配置见根目录 README -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.18.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview go-omni new --module example.com/hello ./hello # 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在 @@ -49,14 +49,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节 ## 版本与迁移 -CLI 与框架依赖固定使用 v0.18.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换: +CLI 与框架依赖固定使用 v0.19.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换: ```sh # 在 go-omni 源码仓库中执行 go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello ``` -当前模板与依赖以 v0.18.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。 +当前模板与依赖以 v0.19.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。 ## 数据库迁移命令 diff --git a/cmd/go-omni/apigen/copyConsumer_test.go b/cmd/go-omni/apigen/copyConsumer_test.go new file mode 100644 index 0000000..629a907 --- /dev/null +++ b/cmd/go-omni/apigen/copyConsumer_test.go @@ -0,0 +1,95 @@ +package apigen + +import ( + "context" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" +) + +func runConsumerGo(t *testing.T, dir string, args ...string) { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "go", args...) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GOWORK=off") + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("go %v: %v\n%s", args, err, out) + } +} +func TestCopyxThreeEndpointConsumer(t *testing.T) { + if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" { + t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1") + } + dir := filepath.Join(t.TempDir(), "app") + o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)} + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + for path, fixture := range map[string]string{ + "internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go", "internal/convert/copy.go": "copy.go", + "internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go", + } { + writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture)) + } + svcFile := filepath.Join(dir, "internal/svc/serviceContext.go") + source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1) + source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1) + writeFile(t, svcFile, source) + configFile := filepath.Join(dir, "internal/config/config.go") + writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1)) + for _, group := range []string{"Enterprise", "Tenant", "Admin"} { + text := fmt.Sprintf(`package %s +import("context";"example.com/composition/internal/types";"example.com/composition/internal/convert";"example.com/composition/internal/domain") +func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){ + input,err:=convert.Employee(*req);if err!=nil{return nil,err} + result,err:=convert.Response(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil +} +`, strings.ToLower(group), group) + writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text) + } + runConsumerGo(t, dir, "mod", "tidy") + runConsumerGo(t, dir, "test", "-race", "-count=1", "./...") + o.API = filepath.Join(dir, "api/service.api") + before := snapshot(t, dir) + o.DryRun = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + o.DryRun = false + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + o.Check = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("generation changed business files") + } + if _, err := os.Stat(filepath.Join(dir, "internal/mapping")); !os.IsNotExist(err) { + t.Fatal("mapping directory generated") + } + // Adding a common field requires one contract edit and one independent + // business field. Three endpoint converters need no new rules. + contract := readFile(t, o.API) + contract = strings.Replace(contract, "type SalaryFields {", "type SalaryFields {\n Note string `json:\"note,optional\"`", 1) + writeFile(t, o.API, contract) + domainPath := filepath.Join(dir, "internal/domain/employee.go") + writeFile(t, domainPath, strings.Replace(readFile(t, domainPath), "type Employee struct {", "type Employee struct {\n Note string", 1)) + o.Check = false + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + o.Check = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + runConsumerGo(t, dir, "test", "-race", "-count=1", "./...") + runConsumerGo(t, dir, "build", ".") +} diff --git a/cmd/go-omni/apigen/generate.go b/cmd/go-omni/apigen/generate.go index 659bd9c..8509872 100644 --- a/cmd/go-omni/apigen/generate.go +++ b/cmd/go-omni/apigen/generate.go @@ -19,7 +19,7 @@ import ( ) const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" -const minimumVersion = "v0.18.0-preview" +const minimumVersion = "v0.19.0-preview" const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n" type Options struct { @@ -252,9 +252,6 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { } } } - if err := renderMappings(o, generated, seeds); err != nil { - return Output{}, err - } currentConfig, stillExists, err := readGeneratorConfig(directory) if err != nil { return Output{}, err diff --git a/cmd/go-omni/apigen/generatorConfig.go b/cmd/go-omni/apigen/generatorConfig.go index fc118ab..7a55c37 100644 --- a/cmd/go-omni/apigen/generatorConfig.go +++ b/cmd/go-omni/apigen/generatorConfig.go @@ -15,14 +15,11 @@ import ( const generatorConfigFile = "omni.gen.json" -// GeneratorConfig is developer-owned and checked on every generation. Mapping -// rules never change the public API or infer a persistence model. +// GeneratorConfig contains developer-owned scaffolding options only. type GeneratorConfig struct { - Repository string `json:"repository,omitempty"` - Middleware string `json:"middleware,omitempty"` - Mappings []Mapping `json:"mappings,omitempty"` - MappingFiles []string `json:"mappingFiles,omitempty"` - inputDigest string + Repository string `json:"repository,omitempty"` + Middleware string `json:"middleware,omitempty"` + inputDigest string } func readGeneratorConfig(root string) (GeneratorConfig, bool, error) { @@ -40,7 +37,7 @@ func readGeneratorConfig(root string) (GeneratorConfig, bool, error) { return cfg, false, err } if previous.GeneratorConfig { - return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options and mappings before regenerating") + return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options before regenerating") } cfg.Middleware = "handlers" } else if !errors.Is(e, os.ErrNotExist) { @@ -54,72 +51,36 @@ func readGeneratorConfig(root string) (GeneratorConfig, bool, error) { if !stat.Mode().IsRegular() || stat.Size() > 1<<20 { return cfg, false, errors.New("omni.gen.json must be a regular file of at most 1 MiB") } - raw, err := readMappingConfigFile(root, generatorConfigFile) + raw, err := readGeneratorConfigFile(root, generatorConfigFile) if err != nil { return cfg, false, err } + var keys map[string]json.RawMessage + if err = apierrors.DecodeCatalogJSON(raw, &keys); err != nil { + return cfg, false, fmt.Errorf("omni.gen.json: %w", err) + } + for _, key := range []string{"mappings", "mappingFiles"} { + if _, exists := keys[key]; exists { + return cfg, false, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go, remove mappings/mappingFiles from omni.gen.json, then remove internal/mapping/mappingsGen.go; see COPY.md") + } + } if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil { return cfg, false, fmt.Errorf("omni.gen.json: %w", err) } - // Child files only declare mappings. No recursive imports, globs, implicit - // directory scans or access outside the project are supported. - if len(cfg.MappingFiles) > 64 { - return cfg, false, errors.New("omni.gen.json: at most 64 mappingFiles supported") - } - digest := sha256.New() - fmt.Fprintf(digest, "%s:%d:", generatorConfigFile, len(raw)) - digest.Write(raw) - total := len(raw) - for i := range cfg.Mappings { - cfg.Mappings[i].origin = generatorConfigFile - } - seen := map[string]bool{generatorConfigFile: true} - for _, name := range cfg.MappingFiles { - key := strings.ToLower(name) - if seen[key] { - return cfg, false, fmt.Errorf("omni.gen.json: duplicate mapping file %q", name) - } - seen[key] = true - if !strings.HasSuffix(name, ".json") { - return cfg, false, fmt.Errorf("mapping file %q must have a .json extension", name) - } - data, err := readMappingConfigFile(root, name) - if err != nil { - return cfg, false, err - } - total += len(data) - if total > 4<<20 { - return cfg, false, errors.New("mapping configuration exceeds 4 MiB in total") - } - var file struct { - Mappings []Mapping `json:"mappings"` - } - if err = apierrors.DecodeCatalogJSON(data, &file); err != nil { - return cfg, false, fmt.Errorf("%s: %w (mapping files only support mappings; nested imports are not supported)", name, err) - } - for i := range file.Mappings { - file.Mappings[i].origin = name - } - cfg.Mappings = append(cfg.Mappings, file.Mappings...) - fmt.Fprintf(digest, "%s:%d:", name, len(data)) - digest.Write(data) - } - cfg.inputDigest = fmt.Sprintf("%x", digest.Sum(nil)) + digest := sha256.Sum256(raw) + cfg.inputDigest = fmt.Sprintf("%x", digest) if cfg.Repository != "none" && cfg.Repository != "group" { return cfg, false, errors.New("omni.gen.json: repository must be none or group") } if cfg.Middleware != "factories" && cfg.Middleware != "handlers" { return cfg, false, errors.New("omni.gen.json: middleware must be factories or handlers") } - if len(cfg.Mappings) > 128 { - return cfg, false, errors.New("omni.gen.json: at most 128 mappings supported") - } return cfg, true, nil } // Read through a bounded reader as well as checking stat: a growing file cannot // make configuration loading unbounded. safePath also rejects symlink escapes. -func readMappingConfigFile(root, name string) ([]byte, error) { +func readGeneratorConfigFile(root, name string) ([]byte, error) { if err := safePath(root, name); err != nil { return nil, err } diff --git a/cmd/go-omni/apigen/generatorConfig_test.go b/cmd/go-omni/apigen/generatorConfig_test.go index 069ad16..a797528 100644 --- a/cmd/go-omni/apigen/generatorConfig_test.go +++ b/cmd/go-omni/apigen/generatorConfig_test.go @@ -2,140 +2,95 @@ package apigen import ( "encoding/json" - "fmt" + "io" "os" "path/filepath" "strings" "testing" ) -func TestMappingFileConfiguration(t *testing.T) { - dir := t.TempDir() - defs := nullableMappings() - writeGeneratorConfig(t, dir, GeneratorConfig{Repository: "none", Middleware: "factories", Mappings: defs[:1], MappingFiles: []string{"mappings/a.json", "mappings/b.json"}}) - writeMappingFile(t, dir, "mappings/a.json", defs[1:5]) - writeMappingFile(t, dir, "mappings/b.json", defs[5:]) - first, exists, err := readGeneratorConfig(dir) - if err != nil || !exists || len(first.Mappings) != len(defs) { - t.Fatal(first, err) - } - for i, m := range first.Mappings { - if m.Name != defs[i].Name { - t.Fatal("non-deterministic load order") +func TestWorkspaceRejectsRemovedMappingWithoutWrites(t *testing.T) { + root := t.TempDir() + for _, name := range []string{"one", "two"} { + if _, err := New(Options{Dir: filepath.Join(root, name), Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) } } - second, _, err := readGeneratorConfig(dir) - if err != nil || first.inputDigest != second.inputDigest { - t.Fatal("unstable config digest", err) - } - path := filepath.Join(dir, "mappings/a.json") - writeFile(t, path, readFile(t, path)+"\n") - changed, _, err := readGeneratorConfig(dir) - if err != nil || changed.inputDigest == first.inputDigest { - t.Fatal("child file change invisible to generation guard", err) - } - // Duplicate definitions name both files instead of silently overriding. - writeMappingFile(t, dir, "mappings/b.json", defs[1:2]) - cfg, _, err := readGeneratorConfig(dir) - if err != nil { - t.Fatal(err) - } - _, err = validateMappings(cfg.Mappings, "example.com/nullable") - if err == nil || !strings.Contains(err.Error(), "mappings/a.json") || !strings.Contains(err.Error(), "mappings/b.json") { - t.Fatal(err) + firstAPI := filepath.Join(root, "one/api/service.api") + writeFile(t, firstAPI, strings.Replace(readFile(t, firstAPI), "Greet a visitor", "Changed description", 1)) + writeFile(t, filepath.Join(root, "two", generatorConfigFile), `{"mappingFiles":["../../private.json"]}`) + workspace := filepath.Join(root, "omni.workspace.json") + writeFile(t, workspace, `{"services":[{"dir":"one"},{"dir":"two"}]}`) + before := snapshot(t, root) + for _, operation := range []string{"generate", "check"} { + if err := Workspace(workspace, operation, io.Discard); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") { + t.Fatal(operation, err) + } + if !sameFiles(before, snapshot(t, root)) { + t.Fatal("workspace migration failure wrote files") + } } } -func TestMappingFileRejection(t *testing.T) { - for _, tc := range []struct{ name, root, child, want string }{ - {"missing", `{"mappingFiles":["maps.json"]}`, "", "maps.json"}, - {"escape", `{"mappingFiles":["../maps.json"]}`, "", "unsafe"}, - {"absolute", `{"mappingFiles":["/maps.json"]}`, "", "unsafe"}, - {"nonCanonical", `{"mappingFiles":["sub/../maps.json"]}`, "", "unsafe"}, - {"duplicate", `{"mappingFiles":["maps.json","maps.json"]}`, `{"mappings":[]}`, "duplicate mapping file"}, - {"caseDuplicate", `{"mappingFiles":["maps.json","MAPS.json"]}`, `{"mappings":[]}`, "duplicate mapping file"}, - {"self", `{"mappingFiles":["omni.gen.json"]}`, "", "duplicate mapping file"}, - {"recursive", `{"mappingFiles":["maps.json"]}`, `{"mappingFiles":["omni.gen.json"]}`, "nested imports are not supported"}, - {"childSettings", `{"mappingFiles":["maps.json"]}`, `{"repository":"none","mappings":[]}`, "maps.json"}, - {"duplicateKeys", `{"mappingFiles":["maps.json"]}`, `{"mappings":[],"mappings":[]}`, "duplicate key"}, - {"rootNull", `null`, "", "JSON object"}, - {"childNull", `{"mappingFiles":["maps.json"]}`, `null`, "JSON object"}, - {"unknownPolicy", `{"mappingFiles":["maps.json"]}`, `{"mappings":[{"unknown":true}]}`, "unknown field"}, - {"trailingValue", `{"mappingFiles":["maps.json"]}`, `{"mappings":[]} {}`, "maps.json"}, - {"tooLarge", `{"mappingFiles":["maps.json"]}`, strings.Repeat(" ", 1<<20) + `{}`, "1 MiB"}, - } { - t.Run(tc.name, func(t *testing.T) { - dir := t.TempDir() - writeFile(t, filepath.Join(dir, generatorConfigFile), tc.root) - if tc.child != "" { - writeFile(t, filepath.Join(dir, "maps.json"), tc.child) - } - before := snapshot(t, dir) - _, _, err := readGeneratorConfig(dir) - if err == nil || !strings.Contains(err.Error(), tc.want) { +func TestRemovedMappingConfiguration(t *testing.T) { + dir := filepath.Join(t.TempDir(), "project") + if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) + } + for _, raw := range []string{`{"mappings":[]}`, `{"mappingFiles":[]}`, `{"mappings":null}`, `{"mappingFiles":["../../private.json"]}`} { + writeFile(t, filepath.Join(dir, generatorConfigFile), raw) + before := snapshot(t, dir) + for _, o := range []Options{{Dir: dir}, {Dir: dir, Check: true}, {Dir: dir, DryRun: true}} { + o.API = filepath.Join(dir, "api/service.api") + if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") { t.Fatal(err) } if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("read changed configuration") + t.Fatal("rejected config wrote project") } - }) - } - for _, directory := range []bool{false, true} { - t.Run(fmt.Sprintf("symlinkDirectory%t", directory), func(t *testing.T) { - dir := t.TempDir() - outside := t.TempDir() - writeMappingFile(t, outside, "maps.json", nil) - name := "maps.json" - target := filepath.Join(outside, name) - link := filepath.Join(dir, name) - if directory { - name = "linked/maps.json" - target = outside - link = filepath.Join(dir, "linked") - } - if err := os.Symlink(target, link); err != nil { - t.Fatal(err) - } - writeGeneratorConfig(t, dir, GeneratorConfig{Repository: "none", Middleware: "factories", MappingFiles: []string{name}}) - if _, _, err := readGeneratorConfig(dir); err == nil || !strings.Contains(err.Error(), "symlink") { - t.Fatal(err) - } - }) + } } } - -func TestMappingFileBounds(t *testing.T) { - dir := t.TempDir() - files := make([]string, 65) - for i := range files { - files[i] = fmt.Sprintf("maps%d.json", i) - } - writeGeneratorConfig(t, dir, GeneratorConfig{Repository: "none", Middleware: "factories", MappingFiles: files}) - if _, _, err := readGeneratorConfig(dir); err == nil || !strings.Contains(err.Error(), "64") { +func TestOldMappingFileRequiresExplicitMigration(t *testing.T) { + dir := filepath.Join(t.TempDir(), "project") + if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil { t.Fatal(err) } - writeGeneratorConfig(t, dir, GeneratorConfig{Repository: "none", Middleware: "factories", MappingFiles: files[:4]}) - for _, name := range files[:4] { - writeFile(t, filepath.Join(dir, name), `{"mappings":[]}`+strings.Repeat(" ", (1<<20)-len(`{"mappings":[]}`))) - } - if _, _, err := readGeneratorConfig(dir); err == nil || !strings.Contains(err.Error(), "4 MiB") { + name := "internal/mapping/mappingsGen.go" + body := generatedHeader + "package mapping\n" + writeFile(t, filepath.Join(dir, name), body) + var m manifest + if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil { t.Fatal(err) } - // The definition limit applies after merging, not separately to each file. - defs := make([]Mapping, 129) - for i := range defs { - defs[i] = nullableMappings()[0] - defs[i].Name = fmt.Sprintf("Map%d", i) + m.Generated[name] = digest([]byte(body)) + raw, _ := json.Marshal(m) + writeFile(t, filepath.Join(dir, manifestPath), string(raw)) + before := snapshot(t, dir) + o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")} + for _, mode := range []string{"generate", "check", "dry-run"} { + o.Check = mode == "check" + o.DryRun = mode == "dry-run" + if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") { + t.Fatal(err) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("legacy file deleted or project changed") + } } - writeGeneratorConfig(t, dir, GeneratorConfig{Repository: "none", Middleware: "factories", MappingFiles: files[:2]}) - writeMappingFile(t, dir, files[0], defs[:65]) - writeMappingFile(t, dir, files[1], defs[65:]) - if _, _, err := readGeneratorConfig(dir); err == nil || !strings.Contains(err.Error(), "128") { + if err := os.Remove(filepath.Join(dir, name)); err != nil { t.Fatal(err) } - // Origins/digests are loader metadata, never serialized into user configuration. - raw, err := json.Marshal(GeneratorConfig{inputDigest: "private"}) - if err != nil || strings.Contains(string(raw), "private") { - t.Fatal(string(raw), err) + o.Check = false + o.DryRun = false + if _, err := Generate(o); err != nil { + t.Fatal(err) + } + o.Check = true + if _, err := Generate(o); err != nil { + t.Fatal(err) + } + if strings.Contains(readFile(t, filepath.Join(dir, manifestPath)), name) { + t.Fatal("old ownership retained") } } diff --git a/cmd/go-omni/apigen/mapping.go b/cmd/go-omni/apigen/mapping.go deleted file mode 100644 index fea2733..0000000 --- a/cmd/go-omni/apigen/mapping.go +++ /dev/null @@ -1,453 +0,0 @@ -package apigen - -import ( - "fmt" - "go/ast" - "go/format" - goparser "go/parser" - gotoken "go/token" - "go/types" - "sort" - "strconv" - "strings" - - "golang.org/x/mod/module" -) - -// GoTypeRef names an existing, exported, non-generic Go struct. -type GoTypeRef struct { - Package string `json:"package"` - Type string `json:"type"` -} -type GoFunctionRef struct { - Package string `json:"package"` - Name string `json:"name"` -} -type MappingField struct { - Source string `json:"source"` - Convert *GoFunctionRef `json:"convert,omitempty"` - Mapping string `json:"mapping,omitempty"` - Elements bool `json:"elements,omitempty"` - Nil string `json:"nil,omitempty"` -} -type Mapping struct { - Name string `json:"name"` - PointerResult bool `json:"pointerResult,omitempty"` - origin string - Source GoTypeRef `json:"source"` - Target GoTypeRef `json:"target"` - Fields map[string]MappingField `json:"fields,omitempty"` - IgnoreSource map[string]string `json:"ignoreSource,omitempty"` - IgnoreTarget map[string]string `json:"ignoreTarget,omitempty"` -} -type mappingField struct { - path, name string - typ types.Type -} -type mappingTypes struct{ source, target types.Type } -type mappingRenderer struct { - importer types.Importer - imports map[string]string - definitions map[string]Mapping - types map[string]mappingTypes -} - -func renderMappings(o Options, generated, seeds map[string][]byte) error { - if len(o.generation.Mappings) == 0 { - return nil - } - refs, err := validateMappings(o.generation.Mappings, o.Module) - if err != nil { - return err - } - importer, close, err := loadMappingTypes(o, refs, generated, seeds) - if err != nil { - return err - } - defer close() - r := mappingRenderer{importer: importer, imports: map[string]string{}, definitions: map[string]Mapping{}, types: map[string]mappingTypes{}} - for _, m := range o.generation.Mappings { - source, err := r.lookup(m.Source) - if err != nil { - return err - } - target, err := r.lookup(m.Target) - if err != nil { - return err - } - r.definitions[m.Name] = m - r.types[m.Name] = mappingTypes{source, target} - } - names := make([]string, 0, len(r.definitions)) - for name := range r.definitions { - names = append(names, name) - } - sort.Strings(names) - var body strings.Builder - for _, name := range names { - source, err := r.render(r.definitions[name]) - if err != nil { - return fmt.Errorf("%s: mapping %s: %w", mappingOrigin(r.definitions[name]), name, err) - } - body.WriteString(source) - } - imports := make([]string, 0, len(r.imports)) - for path := range r.imports { - imports = append(imports, path) - } - sort.Strings(imports) - text := generatedHeader + "package mapping\nimport(\n" - for _, path := range imports { - text += r.imports[path] + " " + strconv.Quote(path) + "\n" - } - text += ")\n" + body.String() - code, err := format.Source([]byte(text)) - if err != nil { - return fmt.Errorf("format mappings: %w", err) - } - // The generated package is type-checked before it is written. No application - // code (including init or converters) is executed during generation. - fset := gotoken.NewFileSet() - file, err := goparser.ParseFile(fset, "mappingsGen.go", code, 0) - if err != nil { - return err - } - check := types.Config{Importer: importer} - if _, err = check.Check(o.Module+"/internal/mapping", fset, []*ast.File{file}, nil); err != nil { - return fmt.Errorf("generated mapping type check: %w", err) - } - generated["internal/mapping/mappingsGen.go"] = code - return nil -} - -func validateMappings(defs []Mapping, project string) ([]string, error) { - refs := map[string]bool{} - names := map[string]string{} - validPackage := func(path string) bool { - if module.CheckImportPath(path) != nil || path == project+"/internal/mapping" { - return false - } - // go/types does not enforce the compiler's internal import visibility. - if strings.HasPrefix(path, "internal/") { - return false - } - if i := strings.LastIndex(path, "/internal/"); i >= 0 { - return strings.HasPrefix(project+"/internal/mapping", path[:i]+"/") - } - return true - } - for _, m := range defs { - if !gotoken.IsIdentifier(m.Name) || !gotoken.IsExported(m.Name) { - return nil, fmt.Errorf("mapping name %q must be unique and exported", m.Name) - } - functions := []string{m.Name} - if m.PointerResult { - functions = append(functions, m.Name+"Ptr") - } - for _, name := range functions { - if prior, exists := names[name]; exists { - return nil, fmt.Errorf("mapping function %q must be unique: %s conflicts with %s", name, mappingOrigin(m), prior) - } - names[name] = mappingOrigin(m) - } - for _, ref := range []GoTypeRef{m.Source, m.Target} { - if !validPackage(ref.Package) || !gotoken.IsIdentifier(ref.Type) || !gotoken.IsExported(ref.Type) { - return nil, fmt.Errorf("mapping %s: invalid Go type reference", m.Name) - } - refs[ref.Package] = true - } - if len(m.Fields) > 4096 || len(m.IgnoreSource) > 4096 || len(m.IgnoreTarget) > 4096 { - return nil, fmt.Errorf("mapping %s: field rule limit exceeded", m.Name) - } - for target, f := range m.Fields { - if target == "" || f.Source == "" { - return nil, fmt.Errorf("mapping %s: explicit field rule requires target and source paths", m.Name) - } - if f.Nil != "" && f.Nil != "preserve" && f.Nil != "zero" && f.Nil != "error" { - return nil, fmt.Errorf("mapping %s.%s: nil must be preserve, zero or error", m.Name, target) - } - if f.Convert != nil && f.Nil != "" { - return nil, fmt.Errorf("mapping %s.%s: nil policy cannot be combined with a converter; the converter owns nil behavior", m.Name, target) - } - if f.Convert != nil { - if f.Mapping != "" || !validPackage(f.Convert.Package) || !gotoken.IsIdentifier(f.Convert.Name) || !gotoken.IsExported(f.Convert.Name) { - return nil, fmt.Errorf("mapping %s.%s: invalid or conflicting converter", m.Name, target) - } - refs[f.Convert.Package] = true - } - } - } - visiting, done := map[string]bool{}, map[string]bool{} - byName := map[string]Mapping{} - for _, m := range defs { - byName[m.Name] = m - } - var visit func(string) error - visit = func(name string) error { - if done[name] { - return nil - } - if visiting[name] { - return fmt.Errorf("mapping cycle at %s", name) - } - m, ok := byName[name] - if !ok { - return fmt.Errorf("unknown mapping %s", name) - } - visiting[name] = true - for _, f := range m.Fields { - if f.Mapping != "" { - if err := visit(f.Mapping); err != nil { - return err - } - } - } - delete(visiting, name) - done[name] = true - return nil - } - for _, m := range defs { - if err := visit(m.Name); err != nil { - return nil, err - } - } - out := make([]string, 0, len(refs)) - for path := range refs { - out = append(out, path) - } - sort.Strings(out) - return out, nil -} - -func (r *mappingRenderer) qualifier(p *types.Package) string { - if alias, ok := r.imports[p.Path()]; ok { - return alias - } - alias := fmt.Sprintf("pkg%d", len(r.imports)) - r.imports[p.Path()] = alias - return alias -} -func (r *mappingRenderer) typeName(t types.Type) string { return types.TypeString(t, r.qualifier) } -func (r *mappingRenderer) lookup(ref GoTypeRef) (types.Type, error) { - pkg, err := r.importer.Import(ref.Package) - if err != nil { - return nil, fmt.Errorf("mapping type %s.%s: %w", ref.Package, ref.Type, err) - } - obj, ok := pkg.Scope().Lookup(ref.Type).(*types.TypeName) - if !ok { - return nil, fmt.Errorf("mapping type %s.%s not found", ref.Package, ref.Type) - } - typ := obj.Type() - if n, ok := typ.(*types.Named); ok && n.TypeParams() != nil && n.TypeParams().Len() > 0 { - return nil, fmt.Errorf("mapping root %s must be a non-generic struct", ref.Type) - } - if _, ok := typ.Underlying().(*types.Struct); !ok { - return nil, fmt.Errorf("mapping root %s must be a struct", ref.Type) - } - return typ, nil -} - -func mappingFields(typ types.Type) (map[string]mappingField, error) { - out := map[string]mappingField{} - leaves := map[string]string{} - var walk func(types.Type, string, int) error - walk = func(typ types.Type, prefix string, depth int) error { - if depth > 32 { - return fmt.Errorf("embedded field nesting exceeds 32") - } - st, ok := typ.Underlying().(*types.Struct) - if !ok { - return fmt.Errorf("embedded field %s must be a value struct", prefix) - } - for i := 0; i < st.NumFields(); i++ { - f := st.Field(i) - path := prefix + f.Name() - if !f.Exported() { - return fmt.Errorf("unexported field %s cannot be mapped; use an explicit converter on the containing value", path) - } - if f.Embedded() { - if err := walk(f.Type(), path+".", depth+1); err != nil { - return err - } - continue - } - if previous, exists := leaves[f.Name()]; exists { - return fmt.Errorf("ambiguous embedded fields %s and %s", previous, path) - } - leaves[f.Name()] = path - out[path] = mappingField{path, f.Name(), f.Type()} - if len(out) > 4096 { - return fmt.Errorf("mapping field limit exceeded") - } - } - return nil - } - if err := walk(typ, "", 0); err != nil { - return nil, err - } - return out, nil -} - -func (r *mappingRenderer) render(m Mapping) (string, error) { - mt := r.types[m.Name] - source, err := mappingFields(mt.source) - if err != nil { - return "", err - } - target, err := mappingFields(mt.target) - if err != nil { - return "", err - } - for key, reason := range m.IgnoreSource { - if _, ok := source[key]; !ok || strings.TrimSpace(reason) == "" { - return "", fmt.Errorf("ignoreSource %s requires an existing field and reason", key) - } - } - for key, reason := range m.IgnoreTarget { - if _, ok := target[key]; !ok || strings.TrimSpace(reason) == "" { - return "", fmt.Errorf("ignoreTarget %s requires an existing field and reason", key) - } - } - for key := range m.Fields { - if _, ok := target[key]; !ok { - return "", fmt.Errorf("unknown target field %s", key) - } - if _, ignored := m.IgnoreTarget[key]; ignored { - return "", fmt.Errorf("target %s is both mapped and ignored", key) - } - } - names := make([]string, 0, len(target)) - for name := range target { - names = append(names, name) - } - sort.Strings(names) - used := map[string]bool{} - text := fmt.Sprintf("// %s maps one value. Pointer/map/slice assignments retain Go reference semantics.\nfunc %s(source %s)(%s,error){\nvar result %s\n", m.Name, m.Name, r.typeName(mt.source), r.typeName(mt.target), r.typeName(mt.target)) - for _, path := range names { - dst := target[path] - if reason, ignored := m.IgnoreTarget[path]; ignored { - text += "// Target " + path + " is intentionally left at zero; see omni.gen.json.\n" - _ = reason - continue - } - rule, explicit := m.Fields[path] - if !explicit { - for key, f := range source { - if f.name == dst.name { - rule.Source = key - break - } - } - } - src, ok := source[rule.Source] - if !ok { - return "", fmt.Errorf("target %s has no source; add a field rule or ignoreTarget reason", path) - } - if _, ignored := m.IgnoreSource[src.path]; ignored { - return "", fmt.Errorf("source %s is both mapped and ignored", src.path) - } - used[src.path] = true - from, to := src.typ, dst.typ - srcExpr, dstExpr := "source."+src.path, "result."+path - if rule.Elements { - switch d := to.Underlying().(type) { - case *types.Slice: - s, ok := from.Underlying().(*types.Slice) - if !ok { - return "", fmt.Errorf("%s: elements requires slice to slice or equal-length array to array", path) - } - text += fmt.Sprintf("if %s!=nil {\n%s=make(%s,len(%s))\nfor index:=range %s {\n", srcExpr, dstExpr, r.typeName(to), srcExpr, srcExpr) - from, to = s.Elem(), d.Elem() - case *types.Array: - s, ok := from.Underlying().(*types.Array) - if !ok || s.Len() != d.Len() { - return "", fmt.Errorf("%s: array lengths must match", path) - } - text += fmt.Sprintf("for index:=range %s {\n", srcExpr) - from, to = s.Elem(), d.Elem() - default: - return "", fmt.Errorf("%s: elements requires a slice or array", path) - } - srcExpr += "[index]" - dstExpr += "[index]" - } - expr, fallible, err := r.expression(rule, from, to, srcExpr) - if err != nil { - return "", fmt.Errorf("%s -> %s: %w", src.path, path, err) - } - if fallible { - fmtPkg, err := r.importer.Import("fmt") - if err != nil { - return "", err - } - alias := r.qualifier(fmtPkg) - message := m.Name + "." + path + ": %w" - args := "err" - if rule.Elements { - message = m.Name + "." + path + "[%d]: %w" - args = "index,err" - } - text += fmt.Sprintf("{ value,err:=%s\nif err!=nil{var zero %s;return zero,%s.Errorf(%q,%s)}\n%s=value\n}\n", expr, r.typeName(mt.target), alias, message, args, dstExpr) - } else { - text += dstExpr + "=" + expr + "\n" - } - if rule.Elements { - text += "}\n" - if _, ok := dst.typ.Underlying().(*types.Slice); ok { - text += "}\n" - } - } - } - keys := make([]string, 0, len(source)) - for key := range source { - keys = append(keys, key) - } - sort.Strings(keys) - for _, key := range keys { - if !used[key] { - if _, ignored := m.IgnoreSource[key]; !ignored { - return "", fmt.Errorf("unmapped source field %s; map it or provide an ignoreSource reason", key) - } - } - } - text += "return result,nil\n}\n" - if m.PointerResult { - text += fmt.Sprintf("// %sPtr returns nil on conversion failure and otherwise a pointer to the result.\nfunc %sPtr(source %s)(*%s,error){\nvalue,err:=%s(source)\nif err!=nil{return nil,err}\nreturn &value,nil\n}\n", m.Name, m.Name, r.typeName(mt.source), r.typeName(mt.target), m.Name) - } - return text, nil -} - -func (r *mappingRenderer) expression(rule MappingField, from, to types.Type, source string) (string, bool, error) { - if rule.Mapping != "" { - return r.nestedExpression(rule, from, to, source) - } - if rule.Nil != "" { - return r.dereferenceExpression(rule, from, to, source) - } - if rule.Convert != nil { - pkg, err := r.importer.Import(rule.Convert.Package) - if err != nil { - return "", false, err - } - fn, ok := pkg.Scope().Lookup(rule.Convert.Name).(*types.Func) - if !ok { - return "", false, fmt.Errorf("converter function %s not found", rule.Convert.Name) - } - sig := fn.Type().(*types.Signature) - if sig.Variadic() || (sig.TypeParams() != nil && sig.TypeParams().Len() > 0) || sig.Params().Len() != 1 || sig.Results().Len() < 1 || sig.Results().Len() > 2 { - return "", false, fmt.Errorf("converter must have signature func(Source) Target or func(Source) (Target,error)") - } - if !types.Identical(sig.Params().At(0).Type(), from) || !types.Identical(sig.Results().At(0).Type(), to) { - return "", false, fmt.Errorf("converter signature does not match field types") - } - fallible := sig.Results().Len() == 2 - if fallible && !types.Identical(sig.Results().At(1).Type(), types.Universe.Lookup("error").Type()) { - return "", false, fmt.Errorf("converter second result must be error") - } - return r.qualifier(pkg) + "." + rule.Convert.Name + "(" + source + ")", fallible, nil - } - if !types.Identical(from, to) { - return "", false, fmt.Errorf("types %s and %s differ; provide an explicit converter", types.TypeString(from, nil), types.TypeString(to, nil)) - } - return source, false, nil -} diff --git a/cmd/go-omni/apigen/mappingLoad.go b/cmd/go-omni/apigen/mappingLoad.go deleted file mode 100644 index 320df90..0000000 --- a/cmd/go-omni/apigen/mappingLoad.go +++ /dev/null @@ -1,175 +0,0 @@ -package apigen - -import ( - "bytes" - "context" - "encoding/json" - "errors" - "fmt" - "go/importer" - gotoken "go/token" - "go/types" - "io" - "os" - "os/exec" - "path/filepath" - "sort" - "strings" - "time" - - "golang.org/x/mod/modfile" -) - -// loadMappingTypes uses the Go compiler's export data, including the pending -// DTOs via an overlay. Module files and sum updates remain in a temporary dir. -// It compiles only explicitly selected packages and their dependencies; never -// runs project code. Build tags/GOOS follow the caller; GOWORK is disabled. -func loadMappingTypes(o Options, refs []string, generated, seeds map[string][]byte) (types.Importer, func(), error) { - // go list resolves its working directory (notably /var -> /private/var on - // macOS). Overlay keys must name that same physical path or stale DTOs win. - o.Dir = canonical(o.Dir) - temp, err := os.MkdirTemp("", "go-omni-mapping-*") - if err != nil { - return nil, nil, err - } - cleanup := func() { _ = os.RemoveAll(temp) } - fail := func(err error) (types.Importer, func(), error) { cleanup(); return nil, nil, err } - raw, err := os.ReadFile(filepath.Join(o.Dir, "go.mod")) - if err != nil { - return fail(errors.New("mapping generation requires an existing Go module; scaffold first and resolve dependencies")) - } - mf, err := modfile.Parse("go.mod", raw, nil) - if err != nil { - return fail(err) - } - replacements := append([]*modfile.Replace(nil), mf.Replace...) - for _, r := range replacements { - if r.New.Version == "" && !filepath.IsAbs(r.New.Path) { - if err = mf.AddReplace(r.Old.Path, r.Old.Version, filepath.Join(o.Dir, r.New.Path), ""); err != nil { - return fail(err) - } - } - } - raw, err = mf.Format() - if err != nil { - return fail(err) - } - modPath := filepath.Join(temp, "go.mod") - if err = os.WriteFile(modPath, raw, 0600); err != nil { - return fail(err) - } - if sums, e := os.ReadFile(filepath.Join(o.Dir, "go.sum")); e == nil { - if err = os.WriteFile(filepath.Join(temp, "go.sum"), sums, 0600); err != nil { - return fail(err) - } - } else if !errors.Is(e, os.ErrNotExist) { - return fail(e) - } - overlay := map[string]string{} - // Hide stale owned Go files, which may otherwise redeclare a renamed DTO. - var previous manifest - if data, e := os.ReadFile(filepath.Join(o.Dir, manifestPath)); e == nil { - if err = json.Unmarshal(data, &previous); err != nil { - return fail(err) - } - for name := range previous.Generated { - if strings.HasSuffix(name, ".go") { - if !ownedPath(name) { - return fail(fmt.Errorf("invalid owned path %s", name)) - } - if err = safePath(o.Dir, name); err != nil { - return fail(err) - } - if _, ok := generated[name]; !ok { - overlay[filepath.Join(o.Dir, filepath.FromSlash(name))] = "" - } - } - } - } else if !errors.Is(e, os.ErrNotExist) { - return fail(e) - } - for i, files := range []map[string][]byte{generated, seeds} { - names := make([]string, 0, len(files)) - for name := range files { - if strings.HasSuffix(name, ".go") { - names = append(names, name) - } - } - sort.Strings(names) - for _, name := range names { - if err = safePath(o.Dir, name); err != nil { - return fail(err) - } - path := filepath.Join(o.Dir, filepath.FromSlash(name)) - if i == 1 { - if _, e := os.Stat(path); e == nil { - continue - } else if !errors.Is(e, os.ErrNotExist) { - return fail(e) - } - } - backing := filepath.Join(temp, fmt.Sprintf("overlay%d.go", len(overlay))) - if err = os.WriteFile(backing, files[name], 0600); err != nil { - return fail(err) - } - overlay[path] = backing - } - } - raw, err = json.Marshal(map[string]any{"Replace": overlay}) - if err != nil { - return fail(err) - } - overlayPath := filepath.Join(temp, "overlay.json") - if err = os.WriteFile(overlayPath, raw, 0600); err != nil { - return fail(err) - } - args := []string{"list", "-deps", "-export", "-json", "-mod=readonly", "-modfile", modPath, "-overlay", overlayPath} - // fmt is used for fallible conversions even if neither struct imports it. - args = append(args, "fmt") - args = append(args, refs...) - ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) - defer cancel() - cmd := exec.CommandContext(ctx, "go", args...) - cmd.Dir = o.Dir - cmd.Env = append(os.Environ(), "GOWORK=off") - cmd.WaitDelay = 2 * time.Second - stdout := &limitedCatalogBuffer{limit: 32 << 20} - stderr := &limitedCatalogBuffer{limit: 64 << 10} - cmd.Stdout = stdout - cmd.Stderr = stderr - if err = cmd.Run(); err != nil { - if ctx.Err() != nil { - return fail(fmt.Errorf("mapping type loading: %w", ctx.Err())) - } - // Go download errors may contain authenticated remote URLs. Keep arbitrary - // command output out of diagnostics; exact mapping errors follow type loading. - return fail(fmt.Errorf("mapping type loading failed (%v); run go list -deps -export for the configured source/target/converter packages; check dependencies, build tags and import cycles", err)) - } - exports := map[string]string{} - dec := json.NewDecoder(bytes.NewReader(stdout.Bytes())) - for { - var p struct { - ImportPath, Export string - Error *struct{ Err string } - } - if err = dec.Decode(&p); err == io.EOF { - break - } else if err != nil { - return fail(err) - } - if p.Error != nil { - return fail(fmt.Errorf("mapping package %s could not be compiled", p.ImportPath)) - } - if p.Export != "" { - exports[p.ImportPath] = p.Export - } - } - imp := importer.ForCompiler(gotoken.NewFileSet(), "gc", func(path string) (io.ReadCloser, error) { - file, ok := exports[path] - if !ok { - return nil, fmt.Errorf("no export data for mapping dependency %s", path) - } - return os.Open(file) - }) - return imp, cleanup, nil -} diff --git a/cmd/go-omni/apigen/mappingNullable_test.go b/cmd/go-omni/apigen/mappingNullable_test.go deleted file mode 100644 index 23b1e53..0000000 --- a/cmd/go-omni/apigen/mappingNullable_test.go +++ /dev/null @@ -1,250 +0,0 @@ -package apigen - -import ( - "context" - "encoding/json" - "fmt" - "io" - "os" - "os/exec" - "path/filepath" - "strings" - "testing" - "time" -) - -func nullableMappings() []Mapping { - pkg := "example.com/nullable/internal/" - ref := func(p, name string) GoTypeRef { return GoTypeRef{pkg + p, name} } - child := Mapping{Name: "ToChild", Source: ref("types", "Child"), Target: ref("domain", "Child"), Fields: map[string]MappingField{ - "Name": {Source: "Name", Convert: &GoFunctionRef{pkg + "convert", "Name"}}, - }} - parent := Mapping{Name: "ToParent", PointerResult: true, Source: ref("types", "Parent"), Target: ref("domain", "Parent"), Fields: map[string]MappingField{ - "Child": {Source: "Child", Mapping: "ToChild"}, "Children": {Source: "Children", Mapping: "ToChild", Elements: true}, - }} - defs := []Mapping{child, parent, - {Name: "ToWireChild", Source: ref("domain", "Child"), Target: ref("types", "Child")}, - {Name: "ToWire", PointerResult: true, Source: ref("domain", "Parent"), Target: ref("types", "Parent"), Fields: map[string]MappingField{ - "Child": {Source: "Child", Mapping: "ToWireChild"}, "Children": {Source: "Children", Mapping: "ToWireChild", Elements: true}, - }}, - {Name: "ToChildDTO", Source: ref("domain", "Child"), Target: ref("domain", "ChildDTO"), Fields: map[string]MappingField{"Name": {Source: "Name", Convert: &GoFunctionRef{pkg + "convert", "Name"}}}}, - {Name: "Lift", Source: ref("domain", "ValueHolder"), Target: ref("domain", "PointerHolder"), Fields: map[string]MappingField{"Child": {Source: "Child", Mapping: "ToChildDTO"}}}, - {Name: "Batch", Source: ref("domain", "InputBatch"), Target: ref("domain", "OutputBatch"), Fields: map[string]MappingField{"Fixed": {Source: "Fixed", Mapping: "ToChildDTO", Elements: true}}}, - {Name: "ShallowCopy", Source: ref("domain", "Parent"), Target: ref("domain", "Parent")}, - {Name: "RequirePointer", Source: ref("types", "Parent"), Target: ref("domain", "Parent"), Fields: map[string]MappingField{ - "Child": {Source: "Child", Mapping: "ToChild", Nil: "error"}, "Children": {Source: "Children", Mapping: "ToChild", Elements: true}, - }}, - } - for _, p := range []struct{ name, policy string }{{"RequiredZero", "zero"}, {"RequiredError", "error"}} { - defs = append(defs, Mapping{Name: p.name, Source: ref("types", "Parent"), Target: ref("domain", "Required"), Fields: map[string]MappingField{ - "Child": {Source: "Child", Mapping: "ToChild", Nil: p.policy}, - }, IgnoreSource: map[string]string{"Children": "this explicit projection only includes the child"}}) - } - for _, p := range []struct{ name, policy string }{{"ScalarZero", "zero"}, {"ScalarError", "error"}} { - m := Mapping{Name: p.name, Source: ref("domain", "ScalarInput"), Target: ref("domain", "ScalarOutput"), Fields: map[string]MappingField{}} - for _, name := range []string{"Text", "Count", "Enabled"} { - m.Fields[name] = MappingField{Source: name, Nil: p.policy} - } - defs = append(defs, m) - } - return defs -} - -func writeMappingFile(t *testing.T, root, name string, defs []Mapping) { - t.Helper() - raw, err := json.MarshalIndent(struct { - Mappings []Mapping `json:"mappings"` - }{defs}, "", " ") - if err != nil { - t.Fatal(err) - } - writeFile(t, filepath.Join(root, name), string(raw)+"\n") -} - -func TestNullableMappingConsumer(t *testing.T) { - if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" { - t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1") - } - root := t.TempDir() - dir := filepath.Join(root, "app") - o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/nullable/service.api"), Dir: dir, Module: "example.com/nullable", FrameworkDir: frameworkRoot(t)} - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - for path, fixture := range map[string]string{"internal/domain/model.go": "domain.go", "internal/convert/value.go": "convert.go"} { - writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/nullable/"+fixture)) - } - runCompositionGo(t, dir, "mod", "tidy") - defs := nullableMappings() - // References go across files, including forward references to ToChild. - writeMappingFile(t, dir, "mappings/parents.json", defs[1:]) - writeMappingFile(t, dir, "mappings/children.json", defs[:1]) - cfg := GeneratorConfig{Repository: "none", Middleware: "factories", MappingFiles: []string{"mappings/parents.json", "mappings/children.json"}} - writeGeneratorConfig(t, dir, cfg) - o.API = filepath.Join(dir, "api/service.api") - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - writeFile(t, filepath.Join(dir, "internal/logic/probe/roundTrip.go"), `package probe -import("context";"example.com/nullable/internal/types";"example.com/nullable/internal/mapping") -func(s *Logic)RoundTrip(ctx context.Context,req *types.Parent)(*types.Parent,error){ - value,err:=mapping.ToParent(*req);if err!=nil{return nil,err} - return mapping.ToWirePtr(value) -} -`) - writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/nullable/consumer_test.go")) - before := snapshot(t, dir) - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("regeneration changed business/config files") - } - o.Check = true - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("check wrote files") - } - runCompositionGo(t, dir, "test", "-race", "-count=1", "./...") - runCompositionGo(t, dir, "build", ".") - // Workspace preflight and check use the same split configuration loader. - workspace := filepath.Join(root, "omni.workspace.json") - writeFile(t, workspace, `{"services":[{"dir":"app"}]}`) - for _, command := range []string{"format", "generate", "check"} { - if err := nullableWorkspace(t, workspace, command); err != nil { - t.Fatal(command, err) - } - } - // All invalid configurations fail before either writing or previewing output. - for _, tc := range []struct { - name, want string - edit func([]Mapping) []Mapping - }{ - {"missingNilPolicy", "requires explicit nil", func(d []Mapping) []Mapping { - d[9].Fields["Child"] = MappingField{Source: "Child", Mapping: "ToChild"} - return d - }}, - {"preserveValue", "requires explicit nil", func(d []Mapping) []Mapping { - d[9].Fields["Child"] = MappingField{Source: "Child", Mapping: "ToChild", Nil: "preserve"} - return d - }}, - {"zeroPointer", "pointer target", func(d []Mapping) []Mapping { - d[1].Fields["Child"] = MappingField{Source: "Child", Mapping: "ToChild", Nil: "zero"} - return d - }}, - {"scalarMismatch", "identical *T to T", func(d []Mapping) []Mapping { - d[11].Fields["Text"] = MappingField{Source: "Count", Nil: "zero"} - return d - }}, - {"valueNil", "requires a pointer source", func(d []Mapping) []Mapping { - d[5].Fields["Child"] = MappingField{Source: "Child", Mapping: "ToChildDTO", Nil: "zero"} - return d - }}, - {"converterNil", "cannot be combined", func(d []Mapping) []Mapping { - f := d[0].Fields["Name"] - f.Nil = "error" - d[0].Fields["Name"] = f - return d - }}, - {"pointerNameCollision", "must be unique", func(d []Mapping) []Mapping { m := d[0]; m.Name = "ToParentPtr"; return append(d, m) }}, - {"mappingCycle", "cycle", func(d []Mapping) []Mapping { - d[0].Fields["Name"] = MappingField{Source: "Name", Mapping: "ToParent"} - return d - }}, - {"unknownMapping", "unknown mapping", func(d []Mapping) []Mapping { - d[1].Fields["Child"] = MappingField{Source: "Child", Mapping: "Missing"} - return d - }}, - {"unhandledField", "unmapped source", func(d []Mapping) []Mapping { d[9].IgnoreSource = nil; return d }}, - } { - t.Run(tc.name, func(t *testing.T) { - changed := tc.edit(nullableMappings()) - writeMappingFile(t, dir, "mappings/parents.json", changed[1:]) - writeMappingFile(t, dir, "mappings/children.json", changed[:1]) - defer writeMappingFile(t, dir, "mappings/parents.json", defs[1:]) - defer writeMappingFile(t, dir, "mappings/children.json", defs[:1]) - before := snapshot(t, dir) - for _, mode := range []string{"check", "preview", "generate"} { - current := o - current.Check = mode == "check" - current.DryRun = mode == "preview" - err := generateManagedTestProject(t, current) - if err == nil || !strings.Contains(err.Error(), tc.want) { - t.Fatal(mode, err) - } - if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("invalid mapping wrote files") - } - } - }) - } - // An edit inside a child file is picked up by --check; the config is never rewritten. - changed := nullableMappings() - changed[0].PointerResult = true - writeMappingFile(t, dir, "mappings/children.json", changed[:1]) - before = snapshot(t, dir) - if err := generateManagedTestProject(t, o); err == nil { - t.Fatal("check missed child mapping drift") - } - if err := nullableWorkspace(t, workspace, "check"); err == nil { - t.Fatal("workspace missed child mapping drift") - } - if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("drift check wrote files") - } - preview := o - preview.Check = false - preview.DryRun = true - if err := generateManagedTestProject(t, preview); err != nil { - t.Fatal(err) - } - if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("preview wrote files") - } - if err := nullableWorkspace(t, workspace, "generate"); err != nil { - t.Fatal(err) - } - if err := nullableWorkspace(t, workspace, "check"); err != nil { - t.Fatal(err) - } - if os.Getenv("GO_OMNI_RELEASE_CLI") != "" { - ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) - defer cancel() - cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule) - cmd.Dir = dir - cmd.Env = append(os.Environ(), "GOWORK=off") - data, err := cmd.CombinedOutput() - if err != nil { - t.Fatal(err, string(data)) - } - var dependency struct { - Version, Sum string - Replace any - } - if err = json.Unmarshal(data, &dependency); err != nil { - t.Fatal(err) - } - if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" { - t.Fatal("not a fixed remote consumer") - } - t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum) - } -} - -func nullableWorkspace(t *testing.T, path, command string) error { - t.Helper() - cli := os.Getenv("GO_OMNI_RELEASE_CLI") - if cli == "" { - return Workspace(path, command, io.Discard) - } - ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) - defer cancel() - cmd := exec.CommandContext(ctx, cli, "workspace", command, "--file", path) - cmd.Dir = filepath.Dir(path) - if data, err := cmd.CombinedOutput(); err != nil { - return fmt.Errorf("released workspace CLI: %w\n%s", err, data) - } - return nil -} diff --git a/cmd/go-omni/apigen/mappingPointer.go b/cmd/go-omni/apigen/mappingPointer.go deleted file mode 100644 index 10eb23a..0000000 --- a/cmd/go-omni/apigen/mappingPointer.go +++ /dev/null @@ -1,97 +0,0 @@ -package apigen - -import ( - "fmt" - "go/types" -) - -func mappingOrigin(m Mapping) string { - if m.origin != "" { - return m.origin - } - return generatorConfigFile -} - -// Pointer adaptation is only selected by an explicit nested mapping. Ordinary -// same-type pointer assignments retain their existing reference semantics. -func (r *mappingRenderer) nestedExpression(rule MappingField, from, to types.Type, source string) (string, bool, error) { - m := r.types[rule.Mapping] - fromValue, toValue := from, to - src, sourcePointer := from.Underlying().(*types.Pointer) - dst, targetPointer := to.Underlying().(*types.Pointer) - if sourcePointer { - fromValue = src.Elem() - } - if targetPointer { - toValue = dst.Elem() - } - if !types.Identical(fromValue, m.source) || !types.Identical(toValue, m.target) { - return "", false, fmt.Errorf("nested mapping %s has incompatible types", rule.Mapping) - } - if !sourcePointer && rule.Nil != "" { - return "", false, fmt.Errorf("nil policy requires a pointer source") - } - if !sourcePointer && !targetPointer { - return rule.Mapping + "(" + source + ")", true, nil - } - policy := rule.Nil - if sourcePointer { - if targetPointer { - if policy == "" { - policy = "preserve" - } - if policy != "preserve" && policy != "error" { - return "", false, fmt.Errorf("pointer target requires nil preserve or error") - } - } else if policy != "zero" && policy != "error" { - return "", false, fmt.Errorf("pointer to value mapping requires explicit nil policy zero or error") - } - } - // Keep a typed local input so the source is evaluated once. The surrounding - // renderer adds the mapping/field/index error path and returns a zero parent. - body := "var zero " + r.typeName(to) + "\n" - input := "input" - if sourcePointer { - nilResult, err := r.nilResult(policy) - if err != nil { - return "", false, err - } - body += "if input==nil {" + nilResult + "}\n" - input = "*input" - } - body += "value,err:=" + rule.Mapping + "(" + input + ")\nif err!=nil{return zero,err}\n" - if targetPointer { - body += "return &value,nil\n" - } else { - body += "return value,nil\n" - } - return "func(input " + r.typeName(from) + ")(" + r.typeName(to) + ",error){\n" + body + "}(" + source + ")", true, nil -} - -// A nil policy without a converter/mapping is deliberately limited to *T -> T. -// It never casts numbers, changes named types, or invents business defaults. -func (r *mappingRenderer) dereferenceExpression(rule MappingField, from, to types.Type, source string) (string, bool, error) { - pointer, ok := from.Underlying().(*types.Pointer) - if !ok || !types.Identical(pointer.Elem(), to) { - return "", false, fmt.Errorf("nil policy without a mapping requires identical *T to T types") - } - if rule.Nil != "zero" && rule.Nil != "error" { - return "", false, fmt.Errorf("pointer to value conversion requires nil zero or error") - } - nilResult, err := r.nilResult(rule.Nil) - if err != nil { - return "", false, err - } - return "func(input " + r.typeName(from) + ")(" + r.typeName(to) + ",error){\nvar zero " + r.typeName(to) + "\nif input==nil{" + nilResult + "}\nreturn *input,nil\n}(" + source + ")", true, nil -} - -func (r *mappingRenderer) nilResult(policy string) (string, error) { - if policy != "error" { - return "return zero,nil", nil - } - pkg, err := r.importer.Import("fmt") - if err != nil { - return "", err - } - return "return zero," + r.qualifier(pkg) + ".Errorf(\"nil source is not allowed\")", nil -} diff --git a/cmd/go-omni/apigen/mapping_test.go b/cmd/go-omni/apigen/mapping_test.go deleted file mode 100644 index 448aa5c..0000000 --- a/cmd/go-omni/apigen/mapping_test.go +++ /dev/null @@ -1,266 +0,0 @@ -package apigen - -import ( - "context" - "encoding/json" - "fmt" - "os" - "os/exec" - "path/filepath" - "strings" - "testing" - "time" -) - -func compositionConfig() GeneratorConfig { - pkg := "example.com/composition/internal/" - cfg := GeneratorConfig{Repository: "none", Middleware: "factories"} - for _, name := range []string{"Enterprise", "Tenant", "Admin"} { - fields := map[string]MappingField{ - "EmployeeName": {Source: "Name", Convert: &GoFunctionRef{pkg + "convert", "Name"}}, - "Salary": {Source: "Salary", Convert: &GoFunctionRef{pkg + "convert", "Money"}}, - "Enabled": {Source: "Enabled", Convert: &GoFunctionRef{pkg + "convert", "Bool"}}, - } - m := Mapping{Name: "From" + name, Source: GoTypeRef{pkg + "types", name + "Request"}, Target: GoTypeRef{pkg + "domain", "Employee"}, Fields: fields} - if name != "Enterprise" { - m.IgnoreSource = map[string]string{"CompanyID": "Authorized separately by Logic; never copy path scope into body"} - } - cfg.Mappings = append(cfg.Mappings, m) - } - cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToResponse", Source: GoTypeRef{pkg + "domain", "Result"}, Target: GoTypeRef{pkg + "types", "EmployeeResponse"}, Fields: map[string]MappingField{"ID": {Source: "ID", Convert: &GoFunctionRef{pkg + "convert", "ID"}}}}) - cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToRow", Source: GoTypeRef{pkg + "domain", "Row"}, Target: GoTypeRef{pkg + "domain", "RowDTO"}}) - cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToBatch", Source: GoTypeRef{pkg + "domain", "Batch"}, Target: GoTypeRef{pkg + "domain", "BatchDTO"}, Fields: map[string]MappingField{ - "Rows": {Source: "Rows", Mapping: "ToRow", Elements: true}, "Fixed": {Source: "Fixed", Mapping: "ToRow", Elements: true}, - }}) - return cfg -} -func writeGeneratorConfig(t *testing.T, dir string, cfg GeneratorConfig) { - t.Helper() - raw, err := json.MarshalIndent(cfg, "", " ") - if err != nil { - t.Fatal(err) - } - writeFile(t, filepath.Join(dir, generatorConfigFile), string(raw)+"\n") -} -func runCompositionGo(t *testing.T, dir string, args ...string) { - t.Helper() - ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) - defer cancel() - cmd := exec.CommandContext(ctx, "go", args...) - cmd.Dir = dir - cmd.Env = append(os.Environ(), "GOWORK=off") - if out, err := cmd.CombinedOutput(); err != nil { - t.Fatalf("go %v: %v\n%s", args, err, out) - } -} - -func TestCompositionMappingConsumer(t *testing.T) { - if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" { - t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1") - } - dir := filepath.Join(t.TempDir(), "app") - o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)} - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - fixtures := map[string]string{ - "internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go", - "internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go", - } - for path, fixture := range fixtures { - writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture)) - } - svcFile := filepath.Join(dir, "internal/svc/serviceContext.go") - source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1) - source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1) - writeFile(t, svcFile, source) - configFile := filepath.Join(dir, "internal/config/config.go") - writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1)) - // Resolve dependencies before opting into compile-time mapping. The first - // generation creates DTOs; the second sees current DTOs through an overlay. - // Logic does not yet import the generated mapping package. - testSource := readFile(t, filepath.Join(dir, "consumer_test.go")) - if err := os.Remove(filepath.Join(dir, "consumer_test.go")); err != nil { - t.Fatal(err) - } - runCompositionGo(t, dir, "mod", "tidy") - cfg := compositionConfig() - writeGeneratorConfig(t, dir, cfg) - o.API = filepath.Join(dir, "api/service.api") - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - writeFile(t, filepath.Join(dir, "consumer_test.go"), testSource) - for _, group := range []string{"Enterprise", "Tenant", "Admin"} { - text := fmt.Sprintf(`package %s -import("context";"example.com/composition/internal/types";"example.com/composition/internal/mapping";"example.com/composition/internal/domain") -func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){ - input,err:=mapping.From%s(*req);if err!=nil{return nil,err} - result,err:=mapping.ToResponse(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil -} -`, strings.ToLower(group), group, group) - writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text) - } - before := snapshot(t, dir) - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("regeneration changed project") - } - o.Check = true - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - if !sameFiles(before, snapshot(t, dir)) { - t.Fatal("check changed project") - } - runCompositionGo(t, dir, "test", "-race", "-count=1", "./...") - runCompositionGo(t, dir, "build", ".") - generated := readFile(t, filepath.Join(dir, "internal/mapping/mappingsGen.go")) - if strings.Contains(generated, "encoding/json") || strings.Contains(generated, "reflect") || !strings.Contains(generated, "result.EmployeeName = value") { - t.Fatal("mapping does not emit typed assignments") - } - // Adding a source or target field is detected before any files are written. - for _, tc := range []struct{ name, path, before, after, want string }{ - {"source", o.API, "Name *string", "NewField *string `json:\"newField,optional\"`\n Name *string", "unmapped source field NewField"}, - {"target", filepath.Join(dir, "internal/domain/employee.go"), "EmployeeName string", "Unmapped string\n EmployeeName string", "target Unmapped has no source"}, - } { - t.Run(tc.name, func(t *testing.T) { - original := readFile(t, tc.path) - writeFile(t, tc.path, strings.Replace(original, tc.before, tc.after, 1)) - defer writeFile(t, tc.path, original) - snap := snapshot(t, dir) - if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) { - t.Fatal(err) - } - if !sameFiles(snap, snapshot(t, dir)) { - t.Fatal("failed check wrote files") - } - o.Check = false - defer func() { o.Check = true }() - if err := generateManagedTestProject(t, o); err == nil { - t.Fatal("generate accepted missing mapping") - } - if !sameFiles(snap, snapshot(t, dir)) { - t.Fatal("failed generate wrote files") - } - }) - } - for _, tc := range []struct { - name string - edit func(*GeneratorConfig) - want string - }{ - {"pointerPolicy", func(c *GeneratorConfig) { delete(c.Mappings[0].Fields, "Salary") }, "provide an explicit converter"}, - {"unknownSource", func(c *GeneratorConfig) { c.Mappings[0].Fields["Salary"] = MappingField{Source: "Missing"} }, "has no source"}, - {"wrongConverter", func(c *GeneratorConfig) { - f := c.Mappings[0].Fields["Salary"] - f.Convert.Name = "ID" - c.Mappings[0].Fields["Salary"] = f - }, "signature does not match"}, - {"unhandledScope", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource = nil }, "unmapped source field CompanyID"}, - {"badIgnore", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource["Missing"] = "stale rule" }, "ignoreSource Missing"}, - {"arrayLength", func(c *GeneratorConfig) { - f := c.Mappings[5].Fields["Fixed"] - f.Source = "Rows" - c.Mappings[5].Fields["Fixed"] = f - }, "array lengths must match"}, - } { - t.Run(tc.name, func(t *testing.T) { - changed := compositionConfig() - tc.edit(&changed) - writeGeneratorConfig(t, dir, changed) - defer writeGeneratorConfig(t, dir, cfg) - snap := snapshot(t, dir) - if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) { - t.Fatal(err) - } - if !sameFiles(snap, snapshot(t, dir)) { - t.Fatal("invalid rule wrote files") - } - }) - } - // All diagnostics exercise current DTOs, not stale exported generated types. - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - // Exercise the repository switch with the same actual CLI as the mapper. - if _, err := os.Stat(filepath.Join(dir, "internal/repository")); !os.IsNotExist(err) { - t.Fatalf("default repository skeleton: %v", err) - } - o.Check = false - cfg.Repository = "group" - writeGeneratorConfig(t, dir, cfg) - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - repo := filepath.Join(dir, "internal/repository/enterprise/repository.go") - writeFile(t, repo, readFile(t, repo)+"\n// Project-owned persistence.\n") - cfg.Repository = "none" - writeGeneratorConfig(t, dir, cfg) - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - if !strings.Contains(readFile(t, repo), "Project-owned persistence.") { - t.Fatal("disabling skeleton overwrote business code") - } - o.Check = true - if err := generateManagedTestProject(t, o); err != nil { - t.Fatal(err) - } - if os.Getenv("GO_OMNI_RELEASE_CLI") != "" { - ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) - defer cancel() - cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule) - cmd.Dir = dir - cmd.Env = append(os.Environ(), "GOWORK=off") - data, err := cmd.CombinedOutput() - if err != nil { - t.Fatal(err, string(data)) - } - var dependency struct { - Version, Sum string - Replace any - } - if err = json.Unmarshal(data, &dependency); err != nil { - t.Fatal(err) - } - if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" { - t.Fatal("not a fixed remote consumer") - } - t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum) - } -} - -func TestMappingConfigurationRejection(t *testing.T) { - cases := []struct { - name string - edit func(*GeneratorConfig) - want string - }{ - {"cycle", func(c *GeneratorConfig) { - c.Mappings[4].Fields = map[string]MappingField{"Value": {Source: "Value", Mapping: "ToRow"}} - }, "cycle"}, - {"unknown", func(c *GeneratorConfig) { - c.Mappings[5].Fields["Rows"] = MappingField{Source: "Rows", Mapping: "Missing", Elements: true} - }, "unknown mapping"}, - {"conflict", func(c *GeneratorConfig) { - f := c.Mappings[0].Fields["Salary"] - f.Mapping = "ToRow" - c.Mappings[0].Fields["Salary"] = f - }, "conflicting converter"}, - {"internal", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.net/private/internal/types" }, "invalid Go type"}, - {"outputCycle", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.com/composition/internal/mapping" }, "invalid Go type"}, - {"duplicates", func(c *GeneratorConfig) { c.Mappings[1].Name = c.Mappings[0].Name }, "must be unique"}, - } - for _, tc := range cases { - t.Run(tc.name, func(t *testing.T) { - cfg := compositionConfig() - tc.edit(&cfg) - if _, err := validateMappings(cfg.Mappings, "example.com/composition"); err == nil || !strings.Contains(err.Error(), tc.want) { - t.Fatal(err) - } - }) - } -} diff --git a/cmd/go-omni/apigen/templates.go b/cmd/go-omni/apigen/templates.go index c1bfee6..1c53758 100644 --- a/cmd/go-omni/apigen/templates.go +++ b/cmd/go-omni/apigen/templates.go @@ -56,10 +56,11 @@ factories for new projects. Named middleware factories run after svc.Init; implement Dependencies accessors on ServiceContext and New(deps) once. Unimplemented/nil factories fail startup and clean registered dependencies. Existing projects retain middleware: handlers until explicitly migrated. -Optional mappings generate typed assignments in internal/mapping/mappingsGen.go; -configure source/target types and explicit conversion rules after resolving module -dependencies. Missing fields fail generation and --check. See GENERATION.md in -the framework repository for composition, null/reference semantics and migration. +Ordinary Logic uses the generated Request/Response types directly. Where an +independent business type is needed, use ordinary Go or the optional copyx package. +There are no mappings/mappingFiles settings or generated mapping files in v0.19. +See COPY.md in the framework repository for copy policies and migration steps, +and GENERATION.md for field composition and middleware factories. ## Run @@ -145,8 +146,7 @@ internal/handler// # generated binding and Logic adapters internal/logic// # developer-owned business operations internal/repository// # optional; repository: group creates editable seeds internal/middleware/ # typed dependency interface, factories and registry -omni.gen.json # developer-owned generator options and mappings -internal/mapping/*Gen.go # optional compile-checked struct mappings +omni.gen.json # developer-owned repository/middleware options docs/openapi.json # generated OpenAPI 3.0.3 .goOmni/apiManifest.json # ownership and content hashes `+"```"+` diff --git a/cmd/go-omni/apigen/testdata/composition/consumer_test.go b/cmd/go-omni/apigen/testdata/composition/consumer_test.go index ee451d0..2eec80a 100644 --- a/cmd/go-omni/apigen/testdata/composition/consumer_test.go +++ b/cmd/go-omni/apigen/testdata/composition/consumer_test.go @@ -18,7 +18,6 @@ import ( "example.com/composition/internal/convert" "example.com/composition/internal/domain" "example.com/composition/internal/handler" - "example.com/composition/internal/mapping" "example.com/composition/internal/middleware" "example.com/composition/internal/svc" "example.com/composition/internal/types" @@ -86,16 +85,16 @@ func TestHTTPComposition(t *testing.T) { } } } -func TestFullFormAndMappingSemantics(t *testing.T) { +func TestFullFormAndCopySemantics(t *testing.T) { name := "employee" money := amount.Money("2.00") yes := true req := types.EnterpriseRequest{Name: &name, Salary: &money, Enabled: &yes, Labels: []string{"one"}} - first, e := mapping.FromEnterprise(req) + first, e := convert.Employee(req) if e != nil { t.Fatal(e) } - second, e := mapping.FromEnterprise(types.EnterpriseRequest{}) + second, e := convert.Employee(types.EnterpriseRequest{}) if e != nil { t.Fatal(e) } @@ -108,26 +107,26 @@ func TestFullFormAndMappingSemantics(t *testing.T) { t.Fatal("unexpected implicit deep copy") } req.Employer = &money - first, e = mapping.FromEnterprise(req) + first, e = convert.Employee(req) if e != nil || first.Employer != req.Employer { t.Fatal("nullable pointer changed", e) } bad := "reject" - failed, e := mapping.FromEnterprise(types.EnterpriseRequest{Name: &bad}) + failed, e := convert.Employee(types.EnterpriseRequest{Name: &bad}) if !errors.Is(e, convert.ErrName) || failed.EmployeeName != "" || !strings.Contains(e.Error(), "EmployeeName") { t.Fatal("converter cause/zero result lost", e) } - empty, e := mapping.ToBatch(domain.Batch{}) + empty, e := convert.ToBatch(domain.Batch{}) if e != nil || empty.Rows != nil { t.Fatal("nil slice changed", e) } src := domain.Batch{Rows: []domain.Row{}, Fixed: [2]domain.Row{{Value: "a"}, {Value: "b"}}, Optional: &name} - out, e := mapping.ToBatch(src) + out, e := convert.ToBatch(src) if e != nil || out.Rows == nil || len(out.Rows) != 0 || out.Fixed[1].Value != "b" || out.Optional != src.Optional { t.Fatal(out, e) } src.Rows = []domain.Row{{Value: "one"}} - out, e = mapping.ToBatch(src) + out, e = convert.ToBatch(src) if e != nil || out.Rows[0].Value != "one" { t.Fatal(out, e) } @@ -140,7 +139,7 @@ func TestFullFormAndMappingSemantics(t *testing.T) { wg.Add(1) go func() { defer wg.Done() - result, err := mapping.FromEnterprise(types.EnterpriseRequest{Name: &name, Salary: &money}) + result, err := convert.Employee(types.EnterpriseRequest{Name: &name, Salary: &money}) if err != nil || result.EmployeeName != name || result.Salary != money { t.Error(result, err) } diff --git a/cmd/go-omni/apigen/testdata/composition/converters.go b/cmd/go-omni/apigen/testdata/composition/converters.go index c26a9ca..f3c520f 100644 --- a/cmd/go-omni/apigen/testdata/composition/converters.go +++ b/cmd/go-omni/apigen/testdata/composition/converters.go @@ -2,7 +2,6 @@ package convert import ( "errors" - "example.com/composition/internal/amount" "strconv" ) @@ -19,11 +18,4 @@ func Name(value *string) (string, error) { } return *value, nil } -func Money(value *amount.Money) amount.Money { - if value == nil { - return "" - } - return *value -} -func Bool(value *bool) bool { return value != nil && *value } func ID(value int64) string { return strconv.FormatInt(value, 10) } diff --git a/cmd/go-omni/apigen/testdata/composition/copy.go b/cmd/go-omni/apigen/testdata/composition/copy.go new file mode 100644 index 0000000..1bd8c8c --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/copy.go @@ -0,0 +1,29 @@ +package convert + +import ( + "example.com/composition/internal/domain" + "example.com/composition/internal/types" + "git.yuntuo.hk/k2safe/go-omni/copyx" +) + +// The three endpoints reuse one ordinary Go helper and their existing scalar. +// There is no field list or package/type name JSON to keep in sync. +func Employee[S any](source S) (domain.Employee, error) { + var target domain.Employee + err := copyx.Copy(&target, source, + copyx.WithFieldNames[S, domain.Employee](map[string]string{"Name": "EmployeeName"}), + copyx.WithNilPolicy(copyx.NilZero), + copyx.WithConverter(Name), + ) + return target, err +} +func Response(source domain.Result) (types.EmployeeResponse, error) { + var target types.EmployeeResponse + err := copyx.Copy(&target, source, copyx.WithConverter(func(v int64) (string, error) { return ID(v), nil })) + return target, err +} +func ToBatch(source domain.Batch) (domain.BatchDTO, error) { + var target domain.BatchDTO + err := copyx.Copy(&target, source) + return target, err +} diff --git a/cmd/go-omni/apigen/testdata/composition/domain.go b/cmd/go-omni/apigen/testdata/composition/domain.go index 25c891b..6400079 100644 --- a/cmd/go-omni/apigen/testdata/composition/domain.go +++ b/cmd/go-omni/apigen/testdata/composition/domain.go @@ -14,7 +14,7 @@ type Result struct { ID int64 } -// Distinct container item types exercise explicitly selected element mappings. +// Distinct container item types exercise explicit Go collection copying. type Row struct{ Value string } type RowDTO struct{ Value string } type Batch struct { diff --git a/cmd/go-omni/apigen/testdata/nullable/consumer_test.go b/cmd/go-omni/apigen/testdata/nullable/consumer_test.go deleted file mode 100644 index 1a552cf..0000000 --- a/cmd/go-omni/apigen/testdata/nullable/consumer_test.go +++ /dev/null @@ -1,170 +0,0 @@ -package main - -import ( - "context" - "encoding/json" - "errors" - "io" - "net/http/httptest" - "strings" - "sync" - "testing" - - "example.com/nullable/internal/convert" - "example.com/nullable/internal/domain" - "example.com/nullable/internal/handler" - "example.com/nullable/internal/mapping" - "example.com/nullable/internal/svc" - "example.com/nullable/internal/types" - "git.yuntuo.hk/k2safe/go-omni/logx" -) - -func TestNullableMappings(t *testing.T) { - for _, src := range []types.Parent{{}, {Children: []*types.Child{}}, {Child: &types.Child{}, Children: []*types.Child{nil, {Name: "two"}}}} { - out, err := mapping.ToParent(src) - if err != nil || (src.Child == nil) != (out.Child == nil) || (src.Children == nil) != (out.Children == nil) || len(src.Children) != len(out.Children) { - t.Fatal(src, out, err) - } - if src.Child != nil && out.Child.Name != src.Child.Name { - t.Fatal("zero child changed") - } - if len(out.Children) > 0 && (out.Children[0] != nil || out.Children[1].Name != "two") { - t.Fatal(out) - } - } - src := types.Parent{Child: &types.Child{Name: "one"}, Children: []*types.Child{{Name: "two"}}} - out, err := mapping.ToParentPtr(src) - if err != nil || out == nil || out.Child.Name != "one" { - t.Fatal(out, err) - } - out.Child.Name = "changed" - out.Children[0].Name = "changed" - if src.Child.Name != "one" || src.Children[0].Name != "two" { - t.Fatal("nested mapped values alias input") - } - plain := domain.Parent{Child: &domain.Child{Name: "shared"}} - copy, err := mapping.ShallowCopy(plain) - if err != nil || copy.Child != plain.Child { - t.Fatal("ordinary assignment semantics changed", err) - } - zero, err := mapping.RequiredZero(types.Parent{}) - if err != nil || zero.Child.Name != "" { - t.Fatal(zero, err) - } - for _, fn := range []func(types.Parent) (domain.Required, error){mapping.RequiredZero, mapping.RequiredError} { - out, err := fn(types.Parent{Child: &types.Child{Name: "required"}}) - if err != nil || out.Child.Name != "required" { - t.Fatal(out, err) - } - } - _, err = mapping.RequiredError(types.Parent{}) - if err == nil || !strings.Contains(err.Error(), "RequiredError.Child") { - t.Fatal(err) - } - _, err = mapping.RequirePointer(types.Parent{}) - if err == nil || !strings.Contains(err.Error(), "RequirePointer.Child") { - t.Fatal(err) - } - lift, err := mapping.Lift(domain.ValueHolder{Child: domain.Child{Name: "lift"}}) - if err != nil || lift.Child == nil || lift.Child.Name != "lift" { - t.Fatal(lift, err) - } - batch, err := mapping.Batch(domain.InputBatch{Fixed: [2]*domain.Child{nil, {Name: "array"}}}) - if err != nil || batch.Fixed[0] != nil || batch.Fixed[1].Name != "array" { - t.Fatal(batch, err) - } - _, err = mapping.Batch(domain.InputBatch{Fixed: [2]*domain.Child{nil, {Name: "bad"}}}) - if !errors.Is(err, convert.ErrValue) || !strings.Contains(err.Error(), "Batch.Fixed[1]") { - t.Fatal(err) - } - for _, input := range []types.Parent{{Child: &types.Child{Name: "bad"}}, {Children: []*types.Child{nil, {Name: "bad"}}}} { - failed, err := mapping.ToParentPtr(input) - if failed != nil || !errors.Is(err, convert.ErrValue) || !strings.Contains(err.Error(), "ToChild.Name") { - t.Fatal(failed, err) - } - if len(input.Children) > 0 && !strings.Contains(err.Error(), "ToParent.Children[1]") { - t.Fatal(err) - } - value, err := mapping.ToParent(input) - if value.Child != nil || value.Children != nil || !errors.Is(err, convert.ErrValue) { - t.Fatal("partial result escaped", err) - } - } -} - -func TestExplicitNilAndConcurrentCalls(t *testing.T) { - zero, err := mapping.ScalarZero(domain.ScalarInput{}) - if err != nil || zero != (domain.ScalarOutput{}) { - t.Fatal(zero, err) - } - _, err = mapping.ScalarError(domain.ScalarInput{}) - if err == nil { - t.Fatal("nil accepted") - } - text, count, enabled := "", int64(0), false - out, err := mapping.ScalarError(domain.ScalarInput{Text: &text, Count: &count, Enabled: &enabled}) - if err != nil || out != (domain.ScalarOutput{}) { - t.Fatal("non-nil zero rejected", out, err) - } - var wg sync.WaitGroup - for i := 0; i < 32; i++ { - wg.Add(1) - go func() { - defer wg.Done() - out, err := mapping.ToParent(types.Parent{Child: &types.Child{Name: "parallel"}}) - if err != nil || out.Child == nil || out.Child.Name != "parallel" { - t.Error(out, err) - } - }() - } - wg.Wait() -} - -func TestHTTPNullableMapping(t *testing.T) { - cfg := logx.Default() - cfg.Writer = io.Discard - logger, err := logx.New(cfg) - if err != nil { - t.Fatal(err) - } - defer logger.Close() - deps, err := svc.New(context.Background(), logger) - if err != nil { - t.Fatal(err) - } - defer deps.Close() - app, err := handler.NewApp(deps) - if err != nil { - t.Fatal(err) - } - defer app.Close() - for _, body := range []string{`{}`, `{"child":null}`, `{"child":{"name":""}}`, `{"child":{"name":"one"},"children":[null,{"name":"two"}]}`, `{"children":[]}`} { - w := httptest.NewRecorder() - req := httptest.NewRequest("POST", "/", strings.NewReader(body)) - req.Header.Set("Content-Type", "application/json") - app.Engine().ServeHTTP(w, req) - if w.Code != 200 { - t.Fatal(w.Code, w.Body.String()) - } - var input, output types.Parent - if err := json.Unmarshal([]byte(body), &input); err != nil { - t.Fatal(err) - } - if err := json.Unmarshal(w.Body.Bytes(), &output); err != nil { - t.Fatal(err) - } - if (input.Child == nil) != (output.Child == nil) || (input.Children == nil) != (output.Children == nil) || len(input.Children) != len(output.Children) { - t.Fatal(body, w.Body.String()) - } - if input.Child != nil && input.Child.Name != output.Child.Name { - t.Fatal(w.Body.String()) - } - } - w := httptest.NewRecorder() - req := httptest.NewRequest("POST", "/", strings.NewReader(`{"child":{"name":"bad"}}`)) - req.Header.Set("Content-Type", "application/json") - app.Engine().ServeHTTP(w, req) - if w.Code != 500 || strings.Contains(w.Body.String(), convert.ErrValue.Error()) || strings.Contains(w.Body.String(), "ToChild") { - t.Fatal("private cause leaked", w.Code, w.Body.String()) - } -} diff --git a/cmd/go-omni/apigen/testdata/nullable/convert.go b/cmd/go-omni/apigen/testdata/nullable/convert.go deleted file mode 100644 index 6b008cc..0000000 --- a/cmd/go-omni/apigen/testdata/nullable/convert.go +++ /dev/null @@ -1,15 +0,0 @@ -package convert - -import ( - "errors" - "fmt" -) - -var ErrValue = errors.New("private conversion cause") - -func Name(value string) (string, error) { - if value == "bad" { - return "", fmt.Errorf("conversion refused: %w", ErrValue) - } - return value, nil -} diff --git a/cmd/go-omni/apigen/testdata/nullable/domain.go b/cmd/go-omni/apigen/testdata/nullable/domain.go deleted file mode 100644 index afb172e..0000000 --- a/cmd/go-omni/apigen/testdata/nullable/domain.go +++ /dev/null @@ -1,23 +0,0 @@ -package domain - -type Child struct{ Name string } -type ChildDTO struct{ Name string } -type Parent struct { - Child *Child - Children []*Child -} -type Required struct{ Child Child } -type ValueHolder struct{ Child Child } -type PointerHolder struct{ Child *ChildDTO } -type InputBatch struct{ Fixed [2]*Child } -type OutputBatch struct{ Fixed [2]*ChildDTO } -type ScalarInput struct { - Text *string - Count *int64 - Enabled *bool -} -type ScalarOutput struct { - Text string - Count int64 - Enabled bool -} diff --git a/cmd/go-omni/apigen/testdata/nullable/service.api b/cmd/go-omni/apigen/testdata/nullable/service.api deleted file mode 100644 index a41cda3..0000000 --- a/cmd/go-omni/apigen/testdata/nullable/service.api +++ /dev/null @@ -1,15 +0,0 @@ -type Child { - Name string `json:"name"` -} -type Parent { - Child *Child `json:"child,optional"` - Children []*Child `json:"children,optional"` -} -@server ( - group: probe - response: raw -) -service probe { - @handler roundTrip - post / (Parent) returns (Parent) -} diff --git a/cmd/go-omni/apigen/write.go b/cmd/go-omni/apigen/write.go index f273777..1d80e7d 100644 --- a/cmd/go-omni/apigen/write.go +++ b/cmd/go-omni/apigen/write.go @@ -117,6 +117,17 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check var changes []change // Check ALL previous outputs, even stale ones, before writing any new file. for name, hash := range old.Generated { + // Migration guard only; old mappings are never regenerated or silently + // deleted. Once the project removes them, forget the stale ownership. + if name == "internal/mapping/mappingsGen.go" { + if err = safePath(root, name); err != nil { + return result, err + } + if _, e := os.Lstat(filepath.Join(root, name)); !errors.Is(e, os.ErrNotExist) { + return result, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go and remove internal/mapping/mappingsGen.go before generating; see COPY.md") + } + continue + } if !ownedPath(name) { return result, fmt.Errorf("api: invalid owned path %q", name) } @@ -273,9 +284,6 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check } func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) } func ownedPath(name string) bool { - if name == "internal/mapping/mappingsGen.go" { - return true - } if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) { return true } diff --git a/cmd/go-omni/main_test.go b/cmd/go-omni/main_test.go index 4265713..89848b9 100644 --- a/cmd/go-omni/main_test.go +++ b/cmd/go-omni/main_test.go @@ -8,7 +8,7 @@ import ( ) func TestNewWithRemoteFrameworkVersion(t *testing.T) { - for _, flag := range [][]string{{"--framework-version", "v0.18.0-preview"}, {"--framework-version=v0.18.0-preview"}} { + for _, flag := range [][]string{{"--framework-version", "v0.19.0-preview"}, {"--framework-version=v0.19.0-preview"}} { project := filepath.Join(t.TempDir(), "billing") args := append([]string{"--module", "example.com/billing"}, flag...) if err := runNew(append(args, project)); err != nil { @@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) { if err != nil { t.Fatal(err) } - if !strings.Contains(string(content), "go-omni v0.18.0-preview") || strings.Contains(string(content), "replace ") { + if !strings.Contains(string(content), "go-omni v0.19.0-preview") || strings.Contains(string(content), "replace ") { t.Fatalf("CLI did not pin remote dependency: %s", content) } } diff --git a/cmd/go-omni/scaffold/version.go b/cmd/go-omni/scaffold/version.go index 59a5160..45b5802 100644 --- a/cmd/go-omni/scaffold/version.go +++ b/cmd/go-omni/scaffold/version.go @@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" // Source builds can carry an unpublished VCS version (including +dirty). // Keep this fallback in sync with the next release; installed CLIs use their // downloaded module version, which also has a module checksum. -const defaultFrameworkVersion = "v0.18.0-preview" +const defaultFrameworkVersion = "v0.19.0-preview" // This target is not a claim that the release already exists. Source users // select a local framework checkout until the module templates are published. diff --git a/copyx/benchmark_test.go b/copyx/benchmark_test.go new file mode 100644 index 0000000..f82e5f7 --- /dev/null +++ b/copyx/benchmark_test.go @@ -0,0 +1,117 @@ +package copyx_test + +import ( + "errors" + "fmt" + "testing" + + "git.yuntuo.hk/k2safe/go-omni/copyx" +) + +func BenchmarkScenarios(b *testing.B) { + type request struct { + Name string + Count int64 + Active bool + } + type result struct { + Label string + Count int64 + Active bool + } + source := request{"sample", 9007199254740993, true} + option := copyx.WithFieldNames[request, result](map[string]string{"Name": "Label"}) + copier, err := copyx.New(option) + if err != nil { + b.Fatal(err) + } + for _, reuse := range []bool{false, true} { + b.Run(fmt.Sprintf("struct/reuse%t", reuse), func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + var target result + var err error + if reuse { + err = copier.Copy(&target, source) + } else { + err = copyx.Copy(&target, source, option) + } + if err != nil { + b.Fatal(err) + } + } + }) + } + deep, _ := copyx.New(copyx.WithDeepCopy()) + b.Run("bytes64KiB", func(b *testing.B) { + src := make([]byte, 64<<10) + b.SetBytes(int64(len(src))) + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + var dst []byte + if err := deep.Copy(&dst, src); err != nil { + b.Fatal(err) + } + } + }) + src := map[string][]int{} + for i := 0; i < 100; i++ { + src[fmt.Sprint(i)] = []int{1, 2, 3} + } + b.Run("map100", func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + var dst map[string][]int + if err := deep.Copy(&dst, src); err != nil { + b.Fatal(err) + } + } + }) + cycle := &detail{Values: []int{1, 2, 3}} + cycle.Next = cycle + b.Run("cycle", func(b *testing.B) { + b.ReportAllocs() + for i := 0; i < b.N; i++ { + var dst *detail + if err := deep.Copy(&dst, cycle); err != nil { + b.Fatal(err) + } + } + }) + b.Run("parallelCycle", func(b *testing.B) { + b.ReportAllocs() + b.RunParallel(func(pb *testing.PB) { + for pb.Next() { + var dst *detail + if err := deep.Copy(&dst, cycle); err != nil { + b.Error(err) + return + } + } + }) + }) + for _, fresh := range []bool{false, true} { + b.Run(fmt.Sprintf("converter/fresh%t", fresh), func(b *testing.B) { + c, err := copyx.New(copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) ([]int, error) { + if fresh { + return append([]int(nil), v...), nil + } + return v, nil + })) + if err != nil { + b.Fatal(err) + } + src := []int{1, 2, 3} + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + var dst []int + err := c.Copy(&dst, src) + if fresh && err != nil || !fresh && !errors.Is(err, copyx.ErrSharedReference) { + b.Fatal(err) + } + } + }) + } +} diff --git a/copyx/copy.go b/copyx/copy.go new file mode 100644 index 0000000..97417d3 --- /dev/null +++ b/copyx/copy.go @@ -0,0 +1,537 @@ +package copyx + +import ( + "errors" + "fmt" + "reflect" + "runtime" + "sort" + "sync" +) + +var ( + ErrType = errors.New("copyx: incompatible types") + ErrNil = errors.New("copyx: nil cannot fill a non-nullable target") + ErrField = errors.New("copyx: unmatched field") + ErrUnsupported = errors.New("copyx: unsupported value; provide an explicit data converter") + ErrSharedReference = errors.New("copyx: deep copy retains a mutable source reference") + ErrLimit = errors.New("copyx: traversal limit exceeded") +) + +// Error describes a field/index path without recording the source value. +type Error struct { + Path string + Cause error +} + +func (e *Error) Error() string { return e.Path + ": " + e.Cause.Error() } +func (e *Error) Unwrap() error { return e.Cause } +func at(path string, err error) error { return &Error{Path: path, Cause: err} } + +func Copy(target, source any, options ...Option) error { + c, err := New(options...) + if err != nil { + return err + } + return c.Copy(target, source) +} + +// Clone always makes a deep copy. Nil and empty collections stay distinct. +func Clone[T any](source T, options ...Option) (T, error) { + var target T + options = append(append([]Option(nil), options...), WithDeepCopy()) + err := Copy(&target, source, options...) + return target, err +} + +// Copy stages the complete result and only assigns target on success. User +// converter side effects (including mutation of source) cannot be rolled back. +// Caller must synchronize concurrent access to source and target themselves. +func (c *Copier) Copy(target, source any) error { + // Alias indexes contain addresses only. Keep the original graph alive until + // validation and commit finish so the GC cannot recycle any indexed address. + defer func() { runtime.KeepAlive(source) }() + if c == nil { + return errors.New("copyx: nil Copier") + } + dst := reflect.ValueOf(target) + if !dst.IsValid() || dst.Kind() != reflect.Pointer || dst.IsNil() { + return errors.New("copyx: target must be a non-nil pointer") + } + src := reflect.ValueOf(source) + s := state{options: &c.options, seen: map[visit]reflect.Value{}} + if c.options.deep { + var err error + s.graph, err = sourceGraph(src) + if err != nil { + return err + } + if src.IsValid() && src.Kind() == reflect.Pointer && !src.IsNil() && dst.Elem().Kind() == reflect.Struct { + id := identity(src) + s.valueRoot = &id + } + } + out, err := s.walk(src, dst.Elem(), dst.Elem().Type(), "$", 0) + if err != nil { + return err + } + if s.graph != nil { + if err = s.graph.audit(out); err != nil { + return err + } + } + dst.Elem().Set(out) + return nil +} + +type visit struct { + reference + target reflect.Type +} +type state struct { + options *settings + seen map[visit]reflect.Value + graph *graph + nodes int + allocated uint64 + valueRoot *reference +} + +func (s *state) reserve(count int, size uintptr, path string) error { + if count < 0 || (count > 0 && uint64(size) > maxCopyBytes/uint64(count)) { + return at(path, ErrLimit) + } + s.allocated += uint64(count) * uint64(size) + if s.allocated > maxCopyBytes { + return at(path, ErrLimit) + } + return nil +} + +func nullable(k reflect.Kind) bool { + switch k { + case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan: + return true + } + return false +} +func baseValue(base reflect.Value, t reflect.Type) reflect.Value { + if base.IsValid() && base.Type() == t { + return base + } + return reflect.Zero(t) +} +func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) { + s.nodes++ + if depth > maxDepth || s.nodes > maxNodes { + return reflect.Value{}, at(path, ErrLimit) + } + base = baseValue(base, t) + if !shallowSafe(t) { + return reflect.Value{}, at(path, ErrUnsupported) + } + if src.IsValid() && !src.CanInterface() { + return reflect.Value{}, at(path, ErrUnsupported) + } + // Exact converters also receive typed nils and zeros. No method calls, + // Scanner/Valuer invocation or implicit numeric conversion is attempted. + if src.IsValid() { + if convert, ok := s.options.converters[pair{src.Type(), t}]; ok { + out, err := convert(src) + if err != nil { + return reflect.Value{}, at(path, err) + } + if s.graph != nil { + if err = s.graph.audit(out); err != nil { + return reflect.Value{}, at(path, err) + } + } + return out, nil + } + } + if !src.IsValid() || (nullable(src.Kind()) && src.IsNil()) { + if src.IsValid() && !s.nilCompatible(src.Type(), t, map[pair]bool{}) { + return reflect.Value{}, at(path, ErrType) + } + if s.options.nilPolicy == NilSkip { + return base, nil + } + if nullable(t.Kind()) || s.options.nilPolicy == NilZero { + return reflect.Zero(t), nil + } + return reflect.Value{}, at(path, ErrNil) + } + if s.options.skipZero && src.IsZero() { + if !s.nilCompatible(src.Type(), t, map[pair]bool{}) { + return reflect.Value{}, at(path, ErrType) + } + return base, nil + } + if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && !s.options.skipZero && s.options.nilPolicy != NilSkip { + if !s.options.deep || simpleValue(t) { + return src, nil + } + } + if src.Kind() == reflect.Interface { + return s.walk(src.Elem(), base, t, path, depth+1) + } + if t.Kind() == reflect.Interface { + if !src.Type().Implements(t) { + return reflect.Value{}, at(path, ErrType) + } + out, err := s.walk(src, reflect.Value{}, src.Type(), path, depth+1) + if err != nil { + return reflect.Value{}, err + } + v := reflect.New(t).Elem() + v.Set(out) + return v, nil + } + if s.options.deep && forbidden(t) { + return reflect.Value{}, at(path, ErrUnsupported) + } + if src.Kind() == reflect.Pointer && t.Kind() != reflect.Pointer { + return s.walk(src.Elem(), base, t, path, depth+1) + } + if t.Kind() == reflect.Pointer { + if !s.options.deep && src.Type() == t { + return src, nil + } + if src.Kind() == reflect.Pointer { + key := visit{identity(src), t} + if s.valueRoot != nil && key.reference == *s.valueRoot { + return reflect.Value{}, at(path, fmt.Errorf("%w: cyclic root must retain its pointer; use Clone(source) or Copy(&targetPointer, source)", ErrUnsupported)) + } + if prior, ok := s.seen[key]; ok { + return prior, nil + } + } + if err := s.reserve(1, t.Elem().Size(), path); err != nil { + return reflect.Value{}, err + } + out := reflect.New(t.Elem()) + if out.Type() != t { + out = out.Convert(t) + } + if src.Kind() == reflect.Pointer { + key := visit{identity(src), t} + s.seen[key] = out + src = src.Elem() + } + var childBase reflect.Value + if !base.IsNil() { + childBase = base.Elem() + } + value, err := s.walk(src, childBase, t.Elem(), path, depth+1) + if err != nil { + return reflect.Value{}, err + } + out.Elem().Set(value) + return out, nil + } + if src.Kind() != t.Kind() { + return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t)) + } + if t == timeType && src.Type() == t { + return src, nil + } + switch t.Kind() { + case reflect.Struct: + return s.structure(src, base, t, path, depth) + case reflect.Slice: + if !s.options.deep && src.Type() == t { + return src, nil + } + key := visit{identity(src), t} + if prior, ok := s.seen[key]; ok { + return prior, nil + } + if err := s.reserve(src.Len(), t.Elem().Size(), path); err != nil { + return reflect.Value{}, err + } + out := reflect.MakeSlice(t, src.Len(), src.Len()) + s.seen[key] = out + if src.Type() == t && len(s.options.converters) == 0 && len(s.options.fields) == 0 && simpleValue(t.Elem()) { + reflect.Copy(out, src) + return out, nil + } + for i := 0; i < src.Len(); i++ { + v, err := s.walk(src.Index(i), reflect.Value{}, t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1) + if err != nil { + return reflect.Value{}, err + } + out.Index(i).Set(v) + } + return out, nil + case reflect.Array: + if src.Len() != t.Len() { + return reflect.Value{}, at(path, ErrType) + } + if err := s.reserve(1, t.Size(), path); err != nil { + return reflect.Value{}, err + } + out := reflect.New(t).Elem() + for i := 0; i < src.Len(); i++ { + v, err := s.walk(src.Index(i), base.Index(i), t.Elem(), fmt.Sprintf("%s[%d]", path, i), depth+1) + if err != nil { + return reflect.Value{}, err + } + out.Index(i).Set(v) + } + return out, nil + case reflect.Map: + if !s.options.deep && src.Type() == t { + return src, nil + } + key := visit{identity(src), t} + if prior, ok := s.seen[key]; ok { + return prior, nil + } + if err := s.reserve(src.Len(), t.Key().Size()+t.Elem().Size()+32, path); err != nil { + return reflect.Value{}, err + } + out := reflect.MakeMapWithSize(t, src.Len()) + s.seen[key] = out + it := src.MapRange() + for it.Next() { + k, err := s.walk(it.Key(), reflect.Value{}, t.Key(), path+"[key]", depth+1) + if err != nil { + return reflect.Value{}, err + } + if !k.Comparable() { + return reflect.Value{}, at(path+"[key]", ErrType) + } + if out.MapIndex(k).IsValid() { + return reflect.Value{}, at(path+"[key]", errors.New("copyx: key conversion collision")) + } + v, err := s.walk(it.Value(), reflect.Value{}, t.Elem(), path+"[value]", depth+1) + if err != nil { + return reflect.Value{}, err + } + out.SetMapIndex(k, v) + } + return out, nil + case reflect.Func, reflect.Chan, reflect.UnsafePointer: + if s.options.deep { + return reflect.Value{}, at(path, ErrUnsupported) + } + } + if src.Type() != t { + return reflect.Value{}, at(path, fmt.Errorf("%w: %s -> %s", ErrType, src.Type(), t)) + } + return src, nil +} + +var fieldTypes sync.Map + +func exportedFields(t reflect.Type) map[string]reflect.StructField { + if known, ok := fieldTypes.Load(t); ok { + return known.(map[string]reflect.StructField) + } + out := map[string]reflect.StructField{} + for _, f := range reflect.VisibleFields(t) { + if f.PkgPath != "" { + continue + } + kind := f.Type.Kind() + if kind == reflect.Pointer { + kind = f.Type.Elem().Kind() + } + if f.Anonymous && kind == reflect.Struct { + continue + } + out[f.Name] = f + } + actual, _ := fieldTypes.LoadOrStore(t, out) + return actual.(map[string]reflect.StructField) +} + +func (s *state) nilCompatible(from, to reflect.Type, seen map[pair]bool) bool { + key := pair{from, to} + if seen[key] { + return true + } + seen[key] = true + if from == to { + return true + } + if _, ok := s.options.converters[key]; ok { + return true + } + if to.Kind() == reflect.Interface { + return from.Implements(to) + } + if from.Kind() == reflect.Pointer { + return s.nilCompatible(from.Elem(), to, seen) + } + if to.Kind() == reflect.Pointer { + return s.nilCompatible(from, to.Elem(), seen) + } + if from.Kind() != to.Kind() { + return false + } + switch from.Kind() { + case reflect.Struct: + // Nil must not hide a structurally incompatible type pair. There are no + // field values to visit, but coverage and conversion types still apply. + for i := 0; i < to.NumField(); i++ { + if to.Field(i).PkgPath != "" { + return false + } + } + sources, targets := exportedFields(from), exportedFields(to) + rules := s.options.fields[key] + if rules == nil { + rules = &fields{} + } + reverse, used := map[string]string{}, map[string]bool{} + for a, b := range rules.names { + reverse[b] = a + } + for name, target := range targets { + if rules.ignoreTarget[name] { + continue + } + srcName := name + if renamed, ok := reverse[name]; ok { + srcName = renamed + } + source, ok := sources[srcName] + if !ok || rules.ignoreSource[srcName] || !s.nilCompatible(source.Type, target.Type, seen) { + return false + } + used[srcName] = true + } + if s.options.strict { + for name := range sources { + if !used[name] && !rules.ignoreSource[name] { + return false + } + } + } + return true + case reflect.Slice: + return s.nilCompatible(from.Elem(), to.Elem(), seen) + case reflect.Map: + return s.nilCompatible(from.Key(), to.Key(), seen) && s.nilCompatible(from.Elem(), to.Elem(), seen) + case reflect.Array: + return from.Len() == to.Len() && s.nilCompatible(from.Elem(), to.Elem(), seen) + } + return false +} +func fieldValue(v reflect.Value, index []int) (reflect.Value, bool) { + for _, i := range index { + if v.Kind() == reflect.Pointer { + if v.IsNil() { + return reflect.Value{}, false + } + v = v.Elem() + } + v = v.Field(i) + } + return v, true +} +func setField(v reflect.Value, index []int, value reflect.Value) error { + for j, i := range index { + field := v.Field(i) + if !field.CanSet() { + return ErrUnsupported + } + if j == len(index)-1 { + field.Set(value) + return nil + } + if field.Kind() == reflect.Pointer { + // Detach embedded destination pointers before editing promoted fields. + fresh := reflect.New(field.Type().Elem()) + if !field.IsNil() { + fresh.Elem().Set(field.Elem()) + } + field.Set(fresh) + v = fresh.Elem() + } else { + v = field + } + } + return nil +} +func (s *state) structure(src, base reflect.Value, t reflect.Type, path string, depth int) (reflect.Value, error) { + if err := s.reserve(1, t.Size(), path); err != nil { + return reflect.Value{}, err + } + out := reflect.New(t).Elem() + out.Set(base) + if src.Type() == t { + // Private scalar state may be assigned as part of its whole value; mutable + // private state requires a converter, never unsafe writes or silent loss. + for i := 0; i < t.NumField(); i++ { + f := t.Field(i) + if f.PkgPath != "" && s.options.deep && privateMutable(f.Type) { + return reflect.Value{}, at(path+"."+f.Name, ErrUnsupported) + } + } + out.Set(src) + } else { + for i := 0; i < t.NumField(); i++ { + if t.Field(i).PkgPath != "" { + return reflect.Value{}, at(path, ErrUnsupported) + } + } + } + from, to := exportedFields(src.Type()), exportedFields(t) + rules := s.options.fields[pair{src.Type(), t}] + if rules == nil { + rules = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}} + } + reverse := map[string]string{} + for a, b := range rules.names { + reverse[b] = a + } + names := make([]string, 0, len(to)) + for n := range to { + names = append(names, n) + } + sort.Strings(names) + used := map[string]bool{} + for _, name := range names { + target := to[name] + old, _ := fieldValue(base, target.Index) + if rules.ignoreTarget[name] { + if !old.IsValid() { + old = reflect.Zero(target.Type) + } + if err := setField(out, target.Index, old); err != nil { + return reflect.Value{}, at(path+"."+name, err) + } + continue + } + srcName := name + if renamed, ok := reverse[name]; ok { + srcName = renamed + } + source, ok := from[srcName] + if !ok || rules.ignoreSource[srcName] { + return reflect.Value{}, at(path+"."+name, ErrField) + } + used[srcName] = true + v, ok := fieldValue(src, source.Index) + if !ok { + return reflect.Value{}, at(path+"."+name, ErrNil) + } + value, err := s.walk(v, old, target.Type, path+"."+name, depth+1) + if err != nil { + return reflect.Value{}, err + } + if err = setField(out, target.Index, value); err != nil { + return reflect.Value{}, at(path+"."+name, err) + } + } + if s.options.strict { + for name := range from { + if !used[name] && !rules.ignoreSource[name] { + return reflect.Value{}, at(path+"."+name, ErrField) + } + } + } + return out, nil +} diff --git a/copyx/copy_test.go b/copyx/copy_test.go new file mode 100644 index 0000000..8ba9904 --- /dev/null +++ b/copyx/copy_test.go @@ -0,0 +1,448 @@ +package copyx_test + +import ( + "errors" + "fmt" + "reflect" + "strings" + "sync" + "testing" + "time" + + "git.yuntuo.hk/k2safe/go-omni/copyx" +) + +type detail struct { + Values []int + Index map[string][]int + Next *detail +} + +func TestDeepGraph(t *testing.T) { + src := &detail{Values: []int{1, 2}, Index: map[string][]int{"a": {3}}} + src.Next = src + dst, err := copyx.Clone(src) + if err != nil { + t.Fatal(err) + } + if dst == src || dst.Next != dst { + t.Fatal("pointer cycle not preserved") + } + dst.Values[0] = 9 + dst.Index["a"][0] = 8 + if src.Values[0] != 1 || src.Index["a"][0] != 3 { + t.Fatal("deep alias") + } + m := map[string]any{} + m["self"] = m + n, err := copyx.Clone(m) + if err != nil { + t.Fatal(err) + } + n["self"].(map[string]any)["new"] = 1 + if n["new"] != 1 || m["new"] != nil { + t.Fatal("map cycle changed") + } + x := []any{nil} + x[0] = x + y, err := copyx.Clone(x) + if err != nil { + t.Fatal(err) + } + y[0].([]any)[0] = "changed" + if y[0] != "changed" { + t.Fatal("slice cycle changed") + } + if _, ok := x[0].([]any); !ok { + t.Fatal("source slice changed") + } +} +func TestShallowAndNil(t *testing.T) { + src := detail{Values: []int{1}} + var dst detail + if err := copyx.Copy(&dst, src); err != nil { + t.Fatal(err) + } + dst.Values[0] = 2 + if src.Values[0] != 2 { + t.Fatal("shallow unexpectedly detached") + } + for _, slice := range [][]int{nil, {}} { + out, err := copyx.Clone(slice) + if err != nil || (out == nil) != (slice == nil) { + t.Fatal(out, err) + } + } + for _, m := range []map[string]int{nil, {}} { + out, err := copyx.Clone(m) + if err != nil || (out == nil) != (m == nil) { + t.Fatal(out, err) + } + } + var p *int + n := 7 + if err := copyx.Copy(&n, p); !errors.Is(err, copyx.ErrNil) || n != 7 { + t.Fatal(n, err) + } + if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilSkip)); err != nil || n != 7 { + t.Fatal(n, err) + } + if err := copyx.Copy(&n, p, copyx.WithNilPolicy(copyx.NilZero)); err != nil || n != 0 { + t.Fatal(n, err) + } + x := 3 + out := &x + if err := copyx.Copy(&out, p); err != nil || out != nil { + t.Fatal(out, err) + } + flag := true + if err := copyx.Copy(&flag, false); err != nil || flag { + t.Fatal(flag, err) + } + flag = true + if err := copyx.Copy(&flag, false, copyx.WithSkipZero()); err != nil || !flag { + t.Fatal(flag, err) + } +} + +func TestConverterAliasesRejected(t *testing.T) { + for _, kind := range []string{"same", "nested", "subslice", "capacity", "fieldPointer", "private", "map", "sibling"} { + t.Run(kind, func(t *testing.T) { + src := &detail{Values: []int{1, 2, 3}, Index: map[string][]int{"x": {1}}} + var err error + switch kind { + case "same": + _, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return v, nil })) + case "nested": + _, err = copyx.Clone(src, copyx.WithConverter(func(v *detail) (*detail, error) { return &detail{Values: v.Values}, nil })) + case "subslice": + _, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[1:], nil })) + case "capacity": + _, err = copyx.Clone(src.Values, copyx.WithConverter(func(v []int) ([]int, error) { return v[:0], nil })) + case "map": + _, err = copyx.Clone(src.Index, copyx.WithConverter(func(v map[string][]int) (map[string][]int, error) { return v, nil })) + case "fieldPointer": + var dst *int + err = copyx.Copy(&dst, src.Values, copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) (*int, error) { return &v[1], nil })) + case "private": + type opaque struct{ data []int } + v := opaque{src.Values} + _, err = copyx.Clone(v, copyx.WithConverter(func(v opaque) (opaque, error) { return v, nil })) + case "sibling": + type object struct{ A, B []int } + v := object{[]int{1}, []int{2}} + _, err = copyx.Clone(v, copyx.WithConverter(func(_ []int) ([]int, error) { return v.B, nil })) + } + if !errors.Is(err, copyx.ErrSharedReference) { + t.Fatalf("got %v", err) + } + }) + } + _, err := copyx.Clone(4, copyx.WithConverter(func(v int) (int, error) { return v, nil })) + if err != nil { + t.Fatal("equal primitive rejected", err) + } + _, err = copyx.Clone("same", copyx.WithConverter(func(v string) (string, error) { return v, nil })) + if err != nil { + t.Fatal(err) + } + good, err := copyx.Clone([]int{1}, copyx.WithConverter(func(v []int) ([]int, error) { return append([]int(nil), v...), nil })) + if err != nil || good[0] != 1 { + t.Fatal(good, err) + } +} + +func TestPrivateAndUnsupported(t *testing.T) { + type money struct{ cents *int64 } + n := int64(9007199254740991) + src := money{&n} + if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatal(err) + } + out, err := copyx.Clone(src, copyx.WithConverter(func(v money) (money, error) { n := *v.cents; return money{&n}, nil })) + if err != nil || *out.cents != n || out.cents == src.cents { + t.Fatal(out, err) + } + tm := time.Now() + cloned, err := copyx.Clone(tm) + if err != nil || !cloned.Equal(tm) { + t.Fatal(cloned, err) + } + for _, v := range []any{make(chan int), func() {}, &sync.Mutex{}, sync.Mutex{}} { + if _, err := copyx.Clone(v); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatalf("%T: %v", v, err) + } + } +} + +func TestNilTypeMismatchAndInterfaceConverter(t *testing.T) { + var source *string + target := 7 + if err := copyx.Copy(&target, source, copyx.WithNilPolicy(copyx.NilZero)); !errors.Is(err, copyx.ErrType) || target != 7 { + t.Fatal(target, err) + } + var list []int + var targetMap map[string]int + if err := copyx.Copy(&targetMap, list); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + type input struct{ Value any } + type output struct{ Value string } + var out output + err := copyx.Copy(&out, input{}, copyx.WithConverter(func(v any) (string, error) { + if v == nil { + return "nil handled", nil + } + return "value", nil + })) + if err != nil || out.Value != "nil handled" { + t.Fatal(out, err) + } + var lock sync.Mutex + if err := copyx.Copy(&lock, sync.Mutex{}); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatal(err) + } + type locked struct { + mu sync.Mutex + Value int + } + var l locked + if err := copyx.Copy(&l, locked{}); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatal(err) + } +} + +func TestAllocationLimitAndPointerRoot(t *testing.T) { + src := make([]byte, (64<<20)+1) + if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) { + t.Fatal(err) + } + n := &detail{} + n.Next = n + var value detail + if err := copyx.Copy(&value, n, copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatal(err) + } + var p *detail + if err := copyx.Copy(&p, n, copyx.WithDeepCopy()); err != nil || p.Next != p || p == n { + t.Fatal(p, err) + } +} +func TestFieldsConvertersAndAtomicity(t *testing.T) { + type input struct { + Name string + Age int + Extra string + } + type output struct { + Label string + Age int + Local string + } + rename := map[string]string{"Name": "Label"} + c, err := copyx.New(copyx.WithFieldNames[input, output](rename), copyx.WithIgnoreTarget[input, output]("Local")) + if err != nil { + t.Fatal(err) + } + rename["Name"] = "Missing" + dst := output{Local: "preserved"} + if err = c.Copy(&dst, input{Name: "hello", Age: 2}); err != nil || dst.Label != "hello" || dst.Local != "preserved" { + t.Fatal(dst, err) + } + if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local")); !errors.Is(err, copyx.ErrField) { + t.Fatal(err) + } + if err = copyx.Copy(&dst, input{}, copyx.WithStrictFields(), copyx.WithFieldNames[input, output](map[string]string{"Name": "Label"}), copyx.WithIgnoreTarget[input, output]("Local"), copyx.WithIgnoreSource[input, output]("Extra")); err != nil { + t.Fatal(err) + } + if err = copyx.Copy(&dst, input{}); !errors.Is(err, copyx.ErrField) { + t.Fatal(err) + } + if _, err = copyx.New(copyx.WithFieldNames[input, output](map[string]string{"Nope": "Age"})); err == nil { + t.Fatal("invalid rule accepted") + } + if _, err = copyx.New(copyx.WithConverter(func(v int) (string, error) { return "", nil }), copyx.WithConverter(func(v int) (string, error) { return "", nil })); err == nil { + t.Fatal("duplicate converter accepted") + } + + sentinel := errors.New("conversion failed") + type node struct { + A []int + Z string + } + old := node{[]int{8}, "old"} + before := old.A + err = copyx.Copy(&old, node{[]int{1}, "bad"}, copyx.WithDeepCopy(), copyx.WithConverter(func(v string) (string, error) { return "", sentinel })) + if !errors.Is(err, sentinel) || !strings.Contains(err.Error(), "Z") || old.Z != "old" || old.A[0] != 8 || &old.A[0] != &before[0] { + t.Fatal(old, err) + } + // Exact typed nil converter must get control before the global nil policy. + var ptr *int + var value int + err = copyx.Copy(&value, ptr, copyx.WithConverter(func(v *int) (int, error) { + if v == nil { + return 0, sentinel + } + return *v, nil + })) + if !errors.Is(err, sentinel) { + t.Fatal(err) + } + var small int8 = 3 + if err = copyx.Copy(&small, int64(300)); !errors.Is(err, copyx.ErrType) || small != 3 { + t.Fatal(small, err) + } +} + +type Embedded struct { + A int + Z string +} + +func TestEmbeddedDestinationDetached(t *testing.T) { + type target struct{ *Embedded } + type source struct { + A int + Z int + } + ptr := &Embedded{5, "keep"} + dst := target{ptr} + if err := copyx.Copy(&dst, source{6, 7}); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + if ptr.A != 5 || dst.Embedded != ptr { + t.Fatal("failure mutated embedded target") + } + if err := copyx.Copy(&dst, Embedded{6, "new"}); err != nil { + t.Fatal(err) + } + if ptr.A != 5 || dst.A != 6 { + t.Fatal("success mutated old embedded object") + } +} +func TestConcurrentCopier(t *testing.T) { + c, err := copyx.New(copyx.WithDeepCopy()) + if err != nil { + t.Fatal(err) + } + src := &detail{Values: []int{7}, Index: map[string][]int{"a": {9}}} + var wg sync.WaitGroup + for i := 0; i < 64; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for n := 0; n < 20; n++ { + var dst *detail + if err := c.Copy(&dst, src); err != nil { + t.Error(err) + return + } + dst.Values[0] = n + if src.Values[0] != 7 { + t.Error("shared") + } + } + }() + } + wg.Wait() +} +func TestBoundAndCollections(t *testing.T) { + src := &detail{} + p := src + for i := 0; i < 300; i++ { + p.Next = &detail{} + p = p.Next + } + if _, err := copyx.Clone(src); !errors.Is(err, copyx.ErrLimit) { + t.Fatal(err) + } + type from struct{ Value string } + type to struct{ Value string } + var out []to + if err := copyx.Copy(&out, []from{{"a"}, {"b"}}); err != nil || len(out) != 2 || out[1].Value != "b" { + t.Fatal(out, err) + } + var array [2]to + if err := copyx.Copy(&array, [2]from{{"a"}, {"b"}}); err != nil { + t.Fatal(err) + } + if err := copyx.Copy(&array, [1]from{{"a"}}); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + var outMap map[string]int + err := copyx.Copy(&outMap, map[int]int{1: 1, 2: 2}, copyx.WithConverter(func(int) (string, error) { return "same", nil })) + if err == nil || outMap != nil { + t.Fatal(outMap, err) + } +} +func FuzzClone(f *testing.F) { + f.Add([]byte{1, 2, 3}) + f.Add([]byte{}) + f.Fuzz(func(t *testing.T, data []byte) { + if len(data) > 1024 { + data = data[:1024] + } + src := map[string]any{"bytes": data, "other": []string{"x"}} + out, err := copyx.Clone(src) + if err != nil { + t.Fatal(err) + } + if !reflect.DeepEqual(src, out) { + t.Fatal("changed value") + } + if len(data) > 0 { + out["bytes"].([]byte)[0] ^= 255 + if data[0] == out["bytes"].([]byte)[0] { + t.Fatal("alias") + } + } + }) +} + +func TestTraversalNodeLimit(t *testing.T) { + source := make([]any, 200001) + if _, err := copyx.Clone(source); !errors.Is(err, copyx.ErrLimit) { + t.Fatal("expected bounded traversal", err) + } +} + +func TestNilCannotHideFieldMismatch(t *testing.T) { + type source struct{ Value string } + type target struct{ Value int } + var src *source + var dst *target + for _, policy := range []copyx.NilPolicy{copyx.NilError, copyx.NilZero, copyx.NilSkip} { + if err := copyx.Copy(&dst, src, copyx.WithNilPolicy(policy)); !errors.Is(err, copyx.ErrType) { + t.Fatal("nil concealed incompatible fields", err) + } + } + if err := copyx.Copy(&dst, src, copyx.WithConverter(func(string) (int, error) { return 0, nil })); err != nil { + t.Fatal(err) + } + text := "keep" + if err := copyx.Copy(&text, 0, copyx.WithSkipZero()); !errors.Is(err, copyx.ErrType) || text != "keep" { + t.Fatal("zero concealed incompatible types", text, err) + } +} +func BenchmarkCopy(b *testing.B) { + for _, n := range []int{1, 100, 1000} { + for _, deep := range []bool{false, true} { + b.Run(fmt.Sprintf("fields%d/deep%t", n, deep), func(b *testing.B) { + src := detail{Values: make([]int, n), Index: map[string][]int{"one": {1, 2}}} + var opts []copyx.Option + if deep { + opts = append(opts, copyx.WithDeepCopy()) + } + c, _ := copyx.New(opts...) + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + var dst detail + if err := c.Copy(&dst, src); err != nil { + b.Fatal(err) + } + } + }) + } + } +} diff --git a/copyx/graph.go b/copyx/graph.go new file mode 100644 index 0000000..a1f33e4 --- /dev/null +++ b/copyx/graph.go @@ -0,0 +1,270 @@ +package copyx + +import ( + "fmt" + "reflect" + "sort" + "sync" + "time" +) + +const maxDepth = 256 +const maxNodes = 200000 +const maxCopyBytes = 64 << 20 + +var timeType = reflect.TypeFor[time.Time]() + +// Immutable type metadata is shared, never values or converter registrations. +var valueTypes sync.Map + +func simpleValue(t reflect.Type) bool { + if known, ok := valueTypes.Load(t); ok { + return known.(bool) + } + result := t == timeType + if !result { + switch t.Kind() { + case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, + reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Uintptr, + reflect.Float32, reflect.Float64, reflect.Complex64, reflect.Complex128, reflect.String: + result = true + case reflect.Array: + result = simpleValue(t.Elem()) + case reflect.Struct: + result = !forbidden(t) + for i := 0; result && i < t.NumField(); i++ { + result = simpleValue(t.Field(i).Type) + } + } + } + valueTypes.Store(t, result) + return result +} + +type reference struct { + kind reflect.Kind + typ reflect.Type + ptr uintptr + length, capacity int +} + +func identity(v reflect.Value) reference { + r := reference{kind: v.Kind(), typ: v.Type(), ptr: uintptr(v.UnsafePointer())} + if v.Kind() == reflect.Slice { + r.length = v.Len() + r.capacity = v.Cap() + } + return r +} + +type span struct{ start, end uintptr } +type graph struct { + maps map[uintptr]bool + spans []span + maxEnd []uintptr +} + +// References are inspected without reading private values through Interface or +// using unsafe writes. Slice capacity matters: returning source[:0] is an alias. +func scanGraph(v reflect.Value, visit func(reflect.Value, string) error) error { + seen := map[reference]bool{} + nodes := 0 + var walk func(reflect.Value, string, int) error + walk = func(v reflect.Value, path string, depth int) error { + nodes++ + if depth > maxDepth || nodes > maxNodes { + return at(path, ErrLimit) + } + if !v.IsValid() { + return nil + } + if err := visit(v, path); err != nil { + return err + } + if simpleValue(v.Type()) { + return nil + } // time.Time is a documented immutable value. + switch v.Kind() { + case reflect.Interface: + if !v.IsNil() { + return walk(v.Elem(), path, depth+1) + } + case reflect.Pointer, reflect.Map, reflect.Slice: + if v.IsNil() { + return nil + } + id := identity(v) + if seen[id] { + return nil + } + seen[id] = true + switch v.Kind() { + case reflect.Pointer: + return walk(v.Elem(), path, depth+1) + case reflect.Map: + it := v.MapRange() + for it.Next() { + if err := walk(it.Key(), path+"[key]", depth+1); err != nil { + return err + } + if err := walk(it.Value(), path+"[value]", depth+1); err != nil { + return err + } + } + case reflect.Slice: + if simpleValue(v.Type().Elem()) { + return nil + } + all := v.Slice(0, v.Cap()) + for i := 0; i < all.Len(); i++ { + if err := walk(all.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil { + return err + } + } + } + case reflect.Struct: + for i := 0; i < v.NumField(); i++ { + if err := walk(v.Field(i), path+"."+v.Type().Field(i).Name, depth+1); err != nil { + return err + } + } + case reflect.Array: + for i := 0; i < v.Len(); i++ { + if err := walk(v.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1); err != nil { + return err + } + } + } + return nil + } + return walk(v, "$", 0) +} +func mutableSpan(v reflect.Value) (span, bool) { + var n uintptr + switch v.Kind() { + case reflect.Pointer: + if v.IsNil() { + return span{}, false + } + n = v.Type().Elem().Size() + case reflect.Slice: + if v.IsNil() { + return span{}, false + } + n = uintptr(v.Cap()) * v.Type().Elem().Size() + default: + return span{}, false + } + if n == 0 { + return span{}, false + } + start := uintptr(v.UnsafePointer()) + return span{start, start + n}, true +} +func sourceGraph(v reflect.Value) (*graph, error) { + g := &graph{maps: map[uintptr]bool{}} + err := scanGraph(v, func(v reflect.Value, _ string) error { + if r, ok := mutableSpan(v); ok { + g.spans = append(g.spans, r) + } + if v.Kind() == reflect.Map && !v.IsNil() { + g.maps[uintptr(v.UnsafePointer())] = true + } + return nil + }) + sort.Slice(g.spans, func(i, j int) bool { return g.spans[i].start < g.spans[j].start }) + var end uintptr + for _, r := range g.spans { + if r.end > end { + end = r.end + } + g.maxEnd = append(g.maxEnd, end) + } + return g, err +} +func (g *graph) audit(v reflect.Value) error { + return scanGraph(v, func(v reflect.Value, path string) error { + if forbidden(v.Type()) { + return at(path, ErrUnsupported) + } + switch v.Kind() { + case reflect.Func, reflect.Chan: + if !v.IsNil() { + return at(path, ErrUnsupported) + } + case reflect.UnsafePointer: + if v.Pointer() != 0 { + return at(path, ErrUnsupported) + } + } + if v.Kind() == reflect.Map && !v.IsNil() && g.maps[uintptr(v.UnsafePointer())] { + return at(path, ErrSharedReference) + } + if r, ok := mutableSpan(v); ok { + i := sort.Search(len(g.spans), func(i int) bool { return g.spans[i].start >= r.end }) + if i > 0 && g.maxEnd[i-1] > r.start { + return at(path, ErrSharedReference) + } + } + return nil + }) +} +func forbidden(t reflect.Type) bool { + // Lock/resource implementations are not data. A custom converter may turn + // them into a different data type but may not manufacture cloned handles. + switch t.PkgPath() { + case "sync", "sync/atomic", "os", "net", "database/sql": + return t.Kind() == reflect.Struct + } + return false +} + +// Shallow copies may share handles through pointers, but must not copy lock or +// resource state by value, including state inside a containing struct/array. +var safeValueTypes sync.Map + +func shallowSafe(t reflect.Type) bool { + if known, ok := safeValueTypes.Load(t); ok { + return known.(bool) + } + result := true + if forbidden(t) { + safeValueTypes.Store(t, false) + return false + } + switch t.Kind() { + case reflect.Struct: + for i := 0; i < t.NumField(); i++ { + if !shallowSafe(t.Field(i).Type) { + result = false + break + } + } + case reflect.Array: + result = shallowSafe(t.Elem()) + } + safeValueTypes.Store(t, result) + return result +} + +func privateMutable(t reflect.Type) bool { + if t == timeType { + return false + } + switch t.Kind() { + case reflect.Pointer, reflect.Map, reflect.Slice, reflect.Interface, reflect.Func, reflect.Chan, reflect.UnsafePointer: + return true + case reflect.Array: + return privateMutable(t.Elem()) + case reflect.Struct: + if forbidden(t) { + return true + } + for i := 0; i < t.NumField(); i++ { + if privateMutable(t.Field(i).Type) { + return true + } + } + } + return false +} diff --git a/copyx/options.go b/copyx/options.go new file mode 100644 index 0000000..f80745b --- /dev/null +++ b/copyx/options.go @@ -0,0 +1,169 @@ +// Package copyx copies Go values without JSON, generated code or global registration. +package copyx + +import ( + "errors" + "fmt" + "reflect" +) + +// NilPolicy controls nil input. NilError preserves nil for nullable targets and +// rejects nil to non-nullable targets. NilSkip retains the existing target. +type NilPolicy uint8 + +const ( + NilError NilPolicy = iota + NilZero + NilSkip +) + +type pair struct{ from, to reflect.Type } +type converter func(reflect.Value) (reflect.Value, error) +type fields struct { + names map[string]string + ignoreSource, ignoreTarget map[string]bool +} +type settings struct { + deep, skipZero, strict bool + nilPolicy NilPolicy + converters map[pair]converter + fields map[pair]*fields +} + +// Option configures one immutable Copier. Options never change global behavior. +type Option func(*settings) error + +func WithDeepCopy() Option { return func(s *settings) error { s.deep = true; return nil } } + +// WithSkipZero explicitly skips Go zero values, including false, 0 and "". +func WithSkipZero() Option { return func(s *settings) error { s.skipZero = true; return nil } } + +// WithStrictFields additionally rejects unconsumed exported source fields. +// Target fields must have a source by default, unless explicitly ignored. +func WithStrictFields() Option { return func(s *settings) error { s.strict = true; return nil } } +func WithNilPolicy(p NilPolicy) Option { + return func(s *settings) error { + if p > NilSkip { + return errors.New("copyx: invalid nil policy") + } + s.nilPolicy = p + return nil + } +} + +// WithConverter registers an exact source/target type pair. Typed nils are +// delivered to converters before default nil handling. The function must not +// mutate its input and must be safe for concurrent calls when Copier is shared. +func WithConverter[S, T any](fn func(S) (T, error)) Option { + return func(s *settings) error { + if fn == nil { + return errors.New("copyx: nil converter") + } + key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()} + if _, exists := s.converters[key]; exists { + return fmt.Errorf("copyx: duplicate converter %s -> %s", key.from, key.to) + } + s.converters[key] = func(v reflect.Value) (reflect.Value, error) { + var input S + reflect.ValueOf(&input).Elem().Set(v) + out, err := fn(input) + return reflect.ValueOf(&out).Elem(), err + } + return nil + } +} + +// WithFieldNames declares source -> target Go field names for one struct pair. +// It is not a JSON tag mapping. Only actual, unambiguous exported fields qualify. +func WithFieldNames[S, T any](names map[string]string) Option { + copyNames := make(map[string]string, len(names)) + for k, v := range names { + copyNames[k] = v + } + return fieldOption[S, T](func(f *fields) error { + for from, to := range copyNames { + if _, exists := f.names[from]; exists { + return fmt.Errorf("copyx: duplicate field rule %s", from) + } + f.names[from] = to + } + return nil + }) +} +func WithIgnoreSource[S, T any](names ...string) Option { + names = append([]string(nil), names...) + return fieldOption[S, T](func(f *fields) error { + for _, n := range names { + f.ignoreSource[n] = true + } + return nil + }) +} + +// WithIgnoreTarget preserves the corresponding existing target fields in Copy. +// Deep mode rejects that preservation if it would retain a source reference. +func WithIgnoreTarget[S, T any](names ...string) Option { + names = append([]string(nil), names...) + return fieldOption[S, T](func(f *fields) error { + for _, n := range names { + f.ignoreTarget[n] = true + } + return nil + }) +} +func fieldOption[S, T any](apply func(*fields) error) Option { + return func(s *settings) error { + key := pair{reflect.TypeFor[S](), reflect.TypeFor[T]()} + if key.from.Kind() != reflect.Struct || key.to.Kind() != reflect.Struct { + return errors.New("copyx: field options require struct types") + } + f := s.fields[key] + if f == nil { + f = &fields{map[string]string{}, map[string]bool{}, map[string]bool{}} + s.fields[key] = f + } + return apply(f) + } +} + +// Copier is immutable after construction. Each Copy owns its traversal state. +type Copier struct{ options settings } + +func New(options ...Option) (*Copier, error) { + s := settings{converters: map[pair]converter{}, fields: map[pair]*fields{}} + for _, option := range options { + if option == nil { + return nil, errors.New("copyx: nil option") + } + if err := option(&s); err != nil { + return nil, err + } + } + for key, f := range s.fields { + from, to := exportedFields(key.from), exportedFields(key.to) + used := map[string]bool{} + for src, dst := range f.names { + if _, ok := from[src]; !ok { + return nil, fmt.Errorf("copyx: unknown source field %s.%s", key.from, src) + } + if _, ok := to[dst]; !ok { + return nil, fmt.Errorf("copyx: unknown target field %s.%s", key.to, dst) + } + if used[dst] || f.ignoreSource[src] || f.ignoreTarget[dst] { + return nil, fmt.Errorf("copyx: conflicting field rule %s", dst) + } + used[dst] = true + } + for n := range f.ignoreSource { + if _, ok := from[n]; !ok { + return nil, fmt.Errorf("copyx: unknown source field %s", n) + } + } + for n := range f.ignoreTarget { + if _, ok := to[n]; !ok { + return nil, fmt.Errorf("copyx: unknown target field %s", n) + } + } + } + return &Copier{options: s}, nil +} diff --git a/examples/copying/README.md b/examples/copying/README.md new file mode 100644 index 0000000..5a2367f --- /dev/null +++ b/examples/copying/README.md @@ -0,0 +1,23 @@ +# 结构复制示例 + +在框架仓库根目录运行 `go run ./examples/copying`;不需要数据库、网络服务或 JSON 映射配置。 + +示例包含:可复用构造器、Go 字段改名、时间自定义转换、保留 false、深拷贝切片、同类型 Clone,以及拒绝自定义转换器返回输入的原切片。 + +预期输出: + +```text +name=sample enabled=false time=2026-01-01T00:00:00Z sourceTag=original +clone detached: true +shared converter rejected: true +``` + +测试:`go test -race ./examples/copying`。完整策略、限制与旧生成映射迁移见 [COPY.md](../../COPY.md)。 + +独立工资三端的实际生成、HTTP 与金额用例见 [集成测试](../../cmd/go-omni/apigen/copyConsumer_test.go) 和 [公共契约](../../cmd/go-omni/apigen/testdata/composition/service.api)。框架根目录运行: + +```sh +GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1 +``` + +该测试在临时独立项目中执行,三个入口共用一个普通 Go copyx helper。公共字段新增时,只修改公共契约和独立业务结构,三个入口的转换规则不增加。 diff --git a/examples/copying/main.go b/examples/copying/main.go new file mode 100644 index 0000000..d652d15 --- /dev/null +++ b/examples/copying/main.go @@ -0,0 +1,61 @@ +// copying demonstrates optional Go-to-Go conversion and checked deep copying. +package main + +import ( + "errors" + "fmt" + "log" + "time" + + "git.yuntuo.hk/k2safe/go-omni/copyx" +) + +type Input struct { + Name string + Enabled bool + Tags []string + CreatedAt time.Time +} +type View struct { + DisplayName string + Enabled bool + Tags []string + CreatedAt string +} + +func run() error { + converter, err := copyx.New( + copyx.WithDeepCopy(), + copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}), + copyx.WithConverter(func(v time.Time) (string, error) { return v.UTC().Format(time.RFC3339), nil }), + ) + if err != nil { + return err + } + source := Input{Name: "sample", Enabled: false, Tags: []string{"original"}, CreatedAt: time.Date(2026, 1, 1, 0, 0, 0, 0, time.UTC)} + var target View + if err = converter.Copy(&target, source); err != nil { + return err + } + target.Tags[0] = "changed" + fmt.Printf("name=%s enabled=%t time=%s sourceTag=%s\n", target.DisplayName, target.Enabled, target.CreatedAt, source.Tags[0]) + + cloned, err := copyx.Clone(source) + if err != nil { + return err + } + cloned.Tags[0] = "separate" + fmt.Println("clone detached:", source.Tags[0] == "original") + + _, err = copyx.Clone(source.Tags, copyx.WithConverter(func(v []string) ([]string, error) { return v, nil })) + fmt.Println("shared converter rejected:", errors.Is(err, copyx.ErrSharedReference)) + if !errors.Is(err, copyx.ErrSharedReference) { + return errors.New("expected alias rejection") + } + return nil +} +func main() { + if err := run(); err != nil { + log.Fatal(err) + } +} diff --git a/examples/copying/main_test.go b/examples/copying/main_test.go new file mode 100644 index 0000000..b0e283d --- /dev/null +++ b/examples/copying/main_test.go @@ -0,0 +1,9 @@ +package main + +import "testing" + +func TestExample(t *testing.T) { + if err := run(); err != nil { + t.Fatal(err) + } +} diff --git a/examples/managedService/.goOmni/apiManifest.json b/examples/managedService/.goOmni/apiManifest.json index 92217cc..9812898 100644 --- a/examples/managedService/.goOmni/apiManifest.json +++ b/examples/managedService/.goOmni/apiManifest.json @@ -1,6 +1,6 @@ { "generatorConfig": true, - "templateVersion": "v0.18.0-preview", + "templateVersion": "v0.19.0-preview", "profile": { "database": "none", "storage": "none", diff --git a/examples/managedService/go.mod b/examples/managedService/go.mod index 00a39f6..edc3717 100644 --- a/examples/managedService/go.mod +++ b/examples/managedService/go.mod @@ -2,7 +2,7 @@ module example.com/managed go 1.22.0 -require git.yuntuo.hk/k2safe/go-omni v0.18.0-preview +require git.yuntuo.hk/k2safe/go-omni v0.19.0-preview require ( github.com/gabriel-vasile/mimetype v1.4.3 // indirect diff --git a/reports/copyxAcceptance.md b/reports/copyxAcceptance.md new file mode 100644 index 0000000..52febfb --- /dev/null +++ b/reports/copyxAcceptance.md @@ -0,0 +1,70 @@ +# v0.19 copyx 与映射移除验收 + +日期:2026-09-25。环境:Apple M2 Max、darwin/arm64、Go 1.27.1,基准默认 GOMAXPROCS=12。最低支持版本另用 Go 1.22.12 验证 copyx 与可运行示例。 + +## 范围与行为 + +本轮新增可选 Go 工具包并删除原映射配置/生成机制,没有修改接入项目代码、数据库或线上服务。没有新增第三方依赖,没有内置工资、金额或权限规则;工资只是生成器的独立通用验收 fixture。 + +| 场景 | 验证内容 | +| --- | --- | +| 浅/深复制 | 值、结构、指针、数组、map、slice、interface;浅共享与深隔离分别验证 | +| 空值与精度 | nil 与空集合、默认保留 false/0、显式 nil/skip 策略;不兼容类型不能借 nil/零值绕过检查 | +| 可变引用 | 原指针/map、子切片、零长度但有容量的视图、源元素指针、私有子引用、其他字段的引用均拒绝 | +| 自定义函数 | 字段改名、精确类型对、金额私有状态复制、nil interface;重复/冲突配置拒绝 | +| 失败语义 | errors.Is 保留 cause、目标不提交部分结果、嵌入目标指针不提前修改 | +| 对象图 | 指针/map/slice 循环,重复引用;循环指针根压成值结构拒绝 | +| 并发与资源 | 同一 Copier 64 goroutine 各复制 20 次,race;深度/节点/估算分配超限拒绝;锁/资源/函数等明确限制 | +| 真正生成的消费者 | 三端工资公共字段,Money scalar,大数精度/舍入、非法金额、缺失/null/零值、完整表单、错误码与错误链 | +| 维护成本 | 公共字段新增:改一处契约和一处独立业务结构;三个接口共用 helper,不增加拷贝规则 | +| 生成与生命周期 | 重复 generate/check/dry-run 保留业务文件;工厂异常、初始化/监听失败及资源关闭由既有回归验证 | +| 移除与 workspace | 旧配置/旧生成文件阻断且不写入,明确迁移后所有权清理;workspace 失败不部分更新服务 | + +FuzzClone 使用动态 JSON 容器、随机 bytes 和嵌套 slice,验证值一致与引用隔离。输入超过 1 KiB 时截取继续验证,运行 30 秒、4 workers:**4,237,917 次执行,未发现失败**。这不等于所有形状都已 fuzz;循环、类型转换及资源拒绝另外用定向用例覆盖。 + +全量 46 个测试包通过 race 回归(含实际生成项目),vet/build、Go 1.22.12、示例运行与生成漂移检查均通过。 + +## 可复现命令 + +```sh +GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -count=1 ./... +go vet ./... +go build ./... +GOTOOLCHAIN=go1.22.12 go test ./copyx ./examples/copying +go test ./copyx -run '^$' -fuzz FuzzClone -fuzztime 30s -parallel 4 +go test ./copyx -run '^$' -bench . -benchmem -benchtime 300ms -count 3 +go run ./examples/copying +go run ./cmd/go-omni api generate --dir examples/managedService --check +# examples/managedService 中另执行 go test -race ./... 与 go build . +``` + +全量回归包含真实临时生成项目,集成测试的子进程使用 go test -race / go build,并不只比较生成字符串。基准测量时没有同时运行本轮 fuzz 或全量测试;原始三轮结果见 [copyxBenchmark.txt](copyxBenchmark.txt)。 + +## 性能与分配 + +下表取三轮中位数。构造器复用与一次性调用都包含真实复制;`fields1000` 指整数 slice 的长度,不是 1,000 个结构字段。所有深拷贝保留引用检查。 + +| 场景 | 中位耗时 | B/op | allocs/op | +| --- | ---: | ---: | ---: | +| 相同结构浅拷贝(含引用字段) | 85.8 ns | 96 | 2 | +| 3 字段跨结构改名,复用 Copier | 697.5 ns | 168 | 7 | +| 相同跨结构,每次构建 Copier | 1,181 ns | 1,120 | 17 | +| 嵌套结构 + 1 个整数 + map,深拷贝 | 2,758 ns | 1,368 | 42 | +| 嵌套结构 + 100 个整数 + map,深拷贝 | 2,787 ns | 2,256 | 42 | +| 嵌套结构 + 1,000 个整数 + map,深拷贝 | 3,806 ns | 9,552 | 42 | +| 64 KiB byte slice 深拷贝 | 5,780 ns | 65,768 | 9 | +| 100 项 map,每项含整数 slice | 88,859 ns | 87,704 | 1,554 | +| 自引用循环对象深拷贝 | 2,128 ns | 640 | 26 | +| converter 新建 slice 并通过检查 | 539.7 ns | 280 | 10 | +| converter 返回原 slice 被拒绝 | 383.8 ns | 320 | 11 | + +并行循环对象基准中位 515.2 ns/op 是多核聚合吞吐均摊,**不是单请求延迟**。复杂 map 的多次图扫描、字段路径和反射分配明显更贵;不据此宣称运行时工具快于手写赋值。热点复用 Copier;大对象和高频映射应按实际业务结构衡量是否改用直接 Go。 + +优化只包括不可变类型元数据缓存、纯值快速路径、纯值 slice 批量复制;没有去掉深拷贝别名检查。内置估算分配上限 64 MiB 不是硬堆内存隔离,converter 的外部分配及副作用不受该预算控制。 + +## 边界 + +- 新方式在运行时做类型与覆盖检查,不再有旧映射生成时的静态保证;字段变更要运行实际类型对测试。 +- converter 是受信项目代码,必须只读输入;不能自动回滚其副作用、证明其未共享外部全局对象,或强杀无限循环。 +- 当前记录是本机库/生成器验收,不是线上服务吞吐、Linux soak、真实业务数据库回归或 OSS/R2 云账号验收。本轮没有对线上服务器施压。 +- 旧消费者仍需按 [COPY.md](../COPY.md#从旧-mapping-迁移) 迁移,不会被这次框架发布自动修改。 diff --git a/reports/copyxBenchmark.txt b/reports/copyxBenchmark.txt new file mode 100644 index 0000000..8c11549 --- /dev/null +++ b/reports/copyxBenchmark.txt @@ -0,0 +1,48 @@ +goos: darwin +goarch: arm64 +pkg: git.yuntuo.hk/k2safe/go-omni/copyx +cpu: Apple M2 Max +BenchmarkScenarios/struct/reusefalse-12 251924 1209 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusefalse-12 316450 1181 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusefalse-12 314466 1140 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusetrue-12 517544 703.0 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/struct/reusetrue-12 517842 691.9 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/struct/reusetrue-12 522170 697.5 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/bytes64KiB-12 67933 5857 ns/op 11189.69 MB/s 65768 B/op 9 allocs/op +BenchmarkScenarios/bytes64KiB-12 61680 5780 ns/op 11337.56 MB/s 65768 B/op 9 allocs/op +BenchmarkScenarios/bytes64KiB-12 61843 5771 ns/op 11355.66 MB/s 65768 B/op 9 allocs/op +BenchmarkScenarios/map100-12 3808 88598 ns/op 87704 B/op 1554 allocs/op +BenchmarkScenarios/map100-12 4136 88859 ns/op 87704 B/op 1554 allocs/op +BenchmarkScenarios/map100-12 4051 89614 ns/op 87704 B/op 1554 allocs/op +BenchmarkScenarios/cycle-12 168308 2128 ns/op 640 B/op 26 allocs/op +BenchmarkScenarios/cycle-12 169057 2163 ns/op 640 B/op 26 allocs/op +BenchmarkScenarios/cycle-12 169892 2120 ns/op 640 B/op 26 allocs/op +BenchmarkScenarios/parallelCycle-12 963274 660.6 ns/op 640 B/op 26 allocs/op +BenchmarkScenarios/parallelCycle-12 582906 515.1 ns/op 640 B/op 26 allocs/op +BenchmarkScenarios/parallelCycle-12 748388 515.2 ns/op 640 B/op 26 allocs/op +BenchmarkScenarios/converter/freshfalse-12 828752 382.8 ns/op 320 B/op 11 allocs/op +BenchmarkScenarios/converter/freshfalse-12 897363 383.8 ns/op 320 B/op 11 allocs/op +BenchmarkScenarios/converter/freshfalse-12 1000000 387.5 ns/op 320 B/op 11 allocs/op +BenchmarkScenarios/converter/freshtrue-12 665454 541.2 ns/op 280 B/op 10 allocs/op +BenchmarkScenarios/converter/freshtrue-12 647089 535.0 ns/op 280 B/op 10 allocs/op +BenchmarkScenarios/converter/freshtrue-12 704719 539.7 ns/op 280 B/op 10 allocs/op +BenchmarkCopy/fields1/deepfalse-12 4058166 87.96 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields1/deepfalse-12 4137807 85.79 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields1/deepfalse-12 4201804 85.07 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields1/deeptrue-12 131413 2739 ns/op 1368 B/op 42 allocs/op +BenchmarkCopy/fields1/deeptrue-12 131658 2771 ns/op 1368 B/op 42 allocs/op +BenchmarkCopy/fields1/deeptrue-12 131880 2758 ns/op 1368 B/op 42 allocs/op +BenchmarkCopy/fields100/deepfalse-12 4209133 85.61 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields100/deepfalse-12 4239586 84.30 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields100/deepfalse-12 4248972 84.59 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields100/deeptrue-12 127617 2787 ns/op 2256 B/op 42 allocs/op +BenchmarkCopy/fields100/deeptrue-12 129816 2826 ns/op 2256 B/op 42 allocs/op +BenchmarkCopy/fields100/deeptrue-12 124041 2780 ns/op 2256 B/op 42 allocs/op +BenchmarkCopy/fields1000/deepfalse-12 4214600 85.59 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields1000/deepfalse-12 4250719 85.20 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields1000/deepfalse-12 4247498 85.05 ns/op 96 B/op 2 allocs/op +BenchmarkCopy/fields1000/deeptrue-12 92329 3818 ns/op 9552 B/op 42 allocs/op +BenchmarkCopy/fields1000/deeptrue-12 93945 3806 ns/op 9552 B/op 42 allocs/op +BenchmarkCopy/fields1000/deeptrue-12 96010 3782 ns/op 9552 B/op 42 allocs/op +PASS +ok git.yuntuo.hk/k2safe/go-omni/copyx 17.513s diff --git a/reports/copyxFuzz.txt b/reports/copyxFuzz.txt new file mode 100644 index 0000000..865abcd --- /dev/null +++ b/reports/copyxFuzz.txt @@ -0,0 +1,15 @@ +fuzz: elapsed: 0s, gathering baseline coverage: 0/10 completed +fuzz: elapsed: 0s, gathering baseline coverage: 10/10 completed, now fuzzing with 4 workers +fuzz: elapsed: 3s, execs: 426851 (142240/sec), new interesting: 5 (total: 15) +fuzz: elapsed: 6s, execs: 837791 (137019/sec), new interesting: 5 (total: 15) +fuzz: elapsed: 9s, execs: 1281087 (147767/sec), new interesting: 5 (total: 15) +fuzz: elapsed: 12s, execs: 1705531 (141463/sec), new interesting: 5 (total: 15) +fuzz: elapsed: 15s, execs: 2125995 (140169/sec), new interesting: 6 (total: 16) +fuzz: elapsed: 18s, execs: 2547285 (140432/sec), new interesting: 6 (total: 16) +fuzz: elapsed: 21s, execs: 2954940 (135854/sec), new interesting: 6 (total: 16) +fuzz: elapsed: 24s, execs: 3389762 (144941/sec), new interesting: 6 (total: 16) +fuzz: elapsed: 27s, execs: 3808273 (139491/sec), new interesting: 6 (total: 16) +fuzz: elapsed: 30s, execs: 4237917 (143257/sec), new interesting: 6 (total: 16) +fuzz: elapsed: 30s, execs: 4237917 (0/sec), new interesting: 6 (total: 16) +PASS +ok git.yuntuo.hk/k2safe/go-omni/copyx 30.363s diff --git a/reports/copyxRegression.txt b/reports/copyxRegression.txt new file mode 100644 index 0000000..917b4d7 --- /dev/null +++ b/reports/copyxRegression.txt @@ -0,0 +1,59 @@ +ok git.yuntuo.hk/k2safe/go-omni 1.519s +ok git.yuntuo.hk/k2safe/go-omni/app 3.503s +ok git.yuntuo.hk/k2safe/go-omni/audit 1.970s +ok git.yuntuo.hk/k2safe/go-omni/auth 2.265s +ok git.yuntuo.hk/k2safe/go-omni/cache 2.566s +ok git.yuntuo.hk/k2safe/go-omni/cache/redis 2.712s +ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni 5.321s +ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen 76.921s +ok git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold 47.757s +ok git.yuntuo.hk/k2safe/go-omni/config 4.374s +ok git.yuntuo.hk/k2safe/go-omni/contract 4.702s +ok git.yuntuo.hk/k2safe/go-omni/copyx 5.171s +ok git.yuntuo.hk/k2safe/go-omni/database 4.141s +ok git.yuntuo.hk/k2safe/go-omni/database/gormx 3.916s +ok git.yuntuo.hk/k2safe/go-omni/database/migrate 3.856s +ok git.yuntuo.hk/k2safe/go-omni/database/mysql 3.816s +ok git.yuntuo.hk/k2safe/go-omni/errors 3.930s +? git.yuntuo.hk/k2safe/go-omni/examples/basic [no test files] +? git.yuntuo.hk/k2safe/go-omni/examples/compatibility [no test files] +? git.yuntuo.hk/k2safe/go-omni/examples/compatibility/ids [no test files] +ok git.yuntuo.hk/k2safe/go-omni/examples/copying 3.198s +ok git.yuntuo.hk/k2safe/go-omni/examples/modules 2.930s +ok git.yuntuo.hk/k2safe/go-omni/examples/platform 2.766s +ok git.yuntuo.hk/k2safe/go-omni/examples/reference 2.794s +? git.yuntuo.hk/k2safe/go-omni/examples/reference/handler [no test files] +? git.yuntuo.hk/k2safe/go-omni/examples/reference/model [no test files] +ok git.yuntuo.hk/k2safe/go-omni/examples/reference/repository 2.880s +? git.yuntuo.hk/k2safe/go-omni/examples/reference/service [no test files] +ok git.yuntuo.hk/k2safe/go-omni/health 3.091s +ok git.yuntuo.hk/k2safe/go-omni/httpc 3.345s +ok git.yuntuo.hk/k2safe/go-omni/httpx 3.870s +ok git.yuntuo.hk/k2safe/go-omni/internal/perfcheck 3.284s +? git.yuntuo.hk/k2safe/go-omni/internal/storagefixture [no test files] +? git.yuntuo.hk/k2safe/go-omni/internal/testutil [no test files] +ok git.yuntuo.hk/k2safe/go-omni/lifecycle 3.390s +ok git.yuntuo.hk/k2safe/go-omni/logx 3.459s +ok git.yuntuo.hk/k2safe/go-omni/metrics 2.732s +ok git.yuntuo.hk/k2safe/go-omni/middleware 2.718s +ok git.yuntuo.hk/k2safe/go-omni/middleware/redis 3.086s +ok git.yuntuo.hk/k2safe/go-omni/openapi 3.319s +ok git.yuntuo.hk/k2safe/go-omni/pagination 3.356s +? git.yuntuo.hk/k2safe/go-omni/queue [no test files] +ok git.yuntuo.hk/k2safe/go-omni/queue/redis 3.060s +ok git.yuntuo.hk/k2safe/go-omni/resilience 2.898s +ok git.yuntuo.hk/k2safe/go-omni/response 3.157s +ok git.yuntuo.hk/k2safe/go-omni/storage 3.414s +ok git.yuntuo.hk/k2safe/go-omni/storage/local 3.617s +ok git.yuntuo.hk/k2safe/go-omni/storage/oss 3.572s +ok git.yuntuo.hk/k2safe/go-omni/storage/s3 3.752s +? git.yuntuo.hk/k2safe/go-omni/storage/storagetest [no test files] +ok git.yuntuo.hk/k2safe/go-omni/task 3.094s +ok git.yuntuo.hk/k2safe/go-omni/telemetry 3.089s +ok git.yuntuo.hk/k2safe/go-omni/transport 3.311s +ok git.yuntuo.hk/k2safe/go-omni/transport/http2 14.938s +? git.yuntuo.hk/k2safe/go-omni/transport/internal/routing [no test files] +ok git.yuntuo.hk/k2safe/go-omni/transport/native 4.422s +ok git.yuntuo.hk/k2safe/go-omni/transport/nethttp 3.401s +? git.yuntuo.hk/k2safe/go-omni/transport/transporttest [no test files] +? git.yuntuo.hk/k2safe/go-omni/validate [no test files]