diff --git a/API.md b/API.md index 776d461..d372cda 100644 --- a/API.md +++ b/API.md @@ -109,7 +109,7 @@ service user-api { - 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`（支持开区间和单边界）。 - `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数；范围边界必须有限且位于 ±2^53 内。 -暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、未通过 scalar 显式接入的自定义编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明，以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。 +暂不支持匿名嵌入、别名、递归请求类型、未声明的外部 Go 类型、未通过 scalar 显式接入的自定义编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明，以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。 业务错误、具名鉴权、多行文档及下述开发命令从 CLI `v0.7.0-preview.1` 提供。公共响应模板从 `v0.8.0-preview` 提供，CLI 与业务项目中的框架依赖需一起升级。 @@ -331,3 +331,5 @@ python3 scripts/verifyManagedProject.py Go 测试覆盖契约拒绝规则、多源绑定、零值/缺失、默认/范围/嵌套集合、文档约束、幂等更新、只读检查、业务保留、删除路由、路径边界及生成项目编译。Python 验收会在临时目录生成项目，分别启动 nethttp/native，验证真实 HTTP 请求、中间件、日志 requestId 和优雅退出；结束后清理自己创建的进程和临时文件。结果记录在 `performance/managedProjectLocal.json`；完整记录见 [第一版验收](performance/managedProjectAcceptance.md)。 内置健康与 OpenAPI 路由可通过配置禁用或修改路径，`.api` 不再固定保留 `/health/live`、`/health/ready`、`/openapi.json`。启用的内置路径与业务 GET 路由重叠时，生成的 NewApp 在监听前返回配置错误；禁用后可以由 `.api` 自己维护这些路径。见 [内置接口配置](PROJECT.md#配置内置接口)。 + +响应 DTO 自引用与相互引用从 v0.28.2 起支持，保留强类型并生成 OpenAPI `$ref`；见[递归响应说明](DEVELOPMENT.md#递归响应-dto)。直接值循环与字段组合循环仍然非法。 diff --git a/CHANGELOG.md b/CHANGELOG.md index f707188..d87c576 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,8 @@ +# v0.28.2-preview + +- 修复响应 DTO 的自引用/相互引用生成，支持 slice、pointer、map；OpenAPI 使用命名组件与循环引用，保留约束、描述、示例及 scalar 输出 schema。 +- 拒绝 Go 非法值循环与递归请求；保留格式化、生成、契约变更和只读检查能力。新增通用树示例与真实生成消费者回归。 + # Changelog ## v0.27.0-preview（2026-09-29） diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index ee0cd9e..db1f750 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,4 +1,4 @@ -# 通用业务开发指南（v0.27.0-preview） +# 通用业务开发指南（v0.28.2-preview） 从 [README](README.md) 创建服务后，业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力；字段、错误码、权限、表结构和事务边界都由项目定义。 @@ -35,6 +35,30 @@ delete /item 基础类型数组还支持 int/int8/int16/int32、uint/uint16/uint32/uint64、float32/float64。nil 根数组都输出 `[]`；`[]any` 在 OpenAPI 中是 `type: array, items: {}`，不限制元素形状，但元素必须可被标准 JSON 编码器序列化。`[]uint8`（Go 的 byte slice）会编码为 base64，故明确拒绝；数字数组可用 `[]int32`，二进制用下载接口。当前不支持根嵌套数组或根自定义 scalar 数组。 +## 递归响应 DTO + +v0.28.2 起，响应 DTO 可引用自身或相互引用；名字与字段由项目定义： + +```text +type Node { + Name string `json:"name"` + Children []Node `json:"children"` + Next *Node `json:"next,omitempty"` + Lookup map[string]Node `json:"lookup,omitempty"` +} + +@handler tree +get /tree returns (Node) +``` + +Go 类型保留 `[]Node`、`*Node`、`map[string]Node`，不降级为 `any`。生成器将递归响应图中的命名 DTO 写入 `components.schemas.DTO_<类型名>`，字段使用 `$ref`；对象、数组、可空根对象、raw 及自定义响应模板都支持。可空引用使用 `anyOf` 的引用与 null 分支，描述和示例通过 `allOf` 保留，遵循 [OpenAPI 3.0.3 引用规则](https://spec.openapis.org/oas/v3.0.3.html#reference-object)。 + +字段命名、标签、约束、重复字段与示例继续校验，自定义 scalar 使用声明的输出类型。有限 JSON 示例按实际内容检查，仍受 32 层/10000 节点的编译限制。类型引用不会无限展开；再次生成和契约变更检查会遍历引用图并保留手写代码。 + +范围是 `.api` 的响应类型。递归请求 DTO（包括间接包含递归类型的请求）仍不支持，`contract.New/Describe` 与手写 `openapi.SchemaOf` 的限制不变；它们不会自动获得递归请求绑定能力。直接值递归如 `Child Node`、仅由值字段形成的相互递归，以及字段组合循环仍会报错。业务返回值必须是可序列化的有限数据，不能构造对象指针环。 + +升级 CLI 后，在项目自己的 `.api` 中将临时 `[]any` 恢复成 `[]项目类型`，执行 `go-omni api generate`、`go-omni api generate --check` 和 `go test ./...`。修改 `.api` 后应先生成再检查；不要手改生成的 DTO。不改变项目的菜单、分类、权限或数据构造规则。完整可运行步骤见 [递归响应示例](examples/recursiveResponse/README.md)。 + ## 契约驱动的请求策略 入口文件可以声明默认策略，单个接口按需覆盖： @@ -201,7 +225,7 @@ go-omni workspace check --file omni.workspace.json ### 混合 API 与通用工程 -v0.28.1-preview 起，workspace 支持在同一份清单中管理 API、通用工程，也支持仅包含通用工程的工作区： +v0.28.2-preview 起，workspace 支持在同一份清单中管理 API、通用工程，也支持仅包含通用工程的工作区： ```json { diff --git a/PROGRAM.md b/PROGRAM.md index f9a338d..109ff20 100644 --- a/PROGRAM.md +++ b/PROGRAM.md @@ -5,7 +5,7 @@ ## 创建与运行 ```sh -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.2-preview go-omni project new demo --module example.com/demo cd demo go mod tidy @@ -78,4 +78,4 @@ go test ./... check 检查模板漂移和必要文件，不检查业务正确性，也不能替代 Go 编译与测试。新的模板版本会重新生成受管文件；不自动合并项目所有文件或更新 go.mod。未知的新清单版本会要求使用兼容 CLI。 -原有 `new` / `api new` 及 API 工程行为保留。通用工程使用自己的清单和命令，不能交给 `api generate` 或 `add` 转换；新增业务包直接写普通 Go。从 v0.28.1-preview 起，`workspace generate/check` 可以同时处理 API 与通用工程。清单可填写 `kind: api | project`，省略时按生成清单识别；format 对通用工程显示 SKIP。见 [完整混合示例](examples/mixedWorkspace/README.md)。各 module 仍独立，workspace 命令不维护 go.work 或模块依赖。 +原有 `new` / `api new` 及 API 工程行为保留。通用工程使用自己的清单和命令，不能交给 `api generate` 或 `add` 转换；新增业务包直接写普通 Go。从 v0.28.2-preview 起，`workspace generate/check` 可以同时处理 API 与通用工程。清单可填写 `kind: api | project`，省略时按生成清单识别；format 对通用工程显示 SKIP。见 [完整混合示例](examples/mixedWorkspace/README.md)。各 module 仍独立，workspace 命令不维护 go.work 或模块依赖。 diff --git a/README.md b/README.md index 0a5ac29..b6e9854 100644 --- a/README.md +++ b/README.md @@ -2,15 +2,17 @@ GoOmni 是面向多个项目复用的 Go 基础框架。通用工程使用 `project new` 生成配置、日志、依赖装配、生命周期和普通 Go 运行入口；需要 HTTP 接口时可选择 `.api` 工程生成，也可以在已有项目中按需引入独立模块。运行方式、业务类型和外部依赖由项目决定。 +**v0.28.2 改动**：API 响应 DTO 支持 `Children []Node`、`Next *Node`、递归 map 和相互引用，生成强类型 Go 字段及有限的 OpenAPI `$ref` 图。字段约束、示例校验、生成与漂移检查继续有效；请求递归和非法值递归仍会报错。见 [可运行示例](examples/recursiveResponse/README.md)与[接入说明](DEVELOPMENT.md#递归响应-dto)。 + **v0.28.1 改动**：`workspace generate/check` 支持 API 与通用工程混合处理，可选 `kind: api | project`，省略时按生成清单识别。`format` 对通用工程显示跳过；检查只读，生成前统一预检。见 [混合工作区示例](examples/mixedWorkspace/README.md)。 **v0.28 改动**：新增独立的 `project new/generate/check`，默认无 API、监听端口、业务任务或外部服务配置；保护手写代码，并等待主运行入口和登记的后台工作结束后释放依赖。见 [通用工程指南](PROGRAM.md)和[可运行示例](examples/program/README.md)。 **v0.27 改动**：统一三种内置服务实现的默认/路由/硬请求体限额，配置冲突在启动前报错；停机以处理函数实际退出为资源清理前提，超时仍有请求时保留共享依赖。见 [HTTP 策略](transport/README.md#统一请求体限额)、[停机规则](lifecycle/README.md#http-请求与依赖的关闭屏障)和[可运行示例](examples/httpSafety/README.md)。 -**v0.26 改动**：通用 [i18n](i18n/README.md)，提供独立词典、语言匹配、回退、命名参数/条件/复数、可选 HTTP 与错误适配、检查/导出 CLI 及前端运行时。文案由项目维护；[完整示例](examples/i18n/README.md)可直接运行。CLI 与框架使用同版 v0.28.1-preview。同时修复 Go 1.22 下请求体限额错误的识别。 +**v0.26 改动**：通用 [i18n](i18n/README.md)，提供独立词典、语言匹配、回退、命名参数/条件/复数、可选 HTTP 与错误适配、检查/导出 CLI 及前端运行时。文案由项目维护；[完整示例](examples/i18n/README.md)可直接运行。CLI 与框架使用同版 v0.28.2-preview。同时修复 Go 1.22 下请求体限额错误的识别。 -**从这份 README 开始接入。** 当前版本 **`v0.28.1-preview`**，通用工程采用根目录入口和 `config / svc / run`；API 工程使用 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。 +**从这份 README 开始接入。** 当前版本 **`v0.28.2-preview`**，通用工程采用根目录入口和 `config / svc / run`；API 工程使用 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。 **v0.25 改动**：`database.InTx` 返回可识别的事务阶段与结果，保留业务/驱动及回滚错误链；提交报错保守标记为结果不确定，不等同于已回滚。[使用规则](database/README.md#事务阶段与结果)、[可运行故障示例](examples/transactions/README.md)。 @@ -75,7 +77,7 @@ Go **1.22+**。框架不依赖 Gin，默认使用标准库网络层，提供可 ## 通用工程快速开始 ```sh -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.2-preview go-omni project new demo --module example.com/demo cd demo go mod tidy @@ -180,7 +182,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可 ```sh export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk" -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.2-preview export PATH="$(go env GOPATH)/bin:$PATH" go-omni --help ``` @@ -337,7 +339,7 @@ go-omni new --api catalog.api \ 在业务项目的 **go.mod 所在目录**执行： ```sh -go get git.yuntuo.hk/k2safe/go-omni@v0.28.1-preview +go get git.yuntuo.hk/k2safe/go-omni@v0.28.2-preview ``` 已有项目可直接复用现有目录、Service、Repository 和数据库连接，先选择一个接口接入。也可以只用 `logx`、`storage` 等包，暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目，按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。 @@ -668,7 +670,7 @@ go test ./... `main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理，手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目，重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段，明确的签名错误会给出原签名与期望签名；生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交，CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。 -CLI 与业务 module 固定使用 `v0.28.1-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load，再升级依赖与 CLI 并重新生成，框架维护的 main/runtimeGen 自动更新；v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理，不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移)，旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 +CLI 与业务 module 固定使用 `v0.28.2-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load，再升级依赖与 CLI 并重新生成，框架维护的 main/runtimeGen 自动更新；v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理，不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移)，旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 ## 按需深入 diff --git a/RELEASE.md b/RELEASE.md index defba7d..92c05b3 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,3 +1,20 @@ +# v0.28.2-preview + +修复 `.api` 响应 DTO 自引用与相互引用的生成限制。`Children []Node`、`Next *Node`、`map[string]Node` 保留强类型，响应文档使用有限的命名组件与 `$ref`；可空引用、字段描述、示例、约束和 scalar 输出类型保持准确。框架不引入业务模型。 + +请求绑定继续拒绝递归类型，Go 非法直接值循环与字段组合循环继续报错。已有非递归类型保持原有生成行为；无需修改运行时配置或响应 JSON。升级 CLI，在项目 `.api` 中恢复强类型后重新生成，再执行检查和项目回归。生成模板最低运行时版本不变。 + +```sh +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.2-preview +go-omni api generate +go-omni api generate --check +go test ./... +``` + +[递归响应说明](DEVELOPMENT.md#递归响应-dto)、[可运行示例](examples/recursiveResponse/README.md)。56 个测试包全量 race 与 Go 1.22 集成回归、vet/build、独立 OpenAPI 校验均通过，见[验收记录](reports/recursiveResponseAcceptance.md)。 + +--- + # v0.28.1-preview workspace generate/check 支持 API 与通用工程混合分发，也可处理纯通用工程工作区。services 新增可选 kind（api/project）；省略时按生成清单识别，保留无清单时显式 api 入口的原有用法。类型与清单冲突、歧义和不合法选项会报错，项目身份、路径与文件归属继续由各生成器校验。 diff --git a/cmd/go-omni/README.md b/cmd/go-omni/README.md index 046ac8d..d1b595a 100644 --- a/cmd/go-omni/README.md +++ b/cmd/go-omni/README.md @@ -20,7 +20,7 @@ API 新项目采用根目录 `main.go` 与 `internal/config、svc、logic、hand ```sh # 仓库读取权限与 PATH 配置见根目录 README -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.2-preview go-omni new --module example.com/hello ./hello # 从自己的契约生成；service.api 及其相对 import 文件需要在本地存在 @@ -33,6 +33,8 @@ go-omni new --api service.api --module example.com/example ./example 存储配置和客户端位于 config/svc，业务使用统一接口。local 存储签名端点由生成器接入；上传、下载等业务路由由 `.api` 定义。认证与存储路径由项目实现。 +响应 DTO 支持自引用及相互引用，生成强类型 Go 字段与 OpenAPI `$ref`；见[递归响应示例](../../examples/recursiveResponse/README.md)。请求递归仍会报错。 + ## 日常生成 ```sh @@ -55,14 +57,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护；业务文件按字节 ## 版本与迁移 -CLI 与框架依赖固定使用 v0.28.1-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换： +CLI 与框架依赖固定使用 v0.28.2-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换： ```sh # 在 go-omni 源码仓库中执行 go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello ``` -当前模板与依赖以 v0.28.1-preview 为基线，继续使用 manifest v2；GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级)：原始模板可自动替换，含业务修改的基础文件按预览合并；通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。 +当前模板与依赖以 v0.28.2-preview 为基线，继续使用 manifest v2；GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级)：原始模板可自动替换，含业务修改的基础文件按预览合并；通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。 ## 数据库迁移命令 diff --git a/cmd/go-omni/apigen/compile.go b/cmd/go-omni/apigen/compile.go index 02aea00..796ed84 100644 --- a/cmd/go-omni/apigen/compile.go +++ b/cmd/go-omni/apigen/compile.go @@ -5,8 +5,6 @@ import ( "encoding/json" "errors" "fmt" - "go/ast" - goparser "go/parser" gotoken "go/token" "reflect" "regexp" @@ -69,56 +67,17 @@ func Compile(spec *Spec) (*Compiled, error) { if err != nil { return nil, err } - resolved := map[string]reflect.Type{} - resolving := map[string]bool{} - var resolve func(string, int) (reflect.Type, error) - var expression func(ast.Expr, int) (reflect.Type, error) - - expression = func(e ast.Expr, depth int) (reflect.Type, error) { - if depth > 32 { - return nil, errors.New("type nesting exceeds 32") - } - switch e := e.(type) { - case *ast.Ident: - if t, ok := scalarTypes[e.Name]; ok { - return t, nil - } - if t, ok := primitiveTypes[e.Name]; ok { - return t, nil - } - return resolve(e.Name, depth+1) - case *ast.StarExpr: - if _, double := e.X.(*ast.StarExpr); double { - return nil, errors.New("multiple pointer levels unsupported") - } - t, err := expression(e.X, depth+1) - if err != nil { - return nil, err - } - return reflect.PointerTo(t), nil - case *ast.ArrayType: - if e.Len != nil { - return nil, errors.New("use slices, not fixed arrays") - } - t, err := expression(e.Elt, depth+1) - if err != nil { - return nil, err - } - return reflect.SliceOf(t), nil - case *ast.MapType: - key, ok := e.Key.(*ast.Ident) - if !ok || key.Name != "string" { - return nil, errors.New("map keys must be string") - } - t, err := expression(e.Value, depth+1) - if err != nil { - return nil, err - } - return reflect.MapOf(reflect.TypeOf(""), t), nil - default: - return nil, errors.New("unsupported type expression") + graph, err := newResponseGraph(spec, definitions, scalarTypes, scalarDefs) + if err != nil { + return nil, err + } + for _, r := range spec.Routes { + if graph.recursive[r.Request] { + return nil, fmt.Errorf("%s: recursive request DTO %s is unsupported; recursive types are supported for responses only", r.Position, r.Request) } } + resolved := map[string]reflect.Type{} + var resolve func(string, int) (reflect.Type, error) resolve = func(name string, depth int) (reflect.Type, error) { if depth > 32 { return nil, errors.New("type nesting exceeds 32") @@ -126,27 +85,13 @@ func Compile(spec *Spec) (*Compiled, error) { if t, ok := resolved[name]; ok { return t, nil } - if resolving[name] { - return nil, fmt.Errorf("recursive type %s is unsupported", name) - } typ, ok := definitions[name] if !ok { return nil, fmt.Errorf("unknown type %s", name) } - resolving[name] = true - defer delete(resolving, name) var fields []reflect.StructField - seen := map[string]bool{} - for _, f := range typ.Fields { - if !exportedName.MatchString(f.Name) || seen[f.Name] { - return nil, fmt.Errorf("%s: invalid or duplicate field %s", name, f.Name) - } - seen[f.Name] = true - expr, err := goparser.ParseExpr(f.Type) - if err != nil { - return nil, err - } - t, err := expression(expr, depth+1) + for i, f := range typ.Fields { + t, err := graph.expression(graph.exprs[name][i], depth+1, resolve) if err != nil { return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err) } @@ -158,6 +103,12 @@ func Compile(spec *Spec) (*Compiled, error) { } c := &Compiled{Spec: spec, Bindings: map[string]*contract.Binder{}} for _, t := range spec.Types { + if graph.recursive[t.Name] { + if _, err := graph.component(t.Name); err != nil { + return nil, err + } + continue + } rt, err := resolve(t.Name, 0) if err != nil { return nil, err @@ -274,7 +225,7 @@ func Compile(spec *Spec) (*Compiled, error) { } var response *openapi.Schema if r.Response != "" { - s, err := responseDataSchema(r.Response, c.Bindings) + s, err := responseDataSchema(r.Response, c.Bindings, graph) if err != nil { return nil, err } @@ -316,6 +267,13 @@ func Compile(spec *Spec) (*Compiled, error) { return nil, err } decorateDocument(doc, spec, c.Bindings) + if len(graph.components) > 0 { + components := map[string]any{} + for name, schema := range graph.components { + components[name] = schema + } + doc["components"].(map[string]any)["schemas"] = components + } compactDocument(doc) c.Document, err = json.MarshalIndent(doc, "", " ") return c, err diff --git a/cmd/go-omni/apigen/generate.go b/cmd/go-omni/apigen/generate.go index aeabea9..6e68986 100644 --- a/cmd/go-omni/apigen/generate.go +++ b/cmd/go-omni/apigen/generate.go @@ -23,7 +23,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" // Runtime-only releases need not raise the managed template compatibility floor. const minimumVersion = "v0.22.0-preview" -const currentVersion = "v0.28.1-preview" +const currentVersion = "v0.28.2-preview" const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n" type Options struct { diff --git a/cmd/go-omni/apigen/recursiveResponse_test.go b/cmd/go-omni/apigen/recursiveResponse_test.go new file mode 100644 index 0000000..412666f --- /dev/null +++ b/cmd/go-omni/apigen/recursiveResponse_test.go @@ -0,0 +1,299 @@ +package apigen + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" +) + +func compileRecursive(t *testing.T, source string) *Compiled { + t.Helper() + spec, err := parseText(t, source) + if err != nil { + t.Fatal(err) + } + c, err := Compile(spec) + if err != nil { + t.Fatal(err) + } + again, err := Compile(spec) + if err != nil || !bytes.Equal(c.Document, again.Document) { + t.Fatal("non-deterministic compilation", err) + } + return c +} + +func recursiveDocument(t *testing.T, c *Compiled) map[string]any { + t.Helper() + var doc map[string]any + if err := json.Unmarshal(c.Document, &doc); err != nil { + t.Fatal(err) + } + // Inspect reference targets without recursively inlining the graph. + var visit func(any) + visit = func(value any) { + switch v := value.(type) { + case map[string]any: + if ref, ok := v["$ref"].(string); ok { + if len(v) != 1 { + t.Fatalf("OpenAPI 3.0 ref has ignored siblings: %v", v) + } + var target any = doc + for _, part := range strings.Split(strings.TrimPrefix(ref, "#/"), "/") { + m, ok := target.(map[string]any) + if !ok { + t.Fatal("bad reference", ref) + } + target = m[part] + } + if target == nil { + t.Fatal("dangling reference", ref) + } + return + } + for key, child := range v { + if key != "example" && key != "examples" { + visit(child) + } + } + case []any: + for _, child := range v { + visit(child) + } + } + } + visit(doc) + return doc +} + +func TestRecursiveResponseGraph(t *testing.T) { + source := readFile(t, filepath.Join(frameworkRoot(t), "examples/recursiveResponse/service.api")) + c := compileRecursive(t, source) + doc := recursiveDocument(t, c) + schemas := doc["components"].(map[string]any)["schemas"].(map[string]any) + node := schemas["DTO_Node"].(map[string]any) + props := node["properties"].(map[string]any) + children := props["children"].(map[string]any) + if children["type"] != "array" || children["maxItems"] != float64(100) || children["items"].(map[string]any)["$ref"] != "#/components/schemas/DTO_Node" { + t.Fatal(children) + } + if props["lookup"].(map[string]any)["additionalProperties"].(map[string]any)["$ref"] != "#/components/schemas/DTO_Node" { + t.Fatal(props) + } + next := props["next"].(map[string]any)["anyOf"].([]any) + if len(next) != 2 || next[0].(map[string]any)["$ref"] != "#/components/schemas/DTO_Node" || next[1].(map[string]any)["nullable"] != true { + t.Fatal(next) + } + for _, pair := range [][2]string{{"Branch", "Leaf"}, {"Leaf", "Branch"}} { + p := schemas["DTO_"+pair[0]].(map[string]any)["properties"].(map[string]any)[strings.ToLower(pair[1])].(map[string]any) + if p["anyOf"].([]any)[0].(map[string]any)["$ref"] != "#/components/schemas/DTO_"+pair[1] { + t.Fatal(p) + } + } + if len(c.Document) > 18000 { + t.Fatal("recursive graph expanded", len(c.Document)) + } + first, _ := json.Marshal(doc) + compactDocument(doc) + second, _ := json.Marshal(doc) + if !bytes.Equal(first, second) { + t.Fatal("document compaction changed references") + } + // Change tracking traverses every named dependency once, including cycles. + snapshot := snapshotContract(c.Spec) + if len(snapshot.Routes) == 0 { + t.Fatal("empty contract snapshot") + } +} + +func TestRecursiveResponseRejections(t *testing.T) { + tests := []struct{ name, types, route, want string }{ + {"valueSelf", "type Node { Child Node `json:\"child\"` }", "get /tree returns (Node)", "recursive value"}, + {"valueMutual", "type Node { Child Other `json:\"child\"` }\ntype Other { Child Node `json:\"child\"` }", "get /tree returns (Node)", "recursive value"}, + {"request", "type Node { Children []Node `json:\"children\"` }", "post /tree (Node) returns (Node)", "recursive request"}, + {"wrappedRequest", "type Node { Children []Node `json:\"children\"` }\ntype Req { Value Node `json:\"value\"` }", "post /tree (Req)", "recursive request"}, + {"unknown", "type Node { Children []Node `json:\"children\"`\n Bad Missing `json:\"bad\"` }", "get /tree returns (Node)", "unknown type"}, + {"duplicate", "type Node { Children []Node `json:\"children\"`\n Again string `json:\"children\"` }", "get /tree returns (Node)", "conflicting composed field"}, + {"nonJSON", "type Node { Children []Node `json:\"children\"`\n Name string `form:\"name\"` }", "get /tree returns (Node)", "response fields must use json"}, + {"nestedNonJSON", "type Node { Children []Node `json:\"children\"`\n Detail Detail `json:\"detail\"` }\ntype Detail { Name string `form:\"name\"` }", "get /tree returns (Node)", "response fields must use json"}, + {"doublePointer", "type Node { Next **Node `json:\"next\"` }", "get /tree returns (Node)", "multiple pointer"}, + {"fixedArray", "type Node { Children [1]Node `json:\"children\"` }", "get /tree returns (Node)", "fixed arrays"}, + {"mapKey", "type Node { Children map[int]Node `json:\"children\"` }", "get /tree returns (Node)", "map keys"}, + {"badRule", "type Node { Next *Node `json:\"next\" binding:\"min=1\"` }", "get /tree returns (Node)", "requires a number"}, + {"badDefault", "type Node { Next *Node `json:\"next,default=x\"` }", "get /tree returns (Node)", "default requires"}, + {"upload", "type Node { Children []Node `json:\"children\"`\n Upload File `file:\"upload\"` }", "get /tree returns (Node)", "response fields must use json"}, + {"composition", "type Node { Node }", "get /tree returns (Node)", "cycle"}, + } + for _, tc := range tests { + t.Run(tc.name, func(t *testing.T) { + spec, err := parseText(t, tc.types+"\nservice demo {\n@handler tree\n"+tc.route+"\n}") + if err == nil { + _, err = Compile(spec) + } + if err == nil || !strings.Contains(err.Error(), tc.want) { + t.Fatalf("want %q, got %v", tc.want, err) + } + }) + } +} + +func TestRecursiveResponseExamples(t *testing.T) { + for _, tc := range []struct { + value string + valid bool + }{ + {`[]`, true}, {`[{"name":"first","children":[{"name":"second","children":[]}]}]`, true}, + {`[{"name":"","children":[]}]`, false}, {`[{"name":"first"}]`, false}, + {`[{"name":"first","children":[],"unknown":true}]`, false}, + {`[{"name":"first","children":[]},{"name":123,"children":[]}]`, false}, + {`[{"name":"first","children":[{"name":"second","children":[{"name":"","children":[]}]}]}]`, false}, + {`null`, false}, {`{} {}`, false}, + } { + t.Run(tc.value, func(t *testing.T) { + source := "type Node {\n Name string `json:\"name\" binding:\"min=1\"`\n Children []Node `json:\"children\" description:\"Nested nodes\" example:" + fmt.Sprintf("%q", tc.value) + "`\n}\nservice demo {\n @handler tree\nget /tree returns (Node)\n}" + spec, err := parseText(t, source) + if err == nil { + _, err = Compile(spec) + } + if (err == nil) != tc.valid { + t.Fatal(err) + } + }) + } + for _, typ := range []string{"*Node", "map[string]*Node", "[]*Node", "*[]Node"} { + source := "type Node {\n Name string `json:\"name\"`\n Next " + typ + " `json:\"next,omitempty\" description:\"Next node\"`\n}\nservice demo {\n@handler tree\nget /tree returns (*Node)\n}" + recursiveDocument(t, compileRecursive(t, source)) + } +} + +func TestRecursiveResponseImportsAndMetadata(t *testing.T) { + root := t.TempDir() + writeFile(t, filepath.Join(root, "service.api"), "import \"node.api\"\nimport \"detail.api\"\nservice demo {\n@handler tree\nget /tree returns (Node)\n}") + writeFile(t, filepath.Join(root, "node.api"), "type Node {\n Children []Node `json:\"children\"`\n Detail Detail `json:\"detail\" description:\"Node detail\" example:\"{\\\"count\\\":9007199254740993,\\\"parent\\\":null}\"`\n}") + writeFile(t, filepath.Join(root, "detail.api"), "type Detail {\n Count int64 `json:\"count\"`\n Parent *Node `json:\"parent\"`\n}") + spec, err := Parse(filepath.Join(root, "service.api")) + if err != nil { + t.Fatal(err) + } + c, err := Compile(spec) + if err != nil { + t.Fatal(err) + } + doc := recursiveDocument(t, c) + schemas := doc["components"].(map[string]any)["schemas"].(map[string]any) + detail := schemas["DTO_Node"].(map[string]any)["properties"].(map[string]any)["detail"].(map[string]any) + if detail["description"] != "Node detail" || detail["allOf"].([]any)[0].(map[string]any)["$ref"] != "#/components/schemas/DTO_Detail" { + t.Fatal(detail) + } + if !bytes.Contains(c.Document, []byte("9007199254740993")) { + t.Fatal("example integer lost precision") + } + // Definition order cannot change cycle classification or component schemas. + spec.Types[0], spec.Types[1] = spec.Types[1], spec.Types[0] + other, err := Compile(spec) + if err != nil || !bytes.Equal(c.Document, other.Document) { + t.Fatal("definition order changed schemas", err) + } + + scalar := `scalar ID ( +goImport: "example.com/scalars" +goType: ID +jsonInput: string | int64 +jsonOutput: string +) +type Node { +Value ID ` + "`json:\"value\" example:\"123\"`" + ` +Children []Node ` + "`json:\"children\"`" + ` +} +service demo { +@handler tree +get /tree returns (Node) +}` + doc = recursiveDocument(t, compileRecursive(t, scalar)) + value := doc["components"].(map[string]any)["schemas"].(map[string]any)["DTO_Node"].(map[string]any)["properties"].(map[string]any)["value"].(map[string]any) + if value["type"] != "string" || value["example"] != nil { + t.Fatal("scalar input leaked into response", value) + } +} + +func TestRecursiveResponseConsumer(t *testing.T) { + if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" { + t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1") + } + root := frameworkRoot(t) + dir := filepath.Join(t.TempDir(), "recursive") + o := Options{Dir: dir, API: filepath.Join(root, "examples/recursiveResponse/service.api"), Module: "example.com/recursive", FrameworkDir: root} + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + doc := recursiveDocument(t, &Compiled{Document: []byte(readFile(t, filepath.Join(dir, "docs/openapi.json")))}) + schemas := doc["components"].(map[string]any)["schemas"].(map[string]any) + children := schemas["DTO_Node"].(map[string]any)["properties"].(map[string]any)["children"].(map[string]any) + if children["items"].(map[string]any)["$ref"] != "#/components/schemas/DTO_Node" { + t.Fatal("generated response lost its recursive schema", children) + } + for _, name := range []string{"tree", "forest", "optional", "raw", "pair"} { + writeFile(t, filepath.Join(dir, "internal/logic/trees/"+name+".go"), "package trees\n") + } + writeFile(t, filepath.Join(dir, "internal/logic/trees/operations.go"), readFile(t, filepath.Join(root, "examples/recursiveResponse/operations.go.txt"))) + writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/recursive/consumer_test.go")) + o.API = filepath.Join(dir, "api/service.api") + before := snapshot(t, dir) + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + o.Check = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("regeneration/check modified business code") + } + runConsumerGo(t, dir, "test", "-mod=mod", "-race", "-count=1", "./...") + // A recursive descendant change must still be visible to read-only checking. + writeFile(t, o.API, strings.Replace(readFile(t, o.API), "max=100", "max=90", 1)) + before = snapshot(t, dir) + if err := generateManagedTestProject(t, o); err == nil { + t.Fatal("check missed recursive DTO drift") + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("drift check wrote files") + } + o.Check = false + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + o.Check = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + if os.Getenv("GO_OMNI_RELEASE_CLI") != "" { + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GOWORK=off") + data, err := cmd.CombinedOutput() + if err != nil { + t.Fatal(err, string(data)) + } + var dep struct { + Version, Sum string + Replace any + } + if err := json.Unmarshal(data, &dep); err != nil { + t.Fatal(err) + } + if dep.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dep.Sum == "" || dep.Replace != nil { + t.Fatal("not a fixed remote consumer") + } + t.Logf("remote framework %s checksum %s; no local replace", dep.Version, dep.Sum) + } +} diff --git a/cmd/go-omni/apigen/responseGraph.go b/cmd/go-omni/apigen/responseGraph.go new file mode 100644 index 0000000..671e9fd --- /dev/null +++ b/cmd/go-omni/apigen/responseGraph.go @@ -0,0 +1,370 @@ +package apigen + +import ( + "encoding/json" + "fmt" + "go/ast" + goparser "go/parser" + "reflect" + "sort" + "strconv" + "strings" + + "git.yuntuo.hk/k2safe/go-omni/contract" + "git.yuntuo.hk/k2safe/go-omni/openapi" +) + +// Named response types form a graph, not an infinitely expanded reflect.StructOf +// tree. Compile each object's local fields with the existing contract compiler, +// then link named fields through OpenAPI references. Request binders still use +// the complete, nonrecursive reflection tree. +type responseGraph struct { + types map[string]Type + exprs map[string][]ast.Expr + recursive map[string]bool // includes types that can reach a cycle + scalars map[string]reflect.Type + scalarDefs []contract.Scalar + components map[string]*openapi.Schema +} + +func newResponseGraph(spec *Spec, types map[string]Type, scalars map[string]reflect.Type, defs []contract.Scalar) (*responseGraph, error) { + g := &responseGraph{types: types, exprs: map[string][]ast.Expr{}, recursive: map[string]bool{}, scalars: scalars, scalarDefs: defs, components: map[string]*openapi.Schema{}} + dependencies, values := map[string][]string{}, map[string][]string{} + for _, typ := range spec.Types { + seen := map[string]bool{} + for _, f := range typ.Fields { + if !exportedName.MatchString(f.Name) || seen[f.Name] { + return nil, fmt.Errorf("%s: invalid or duplicate field %s", typ.Name, f.Name) + } + seen[f.Name] = true + e, err := goparser.ParseExpr(f.Type) + if err != nil { + return nil, err + } + // Validate every expression, including otherwise unused declarations. + if _, err = g.expression(e, 0, func(name string, _ int) (reflect.Type, error) { + dependencies[typ.Name] = append(dependencies[typ.Name], name) + return reflect.TypeOf(struct{}{}), nil + }); err != nil { + return nil, fmt.Errorf("%s.%s: %w", typ.Name, f.Name, err) + } + g.exprs[typ.Name] = append(g.exprs[typ.Name], e) + if id, ok := e.(*ast.Ident); ok && types[id.Name].Name != "" { + values[typ.Name] = append(values[typ.Name], id.Name) + } + } + } + // Only pointer/slice/map indirection can break a Go struct layout cycle. + state := map[string]int{} + var valueCycle func(string) error + valueCycle = func(name string) error { + if state[name] == 1 { + return fmt.Errorf("%s: invalid recursive value type; use a pointer, slice or map", name) + } + if state[name] == 2 { + return nil + } + state[name] = 1 + for _, next := range values[name] { + if err := valueCycle(next); err != nil { + return err + } + } + state[name] = 2 + return nil + } + for _, typ := range spec.Types { + if err := valueCycle(typ.Name); err != nil { + return nil, err + } + } + state = map[string]int{} + var cyclic func(string) bool + cyclic = func(name string) bool { + if state[name] == 1 { + return true + } + if state[name] == 2 { + return g.recursive[name] + } + state[name] = 1 + for _, next := range dependencies[name] { + if cyclic(next) { + g.recursive[name] = true + } + } + state[name] = 2 + return g.recursive[name] + } + for _, typ := range spec.Types { + cyclic(typ.Name) + } + return g, nil +} + +func (g *responseGraph) expression(e ast.Expr, depth int, named func(string, int) (reflect.Type, error)) (reflect.Type, error) { + if depth > 32 { + return nil, fmt.Errorf("type nesting exceeds 32") + } + switch e := e.(type) { + case *ast.Ident: + if t := g.scalars[e.Name]; t != nil { + return t, nil + } + if t := primitiveTypes[e.Name]; t != nil { + return t, nil + } + if g.types[e.Name].Name == "" { + return nil, fmt.Errorf("unknown type %s", e.Name) + } + return named(e.Name, depth+1) + case *ast.StarExpr: + if _, double := e.X.(*ast.StarExpr); double { + return nil, fmt.Errorf("multiple pointer levels unsupported") + } + t, err := g.expression(e.X, depth+1, named) + if err != nil { + return nil, err + } + return reflect.PointerTo(t), nil + case *ast.ArrayType: + if e.Len != nil { + return nil, fmt.Errorf("use slices, not fixed arrays") + } + t, err := g.expression(e.Elt, depth+1, named) + if err != nil { + return nil, err + } + return reflect.SliceOf(t), nil + case *ast.MapType: + key, ok := e.Key.(*ast.Ident) + if !ok || key.Name != "string" { + return nil, fmt.Errorf("map keys must be string") + } + t, err := g.expression(e.Value, depth+1, named) + if err != nil { + return nil, err + } + return reflect.MapOf(reflect.TypeOf(""), t), nil + default: + return nil, fmt.Errorf("unsupported type expression") + } +} + +func (g *responseGraph) component(name string) (*openapi.Schema, error) { + key := "DTO_" + name + ref := &openapi.Schema{Ref: "#/components/schemas/" + key} + if _, ok := g.components[key]; ok { + return ref, nil + } + // Register before following edges so both self and mutual references terminate. + g.components[key] = nil + typ := g.types[name] + var fields []reflect.StructField + for i, f := range typ.Fields { + t, err := g.expression(g.exprs[name][i], 0, func(string, int) (reflect.Type, error) { return reflect.TypeOf(struct{}{}), nil }) + if err != nil { + return nil, err + } + tag, err := withoutExample(f) + if err != nil { + return nil, err + } + fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: tag}) + } + b, err := contract.Describe(reflect.StructOf(fields), g.scalarDefs...) + if err != nil { + return nil, fmt.Errorf("%s: %w", typ.Position, err) + } + s, err := b.ResponseSchema() + if err != nil { + return nil, fmt.Errorf("%s: %w", typ.Position, err) + } + for i, f := range typ.Fields { + wire := strings.Split(reflect.StructTag(fieldTag(f)).Get("json"), ",")[0] + property := s.Properties[wire] + // Examples are checked against a finite projection of their actual values, + // with the same contract rules and without running application decoders. + if _, ok := reflect.StructTag(fieldTag(f)).Lookup("example"); ok { + example, err := g.example(f, g.exprs[name][i]) + if err != nil { + return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err) + } + property.Example = example + } + property, err = g.link(g.exprs[name][i], property) + if err != nil { + return nil, err + } + s.Properties[wire] = property + } + g.components[key] = s + return ref, nil +} + +func (g *responseGraph) link(e ast.Expr, s *openapi.Schema) (*openapi.Schema, error) { + switch e := e.(type) { + case *ast.Ident: + if g.types[e.Name].Name == "" { + return s, nil + } + ref, err := g.component(e.Name) + if err != nil { + return nil, err + } + if s.Nullable { + ref = nullableReference(ref) + } + if s.Description != "" || s.Example != nil { + // OpenAPI 3.0 ignores siblings on Reference Objects. + ref = &openapi.Schema{AllOf: []*openapi.Schema{ref}, Description: s.Description, Example: s.Example} + } + return ref, nil + case *ast.StarExpr: + return g.link(e.X, s) + case *ast.ArrayType: + item, err := g.link(e.Elt, s.Items) + if err != nil { + return nil, err + } + s.Items = item + case *ast.MapType: + item, err := g.link(e.Value, s.AdditionalProperties) + if err != nil { + return nil, err + } + s.AdditionalProperties = item + } + return s, nil +} + +func nullableReference(s *openapi.Schema) *openapi.Schema { + return &openapi.Schema{AnyOf: []*openapi.Schema{s, {Type: "object", Nullable: true, Enum: []any{nil}}}} +} + +func withoutExample(f Field) (reflect.StructTag, error) { + tags, err := contract.ParseTags(fieldTag(f)) + if err != nil { + return "", err + } + delete(tags, "example") + keys := make([]string, 0, len(tags)) + for key := range tags { + keys = append(keys, key) + } + sort.Strings(keys) + var parts []string + for _, key := range keys { + parts = append(parts, key+":"+strconv.Quote(tags[key])) + } + return reflect.StructTag(strings.Join(parts, " ")), nil +} + +func (g *responseGraph) example(f Field, e ast.Expr) (any, error) { + raw := reflect.StructTag(fieldTag(f)).Get("example") + var value any + // Plain string examples intentionally are not JSON literals. + if f.Type == "string" || f.Type == "*string" { + value = raw + } else { + decoder := json.NewDecoder(strings.NewReader(raw)) + decoder.UseNumber() + if err := decoder.Decode(&value); err != nil { + return nil, fmt.Errorf("invalid example JSON: %w", err) + } + } + budget := 10000 + t, err := g.projectExample(e, []any{value}, 0, &budget) + if err != nil { + return nil, err + } + b, err := contract.Describe(reflect.StructOf([]reflect.StructField{{Name: f.Name, Type: t, Tag: reflect.StructTag(fieldTag(f))}}), g.scalarDefs...) + if err != nil { + return nil, err + } + s, err := b.ResponseSchema() + if err != nil { + return nil, err + } + wire := strings.Split(reflect.StructTag(fieldTag(f)).Get("json"), ",")[0] + return s.Properties[wire].Example, nil +} + +// Only materialize named objects where the finite example has a non-null value. +// Empty slices/maps and null pointers stop expansion. Unioning collection values +// checks every item using one reflection type, including missing/unknown fields. +func (g *responseGraph) projectExample(e ast.Expr, values []any, depth int, budget *int) (reflect.Type, error) { + *budget-- + if depth > 32 || *budget < 0 { + return nil, fmt.Errorf("example type expansion exceeds 32 levels or 10000 nodes") + } + var present []any + for _, v := range values { + if v != nil { + present = append(present, v) + } + } + switch e := e.(type) { + case *ast.Ident: + return g.expression(e, 0, func(name string, _ int) (reflect.Type, error) { + if len(present) == 0 { + return reflect.TypeOf(struct{}{}), nil + } + var fields []reflect.StructField + for i, f := range g.types[name].Fields { + wire := strings.Split(reflect.StructTag(fieldTag(f)).Get("json"), ",")[0] + var nested []any + for _, v := range present { + if obj, ok := v.(map[string]any); ok { + if child, found := obj[wire]; found { + nested = append(nested, child) + } + } + } + t, err := g.projectExample(g.exprs[name][i], nested, depth+1, budget) + if err != nil { + return nil, err + } + tag, err := withoutExample(f) + if err != nil { + return nil, err + } + fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: tag}) + } + return reflect.StructOf(fields), nil + }) + case *ast.StarExpr: + t, err := g.projectExample(e.X, present, depth+1, budget) + if err != nil { + return nil, err + } + return reflect.PointerTo(t), nil + case *ast.ArrayType: + var nested []any + for _, v := range present { + if list, ok := v.([]any); ok { + nested = append(nested, list...) + } + } + t, err := g.projectExample(e.Elt, nested, depth+1, budget) + if err != nil { + return nil, err + } + return reflect.SliceOf(t), nil + case *ast.MapType: + var nested []any + for _, v := range present { + if obj, ok := v.(map[string]any); ok { + for _, v := range obj { + nested = append(nested, v) + } + } + } + t, err := g.projectExample(e.Value, nested, depth+1, budget) + if err != nil { + return nil, err + } + return reflect.MapOf(reflect.TypeOf(""), t), nil + } + return nil, fmt.Errorf("unsupported example type") +} diff --git a/cmd/go-omni/apigen/responseShape.go b/cmd/go-omni/apigen/responseShape.go index cf23a15..465f3f0 100644 --- a/cmd/go-omni/apigen/responseShape.go +++ b/cmd/go-omni/apigen/responseShape.go @@ -35,7 +35,7 @@ func responseGoType(value string) string { } return "*types." + name } -func responseDataSchema(value string, bindings map[string]*contract.Binder) (*openapi.Schema, error) { +func responseDataSchema(value string, bindings map[string]*contract.Binder, graph *responseGraph) (*openapi.Schema, error) { kind, name := responseShape(value) if kind == "null" { return &openapi.Schema{Type: "object", Nullable: true, Enum: []any{nil}}, nil @@ -56,11 +56,15 @@ func responseDataSchema(value string, bindings map[string]*contract.Binder) (*op } return s.Properties["value"], nil } - b, ok := bindings[name] - if !ok { + var s *openapi.Schema + var err error + if graph.recursive[name] { + s, err = graph.component(name) + } else if b := bindings[name]; b != nil { + s, err = b.ResponseSchema() + } else { return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, []string/[]int64/[]bool/[]any, *Name or null", name) } - s, err := b.ResponseSchema() if err != nil { return nil, err } @@ -68,6 +72,9 @@ func responseDataSchema(value string, bindings map[string]*contract.Binder) (*op case "array": return &openapi.Schema{Type: "array", Items: s}, nil case "nullable": + if s.Ref != "" { + return nullableReference(s), nil + } s.Nullable = true } return s, nil diff --git a/cmd/go-omni/apigen/testdata/recursive/consumer_test.go b/cmd/go-omni/apigen/testdata/recursive/consumer_test.go new file mode 100644 index 0000000..49949be --- /dev/null +++ b/cmd/go-omni/apigen/testdata/recursive/consumer_test.go @@ -0,0 +1,77 @@ +package main_test + +import ( + "context" + "encoding/json" + "io" + "net/http/httptest" + "testing" + + "example.com/recursive/internal/handler" + "example.com/recursive/internal/svc" + "example.com/recursive/internal/types" + omni "git.yuntuo.hk/k2safe/go-omni" + "git.yuntuo.hk/k2safe/go-omni/logx" +) + +func TestRecursiveHTTP(t *testing.T) { + app := omni.New() + defer app.Close() + cfg := logx.Default() + cfg.Writer = io.Discard + logger, err := logx.New(cfg) + if err != nil { + t.Fatal(err) + } + defer logger.Close() + deps, err := svc.New(context.Background(), logger) + if err != nil { + t.Fatal(err) + } + defer deps.Close() + if err := handler.RegisterRoutes(app, deps); err != nil { + t.Fatal(err) + } + for _, path := range []string{"/tree", "/forest", "/optional", "/raw", "/pair"} { + w := httptest.NewRecorder() + app.ServeHTTP(w, httptest.NewRequest("GET", path, nil)) + if w.Code != 200 { + t.Fatal(path, w.Code, w.Body.String()) + } + data := w.Body.Bytes() + if path != "/raw" { + var envelope struct { + Data json.RawMessage `json:"data"` + } + if err := json.Unmarshal(data, &envelope); err != nil { + t.Fatal(err) + } + data = envelope.Data + } + switch path { + case "/optional": + if string(data) != "null" { + t.Fatal(string(data)) + } + case "/pair": + var pair types.Branch + if err := json.Unmarshal(data, &pair); err != nil || pair.Leaf == nil || pair.Leaf.Branch != nil { + t.Fatal(err, string(data)) + } + default: + var node types.Node + if path == "/forest" { + var forest []types.Node + if err := json.Unmarshal(data, &forest); err != nil || len(forest) != 1 { + t.Fatal(err, string(data)) + } + node = forest[0] + } else if err := json.Unmarshal(data, &node); err != nil { + t.Fatal(err) + } + if len(node.Children) != 1 || len(node.Children[0].Children) != 1 || node.Children[0].Children[0].Name != "leaf" || node.Lookup["entry"].Name != "lookup" { + t.Fatal(node) + } + } + } +} diff --git a/cmd/go-omni/projectgen/generate.go b/cmd/go-omni/projectgen/generate.go index fd5805a..88434a6 100644 --- a/cmd/go-omni/projectgen/generate.go +++ b/cmd/go-omni/projectgen/generate.go @@ -25,7 +25,7 @@ import ( ) const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" -const currentVersion = "v0.28.1-preview" +const currentVersion = "v0.28.2-preview" const minimumVersion = "v0.28.0-preview" const manifestPath = ".goOmni/projectManifest.json" const lockPath = ".goOmni/project.lock" diff --git a/cmd/go-omni/scaffold/version.go b/cmd/go-omni/scaffold/version.go index 0aa895e..a975601 100644 --- a/cmd/go-omni/scaffold/version.go +++ b/cmd/go-omni/scaffold/version.go @@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" // Source builds can carry an unpublished VCS version (including +dirty). // Keep this fallback in sync with the next release; installed CLIs use their // downloaded module version, which also has a module checksum. -const defaultFrameworkVersion = "v0.28.1-preview" +const defaultFrameworkVersion = "v0.28.2-preview" // This target is not a claim that the release already exists. Source users // select a local framework checkout until the module templates are published. diff --git a/contract/README.md b/contract/README.md index 5e25a02..5c57855 100644 --- a/contract/README.md +++ b/contract/README.md @@ -28,3 +28,5 @@ overrides it. Application transport read timeouts still control slow clients. See [API.md](../API.md) for source tags, validation rules, null/presence behavior, strict decoding and supported type limits. This API leaves the existing `Context.BindJSON/BindQuery/BindURI` behavior unchanged. + +Recursive response DTOs in `.api` are compiled by the CLI as named OpenAPI references; see [the response guide](../DEVELOPMENT.md#递归响应-dto). This does not change the nonrecursive request type requirement of `New`, `ForType`, or `Describe`. diff --git a/examples/mixedWorkspace/README.md b/examples/mixedWorkspace/README.md index 2ebbb25..035cedc 100644 --- a/examples/mixedWorkspace/README.md +++ b/examples/mixedWorkspace/README.md @@ -1,11 +1,11 @@ # API 与通用工程混合工作区 -本例只定义生成工程的类型与目录，不包含业务模型、外部服务或特定调度方式。使用 v0.28.1-preview 或更新版本 CLI。 +本例只定义生成工程的类型与目录，不包含业务模型、外部服务或特定调度方式。使用 v0.28.2-preview 或更新版本 CLI。 下面从一个空目录创建两个独立 Go module；API 工程来自 API 模板，通用工程默认不监听端口。 ```sh -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.2-preview mkdir mixed-demo cd mixed-demo go-omni api new apps/interface --module example.com/interface diff --git a/examples/recursiveResponse/README.md b/examples/recursiveResponse/README.md new file mode 100644 index 0000000..8f89561 --- /dev/null +++ b/examples/recursiveResponse/README.md @@ -0,0 +1,32 @@ +# 递归响应类型 + +通用节点、树集合和相互引用对象的响应示例。没有数据库、权限或业务服务依赖。 + +在 go-omni 仓库根目录执行（新建独立临时目录）： + +```sh +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.2-preview +example_dir="$(mktemp -d)/recursive" +go-omni api generate --api examples/recursiveResponse/service.api --dir "$example_dir" --module example.com/recursive +# 以下文件仅是本次新生成示例的占位实现，替换为示例自身维护的逻辑。 +for name in tree forest optional raw pair; do + printf 'package trees\n' > "$example_dir/internal/logic/trees/$name.go" +done +cp examples/recursiveResponse/operations.go.txt "$example_dir/internal/logic/trees/operations.go" +cd "$example_dir" +go mod tidy +go-omni api generate --check +go test ./... +go run . +``` + +按启动日志显示的监听地址访问 `/tree`、`/forest`、`/optional`、`/raw`、`/pair`。前三层节点通过 `[]types.Node` 编译和返回；`/optional` 返回 null，`/raw` 直接返回节点，`/pair` 返回相互引用类型的有限实例。`docs/openapi.json` 中的 `DTO_Node` 通过 `$ref` 引用自身，`DTO_Branch`、`DTO_Leaf` 互相引用。 + +嵌套集合遵循普通 Go JSON 规则，示例将叶节点的 Children 初始化为空切片，使其输出 `[]`；框架不会递归改写项目返回数据。业务数据不能包含实际指针环。 + +框架开发时可在首次生成命令后加 `--framework-dir .` 使用本地代码。回归同时覆盖本地与已发布远程 CLI、真实 HTTP 响应、生成幂等与只读漂移检查： + +```sh +# 回到 go-omni 仓库根目录执行 +GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -count=1 ./cmd/go-omni/apigen -run Recursive +``` diff --git a/examples/recursiveResponse/operations.go.txt b/examples/recursiveResponse/operations.go.txt new file mode 100644 index 0000000..33a75f2 --- /dev/null +++ b/examples/recursiveResponse/operations.go.txt @@ -0,0 +1,22 @@ +package trees + +import ( + "context" + "example.com/recursive/internal/types" +) + +func sample() types.Node { + return types.Node{Name: "root", Children: []types.Node{ + {Name: "child", Children: []types.Node{ + {Name: "leaf", Children: []types.Node{}}, + }}, + }, Lookup: map[string]types.Node{"entry": {Name: "lookup", Children: []types.Node{}}}} +} + +func (s *Logic) Tree(context.Context) (*types.Node, error) { node := sample(); return &node, nil } +func (s *Logic) Forest(context.Context) ([]types.Node, error) { return []types.Node{sample()}, nil } +func (s *Logic) Optional(context.Context) (*types.Node, error) { return nil, nil } +func (s *Logic) Raw(context.Context) (*types.Node, error) { node := sample(); return &node, nil } +func (s *Logic) Pair(context.Context) (*types.Branch, error) { + return &types.Branch{Leaf: &types.Leaf{}}, nil +} diff --git a/examples/recursiveResponse/service.api b/examples/recursiveResponse/service.api new file mode 100644 index 0000000..752dfb2 --- /dev/null +++ b/examples/recursiveResponse/service.api @@ -0,0 +1,33 @@ +type Node { + Name string `json:"name" binding:"min=1"` + Children []Node `json:"children" binding:"max=100"` + Next *Node `json:"next,omitempty"` + Lookup map[string]Node `json:"lookup,omitempty"` +} + +type Branch { + Leaf *Leaf `json:"leaf"` +} + +type Leaf { + Branch *Branch `json:"branch"` +} + +@server (group: trees) +service recursive-example { + @handler tree + get /tree returns (Node) + + @handler forest + get /forest returns ([]Node) + + @handler optional + get /optional returns (*Node) + + @handler raw + @response raw + get /raw returns (Node) + + @handler pair + get /pair returns (Branch) +} diff --git a/openapi/openapi.go b/openapi/openapi.go index ad5d991..329142b 100644 --- a/openapi/openapi.go +++ b/openapi/openapi.go @@ -17,6 +17,8 @@ import ( ) type Schema struct { + Ref string `json:"$ref,omitempty"` + AllOf []*Schema `json:"allOf,omitempty"` MaxBytes *int64 `json:"x-omni-max-bytes,omitempty"` Example any `json:"example,omitempty"` EnumDescriptions []string `json:"x-enum-descriptions,omitempty"` diff --git a/openapi/schemaJSON_test.go b/openapi/schemaJSON_test.go index b991a68..255865b 100644 --- a/openapi/schemaJSON_test.go +++ b/openapi/schemaJSON_test.go @@ -2,6 +2,7 @@ package openapi import ( "encoding/json" + "reflect" "strings" "testing" @@ -14,6 +15,8 @@ func TestSchemaAdditionalPropertiesAndExactNumbers(t *testing.T) { `{"type":"object","additionalProperties":true}`, `{"type":"object","additionalProperties":{"type":"string"}}`, `{"type":"integer","default":18446744073709551615,"enum":[18446744073709551615]}`, + `{"$ref":"#/components/schemas/Node"}`, + `{"description":"Node","allOf":[{"anyOf":[{"$ref":"#/components/schemas/Node"},{"type":"object","nullable":true,"enum":[null]}]}]}`, } { var s Schema if err := json.Unmarshal([]byte(text), &s); err != nil { @@ -30,6 +33,9 @@ func TestSchemaAdditionalPropertiesAndExactNumbers(t *testing.T) { if err = json.Unmarshal(b, &roundTrip); err != nil { t.Fatal(err) } + if !reflect.DeepEqual(s, roundTrip) { + t.Fatalf("schema round trip changed: %s", b) + } } api, _ := New("Exact numbers", "1") s := &Schema{Type: "integer", Enum: []any{json.Number("18446744073709551615")}} diff --git a/reports/recursiveResponseAcceptance.md b/reports/recursiveResponseAcceptance.md new file mode 100644 index 0000000..906ed00 --- /dev/null +++ b/reports/recursiveResponseAcceptance.md @@ -0,0 +1,26 @@ +# v0.28.2 递归响应验收 + +范围：`.api` 响应 DTO 的自引用与相互引用，使用通用 Node/Branch/Leaf；不加入接入项目的业务字段或业务行为。非递归响应保持原生成路径，递归请求与非法值循环仍拒绝。 + +## 本地回归 + +2026-09-30，macOS arm64： + +- `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -count=1 ./...`：56 个测试包通过，apigen 集成包 112.675 秒。 +- `GOTOOLCHAIN=go1.22.12 GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -count=1 ./...`：56 个测试包通过，apigen 集成包 91.536 秒。 +- 新增生成文档断言后，重新运行专项 `go test -race -count=1 ./cmd/go-omni/apigen -run Recursive`（开启集成）：通过，4.318 秒。 +- `go vet ./...`、`go build ./...`、`git diff --check`：通过。 +- 示例通过实际 CLI 导出后，`@apidevtools/swagger-cli@4.0.4 validate` 确认完整 OpenAPI 3.0.3 文档有效。 + +## 覆盖内容 + +- 强类型三层树、递归 slice/map/pointer、可空对象、raw、数组根响应以及相互引用的有限值，生成消费者真实 HTTP 返回和 Go JSON 解码。 +- `$ref` 目标完整、没有被忽略的引用同级属性、可空引用的 null 分支、循环引用不展开、文档压缩幂等。 +- 跨源文件引用、声明顺序无关、直接值与引用混合的合法类型、scalar 输出 schema、字段约束、描述、精确 int64 示例。 +- 有限递归示例、空集合、缺失/未知字段、深层非法值、异构数组错误、非法 null 与额外 JSON 值。 +- 非法直接值循环、相互值循环、字段组合循环、递归请求及间接递归请求、未知类型、非法标签、重复字段、上传字段、双指针、固定数组和错误 map 键。 +- 反复生成、手写代码保留、只读检查、递归字段变更诊断；请求绑定与原有生成消费者继续全量回归。 + +## 发布验收 + +远程验收使用 `scripts/verifyManagedRelease.py`；新增递归消费者与其他消费者一起用空模块缓存安装固定版本 CLI，并核对 module checksum 和无本地 replace。通用工程与混合工作区分别运行其原有发布验收脚本。结果在发布后记录。 diff --git a/scripts/verifyManagedRelease.py b/scripts/verifyManagedRelease.py index 9cf8d92..692d3fa 100644 --- a/scripts/verifyManagedRelease.py +++ b/scripts/verifyManagedRelease.py @@ -124,7 +124,7 @@ def main(): checks += ["sharedGoCatalogExport", "documentOnlyErrorSelection", "workspaceCheckNoWrites", "staleCatalogRejected", "twoIndependentServiceBuilds"] env.update(GO_OMNI_SCAFFOLD_INTEGRATION="1", GO_OMNI_RELEASE_CLI=str(cli), GO_OMNI_RELEASE_VERSION=args.version) - tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer", "TestCopyxThreeEndpointConsumer", "TestGeneratedConfigurationConsumer", "TestGeneratedCORSConsumer", "TestI18nGeneratedConsumer"] + tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer", "TestCopyxThreeEndpointConsumer", "TestGeneratedConfigurationConsumer", "TestGeneratedCORSConsumer", "TestI18nGeneratedConsumer", "TestRecursiveResponseConsumer"] print("Running generated consumers against the remote framework, including lifecycle failure cleanup", flush=True) output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=240s", "./cmd/go-omni/apigen", "-run", "^(" + "|".join(tests) + ")$"], ROOT, env) events = [json.loads(line) for line in output.splitlines() if line.startswith("{")] @@ -135,6 +135,7 @@ def main(): assert len(sums) == len(tests) and all(s == dependency["Sum"] for s in sums), "remote consumer checksum mismatch" checks += ["initializationAndListenFailureCleanup", "drainBeforeDependencyClose", "generatedConsumerRaceTests", "errorParamsAndInheritance", "manualHooksAndScalars", "dependencyFatalShutdown", "fatalProcessExitCode", "initializationShutdownNotification", "configuredBuiltinEndpoints", "projectHealthEnvelope", "endpointConflictCleanup", "responseArrayNullableNull", "primitiveAndDynamicArrays", "strictRequestPolicies", "declarativeMultipartBinding", "backgroundStopDrainResourceOrder", "backgroundInitAndListenFailureCleanup", "backgroundTimeoutRetainsResourcesAndExits"] checks += ["generatedIncompleteHTTPDrainRetainsDependenciesAndLogger"] + checks += ["recursiveResponseTypedHTTP", "recursiveResponseReferenceGraph", "recursiveResponseRegenerationAndDrift"] # Run tests from the downloaded module, not the framework checkout. http_output = run(["go", "test", "-race", "-count=1", "-json", MODULE + "/httpc"], starter, env)