diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index a7672b1..746b0d7 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,7 +1,9 @@ -# 通用业务开发指南(v0.16.0-preview) +# 通用业务开发指南(v0.17.0-preview) 从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。 +本版新增的公共字段组合、可选结构映射、自动中间件工厂和 Repository 开关,见 [生成能力指南](GENERATION.md)。 + ## 返回数组、可空对象和 null ```text @@ -145,40 +147,15 @@ func (c *ServiceContext) Init(ctx context.Context) error { if err != nil { return err } c.Client = client c.OnClose(client.Close) - c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(client) return nil } ``` 每个资源初始化成功就登记清理;后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求,再执行后台组件的 Stop/Drain 阶段,确认收尾后逆序关闭依赖,最后关闭日志。关闭幂等,单个清理函数 panic/返回错误不会跳过其他清理;初始化 panic 清理后继续传播。后台组件按 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭) 登记;无法确认收尾完成时保留共享依赖并异常退出,不能把取消等同于任务已经结束。 -中间件用业务所需的小接口接收依赖,避免反向导入 svc: +新项目使用中间件工厂:项目维护 `middleware.Dependencies` 小接口与 `New(deps) (httpx.Middleware, error)`,svc 提供对应访问器。生成的 runtime 在 Init 成功后自动调用工厂并注册,不再按名字替换 map;工厂失败会释放已经登记的依赖。示例与现有项目迁移步骤见 [中间件工厂](GENERATION.md#中间件工厂自动注册)。 -```go -// internal/middleware/accessFactory.go -package middleware - -import ( - "context" - "git.yuntuo.hk/k2safe/go-omni/httpx" -) - -type AccessChecker interface { - CheckAccess(context.Context, string) error -} -func NewRequireAccess(checker AccessChecker) httpx.Middleware { - return func(next httpx.Handler) httpx.Handler { - return func(c *httpx.Context) error { - if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil { - return err - } - return next(c) - } - } -} -``` - -`.api` 声明对应 middleware 后,保留生成的初始 hook,并另建上述工厂;在 Init 用工厂替换 registry 的同名项即可。该包只依赖接口和 HTTP 抽象,不导入 svc,也不把数据库藏到 context 里。 +v0.16 项目及 `omni.gen.json` 中选择 `middleware: handlers` 的项目继续使用原有 `Defaults()`,可以在 Init 设置 `c.Middlewares["RequireAccess"]`。更新时不会重写这部分业务代码;只有显式切换到 factories 后才使用新流程。 生成入口为每个 App 注入独立 Logger,不修改进程默认 Logger。业务可用 `c.Logger.InfoContext(...)`,或请求链中的 `logx.InfoContext(ctx,...)`。标准库/第三方需要 `*log.Logger` 时使用 `c.Logger.Standard(slog.LevelWarn)`;适配器借用原 Logger,不负责关闭。手写多 App 用 `omni.WithLogInstance(logger)` 显式注入。 @@ -301,7 +278,7 @@ go-omni workspace check --file omni.workspace.json ## v0.13 项目升级 -先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.16.0-preview(或显式引用当前源码)。 +先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.17.0-preview(或显式引用当前源码)。 ```sh go-omni api upgrade --dir . --dry-run diff --git a/GENERATION.md b/GENERATION.md new file mode 100644 index 0000000..99bfff6 --- /dev/null +++ b/GENERATION.md @@ -0,0 +1,212 @@ +# 公共字段、结构映射与生成选项 + +本页对应 `v0.17.0-preview`。CLI 与框架依赖固定到同一版本: + +```sh +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview +# 已有业务项目内执行: +go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview +``` + +私有仓库访问设置见 [安装说明](README.md#准备环境与安装)。升级不会覆盖业务文件;已有中间件保持 handlers 模式,切换工厂模式按本页迁移步骤完成。 + +## 一份项目生成配置 + +新项目根目录生成开发者维护的 `omni.gen.json`: + +```json +{ + "repository": "none", + "middleware": "factories" +} +``` + +该文件与 `.api` 一起提交。每次 `api generate`、`--check`、`--dry-run` 和 `workspace generate/check` 都会读取;`api validate/export/format` 只处理契约,不加载业务 Go 类型。 + +- `repository: none` 是默认值,不创建空 Repository 文件。`group` 按 `.api` 的 group 补齐 `internal/repository//repository.go`;文件始终属于业务。切回 none 后不再补骨架,已有文件不删除、不覆盖。实际 SQL、ORM 和仓储划分由项目决定。 +- `middleware: factories` 是新项目默认值,按下文自动初始化。`handlers` 保留原有 `Defaults()` 与 `func(next Handler) Handler` 接口,供既有项目逐步迁移。 +- `mappings` 可省略;只有明确列出的类型对才生成映射。未启用时不会运行 Go 类型加载命令,也不增加运行时依赖。 +- 升级首次遇到没有配置的旧 manifest 时,补一份 `middleware: handlers` 的配置,保留原有手写注册;Repository 默认 none。生成配置建立后不能直接删除,否则生成失败,避免误关闭映射或改变中间件模式。 + +## 公共字段组合 + +```text +type EmployeeFields { + // 员工姓名。 + Name string `json:"name" binding:"max=80"` + Salary Money `json:"salary"` +} + +type EnterpriseRequest { + EmployeeFields +} + +type AdminRequest { + EmployeeFields + CompanyID string `path:"companyId"` +} +``` + +`Money` 由项目按既有 scalar 语法声明。组合行只有已声明的对象类型名,不加指针或字段 tag。公共对象可以通过已有相对 import 或声明过的共享根 `@common/...` 引入。 + +生成器展开为普通 Go 字段。JSON 保持平铺;字段的绑定来源、校验、注释、可空性和 OpenAPI 使用同一份展开结果。不会在 JSON 中增加 `EmployeeFields` 一层。重复 Go 字段、重复来源字段名、循环组合、未知对象和过深展开均失败;即使重复字段类型相同,也不能隐式覆盖。要改变某个字段的规则,应拆出更小的公共组合。 + +允许多层组合,上限 32 层、单类型 4096 字段、全契约展开 65536 字段。Go 匿名指针嵌入并非本功能;生成 DTO 中不会使用匿名嵌入。 + +## 可选结构映射 + +`.api` 生成 DTO,业务独立维护输入/结果类型。项目明确选择需要自动赋值的类型对;不从数据库反向生成公开接口,不自动推断反向映射。 + +假设 `types.AdminRequest` 有 `Name string`、`Salary amount.Money`、`CompanyID string`,业务的 `domain.EmployeeInput` 有 `EmployeeName string` 和同类型 `Salary`: + +```json +{ + "repository": "none", + "middleware": "factories", + "mappings": [ + { + "name": "ToEmployeeInput", + "source": {"package": "example.com/payroll/internal/types", "type": "AdminRequest"}, + "target": {"package": "example.com/payroll/internal/domain", "type": "EmployeeInput"}, + "fields": { + "EmployeeName": {"source": "Name"} + }, + "ignoreSource": { + "CompanyID": "由 Logic 单独做企业范围授权,不属于请求体输入" + } + } + ] +} +``` + +执行 `go-omni api generate` 后,产生独立的生成文件 `internal/mapping/mappingsGen.go`: + +```go +func ToEmployeeInput(source types.AdminRequest) (domain.EmployeeInput, error) { + var result domain.EmployeeInput + result.EmployeeName = source.Name + result.Salary = source.Salary + return result, nil +} +``` + +包别名由生成器决定。业务调用 `mapping.ToEmployeeInput(*req)`;反向响应映射按另一条 source/target 规则声明。两个类型完全相同时通常无需映射。 + +### 类型与覆盖检查 + +自动赋值要求同名且 **Go 类型一致**(类型别名按 Go 类型系统处理),字段名使用 Go 名称,不按 JSON 标签猜对应关系。int32 → int64、`*Money` → Money、数值 ID → string 都必须显式转换。不会暗中补零、解引用或执行数字强转。 + +每个源字段必须被使用或出现在 `ignoreSource`;每个目标字段必须被赋值或出现在 `ignoreTarget`。忽略规则必须给出非空原因,目标忽略明确意味着保留零值;规则引用不存在的字段也会失败。新字段若无法自动匹配,会在生成阶段报错。两边同时添加的同名同类型字段会自动加入生成代码,`--check` 检出未重新生成的差异。 + +根类型是项目可导入的具名结构;支持值类型匿名嵌入,规则可用 `Employee.EmployeeName` 这样的实际 Go 字段路径。同名提升歧义、未导出字段和匿名指针嵌入拒绝处理。普通具名嵌套字段使用独立映射或转换函数,不猜测内部结构。泛型声明不能直接作为根;具体字段类型按 Go 类型系统检查。 + +配置变化、遗漏映射、类型错误会在写入前失败。映射生成与其他生成产物使用同一套文件所有权机制;手改映射生成文件会被拒绝覆盖。 + +### 显式转换函数 + +```json +"fields": { + "Salary": { + "source": "Salary", + "convert": {"package": "example.com/payroll/internal/convert", "name": "SalaryValue"} + } +} +``` + +支持这两种精确签名: + +```go +func SalaryValue(source *amount.Money) amount.Money +func SalaryValue(source *amount.Money) (amount.Money, error) +``` + +项目选择一种实现。nil、默认值、金额精度、日期格式和错误码由函数负责;若两边已经是同一个 Money 类型,无需再次解析或序列化。转换函数应仅负责值转换,不执行 SQL 或改变授权范围。 + +错误通过 `%w` 加上映射名/目标字段路径,保留 `errors.Is/As` 和框架错误链;返回零目标值,不交付部分完成的结果。包装不主动添加源值或转换后的值。错误是否公开仍由项目错误目录和响应策略决定。 + +### 嵌套对象、数组与引用 + +`mapping` 指向同配置中的另一条映射,`elements` 明确要求逐项映射: + +```json +"fields": { + "Summary": {"source": "Summary", "mapping": "ToSummary"}, + "Items": {"source": "Items", "mapping": "ToItem", "elements": true} +} +``` + +`mapping` 与 `convert` 互斥。`elements` 可以配合其中任意一种,也可以对类型相同的元素直接赋值;支持 slice→slice、等长 array→array,禁止隐式互转。nil slice 保持 nil,非 nil 空 slice 保持非 nil,逐项失败的错误包含索引,循环映射声明被拒绝。 + +未选择 elements 的同类型指针、map、slice 使用普通 Go 赋值,**共享引用**,不做隐式深拷贝。elements 会分配新的目标 slice,但元素内的引用仍遵循其映射规则。需要深拷贝时显式提供转换函数。 + +映射不保存原 JSON 的字段存在信息。普通 `*T` 在绑定后可能已把缺失和 null 合并为 nil;无法在转换阶段恢复区分。有三态需要的项目必须先使用能携带该信息的输入类型及绑定扩展;没有声明三态时,保持当前输入类型的语义。完整表单更新和局部更新由业务决定,框架不把 nil 自动解释成“保留旧值”。 + +### 生成顺序与 CI + +1. 先从 `.api` 生成项目,添加业务结构、scalar 和转换函数,执行 `go mod tidy`。 +2. 在 `omni.gen.json` 列出映射,执行 `go-omni api generate`。 +3. Logic 调用生成函数。后续改字段后重复生成,不修改 `*Gen.go`。 +4. CI 同时运行 `go-omni api generate --check` 与 `go test ./...`。 + +启用映射后,CLI 通过 `go list -deps -export` 加载明确声明的包与依赖,按当前构建环境做类型检查,不执行业务代码、init 或转换函数。采用待生成 DTO 的临时 overlay,避免“新增字段还看旧 DTO”。go.mod/go.sum 的副本及 overlay 都在临时目录,check 不修改项目文件;Go 自己的构建/模块缓存可能更新。使用当前 GOOS/GOARCH/GOFLAGS、`GOWORK=off`;依赖用 go.mod 与 replace 表达,生成超时为 90 秒。 + +类型包/转换函数包不能依赖输出包 `internal/mapping`;把转换函数放在独立包中,避免导入环。若包本身无法编译,先运行 `go list -deps -export <相关包>` 排查;CLI 不回显可能含私有仓库凭据的 Go 下载输出。纯值字段转换只生成标准 Go 代码,不引入运行时反射搬运、JSON 往返、ORM 或队列依赖。 + +## 中间件工厂自动注册 + +`.api` 沿用 `middleware: RequireAccess` 或 auth 对应的中间件声明。新项目会生成可编辑的 `internal/middleware/requireAccessFactory.go`,只需实现一次: + +```go +// internal/middleware/dependencies.go,由项目维护。 +type AccessChecker interface { + CheckAccess(context.Context, string) error +} +type Dependencies interface { + AccessChecker() AccessChecker +} + +// internal/middleware/requireAccessFactory.go,由项目维护。 +func NewRequireAccess(deps Dependencies) (httpx.Middleware, error) { + checker := deps.AccessChecker() + if checker == nil { return nil, errors.New("access checker is required") } + return func(next httpx.Handler) httpx.Handler { + return func(c *httpx.Context) error { + if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil { + return err + } + return next(c) + } + }, nil +} + +// internal/svc/access.go,由项目维护;ServiceContext.Init 初始化 c.Auth。 +func (c *ServiceContext) AccessChecker() middleware.AccessChecker { return c.Auth } +``` + +编译器检查 ServiceContext 是否实现依赖接口,middleware 不导入 svc。多个工厂可以共用这些小接口。所需 imports 和具体鉴权服务仍属于项目。 + +启动顺序为:框架基础依赖 → `svc.Init` → 生成的 `middleware.New(c)` 调用各具名工厂 → 创建 App/注册路由 → 监听。中间件实例创建一次,由请求并发复用;不要把单次请求信息存在实例字段里。工厂之间没有初始化顺序依赖;路由执行顺序仍以 `.api` 声明为准。 + +工厂返回错误或 nil 中间件均阻止监听;错误链保留,已登记资源按现有生命周期清理。工厂 panic 也先清理依赖再传播。资源初始化和清理登记放在 svc.Init,工厂借用资源;HTTP 请求排空之前不会关闭这些依赖。 + +factory 模式不再在 Init 写 `Middlewares["..."]`;残留非空手写注册表会明确报错,防止生成注册静默覆盖它。未实现的工厂让启动失败;未实现的 Logic 仍返回 501。 + +### 既有项目迁移 + +1. 首次更新保持 `middleware: handlers`,现有 hook 和手写 Configure 不变。 +2. 改为 factories 并重新生成,会补 `dependencies.go` 和 `*Factory.go`,保留原来的手写文件。 +3. 将旧实现迁入工厂,提供 Dependencies 的访问器;移除 Init 中按名字覆盖 map 的代码。 +4. 旧函数不再需要时由项目删除。若新工厂符号与旧文件中的同名函数冲突,整理旧文件;生成器不覆盖业务代码。 +5. `generate --check` 与 `go test ./...` 检查后再运行。工厂没有全部实现前,服务不会启动。 + +## 文件维护边界 + +| 项目维护 | 生成器持续维护 | +| --- | --- | +| `.api` 公共字段与接口 | 展开后的 DTO、Handler、OpenAPI | +| `omni.gen.json` 的映射与选项 | `internal/mapping/mappingsGen.go` | +| Go 业务类型、scalar、转换函数 | 编译期可检查的赋值与调用代码 | +| middleware Dependencies、具名工厂、svc 访问器 | 中间件 registry、启动调用 | +| Logic、按需 Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 | + +回归样例位于 `cmd/go-omni/apigen/testdata/composition`,为独立三端服务,不导入任何 eTax 代码。 diff --git a/PROJECT.md b/PROJECT.md index 2b287e2..16abc34 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -1,6 +1,6 @@ # 生成项目与文件维护边界 -v0.16.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 +v0.17.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 ## 创建和日常开发 @@ -38,6 +38,8 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name): 这是轻量源码分析,不运行项目 init、不执行 Go 命令、不下载依赖。项目自定义类型别名、嵌入方法或 build tags 等无法可靠判断时,会给 REVIEW 并交给 Go 编译器验证;不把这些情况强行判为签名错误。它不检查方法体是否引用已删除字段,也不证明业务逻辑正确。CI 中应同时执行 `go-omni api generate --check` 和 `go test ./...`,按项目实际构建参数测试。当前版本首次接手缺少快照的 manifest v2 时,会明确提示建立基线,不猜测旧契约;Logic 签名检查仍会执行。 +本版新增生成选项、公共字段组合和映射,详见 [生成能力指南](GENERATION.md)。 + ## 哪些文件由用户维护 | 文件 | 首次生成后由项目维护 | @@ -50,15 +52,16 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name): | `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 | | `internal/svc/storage.go` | 选用存储时生成,之后保留 | | `internal/logic//*.go` | 业务方法,使用标准 context.Context | -| `internal/repository//*.go` | 数据访问,按需实现 | -| `internal/middleware/.go` | 项目鉴权/权限等具名中间件实现 | +| `internal/repository//*.go` | 数据访问;默认不补空骨架,可选 group 模式 | +| `internal/middleware/*Factory.go`、`dependencies.go` | 中间件工厂与依赖小接口,svc 实现访问器 | +| `omni.gen.json`、映射转换函数 | 生成选项、显式类型对与差异规则 | | 项目 Go 错误包 | 自行维护 Catalog 和常量,可跨服务共用 | | 自定义标量文件、`*Bind.go`、`*Manual.go` | 只有显式选择相应扩展时才生成/使用 | | 项目测试、迁移、部署脚本、go.mod/go.sum | 项目所有,生成器不重置其内容 | 以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。 -生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。 +生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、可选映射 `internal/mapping/mappingsGen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。 Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。 @@ -208,7 +211,7 @@ get /download (DownloadReq) ## 已有服务接入步骤 -每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.16.0-preview: +每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.17.0-preview: 1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。 2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。 diff --git a/README.md b/README.md index dbed247..f97df39 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,9 @@ GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。 -**从这份 README 开始接入。** 当前版本 **`v0.16.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。 +**从这份 README 开始接入。** 当前版本 **`v0.17.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。 + +**本版新增**:公共字段组合、可选结构映射、带依赖的中间件工厂、按需 Repository 骨架,使用与迁移见 [生成能力指南](GENERATION.md)。 Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。 @@ -36,7 +38,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可 ```sh export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk" -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.16.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview export PATH="$(go env GOPATH)/bin:$PATH" go-omni --help ``` @@ -77,21 +79,22 @@ hello/ ├── api/service.api # 维护接口定义,也可 import 其他 .api ├── main.go # 生成器维护:启动、路由注册、优雅退出 ├── etc/config.yaml # 项目配置 +├── omni.gen.json # 项目维护的生成选项与可选映射 ├── internal/ │ ├── config/ # 配置读取 │ ├── svc/ # 数据库、日志、外部客户端等依赖 │ ├── types/*Gen.go # 按 .api 文件生成 DTO │ ├── handler/ # 生成的绑定、响应策略、路由及文档 │ ├── logic/greet/ # 在这里实现业务逻辑 -│ ├── repository/greet/ # 在这里实现数据访问 -│ └── middleware/ # 声明鉴权等中间件后生成对应 hook +│ ├── repository/ # 按需创建,默认不生成空骨架 +│ └── middleware/ # 声明鉴权等中间件后生成工厂 ├── docs/openapi.json # 生成的 OpenAPI 3.0.3 └── .goOmni/apiManifest.json # 生成文件所有权与校验信息,一起提交 Git ``` 修改 `api/service.api` 后重新生成,业务写在 `internal/logic`,数据访问写在 `internal/repository`,依赖在 `internal/svc` 装配。声明业务错误后还会生成 `internal/apierrors/errorsGen.go`。 -`api new` 的问候接口可以直接运行;自定义接口新生成的 Logic 和鉴权 hook 返回 **501**,需要实现后才可用。生成器不会根据 HTTP 字段猜数据库表或业务规则。 +`api new` 的问候接口可以直接运行;自定义接口未实现的 Logic 返回 **501**。新项目的具名中间件工厂未实现时直接阻止启动;v0.16 的原有 handler 模式仍返回 501。生成器不会根据 HTTP 字段猜数据库表或业务规则。 ### 从自己的接口定义生成 @@ -192,7 +195,7 @@ go-omni new --api catalog.api \ 在业务项目的 **go.mod 所在目录**执行: ```sh -go get git.yuntuo.hk/k2safe/go-omni@v0.16.0-preview +go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview ``` 已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。 @@ -316,7 +319,7 @@ auth UserAuth ( ) ``` -生成器会创建 `internal/middleware/auth.go`,由项目实现凭据校验和身份注入。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的 hook 返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。 +新项目生成 `internal/middleware/authFactory.go`,由项目实现 `NewAuth(deps)`;生成器负责初始化和注册。v0.16 或 handler 模式继续使用 `internal/middleware/auth.go`。凭据校验和身份注入由项目实现。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的工厂阻止启动,原有 hook 模式返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。 业务错误可以集中定义,业务只选择发生了哪个错误。**HTTP 状态由项目配置,业务失败可以返回 HTTP 200**;框架不根据 HTTP 状态单独判断成功与否。 @@ -460,9 +463,9 @@ go-omni api generate --check go test ./... ``` -`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。 +`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。 -CLI 与业务 module 固定使用 `v0.16.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 +CLI 与业务 module 固定使用 `v0.17.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 ## 按需深入 diff --git a/RELEASE.md b/RELEASE.md index 3dbf0bc..df0cec1 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,4 +1,27 @@ -# v0.16.0-preview +# v0.17.0-preview + +减少项目里的重复声明、字段赋值和中间件装配代码,业务规则继续由项目控制: + +- `.api` 公共字段组合:保持 JSON 平铺,DTO、绑定、校验和 OpenAPI 同步;字段冲突、循环和超限明确报错。 +- 可选结构映射:在 `omni.gen.json` 显式声明源与目标,生成编译期检查的普通 Go 赋值。不同字段名、金额、时间和指针转换由项目指定函数;遗漏字段与类型错误在写文件前失败。不使用 JSON 往返或运行时反射。 +- 带依赖的中间件工厂:svc 初始化依赖后自动调用 `New(deps)` 并注册;错误、nil 或 panic 阻止启动,已登记资源按生命周期清理。 +- Repository 骨架默认关闭,可按 group 启用;已有业务文件始终保留。生成选项与映射纳入重新生成和 `--check`。 + +```sh +go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview +go-omni api generate +go-omni api generate --check +go test ./... +``` + +新项目默认使用 factories;已有项目首次升级保留 handlers 模式,需要显式迁移后再切换。映射按需开启,先准备业务类型、转换函数和依赖,再声明映射;普通引用赋值保留 Go 语义,缺失/null 语义由项目类型与转换函数定义。 + +使用方法、文件归属和升级步骤见 [生成能力指南](GENERATION.md),回归与发布验收见 [验收记录](reports/generationExperienceAcceptance.md)。本版仍为 preview;框架验收不等于业务项目完成迁移。 + +--- + +# v0.16.0-preview(历史版本) 补齐通用契约与生成体验,业务响应协议继续由项目定义: diff --git a/cmd/go-omni/README.md b/cmd/go-omni/README.md index b6d81fb..d784159 100644 --- a/cmd/go-omni/README.md +++ b/cmd/go-omni/README.md @@ -1,5 +1,7 @@ # go-omni CLI +v0.17 增加公共字段组合、可选结构映射、自动中间件工厂和按需 Repository。配置和迁移见 [生成能力指南](../../GENERATION.md)。 + v0.16 增加基础类型/动态 JSON 数组返回、OpenAPI 公共引用和 workspace Go Catalog 统一导出。见 [接入与筛选规则](../../DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。 v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目录>|--apply]`,以及 `workspace format|generate|check --file omni.workspace.json`。详见 [通用开发指南](../../DEVELOPMENT.md)。 @@ -10,14 +12,14 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目 ```sh # 仓库读取权限与 PATH 配置见根目录 README -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.16.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview go-omni new --module example.com/hello ./hello # 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在 go-omni new --api service.api --module example.com/example ./example ``` -`new` 与 `api new` 统一生成契约项目;不再提供另一套 CRUD 启动目录。未实现的 Logic / 中间件骨架返回 501。进入项目执行 `go mod tidy`、`go run .`、`go build .`。 +`new` 与 `api new` 统一生成契约项目;不再提供另一套 CRUD 启动目录。未实现的 Logic 返回 501;中间件工厂未实现时阻止启动,既有 handler 模式仍返回 501。进入项目执行 `go mod tidy`、`go run .`、`go build .`。 参数:`--database none|memory|mysql`(默认 none;memory 等同 none),`--storage none|local|s3|r2|oss`,`--log-output stdout|stderr|file`,`--log-format json|text`。基础设施按需生成,连接凭据由环境提供,不自动创建数据库、表或云资源。选择 MySQL 不会凭空生成业务表或 Repository 查询。 @@ -45,14 +47,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节 ## 版本与迁移 -CLI 与框架依赖固定使用 v0.16.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换: +CLI 与框架依赖固定使用 v0.17.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换: ```sh # 在 go-omni 源码仓库中执行 go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello ``` -模板最低需要 v0.16.0-preview 的统一环境变量别名能力,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。 +当前模板与依赖以 v0.17.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。 ## 数据库迁移命令 diff --git a/cmd/go-omni/apigen/apigen_test.go b/cmd/go-omni/apigen/apigen_test.go index 8825294..49e650d 100644 --- a/cmd/go-omni/apigen/apigen_test.go +++ b/cmd/go-omni/apigen/apigen_test.go @@ -170,6 +170,7 @@ func TestRegenerationAndConflicts(t *testing.T) { servicePath := filepath.Join(dir, "internal/logic/greet/greet.go") service := readFile(t, servicePath) + "\n// user edit must survive\n" writeFile(t, servicePath, service) + writeFile(t, filepath.Join(dir, "internal/repository/greet/repository.go"), "package greet\n") for _, file := range []string{"etc/config.yaml", "internal/svc/serviceContext.go", "internal/repository/greet/repository.go"} { body := readFile(t, filepath.Join(dir, file)) if strings.HasSuffix(file, ".go") { @@ -301,6 +302,7 @@ func TestGeneratedProjectsBuildAndRun(t *testing.T) { if starter { _, err = New(o) } else { + writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`) o.API = filepath.Join(root, "examples/contracts/user.api") _, err = Generate(o) } diff --git a/cmd/go-omni/apigen/business_test.go b/cmd/go-omni/apigen/business_test.go index 0008727..04bdc5b 100644 --- a/cmd/go-omni/apigen/business_test.go +++ b/cmd/go-omni/apigen/business_test.go @@ -40,6 +40,7 @@ func TestBusinessGeneratedProject(t *testing.T) { t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1") } dir := filepath.Join(t.TempDir(), "business") + writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`) o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/business.api"), Module: "example.com/business", FrameworkDir: frameworkRoot(t)} if err := generateManagedTestProject(t, o); err != nil { t.Fatal(err) diff --git a/cmd/go-omni/apigen/compile.go b/cmd/go-omni/apigen/compile.go index 8130112..02aea00 100644 --- a/cmd/go-omni/apigen/compile.go +++ b/cmd/go-omni/apigen/compile.go @@ -40,6 +40,9 @@ func Compile(spec *Spec) (*Compiled, error) { if prepareErr != nil { return nil, prepareErr } + if err := expandFields(spec); err != nil { + return nil, err + } if err := validateDocumentCatalog(spec); err != nil { return nil, err } diff --git a/cmd/go-omni/apigen/composition.go b/cmd/go-omni/apigen/composition.go new file mode 100644 index 0000000..b048fb3 --- /dev/null +++ b/cmd/go-omni/apigen/composition.go @@ -0,0 +1,102 @@ +package apigen + +import ( + "fmt" + "reflect" + "strings" +) + +// expandFields copies definitions rather than embedding Go structs. Binding, +// validation, DTOs, snapshots and OpenAPI all consume the same flattened fields. +func expandFields(spec *Spec) error { + definitions := map[string]Type{} + for _, t := range spec.Types { + if _, exists := definitions[t.Name]; exists { + return fmt.Errorf("%s: duplicate type %s", t.Position, t.Name) + } + definitions[t.Name] = t + } + expanded := map[string]Type{} + heights := map[string]int{} + visiting := map[string]bool{} + total := 0 + var expand func(string, int) (Type, error) + expand = func(name string, depth int) (Type, error) { + if t, ok := expanded[name]; ok { + if depth+heights[name] > 32 { + return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name) + } + return t, nil + } + if visiting[name] { + return Type{}, fmt.Errorf("field composition cycle at %s", name) + } + if depth > 32 { + return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name) + } + t, ok := definitions[name] + if !ok { + return Type{}, fmt.Errorf("unknown composed object type %s", name) + } + visiting[name] = true + defer delete(visiting, name) + fields := []Field{} + height := 0 + seen, wire := map[string]bool{}, map[string]bool{} + for _, f := range t.Fields { + add := []Field{f} + if f.Type == "" { + child, err := expand(f.Name, depth+1) + if err != nil { + return Type{}, fmt.Errorf("%s.%s: %w", name, f.Name, err) + } + add = child.Fields + if heights[f.Name]+1 > height { + height = heights[f.Name] + 1 + } + if height > 32 { + return Type{}, fmt.Errorf("field composition exceeds 32 levels at %s", name) + } + } + for _, field := range add { + if seen[field.Name] { + return Type{}, fmt.Errorf("%s: conflicting composed field %s", name, field.Name) + } + seen[field.Name] = true + for _, source := range []string{"json", "form", "path", "header", "multipart", "multipartJSON", "file"} { + tag := strings.Split(reflect.StructTag(field.Tag).Get(source), ",")[0] + if tag == "" || tag == "-" { + continue + } + if source == "header" { + tag = strings.ToLower(tag) + } + key := source + ":" + tag + if wire[key] { + return Type{}, fmt.Errorf("%s: conflicting composed field tag %s", name, key) + } + wire[key] = true + } + fields = append(fields, field) + total++ + if len(fields) > 4096 || total > 65536 { + return Type{}, fmt.Errorf("field composition expansion limit exceeded") + } + } + } + t.Fields = fields + expanded[name] = t + heights[name] = height + return t, nil + } + result := make([]Type, 0, len(spec.Types)) + for _, t := range spec.Types { + out, err := expand(t.Name, 0) + if err != nil { + return err + } + result = append(result, out) + } + spec.Types = result + return nil +} diff --git a/cmd/go-omni/apigen/composition_test.go b/cmd/go-omni/apigen/composition_test.go new file mode 100644 index 0000000..bf9ec3e --- /dev/null +++ b/cmd/go-omni/apigen/composition_test.go @@ -0,0 +1,163 @@ +package apigen + +import ( + "bytes" + "encoding/json" + "fmt" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestFieldComposition(t *testing.T) { + source := readFile(t, "testdata/composition/service.api") + spec, err := parseText(t, source) + if err != nil { + t.Fatal(err) + } + first, err := Compile(spec) + if err != nil { + t.Fatal(err) + } + second, err := Compile(spec) + if err != nil { + t.Fatal(err) + } + if !bytes.Equal(first.Document, second.Document) || len(spec.Types[1].Fields) != 1 { + t.Fatal("composition mutates input") + } + for _, name := range []string{"EnterpriseRequest", "TenantRequest", "AdminRequest"} { + schema := first.Bindings[name].BodySchema() + raw, _ := json.Marshal(schema) + if !bytes.Contains(raw, []byte("Employee display name.")) || bytes.Contains(raw, []byte("SalaryFields")) { + t.Fatalf("flattened schema/comment missing: %s", raw) + } + } + cases := []struct{ name, decl, want string }{ + {"unknown", "type Req {\n Missing\n}", "unknown composed"}, + {"cycle", "type Req {\n Other\n}\ntype Other {\n Req\n}", "cycle"}, + {"conflict", "type Base {\n Name string `json:\"name\"`\n}\ntype Req {\n Base\n Name string `json:\"other\"`\n}", "conflicting"}, + {"wireConflict", "type Base {\n Name string `json:\"name\"`\n}\ntype Req {\n Base\n Other string `json:\"name\"`\n}", "conflicting"}, + {"tagged", "type Req {\n Other `json:\"other\"`\n}", "cannot carry"}, + {"diamond", "type Base {\n Name string `json:\"name\"`\n}\ntype Left {\n Base\n}\ntype Right {\n Base\n}\ntype Req {\n Left\n Right\n}", "conflicting"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + s, e := parseText(t, tc.decl+"\nservice demo{\n@handler post\npost / (Req)\n}") + if e == nil { + _, e = Compile(s) + } + if e == nil || !strings.Contains(e.Error(), tc.want) { + t.Fatal(e) + } + }) + } + // Imports can contain reusable definitions and retain the existing root policy. + dir := t.TempDir() + writeFile(t, filepath.Join(dir, "base.api"), "type Base {\n Name string `json:\"name\"`\n}\n") + writeFile(t, filepath.Join(dir, "service.api"), "import \"base.api\"\ntype Req {\n Base\n}\nservice demo{\n@handler post\npost / (Req)\n}") + if _, err := FormatProject(filepath.Join(dir, "service.api"), false); err != nil { + t.Fatal(err) + } + if _, err := FormatProject(filepath.Join(dir, "service.api"), true); err != nil { + t.Fatal(err) + } +} + +func TestRepositoryGenerationOptions(t *testing.T) { + dir := filepath.Join(t.TempDir(), "app") + if _, err := New(Options{Dir: dir, Module: "example.com/repository", FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Join(dir, "internal/repository")); !os.IsNotExist(err) { + t.Fatal("default creates repository") + } + o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")} + writeFile(t, filepath.Join(dir, generatorConfigFile), `{"repository":"group","middleware":"factories"}`) + if _, err := Generate(o); err != nil { + t.Fatal(err) + } + repo := filepath.Join(dir, "internal/repository/greet/repository.go") + original := readFile(t, repo) + "\n// business SQL belongs here\n" + writeFile(t, repo, original) + writeFile(t, filepath.Join(dir, generatorConfigFile), `{"repository":"none","middleware":"factories"}`) + if _, err := Generate(o); err != nil { + t.Fatal(err) + } + o.Check = true + if _, err := Generate(o); err != nil { + t.Fatal(err) + } + if readFile(t, repo) != original { + t.Fatal("repository removed or overwritten") + } + for _, raw := range []string{`{"repository":"oops"}`, `{"middleware":"oops"}`, `{"unknown":true}`, `{"repository":"none","repository":"group"}`} { + writeFile(t, filepath.Join(dir, generatorConfigFile), raw) + before := snapshot(t, dir) + if _, err := Generate(o); err == nil { + t.Fatal("accepted", raw) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("invalid config wrote files") + } + } +} + +func TestCompositionExpansionBound(t *testing.T) { + var source strings.Builder + source.WriteString("type Base {\n Value string `json:\"value\"`\n}\n") + last := "Base" + for i := 0; i < 35; i++ { + name := fmt.Sprintf("Level%d", i) + fmt.Fprintf(&source, "type %s {\n %s\n}\n", name, last) + last = name + } + // Reverse order forces traversal instead of hitting pre-expanded definitions. + spec, err := parseText(t, source.String()+"service demo{\n@handler post\npost / ("+last+")\n}") + if err != nil { + t.Fatal(err) + } + for i, j := 0, len(spec.Types)-1; i < j; i, j = i+1, j-1 { + spec.Types[i], spec.Types[j] = spec.Types[j], spec.Types[i] + } + if _, err = Compile(spec); err == nil { + t.Fatal("unbounded composition accepted") + } +} + +func TestGenerationConfigMissingAndLegacy(t *testing.T) { + dir := filepath.Join(t.TempDir(), "app") + if _, err := New(Options{Dir: dir, Module: "example.com/config", FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) + } + o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")} + if err := os.Remove(filepath.Join(dir, generatorConfigFile)); err != nil { + t.Fatal(err) + } + before := snapshot(t, dir) + if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "restore") { + t.Fatal(err) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("missing config changed project") + } + // A genuine old manifest has never tracked a generator config. + var old manifest + if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &old); err != nil { + t.Fatal(err) + } + old.GeneratorConfig = false + raw, _ := json.Marshal(old) + writeFile(t, filepath.Join(dir, manifestPath), string(raw)) + if _, err := Generate(o); err != nil { + t.Fatal(err) + } + cfg, exists, err := readGeneratorConfig(dir) + if err != nil || !exists || cfg.Middleware != "handlers" || cfg.Repository != "none" { + t.Fatal(cfg, exists, err) + } + if !strings.Contains(readFile(t, filepath.Join(dir, "internal/svc/runtimeGen.go")), "middleware.Defaults()") { + t.Fatal("legacy registration replaced") + } +} diff --git a/cmd/go-omni/apigen/generate.go b/cmd/go-omni/apigen/generate.go index 42e319b..de4af16 100644 --- a/cmd/go-omni/apigen/generate.go +++ b/cmd/go-omni/apigen/generate.go @@ -19,10 +19,11 @@ import ( ) const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" -const minimumVersion = "v0.16.0-preview" +const minimumVersion = "v0.17.0-preview" const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n" type Options struct { + generation GeneratorConfig lockHeld bool documentCatalogs map[string][]byte Database, Storage string @@ -97,6 +98,11 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { return Output{}, err } o.Dir = directory + genConfig, hasGenConfig, err := readGeneratorConfig(directory) + if err != nil { + return Output{}, err + } + o.generation = genConfig if err = checkRuntimeExtension(directory); err != nil { return Output{}, err } @@ -192,6 +198,10 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { for name, data := range base { seeds[name] = data } + if !hasGenConfig { + raw, _ := json.MarshalIndent(genConfig, "", " ") + seeds[generatorConfigFile] = append(raw, '\n') + } // The API source remains developer-owned. A relocated project regenerates // from its own api/ tree, including all imports. entry := "api/" + c.Spec.Entry @@ -242,6 +252,18 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { } } } + if err := renderMappings(o, generated, seeds); err != nil { + return Output{}, err + } + currentConfig, stillExists, err := readGeneratorConfig(directory) + if err != nil { + return Output{}, err + } + beforeConfig, _ := json.Marshal(genConfig) + afterConfig, _ := json.Marshal(currentConfig) + if hasGenConfig != stillExists || string(beforeConfig) != string(afterConfig) { + return Output{}, errors.New("omni.gen.json changed during generation; retry") + } out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld) expectedVersion := minimumVersion if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) { @@ -602,7 +624,9 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map } put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text) put(seeds, "internal/logic/"+group+"/logic.go", fmt.Sprintf("package %s\nimport %q\ntype Logic struct {deps *svc.ServiceContext}\nfunc New(deps *svc.ServiceContext)*Logic{return &Logic{deps:deps}}\n", packageName, o.Module+"/internal/svc")) - put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n") + if o.generation.Repository == "group" { + put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n") + } } put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n") registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n" @@ -611,6 +635,9 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m))) } put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n") + if o.generation.Middleware == "factories" { + renderMiddlewareFactories(middlewareNames, generated, seeds) + } for name, source := range baseTemplates { content := strings.ReplaceAll(source, "MODULE", o.Module) content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat) @@ -630,6 +657,16 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map return nil, nil, err } for name, data := range infra { + if o.generation.Middleware == "factories" && name == "internal/svc/runtimeGen.go" { + formatted, err := format.Source(data) + if err != nil { + return nil, nil, err + } + source := string(formatted) + source = strings.Replace(source, "Middlewares: middleware.Defaults(),", "", 1) + source = strings.Replace(source, "return c, nil", "if len(c.Middlewares) != 0 { return nil, errors.New(\"factory middleware is registered automatically; remove manual Middlewares assignments from Init\") }\nc.Middlewares, err = middleware.New(c)\nif err != nil { return nil, err }\nreturn c, nil", 1) + data = []byte(source) + } if strings.HasSuffix(name, "Gen.go") { generated[name] = append([]byte(generatedHeader), data...) } else { diff --git a/cmd/go-omni/apigen/generatorConfig.go b/cmd/go-omni/apigen/generatorConfig.go new file mode 100644 index 0000000..25b1066 --- /dev/null +++ b/cmd/go-omni/apigen/generatorConfig.go @@ -0,0 +1,69 @@ +package apigen + +import ( + "encoding/json" + "errors" + "fmt" + "os" + "path/filepath" + + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" +) + +const generatorConfigFile = "omni.gen.json" + +// GeneratorConfig is developer-owned and checked on every generation. Mapping +// rules never change the public API or infer a persistence model. +type GeneratorConfig struct { + Repository string `json:"repository,omitempty"` + Middleware string `json:"middleware,omitempty"` + Mappings []Mapping `json:"mappings,omitempty"` +} + +func readGeneratorConfig(root string) (GeneratorConfig, bool, error) { + cfg := GeneratorConfig{Repository: "none", Middleware: "factories"} + if err := safePath(root, generatorConfigFile); err != nil { + return cfg, false, err + } + stat, err := os.Lstat(filepath.Join(root, generatorConfigFile)) + if errors.Is(err, os.ErrNotExist) { + // Existing projects keep their handwritten middleware signature until they + // explicitly select factories; do not rewrite or adopt developer files. + if raw, e := os.ReadFile(filepath.Join(root, manifestPath)); e == nil { + var previous manifest + if err := json.Unmarshal(raw, &previous); err != nil { + return cfg, false, err + } + if previous.GeneratorConfig { + return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options and mappings before regenerating") + } + cfg.Middleware = "handlers" + } else if !errors.Is(e, os.ErrNotExist) { + return cfg, false, e + } + return cfg, false, nil + } + if err != nil { + return cfg, false, err + } + if !stat.Mode().IsRegular() || stat.Size() > 1<<20 { + return cfg, false, errors.New("omni.gen.json must be a regular file of at most 1 MiB") + } + raw, err := os.ReadFile(filepath.Join(root, generatorConfigFile)) + if err != nil { + return cfg, false, err + } + if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil { + return cfg, false, fmt.Errorf("omni.gen.json: %w", err) + } + if cfg.Repository != "none" && cfg.Repository != "group" { + return cfg, false, errors.New("omni.gen.json: repository must be none or group") + } + if cfg.Middleware != "factories" && cfg.Middleware != "handlers" { + return cfg, false, errors.New("omni.gen.json: middleware must be factories or handlers") + } + if len(cfg.Mappings) > 128 { + return cfg, false, errors.New("omni.gen.json: at most 128 mappings supported") + } + return cfg, true, nil +} diff --git a/cmd/go-omni/apigen/managed_test.go b/cmd/go-omni/apigen/managed_test.go index 70bd378..57e7612 100644 --- a/cmd/go-omni/apigen/managed_test.go +++ b/cmd/go-omni/apigen/managed_test.go @@ -170,7 +170,7 @@ func generateManagedTestProject(t *testing.T, o Options) error { _, err := Generate(o) return err } - ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) defer cancel() args := []string{"api", "generate", "--api", o.API, "--dir", o.Dir, "--module", o.Module} if o.Check { diff --git a/cmd/go-omni/apigen/mapping.go b/cmd/go-omni/apigen/mapping.go new file mode 100644 index 0000000..45d5a91 --- /dev/null +++ b/cmd/go-omni/apigen/mapping.go @@ -0,0 +1,433 @@ +package apigen + +import ( + "fmt" + "go/ast" + "go/format" + goparser "go/parser" + gotoken "go/token" + "go/types" + "sort" + "strconv" + "strings" + + "golang.org/x/mod/module" +) + +// GoTypeRef names an existing, exported, non-generic Go struct. +type GoTypeRef struct { + Package string `json:"package"` + Type string `json:"type"` +} +type GoFunctionRef struct { + Package string `json:"package"` + Name string `json:"name"` +} +type MappingField struct { + Source string `json:"source"` + Convert *GoFunctionRef `json:"convert,omitempty"` + Mapping string `json:"mapping,omitempty"` + Elements bool `json:"elements,omitempty"` +} +type Mapping struct { + Name string `json:"name"` + Source GoTypeRef `json:"source"` + Target GoTypeRef `json:"target"` + Fields map[string]MappingField `json:"fields,omitempty"` + IgnoreSource map[string]string `json:"ignoreSource,omitempty"` + IgnoreTarget map[string]string `json:"ignoreTarget,omitempty"` +} +type mappingField struct { + path, name string + typ types.Type +} +type mappingTypes struct{ source, target types.Type } +type mappingRenderer struct { + importer types.Importer + imports map[string]string + definitions map[string]Mapping + types map[string]mappingTypes +} + +func renderMappings(o Options, generated, seeds map[string][]byte) error { + if len(o.generation.Mappings) == 0 { + return nil + } + refs, err := validateMappings(o.generation.Mappings, o.Module) + if err != nil { + return err + } + importer, close, err := loadMappingTypes(o, refs, generated, seeds) + if err != nil { + return err + } + defer close() + r := mappingRenderer{importer: importer, imports: map[string]string{}, definitions: map[string]Mapping{}, types: map[string]mappingTypes{}} + for _, m := range o.generation.Mappings { + source, err := r.lookup(m.Source) + if err != nil { + return err + } + target, err := r.lookup(m.Target) + if err != nil { + return err + } + r.definitions[m.Name] = m + r.types[m.Name] = mappingTypes{source, target} + } + names := make([]string, 0, len(r.definitions)) + for name := range r.definitions { + names = append(names, name) + } + sort.Strings(names) + var body strings.Builder + for _, name := range names { + source, err := r.render(r.definitions[name]) + if err != nil { + return fmt.Errorf("mapping %s: %w", name, err) + } + body.WriteString(source) + } + imports := make([]string, 0, len(r.imports)) + for path := range r.imports { + imports = append(imports, path) + } + sort.Strings(imports) + text := generatedHeader + "package mapping\nimport(\n" + for _, path := range imports { + text += r.imports[path] + " " + strconv.Quote(path) + "\n" + } + text += ")\n" + body.String() + code, err := format.Source([]byte(text)) + if err != nil { + return fmt.Errorf("format mappings: %w", err) + } + // The generated package is type-checked before it is written. No application + // code (including init or converters) is executed during generation. + fset := gotoken.NewFileSet() + file, err := goparser.ParseFile(fset, "mappingsGen.go", code, 0) + if err != nil { + return err + } + check := types.Config{Importer: importer} + if _, err = check.Check(o.Module+"/internal/mapping", fset, []*ast.File{file}, nil); err != nil { + return fmt.Errorf("generated mapping type check: %w", err) + } + generated["internal/mapping/mappingsGen.go"] = code + return nil +} + +func validateMappings(defs []Mapping, project string) ([]string, error) { + refs := map[string]bool{} + names := map[string]bool{} + validPackage := func(path string) bool { + if module.CheckImportPath(path) != nil || path == project+"/internal/mapping" { + return false + } + // go/types does not enforce the compiler's internal import visibility. + if strings.HasPrefix(path, "internal/") { + return false + } + if i := strings.LastIndex(path, "/internal/"); i >= 0 { + return strings.HasPrefix(project+"/internal/mapping", path[:i]+"/") + } + return true + } + for _, m := range defs { + if !gotoken.IsIdentifier(m.Name) || !gotoken.IsExported(m.Name) || names[m.Name] { + return nil, fmt.Errorf("mapping name %q must be unique and exported", m.Name) + } + names[m.Name] = true + for _, ref := range []GoTypeRef{m.Source, m.Target} { + if !validPackage(ref.Package) || !gotoken.IsIdentifier(ref.Type) || !gotoken.IsExported(ref.Type) { + return nil, fmt.Errorf("mapping %s: invalid Go type reference", m.Name) + } + refs[ref.Package] = true + } + if len(m.Fields) > 4096 || len(m.IgnoreSource) > 4096 || len(m.IgnoreTarget) > 4096 { + return nil, fmt.Errorf("mapping %s: field rule limit exceeded", m.Name) + } + for target, f := range m.Fields { + if target == "" || f.Source == "" { + return nil, fmt.Errorf("mapping %s: explicit field rule requires target and source paths", m.Name) + } + if f.Convert != nil { + if f.Mapping != "" || !validPackage(f.Convert.Package) || !gotoken.IsIdentifier(f.Convert.Name) || !gotoken.IsExported(f.Convert.Name) { + return nil, fmt.Errorf("mapping %s.%s: invalid or conflicting converter", m.Name, target) + } + refs[f.Convert.Package] = true + } + } + } + visiting, done := map[string]bool{}, map[string]bool{} + byName := map[string]Mapping{} + for _, m := range defs { + byName[m.Name] = m + } + var visit func(string) error + visit = func(name string) error { + if done[name] { + return nil + } + if visiting[name] { + return fmt.Errorf("mapping cycle at %s", name) + } + m, ok := byName[name] + if !ok { + return fmt.Errorf("unknown mapping %s", name) + } + visiting[name] = true + for _, f := range m.Fields { + if f.Mapping != "" { + if err := visit(f.Mapping); err != nil { + return err + } + } + } + delete(visiting, name) + done[name] = true + return nil + } + for _, m := range defs { + if err := visit(m.Name); err != nil { + return nil, err + } + } + out := make([]string, 0, len(refs)) + for path := range refs { + out = append(out, path) + } + sort.Strings(out) + return out, nil +} + +func (r *mappingRenderer) qualifier(p *types.Package) string { + if alias, ok := r.imports[p.Path()]; ok { + return alias + } + alias := fmt.Sprintf("pkg%d", len(r.imports)) + r.imports[p.Path()] = alias + return alias +} +func (r *mappingRenderer) typeName(t types.Type) string { return types.TypeString(t, r.qualifier) } +func (r *mappingRenderer) lookup(ref GoTypeRef) (types.Type, error) { + pkg, err := r.importer.Import(ref.Package) + if err != nil { + return nil, fmt.Errorf("mapping type %s.%s: %w", ref.Package, ref.Type, err) + } + obj, ok := pkg.Scope().Lookup(ref.Type).(*types.TypeName) + if !ok { + return nil, fmt.Errorf("mapping type %s.%s not found", ref.Package, ref.Type) + } + typ := obj.Type() + if n, ok := typ.(*types.Named); ok && n.TypeParams() != nil && n.TypeParams().Len() > 0 { + return nil, fmt.Errorf("mapping root %s must be a non-generic struct", ref.Type) + } + if _, ok := typ.Underlying().(*types.Struct); !ok { + return nil, fmt.Errorf("mapping root %s must be a struct", ref.Type) + } + return typ, nil +} + +func mappingFields(typ types.Type) (map[string]mappingField, error) { + out := map[string]mappingField{} + leaves := map[string]string{} + var walk func(types.Type, string, int) error + walk = func(typ types.Type, prefix string, depth int) error { + if depth > 32 { + return fmt.Errorf("embedded field nesting exceeds 32") + } + st, ok := typ.Underlying().(*types.Struct) + if !ok { + return fmt.Errorf("embedded field %s must be a value struct", prefix) + } + for i := 0; i < st.NumFields(); i++ { + f := st.Field(i) + path := prefix + f.Name() + if !f.Exported() { + return fmt.Errorf("unexported field %s cannot be mapped; use an explicit converter on the containing value", path) + } + if f.Embedded() { + if err := walk(f.Type(), path+".", depth+1); err != nil { + return err + } + continue + } + if previous, exists := leaves[f.Name()]; exists { + return fmt.Errorf("ambiguous embedded fields %s and %s", previous, path) + } + leaves[f.Name()] = path + out[path] = mappingField{path, f.Name(), f.Type()} + if len(out) > 4096 { + return fmt.Errorf("mapping field limit exceeded") + } + } + return nil + } + if err := walk(typ, "", 0); err != nil { + return nil, err + } + return out, nil +} + +func (r *mappingRenderer) render(m Mapping) (string, error) { + mt := r.types[m.Name] + source, err := mappingFields(mt.source) + if err != nil { + return "", err + } + target, err := mappingFields(mt.target) + if err != nil { + return "", err + } + for key, reason := range m.IgnoreSource { + if _, ok := source[key]; !ok || strings.TrimSpace(reason) == "" { + return "", fmt.Errorf("ignoreSource %s requires an existing field and reason", key) + } + } + for key, reason := range m.IgnoreTarget { + if _, ok := target[key]; !ok || strings.TrimSpace(reason) == "" { + return "", fmt.Errorf("ignoreTarget %s requires an existing field and reason", key) + } + } + for key := range m.Fields { + if _, ok := target[key]; !ok { + return "", fmt.Errorf("unknown target field %s", key) + } + if _, ignored := m.IgnoreTarget[key]; ignored { + return "", fmt.Errorf("target %s is both mapped and ignored", key) + } + } + names := make([]string, 0, len(target)) + for name := range target { + names = append(names, name) + } + sort.Strings(names) + used := map[string]bool{} + text := fmt.Sprintf("// %s maps one value. Pointer/map/slice assignments retain Go reference semantics.\nfunc %s(source %s)(%s,error){\nvar result %s\n", m.Name, m.Name, r.typeName(mt.source), r.typeName(mt.target), r.typeName(mt.target)) + for _, path := range names { + dst := target[path] + if reason, ignored := m.IgnoreTarget[path]; ignored { + text += "// Target " + path + " is intentionally left at zero; see omni.gen.json.\n" + _ = reason + continue + } + rule, explicit := m.Fields[path] + if !explicit { + for key, f := range source { + if f.name == dst.name { + rule.Source = key + break + } + } + } + src, ok := source[rule.Source] + if !ok { + return "", fmt.Errorf("target %s has no source; add a field rule or ignoreTarget reason", path) + } + if _, ignored := m.IgnoreSource[src.path]; ignored { + return "", fmt.Errorf("source %s is both mapped and ignored", src.path) + } + used[src.path] = true + from, to := src.typ, dst.typ + srcExpr, dstExpr := "source."+src.path, "result."+path + if rule.Elements { + switch d := to.Underlying().(type) { + case *types.Slice: + s, ok := from.Underlying().(*types.Slice) + if !ok { + return "", fmt.Errorf("%s: elements requires slice to slice or equal-length array to array", path) + } + text += fmt.Sprintf("if %s!=nil {\n%s=make(%s,len(%s))\nfor index:=range %s {\n", srcExpr, dstExpr, r.typeName(to), srcExpr, srcExpr) + from, to = s.Elem(), d.Elem() + case *types.Array: + s, ok := from.Underlying().(*types.Array) + if !ok || s.Len() != d.Len() { + return "", fmt.Errorf("%s: array lengths must match", path) + } + text += fmt.Sprintf("for index:=range %s {\n", srcExpr) + from, to = s.Elem(), d.Elem() + default: + return "", fmt.Errorf("%s: elements requires a slice or array", path) + } + srcExpr += "[index]" + dstExpr += "[index]" + } + expr, fallible, err := r.expression(rule, from, to, srcExpr) + if err != nil { + return "", fmt.Errorf("%s -> %s: %w", src.path, path, err) + } + if fallible { + fmtPkg, err := r.importer.Import("fmt") + if err != nil { + return "", err + } + alias := r.qualifier(fmtPkg) + message := m.Name + "." + path + ": %w" + args := "err" + if rule.Elements { + message = m.Name + "." + path + "[%d]: %w" + args = "index,err" + } + text += fmt.Sprintf("{ value,err:=%s\nif err!=nil{var zero %s;return zero,%s.Errorf(%q,%s)}\n%s=value\n}\n", expr, r.typeName(mt.target), alias, message, args, dstExpr) + } else { + text += dstExpr + "=" + expr + "\n" + } + if rule.Elements { + text += "}\n" + if _, ok := dst.typ.Underlying().(*types.Slice); ok { + text += "}\n" + } + } + } + keys := make([]string, 0, len(source)) + for key := range source { + keys = append(keys, key) + } + sort.Strings(keys) + for _, key := range keys { + if !used[key] { + if _, ignored := m.IgnoreSource[key]; !ignored { + return "", fmt.Errorf("unmapped source field %s; map it or provide an ignoreSource reason", key) + } + } + } + text += "return result,nil\n}\n" + return text, nil +} + +func (r *mappingRenderer) expression(rule MappingField, from, to types.Type, source string) (string, bool, error) { + if rule.Mapping != "" { + m := r.types[rule.Mapping] + if !types.Identical(from, m.source) || !types.Identical(to, m.target) { + return "", false, fmt.Errorf("nested mapping %s has incompatible types", rule.Mapping) + } + return rule.Mapping + "(" + source + ")", true, nil + } + if rule.Convert != nil { + pkg, err := r.importer.Import(rule.Convert.Package) + if err != nil { + return "", false, err + } + fn, ok := pkg.Scope().Lookup(rule.Convert.Name).(*types.Func) + if !ok { + return "", false, fmt.Errorf("converter function %s not found", rule.Convert.Name) + } + sig := fn.Type().(*types.Signature) + if sig.Variadic() || (sig.TypeParams() != nil && sig.TypeParams().Len() > 0) || sig.Params().Len() != 1 || sig.Results().Len() < 1 || sig.Results().Len() > 2 { + return "", false, fmt.Errorf("converter must have signature func(Source) Target or func(Source) (Target,error)") + } + if !types.Identical(sig.Params().At(0).Type(), from) || !types.Identical(sig.Results().At(0).Type(), to) { + return "", false, fmt.Errorf("converter signature does not match field types") + } + fallible := sig.Results().Len() == 2 + if fallible && !types.Identical(sig.Results().At(1).Type(), types.Universe.Lookup("error").Type()) { + return "", false, fmt.Errorf("converter second result must be error") + } + return r.qualifier(pkg) + "." + rule.Convert.Name + "(" + source + ")", fallible, nil + } + if !types.Identical(from, to) { + return "", false, fmt.Errorf("types %s and %s differ; provide an explicit converter", types.TypeString(from, nil), types.TypeString(to, nil)) + } + return source, false, nil +} diff --git a/cmd/go-omni/apigen/mappingLoad.go b/cmd/go-omni/apigen/mappingLoad.go new file mode 100644 index 0000000..320df90 --- /dev/null +++ b/cmd/go-omni/apigen/mappingLoad.go @@ -0,0 +1,175 @@ +package apigen + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "fmt" + "go/importer" + gotoken "go/token" + "go/types" + "io" + "os" + "os/exec" + "path/filepath" + "sort" + "strings" + "time" + + "golang.org/x/mod/modfile" +) + +// loadMappingTypes uses the Go compiler's export data, including the pending +// DTOs via an overlay. Module files and sum updates remain in a temporary dir. +// It compiles only explicitly selected packages and their dependencies; never +// runs project code. Build tags/GOOS follow the caller; GOWORK is disabled. +func loadMappingTypes(o Options, refs []string, generated, seeds map[string][]byte) (types.Importer, func(), error) { + // go list resolves its working directory (notably /var -> /private/var on + // macOS). Overlay keys must name that same physical path or stale DTOs win. + o.Dir = canonical(o.Dir) + temp, err := os.MkdirTemp("", "go-omni-mapping-*") + if err != nil { + return nil, nil, err + } + cleanup := func() { _ = os.RemoveAll(temp) } + fail := func(err error) (types.Importer, func(), error) { cleanup(); return nil, nil, err } + raw, err := os.ReadFile(filepath.Join(o.Dir, "go.mod")) + if err != nil { + return fail(errors.New("mapping generation requires an existing Go module; scaffold first and resolve dependencies")) + } + mf, err := modfile.Parse("go.mod", raw, nil) + if err != nil { + return fail(err) + } + replacements := append([]*modfile.Replace(nil), mf.Replace...) + for _, r := range replacements { + if r.New.Version == "" && !filepath.IsAbs(r.New.Path) { + if err = mf.AddReplace(r.Old.Path, r.Old.Version, filepath.Join(o.Dir, r.New.Path), ""); err != nil { + return fail(err) + } + } + } + raw, err = mf.Format() + if err != nil { + return fail(err) + } + modPath := filepath.Join(temp, "go.mod") + if err = os.WriteFile(modPath, raw, 0600); err != nil { + return fail(err) + } + if sums, e := os.ReadFile(filepath.Join(o.Dir, "go.sum")); e == nil { + if err = os.WriteFile(filepath.Join(temp, "go.sum"), sums, 0600); err != nil { + return fail(err) + } + } else if !errors.Is(e, os.ErrNotExist) { + return fail(e) + } + overlay := map[string]string{} + // Hide stale owned Go files, which may otherwise redeclare a renamed DTO. + var previous manifest + if data, e := os.ReadFile(filepath.Join(o.Dir, manifestPath)); e == nil { + if err = json.Unmarshal(data, &previous); err != nil { + return fail(err) + } + for name := range previous.Generated { + if strings.HasSuffix(name, ".go") { + if !ownedPath(name) { + return fail(fmt.Errorf("invalid owned path %s", name)) + } + if err = safePath(o.Dir, name); err != nil { + return fail(err) + } + if _, ok := generated[name]; !ok { + overlay[filepath.Join(o.Dir, filepath.FromSlash(name))] = "" + } + } + } + } else if !errors.Is(e, os.ErrNotExist) { + return fail(e) + } + for i, files := range []map[string][]byte{generated, seeds} { + names := make([]string, 0, len(files)) + for name := range files { + if strings.HasSuffix(name, ".go") { + names = append(names, name) + } + } + sort.Strings(names) + for _, name := range names { + if err = safePath(o.Dir, name); err != nil { + return fail(err) + } + path := filepath.Join(o.Dir, filepath.FromSlash(name)) + if i == 1 { + if _, e := os.Stat(path); e == nil { + continue + } else if !errors.Is(e, os.ErrNotExist) { + return fail(e) + } + } + backing := filepath.Join(temp, fmt.Sprintf("overlay%d.go", len(overlay))) + if err = os.WriteFile(backing, files[name], 0600); err != nil { + return fail(err) + } + overlay[path] = backing + } + } + raw, err = json.Marshal(map[string]any{"Replace": overlay}) + if err != nil { + return fail(err) + } + overlayPath := filepath.Join(temp, "overlay.json") + if err = os.WriteFile(overlayPath, raw, 0600); err != nil { + return fail(err) + } + args := []string{"list", "-deps", "-export", "-json", "-mod=readonly", "-modfile", modPath, "-overlay", overlayPath} + // fmt is used for fallible conversions even if neither struct imports it. + args = append(args, "fmt") + args = append(args, refs...) + ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "go", args...) + cmd.Dir = o.Dir + cmd.Env = append(os.Environ(), "GOWORK=off") + cmd.WaitDelay = 2 * time.Second + stdout := &limitedCatalogBuffer{limit: 32 << 20} + stderr := &limitedCatalogBuffer{limit: 64 << 10} + cmd.Stdout = stdout + cmd.Stderr = stderr + if err = cmd.Run(); err != nil { + if ctx.Err() != nil { + return fail(fmt.Errorf("mapping type loading: %w", ctx.Err())) + } + // Go download errors may contain authenticated remote URLs. Keep arbitrary + // command output out of diagnostics; exact mapping errors follow type loading. + return fail(fmt.Errorf("mapping type loading failed (%v); run go list -deps -export for the configured source/target/converter packages; check dependencies, build tags and import cycles", err)) + } + exports := map[string]string{} + dec := json.NewDecoder(bytes.NewReader(stdout.Bytes())) + for { + var p struct { + ImportPath, Export string + Error *struct{ Err string } + } + if err = dec.Decode(&p); err == io.EOF { + break + } else if err != nil { + return fail(err) + } + if p.Error != nil { + return fail(fmt.Errorf("mapping package %s could not be compiled", p.ImportPath)) + } + if p.Export != "" { + exports[p.ImportPath] = p.Export + } + } + imp := importer.ForCompiler(gotoken.NewFileSet(), "gc", func(path string) (io.ReadCloser, error) { + file, ok := exports[path] + if !ok { + return nil, fmt.Errorf("no export data for mapping dependency %s", path) + } + return os.Open(file) + }) + return imp, cleanup, nil +} diff --git a/cmd/go-omni/apigen/mapping_test.go b/cmd/go-omni/apigen/mapping_test.go new file mode 100644 index 0000000..c44b467 --- /dev/null +++ b/cmd/go-omni/apigen/mapping_test.go @@ -0,0 +1,266 @@ +package apigen + +import ( + "context" + "encoding/json" + "fmt" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" +) + +func compositionConfig() GeneratorConfig { + pkg := "example.com/composition/internal/" + cfg := GeneratorConfig{Repository: "none", Middleware: "factories"} + for _, name := range []string{"Enterprise", "Tenant", "Admin"} { + fields := map[string]MappingField{ + "EmployeeName": {Source: "Name", Convert: &GoFunctionRef{pkg + "convert", "Name"}}, + "Salary": {Source: "Salary", Convert: &GoFunctionRef{pkg + "convert", "Money"}}, + "Enabled": {Source: "Enabled", Convert: &GoFunctionRef{pkg + "convert", "Bool"}}, + } + m := Mapping{Name: "From" + name, Source: GoTypeRef{pkg + "types", name + "Request"}, Target: GoTypeRef{pkg + "domain", "Employee"}, Fields: fields} + if name != "Enterprise" { + m.IgnoreSource = map[string]string{"CompanyID": "Authorized separately by Logic; never copy path scope into body"} + } + cfg.Mappings = append(cfg.Mappings, m) + } + cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToResponse", Source: GoTypeRef{pkg + "domain", "Result"}, Target: GoTypeRef{pkg + "types", "EmployeeResponse"}, Fields: map[string]MappingField{"ID": {Source: "ID", Convert: &GoFunctionRef{pkg + "convert", "ID"}}}}) + cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToRow", Source: GoTypeRef{pkg + "domain", "Row"}, Target: GoTypeRef{pkg + "domain", "RowDTO"}}) + cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToBatch", Source: GoTypeRef{pkg + "domain", "Batch"}, Target: GoTypeRef{pkg + "domain", "BatchDTO"}, Fields: map[string]MappingField{ + "Rows": {Source: "Rows", Mapping: "ToRow", Elements: true}, "Fixed": {Source: "Fixed", Mapping: "ToRow", Elements: true}, + }}) + return cfg +} +func writeGeneratorConfig(t *testing.T, dir string, cfg GeneratorConfig) { + t.Helper() + raw, err := json.MarshalIndent(cfg, "", " ") + if err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(dir, generatorConfigFile), string(raw)+"\n") +} +func runCompositionGo(t *testing.T, dir string, args ...string) { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "go", args...) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GOWORK=off") + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("go %v: %v\n%s", args, err, out) + } +} + +func TestCompositionMappingConsumer(t *testing.T) { + if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" { + t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1") + } + dir := filepath.Join(t.TempDir(), "app") + o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)} + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + fixtures := map[string]string{ + "internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go", + "internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go", + } + for path, fixture := range fixtures { + writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture)) + } + svcFile := filepath.Join(dir, "internal/svc/serviceContext.go") + source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1) + source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1) + writeFile(t, svcFile, source) + configFile := filepath.Join(dir, "internal/config/config.go") + writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1)) + // Resolve dependencies before opting into compile-time mapping. The first + // generation creates DTOs; the second sees current DTOs through an overlay. + // Logic does not yet import the generated mapping package. + testSource := readFile(t, filepath.Join(dir, "consumer_test.go")) + if err := os.Remove(filepath.Join(dir, "consumer_test.go")); err != nil { + t.Fatal(err) + } + runCompositionGo(t, dir, "mod", "tidy") + cfg := compositionConfig() + writeGeneratorConfig(t, dir, cfg) + o.API = filepath.Join(dir, "api/service.api") + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(dir, "consumer_test.go"), testSource) + for _, group := range []string{"Enterprise", "Tenant", "Admin"} { + text := fmt.Sprintf(`package %s +import("context";"example.com/composition/internal/types";"example.com/composition/internal/mapping";"example.com/composition/internal/domain") +func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){ + input,err:=mapping.From%s(*req);if err!=nil{return nil,err} + result,err:=mapping.ToResponse(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil +} +`, strings.ToLower(group), group, group) + writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text) + } + before := snapshot(t, dir) + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("regeneration changed project") + } + o.Check = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("check changed project") + } + runCompositionGo(t, dir, "test", "-race", "-count=1", "./...") + runCompositionGo(t, dir, "build", ".") + generated := readFile(t, filepath.Join(dir, "internal/mapping/mappingsGen.go")) + if strings.Contains(generated, "encoding/json") || strings.Contains(generated, "reflect") || !strings.Contains(generated, "result.EmployeeName = value") { + t.Fatal("mapping does not emit typed assignments") + } + // Adding a source or target field is detected before any files are written. + for _, tc := range []struct{ name, path, before, after, want string }{ + {"source", o.API, "Name *string", "NewField *string `json:\"newField,optional\"`\n Name *string", "unmapped source field NewField"}, + {"target", filepath.Join(dir, "internal/domain/employee.go"), "EmployeeName string", "Unmapped string\n EmployeeName string", "target Unmapped has no source"}, + } { + t.Run(tc.name, func(t *testing.T) { + original := readFile(t, tc.path) + writeFile(t, tc.path, strings.Replace(original, tc.before, tc.after, 1)) + defer writeFile(t, tc.path, original) + snap := snapshot(t, dir) + if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) { + t.Fatal(err) + } + if !sameFiles(snap, snapshot(t, dir)) { + t.Fatal("failed check wrote files") + } + o.Check = false + defer func() { o.Check = true }() + if err := generateManagedTestProject(t, o); err == nil { + t.Fatal("generate accepted missing mapping") + } + if !sameFiles(snap, snapshot(t, dir)) { + t.Fatal("failed generate wrote files") + } + }) + } + for _, tc := range []struct { + name string + edit func(*GeneratorConfig) + want string + }{ + {"pointerPolicy", func(c *GeneratorConfig) { delete(c.Mappings[0].Fields, "Salary") }, "provide an explicit converter"}, + {"unknownSource", func(c *GeneratorConfig) { c.Mappings[0].Fields["Salary"] = MappingField{Source: "Missing"} }, "has no source"}, + {"wrongConverter", func(c *GeneratorConfig) { + f := c.Mappings[0].Fields["Salary"] + f.Convert.Name = "ID" + c.Mappings[0].Fields["Salary"] = f + }, "signature does not match"}, + {"unhandledScope", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource = nil }, "unmapped source field CompanyID"}, + {"badIgnore", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource["Missing"] = "stale rule" }, "ignoreSource Missing"}, + {"arrayLength", func(c *GeneratorConfig) { + f := c.Mappings[5].Fields["Fixed"] + f.Source = "Rows" + c.Mappings[5].Fields["Fixed"] = f + }, "array lengths must match"}, + } { + t.Run(tc.name, func(t *testing.T) { + changed := compositionConfig() + tc.edit(&changed) + writeGeneratorConfig(t, dir, changed) + defer writeGeneratorConfig(t, dir, cfg) + snap := snapshot(t, dir) + if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) { + t.Fatal(err) + } + if !sameFiles(snap, snapshot(t, dir)) { + t.Fatal("invalid rule wrote files") + } + }) + } + // All diagnostics exercise current DTOs, not stale exported generated types. + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + // Exercise the repository switch with the same actual CLI as the mapper. + if _, err := os.Stat(filepath.Join(dir, "internal/repository")); !os.IsNotExist(err) { + t.Fatalf("default repository skeleton: %v", err) + } + o.Check = false + cfg.Repository = "group" + writeGeneratorConfig(t, dir, cfg) + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + repo := filepath.Join(dir, "internal/repository/enterprise/repository.go") + writeFile(t, repo, readFile(t, repo)+"\n// Project-owned persistence.\n") + cfg.Repository = "none" + writeGeneratorConfig(t, dir, cfg) + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + if !strings.Contains(readFile(t, repo), "Project-owned persistence.") { + t.Fatal("disabling skeleton overwrote business code") + } + o.Check = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + if os.Getenv("GO_OMNI_RELEASE_CLI") != "" { + ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GOWORK=off") + data, err := cmd.CombinedOutput() + if err != nil { + t.Fatal(err, string(data)) + } + var dependency struct { + Version, Sum string + Replace any + } + if err = json.Unmarshal(data, &dependency); err != nil { + t.Fatal(err) + } + if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" { + t.Fatal("not a fixed remote consumer") + } + t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum) + } +} + +func TestMappingConfigurationRejection(t *testing.T) { + cases := []struct { + name string + edit func(*GeneratorConfig) + want string + }{ + {"cycle", func(c *GeneratorConfig) { + c.Mappings[4].Fields = map[string]MappingField{"Value": {Source: "Value", Mapping: "ToRow"}} + }, "cycle"}, + {"unknown", func(c *GeneratorConfig) { + c.Mappings[5].Fields["Rows"] = MappingField{Source: "Rows", Mapping: "Missing", Elements: true} + }, "unknown mapping"}, + {"conflict", func(c *GeneratorConfig) { + f := c.Mappings[0].Fields["Salary"] + f.Mapping = "ToRow" + c.Mappings[0].Fields["Salary"] = f + }, "conflicting converter"}, + {"internal", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.net/private/internal/types" }, "invalid Go type"}, + {"outputCycle", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.com/composition/internal/mapping" }, "invalid Go type"}, + {"duplicates", func(c *GeneratorConfig) { c.Mappings[1].Name = c.Mappings[0].Name }, "unique and exported"}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + cfg := compositionConfig() + tc.edit(&cfg) + if _, err := validateMappings(cfg.Mappings, "example.com/composition"); err == nil || !strings.Contains(err.Error(), tc.want) { + t.Fatal(err) + } + }) + } +} diff --git a/cmd/go-omni/apigen/metadata_test.go b/cmd/go-omni/apigen/metadata_test.go index 3cc23b0..19c3abc 100644 --- a/cmd/go-omni/apigen/metadata_test.go +++ b/cmd/go-omni/apigen/metadata_test.go @@ -238,6 +238,7 @@ func TestDeclaredContractConsumer(t *testing.T) { source := filepath.Join(root, "input/service.api") writeFile(t, source, declaredAPI) dir := filepath.Join(root, "project") + writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`) if _, err := Generate(Options{API: source, Dir: dir, Module: "example.com/dx", FrameworkDir: frameworkRoot(t)}); err != nil { t.Fatal(err) } diff --git a/cmd/go-omni/apigen/middlewareFactory.go b/cmd/go-omni/apigen/middlewareFactory.go new file mode 100644 index 0000000..b529bfa --- /dev/null +++ b/cmd/go-omni/apigen/middlewareFactory.go @@ -0,0 +1,19 @@ +package apigen + +import "fmt" + +func renderMiddlewareFactories(names []string, generated, seeds map[string][]byte) { + // All project-specific types live behind this project-owned interface. svc + // implements it; middleware never imports svc (and cannot form that cycle). + seeds["internal/middleware/dependencies.go"] = []byte("package middleware\n// Dependencies is implemented by *svc.ServiceContext. Add typed accessors here\n// and implement them on ServiceContext. Factories borrow dependencies; register\n// their cleanup in ServiceContext.Init, not in a middleware factory.\ntype Dependencies interface {}\n") + text := fmt.Sprintf("package middleware\nimport(\"fmt\";%q)\n// New constructs declared middleware after ServiceContext.Init succeeds.\nfunc New(deps Dependencies)(map[string]httpx.Middleware,error){\nresult:=map[string]httpx.Middleware{}\n", frameworkModule+"/httpx") + if len(names) == 0 { + text = fmt.Sprintf("package middleware\nimport %q\nfunc New(deps Dependencies)(map[string]httpx.Middleware,error){\nresult:=map[string]httpx.Middleware{}\n", frameworkModule+"/httpx") + } + for i, name := range names { + delete(seeds, "internal/middleware/"+lower(name)+".go") + text += fmt.Sprintf("m%d,err:=New%s(deps)\nif err!=nil{return nil,fmt.Errorf(%q,err)}\nif m%d==nil{return nil,fmt.Errorf(%q)}\nresult[%q]=m%d\n", i, upper(name), "middleware "+name+": %w", i, "middleware "+name+": factory returned nil", name, i) + seeds["internal/middleware/"+lower(name)+"Factory.go"] = []byte(fmt.Sprintf("package middleware\nimport(\"errors\";%q)\n// New%s is project-owned. Implement this factory once; registration is generated.\n// Dependencies have already been initialized. Do not acquire owned resources here.\nfunc New%s(deps Dependencies)(httpx.Middleware,error){\nreturn nil,errors.New(%q)\n}\n", frameworkModule+"/httpx", upper(name), upper(name), "implement "+name+" middleware factory")) + } + generated["internal/middleware/registryGen.go"] = []byte(generatedHeader + text + "return result,nil\n}\n") +} diff --git a/cmd/go-omni/apigen/parse.go b/cmd/go-omni/apigen/parse.go index 1b0f1f4..cfa04b2 100644 --- a/cmd/go-omni/apigen/parse.go +++ b/cmd/go-omni/apigen/parse.go @@ -39,6 +39,8 @@ type Type struct { Fields []Field Position string } + +// An empty Type denotes a named field composition, expanded before compilation. type Field struct{ Name, Type, Tag, Description string } type Route struct { BoundMultipart bool @@ -517,7 +519,11 @@ func (p *parser) parseType() error { } } if expression == "" { - return p.err("field type required; embedded types are not supported") + if p.tokens[p.at].kind == scanner.RawString { + return p.err("composed types cannot carry field tags") + } + typ.Fields = append(typ.Fields, Field{Name: name, Description: description}) + continue } if p.tokens[p.at].kind != scanner.RawString { return p.err("field source tag required") diff --git a/cmd/go-omni/apigen/response_test.go b/cmd/go-omni/apigen/response_test.go index fc766dd..e91e617 100644 --- a/cmd/go-omni/apigen/response_test.go +++ b/cmd/go-omni/apigen/response_test.go @@ -251,6 +251,7 @@ func TestResponseTemplateConsumer(t *testing.T) { t.Fatal(err) } } + writeFile(t, filepath.Join(dir, generatorConfigFile), `{"middleware":"handlers"}`) generate(source, true) writeFile(t, filepath.Join(dir, "response_test.go"), responseConsumerTest) for _, method := range []string{"Item", "Callback", "Partner"} { diff --git a/cmd/go-omni/apigen/templates.go b/cmd/go-omni/apigen/templates.go index a4efea4..c1bfee6 100644 --- a/cmd/go-omni/apigen/templates.go +++ b/cmd/go-omni/apigen/templates.go @@ -51,6 +51,16 @@ This project is generated from an editable API contract. Business code uses standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx. main.go and HTTP composition are continuously generator-owned. +omni.gen.json selects repository: none (default) or group, and middleware: +factories for new projects. Named middleware factories run after svc.Init; +implement Dependencies accessors on ServiceContext and New(deps) once. +Unimplemented/nil factories fail startup and clean registered dependencies. +Existing projects retain middleware: handlers until explicitly migrated. +Optional mappings generate typed assignments in internal/mapping/mappingsGen.go; +configure source/target types and explicit conversion rules after resolving module +dependencies. Missing fields fail generation and --check. See GENERATION.md in +the framework repository for composition, null/reference semantics and migration. + ## Run `+"```sh"+` @@ -133,8 +143,10 @@ internal/svc/runtimeGen.go # GENERATED lifecycle and infrastructure startup internal/apierrors/errorsGen.go # generated fresh error constructors, when declared internal/handler// # generated binding and Logic adapters internal/logic// # developer-owned business operations -internal/repository// # developer-owned persistence contracts -internal/middleware/ # named hooks and generated registry +internal/repository// # optional; repository: group creates editable seeds +internal/middleware/ # typed dependency interface, factories and registry +omni.gen.json # developer-owned generator options and mappings +internal/mapping/*Gen.go # optional compile-checked struct mappings docs/openapi.json # generated OpenAPI 3.0.3 .goOmni/apiManifest.json # ownership and content hashes `+"```"+` diff --git a/cmd/go-omni/apigen/testdata/composition/accessFactory.go b/cmd/go-omni/apigen/testdata/composition/accessFactory.go new file mode 100644 index 0000000..cc30b1f --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/accessFactory.go @@ -0,0 +1,30 @@ +package middleware + +import ( + "errors" + "git.yuntuo.hk/k2safe/go-omni/httpx" +) + +var ErrFactory = errors.New("factory fault") + +func NewAccess(deps Dependencies) (httpx.Middleware, error) { + if deps.Access() == nil { + return nil, errors.New("dependencies not initialized") + } + switch deps.FactoryMode() { + case "fail": + return nil, ErrFactory + case "nil": + return nil, nil + case "panic": + panic(ErrFactory) + } + return func(next httpx.Handler) httpx.Handler { + return func(c *httpx.Context) error { + if err := deps.Access().Check(c.Context()); err != nil { + return err + } + return next(c) + } + }, nil +} diff --git a/cmd/go-omni/apigen/testdata/composition/amount.go b/cmd/go-omni/apigen/testdata/composition/amount.go new file mode 100644 index 0000000..1b66132 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/amount.go @@ -0,0 +1,52 @@ +package amount + +import ( + "encoding/json" + "errors" + "fmt" + "math/big" + "regexp" +) + +// This fixture's business policy is exact decimal HALF_UP. The framework does +// not choose its precision, rounding mode or currency. +type Money string + +var decimalPattern = regexp.MustCompile(`^-?[0-9]+(\.[0-9]+)?$`) + +func (m Money) String() string { + if m == "" { + return "0.00" + } + return string(m) +} +func (m Money) MarshalJSON() ([]byte, error) { return json.Marshal(m.String()) } +func (m *Money) UnmarshalJSON(data []byte) error { + var text string + if err := json.Unmarshal(data, &text); err != nil { + return err + } + if !decimalPattern.MatchString(text) { + return errors.New("invalid decimal") + } + value, ok := new(big.Rat).SetString(text) + if !ok { + return errors.New("invalid decimal") + } + value.Mul(value, big.NewRat(100, 1)) + sign := value.Sign() + value.Abs(value) + cents, rem := new(big.Int), new(big.Int) + cents.QuoRem(value.Num(), value.Denom(), rem) + if rem.Mul(rem, big.NewInt(2)).Cmp(value.Denom()) >= 0 { + cents.Add(cents, big.NewInt(1)) + } + whole, fraction := new(big.Int), new(big.Int) + whole.QuoRem(cents, big.NewInt(100), fraction) + prefix := "" + if sign < 0 && cents.Sign() != 0 { + prefix = "-" + } + *m = Money(fmt.Sprintf("%s%s.%02d", prefix, whole.String(), fraction.Int64())) + return nil +} diff --git a/cmd/go-omni/apigen/testdata/composition/client.go b/cmd/go-omni/apigen/testdata/composition/client.go new file mode 100644 index 0000000..4a14ecf --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/client.go @@ -0,0 +1,36 @@ +package svc + +import ( + "context" + "errors" + "example.com/composition/internal/middleware" + "sync/atomic" +) + +var Closes atomic.Int64 +var ErrInit = errors.New("init fault") + +type Client struct{ closed atomic.Bool } + +func (c *Client) Check(ctx context.Context) error { + if c.closed.Load() { + return errors.New("client closed before HTTP drained") + } + return ctx.Err() +} +func (c *ServiceContext) initClient(ctx context.Context) error { + c.client = &Client{} + c.OnClose(func() error { + if !c.client.closed.CompareAndSwap(false, true) { + return errors.New("double close") + } + Closes.Add(1) + return nil + }) + if c.Config.FactoryMode == "initFail" { + return ErrInit + } + return nil +} +func (c *ServiceContext) Access() middleware.AccessChecker { return c.client } +func (c *ServiceContext) FactoryMode() string { return c.Config.FactoryMode } diff --git a/cmd/go-omni/apigen/testdata/composition/consumer_test.go b/cmd/go-omni/apigen/testdata/composition/consumer_test.go new file mode 100644 index 0000000..ee451d0 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/consumer_test.go @@ -0,0 +1,218 @@ +package main + +import ( + "context" + "encoding/json" + "errors" + "io" + "net" + "net/http/httptest" + "os" + "path/filepath" + "strings" + "sync" + "testing" + + "example.com/composition/internal/amount" + "example.com/composition/internal/config" + "example.com/composition/internal/convert" + "example.com/composition/internal/domain" + "example.com/composition/internal/handler" + "example.com/composition/internal/mapping" + "example.com/composition/internal/middleware" + "example.com/composition/internal/svc" + "example.com/composition/internal/types" + "git.yuntuo.hk/k2safe/go-omni/logx" +) + +func logger(t *testing.T) *logx.Logger { + t.Helper() + cfg := logx.Default() + cfg.Writer = io.Discard + l, e := logx.New(cfg) + if e != nil { + t.Fatal(e) + } + t.Cleanup(func() { _ = l.Close() }) + return l +} +func TestHTTPComposition(t *testing.T) { + deps, e := svc.New(context.Background(), logger(t)) + if e != nil { + t.Fatal(e) + } + defer deps.Close() + app, e := handler.NewApp(deps) + if e != nil { + t.Fatal(e) + } + defer app.Close() + for _, path := range []string{"/enterprise", "/tenant/123", "/admin/123"} { + for _, tc := range []struct { + name, body string + status int + salary, nameValue string + employer any + }{ + {"exact", `{"name":"employee","salary":"90071992547409.91","employer":null,"enabled":false}`, 200, "90071992547409.91", "employee", nil}, + {"round", `{"salary":"1.005","employer":"0"}`, 200, "1.01", "", "0.00"}, + {"missing", `{}`, 200, "0.00", "", nil}, + {"null", `{"name":null,"salary":null,"employer":null,"enabled":null}`, 200, "0.00", "", nil}, + {"zero", `{"salary":"0","enabled":false}`, 200, "0.00", "", nil}, + {"invalid", `{"salary":"abc"}`, 400, "", "", nil}, + {"empty", `{"salary":""}`, 400, "", "", nil}, + {"numeric", `{"salary":100}`, 400, "", "", nil}, + {"flat", `{"SalaryFields":{"salary":"1"}}`, 400, "", "", nil}, + {"validation", `{"name":"` + strings.Repeat("x", 81) + `"}`, 400, "", "", nil}, + } { + t.Run(path+tc.name, func(t *testing.T) { + req := httptest.NewRequest("POST", path, strings.NewReader(tc.body)) + req.Header.Set("Content-Type", "application/json") + w := httptest.NewRecorder() + app.Engine().ServeHTTP(w, req) + if w.Code != tc.status { + t.Fatalf("status=%d body=%s", w.Code, w.Body.String()) + } + if w.Code == 200 { + var out map[string]any + if e := json.Unmarshal(w.Body.Bytes(), &out); e != nil { + t.Fatal(e) + } + if out["salary"] != tc.salary || out["employeeName"] != tc.nameValue || out["employer"] != tc.employer || out["id"] != "9007199254740993" { + t.Fatal(out) + } + } + }) + } + } +} +func TestFullFormAndMappingSemantics(t *testing.T) { + name := "employee" + money := amount.Money("2.00") + yes := true + req := types.EnterpriseRequest{Name: &name, Salary: &money, Enabled: &yes, Labels: []string{"one"}} + first, e := mapping.FromEnterprise(req) + if e != nil { + t.Fatal(e) + } + second, e := mapping.FromEnterprise(types.EnterpriseRequest{}) + if e != nil { + t.Fatal(e) + } + if first.EmployeeName != "employee" || !first.Enabled || second.EmployeeName != "" || second.Enabled || second.Salary.String() != "0.00" || second.Employer != nil { + t.Fatal("full-form update became patch behavior") + } + // Direct assignments retain standard Go aliasing, including scalar pointers. + first.Labels[0] = "shared" + if req.Labels[0] != "shared" { + t.Fatal("unexpected implicit deep copy") + } + req.Employer = &money + first, e = mapping.FromEnterprise(req) + if e != nil || first.Employer != req.Employer { + t.Fatal("nullable pointer changed", e) + } + bad := "reject" + failed, e := mapping.FromEnterprise(types.EnterpriseRequest{Name: &bad}) + if !errors.Is(e, convert.ErrName) || failed.EmployeeName != "" || !strings.Contains(e.Error(), "EmployeeName") { + t.Fatal("converter cause/zero result lost", e) + } + empty, e := mapping.ToBatch(domain.Batch{}) + if e != nil || empty.Rows != nil { + t.Fatal("nil slice changed", e) + } + src := domain.Batch{Rows: []domain.Row{}, Fixed: [2]domain.Row{{Value: "a"}, {Value: "b"}}, Optional: &name} + out, e := mapping.ToBatch(src) + if e != nil || out.Rows == nil || len(out.Rows) != 0 || out.Fixed[1].Value != "b" || out.Optional != src.Optional { + t.Fatal(out, e) + } + src.Rows = []domain.Row{{Value: "one"}} + out, e = mapping.ToBatch(src) + if e != nil || out.Rows[0].Value != "one" { + t.Fatal(out, e) + } + out.Rows[0].Value = "changed" + if src.Rows[0].Value != "one" { + t.Fatal("element mapping reused input slice") + } + var wg sync.WaitGroup + for i := 0; i < 32; i++ { + wg.Add(1) + go func() { + defer wg.Done() + result, err := mapping.FromEnterprise(types.EnterpriseRequest{Name: &name, Salary: &money}) + if err != nil || result.EmployeeName != name || result.Salary != money { + t.Error(result, err) + } + }() + } + wg.Wait() +} +func TestFactoryLifecycle(t *testing.T) { + for _, mode := range []string{"fail", "nil", "initFail", "panic"} { + t.Run(mode, func(t *testing.T) { + before := svc.Closes.Load() + cfg := config.Default() + cfg.FactoryMode = mode + if mode == "panic" { + func() { + defer func() { + if recover() != middleware.ErrFactory { + t.Error("panic cause changed") + } + }() + _, _ = svc.New(context.Background(), logger(t), cfg) + }() + } else { + deps, err := svc.New(context.Background(), logger(t), cfg) + if deps != nil || err == nil { + t.Fatal("factory did not fail closed") + } + if mode == "fail" && !errors.Is(err, middleware.ErrFactory) { + t.Fatal("factory cause lost", err) + } + if mode == "initFail" && !errors.Is(err, svc.ErrInit) { + t.Fatal("init cause lost", err) + } + } + if svc.Closes.Load() != before+1 { + t.Fatal("dependency not closed exactly once") + } + }) + } + deps, e := svc.New(context.Background(), logger(t)) + if e != nil { + t.Fatal(e) + } + if e = deps.Access().Check(context.Background()); e != nil { + t.Fatal(e) + } + before := svc.Closes.Load() + if e = deps.Close(); e != nil { + t.Fatal(e) + } + if e = deps.Close(); e != nil { + t.Fatal(e) + } + if svc.Closes.Load() != before+1 { + t.Fatal("normal close repeated") + } +} +func TestGeneratedEntryListenFailure(t *testing.T) { + listener, err := net.Listen("tcp", "127.0.0.1:0") + if err != nil { + t.Fatal(err) + } + defer listener.Close() + path := filepath.Join(t.TempDir(), "config.yaml") + if err = os.WriteFile(path, []byte("server:\n address: "+listener.Addr().String()+"\n"), 0600); err != nil { + t.Fatal(err) + } + before := svc.Closes.Load() + if err = run(context.Background(), path); err == nil { + t.Fatal("listen should fail") + } + if svc.Closes.Load() != before+1 { + t.Fatal("listen failure did not close dependencies") + } +} diff --git a/cmd/go-omni/apigen/testdata/composition/converters.go b/cmd/go-omni/apigen/testdata/composition/converters.go new file mode 100644 index 0000000..c26a9ca --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/converters.go @@ -0,0 +1,29 @@ +package convert + +import ( + "errors" + "example.com/composition/internal/amount" + "strconv" +) + +var ErrName = errors.New("rejected name") + +// Explicit project rules: full-form omission clears strings/bools and uses zero +// for required money. Nullable employer amounts remain nil by direct assignment. +func Name(value *string) (string, error) { + if value == nil { + return "", nil + } + if *value == "reject" { + return "", ErrName + } + return *value, nil +} +func Money(value *amount.Money) amount.Money { + if value == nil { + return "" + } + return *value +} +func Bool(value *bool) bool { return value != nil && *value } +func ID(value int64) string { return strconv.FormatInt(value, 10) } diff --git a/cmd/go-omni/apigen/testdata/composition/dependencies.go b/cmd/go-omni/apigen/testdata/composition/dependencies.go new file mode 100644 index 0000000..3f05a94 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/dependencies.go @@ -0,0 +1,9 @@ +package middleware + +import "context" + +type AccessChecker interface{ Check(context.Context) error } +type Dependencies interface { + Access() AccessChecker + FactoryMode() string +} diff --git a/cmd/go-omni/apigen/testdata/composition/domain.go b/cmd/go-omni/apigen/testdata/composition/domain.go new file mode 100644 index 0000000..25c891b --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/domain.go @@ -0,0 +1,29 @@ +package domain + +import "example.com/composition/internal/amount" + +type Employee struct { + EmployeeName string + Salary amount.Money + Employer *amount.Money + Enabled bool + Labels []string +} +type Result struct { + Employee + ID int64 +} + +// Distinct container item types exercise explicitly selected element mappings. +type Row struct{ Value string } +type RowDTO struct{ Value string } +type Batch struct { + Rows []Row + Fixed [2]Row + Optional *string +} +type BatchDTO struct { + Rows []RowDTO + Fixed [2]RowDTO + Optional *string +} diff --git a/cmd/go-omni/apigen/testdata/composition/service.api b/cmd/go-omni/apigen/testdata/composition/service.api new file mode 100644 index 0000000..8d0e7a5 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/composition/service.api @@ -0,0 +1,63 @@ +scalar Money ( + goImport: "example.com/composition/internal/amount" + goType: Money + jsonInput: string + jsonOutput: string +) + +type SalaryFields { + // Employee display name. + Name *string `json:"name,optional" binding:"max=80"` + Salary *Money `json:"salary,optional"` + Employer *Money `json:"employer,optional"` + Enabled *bool `json:"enabled,optional"` + Labels []string `json:"labels,optional"` +} + +type EnterpriseRequest { + SalaryFields +} +type TenantRequest { + SalaryFields + CompanyID string `path:"companyId"` +} +type AdminRequest { + SalaryFields + CompanyID string `path:"companyId"` +} +type EmployeeResponse { + ID string `json:"id"` + EmployeeName string `json:"employeeName"` + Salary Money `json:"salary"` + Employer *Money `json:"employer"` + Enabled bool `json:"enabled"` + Labels []string `json:"labels"` +} + +@server ( + group: enterprise + middleware: Access + response: raw +) +service composition { + @handler save + post /enterprise (EnterpriseRequest) returns (EmployeeResponse) +} +@server ( + group: tenant + middleware: Access + response: raw +) +service composition { + @handler save + post /tenant/:companyId (TenantRequest) returns (EmployeeResponse) +} +@server ( + group: admin + middleware: Access + response: raw +) +service composition { + @handler save + post /admin/:companyId (AdminRequest) returns (EmployeeResponse) +} diff --git a/cmd/go-omni/apigen/write.go b/cmd/go-omni/apigen/write.go index 336e8f6..f273777 100644 --- a/cmd/go-omni/apigen/write.go +++ b/cmd/go-omni/apigen/write.go @@ -16,6 +16,7 @@ import ( const manifestPath = ".goOmni/apiManifest.json" type manifest struct { + GeneratorConfig bool `json:"generatorConfig,omitempty"` TemplateVersion string `json:"templateVersion,omitempty"` Profile *projectProfile `json:"profile,omitempty"` Version int `json:"version"` @@ -109,7 +110,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check return result, err } } - next := manifest{TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract} + next := manifest{GeneratorConfig: true, TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract} for name, data := range generated { next.Generated[name] = digest(data) } @@ -272,6 +273,9 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check } func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) } func ownedPath(name string) bool { + if name == "internal/mapping/mappingsGen.go" { + return true + } if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) { return true } diff --git a/cmd/go-omni/main_test.go b/cmd/go-omni/main_test.go index c1f68dc..f10b959 100644 --- a/cmd/go-omni/main_test.go +++ b/cmd/go-omni/main_test.go @@ -8,7 +8,7 @@ import ( ) func TestNewWithRemoteFrameworkVersion(t *testing.T) { - for _, flag := range [][]string{{"--framework-version", "v0.16.0-preview"}, {"--framework-version=v0.16.0-preview"}} { + for _, flag := range [][]string{{"--framework-version", "v0.17.0-preview"}, {"--framework-version=v0.17.0-preview"}} { project := filepath.Join(t.TempDir(), "billing") args := append([]string{"--module", "example.com/billing"}, flag...) if err := runNew(append(args, project)); err != nil { @@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) { if err != nil { t.Fatal(err) } - if !strings.Contains(string(content), "go-omni v0.16.0-preview") || strings.Contains(string(content), "replace ") { + if !strings.Contains(string(content), "go-omni v0.17.0-preview") || strings.Contains(string(content), "replace ") { t.Fatalf("CLI did not pin remote dependency: %s", content) } } diff --git a/cmd/go-omni/scaffold/version.go b/cmd/go-omni/scaffold/version.go index 4c889ca..8b5eb7b 100644 --- a/cmd/go-omni/scaffold/version.go +++ b/cmd/go-omni/scaffold/version.go @@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" // Source builds can carry an unpublished VCS version (including +dirty). // Keep this fallback in sync with the next release; installed CLIs use their // downloaded module version, which also has a module checksum. -const defaultFrameworkVersion = "v0.16.0-preview" +const defaultFrameworkVersion = "v0.17.0-preview" // This target is not a claim that the release already exists. Source users // select a local framework checkout until the module templates are published. diff --git a/examples/managedService/.goOmni/apiManifest.json b/examples/managedService/.goOmni/apiManifest.json index c588208..d9de76a 100644 --- a/examples/managedService/.goOmni/apiManifest.json +++ b/examples/managedService/.goOmni/apiManifest.json @@ -1,5 +1,6 @@ { - "templateVersion": "v0.16.0-preview", + "generatorConfig": true, + "templateVersion": "v0.17.0-preview", "profile": { "database": "none", "storage": "none", diff --git a/examples/managedService/go.mod b/examples/managedService/go.mod index 9c1e01f..a3c46e8 100644 --- a/examples/managedService/go.mod +++ b/examples/managedService/go.mod @@ -2,7 +2,7 @@ module example.com/managed go 1.22.0 -require git.yuntuo.hk/k2safe/go-omni v0.16.0-preview +require git.yuntuo.hk/k2safe/go-omni v0.17.0-preview require ( github.com/gabriel-vasile/mimetype v1.4.3 // indirect diff --git a/examples/managedService/omni.gen.json b/examples/managedService/omni.gen.json new file mode 100644 index 0000000..fef1b70 --- /dev/null +++ b/examples/managedService/omni.gen.json @@ -0,0 +1,4 @@ +{ + "repository": "none", + "middleware": "handlers" +} diff --git a/reports/generationExperienceAcceptance.md b/reports/generationExperienceAcceptance.md new file mode 100644 index 0000000..0c9b79f --- /dev/null +++ b/reports/generationExperienceAcceptance.md @@ -0,0 +1,30 @@ +# 通用生成体验验收(v0.17.0-preview) + +本轮完成公共字段组合、可选结构映射、带依赖的中间件工厂与 Repository 骨架开关。没有添加项目特定金额、授权或数据库规则。 + +## 实际生成项目 + +`TestCompositionMappingConsumer` 从 `.api` 生成独立模块,安装项目自己的 Money scalar、业务类型、转换函数、依赖与中间件,然后生成企业/租户/管理三端的请求和响应映射。在生成模块内执行 `go test -race ./...`、`go build .`,用真实生成 Handler 处理 HTTP 请求。 + +验证: + +- 公共字段生成平铺 DTO;字段注释、长度校验、绑定和 OpenAPI 同步;未知组合、同名字段/来源冲突、循环和展开上限均拒绝。 +- 三端正确处理正常输入、缺失、null、零值、非法金额、空金额、数值金额拒绝和完整表单更新。项目示例的精确十进制保留 `90071992547409.91`,`1.005` 按项目规则得到 `1.01`;可空单位金额保持 nil,ID 字符串保持大整数精度。 +- 同名同类型直接赋值;重命名、指针取值、金额默认值和 ID 字符串由显式转换函数控制;Go 值嵌入按明确路径映射。 +- slice/等长 array 的显式逐项映射、nil 与非 nil 空 slice、普通引用赋值、错误链、零结果与 32 个并发调用通过 race 检查。 +- 新增源/目标字段未处理、错误转换签名、指针差异未声明、遗漏企业范围字段处理、无效忽略规则、数组长度不匹配均在写入前失败;Go 文件、模块文件和 manifest 快照不变。未知/循环映射、冲突转换声明、不可访问 internal 包等在类型加载前拒绝。 +- 重复生成字节一致、业务文件保留,`--check` 只读。macOS `/var` 别名统一到真实路径,新增 DTO 字段不会被旧 Go 导出数据掩盖。 +- 依赖先初始化,再调用工厂自动注册。工厂失败、返回 nil、panic、依赖初始化失败和监听失败都会正确释放已登记资源,真实错误链保留;正常重复 Close 不重复释放资源。 +- Repository 默认关闭,group 模式补齐骨架,切回 none 保留已有业务实现。旧项目默认保持 handlers 模式;新配置建立后若丢失,明确报错,防止静默切换生成策略。 + +## 回归结果 + +[回归记录](generationExperienceRegression.json):全仓库 **44 个有测试的包**通过 race,发布版本的 go vet、go build 通过。记录包含命令、日志摘要和变更 Go 源码 SHA256。 + +[实际进程验收](generationExperienceLocal.json):根目录 go run/go build、重复生成、生成检查、业务保留、nethttp/native 的 SIGINT/SIGTERM、监听端口释放、下载/原始回调/multipart、错误目录与默认 404/405 通过,耗时 4.99 秒;临时进程和目录均由脚本清理。 + +现有 `examples/managedService` 更新生成配置基线后 `generate --check` 与模块编译测试通过。远程发布验收将在固定标签发布后补充;本轮不涉及 eTax 业务迁移。 + +## 使用边界 + +映射是显式启用的 CLI 能力,使用 Go 编译器类型信息,不执行项目初始化或转换函数;运行时只有生成的赋值/调用代码。转换函数包应独立于生成输出包,类型依赖需提前解析。普通指针输入中已经合并的缺失/null 不会被重新推断;普通指针/map/slice 赋值保持 Go 共享引用语义。完整使用与迁移见 [生成能力指南](../GENERATION.md)。 diff --git a/reports/generationExperienceLocal.json b/reports/generationExperienceLocal.json new file mode 100644 index 0000000..910648f --- /dev/null +++ b/reports/generationExperienceLocal.json @@ -0,0 +1,59 @@ +{ + "passed": true, + "mode": "localSource", + "durationSeconds": 4.99, + "checks": [ + "rootGoRun", + "rootGoBuild", + "stableRegeneration", + "checkMainOwnership", + "contractChangePreview", + "logicSignatureCheckPersists", + "businessSourcePreserved", + "fixedConsumerBuild", + "nethttpSIGINT", + "nethttpSIGTERM", + "nativeSIGINT", + "nativeSIGTERM", + "listenerReleased", + "temporaryProcessesRemoved", + "nethttpGeneratedTransfers", + "nativeGeneratedTransfers", + "malformedMultipart400", + "goOwnedErrors", + "default404And405" + ], + "sourceHashes": { + "cmd/go-omni/apigen/generate.go": "5a25134afc402c62ca6afe04320b5ef14141a5ff4caedb9a5eeca4952b302f03", + "cmd/go-omni/apigen/templates.go": "e3177491934aef8cec783bed4cbeb2cfa3cb9b0815c9f7f9c63b85270bb07d24", + "cmd/go-omni/apigen/startup.go": "e3322fa3e2aff9f45d66427f44187535c7c0960edc0f5fdfd5d7b4943b349334", + "cmd/go-omni/apigen/managed_test.go": "52c8ab8a0919c6503cb8835eb258f91f237204c9fc13d9dc735bb913944168b0", + "cmd/go-omni/apigen/contractChanges.go": "aeb83990e908738faaa3cdfa20c3b8938023db05b6e55f2d24352c44144056a8", + "cmd/go-omni/apigen/logicCheck.go": "ca1af4f876a3a0db7b7b9ba9b82c1d6399fd89d29f667dad39cfd3de526eb1cb", + "cmd/go-omni/apigen/write.go": "eab29555906c1d6675ffdf07a30f00536065ec8d406cc1a4e53a05d8db720ff8", + "cmd/go-omni/api.go": "5203a17c5f66977a9eba838d61cf5a7356de53d5f72a1196fd4f2046aa43ac85", + "config/config.go": "b4935ebb452c9df34b733842687ba11eec7e006a54ce42ff767273dd74d6e672", + "httpx/transfer.go": "c9fe4e98c0e599d367eaf19ebecf367db914edfca9ad3a65527496d1dea80a3d", + "httpx/multipart.go": "b942ede7a810b268cd43b395c3bddcfaaacad92812bbb60e653466b8d86bc875", + "scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18", + "lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31", + "health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866", + "config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d", + "httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb", + "app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0", + "app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73", + "cmd/go-omni/scaffold/templates/api-config.go.tmpl": "715d29a9be33ba232eac339547cc0a13ae09c8bcb38a65a69a4f32f09fea5346", + "cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "e8575c04b0923d924d96e4334731313076284fd068c4990715b7a07a2724f969", + "cmd/go-omni/scaffold/templates/api-close.go.tmpl": "3f22d272e92f3b64b62123b3d80ec1a6ce8834257d64752fd75facf12623902b", + "cmd/go-omni/scaffold/templates/api-databaseOptions.go.tmpl": "6e5395b4a787c77a6cabfd85f16049a39578d6ec751a9b78203f3a03baabdf28", + "cmd/go-omni/scaffold/templates/api-options.go.tmpl": "11fe34ce45e8c98f7b5c9c0a2c512eb0b3abf86f95b4b576e083bcb6f5d0f824", + "cmd/go-omni/scaffold/templates/api-context.go.tmpl": "6cf67d37ef64f388b23a98781c3ee3abb71a2e0b9e71eeb70f1e5492ff7e32b4", + "cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247", + "cmd/go-omni/apigen/testdata/managed/background.go": "fb357b5deefd5187f2df406be5189fc4d57da857bd55b474f4726bf065c77167", + "cmd/go-omni/apigen/testdata/managed/client.go": "cdd7d394ddad4bcdc10224890a884aa513ed2ffd8275ecdad8de9963dd10986a", + "cmd/go-omni/apigen/testdata/managed/phases_test.go": "4d826d7133027ac2399f8209f03efa46ed54aa45dcd0a35274a9009314941dca", + "cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319", + "cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f", + "cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1" + } +} diff --git a/reports/generationExperienceRegression.json b/reports/generationExperienceRegression.json new file mode 100644 index 0000000..4b1f8f8 --- /dev/null +++ b/reports/generationExperienceRegression.json @@ -0,0 +1,63 @@ +{ + "passed": true, + "status": "releaseCandidateValidated", + "testedPackages": 44, + "commands": [ + "GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./... -count=1 -timeout=300s", + "go vet ./...", + "go build ./...", + "python3 scripts/verifyManagedProject.py --report reports/generationExperienceLocal.json", + "go run ./cmd/go-omni api generate --dir examples/managedService --check", + "cd examples/managedService && go test ./...", + "git diff --check" + ], + "goVersion": "go version go1.27.1 darwin/arm64", + "logs": { + "fullRace": { + "path": "/tmp/go-omni-v017-race.log", + "sha256": "8c7a1c60dc33d11105eac09181888e2f53af1e78027cc8c4b25bc14f766855d4" + }, + "vet": { + "path": "/tmp/go-omni-v017-vet.log", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + }, + "build": { + "path": "/tmp/go-omni-v017-build.log", + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + }, + "process": { + "path": "/tmp/go-omni-v017-process.log", + "sha256": "748bf31b88405f7f1d550b1d74bbbc5543ea09c5d09b59ec98da6ad040694b12" + } + }, + "sourceHashes": { + "cmd/go-omni/apigen/apigen_test.go": "cc94763c3d55b7cb1025f2d0ac7946be42f7f274234437ead8a29cf1a98f2021", + "cmd/go-omni/apigen/business_test.go": "3b484352282d945f5a58047682079e3d64d4b0bf6ec6bafc23151b1197bb3942", + "cmd/go-omni/apigen/compile.go": "442d1b8c8e9c16e23dc4125c49141e15391f34ff726543d860af849f8c3e1bab", + "cmd/go-omni/apigen/composition.go": "7ed3761b27d9c932d8ddc43e63c2aca798bdbe38f2b40571af8916c8401ea650", + "cmd/go-omni/apigen/composition_test.go": "ea3a907dfb48c12d54fda79ee8a0d0fa9c17f17db9f1b46b344e9b62fcc9c94f", + "cmd/go-omni/apigen/generate.go": "5a25134afc402c62ca6afe04320b5ef14141a5ff4caedb9a5eeca4952b302f03", + "cmd/go-omni/apigen/generatorConfig.go": "467112eeb5180fc1571e811e38e90e6dee3f128fe39e17bb978591cee0fa1b22", + "cmd/go-omni/apigen/managed_test.go": "52c8ab8a0919c6503cb8835eb258f91f237204c9fc13d9dc735bb913944168b0", + "cmd/go-omni/apigen/mapping.go": "1d28f3b1300f50654b392735be546ff9c588664bfc0563e4ff5f838f495e091e", + "cmd/go-omni/apigen/mappingLoad.go": "b301bd49e4b67160ba70be4163c8ab1f3620f5963ec26f4b28af5df185a75170", + "cmd/go-omni/apigen/mapping_test.go": "6660f3183e6078ae5d6baecc654b0d2bcdfc243b6bc1da28811af8d25fa73efa", + "cmd/go-omni/apigen/metadata_test.go": "00086cdf08296ced7ebe2d377bf0dbe3be833bb7e3bfa37216a94adbda50ef31", + "cmd/go-omni/apigen/middlewareFactory.go": "cc0220edb3eff25d7f66b2c37e206455f8730a2b909613713322bee7caf72303", + "cmd/go-omni/apigen/parse.go": "b3f25a260b976dbe0f82e029e5d76ecf35beb9da1e06b0b1235ff5d8a211f015", + "cmd/go-omni/apigen/response_test.go": "789c8e009108e2b7d9d787771bead423b50b80f486c41894c91222097fadf3ec", + "cmd/go-omni/apigen/templates.go": "e3177491934aef8cec783bed4cbeb2cfa3cb9b0815c9f7f9c63b85270bb07d24", + "cmd/go-omni/apigen/testdata/composition/accessFactory.go": "e1e5057ff79f806b3218e93249e8191d097b5c4844977204a8c32848443cba33", + "cmd/go-omni/apigen/testdata/composition/amount.go": "f2d1834502d71d9ecf72ebe3b1077e5ef41f44fd816abd4617ac19f9bb962fdd", + "cmd/go-omni/apigen/testdata/composition/client.go": "19102e191c0471cae9e7405623906d0d804865c40b80ff1d5b240a41ceab6a4e", + "cmd/go-omni/apigen/testdata/composition/consumer_test.go": "9021c47c8af7d0c338c005d9f0a1f97ecdaaffc0b17b01c09d58023cc8c69fd7", + "cmd/go-omni/apigen/testdata/composition/converters.go": "3ded306c68182d0c2184e44eecb6307c366f0ae43a4d23a6a3ef70fc14f45833", + "cmd/go-omni/apigen/testdata/composition/dependencies.go": "edc153204b06da00c0e45c52db43a4a22f8ffe22c4a6d3f3ebec0137b861f84d", + "cmd/go-omni/apigen/testdata/composition/domain.go": "19bc978d173081f18efda643ffad3510287c03b4e47436c7019ea91973f32b72", + "cmd/go-omni/apigen/write.go": "eab29555906c1d6675ffdf07a30f00536065ec8d406cc1a4e53a05d8db720ff8", + "cmd/go-omni/main_test.go": "6f3dd191f94094bb2406b4c25cba1b6b6effd871984a4f75a4fdfa1b8dccbc9e", + "cmd/go-omni/scaffold/version.go": "1b93322fe3156a325ffec8ff33e0c5b054c90625618a52485628691ff2c40ad5", + "scripts/verifyManagedRelease.py": "3a736a308d9dbe3a51555e2864dbcb39dc38cd01bd4e103095f4305a41bde958" + }, + "frameworkVersion": "v0.17.0-preview" +} diff --git a/scripts/verifyManagedRelease.py b/scripts/verifyManagedRelease.py index fd4bd97..a7d3592 100644 --- a/scripts/verifyManagedRelease.py +++ b/scripts/verifyManagedRelease.py @@ -124,22 +124,26 @@ def main(): checks += ["sharedGoCatalogExport", "documentOnlyErrorSelection", "workspaceCheckNoWrites", "staleCatalogRejected", "twoIndependentServiceBuilds"] env.update(GO_OMNI_SCAFFOLD_INTEGRATION="1", GO_OMNI_RELEASE_CLI=str(cli), GO_OMNI_RELEASE_VERSION=args.version) - tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer"] + tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer", "TestCompositionMappingConsumer"] print("Running generated consumers against the remote framework, including lifecycle failure cleanup", flush=True) output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=240s", "./cmd/go-omni/apigen", "-run", "^(" + "|".join(tests) + ")$"], ROOT, env) events = [json.loads(line) for line in output.splitlines() if line.startswith("{")] - passed = [e["Test"] for e in events if e.get("Action") == "pass" and "Test" in e] + passed = [e["Test"] for e in events if e.get("Action") == "pass" and "Test" in e and "/" not in e["Test"]] assert sorted(passed) == sorted(tests), output evidence = "".join(e.get("Output", "") for e in events) sums = re.findall(r"checksum (h1:\S+); no local replace", evidence) assert len(sums) == len(tests) and all(s == dependency["Sum"] for s in sums), "remote consumer checksum mismatch" checks += ["initializationAndListenFailureCleanup", "drainBeforeDependencyClose", "generatedConsumerRaceTests", "errorParamsAndInheritance", "manualHooksAndScalars", "dependencyFatalShutdown", "fatalProcessExitCode", "initializationShutdownNotification", "configuredBuiltinEndpoints", "projectHealthEnvelope", "endpointConflictCleanup", "responseArrayNullableNull", "primitiveAndDynamicArrays", "strictRequestPolicies", "declarativeMultipartBinding", "backgroundStopDrainResourceOrder", "backgroundInitAndListenFailureCleanup", "backgroundTimeoutRetainsResourcesAndExits"] + checks += ["flatFieldComposition", "typedStructureMapping", "explicitNullAndMoneyPolicies", "mappingFailureBeforeWrites", "mappingConsumerRaceTests", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"] + sources = ["lifecycle/runtime.go", "lifecycle/phases.go", "config/lifecycle_test.go", "task/task.go", "cmd/go-omni/scaffold/templates/api-runtime.go.tmpl", "cmd/go-omni/apigen/templates.go", "config/project.go", "contract/bind.go", "contract/policy.go", "contract/multipart.go", "contract/file.go", "cmd/go-omni/apigen/business_test.go", "cmd/go-omni/apigen/generate.go", "lifecycle/controller.go", "config/endpoints.go", "health/health.go", "httpx/response.go", "app/app.go", "app/transport.go", "scripts/verifyManagedRelease.py", "scripts/verifyManagedProject.py", "cmd/go-omni/apigen/managed_test.go"] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/managed").glob("*.go")] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/business").glob("*.go")] sources += ["cmd/go-omni/apigen/responseShape.go", "cmd/go-omni/apigen/documentRefs.go", "cmd/go-omni/apigen/catalogDocument.go", "cmd/go-omni/apigen/workspaceCatalog.go", "config/env.go", "cmd/go-omni/apigen/arrayResponse_test.go"] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/arrays").glob("*.go")] + sources += ["cmd/go-omni/apigen/" + name for name in ("composition.go", "generatorConfig.go", "mapping.go", "mappingLoad.go", "mapping_test.go", "middlewareFactory.go")] + sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/composition").iterdir() if p.is_file()] report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"], "freshModuleCache": True, "noLocalReplace": True, "durationSeconds": round(time.monotonic() - started, 2), "checks": checks,