diff --git a/API.md b/API.md index e925285..d3dc213 100644 --- a/API.md +++ b/API.md @@ -6,6 +6,8 @@ 本阶段目标：从 `.api` 定义生成可运行的 Go 项目；定义、绑定、校验与 OpenAPI 保持一致；重复生成不覆盖业务实现；业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`，运行时契约编译器是 `contract`。 +v0.14 的 [完整开发指南](DEVELOPMENT.md) 说明响应根类型、`@request`、multipart 字段、注释/格式化、Go 错误目录导出与共享契约。 + ## 与 go-zero 的对应关系 调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式：统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写，不引入 go-zero 或 Gin 运行时依赖。 @@ -61,7 +63,7 @@ go-omni new --api user.api --module example.com/userService \ 初次选项记录在 `.goOmni/apiManifest.json`，以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错，因为初始化代码属于业务方；可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。new 与 api new 统一生成契约项目，不再使用旧式 CRUD 目录。 -MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取，非秘密池配置位于 YAML，环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`；API 项目不会额外生成公开的签名发放接口，鉴权和对象权限由业务实现。初始化失败会清理已打开资源，退出清理可重复调用。 +MySQL 使用官方 `database/mysql` 结构化适配器，密码从 `GO_OMNI_DB_PASSWORD` 覆盖，非秘密池配置位于 YAML，环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`；API 项目不会额外生成公开的签名发放接口，鉴权和对象权限由业务实现。初始化失败会清理已打开资源，退出清理可重复调用。 HTTP 请求类型不能推导数据库表结构；迁移和 Repository 由业务自己维护。当前目录的完整示例见 [managedService](examples/managedService/README.md)。MySQL、事务审计和鉴权实现可参考 [旧目录用户管理示例](examples/userService/README.md)，接入时按 [项目指南](PROJECT.md) 装配到 config/svc/logic。 @@ -99,21 +101,21 @@ service user-api { - 可省略的 `syntax = "v1"`；`info` 中的 title、version 和描述性元数据。 - 单独的 `type X { ... }` 与 `type (...)`；基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。 -- 相对 `.api` 导入及 `import (...)`，重复导入去重；拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件，限制导入/类型深度及 schema 展开规模。 +- 相对 `.api` 导入及 `import (...)`，重复导入去重；拒绝循环和未授权的目录越界。跨服务共享须在 `omni.imports.json` 声明 roots，见 [共享契约](DEVELOPMENT.md#两个服务共享契约)。单文件 1 MiB、总计 8 MiB、最多 64 个文件，限制导入/类型深度及 schema 展开规模。 - 同一 service 名称的多个块；`@server` 的 group、prefix、middleware、auth、errors、response，group 可以写 `admin/user`。 - 每个路由必须有 `@handler`；可选单行/多行 `@doc`、`@auth`、`@errors`、`@response`；GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。 - 可省略请求或响应；有响应默认 200，无响应默认 204。Omni 扩展 `@status 201`、`@status 202`；204 不允许响应类型，HEAD 不允许响应体，GET/HEAD 不允许 JSON 请求体。 -- 每个请求字段只使用一个来源：`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query，不解析 multipart 或表单 body。query 切片用重复键；header/path 使用标量。 +- 每个请求字段只使用一个来源：`json`、`path`、`form`、`header`，或声明式上传的 `multipart`、`multipartJSON`、`file`。`form` 始终表示 URL query；query 切片用重复键，header/path 使用标量。上传示例与限额见 [声明式 multipart](DEVELOPMENT.md#声明式-multipart)。 - 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`（支持开区间和单边界）。 - `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数；范围边界必须有限且位于 ±2^53 内。 -暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明，以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。 +暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、未通过 scalar 显式接入的自定义编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明，以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。 业务错误、具名鉴权、多行文档及下述开发命令从 CLI `v0.7.0-preview.1` 提供。公共响应模板从 `v0.8.0-preview` 提供，CLI 与业务项目中的框架依赖需一起升级。 ## 业务错误：声明、代码和文档共用一份定义 -**错误目录也可以只在 Go 中维护。** 从 `v0.11.1-preview` 起，生成路由没有声明错误列表时，继承 `omni.WithErrors(catalog)` 注册的 App 目录；`.api` 继续定义接口和返回格式，无需重复登记错误。全局中间件提前返回同样适用。显式引用 `.api` 错误时才使用路由独立目录，规则见 [目录继承](errors/README.md)。CLI 不执行运行时 Go 配置，因此静态文档不会自动枚举只在 Go 中定义的具体错误。 +**错误目录也可以只在 Go 中维护。** 从 `v0.11.1-preview` 起，生成路由没有声明错误列表时，继承 `omni.WithErrors(catalog)` 注册的 App 目录；`.api` 继续定义接口和返回格式，无需重复登记错误。全局中间件提前返回同样适用。显式引用 `.api` 错误时才使用路由独立目录，规则见 [目录继承](errors/README.md)。CLI 不执行项目 Go 配置；可显式用 `errors.ExportCatalog` 导出 `omni.errors.json`，供文档合并同一目录，见 [导出步骤](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。 从 `v0.10.0-preview` 起，推荐用 `errorCatalog` 同时定义逻辑标识、对外业务码、提示和 HTTP 状态： @@ -258,11 +260,11 @@ go test ./... - 动态 JSON 字段 `any` / `interface{}`，包含切片和 map；外围契约仍严格。 - `scalar Name (...)` 显式引用项目类型，声明 JSON 输入/输出类型及文本绑定；支持已有 JSON/Text 转换方法。 - `@body (mode: json/raw/multipart; maxBytes: ...)`：按路由限制请求体。 -- `@binding custom` 生成保留的绑定 hook；`@handlerMode manual` 生成保留的整个 Handler，仅用于自定义协议。raw/multipart 默认生成自动 Handler，将受限流交给 Logic。 +- `@binding custom` 生成保留的绑定 hook；`@handlerMode manual` 生成保留的整个 Handler，仅用于自定义协议。raw 与未声明上传字段的 multipart 将受限流交给 Logic；声明了上传字段时则交给 Logic 已校验的 DTO。 - `@produces "text/plain"` 等声明无 DTO 成功输出；自动模式的 Logic 返回 `*httpx.Stream`，Handler 负责流式发送和关闭。默认状态 200。manual 模式保留项目自己的输出实现。 - 自动 multipart Handler 检查未读部分和结束边界；损坏/截断默认返回 400，超限为 413。公共状态、码和文案仍可用项目错误目录配置，见 [流式接入与错误处理](PROJECT.md#自动上传回调与下载)。 -迁移扩展起始于 v0.9.0，原有手写扩展说明见 [迁移扩展指南](MIGRATION.md)。当前根目录入口和自动流式 Handler 要求 CLI 与框架依赖一起使用 v0.13.0-preview；版本说明和新目录维护规则见 [项目指南](PROJECT.md)。不兼容全部 goctl 语法。 +迁移扩展起始于 v0.9.0，原有手写扩展说明见 [迁移扩展指南](MIGRATION.md)。当前根目录入口和自动流式 Handler 要求 CLI 与框架依赖一起使用 v0.14.0-preview；版本说明和新目录维护规则见 [项目指南](PROJECT.md)。不兼容全部 goctl 语法。 ## 参数语义 @@ -270,9 +272,9 @@ go test ./... 业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null；普通非指针字段拒绝 null（动态 `any` 允许）；`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil，本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`；需要省略零值时显式使用 `omitempty`。 -JSON 必须是一个 object，Content-Type 为 application/json。默认最多 1 MiB，包含 chunked/未知长度和尾部空白；超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400；错误 Content-Type 返回 415。失败不会部分修改目标 DTO；错误详情只保留在内部 cause，公共响应不回显原始参数。 +普通 JSON 请求体必须是一个 object，Content-Type 为 application/json；响应根类型另支持数组、nullable 对象和 null。默认最多 1 MiB，包含 chunked/未知长度和尾部空白；超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400；错误 Content-Type 返回 415。失败不会部分修改目标 DTO；错误详情只保留在内部 cause，公共响应不回显原始参数。 -`contract.Binder` 在服务启动时编译，随后并发复用，不会每次请求重新分析标签。CLI 用相同编译器产生文档，因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守，不在每次响应上再次校验；非 nullable 集合请返回空集合而不是 nil。 +`contract.Binder` 在服务启动时编译，随后并发复用，不会每次请求重新分析标签。CLI 用相同编译器产生文档，因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守，不在每次响应上再次校验；嵌套的非 nullable 集合请返回空集合而不是 nil，根 `returns ([]Item)` 会把 nil slice 规范为 `[]`。 ## 生成目录和业务边界 @@ -282,7 +284,7 @@ main.go # 启动、日志、优雅退出 etc/config.yaml internal/config/ internal/svc/ # 依赖初始化和释放 -internal/types/typesGen.go +internal/types/*Gen.go internal/apierrors/errorsGen.go # 声明错误后生成 internal/handler/routesGen.go internal/handler/openapiGen.go diff --git a/BUSINESS_DEVELOPMENT.md b/BUSINESS_DEVELOPMENT.md new file mode 100644 index 0000000..8b58561 --- /dev/null +++ b/BUSINESS_DEVELOPMENT.md @@ -0,0 +1,24 @@ +# 通用业务开发能力完善计划 + +本轮目标：业务维护契约、配置、依赖装配与业务逻辑，框架持续维护 HTTP 接入、契约校验、通用运行期与文档。按实际生成项目和可删除的业务适配点验收，不写入 eTax 业务规则。 + +## 实施与验收清单 + +- [x] 运行期：资源登记、逆序释放、幂等关闭、失败回滚、故障原因保留归框架；生成入口与项目依赖初始化边界明确。多 App 日志隔离、标准 log.Logger 适配、可信请求元数据与依赖中间件示例。 +- [x] 配置：项目完整结构体环境覆盖；默认值→文件→环境；嵌套、duration、slice、显式零值、空值策略、必填与脱敏；日志无副作用完整校验。 +- [x] 数据库：官方 MySQL 结构化配置/TLS/驱动日志适配，继续复用 OpenConnector 和唯一连接池；独立启动检查、失败清理、生成器接入；不内置版本/表结构/业务错误策略。 +- [x] 响应契约：数组、可空对象、成功 null、独立 HTTP 204；Logic/Handler/OpenAPI/漂移检查一致。 +- [x] 请求契约：未知 query、JSON UTF-8/charset、无 Body/必填空对象/可选 Body，App 默认与路由覆盖；scalar/any/数组/大小限制一致，普通 JSON 只绑定一次。 +- [x] 上传：声明文件/文本/JSON 字段，重复/未知/必填/单字段/单文件/总量限制；有界暂存与清理，调用 Logic 前完成普通表单校验；保留流式扩展与 Complete 约定。 +- [x] 文档工具：字段注释、example/枚举描述进入 DTO 与 OpenAPI；保注释、幂等格式化及 check；显式 Go 错误目录元数据导出与文档合并，不自动执行项目代码。 +- [x] 共享契约：受控 roots/摘要、循环/深度/文件数/路径保护；分源文件 DTO、多个独立 module 的工作区检查；版本提示与基础模板升级预览，保留业务文件。 +- [x] 集成：实际生成项目验证新能力、连续生成、业务文件保留、失败清理、nethttp/native 请求与退出；现有错误继承/参数/上传下载回归。 +- [ ] 最终回归：全量 race（含生成器集成）、vet、build、格式化/链接检查；使用说明、迁移步骤及验收报告完整，发布后远程无 replace 验收。 + +## 核心约定 + +响应与错误码由项目配置。字段与请求策略由契约表达；基础设施采用可选适配器，保留标准 Connector。所有业务依赖在请求排空后关闭；框架不重试写操作、不自动迁移业务表、不关闭借用资源。 + +普通 multipart 为保证进入 Logic 前完成校验，以有界复制暂存到临时文件；框架清理自己创建的资源。纯流式路径明确由调用方在提交持久副作用前完成输入检查。 + +不修改 eTax/ePay 业务，不连接既有业务服务器。真实基础设施如需验证，只使用本轮隔离资源。验收报告明确区分单元测试、生成消费者测试与真实服务测试。 diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md new file mode 100644 index 0000000..1bc9f65 --- /dev/null +++ b/DEVELOPMENT.md @@ -0,0 +1,261 @@ +# 通用业务开发指南（v0.14.0-preview） + +从 [README](README.md) 创建服务后，业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力；字段、错误码、权限、表结构和事务边界都由项目定义。 + +## 返回数组、可空对象和 null + +```text +@handler list +get /items returns ([]Item) + +@handler find +get /item returns (*Item) + +@handler save +post /item returns (null) + +@handler remove +delete /item +``` + +| 契约 | Logic 返回 | 成功数据 | +| --- | --- | --- | +| `returns (Item)` | `(*types.Item, error)` | 对象；nil 被视为实现错误 | +| `returns ([]Item)` | `([]types.Item, error)` | 数组；nil 统一输出 `[]` | +| `returns (*Item)` | `(*types.Item, error)` | 对象或 `null` | +| `returns (null)` | `error` | HTTP 200 和空业务数据 | +| 不写 returns | `error` | HTTP 204，无响应体 | + +统一响应中的空数据是否显示，仍遵循项目模板的 `dataMode`；需要 `data:null` 就设置 `dataMode: always`。`@response raw` 输出业务数据本身，包括 `[]`、对象或 `null`。可用 `@status 201/202`；204 不能声明响应类型。根类型当前支持以上四种形式，嵌套字段继续支持既有 slice/map/pointer/scalar/any。 + +## 契约驱动的请求策略 + +入口文件可以声明默认策略，单个接口按需覆盖： + +```text +@app ( + defaultResponse: CommonResponse + request: {"unknownQuery":"reject", "utf8":"reject", "charset":"utf8"} +) + +@handler create +@request (body: required) +post /items (CreateReq) returns (Item) + +@handler touch +@request (body: emptyObject) +post /touch returns (null) +``` + +- `unknownQuery: reject|ignore`：是否拒绝未声明的 URL 查询字段；重复标量仍拒绝，数组用重复的同名 query。 +- `utf8: reject|replace`：拒绝原始非法 UTF-8，或采用 JSON 的替换字符行为。不是只检查解码后的字符串。 +- `charset: utf8|any`：`utf8` 仅接受未声明 charset 或 `utf-8`；不会对其他编码做转码。 +- `body: auto|required|optional|none|emptyObject`：auto 按 JSON 字段推断；required 要求一个对象；optional 允许完全没有 body，但提供 body 后仍完整校验；none 禁止任何 body 字节；emptyObject 必须是 `{}`。 + +未设置的项继承 App 的 `config.Config.Request`，最后采用既有默认行为。`.api` 的 `@app` 显式字段优先于项目配置；路由 `@request` 优先于 App。raw/multipart 使用各自解析流程，不能套 JSON body 策略。普通 JSON 只解析一次；重复/未知 JSON 字段、约束、scalar 和 `any` 仍使用同一编译器。自定义 binder/manual Handler 的输入逻辑由项目显式负责。 + +没有 body 与提供 `{}` 不等价。optional 无 body 时 JSON 字段保留零值；需要业务区分是否出现某字段时使用 optional 指针字段。raw/multipart 不继承 App 的 JSON body 策略；GET/HEAD 如遇 App 的 required/optional/emptyObject 默认值，须明确覆盖为 auto 或 none。参数校验不会替代企业权限、记账规则或数据库约束。 + +## 声明式 multipart + +```text +type UploadReq { + Title string `multipart:"title" binding:"required,max=80"` + Metadata AttachmentMeta `multipartJSON:"metadata,maxBytes=16384"` + Attachment File `file:"attachment,maxBytes=10485760"` + Extras []File `file:"extras,optional,maxFiles=3,maxBytes=1048576"` +} +type AttachmentMeta { + Category string `json:"category"` +} + +@handler upload +@body (mode: multipart; maxBytes: 16777216; maxParts: 8; maxFieldBytes: 65536) +post /attachments (UploadReq) returns (AttachmentResp) +``` + +Logic 直接得到 `*types.UploadReq`，无需维护 Handler。`form` 仍表示 URL query；`multipart` 表示文本表单字段，可用基础类型、文本 scalar 和重复 scalar 数组；`multipartJSON` 是一个严格解码的 JSON 表单字段，可包含对象、数组和已声明的 scalar。 + +`File` 提供 `Name`、`ContentType`、`Size`、`Present()` 和 `Open() (io.ReadCloser, error)`。文件名和媒体类型属于客户端元数据，项目继续决定内容检测、存储权限和目标对象名。不要把客户端文件名直接用作存储路径。 + +```go +reader, err := req.Attachment.Open() +if err != nil { return nil, err } +defer reader.Close() +// 将 reader 传给项目选定的流式存储接口；不要先 io.ReadAll 整个大文件。 +``` + +所有声明字段、重复/未知字段、文件数量和各级限额都会在进入 Logic 前校验。缺少必填项、重复标量/JSON 字段、文件与文本类型混用、畸形 JSON 均失败。JSON 的类型、未知字段和重复 key 检查与普通 JSON 一致。 + +默认总量 1 MiB、最多 128 个 part、单文本/JSON 字段 64 KiB；单文件默认受总量约束，文件数组默认最多 16 个。`@body` 可设置 `maxBytes/maxParts/maxFieldBytes/maxFileBytes`；字段上的 `maxBytes/maxFiles` 再收紧对应限额。总量包括 MIME 头和结尾数据；传输编码的 part 被拒绝，避免字节含义不明确。 + +文件使用有界复制写入权限为 0600 的临时文件，文本/JSON 字段在各自限额内缓冲。生成 Handler 返回或 panic 时关闭打开的文件 reader 并删除临时文件；不能把请求的 File 留到异步任务使用。失败不会交付半个 DTO，也不会进入 Logic。临时磁盘容量、并发请求上限与 HTTP 超时由部署配置共同约束。 + +不声明 multipart 字段时，保留原来的 `body *httpx.Body` 流式入口，不创建临时文件。该方式适合特殊协议；项目应在不可撤销副作用之前完成校验并调用 `body.Complete()`。下载和原始回调继续使用 `@produces` / `@body(mode: raw)`。 + +## 完整业务配置的环境覆盖 + +生成项目默认执行：代码与 `envDefault` 默认值 → JSON/YAML 文件 → 全结构体环境覆盖 → 既有框架环境别名 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。 + +```go +type ClientConfig struct { + Endpoint string `json:"endpoint" env:"CLIENT_ENDPOINT,required"` + Token string `json:"-" env:"CLIENT_TOKEN,required,secret,empty=reject"` + Timeout time.Duration `json:"timeout" envDefault:"3s"` + Retries []int `json:"retries"` + Enabled bool `json:"enabled"` +} +``` + +把字段加入项目 `Config` 后，默认环境名取 `GO_OMNI_` 加 JSON 路径的 UPPER_SNAKE_CASE，如 `GO_OMNI_CLIENT_TIMEOUT`。显式 env 名是完整名称，可以使用自己的业务前缀。`env:"-"` 排除整个子树；`json:"-"` 字段只有显式 env 标签才参与。 + +duration 使用 `250ms`、`3s`；slice/map 使用 JSON，如 `[1,3,5]`、`{"retry":"2s"}`。支持嵌套、零/false、空数组、自定义 TextUnmarshaler。空环境值默认允许；可选择 `empty=ignore` 或 `empty=reject`。required 要求字符串非空、指针/集合非 nil；0 和 false 可以是合法业务值，范围由项目 Validate 决定。 + +手写项目可使用 `config.LoadProject(path, defaults, config.EnvOptions{Prefix:"MY_APP_"})`，或分别调用 `ApplyDefaults/DecodeFile/OverlayEnv`。读取失败不修改输入默认值；诊断不会打印输入值。不要主动打印含密钥的整个配置，或把底层解析错误展开到公开响应。Validate 不打开日志文件、数据库或存储连接；证书的可读性和握手信任在依赖初始化阶段验证。 + +## 依赖、日志与中间件 + +`internal/svc/runtimeGen.go` 持续由生成器维护。项目的 ServiceContext 嵌入 `*lifecycle.Runtime`，只在 `Init(ctx)` 中装配额外依赖： + +```go +func (c *ServiceContext) Init(ctx context.Context) error { + client, err := NewClient(c.Config.Client) + if err != nil { return err } + c.Client = client + c.OnClose(client.Close) + c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(client) + return nil +} +``` + +每个资源初始化成功就登记清理；后续初始化失败自动逆序回滚。正常退出先排空 HTTP 请求，再逆序关闭依赖，最后关闭日志。关闭幂等，单个清理函数 panic/返回错误不会跳过其他清理；初始化 panic 清理后继续传播。项目清理函数仍须自行停止、等待后台任务并保证能返回，框架不能强制杀死任意 goroutine。 + +中间件用业务所需的小接口接收依赖，避免反向导入 svc： + +```go +// internal/middleware/accessFactory.go +package middleware + +import ( + "context" + "git.yuntuo.hk/k2safe/go-omni/httpx" +) + +type AccessChecker interface { + CheckAccess(context.Context, string) error +} +func NewRequireAccess(checker AccessChecker) httpx.Middleware { + return func(next httpx.Handler) httpx.Handler { + return func(c *httpx.Context) error { + if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil { + return err + } + return next(c) + } + } +} +``` + +`.api` 声明对应 middleware 后，保留生成的初始 hook，并另建上述工厂；在 Init 用工厂替换 registry 的同名项即可。该包只依赖接口和 HTTP 抽象，不导入 svc，也不把数据库藏到 context 里。 + +生成入口为每个 App 注入独立 Logger，不修改进程默认 Logger。业务可用 `c.Logger.InfoContext(...)`，或请求链中的 `logx.InfoContext(ctx,...)`。标准库/第三方需要 `*log.Logger` 时使用 `c.Logger.Standard(slog.LevelWarn)`；适配器借用原 Logger，不负责关闭。手写多 App 用 `omni.WithLogInstance(logger)` 显式注入。 + +`httpx.Metadata(ctx)` 在业务的标准 context 中提供可信代理规则解析后的 ClientIP、HTTP Method 和路由模板；不放入凭据、完整 URL、查询或 body。业务 userId/tenantId 等仍由项目选择，框架不预设身份模型。 + +`middleware.ContextErrors("REQUEST_CANCELED", "REQUEST_TIMEOUT")` 可将原始取消/超时错误映射到项目错误目录；显式 APIError 优先，内部 cause 链保留。状态、业务码与提示仍在项目 Catalog 配置。 + +## 字段文档、格式化、Go 错误目录导出 + +```text +type Item { + // 企业名称 + Name string `json:"name" example:"示例企业"` + State int `json:"state,options=0|1" example:"1" enumDescriptions:"[\"禁用\",\"启用\"]"` // 状态 +} +``` + +字段前/同行注释同步到 Go 注释和 OpenAPI description。显式 `description` 标签优先；example 的基础字段值也检查类型和约束，枚举说明按声明值顺序填写。字符串 example 直接写内容，其他类型写 JSON。自定义 scalar 的示例不能替代其项目解码器验证；输入输出类型不同时，不把输入示例冒充输出示例。 + +```sh +go-omni api format --api api/service.api +go-omni api format --api api/service.api --check +go-omni api generate +go-omni api generate --check +``` + +格式化保留注释、token 和多行字面量，规范四空格缩进、空行、行首尾空白及相邻单行字段的类型/标签/行尾注释对齐；结果幂等。先校验完整契约，check 不写文件；导入的共享目录不被某个服务的格式化命令修改。DTO 按源 `.api` 输出 `dto<摘要>Gen.go`，都属于 `types` 包，跨文件引用无需改包名。 + +Go Catalog 是错误的唯一来源时，显式导出文档元数据： + +```go +// 在项目自己的导出命令中；appErrors.Catalog 与 AppOptions 注册同一函数。 +if err := omnierrors.ExportCatalog(os.Stdout, appErrors.Catalog()); err != nil { + log.Fatal(err) +} +``` + +```sh +go run ./cmd/exportErrors > api/omni.errors.json +go-omni api generate +``` + +文件是派生产物，应纳入版本控制并由业务 CI 检查导出漂移。CLI 自动读取入口目录的 `omni.errors.json`，校验错误目录、合并状态/模板参数/具体示例到 OpenAPI；不会运行项目 Go，不创建另一份运行期注册器。带独立错误目录的路由继续使用自己的目录。没有路由级精确错误列表时，文档描述 App 目录的公共错误集合；特殊自定义 Encoder 的文档由项目显式扩展。 + +## 两个服务共享契约 + +```text +workspace/ +├── contracts/common/response.api +├── services/api/api/service.api +├── services/api/api/omni.imports.json +├── services/id/api/service.api +├── services/id/api/omni.imports.json +└── omni.workspace.json +``` + +两个服务的 `omni.imports.json` 可以分别声明： + +```json +{"roots":{"common":{"path":"../../../contracts/common","version":"1.0.0"}}} +``` + +```text +import "@common/response.api" +``` + +只读取入口目录和显式声明的根。保留真实路径检查、symlink 越界拒绝、循环检测、32 层、64 文件、单文件 1 MiB/总量 8 MiB 限额；最多 16 个共享根。不会下载依赖或复制共享定义进服务。移动目录时显式更新 roots。 + +`version` 是项目管理的版本标签。生成文档记录每个文件 SHA-256，以及每个共享根所导入文件集合的摘要；可把文档中的 `x-omni-contract-sources.shared.common.sha256` 填入 root 的 `sha256`，固定输入集合，内容变化后必须显式更新。摘要算法是对“该根内相对路径→文件 SHA-256”的键排序 JSON 对象再做 SHA-256，未导入的文件不参与。 + +工作区清单： + +```json +{"services":[{"dir":"services/api"},{"dir":"services/id"}]} +``` + +```sh +go-omni workspace format --file omni.workspace.json +go-omni workspace generate --file omni.workspace.json +go-omni workspace check --file omni.workspace.json +``` + +各服务保持独立 module；可显式填写各服务的 `api`（相对服务目录）和 `module`。所有服务预检通过后再写；每个服务独立提交文件事务，不承诺跨项目的崩溃原子性。check 汇总格式与生成漂移；业务编译/测试仍在各 module 执行。 + +## v0.13 项目升级 + +先使用与目标框架匹配的 CLI；CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.14.0-preview（或显式引用当前源码）。 + +```sh +go-omni api upgrade --dir . --dry-run +go-omni api upgrade --dir . --preview .goOmni/upgradePreview +# 只有标为 safe 的原始模板才会自动替换；review 阻止全部写入。 +go-omni api upgrade --dir . --apply +go mod tidy +go-omni api generate --check +go test ./... +go build . +``` + +自动迁移只替换与 v0.13 原始模板一致的配置/svc 基础文件、移除原始 close.go，并更新最低依赖及生成产物；保留 Logic、Repository、AppOptions、独立命令。迁移失败回滚已改的项目文件。存在业务修改时，仅输出这些基础文件的候选版本和清单，不生成整棵业务代码让人重新拼接。 + +定制项目按预览定点迁移：保留依赖字段，把初始化移到 Init，嵌入 Runtime，清理改为 OnClose 登记；移除旧 New/Close/OnClose；合并业务配置和结构化 MySQL 配置，更新 go.mod，再正常 generate。生成器会拒绝遗留构造/关闭实现，避免重复函数或悄悄绕过清理。v0.12 及更早的定制结构不能被精确自动合并，使用同一预览逐项迁移并编译验证。 diff --git a/PROJECT.md b/PROJECT.md index ee4ed81..b3b926b 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -1,6 +1,6 @@ # 生成项目与文件维护边界 -v0.13.0-preview 提供受管理的生成项目。安装同版远程 CLI，生成的项目直接引用该版本，无需本地 replace。根目录入口支持 `go run .` 和 `go build .`；框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 +v0.14.0-preview 提供受管理的生成项目。安装同版远程 CLI，生成的项目直接引用该版本，无需本地 replace。根目录入口支持 `go run .` 和 `go build .`；框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 ## 创建和日常开发 @@ -47,7 +47,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name): | `internal/config/config.go` | 自定义配置字段、加载、默认值与校验 | | `internal/svc/serviceContext.go` | 依赖字段和初始化，持有加载后的 Config | | `internal/svc/options.go` | AppOptions、GlobalMiddleware；注册 Go 错误目录和全局扩展 | -| `internal/svc/close.go` | 资源关闭；OnClose 为自定义资源提供标准清理入口 | +| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 | | `internal/svc/storage.go` | 选用存储时生成，之后保留 | | `internal/logic//*.go` | 业务方法，使用标准 context.Context | | `internal/repository//*.go` | 数据访问，按需实现 | @@ -58,7 +58,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name): 以上骨架只补缺失文件，既有内容按字节保留。删除已废弃接口后，对应业务文件也不会被自动删除；由开发者整理编译器指出的失效类型或签名。 -生成器持续管理：`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误，还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT；手改会被拒绝覆盖，需先移走自定义内容。 +生成器持续管理：`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误，还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT；手改会被拒绝覆盖，需先移走自定义内容。 Logic 实例在注册时创建并由请求并发复用；请求数据用方法参数和局部变量传递，不存入共享 Logic/svc 字段。 @@ -66,16 +66,16 @@ Logic 实例在注册时创建并由请求并发复用；请求数据用方法 ## 依赖与启动扩展 -`svc.New(ctx, logger, cfg)` 初始化依赖，可返回错误；Logger 由入口创建并借用。新增客户端时只需要扩展配置和 svc： +`svc.New(ctx, logger, cfg)` 初始化依赖，可返回错误；Logger 由入口创建并借用。New 本身由生成器维护；新增客户端时扩展配置和 svc 的 `Init(ctx) error`： ```go -client, err := newClient(cfg.Client) -if err != nil { return nil, err } +client, err := newClient(c.Config.Client) +if err != nil { return err } c.Client = client c.OnClose(client.Close) // 创建成功后立即登记，后续失败也能释放 ``` -`New` 的失败清理骨架已生成。OnClose 按登记的相反顺序执行；Close 幂等。项目不要重复关闭同一资源，借用的 Logger 也不应由 svc 关闭。构造器在返回错误前若创建了尚未交付的资源，该构造器自己负责释放。 +ServiceContext 嵌入 `*lifecycle.Runtime`；框架维护锁、清理栈和失败回滚，项目不再复制 Close/OnClose 实现。OnClose 按登记的相反顺序执行；Close 幂等。项目不要重复关闭同一资源，借用的 Logger 也不应由 svc 关闭。构造器在返回错误前若创建了尚未交付的资源，该构造器自己负责释放。 错误注册写在 `internal/svc/options.go`，错误表可以来自任意项目共用包： @@ -85,7 +85,7 @@ func (c *ServiceContext) AppOptions() []omni.Option { } ``` -业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`，可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录；独立目录不与 App 合并。静态 OpenAPI 不执行 Go，不能自动枚举仅存在于 Go 中的错误目录。 +业务调用 `omnierrors.ByCode(apierrors.CodeUsernameInvalid)`，可接 `WithParams` / `WithCause`。无需在 `.api` 重复错误目录。未声明独立错误目录的路由继承 App 目录；独立目录不与 App 合并。静态 OpenAPI 不执行 Go；可从同一 Catalog 显式导出 `api/omni.errors.json`，详见 [元数据导出](DEVELOPMENT.md#字段文档格式化go-错误目录导出)。 `GlobalMiddleware()` 提供全局中间件；`Middlewares` 映射提供 `.api` 声明的具名中间件。全部显式调用，不依赖项目 init 注册，也不要求 svc 导入 handler。框架请求链按原有次序运行。 @@ -140,16 +140,11 @@ server: 关闭或改名后，旧路径不再由框架保留，项目可以在 `.api` 自己声明。启用的内置 GET 路由与业务路由相同或被业务参数/通配路径覆盖时，在创建 App、开始监听之前报错；不会覆盖业务路由。内置接口仍经过全局中间件；项目在鉴权中间件中明确决定这些路径的访问规则，框架不自动绕过鉴权。 -## 从 v0.12 生成项目升级 + -manifest v2 和现有目录保留。升级同版 CLI 与框架依赖后执行 `api generate`，生成器自动更新 main 和 Handler；`svc`、配置与业务文件仍按字节保留，所以需要在项目拥有的 svc 文件补入以下扩展： +## 从 v0.13 或更早版本升级 -1. `serviceContext.go` 导入 `git.yuntuo.hk/k2safe/go-omni/lifecycle`，给 ServiceContext 增加 `Lifecycle *lifecycle.Controller`；创建 c 时用 `lifecycle.New(ctx)` 初始化，并在后续依赖初始化前设置 `ctx = c.Lifecycle.Context()`。 -2. 增加 `func (c *ServiceContext) RequestShutdown(err error) { c.Lifecycle.RequestShutdown(err) }`。 -3. 构造失败的 defer 在 `c.Close()` 后保留 `c.Lifecycle.Err()`；Close 的 `closeOnce.Do` 开头调用 `c.Lifecycle.RequestShutdown(nil)`，重复关闭仍由已有 once 保护。 -4. 配置默认值若来自 `config.Default()`，自动具有默认端点配置；如需禁用或改路径，在 YAML 或项目 Config 中显式设置。手动拼 Config 的项目需要自行填写要启用的端点。 - -精确实现可对照 [示例 svc](examples/managedService/internal/svc/serviceContext.go) 和 [关闭逻辑](examples/managedService/internal/svc/close.go)，也可在临时目录用新版 CLI 生成参考。不要覆盖原有数据库/客户端初始化；不需要手改 main、Handler 或引入另一套启动器。最后执行 `api generate --check`、`go test ./...`、`go build .`，并验证故障退出。 +请按 [v0.14 定点迁移说明](DEVELOPMENT.md#v013-项目升级) 使用 `api upgrade`。原始模板可以自动升级；含业务修改的基础文件必须参考局部预览合并。旧 New/Close/OnClose 移到框架 Runtime，项目依赖初始化保留在 Init。不要继续向项目里复制生命周期锁和清理栈。 ## App 默认响应格式 @@ -193,7 +188,7 @@ post /callback get /download (DownloadReq) ``` -- raw/multipart 的 Logic 增加 `body *httpx.Body` 参数。`body.Raw` 是受路由上限约束的原始字节流；`body.Header` 是请求头副本。 +- 以下为未声明 multipart 字段的流式用法；声明式绑定见 [上传字段](DEVELOPMENT.md#声明式-multipart)。raw/流式 multipart 的 Logic 增加 `body *httpx.Body` 参数。`body.Raw` 是受路由上限约束的原始字节流；`body.Header` 是请求头副本。 - multipart 用 `body.Parts.NextPart()` 顺序读取，关闭每个 Part；不调用 ParseMultipartForm，不产生上传临时文件，也不把整个文件放入内存。每个字段/文件的业务要求由 Logic 校验。请求 DTO 可定义 path/form/header；`form` 仍是 URL 查询参数，不能用 json DTO 重复读取上传体。 - Parts 是 `*httpx.MultipartReader`，返回的 `*httpx.MultipartPart` 提供 Read、Close、Header、FormName、FileName；NextRawPart 可读取未做 quoted-printable 解码的部分。NextPart 自动关闭上一个未读完的 Part；Close 会报告读取错误。不要混用 Parts 与 Raw 读取同一个 multipart 请求。 - multipart 缺少结束边界、头部/内容损坏默认返回 `400 / INVALID_MULTIPART`；超过路由限额仍为 413，媒体类型/缺少 boundary 为 415。框架只提供通用标识和默认提示，项目可以在 Go 错误目录中配置自己的状态、业务码和文案。内部解析原因保留在错误链，不进入公开响应；Logic 自己返回的数据库等错误不会被统一改成 400。 @@ -213,7 +208,7 @@ get /download (DownloadReq) ## 已有服务接入步骤 -每个独立服务生成一个项目，module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.13.0-preview： +每个独立服务生成一个项目，module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.14.0-preview： 1. 先保留旧服务，整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`；不对旧入口强制覆盖。 2. 在 `.api` 中选择 `@app(defaultResponse: ...)`，显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。 diff --git a/README.md b/README.md index b5045b4..e3a1b64 100644 --- a/README.md +++ b/README.md @@ -2,12 +2,14 @@ GoOmni 是面向多个业务项目复用的 Go 后端框架：用 `.api` 定义接口、请求和返回类型，生成项目目录、HTTP 接入代码及 OpenAPI，业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`，或单独引入日志、存储等模块。 -**从这份 README 开始接入。** 当前版本 **`v0.13.0-preview`**，新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。 +**从这份 README 开始接入。** 当前版本 **`v0.14.0-preview`**，新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。 Go **1.22+**。框架不依赖 Gin，默认使用标准库网络层，提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。 响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置，可以替换。 +新增通用开发能力见 [业务开发指南](DEVELOPMENT.md)：数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。 + ## 选择接入方式 | 你的项目 | 从哪里开始 | @@ -28,7 +30,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可 ```sh export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk" -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.13.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.0-preview export PATH="$(go env GOPATH)/bin:$PATH" go-omni --help ``` @@ -72,7 +74,7 @@ hello/ ├── internal/ │ ├── config/ # 配置读取 │ ├── svc/ # 数据库、日志、外部客户端等依赖 -│ ├── types/typesGen.go # 生成的请求与业务返回类型 +│ ├── types/*Gen.go # 按 .api 文件生成 DTO │ ├── handler/ # 生成的绑定、响应策略、路由及文档 │ ├── logic/greet/ # 在这里实现业务逻辑 │ ├── repository/greet/ # 在这里实现数据访问 @@ -175,7 +177,7 @@ go-omni new --api catalog.api \ --log-output file --log-format json ./catalogFull ``` -该命令在 `catalog.api` 所在目录执行，目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护，API 模板不自动建表；MySQL DSN 通过 `GO_OMNI_DB_DSN` 提供。存储还可选 `s3`、`r2`、`oss`，按生成项目的配置说明提供桶、端点和凭据。完整生成项目示例见 [managedService](examples/managedService/README.md)。 +该命令在 `catalog.api` 所在目录执行，目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护，API 模板不自动建表；MySQL 使用结构化配置，密码通过 `GO_OMNI_DB_PASSWORD` 提供。存储还可选 `s3`、`r2`、`oss`，按生成项目的配置说明提供桶、端点和凭据。完整生成项目示例见 [managedService](examples/managedService/README.md)。 `go-omni new` 与 `go-omni api new` 统一生成契约项目。需要手写项目时，可直接引用框架包；生成器不再提供另一套 CRUD 启动目录。 @@ -184,7 +186,7 @@ go-omni new --api catalog.api \ 在业务项目的 **go.mod 所在目录**执行： ```sh -go get git.yuntuo.hk/k2safe/go-omni@v0.13.0-preview +go get git.yuntuo.hk/k2safe/go-omni@v0.14.0-preview ``` 已有项目可直接复用现有目录、Service、Repository 和数据库连接，先选择一个接口接入。也可以只用 `logx`、`storage` 等包，暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目，按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。 @@ -382,7 +384,7 @@ return apierrors.TextTooLong("企业名称", 100) 生成项目在 `internal/svc` 中创建依赖，提供给 Logic/Repository，并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc，再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`；不要对同一资源重复登记。 -已有项目使用 `driver.Connector` 时，直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 等驱动的独立 TLS、Logger、时区与拨号参数由项目构造 Connector；框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN，也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。 +已有项目使用 `driver.Connector` 时，直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 可直接用 [database/mysql](database/mysql/README.md) 配置独立 TLS、Logger、时区与拨号参数；任意驱动也可由项目构造 Connector。框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN，也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。 | 能力 | 包与接入说明 | | --- | --- | @@ -409,7 +411,7 @@ API 项目的数据库迁移需要单独维护并显式执行；启动不会自 | `GO_OMNI_SHUTDOWN_TIMEOUT` | 优雅退出等待时间，默认 `10s` | | `GO_OMNI_ENABLE_LOGGER` / `GO_OMNI_LOG_LEVEL` | 访问日志开关、日志级别 | | `GO_OMNI_TRUSTED_PROXIES` | 可解析转发地址头的可信代理 IP/CIDR，逗号分隔 | -| `GO_OMNI_DB_DSN` | 选择 MySQL 初始化时的数据库连接配置 | +| `GO_OMNI_DB_HOST/USER/PASSWORD/DATABASE` | 官方 MySQL 适配器的结构化连接配置 | 例如在生成项目中切换 native： @@ -454,7 +456,7 @@ go test ./... `main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理，手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目，重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段，明确的签名错误会给出原签名与期望签名；生成后 `--check` 仍能检出未修复的问题。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交，CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。 -CLI 与业务 module 固定使用 `v0.13.0-preview`。新布局不兼容旧 manifest；旧试验项目需在新目录生成并搬入业务文件，不能直接覆盖手写入口。新布局内部升级再执行上述生成与测试步骤，手写框架项目只需依赖升级及业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 +CLI 与业务 module 固定使用 `v0.14.0-preview`。新布局不兼容旧 manifest；旧试验项目需在新目录生成并搬入业务文件，不能直接覆盖手写入口。新布局内部升级再执行上述生成与测试步骤，手写框架项目只需依赖升级及业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 ## 按需深入 diff --git a/RELEASE.md b/RELEASE.md index 79e3661..226b0bd 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,4 +1,34 @@ -# v0.13.0-preview +# v0.14.0-preview + +本版补齐契约驱动的业务开发与通用基础设施。项目继续维护接口、配置、依赖装配、Logic、SQL 和业务规则；生成器维护入口、请求绑定、路由、响应与文档。 + +## 安装与升级 + +```sh +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.0-preview +go-omni new --module example.com/billing ./billing +``` + +新生成器与运行时一起升级。已有 v0.13 项目先运行 `go-omni api upgrade --dry-run`，再按 [定点迁移步骤](DEVELOPMENT.md#v013-项目升级) 处理；不会静默覆盖自定义 svc/config。运行期的新构造器由生成器维护，项目初始化迁入 `svc.Init`，使用 `OnClose` 登记资源。 + +## 本版能力 + +- `.api` 支持根数组、可空对象、成功 null；真正 204 独立保留。生成 Logic、Handler、OpenAPI 和漂移检查使用相同语义。 +- 请求契约控制未知 query、原始 UTF-8、charset、必须/可选/禁止 body 和严格空对象。路由覆盖 App 策略，普通 JSON 只解析一次。 +- multipart 声明文件、文本和 JSON 表单字段。Logic 收到校验后的 DTO；文件有界暂存、流式读取、请求结束清理，保留底层 raw/multipart/下载扩展。 +- 完整项目配置支持环境覆盖、显式零值、空值策略、必填、嵌套及复杂类型。独立校验先于日志和连接创建。 +- `lifecycle.Runtime` 统一逆序清理、幂等关闭、失败回滚及故障保留；App Logger 独立，支持标准 `log.Logger` 与可信请求元数据。提供依赖中间件装配示例和可选 context 错误映射。 +- 官方 `database/mysql` 适配器提供结构化配置、每连接池独立 TLS/mTLS、驱动日志、启动检查与失败清理。继续返回同一 `database.DB`，不自动迁移表或重试业务事务。 +- 字段注释、示例和枚举说明进入 DTO/OpenAPI；`api format` 幂等格式化。Go 错误目录可显式导出供文档使用，无需复制错误定义。 +- 显式共享契约 roots、来源摘要、分文件 DTO、多服务 workspace 命令、基础模板升级预览；保留目录边界及业务文件所有权。 + +详见 [业务开发指南](DEVELOPMENT.md)、[数据库接入](database/mysql/README.md) 和 [验收记录](reports/businessDevelopmentAcceptance.md)。 + +仍需业务维护：契约、配置、svc 依赖创建/探针/清理登记、错误目录、Logic、Repository 和权限中间件。特殊协议可使用保留的扩展点。声明式上传的暂存磁盘和并发量需配置容量；共享契约版本标签由项目管理，框架校验摘要但不下载依赖。此版仍为 preview，不代表 eTax/ePay 已完成业务迁移。 + +--- + +# v0.13.0-preview（历史版本） 本版补齐项目依赖主动退出与内置接口配置。ID 租约、数据库或后台任务等故障由项目判断，框架通过统一生命周期完成退出；不内置业务规则。 diff --git a/app/logIsolation_test.go b/app/logIsolation_test.go new file mode 100644 index 0000000..8990d6b --- /dev/null +++ b/app/logIsolation_test.go @@ -0,0 +1,58 @@ +package app_test + +import ( + "bytes" + "context" + "net/http/httptest" + "strings" + "testing" + + omni "git.yuntuo.hk/k2safe/go-omni" + "git.yuntuo.hk/k2safe/go-omni/httpx" + "git.yuntuo.hk/k2safe/go-omni/logx" +) + +func TestAppLogAndRequestMetadataIsolation(t *testing.T) { + var a, b bytes.Buffer + la, _ := logx.New(logx.Config{Writer: &a}) + lb, _ := logx.New(logx.Config{Writer: &b}) + apps := []*omni.App{ + omni.New(omni.WithLogger(true), omni.WithLogInstance(la), omni.WithTrustedProxies("127.0.0.1/32")), + omni.New(omni.WithLogger(true), omni.WithLogInstance(lb)), + } + for i, app := range apps { + app.GET("/probe/:id", func(c *httpx.Context) error { + ctx := c.Context() + meta := httpx.Metadata(ctx) + expected := "127.0.0.1" + if i == 0 { + expected = "203.0.113.7" + } + if meta.ClientIP != expected || meta.Route != "/probe/:id" || meta.Method != "GET" { + t.Error(meta) + } + logx.InfoContext(ctx, "business", "app", i) + clone := c.Clone(context.WithValue(ctx, struct{}{}, true), httptest.NewRecorder()) + if httpx.Metadata(clone.Context()) != meta { + t.Error("metadata lost") + } + return c.OK(nil) + }) + } + for _, app := range apps { + r := httptest.NewRequest("GET", "/probe/42", nil) + r.RemoteAddr = "127.0.0.1:1234" + r.Header.Set("X-Forwarded-For", "203.0.113.7") + app.ServeHTTP(httptest.NewRecorder(), r) + } + for i, out := range []string{a.String(), b.String()} { + if !strings.Contains(out, "http.request") || !strings.Contains(out, "business") || strings.Contains(out, `"app":`+string(rune('0'+(1-i)))) { + t.Fatal(out) + } + } + _ = apps[0].Close() + lb.Info("still open") + if !strings.Contains(b.String(), "still open") { + t.Fatal("borrowed logger closed") + } +} diff --git a/app/router.go b/app/router.go index b723ed5..3a89642 100644 --- a/app/router.go +++ b/app/router.go @@ -3,7 +3,9 @@ package app import ( "net/http" + "git.yuntuo.hk/k2safe/go-omni/contract" "git.yuntuo.hk/k2safe/go-omni/httpx" + "git.yuntuo.hk/k2safe/go-omni/logx" "git.yuntuo.hk/k2safe/go-omni/response" ) @@ -26,10 +28,14 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid } } return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) { + request = request.WithContext(logx.WithLogger(contract.WithRequestPolicy(request.Context(), a.config.Request), a.config.LogInstance)) if len(a.config.TrustedProxies) > 0 { request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver)) } requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context())) + requestContext.WithContext(httpx.WithRequestMetadata(request.Context(), httpx.RequestMetadata{ + ClientIP: requestContext.ClientIP(), Method: request.Method, Route: httpx.RoutePattern(request.Context()), + })) if provider, ok := a.responseEncoder.(response.ErrorRegistryProvider); ok { requestContext.SetErrorRegistry(provider.ErrorRegistry()) } diff --git a/cmd/go-omni/README.md b/cmd/go-omni/README.md index b7784d4..921987a 100644 --- a/cmd/go-omni/README.md +++ b/cmd/go-omni/README.md @@ -1,12 +1,14 @@ # go-omni CLI +v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目录>|--apply]`，以及 `workspace format|generate|check --file omni.workspace.json`。详见 [通用开发指南](../../DEVELOPMENT.md)。 + 新项目采用根目录 `main.go` 与 `internal/config、svc、logic、handler、types`。完整职责、启动扩展、流式接口和生成文件边界见 [项目指南](../../PROJECT.md)。 ## 创建项目 ```sh # 仓库读取权限与 PATH 配置见根目录 README -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.13.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.14.0-preview go-omni new --module example.com/hello ./hello # 从自己的契约生成；service.api 及其相对 import 文件需要在本地存在 @@ -41,14 +43,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护；业务文件按字节 ## 版本与迁移 -CLI 与框架依赖固定使用 v0.13.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换： +CLI 与框架依赖固定使用 v0.14.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换： ```sh # 在 go-omni 源码仓库中执行 go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello ``` -新模板需要 v0.13.0-preview 的依赖退出与端点配置能力，继续使用 manifest v2。v0.12 项目需按 [升级步骤](../../PROJECT.md#从-v012-生成项目升级) 补入 svc 生命周期字段和方法，生成器保留这些项目文件。不兼容旧目录，旧生成项目请在新目录重新生成后移入业务代码，不覆盖手写 main。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。 +新模板需要 v0.14.0-preview 的通用 Runtime、配置和绑定能力，继续使用 manifest v2。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级)：原始模板可自动替换，含业务修改的基础文件按预览合并；通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。 ## 数据库迁移命令 diff --git a/cmd/go-omni/api.go b/cmd/go-omni/api.go index de8aebd..18ca9af 100644 --- a/cmd/go-omni/api.go +++ b/cmd/go-omni/api.go @@ -15,16 +15,19 @@ func runAPI(args []string) error { return nil } command := args[0] + if command == "upgrade" { + return runUpgrade(args[1:]) + } if command == "go" { command = "generate" } if command == "check" { command = "validate" } - if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" { + if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" && command != "format" { return fmt.Errorf("unknown api command %q", command) } - readOnly := command == "validate" || command == "inspect" || command == "export" + readOnly := command == "validate" || command == "inspect" || command == "export" || command == "format" o := apigen.Options{} for i := 1; i < len(args); i++ { arg := args[i] @@ -33,7 +36,7 @@ func runAPI(args []string) error { return nil } if arg == "--check" || arg == "--dry-run" { - if command != "generate" { + if command != "generate" && !(command == "format" && arg == "--check") { return fmt.Errorf("%s belongs to api generate", arg) } if arg == "--check" { @@ -108,6 +111,14 @@ func runAPI(args []string) error { return err } } + if command == "format" { + changes, err := apigen.FormatProject(o.API, o.Check) + if err != nil { + return err + } + fmt.Printf("API format: %d files updated\n", len(changes)) + return nil + } if readOnly { spec, err := apigen.Parse(o.API) if err != nil { diff --git a/cmd/go-omni/apigen/business_test.go b/cmd/go-omni/apigen/business_test.go new file mode 100644 index 0000000..0008727 --- /dev/null +++ b/cmd/go-omni/apigen/business_test.go @@ -0,0 +1,91 @@ +package apigen + +import ( + "context" + "encoding/json" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" +) + +func TestBusinessContract(t *testing.T) { + spec, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/business.api")) + if err != nil { + t.Fatal(err) + } + c, err := Compile(spec) + if err != nil { + t.Fatal(err) + } + var doc map[string]any + if err = json.Unmarshal(c.Document, &doc); err != nil { + t.Fatal(err) + } + paths := doc["paths"].(map[string]any) + body := paths["/optional"].(map[string]any)["post"].(map[string]any)["requestBody"].(map[string]any) + if body["required"] != false { + t.Fatal("optional body documented as required") + } + for _, path := range []string{"/empty", "/absent"} { + if paths[path].(map[string]any)["post"].(map[string]any)["x-omni-request-policy"] == nil { + t.Fatal("missing policy metadata") + } + } +} +func TestBusinessGeneratedProject(t *testing.T) { + if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" { + t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1") + } + dir := filepath.Join(t.TempDir(), "business") + o := Options{Dir: dir, API: filepath.Join(frameworkRoot(t), "examples/contracts/business.api"), Module: "example.com/business", FrameworkDir: frameworkRoot(t)} + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + for _, name := range []string{"upload", "nullable", "nothing", "rawNull", "rawArray", "empty", "absent", "optional", "echo", "tolerant", "noContent"} { + writeFile(t, filepath.Join(dir, "internal/logic/demo/"+name+".go"), "package demo\n") + } + writeFile(t, filepath.Join(dir, "internal/logic/demo/list.go"), readFile(t, "testdata/business/operations.go")) + writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/business/consumer_test.go")) + writeFile(t, filepath.Join(dir, "internal/middleware/accessFactory.go"), readFile(t, "testdata/business/accessFactory.go")) + writeFile(t, filepath.Join(dir, "internal/svc/client.go"), readFile(t, "testdata/business/client.go")) + svcFile := filepath.Join(dir, "internal/svc/serviceContext.go") + source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n Client *accessClient", 1) + source = strings.Replace(source, "return nil", "return c.initAccess(ctx)", 1) + writeFile(t, svcFile, source) + o.API = filepath.Join(dir, "api/business.api") + o.Check = true + if err := generateManagedTestProject(t, o); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second) + defer cancel() + cmd := exec.CommandContext(ctx, "go", "test", "-race", "-mod=mod", "-count=1", "./...") + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GOWORK=off") + if out, err := cmd.CombinedOutput(); err != nil { + t.Fatalf("%v\n%s", err, out) + } + if os.Getenv("GO_OMNI_RELEASE_CLI") != "" { + cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GOWORK=off") + raw, err := cmd.CombinedOutput() + if err != nil { + t.Fatal(err, string(raw)) + } + var module struct { + Version, Sum string + Replace any + } + if err = json.Unmarshal(raw, &module); err != nil { + t.Fatal(err) + } + if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" { + t.Fatal("not a fixed remote consumer") + } + t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum) + } +} diff --git a/cmd/go-omni/apigen/catalogDocument.go b/cmd/go-omni/apigen/catalogDocument.go new file mode 100644 index 0000000..a18bc7d --- /dev/null +++ b/cmd/go-omni/apigen/catalogDocument.go @@ -0,0 +1,69 @@ +package apigen + +import ( + "errors" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "git.yuntuo.hk/k2safe/go-omni/response" + "io" + "os" + "path/filepath" +) + +const catalogDocumentFile = "omni.errors.json" + +func readDocumentCatalog(root string, spec *Spec) error { + path := filepath.Join(root, catalogDocumentFile) + stat, err := os.Lstat(path) + if errors.Is(err, os.ErrNotExist) { + return nil + } + if err != nil { + return err + } + if !stat.Mode().IsRegular() { + return errors.New("api: error metadata must be a regular file") + } + file, err := os.Open(path) + if err != nil { + return err + } + defer file.Close() + raw, err := io.ReadAll(io.LimitReader(file, (1<<20)+1)) + if err != nil { + return err + } + if len(raw) > 1<<20 { + return errors.New("api: error metadata exceeds 1 MiB") + } + var catalog apierrors.Catalog + if err = apierrors.DecodeCatalogJSON(raw, &catalog); err != nil { + return err + } + registry, err := apierrors.NewRegistry(catalog) + if err != nil { + return err + } + normalized := registry.Catalog() + spec.DocumentCatalog = &normalized + spec.Sources[catalogDocumentFile] = raw + spec.SourcePaths[catalogDocumentFile] = path + return nil +} +func validateDocumentCatalog(spec *Spec) error { + if spec.DocumentCatalog == nil { + return nil + } + for _, r := range spec.Routes { + if len(r.Errors) > 0 { + continue + } + policy, err := response.New(responseConfig(spec, r)) + if err != nil { + return err + } + if err = policy.ValidateErrors(*spec.DocumentCatalog); err != nil { + return err + } + } + return nil +} diff --git a/cmd/go-omni/apigen/compile.go b/cmd/go-omni/apigen/compile.go index a62b584..659dfbf 100644 --- a/cmd/go-omni/apigen/compile.go +++ b/cmd/go-omni/apigen/compile.go @@ -40,8 +40,17 @@ func Compile(spec *Spec) (*Compiled, error) { if prepareErr != nil { return nil, prepareErr } + if err := validateDocumentCatalog(spec); err != nil { + return nil, err + } + if err := spec.RequestPolicy.Validate(); err != nil { + return nil, err + } definitions := map[string]Type{} for _, t := range spec.Types { + if t.Name == "File" { + return nil, errors.New("File is reserved for multipart uploads") + } if t.Name == "ContractScalars" && len(spec.Scalars) > 0 { return nil, errors.New("ContractScalars is reserved when declaring scalars") } @@ -61,7 +70,7 @@ func Compile(spec *Spec) (*Compiled, error) { resolving := map[string]bool{} var resolve func(string, int) (reflect.Type, error) var expression func(ast.Expr, int) (reflect.Type, error) - primitives := map[string]reflect.Type{"any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))} + primitives := map[string]reflect.Type{"File": reflect.TypeOf(contract.File{}), "any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))} expression = func(e ast.Expr, depth int) (reflect.Type, error) { if depth > 32 { return nil, errors.New("type nesting exceeds 32") @@ -138,7 +147,7 @@ func Compile(spec *Spec) (*Compiled, error) { if err != nil { return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err) } - fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)}) + fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(fieldTag(f))}) } t := reflect.StructOf(fields) resolved[name] = t @@ -162,7 +171,7 @@ func Compile(spec *Spec) (*Compiled, error) { } handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{} middlewareSymbols := map[string]string{} - for _, r := range spec.Routes { + for routeIndex, r := range spec.Routes { if !goName.MatchString(r.Handler) { return nil, fmt.Errorf("%s: invalid handler name", r.Position) } @@ -233,14 +242,26 @@ func Compile(spec *Spec) (*Compiled, error) { if err := validateRouteExtensions(r); err != nil { return nil, fmt.Errorf("%s: %w", r.Position, err) } + effectivePolicy := contract.MergePolicy(spec.RequestPolicy, r.RequestPolicy) + if (r.Method == "get" || r.Method == "head") && effectivePolicy.Body != "" && effectivePolicy.Body != "auto" && effectivePolicy.Body != "none" { + return nil, errors.New("GET/HEAD cannot inherit a JSON body policy; override body with auto or none") + } op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}} if r.Request != "" { b, ok := c.Bindings[r.Request] if !ok { return nil, fmt.Errorf("unknown request %s", r.Request) } + r.BoundMultipart = b.HasMultipart() && !r.CustomBinding && !r.Manual + spec.Routes[routeIndex].BoundMultipart = r.BoundMultipart + if b.HasMultipart() && r.BodyMode != "multipart" { + return nil, errors.New("multipart fields require @body (mode: multipart)") + } op.Parameters = b.Parameters() op.Request = b.BodySchema() + if op.Request != nil && (effectivePolicy.Body == "none" || effectivePolicy.Body == "emptyObject") { + return nil, errors.New("request policy conflicts with JSON fields") + } if op.Request != nil && (r.BodyMode == "raw" || r.BodyMode == "multipart") { return nil, errors.New("raw/multipart request DTOs may declare only path/form/header fields; use the generated body stream for content") } @@ -250,11 +271,7 @@ func Compile(spec *Spec) (*Compiled, error) { } var response *openapi.Schema if r.Response != "" { - b, ok := c.Bindings[r.Response] - if !ok { - return nil, fmt.Errorf("unknown response %s", r.Response) - } - s, err := b.ResponseSchema() + s, err := responseDataSchema(r.Response, c.Bindings) if err != nil { return nil, err } @@ -295,7 +312,7 @@ func Compile(spec *Spec) (*Compiled, error) { if err = decoder.Decode(&doc); err != nil { return nil, err } - decorateDocument(doc, spec) + decorateDocument(doc, spec, c.Bindings) c.Document, err = json.MarshalIndent(doc, "", " ") return c, err } diff --git a/cmd/go-omni/apigen/contractChanges.go b/cmd/go-omni/apigen/contractChanges.go index 95bbd73..aed5f37 100644 --- a/cmd/go-omni/apigen/contractChanges.go +++ b/cmd/go-omni/apigen/contractChanges.go @@ -90,7 +90,8 @@ func snapshotContract(spec *Spec) *contractSnapshot { } } visit(r.Request) - visit(r.Response) + _, responseName := responseShape(r.Response) + visit(responseName) op := routeContract{ Method: strings.ToUpper(r.Method), Path: r.Path, Group: r.Group, Handler: r.Handler, Request: r.Request, Response: r.Response, BodyMode: r.BodyMode, Produces: r.Produces, @@ -223,14 +224,14 @@ func expectedLogicSignature(r Route) string { if r.Request != "" { params = append(params, "*types."+r.Request) } - if r.BodyMode == "raw" || r.BodyMode == "multipart" { + if streamsBody(r) { params = append(params, "*httpx.Body") } results := []string{"error"} if r.Produces != "" { results = []string{"*httpx.Stream", "error"} - } else if r.Response != "" { - results = []string{"*types." + r.Response, "error"} + } else if result := responseGoType(r.Response); result != "" { + results = []string{result, "error"} } return displaySignature(upper(r.Handler), params, results) } diff --git a/cmd/go-omni/apigen/contractMetadata.go b/cmd/go-omni/apigen/contractMetadata.go index 8b0a051..24dee16 100644 --- a/cmd/go-omni/apigen/contractMetadata.go +++ b/cmd/go-omni/apigen/contractMetadata.go @@ -3,6 +3,7 @@ package apigen import ( "encoding/json" "fmt" + "git.yuntuo.hk/k2safe/go-omni/contract" "net/http" "regexp" "sort" @@ -119,7 +120,8 @@ func prepareMetadata(original *Spec) (*Spec, error) { return &spec, nil } -func decorateDocument(doc map[string]any, spec *Spec) { +func decorateDocument(doc map[string]any, spec *Spec, bindings map[string]*contract.Binder) { + doc["x-omni-contract-sources"] = sourceMetadata(spec) if spec.DefaultResponse != "" { doc["x-omni-default-response"] = spec.DefaultResponse } @@ -144,7 +146,12 @@ func decorateDocument(doc map[string]any, spec *Spec) { } doc["components"].(map[string]any)["securitySchemes"] = schemes definitions := map[string]ErrorDefinition{} - registry := apierrors.MustRegistry(errorCatalog(spec, nil)) + declaredRegistry := apierrors.MustRegistry(errorCatalog(spec, nil)) + var documentRegistry *apierrors.Registry + if spec.DocumentCatalog != nil { + documentRegistry = apierrors.MustRegistry(*spec.DocumentCatalog) + doc["x-omni-app-error-catalog"] = documentRegistry.Catalog() + } for _, e := range spec.Errors { definitions[e.Name] = e } @@ -152,6 +159,7 @@ func decorateDocument(doc map[string]any, spec *Spec) { doc["x-omni-errors"] = spec.Errors } for _, r := range spec.Routes { + registry := declaredRegistry policy := response.MustNew(responseConfig(spec, r)) cfg := policy.Config() parts := strings.Split(r.Path, "/") @@ -161,7 +169,16 @@ func decorateDocument(doc map[string]any, spec *Spec) { } } op := doc["paths"].(map[string]any)[strings.Join(parts, "/")].(map[string]any)[r.Method].(map[string]any) - decorateRequestOptions(op, r) + requestRoute := r + requestRoute.RequestPolicy = contract.MergePolicy(spec.RequestPolicy, r.RequestPolicy) + decorateRequestOptions(op, requestRoute) + if b := bindings[r.Request]; b != nil && b.HasMultipart() { + encoding := map[string]any{} + for _, name := range b.MultipartJSONFields() { + encoding[name] = map[string]any{"contentType": "application/json"} + } + op["requestBody"] = map[string]any{"required": true, "content": map[string]any{"multipart/form-data": map[string]any{"schema": b.MultipartSchema(), "encoding": encoding}}} + } if r.ResponseFormat != "" { op["x-omni-response"] = r.ResponseFormat } @@ -193,6 +210,13 @@ func decorateDocument(doc map[string]any, spec *Spec) { e := definitions[name] grouped[e.Status] = append(grouped[e.Status], e) } + if len(r.Errors) == 0 && documentRegistry != nil { + registry = documentRegistry + op["x-omni-error-catalog-scope"] = "app" + for key, definition := range registry.Catalog().Definitions { + grouped[definition.Status] = append(grouped[definition.Status], ErrorDefinition{Name: key, Code: key, Status: definition.Status, Message: definition.Message, BusinessCode: definition.Code, Params: definition.Params}) + } + } if len(r.Errors) > 0 { op["x-omni-errors"] = r.Errors } diff --git a/cmd/go-omni/apigen/documentation_test.go b/cmd/go-omni/apigen/documentation_test.go new file mode 100644 index 0000000..c580542 --- /dev/null +++ b/cmd/go-omni/apigen/documentation_test.go @@ -0,0 +1,127 @@ +package apigen + +import ( + "bytes" + "encoding/json" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestFieldDocumentationAndFormat(t *testing.T) { + source := "// 项目说明\n type Req{\n// 状态\n State int `json:\"state,options=0|1\" example:\"1\" enumDescriptions:\"[\\\"禁用\\\",\\\"启用\\\"]\"` // 状态值\n /* 第一行\n 第二行 */\nName string `json:\"name\" example:\"企业名称\"`\n}\nservice demo{\n @handler echo\npost /echo (Req) returns (Req)\n}\n" + formatted, err := Format([]byte(source)) + if err != nil { + t.Fatal(err) + } + again, err := Format(formatted) + if err != nil || !bytes.Equal(formatted, again) { + t.Fatal("format is not idempotent", err, string(again)) + } + if !bytes.Contains(formatted, []byte(" // 状态\n State")) { + t.Fatal(string(formatted)) + } + spec, err := parseText(t, string(formatted)) + if err != nil { + t.Fatal(err) + } + c, err := Compile(spec) + if err != nil { + t.Fatal(err) + } + if spec.Types[0].Fields[0].Description != "状态\n状态值" || spec.Types[0].Fields[1].Description != "第一行\n第二行" { + t.Fatalf("%+v", spec.Types[0].Fields) + } + schema, err := c.Bindings["Req"].ResponseSchema() + if err != nil { + t.Fatal(err) + } + state := schema.Properties["state"] + if state.Description != "状态\n状态值" || len(state.EnumDescriptions) != 2 || state.Example != json.Number("1") { + t.Fatalf("%+v", state) + } + dir := t.TempDir() + entry := filepath.Join(dir, "demo.api") + writeFile(t, entry, source) + if _, err = FormatProject(entry, true); err == nil { + t.Fatal("check accepted drift") + } + if readFile(t, entry) != source { + t.Fatal("check wrote source") + } + if _, err = FormatProject(entry, false); err != nil { + t.Fatal(err) + } + if _, err = FormatProject(entry, true); err != nil { + t.Fatal(err) + } + project := filepath.Join(t.TempDir(), "demo") + if _, err = Generate(Options{API: entry, Dir: project, Module: "example.com/docs", FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) + } + dto := readFile(t, filepath.Join(project, "internal/types/"+dtoFile("demo.api:1:1"))) + if !strings.Contains(dto, "// 状态值") || !strings.Contains(dto, `description:"状态\n状态值"`) { + t.Fatal(dto) + } + // Invalid examples fail before project output is created. + bad := strings.Replace(source, "example:\"1\"", "example:\"9\"", 1) + writeFile(t, entry, bad) + if _, err = FormatProject(entry, false); err == nil { + t.Fatal("invalid enum example accepted") + } + if data, _ := os.ReadFile(entry); string(data) != bad { + t.Fatal("invalid source changed") + } +} + +func TestFormatAlignsFieldsAndPreservesTokens(t *testing.T) { + source := []byte("type Req {\n Name string `json:\"name\"` // 名称\n Sequence int64 `json:\"sequence\"` // 编号\n}\n") + out, err := Format(source) + if err != nil { + t.Fatal(err) + } + again, err := Format(out) + if err != nil || !bytes.Equal(out, again) { + t.Fatal("alignment not idempotent", err) + } + lines := strings.Split(string(out), "\n") + if strings.Index(lines[1], "`") != strings.Index(lines[2], "`") || strings.Index(lines[1], "//") != strings.Index(lines[2], "//") { + t.Fatal("fields not aligned", string(out)) + } + for _, literal := range []string{"`json:\"name\"`", "`json:\"sequence\"`", "// 名称", "// 编号"} { + if !bytes.Contains(out, []byte(literal)) { + t.Fatal("literal changed", literal) + } + } +} + +func TestRequestPolicyCannotCreateGETBody(t *testing.T) { + for _, source := range []string{ + "service demo { @handler get\n @request(body: required)\n get /x }", + "@app(request: {\"body\":\"optional\"})\n service demo { @handler get\n get /x }", + } { + spec, err := parseText(t, source) + if err == nil { + _, err = Compile(spec) + } + if err == nil { + t.Fatal("GET body policy accepted") + } + } +} +func TestFormatLiteralPreservation(t *testing.T) { + source := []byte("type Demo {\n Body string `json:\"body\"\ndescription:\"literal space\"`\n}\n") + out, err := Format(source) + if err != nil { + t.Fatal(err) + } + if !bytes.Contains(out, []byte("`json:\"body\"\ndescription:\"literal space\"`")) { + t.Fatal(string(out)) + } + for _, src := range []string{"type X {", "}", "info (title:\"unterminated)"} { + if _, err := Format([]byte(src)); err == nil { + t.Fatal("invalid input accepted") + } + } +} diff --git a/cmd/go-omni/apigen/format.go b/cmd/go-omni/apigen/format.go new file mode 100644 index 0000000..eb34bff --- /dev/null +++ b/cmd/go-omni/apigen/format.go @@ -0,0 +1,207 @@ +package apigen + +import ( + "bytes" + "errors" + "fmt" + "os" + "path/filepath" + "regexp" + "sort" + "strings" + "text/scanner" + "unicode/utf8" +) + +// Format normalizes indentation and blank lines without rewriting tokens, +// comments, quoted metadata, struct tags or multiline literal contents. +func Format(source []byte) ([]byte, error) { + if len(source) > 1<<20 { + return nil, errors.New("api: source size limit exceeded") + } + var scan scanner.Scanner + scan.Init(bytes.NewReader(source)) + scan.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments + var scanErr error + scan.Error = func(s *scanner.Scanner, msg string) { + if scanErr == nil { + scanErr = fmt.Errorf("%s: %s", s.Position, msg) + } + } + depths := map[int]int{} + protected := map[int]bool{} + depth := 0 + for k := scan.Scan(); k != scanner.EOF; k = scan.Scan() { + line := scan.Position.Line + value := scan.TokenText() + if k == '}' || k == ')' { + depth-- + if depth < 0 { + return nil, errors.New("api: unmatched closing delimiter") + } + } + if _, ok := depths[line]; !ok { + depths[line] = depth + } + // Preserve interior lines exactly, including the closing line of a literal. + count := strings.Count(value, "\n") + for n := 1; n <= count; n++ { + protected[line+n] = true + } + if k == '{' || k == '(' { + depth++ + } + } + if scanErr != nil { + return nil, scanErr + } + if depth != 0 { + return nil, errors.New("api: unclosed delimiter") + } + lines := strings.Split(string(source), "\n") + out := make([]string, 0, len(lines)) + outProtected := map[int]bool{} + blank := true + lastDepth := 0 + for i, line := range lines { + if protected[i+1] { + out = append(out, line) + outProtected[len(out)] = true + blank = false + continue + } + trimmed := strings.TrimSpace(line) + if trimmed == "" { + if !blank { + out = append(out, "") + blank = true + } + continue + } + d, ok := depths[i+1] + if ok { + lastDepth = d + } + out = append(out, strings.Repeat(" ", lastDepth)+trimmed) + blank = false + } + for len(out) > 0 && out[len(out)-1] == "" { + out = out[:len(out)-1] + } + alignFieldLines(out, outProtected) + return []byte(strings.Join(out, "\n") + "\n"), nil +} + +// Only complete, single-line fields are aligned. Literal bytes and multiline +// comment contents are never candidates, even when they resemble a field. +var singleLineField = regexp.MustCompile("^([ \\t]*)([A-Z][A-Za-z0-9_]*)[ \\t]+([^ \\t`]+)[ \\t]+(`[^`]*`)[ \\t]*(//.*)?$") + +func alignFieldLines(lines []string, protected map[int]bool) { + for start := 0; start < len(lines); { + var fields [][]string + widths := [3]int{} + end := start + for end < len(lines) && !protected[end+1] { + parts := singleLineField.FindStringSubmatch(lines[end]) + if parts == nil || (len(fields) > 0 && parts[1] != fields[0][1]) { + break + } + for col := range widths { + if n := utf8.RuneCountInString(parts[col+2]); n > widths[col] { + widths[col] = n + } + } + fields = append(fields, parts) + end++ + } + for i, parts := range fields { + line := parts[1] + for col := range widths { + line += parts[col+2] + if col < 2 || parts[5] != "" { + line += strings.Repeat(" ", widths[col]-utf8.RuneCountInString(parts[col+2])+1) + } + } + lines[start+i] = line + parts[5] + } + if end == start { + end++ + } + start = end + } +} + +// FormatProject validates the full contract before writing any local source. +// Check mode is read-only and reports every file needing formatting. +func FormatProject(entry string, check bool) ([]string, error) { + spec, err := Parse(entry) + if err != nil { + return nil, err + } + if _, err = Compile(spec); err != nil { + return nil, err + } + real, err := filepath.EvalSymlinks(entry) + if err != nil { + return nil, err + } + root := filepath.Dir(real) + names := make([]string, 0, len(spec.Sources)) + for name := range spec.Sources { + if strings.HasPrefix(name, "@") || filepath.Ext(name) != ".api" { + continue + } + names = append(names, name) + } + sort.Strings(names) + next := map[string][]byte{} + var changed []string + for _, name := range names { + data, err := Format(spec.Sources[name]) + if err != nil { + return nil, fmt.Errorf("%s: %w", name, err) + } + if !bytes.Equal(data, spec.Sources[name]) { + next[name] = data + changed = append(changed, name) + } + } + if check { + if len(changed) > 0 { + return changed, fmt.Errorf("API format drift: %s", strings.Join(changed, ", ")) + } + return changed, nil + } + for _, name := range changed { + path := filepath.Join(root, filepath.FromSlash(name)) + old, err := os.ReadFile(path) + if err != nil { + return changed, err + } + if !bytes.Equal(old, spec.Sources[name]) { + return changed, fmt.Errorf("API source changed during formatting: %s", name) + } + stat, err := os.Stat(path) + if err != nil { + return changed, err + } + f, err := os.CreateTemp(filepath.Dir(path), ".omni-format-*") + if err != nil { + return changed, err + } + tmp := f.Name() + err = f.Chmod(stat.Mode().Perm()) + if err == nil { + _, err = f.Write(next[name]) + } + err = errors.Join(err, f.Close()) + if err == nil { + err = os.Rename(tmp, path) + } + _ = os.Remove(tmp) + if err != nil { + return changed, err + } + } + return changed, nil +} diff --git a/cmd/go-omni/apigen/generate.go b/cmd/go-omni/apigen/generate.go index d377776..84b8004 100644 --- a/cmd/go-omni/apigen/generate.go +++ b/cmd/go-omni/apigen/generate.go @@ -19,10 +19,11 @@ import ( ) const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" -const minimumVersion = "v0.13.0-preview" +const minimumVersion = "v0.14.0-preview" const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n" type Options struct { + lockHeld bool Database, Storage string API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string Check bool @@ -95,6 +96,9 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { return Output{}, err } o.Dir = directory + if err = checkRuntimeExtension(directory); err != nil { + return Output{}, err + } for _, r := range c.Spec.Routes { if !r.Manual { path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go") @@ -109,6 +113,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { return Output{}, err } seeds := map[string][]byte{} + requiredVersion := "" moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod")) if readErr == nil { mf, err := modfile.Parse("go.mod", moduleData, nil) @@ -120,7 +125,6 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { } o.Module = mf.Module.Mod.Path found := false - requiredVersion := "" for _, r := range mf.Require { if r.Mod.Path == frameworkModule { found = true @@ -141,13 +145,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { if !filepath.IsAbs(path) { path = filepath.Join(directory, path) } - if _, err := os.Stat(filepath.Join(path, "lifecycle", "controller.go")); err == nil { + if _, err := os.Stat(filepath.Join(path, "lifecycle", "runtime.go")); err == nil { localCurrent = true } } } if !localCurrent { - return Output{}, fmt.Errorf("existing framework dependency predates dependency shutdown and configurable endpoint support; use %s or a current local replacement", minimumVersion) + return Output{}, fmt.Errorf("existing framework dependency predates the generated Runtime, request policy and structured MySQL support; use %s or a current local replacement", minimumVersion) } } if o.FrameworkDir != "" { @@ -191,7 +195,16 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { // from its own api/ tree, including all imports. entry := "api/" + c.Spec.Entry for name, data := range c.Spec.Sources { - seeds["api/"+name] = data + if !strings.HasPrefix(name, "@") { + seeds["api/"+name] = data + } + } + if len(c.Spec.SharedRoots) > 0 { + raw, e := sharedSeeds(c.Spec, directory) + if e != nil { + return Output{}, e + } + seeds["api/"+importsFile] = raw } if o.API != "" { source, _ := filepath.Abs(o.API) @@ -200,6 +213,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { rel, e := filepath.Rel(targetReal, sourceReal) if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) { entry = filepath.ToSlash(rel) + delete(seeds, "api/"+importsFile) for name := range c.Spec.Sources { delete(seeds, "api/"+name) } @@ -208,7 +222,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { // Source collisions must never leave a copied contract different from the // generated code. On updates, use the developer-owned API in the project. for name, data := range seeds { - if strings.HasSuffix(name, ".api") { + if strings.HasSuffix(name, ".api") || filepath.Base(name) == catalogDocumentFile || filepath.Base(name) == importsFile { current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name))) if e == nil && string(current) != string(data) { return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name) @@ -227,7 +241,12 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) { } } } - return apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec)) + out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld) + if requiredVersion != "" && requiredVersion != minimumVersion { + out.Diagnostics = append(out.Diagnostics, Diagnostic{Severity: "review", Code: "frameworkVersion", Path: "go.mod", Before: requiredVersion, After: minimumVersion, Message: "project dependency differs from this CLI template version; use matching CLI/framework versions, or verify the explicit local replacement"}) + sortDiagnostics(out.Diagnostics) + } + return out, err } func moduleFile(o Options) ([]byte, error) { @@ -336,6 +355,13 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map } types := "package types\n\n" + needsUpload := false + for _, b := range c.Bindings { + needsUpload = needsUpload || b.HasMultipart() + } + if needsUpload { + types += fmt.Sprintf("import %q\n", frameworkModule+"/contract") + } if len(c.Spec.Scalars) > 0 { types += "import(" for i, d := range c.Spec.Scalars { @@ -350,12 +376,28 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map registry += "}}\n" put(generated, "internal/types/scalarsGen.go", generatedHeader+registry) } + if needsUpload { + types += "type File = contract.File\n" + } + bySource := map[string]string{} for _, t := range c.Spec.Types { - types += "type " + t.Name + " struct {\n" - for _, f := range t.Fields { - types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag) + name := dtoFile(t.Position) + if bySource[name] == "" { + bySource[name] = "package types\n\n" } - types += "}\n" + text := "type " + t.Name + " struct {\n" + for _, f := range t.Fields { + if f.Description != "" { + for _, line := range strings.Split(f.Description, "\n") { + text += "// " + line + "\n" + } + } + text += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, fieldTag(f)) + } + bySource[name] += text + "}\n" + } + for name, text := range bySource { + put(generated, "internal/types/"+name, generatedHeader+text) } put(generated, "internal/types/typesGen.go", generatedHeader+types) put(generated, "docs/openapi.json", string(c.Document)+"\n") @@ -408,8 +450,8 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map continue } hasAutomatic = true - needsTypes = needsTypes || r.Request != "" || r.Response != "" - needsBinder = needsBinder || (r.Request != "" && !r.CustomBinding) + needsTypes = needsTypes || r.Request != "" || responseGoType(r.Response) != "" + needsBinder = needsBinder || !r.CustomBinding } text += fmt.Sprintf("%q;%q;", frameworkModule+"/httpx", o.Module+"/internal/svc") if needsTypes { @@ -442,12 +484,18 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Manual.go", fmt.Sprintf("package %s\nimport(%q;apierrors %q)\n// %s owns raw/multipart binding and response output. h.deps holds project dependencies.\n// This file is preserved on regeneration.\nfunc(h *Handler)%s(c *httpx.Context)error{return apierrors.New(501,\"NOT_IMPLEMENTED\",\"manual handler is not implemented\")}\n", packageName, frameworkModule+"/httpx", frameworkModule+"/errors", name, name)) continue } - if r.Request != "" && !r.CustomBinding { + if !r.CustomBinding { scalarArgs := "" if len(c.Spec.Scalars) > 0 { scalarArgs = "types.ContractScalars()..." } - text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s](%s)\n", lower(r.Handler), r.Request, scalarArgs) + requestType := "struct{}" + if r.Request != "" { + requestType = "types." + r.Request + } else { + scalarArgs = "" + } + text += fmt.Sprintf("var %sRequest = contract.MustNew[%s](%s)\n", lower(r.Handler), requestType, scalarArgs) } text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name) call := "c.Context()" @@ -455,16 +503,32 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map if r.CustomBinding { text += fmt.Sprintf("var req types.%s\nif err:=h.bind%s(c,&req);err!=nil{return err}\n", r.Request, name) put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Bind.go", fmt.Sprintf("package %s\nimport(%q;%q;apierrors %q)\n// bind%s is project-owned; repeated generation preserves it.\nfunc(h *Handler)bind%s(c *httpx.Context,req *types.%s)error{return apierrors.New(501,\"BINDING_NOT_IMPLEMENTED\",\"custom binding is not implemented\")}\n", packageName, frameworkModule+"/httpx", o.Module+"/internal/types", frameworkModule+"/errors", name, name, r.Request)) + } else if r.BoundMultipart { + text += fmt.Sprintf("var req types.%s\ncleanup,err:=%sRequest.BindMultipart(c.Request(),httpx.RouteParams(c.Context()),&req,contract.MultipartOptions{MaxBodyBytes:%d,MaxParts:%d,MaxFieldBytes:%d,MaxFileBytes:%d,Policy:%s});if err!=nil{return err};defer cleanup()\n", r.Request, lower(r.Handler), r.MaxBodyBytes, r.MaxParts, r.MaxFieldBytes, r.MaxFileBytes, policyLiteral(r.RequestPolicy)) } else { limit := "" if r.MaxBodyBytes > 0 { limit = fmt.Sprintf(",%d", r.MaxBodyBytes) } - text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req%s);err!=nil{return err}\n", r.Request, lower(r.Handler), limit) + policy := r.RequestPolicy + if r.BodyMode == "raw" || r.BodyMode == "multipart" { + policy.Body = "auto" + } + text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.BindWithPolicy(c.Request(),httpx.RouteParams(c.Context()),&req,%s%s);err!=nil{return err}\n", r.Request, lower(r.Handler), policyLiteral(policy), limit) } call += ", &req" + } else { + limit := "" + if r.MaxBodyBytes > 0 { + limit = fmt.Sprintf(",%d", r.MaxBodyBytes) + } + policy := r.RequestPolicy + if r.BodyMode == "raw" || r.BodyMode == "multipart" { + policy.Body = "auto" + } + text += fmt.Sprintf("if err:=%sRequest.BindWithPolicy(c.Request(),httpx.RouteParams(c.Context()),&struct{}{},%s%s);err!=nil{return err}\n", lower(r.Handler), policyLiteral(policy), limit) } - streamingBody := r.BodyMode == "raw" || r.BodyMode == "multipart" + streamingBody := streamsBody(r) if streamingBody { text += fmt.Sprintf("body,err:=httpx.NewBody(c.Request(),%q);if err!=nil{return err}\n", r.BodyMode) call += ",body" @@ -475,8 +539,15 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map text += "if err:=body.Complete();err!=nil{if resp!=nil{_ =resp.Close()};return err}\n" } text += fmt.Sprintf("return c.WriteStream(%d,%q,resp)\n}\n", r.Status, r.Produces) - } else if r.Response != "" { - text += fmt.Sprintf("resp,err:=h.logic.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name) + } else if responseGoType(r.Response) != "" { + text += fmt.Sprintf("resp,err:=h.logic.%s(%s);if err!=nil{return err};\n", name, call) + kind, _ := responseShape(r.Response) + if kind == "object" { + text += fmt.Sprintf("if resp==nil{return fmt.Errorf(%q)}\n", name+" returned a nil response") + } + if kind == "array" { + text += "if resp==nil{resp=make(" + responseGoType(r.Response) + ",0)}\n" + } if streamingBody { text += "if err:=body.Complete();err!=nil{return err}\n" } @@ -487,10 +558,15 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map if streamingBody { text += "if err:=body.Complete();err!=nil{return err};" } - text += "return c.NoContent()\n}\n" + if r.Response == "null" { + method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status] + text += "return c." + method + "(nil)\n}\n" + } else { + text += "return c.NoContent()\n}\n" + } } service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors") - if r.Request != "" || r.Response != "" { + if r.Request != "" || responseGoType(r.Response) != "" { service += fmt.Sprintf("%q;", o.Module+"/internal/types") } if streamingBody || r.Produces != "" { @@ -503,7 +579,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}" } else { service += "// TODO: implement business behavior and use s.deps for dependencies.\n" - if r.Response != "" || r.Produces != "" { + if responseGoType(r.Response) != "" || r.Produces != "" { service += "return nil, " } else { service += "return " @@ -549,7 +625,11 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map return nil, nil, err } for name, data := range infra { - seeds[name] = data + if strings.HasSuffix(name, "Gen.go") { + generated[name] = append([]byte(generatedHeader), data...) + } else { + seeds[name] = data + } } if o.Storage == "local" { for _, r := range c.Spec.Routes { @@ -564,7 +644,7 @@ func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map } func hasResponse(routes []Route) bool { for _, r := range routes { - if r.Response != "" && !r.Manual && r.Produces == "" { + if kind, _ := responseShape(r.Response); r.Response != "" && kind == "object" && !r.Manual && r.Produces == "" { return true } } @@ -575,17 +655,41 @@ func signature(r Route) string { if r.Request != "" { s += ",req *types." + r.Request } - if r.BodyMode == "raw" || r.BodyMode == "multipart" { + if streamsBody(r) { s += ",body *httpx.Body" } s += ") " if r.Produces != "" { return s + "(*httpx.Stream,error)" } - if r.Response != "" { - s += "(*types." + r.Response + ",error)" + if result := responseGoType(r.Response); result != "" { + s += "(" + result + ",error)" } else { s += "error" } return s } + +func streamsBody(r Route) bool { + return r.BodyMode == "raw" || (r.BodyMode == "multipart" && !r.BoundMultipart) +} + +func dtoFile(position string) string { + source := position + for i := 0; i < 2; i++ { + if idx := strings.LastIndex(source, ":"); idx >= 0 { + source = source[:idx] + } + } + stem := strings.TrimSuffix(filepath.Base(source), ".api") + var b strings.Builder + for _, r := range stem { + if r >= 'a' && r <= 'z' || r >= 'A' && r <= 'Z' || r >= '0' && r <= '9' { + b.WriteRune(r) + } + } + if b.Len() == 0 { + b.WriteString("contract") + } + return "dto" + upper(b.String()) + digest([]byte(source))[:12] + "Gen.go" +} diff --git a/cmd/go-omni/apigen/logicCheck.go b/cmd/go-omni/apigen/logicCheck.go index 51f5359..92f732c 100644 --- a/cmd/go-omni/apigen/logicCheck.go +++ b/cmd/go-omni/apigen/logicCheck.go @@ -42,7 +42,7 @@ func checkLogic(root, module string, contract *contractSnapshot, seeds map[strin typeExtensions := false if entries, err := os.ReadDir(filepath.Join(root, "internal/types")); err == nil { for _, e := range entries { - if sourceFile(e.Name()) && e.Name() != "typesGen.go" && e.Name() != "scalarsGen.go" { + if sourceFile(e.Name()) && !ownedPath("internal/types/"+e.Name()) { typeExtensions = true } } diff --git a/cmd/go-omni/apigen/managed_test.go b/cmd/go-omni/apigen/managed_test.go index a1ba2fa..ac6e94e 100644 --- a/cmd/go-omni/apigen/managed_test.go +++ b/cmd/go-omni/apigen/managed_test.go @@ -82,7 +82,7 @@ func TestManagedGeneratedProject(t *testing.T) { writeFile(t, configPath, strings.Replace(readFile(t, configPath), "type Config struct {", "type Config struct {\n FailInit bool `json:\"failInit\"`\n ShutdownInit bool `json:\"shutdownInit\"`", 1)) svcPath := filepath.Join(dir, "internal/svc/serviceContext.go") source := strings.Replace(readFile(t, svcPath), "type ServiceContext struct {", "type ServiceContext struct {\n Client *Client", 1) - source = strings.Replace(source, "return c, nil", "if err=initClient(ctx,c);err!=nil{return nil,err}\n return c,nil", 1) + source = strings.Replace(source, "return nil", "return initClient(ctx,c)", 1) writeFile(t, svcPath, source) // Independent tooling stays outside generator ownership. writeFile(t, filepath.Join(dir, "cmd/migrate/main.go"), "package main\nfunc main(){}\n") diff --git a/cmd/go-omni/apigen/parse.go b/cmd/go-omni/apigen/parse.go index 61b526a..296bed8 100644 --- a/cmd/go-omni/apigen/parse.go +++ b/cmd/go-omni/apigen/parse.go @@ -5,6 +5,8 @@ package apigen import ( "errors" "fmt" + "git.yuntuo.hk/k2safe/go-omni/contract" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" "io" "os" "path/filepath" @@ -14,6 +16,10 @@ import ( ) type Spec struct { + DocumentCatalog *apierrors.Catalog + SharedRoots map[string]SharedRoot + SourcePaths map[string]string + RequestPolicy contract.RequestPolicy DefaultResponse string appDeclared bool Title, Version, Service, Description string @@ -32,8 +38,12 @@ type Type struct { Fields []Field Position string } -type Field struct{ Name, Type, Tag string } +type Field struct{ Name, Type, Tag, Description string } type Route struct { + BoundMultipart bool + MaxParts int + MaxFieldBytes, MaxFileBytes int64 + RequestPolicy contract.RequestPolicy Manual, CustomBinding bool BodyMode, Produces string MaxBodyBytes int64 @@ -51,6 +61,7 @@ type Route struct { } type token struct { text, position string + comment string kind rune space bool } @@ -71,11 +82,15 @@ func Parse(path string) (*Spec, error) { return nil, err } root := filepath.Dir(entry) - spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)} + roots, err := readSharedRoots(root) + if err != nil { + return nil, err + } + spec := &Spec{Version: "1.0.0", SharedRoots: roots, SourcePaths: map[string]string{}, Sources: map[string][]byte{}, Entry: filepath.Base(entry)} visiting, visited := map[string]bool{}, map[string]bool{} total := 0 - var visit func(string, int) error - visit = func(name string, depth int) error { + var visit func(string, string, int) error + visit = func(name, alias string, depth int) error { if depth > 32 || len(visited)+len(visiting) >= 64 { return errors.New("api: import limit exceeded") } @@ -83,9 +98,16 @@ func Parse(path string) (*Spec, error) { if err != nil { return err } - rel, err := filepath.Rel(root, real) - if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { - return errors.New("api: imports must stay within the entry API directory") + sourceRoot := root + if alias != "" { + sourceRoot = roots[alias].Path + } + rel, err := sourceWithin(sourceRoot, real) + if err != nil { + return err + } + if alias != "" { + rel = "@" + alias + "/" + rel } if filepath.Ext(real) != ".api" { return errors.New("api: expected .api file") @@ -120,18 +142,37 @@ func Parse(path string) (*Spec, error) { return err } spec.Sources[filepath.ToSlash(rel)] = data + spec.SourcePaths[filepath.ToSlash(rel)] = real for _, imp := range p.imports { if filepath.IsAbs(imp) || strings.Contains(imp, "\\") { return errors.New("api: imports must use relative slash paths") } - if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil { + nextAlias := alias + next := filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)) + if strings.HasPrefix(imp, "@") { + var sub string + var ok bool + nextAlias, sub, ok = strings.Cut(strings.TrimPrefix(imp, "@"), "/") + declared, found := roots[nextAlias] + if !ok || !found || sub == "" || strings.HasPrefix(sub, "/") || strings.HasPrefix(sub, "../") || filepath.ToSlash(filepath.Clean(sub)) != sub { + return errors.New("api: import needs a declared @root and a clean relative path") + } + next = filepath.Join(declared.Path, filepath.FromSlash(sub)) + } + if err = visit(next, nextAlias, depth+1); err != nil { return err } } visited[real] = true return nil } - if err = visit(entry, 0); err != nil { + if err = visit(entry, "", 0); err != nil { + return nil, err + } + if err = readDocumentCatalog(root, spec); err != nil { + return nil, err + } + if err = validateSharedPins(spec); err != nil { return nil, err } if spec.Service == "" || len(spec.Routes) == 0 { @@ -146,7 +187,7 @@ func lex(name, source string) ([]token, error) { var s scanner.Scanner s.Init(strings.NewReader(source)) s.Filename = name - s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments + s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments s.Whitespace ^= 1 << '\n' var scanErr error s.Error = func(s *scanner.Scanner, message string) { @@ -156,9 +197,29 @@ func lex(name, source string) ([]token, error) { } var out []token end := 0 + pending := "" + lastLine := 0 + lastToken := -1 for { kind := s.Scan() - out = append(out, token{s.TokenText(), s.Position.String(), kind, s.Position.Offset > end}) + if kind == scanner.Comment { + comment := cleanComment(s.TokenText()) + if lastToken >= 0 && s.Position.Line == lastLine { + out[lastToken].comment = joinComment(out[lastToken].comment, comment) + } else { + pending = joinComment(pending, comment) + } + end = s.Position.Offset + len(s.TokenText()) + continue + } + current := token{text: s.TokenText(), position: s.Position.String(), kind: kind, space: s.Position.Offset > end} + if kind != '\n' { + current.comment = pending + pending = "" + lastLine = s.Position.Line + lastToken = len(out) + } + out = append(out, current) end = s.Position.Offset + len(s.TokenText()) if kind == scanner.EOF { break @@ -315,8 +376,20 @@ func (p *parser) parse() error { if err != nil { return err } - if len(m) != 1 || m["defaultResponse"] == "" { - return p.err("@app requires only defaultResponse: TemplateName") + if len(m) == 0 { + return p.err("@app requires defaultResponse or request") + } + for key, value := range m { + switch key { + case "defaultResponse": + case "request": + p.spec.RequestPolicy, err = decodeRequestPolicy(value) + if err != nil { + return err + } + default: + return p.err("unsupported @app property " + key) + } } p.spec.DefaultResponse = m["defaultResponse"] p.spec.appDeclared = true @@ -422,6 +495,7 @@ func (p *parser) parseType() error { } typ := Type{Name: name, Position: pos} for p.skip(); p.peek() != "}"; p.skip() { + description := p.tokens[p.at].comment name, err := p.ident() if err != nil { return err @@ -444,11 +518,13 @@ func (p *parser) parseType() error { if p.tokens[p.at].kind != scanner.RawString { return p.err("field source tag required") } - tag, err := strconv.Unquote(p.next().text) + tagToken := p.next() + description = joinComment(description, tagToken.comment) + tag, err := strconv.Unquote(tagToken.text) if err != nil { return err } - typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag}) + typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag, Description: description}) } p.next() p.spec.Types = append(p.spec.Types, typ) @@ -496,6 +572,12 @@ func (p *parser) parseService(server map[string]string) error { } seen[annotation] = true switch annotation { + case "request": + var props map[string]string + props, e = p.metadata() + if e == nil { + route.RequestPolicy, e = parseRequestPolicy(props) + } case "handlerMode": mode, err := p.ident() if err != nil { @@ -589,9 +671,48 @@ func (p *parser) routeType() (string, error) { if err := p.expect("("); err != nil { return "", err } + prefix := "" + if p.peek() == "[" { + p.next() + if err := p.expect("]"); err != nil { + return "", err + } + prefix = "[]" + } else if p.peek() == "*" { + p.next() + prefix = "*" + } name, err := p.ident() if err != nil { return "", err } - return name, p.expect(")") + return prefix + name, p.expect(")") +} + +func joinComment(a, b string) string { + if a == "" { + return b + } + if b == "" { + return a + } + return a + "\n" + b +} +func cleanComment(s string) string { + if strings.HasPrefix(s, "//") { + return strings.TrimSpace(strings.TrimPrefix(s, "//")) + } + s = strings.TrimSuffix(strings.TrimPrefix(s, "/*"), "*/") + lines := strings.Split(s, "\n") + for i, line := range lines { + lines[i] = strings.TrimSpace(strings.TrimPrefix(strings.TrimSpace(line), "*")) + } + return strings.TrimSpace(strings.Join(lines, "\n")) +} +func fieldTag(f Field) string { + tags, err := contract.ParseTags(f.Tag) + if err != nil || f.Description == "" || tags["description"] != "" { + return f.Tag + } + return f.Tag + " description:" + strconv.Quote(f.Description) } diff --git a/cmd/go-omni/apigen/requestOptions.go b/cmd/go-omni/apigen/requestOptions.go index 5ed7e1d..4b03c9c 100644 --- a/cmd/go-omni/apigen/requestOptions.go +++ b/cmd/go-omni/apigen/requestOptions.go @@ -1,9 +1,11 @@ package apigen import ( + "encoding/json" "errors" "fmt" "git.yuntuo.hk/k2safe/go-omni/contract" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" "mime" "strconv" "strings" @@ -16,6 +18,21 @@ func (p *parser) parseBody(r *Route) error { } for key, value := range props { switch key { + case "maxParts": + r.MaxParts, err = strconv.Atoi(value) + if err != nil || r.MaxParts < 1 || r.MaxParts > 10000 { + return p.err("maxParts must be 1..10000") + } + case "maxFieldBytes", "maxFileBytes": + limit, e := strconv.ParseInt(value, 10, 64) + if e != nil || limit < 1 || limit > 1<<30 { + return p.err("part limits must be 1..1073741824 bytes") + } + if key == "maxFieldBytes" { + r.MaxFieldBytes = limit + } else { + r.MaxFileBytes = limit + } case "mode": r.BodyMode = value case "maxBytes": @@ -33,6 +50,16 @@ func (p *parser) parseBody(r *Route) error { return nil } func validateRouteExtensions(r Route) error { + if (r.MaxParts > 0 || r.MaxFieldBytes > 0 || r.MaxFileBytes > 0) && r.BodyMode != "multipart" { + return errors.New("part limits require multipart mode") + } + + if err := r.RequestPolicy.Validate(); err != nil { + return err + } + if (r.BodyMode == "raw" || r.BodyMode == "multipart") && r.RequestPolicy.Body != "" && r.RequestPolicy.Body != "auto" { + return errors.New("raw/multipart cannot declare a JSON body policy") + } if r.BodyMode != "" && r.BodyMode != "json" && r.BodyMode != "raw" && r.BodyMode != "multipart" { return errors.New("body mode must be json, raw or multipart") } @@ -42,6 +69,9 @@ func validateRouteExtensions(r Route) error { if r.MaxBodyBytes > 0 && (r.Method == "get" || r.Method == "head") { return errors.New("GET/HEAD cannot declare a body limit") } + if (r.Method == "get" || r.Method == "head") && r.RequestPolicy.Body != "" && r.RequestPolicy.Body != "auto" && r.RequestPolicy.Body != "none" { + return errors.New("GET/HEAD cannot declare a JSON body policy") + } if r.Produces != "" { media, params, err := mime.ParseMediaType(r.Produces) if err != nil || len(params) != 0 || media != r.Produces || strings.Contains(media, "*") { @@ -58,6 +88,23 @@ func validateRouteExtensions(r Route) error { } func decorateRequestOptions(op map[string]any, r Route) { + if r.BodyMode == "raw" || r.BodyMode == "multipart" { + r.RequestPolicy.Body = "auto" + } + op["x-omni-request-policy"] = r.RequestPolicy + if r.BodyMode == "multipart" { + limits, _ := (contract.MultipartOptions{MaxBodyBytes: r.MaxBodyBytes, MaxParts: r.MaxParts, MaxFieldBytes: r.MaxFieldBytes, MaxFileBytes: r.MaxFileBytes}).Normalized() + op["x-omni-multipart-limits"] = map[string]any{"maxBytes": limits.MaxBodyBytes, "maxParts": limits.MaxParts, "maxFieldBytes": limits.MaxFieldBytes, "maxFileBytes": limits.MaxFileBytes} + } + if r.BodyMode != "raw" && r.BodyMode != "multipart" { + if body, ok := op["requestBody"].(map[string]any); ok && r.RequestPolicy.Body == "optional" { + body["required"] = false + } + if _, ok := op["requestBody"]; !ok && (r.RequestPolicy.Body == "emptyObject" || r.RequestPolicy.Body == "required" || r.RequestPolicy.Body == "optional") { + op["requestBody"] = map[string]any{"required": r.RequestPolicy.Body != "optional", "content": map[string]any{"application/json": map[string]any{"schema": map[string]any{"type": "object", "additionalProperties": false}}}} + } + } + if r.Manual { op["x-omni-handler-mode"] = "manual" } @@ -87,3 +134,21 @@ func decorateRequestOptions(op map[string]any, r Route) { op["responses"].(map[string]any)[strconv.Itoa(r.Status)] = map[string]any{"description": "Success", "content": map[string]any{r.Produces: map[string]any{"schema": schema}}} } } + +func parseRequestPolicy(props map[string]string) (contract.RequestPolicy, error) { + data, err := json.Marshal(props) + if err != nil { + return contract.RequestPolicy{}, err + } + return decodeRequestPolicy(string(data)) +} +func decodeRequestPolicy(raw string) (contract.RequestPolicy, error) { + var p contract.RequestPolicy + if err := apierrors.DecodeCatalogJSON([]byte(raw), &p); err != nil { + return p, fmt.Errorf("request policy: %w", err) + } + return p, p.Validate() +} +func policyLiteral(p contract.RequestPolicy) string { + return fmt.Sprintf("contract.RequestPolicy{UnknownQuery:%q,UTF8:%q,Charset:%q,Body:%q}", p.UnknownQuery, p.UTF8, p.Charset, p.Body) +} diff --git a/cmd/go-omni/apigen/responseShape.go b/cmd/go-omni/apigen/responseShape.go new file mode 100644 index 0000000..c7cea59 --- /dev/null +++ b/cmd/go-omni/apigen/responseShape.go @@ -0,0 +1,54 @@ +package apigen + +import ( + "fmt" + "git.yuntuo.hk/k2safe/go-omni/contract" + "git.yuntuo.hk/k2safe/go-omni/openapi" + "strings" +) + +// Response roots deliberately allow named DTOs, []DTO, *DTO and null. Keeping +// these four shapes explicit makes nil/empty behavior uniform across encoders. +func responseShape(value string) (kind, name string) { + if value == "null" { + return "null", "" + } + if strings.HasPrefix(value, "[]") { + return "array", strings.TrimPrefix(value, "[]") + } + if strings.HasPrefix(value, "*") { + return "nullable", strings.TrimPrefix(value, "*") + } + return "object", value +} +func responseGoType(value string) string { + kind, name := responseShape(value) + if kind == "null" || value == "" { + return "" + } + if kind == "array" { + return "[]types." + name + } + return "*types." + name +} +func responseDataSchema(value string, bindings map[string]*contract.Binder) (*openapi.Schema, error) { + kind, name := responseShape(value) + if kind == "null" { + return &openapi.Schema{Type: "object", Nullable: true, Enum: []any{nil}}, nil + } + b, ok := bindings[name] + if !ok { + return nil, fmt.Errorf("unknown response DTO %q; use Name, []Name, *Name or null", name) + } + s, err := b.ResponseSchema() + if err != nil { + return nil, err + } + switch kind { + case "array": + return &openapi.Schema{Type: "array", Items: s}, nil + case "nullable": + s.Nullable = true + } + return s, nil +} diff --git a/cmd/go-omni/apigen/shared.go b/cmd/go-omni/apigen/shared.go new file mode 100644 index 0000000..e788fad --- /dev/null +++ b/cmd/go-omni/apigen/shared.go @@ -0,0 +1,150 @@ +package apigen + +import ( + "encoding/json" + "errors" + "fmt" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "io" + "os" + "path/filepath" + "sort" + "strings" +) + +const importsFile = "omni.imports.json" + +// SharedRoot explicitly authorizes reads from one canonical local directory. +// Version is a project-owned label; SHA256 optionally pins the imported source +// set, not a remote tag. The CLI never downloads or executes shared contracts. +type SharedRoot struct { + Path string `json:"path"` + Version string `json:"version,omitempty"` + SHA256 string `json:"sha256,omitempty"` +} +type sharedConfig struct { + Roots map[string]SharedRoot `json:"roots"` +} + +func readSharedRoots(root string) (map[string]SharedRoot, error) { + stat, err := os.Lstat(filepath.Join(root, importsFile)) + if errors.Is(err, os.ErrNotExist) { + return nil, nil + } + if err != nil { + return nil, err + } + if !stat.Mode().IsRegular() { + return nil, errors.New("api: shared root configuration must be a regular file") + } + f, err := os.Open(filepath.Join(root, importsFile)) + if errors.Is(err, os.ErrNotExist) { + return nil, nil + } + if err != nil { + return nil, err + } + defer f.Close() + raw, err := io.ReadAll(io.LimitReader(f, 65537)) + if err != nil { + return nil, err + } + if len(raw) > 65536 { + return nil, errors.New("api: shared root config too large") + } + var cfg sharedConfig + if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil { + return nil, err + } + if len(cfg.Roots) > 16 { + return nil, errors.New("api: at most 16 shared roots") + } + for name, r := range cfg.Roots { + if !lowerName.MatchString(name) || r.Path == "" || len(r.Version) > 128 { + return nil, errors.New("api: invalid shared root") + } + if r.SHA256 != "" && (len(r.SHA256) != 64 || strings.Trim(r.SHA256, "0123456789abcdef") != "") { + return nil, errors.New("api: shared sha256 must be 64 lowercase hex characters") + } + path := r.Path + if !filepath.IsAbs(path) { + path = filepath.Join(root, path) + } + r.Path, err = filepath.EvalSymlinks(path) + if err != nil { + return nil, err + } + r.Path, err = filepath.Abs(r.Path) + if err != nil { + return nil, err + } + stat, err := os.Stat(r.Path) + if err != nil { + return nil, err + } + if !stat.IsDir() { + return nil, errors.New("api: shared root must be a directory") + } + cfg.Roots[name] = r + } + return cfg.Roots, nil +} +func sourceWithin(root, name string) (string, error) { + real, err := filepath.EvalSymlinks(name) + if err != nil { + return "", err + } + rel, err := filepath.Rel(root, real) + if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) { + return "", errors.New("api: imports must stay within their declared contract root") + } + return filepath.ToSlash(rel), nil +} +func sharedHash(sources map[string][]byte, alias string) string { + values := map[string]string{} + for name, data := range sources { + if strings.HasPrefix(name, "@"+alias+"/") { + values[strings.TrimPrefix(name, "@"+alias+"/")] = digest(data) + } + } + raw, _ := json.Marshal(values) + return digest(raw) +} +func sourceMetadata(spec *Spec) map[string]any { + hashes := map[string]string{} + for name, data := range spec.Sources { + hashes[name] = digest(data) + } + shared := map[string]any{} + for alias, r := range spec.SharedRoots { + shared[alias] = map[string]string{"version": r.Version, "sha256": sharedHash(spec.Sources, alias)} + } + return map[string]any{"files": hashes, "shared": shared} +} +func sharedSeeds(spec *Spec, directory string) ([]byte, error) { + cfg := sharedConfig{Roots: map[string]SharedRoot{}} + for name, r := range spec.SharedRoots { + rel, err := filepath.Rel(filepath.Join(directory, "api"), r.Path) + if err != nil { + return nil, err + } + r.Path = filepath.ToSlash(rel) + cfg.Roots[name] = r + } + data, err := json.MarshalIndent(cfg, "", " ") + return append(data, '\n'), err +} +func validateSharedPins(spec *Spec) error { + names := make([]string, 0, len(spec.SharedRoots)) + for name := range spec.SharedRoots { + names = append(names, name) + } + sort.Strings(names) + for _, name := range names { + r := spec.SharedRoots[name] + if r.SHA256 != "" && r.SHA256 != sharedHash(spec.Sources, name) { + return fmt.Errorf("api: shared contract %s checksum differs (got %s)", name, sharedHash(spec.Sources, name)) + } + } + return nil +} diff --git a/cmd/go-omni/apigen/shared_test.go b/cmd/go-omni/apigen/shared_test.go new file mode 100644 index 0000000..0777cf7 --- /dev/null +++ b/cmd/go-omni/apigen/shared_test.go @@ -0,0 +1,96 @@ +package apigen + +import ( + "bytes" + "encoding/json" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestControlledSharedContracts(t *testing.T) { + root := t.TempDir() + shared := filepath.Join(root, "shared") + writeFile(t, filepath.Join(shared, "response.api"), "response Common (codeType: int32; successCode: 0; errorCode: 9999; dataMode: always)\ntype Item { Name string `json:\"name\"` }\n") + for _, service := range []string{"api", "id"} { + dir := filepath.Join(root, service) + entry := filepath.Join(dir, "api/service.api") + writeFile(t, entry, "import \"@common/response.api\"\n@app(defaultResponse: Common)\nservice demo {\n@handler get\nget /get returns (Item)\n}\n") + config := `{"roots":{"common":{"path":"../../shared","version":"1.0.0"}}}` + writeFile(t, filepath.Join(dir, "api/omni.imports.json"), config) + spec, err := Parse(entry) + if err != nil { + t.Fatal(err) + } + if len(spec.Sources) != 2 || len(spec.SourcePaths) != 2 { + t.Fatalf("%+v", spec) + } + pin := sharedHash(spec.Sources, "common") + config = strings.Replace(config, `"version":"1.0.0"`, `"version":"1.0.0","sha256":"`+pin+`"`, 1) + writeFile(t, filepath.Join(dir, "api/omni.imports.json"), config) + o := Options{Dir: dir, API: entry, Module: "example.com/" + service, FrameworkDir: frameworkRoot(t)} + if _, err = Generate(o); err != nil { + t.Fatal(err) + } + o.Check = true + if _, err = Generate(o); err != nil { + t.Fatal(err) + } + if _, err = os.Stat(filepath.Join(dir, "api/@common")); !os.IsNotExist(err) { + t.Fatal("shared contract copied into service") + } + document := readFile(t, filepath.Join(dir, "docs/openapi.json")) + if !strings.Contains(document, pin) || !strings.Contains(document, "@common/response.api") { + t.Fatal("missing provenance") + } + writeFile(t, filepath.Join(dir, "api/omni.imports.json"), strings.Replace(config, pin, strings.Repeat("0", 64), 1)) + if _, err = Parse(entry); err == nil { + t.Fatal("checksum mismatch accepted") + } + } + // An undeclared alias, traversal, symlink escape and cycle are all rejected. + for _, imp := range []string{"@missing/response.api", "@common/../outside.api", "../shared/response.api", "@common/escape.api", "@common/cycle.api"} { + local := filepath.Join(root, "consumer") + writeFile(t, filepath.Join(local, "omni.imports.json"), `{"roots":{"common":{"path":"../shared"}}}`) + writeFile(t, filepath.Join(root, "outside.api"), "type Outside {}\n") + _ = os.Symlink(filepath.Join(root, "outside.api"), filepath.Join(shared, "escape.api")) + writeFile(t, filepath.Join(shared, "cycle.api"), "import \"cycle.api\"\n") + entry := filepath.Join(local, "service.api") + writeFile(t, entry, "import "+strconvQuote(imp)+"\nservice demo{\n@handler get\nget /get\n}\n") + if _, err := Parse(entry); err == nil { + t.Fatal("unsafe import accepted", imp) + } + } +} +func strconvQuote(value string) string { data, _ := json.Marshal(value); return string(data) } +func TestCatalogDocumentExportDoesNotGenerateRuntimeRegistry(t *testing.T) { + dir := t.TempDir() + entry := filepath.Join(dir, "service.api") + writeFile(t, entry, "@app(defaultResponse: Common)\nresponse Common(codeType: int32; successCode: 0; errorCode: 9999; messageField: msg; dataMode: always)\nservice demo{\n@handler get\nget /get returns (null)\n}\n") + var exported bytes.Buffer + err := apierrors.ExportCatalog(&exported, apierrors.Catalog{DefaultStatus: 200, Definitions: map[string]apierrors.Definition{"TOO_LONG": {Code: 400002, Message: "{label}不能超过{max}个字符", Params: []apierrors.Parameter{{Name: "label", Type: "string", Example: "企业名称"}, {Name: "max", Type: "int", Example: 100}}}}}) + if err != nil { + t.Fatal(err) + } + writeFile(t, filepath.Join(dir, "omni.errors.json"), exported.String()) + spec, err := Parse(entry) + if err != nil { + t.Fatal(err) + } + c, err := Compile(spec) + if err != nil { + t.Fatal(err) + } + if len(c.Spec.Errors) != 0 || !strings.Contains(string(c.Document), "企业名称不能超过100个字符") { + t.Fatal(string(c.Document)) + } + target := filepath.Join(t.TempDir(), "catalog") + if _, err = Generate(Options{Dir: target, API: entry, Module: "example.com/catalog", FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) + } + if _, err = os.Stat(filepath.Join(target, "internal/apierrors/errorsGen.go")); !os.IsNotExist(err) { + t.Fatal("document metadata became a runtime error registry") + } +} diff --git a/cmd/go-omni/apigen/startup.go b/cmd/go-omni/apigen/startup.go index 008cbb4..ce6546a 100644 --- a/cmd/go-omni/apigen/startup.go +++ b/cmd/go-omni/apigen/startup.go @@ -20,8 +20,11 @@ import ( func NewApp(deps *svc.ServiceContext)(*omni.App,error){ if deps==nil || deps.Lifecycle==nil{return nil,fmt.Errorf("service context and lifecycle controller required; see PROJECT.md upgrade instructions")} cfg:=deps.Config.Server + cfg.LogInstance=deps.Logger for _,option:=range deps.AppOptions(){if option!=nil{option(&cfg)}} `, frameworkModule, frameworkModule+"/httpx", frameworkModule+"/config", frameworkModule+"/health", o.Module+"/internal/svc") + source += fmt.Sprintf("cfg.Request=contract.MergePolicy(cfg.Request,%#v)\n", spec.RequestPolicy) + source = strings.Replace(source, "\"fmt\"", "\"fmt\";"+fmt.Sprintf("%q", frameworkModule+"/contract"), 1) if spec.DefaultResponse != "" { for _, r := range spec.Responses { if r.Name == spec.DefaultResponse { diff --git a/cmd/go-omni/apigen/templates.go b/cmd/go-omni/apigen/templates.go index cd9d673..69b7d38 100644 --- a/cmd/go-omni/apigen/templates.go +++ b/cmd/go-omni/apigen/templates.go @@ -28,14 +28,10 @@ func run(ctx context.Context,path string)(runErr error){ cfg,err:=config.Load(path);if err!=nil{return err} logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err} defer func(){runErr=errors.Join(runErr,logger.Close())}() - logx.SetDefault(logger) deps,err:=svc.New(ctx,logger,cfg);if err!=nil{return err} // RunContext drains HTTP requests before returning. This scope owns dependencies // and closes them exactly once, before closing the borrowed logger. - defer func(){ - runErr=errors.Join(runErr,deps.Close()) - if cause:=deps.Lifecycle.Err();cause!=nil && !errors.Is(runErr,cause){runErr=errors.Join(runErr,cause)} - }() + defer func(){runErr=deps.Finish(runErr)}() app,err:=handler.NewApp(deps);if err!=nil{return err} defer func(){runErr=errors.Join(runErr,app.Close())}() logger.InfoContext(ctx,"service.starting","address",app.Config().Address) @@ -111,6 +107,16 @@ and produce REVIEW notices. This source scan does not execute project code or replace go test ./... . Snapshot-based REVIEW notices describe changes since the last successful generation; they do not persist as an acknowledgement checklist. +## Contract tools + +api format [--check] preserves comments and normalizes indentation. api upgrade +previews changed infrastructure; --apply only replaces recognized prior seeds. +Use @request policy overrides and @app(request: {...}) for query/JSON strictness. +Explicitly exported api/omni.errors.json documents the Go-owned error catalog; +the CLI never executes project Go. omni.imports.json permits named shared roots; +workspace format/generate/check processes explicitly listed independent services. +Full reference: https://git.yuntuo.hk/k2safe/go-omni/src/branch/main/DEVELOPMENT.md + ## Layout `+"```text"+` @@ -119,7 +125,8 @@ main.go # GENERATED startup, lifecycle and route registrati etc/config.yaml # lowerCamelCase configuration internal/config/ # defaults -> file -> environment internal/svc/ # dependency initialization and cleanup -internal/types/typesGen.go # generated request/response types +internal/types/*Gen.go # DTOs split by source API, one types package +internal/svc/runtimeGen.go # GENERATED lifecycle and infrastructure startup internal/apierrors/errorsGen.go # generated fresh error constructors, when declared internal/handler// # generated binding and Logic adapters internal/logic// # developer-owned business operations @@ -138,7 +145,7 @@ use the same contract. 404/405 and ordinary App routes use the declared App defa No template is selected by declaration/import order. Without @app, existing framework response defaults apply. An explicit API App default takes precedence over response options in svc, keeping generated routes and OpenAPI consistent. -Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default. +Object responses reject nil; []DTO maps nil to []; *DTO permits null; returns(null) is a successful null value. No returns means 204. Body DTOs use application/json and have a 1 MiB limit by default. Use @body(maxBytes: ...) for route limits. @body(mode: raw|multipart; maxBytes: ...) adds a *httpx.Body argument to Logic, without handwritten handlers. Raw is the exact limited request stream; Parts supports sequential multipart reading with no temp files. @@ -146,11 +153,10 @@ limited request stream; Parts supports sequential multipart reading with no temp Logic result of *httpx.Stream; the handler closes Content and streams it unwrapped. Signature verification and storage policy belong to Logic/middleware. A consumed request stream cannot be reread automatically; use one reader owner or explicitly -bounded buffering. Stream reads respect configured server timeouts. Multipart parts -are not DTO form fields: form tags remain URL query parameters. The optional manual +bounded buffering. Stream reads respect configured server timeouts. Declare multipart/multipartJSON/file DTO fields for validated uploads; without those fields the existing streaming Body is passed. form tags remain URL query parameters. The optional manual handler/custom binder hooks are preserved, and are needed only for custom protocols. -Fields require exactly one source tag: json, path, form (URL query only), header. +Fields require one source tag: json, path, form (URL query), header, multipart, multipartJSON or file. Multipart fields select full automatic binding before Logic; File/[]File are request-scoped disk-backed uploads. Use File.Open to stream them; generated cleanup owns their lifetime. All are required by presence unless optional, omitempty or default is specified. Zero and false are valid present values; binding:"required" rejects scalar zero. Use *T with optional when business logic needs to distinguish missing from zero. @@ -171,7 +177,7 @@ The selected auth runs first; @auth none removes inherited declarative auth. Validation policy remains in the middleware implementation. Public messages must not contain secrets; error causes may be included in server logs. -svc.New receives the loaded Config and borrowed Logger. Add dependencies there, +Generated svc.New receives Config and a borrowed Logger. Add dependencies in svc.Init, then immediately register each cleanup with OnClose. A later initialization failure runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated @@ -208,7 +214,7 @@ func infrastructureREADME(o Options) string { Selected database: %s; storage: %s; logging: %s/%s. MySQL uses database/sql through deps.DB, including InTx and a readiness probe. -Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*. +Use structured database/mysql configuration and GO_OMNI_DB_PASSWORD. Other connection/pool/TLS settings live in etc/config.yaml and GO_OMNI_DB_*. No tables are invented or migrated automatically from request/response DTOs. The none/memory selection opens no database; implement repositories per business. diff --git a/cmd/go-omni/apigen/testdata/business/accessFactory.go b/cmd/go-omni/apigen/testdata/business/accessFactory.go new file mode 100644 index 0000000..94b6388 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/business/accessFactory.go @@ -0,0 +1,22 @@ +package middleware + +import ( + "context" + "git.yuntuo.hk/k2safe/go-omni/httpx" +) + +// This package deliberately imports no svc; the dependency is supplied explicitly. +type AccessChecker interface { + CheckAccess(context.Context, string) error +} + +func NewRequireAccess(checker AccessChecker) httpx.Middleware { + return func(next httpx.Handler) httpx.Handler { + return func(c *httpx.Context) error { + if err := checker.CheckAccess(c.Context(), c.Header("Authorization")); err != nil { + return err + } + return next(c) + } + } +} diff --git a/cmd/go-omni/apigen/testdata/business/client.go b/cmd/go-omni/apigen/testdata/business/client.go new file mode 100644 index 0000000..a09bb42 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/business/client.go @@ -0,0 +1,32 @@ +package svc + +import ( + "context" + "example.com/business/internal/middleware" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "git.yuntuo.hk/k2safe/go-omni/httpx" + "sync/atomic" +) + +// A local fixture checks wiring; production authentication is project-owned. +type accessClient struct{ closed atomic.Bool } + +func (c *accessClient) Close() error { c.closed.Store(true); return nil } +func (c *accessClient) CheckAccess(ctx context.Context, token string) error { + if c.closed.Load() { + return apierrors.New(503, "CLOSED", "closed") + } + if httpx.Metadata(ctx).Route == "" { + return apierrors.New(500, "NO_METADATA", "missing route") + } + if token == "reject" { + return apierrors.New(403, "DENIED", "denied") + } + return nil +} +func (c *ServiceContext) initAccess(ctx context.Context) error { + c.Client = &accessClient{} + c.OnClose(c.Client.Close) + c.Middlewares["RequireAccess"] = middleware.NewRequireAccess(c.Client) + return nil +} diff --git a/cmd/go-omni/apigen/testdata/business/consumer_test.go b/cmd/go-omni/apigen/testdata/business/consumer_test.go new file mode 100644 index 0000000..9fa1272 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/business/consumer_test.go @@ -0,0 +1,119 @@ +package main + +import ( + "bytes" + "context" + "encoding/json" + "example.com/business/internal/config" + "example.com/business/internal/handler" + "example.com/business/internal/svc" + "git.yuntuo.hk/k2safe/go-omni/logx" + "io" + "mime/multipart" + "net/http/httptest" + "strings" + "testing" +) + +func TestShapesAndPolicy(t *testing.T) { + cfg := config.Default() + cfg.Server.Log.Writer = io.Discard + logger, err := logx.New(cfg.Server.Log) + if err != nil { + t.Fatal(err) + } + defer logger.Close() + deps, err := svc.New(context.Background(), logger, cfg) + if err != nil { + t.Fatal(err) + } + defer deps.Close() + app, err := handler.NewApp(deps) + if err != nil { + t.Fatal(err) + } + defer app.Close() + for _, tt := range []struct { + method, path, body, media string + status int + data string + raw bool + }{ + {"GET", "/list", "", "", 200, `[]`, false}, + {"GET", "/list?mode=full&tags=a&tags=b", "", "", 200, `[{"name":"one"}]`, false}, + {"GET", "/nullable", "", "", 200, `null`, false}, + {"GET", "/nullable?mode=full", "", "", 200, `{"name":"one"}`, false}, + {"POST", "/nothing", "", "", 200, `null`, false}, + {"GET", "/rawNull", "", "", 200, `null`, true}, + {"GET", "/rawArray", "", "", 200, `[]`, true}, + {"DELETE", "/noContent", "", "", 204, "", true}, + {"POST", "/empty", `{}`, "application/json", 200, `null`, false}, + {"POST", "/empty", ``, "application/json", 400, "", false}, + {"POST", "/empty", `{"unknown":1}`, "application/json", 400, "", false}, + {"POST", "/absent", ``, "", 200, `null`, false}, + {"POST", "/absent", `{}`, "application/json", 400, "", false}, + {"POST", "/optional", ``, "", 200, `{"name":""}`, false}, + {"POST", "/optional", `{}`, "application/json", 400, "", false}, + {"POST", "/echo", `{"name":"ok"}`, "application/json", 200, `{"name":"ok"}`, false}, + {"POST", "/echo", `{"name":"ok"}`, "application/json; charset=latin1", 415, "", false}, + {"POST", "/echo", "{\"name\":\"\xff\"}", "application/json", 400, "", false}, + {"GET", "/list?unknown=1", "", "", 400, "", false}, + {"GET", "/rawNull?unknown=1", "", "", 400, "", false}, + {"POST", "/tolerant?unknown=1", "{\"name\":\"\xff\"}", "application/json; charset=latin1", 200, `{"name":"�"}`, false}, + } { + t.Run(tt.method+tt.path+tt.body+tt.media, func(t *testing.T) { + r := httptest.NewRequest(tt.method, tt.path, strings.NewReader(tt.body)) + if tt.media != "" { + r.Header.Set("Content-Type", tt.media) + } + w := httptest.NewRecorder() + app.ServeHTTP(w, r) + if w.Code != tt.status { + t.Fatalf("%d %s", w.Code, w.Body) + } + if tt.status >= 400 { + return + } + data := strings.TrimSpace(w.Body.String()) + if !tt.raw { + var obj map[string]json.RawMessage + if err := json.Unmarshal(w.Body.Bytes(), &obj); err != nil { + t.Fatal(err) + } + v, ok := obj["data"] + if !ok { + t.Fatal("missing data") + } + data = string(v) + } + if data != tt.data { + t.Fatalf("data %s expected %s", data, tt.data) + } + }) + } + var body bytes.Buffer + form := multipart.NewWriter(&body) + _ = form.WriteField("title", "title") + _ = form.WriteField("metadata", `{"name":"meta"}`) + f, err := form.CreateFormFile("attachment", "a.txt") + if err != nil { + t.Fatal(err) + } + _, _ = io.WriteString(f, "content") + _ = form.Close() + r := httptest.NewRequest("POST", "/upload", &body) + r.Header.Set("Content-Type", form.FormDataContentType()) + w := httptest.NewRecorder() + app.ServeHTTP(w, r) + if w.Code != 200 || !strings.Contains(w.Body.String(), "title:meta:content") { + t.Fatal(w.Code, w.Body.String()) + } + denied := httptest.NewRequest("GET", "/list", nil) + denied.Header.Set("Authorization", "reject") + result := httptest.NewRecorder() + app.ServeHTTP(result, denied) + if result.Code != 403 { + t.Fatal("injected middleware did not run", result.Code, result.Body.String()) + } + +} diff --git a/cmd/go-omni/apigen/testdata/business/operations.go b/cmd/go-omni/apigen/testdata/business/operations.go new file mode 100644 index 0000000..8a59b79 --- /dev/null +++ b/cmd/go-omni/apigen/testdata/business/operations.go @@ -0,0 +1,42 @@ +package demo + +import ( + "context" + "example.com/business/internal/types" + "io" +) + +func (s *Logic) List(ctx context.Context, req *types.Query) ([]types.Item, error) { + if req.Mode == "full" { + return []types.Item{{Name: "one"}}, nil + } + return nil, nil +} +func (s *Logic) Nullable(ctx context.Context, req *types.Query) (*types.Item, error) { + if req.Mode == "full" { + return &types.Item{Name: "one"}, nil + } + return nil, nil +} +func (s *Logic) Nothing(ctx context.Context) error { return nil } +func (s *Logic) RawNull(ctx context.Context) error { return nil } +func (s *Logic) RawArray(ctx context.Context) ([]types.Item, error) { return nil, nil } +func (s *Logic) Empty(ctx context.Context) error { return nil } +func (s *Logic) Absent(ctx context.Context) error { return nil } +func (s *Logic) Optional(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil } +func (s *Logic) Echo(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil } +func (s *Logic) Tolerant(ctx context.Context, req *types.Item) (*types.Item, error) { return req, nil } +func (s *Logic) NoContent(ctx context.Context) error { return nil } + +func (s *Logic) Upload(ctx context.Context, req *types.Upload) (*types.Item, error) { + f, err := req.Attachment.Open() + if err != nil { + return nil, err + } + defer f.Close() + data, err := io.ReadAll(f) + if err != nil { + return nil, err + } + return &types.Item{Name: req.Title + ":" + req.Metadata.Name + ":" + string(data)}, nil +} diff --git a/cmd/go-omni/apigen/upgrade.go b/cmd/go-omni/apigen/upgrade.go new file mode 100644 index 0000000..c75a004 --- /dev/null +++ b/cmd/go-omni/apigen/upgrade.go @@ -0,0 +1,296 @@ +package apigen + +import ( + "bytes" + "encoding/json" + "errors" + "fmt" + "git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold" + "go/ast" + "go/format" + goparser "go/parser" + gotoken "go/token" + "golang.org/x/mod/modfile" + "golang.org/x/mod/semver" + "os" + "path/filepath" + "sort" + "strings" +) + +type UpgradeChange struct { + Path, Action string + Manual bool + before, after []byte + mode os.FileMode + exists bool +} +type UpgradePlan struct { + Changes []UpgradeChange + Module, Entry string +} + +// PlanUpgrade compares infrastructure against the exact previous release seeds. +// A custom file is never classified as safe to replace. Logic, middleware, +// options and repositories outside the changed templates are left untouched. +func PlanUpgrade(dir string) (UpgradePlan, error) { + var plan UpgradePlan + root := canonical(dir) + if err := safePath(root, manifestPath); err != nil { + return plan, err + } + raw, err := os.ReadFile(filepath.Join(root, manifestPath)) + if err != nil { + return plan, err + } + var m manifest + if err = json.Unmarshal(raw, &m); err != nil { + return plan, err + } + if m.Version != 2 || m.Profile == nil { + return plan, errors.New("upgrade requires a managed v0.13+ project") + } + if filepath.Ext(m.Entry) != ".api" { + return plan, errors.New("upgrade requires a valid API entry") + } + if err = safePath(root, m.Entry); err != nil { + return plan, err + } + plan.Module, plan.Entry = m.Module, m.Entry + if semver.IsValid(m.TemplateVersion) && semver.Compare(m.TemplateVersion, minimumVersion) >= 0 { + return plan, nil + } + p := m.Profile + legacy, err := scaffold.LegacyAPIInfrastructure(m.Module, p.Database, p.Storage, p.LogOutput, p.LogFormat) + if err != nil { + return plan, err + } + current, err := scaffold.APIInfrastructure(m.Module, p.Database, p.Storage, p.LogOutput, p.LogFormat) + if err != nil { + return plan, err + } + names := map[string]bool{} + for name := range legacy { + names[name] = true + } + for name := range current { + if !strings.HasSuffix(name, "Gen.go") { + names[name] = true + } + } + sorted := make([]string, 0, len(names)) + for name := range names { + sorted = append(sorted, name) + } + sort.Strings(sorted) + for _, name := range sorted { + // options.go is unchanged by this release. Its application customizations remain untouched. + previous, next := legacy[name], current[name] + if strings.HasSuffix(name, ".go") { + if previous != nil { + previous, err = format.Source(previous) + if err != nil { + return plan, err + } + } + if next != nil { + next, err = format.Source(next) + if err != nil { + return plan, err + } + } + } + if bytes.Equal(previous, next) { + continue + } + if err = safePath(root, name); err != nil { + return plan, err + } + existing, readErr := os.ReadFile(filepath.Join(root, name)) + if readErr != nil && !errors.Is(readErr, os.ErrNotExist) { + return plan, readErr + } + exists := readErr == nil + if bytes.Equal(existing, next) { + continue + } + mode := os.FileMode(0644) + if exists { + stat, e := os.Stat(filepath.Join(root, name)) + if e != nil { + return plan, e + } + mode = stat.Mode().Perm() + } + action := "update" + if next == nil { + action = "remove" + } else if !exists { + action = "create" + } + plan.Changes = append(plan.Changes, UpgradeChange{Path: name, Action: action, Manual: exists && !bytes.Equal(existing, previous), before: existing, after: next, mode: mode, exists: exists}) + } + if err = safePath(root, "go.mod"); err != nil { + return plan, err + } + raw, err = os.ReadFile(filepath.Join(root, "go.mod")) + if err != nil { + return plan, err + } + mf, err := modfile.Parse("go.mod", raw, nil) + if err != nil { + return plan, err + } + for _, r := range mf.Require { + if r.Mod.Path == frameworkModule && semver.Compare(r.Mod.Version, minimumVersion) < 0 { + if err = mf.AddRequire(frameworkModule, minimumVersion); err != nil { + return plan, err + } + next, e := mf.Format() + if e != nil { + return plan, e + } + stat, e := os.Stat(filepath.Join(root, "go.mod")) + if e != nil { + return plan, e + } + plan.Changes = append(plan.Changes, UpgradeChange{Path: "go.mod", Action: "update", before: raw, after: next, exists: true, mode: stat.Mode().Perm()}) + break + } + } + return plan, nil +} + +// PreviewUpgrade writes only proposed infrastructure files into an empty directory. +// It never mutates the service or emits a replacement business tree. +func PreviewUpgrade(dir, destination string) (UpgradePlan, error) { + plan, err := PlanUpgrade(dir) + if err != nil { + return plan, err + } + if entries, e := os.ReadDir(destination); e == nil && len(entries) > 0 { + return plan, errors.New("upgrade preview directory must be empty") + } else if e != nil && !errors.Is(e, os.ErrNotExist) { + return plan, e + } + for _, c := range plan.Changes { + if c.after == nil { + continue + } + if err = safePath(destination, c.Path); err != nil { + return plan, err + } + if err = atomicWrite(destination, c.Path, c.after, 0644); err != nil { + return plan, err + } + } + note := "# Infrastructure upgrade preview\n\nOnly untouched v0.13 seed files can be replaced automatically. Custom files require a targeted edit; this preview is never copied over the project automatically.\n\nMove dependency initialization into ServiceContext.Init; embed *lifecycle.Runtime; register cleanup with OnClose. Remove old New/Close/OnClose implementations after preserving their project work in Init or cleanup hooks. Use structured database/mysql.Config and DatabaseOptions. Preserve business configuration fields, AppOptions, middleware and Logic.\n\n" + for _, c := range plan.Changes { + note += fmt.Sprintf("- %s %s (manual review: %t)\n", c.Action, c.Path, c.Manual) + } + if err = safePath(destination, "UPGRADE.md"); err != nil { + return plan, err + } + return plan, atomicWrite(destination, "UPGRADE.md", []byte(note), 0644) +} + +// ApplyUpgrade applies only recognized seed changes under the same generation +// lock. A generation failure restores every changed developer file. Any manual +// review item blocks all writes; use PreviewUpgrade and migrate that file first. +func ApplyUpgrade(dir string) (plan UpgradePlan, err error) { + root := canonical(dir) + if err = safePath(root, ".goOmni/api.lock"); err != nil { + return plan, err + } + lock, err := os.OpenFile(filepath.Join(root, ".goOmni/api.lock"), os.O_CREATE|os.O_EXCL|os.O_WRONLY, 0600) + if err != nil { + return plan, err + } + _ = lock.Close() + defer os.Remove(filepath.Join(root, ".goOmni/api.lock")) + plan, err = PlanUpgrade(root) + if err != nil { + return plan, err + } + for _, c := range plan.Changes { + if c.Manual { + return plan, fmt.Errorf("upgrade: %s contains project changes; inspect an upgrade preview and migrate it explicitly", c.Path) + } + } + var changed []UpgradeChange + defer func() { + if err != nil { + for i := len(changed) - 1; i >= 0; i-- { + c := changed[i] + if c.exists { + err = errors.Join(err, atomicWrite(root, c.Path, c.before, c.mode)) + } else { + e := os.Remove(filepath.Join(root, c.Path)) + if !errors.Is(e, os.ErrNotExist) { + err = errors.Join(err, e) + } + } + } + } + }() + for _, c := range plan.Changes { + if c.after == nil { + err = os.Remove(filepath.Join(root, c.Path)) + } else { + err = atomicWrite(root, c.Path, c.after, c.mode) + } + if err != nil { + return plan, err + } + changed = append(changed, c) + } + preview, e := Generate(Options{Dir: root, API: filepath.Join(root, plan.Entry), DryRun: true, lockHeld: true}) + if e != nil { + return plan, e + } + if diagnosticErrors(preview.Diagnostics) > 0 { + return plan, errors.New("upgrade: Logic checks failed; resolve diagnostics before applying") + } + _, err = Generate(Options{Dir: root, API: filepath.Join(root, plan.Entry), lockHeld: true}) + return plan, err +} + +func checkRuntimeExtension(root string) error { + directory := filepath.Join(root, "internal/svc") + files, err := os.ReadDir(directory) + if errors.Is(err, os.ErrNotExist) { + return nil + } + if err != nil { + return err + } + for _, entry := range files { + if !sourceFile(entry.Name()) || strings.HasSuffix(entry.Name(), "Gen.go") { + continue + } + path := filepath.Join(directory, entry.Name()) + file, err := goparser.ParseFile(gotoken.NewFileSet(), path, nil, goparser.SkipObjectResolution) + if err != nil { + return err + } + for _, decl := range file.Decls { + fn, ok := decl.(*ast.FuncDecl) + if !ok { + continue + } + if fn.Recv == nil && fn.Name.Name == "New" { + return fmt.Errorf("%s owns New; run go-omni api upgrade --dir %s and move project initialization into ServiceContext.Init", path, root) + } + if fn.Recv != nil && (fn.Name.Name == "Close" || fn.Name.Name == "OnClose" || fn.Name.Name == "Finish") { + receiver := fn.Recv.List[0].Type + if p, ok := receiver.(*ast.StarExpr); ok { + receiver = p.X + } + if id, ok := receiver.(*ast.Ident); ok && id.Name == "ServiceContext" { + return fmt.Errorf("%s overrides framework resource lifecycle; inspect go-omni api upgrade and register cleanup in Init", path) + } + } + } + } + return nil +} diff --git a/cmd/go-omni/apigen/upgrade_test.go b/cmd/go-omni/apigen/upgrade_test.go new file mode 100644 index 0000000..b6f007f --- /dev/null +++ b/cmd/go-omni/apigen/upgrade_test.go @@ -0,0 +1,124 @@ +package apigen + +import ( + "encoding/json" + "git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold" + "go/format" + "os" + "path/filepath" + "strings" + "testing" +) + +func oldInfrastructureProject(t *testing.T) string { + t.Helper() + dir := filepath.Join(t.TempDir(), "app") + if _, err := New(Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) + } + previous, err := scaffold.LegacyAPIInfrastructure("example.com/upgrade", "none", "none", "stderr", "json") + if err != nil { + t.Fatal(err) + } + for name, content := range previous { + if strings.HasSuffix(name, ".go") { + content, err = format.Source(content) + if err != nil { + t.Fatal(err) + } + } + writeFile(t, filepath.Join(dir, name), string(content)) + } + var m manifest + if err = json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil { + t.Fatal(err) + } + delete(m.Generated, "internal/svc/runtimeGen.go") + m.TemplateVersion = "" + _ = os.Remove(filepath.Join(dir, "internal/svc/runtimeGen.go")) + raw, _ := json.Marshal(m) + writeFile(t, filepath.Join(dir, manifestPath), string(raw)) + path := filepath.Join(dir, "go.mod") + writeFile(t, path, strings.ReplaceAll(readFile(t, path), minimumVersion, "v0.13.0-preview")) + return dir +} + +func TestUpgradeRejectsEscapingEntry(t *testing.T) { + dir := oldInfrastructureProject(t) + var m manifest + if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil { + t.Fatal(err) + } + m.Entry = "../outside.api" + raw, _ := json.Marshal(m) + writeFile(t, filepath.Join(dir, manifestPath), string(raw)) + before := snapshot(t, dir) + if _, err := ApplyUpgrade(dir); err == nil { + t.Fatal("escaping API entry accepted") + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("invalid upgrade wrote project") + } +} +func TestSafeInfrastructureUpgrade(t *testing.T) { + dir := oldInfrastructureProject(t) + before := snapshot(t, dir) + if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}); err == nil { + t.Fatal("legacy constructor accepted") + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("failed generate wrote files") + } + plan, err := PlanUpgrade(dir) + if err != nil || len(plan.Changes) == 0 { + t.Fatal(plan, err) + } + for _, c := range plan.Changes { + if c.Manual { + t.Fatal(c.Path) + } + } + preview := filepath.Join(t.TempDir(), "preview") + if _, err = PreviewUpgrade(dir, preview); err != nil { + t.Fatal(err) + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("preview changed project") + } + logic := readFile(t, filepath.Join(dir, "internal/logic/greet/greet.go")) + if _, err = ApplyUpgrade(dir); err != nil { + t.Fatal(err) + } + if _, err = os.Stat(filepath.Join(dir, "internal/svc/close.go")); !os.IsNotExist(err) { + t.Fatal("old lifecycle remains") + } + if readFile(t, filepath.Join(dir, "internal/logic/greet/greet.go")) != logic { + t.Fatal("business changed") + } + if _, err = Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil { + t.Fatal(err) + } + plan, err = PlanUpgrade(dir) + if err != nil || len(plan.Changes) != 0 { + t.Fatal("upgrade not idempotent", plan, err) + } +} +func TestUpgradePreservesCustomFilesAndRollsBack(t *testing.T) { + for _, custom := range []bool{true, false} { + dir := oldInfrastructureProject(t) + if custom { + path := filepath.Join(dir, "internal/svc/serviceContext.go") + writeFile(t, path, readFile(t, path)+"\n// custom dependency\n") + } else { + path := filepath.Join(dir, "internal/logic/greet/greet.go") + writeFile(t, path, "package greet;func(s *Logic)Greet(){}") + } + before := snapshot(t, dir) + if _, err := ApplyUpgrade(dir); err == nil { + t.Fatal("unsafe upgrade accepted") + } + if !sameFiles(before, snapshot(t, dir)) { + t.Fatal("failed upgrade changed project") + } + } +} diff --git a/cmd/go-omni/apigen/workspace.go b/cmd/go-omni/apigen/workspace.go new file mode 100644 index 0000000..39feef8 --- /dev/null +++ b/cmd/go-omni/apigen/workspace.go @@ -0,0 +1,126 @@ +package apigen + +import ( + "errors" + "fmt" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "io" + "os" + "path/filepath" + "strings" +) + +type WorkspaceService struct { + Dir string `json:"dir"` + API string `json:"api,omitempty"` + Module string `json:"module,omitempty"` +} +type WorkspaceConfig struct { + Services []WorkspaceService `json:"services"` +} + +// Workspace operates independent Go modules listed in an explicit local file. +// All projects are preflighted before generate/format writes. Each generator +// transaction is atomic within its project; no cross-module transaction is promised. +func Workspace(path, command string, w io.Writer) error { + if command != "generate" && command != "check" && command != "format" { + return errors.New("workspace command must be generate, check or format") + } + if w == nil { + w = io.Discard + } + real, err := filepath.Abs(path) + if err != nil { + return err + } + stat, err := os.Lstat(real) + if err != nil { + return err + } + if !stat.Mode().IsRegular() || stat.Size() > 65536 { + return errors.New("workspace configuration must be a regular file of at most 64 KiB") + } + raw, err := os.ReadFile(real) + if err != nil { + return err + } + var cfg WorkspaceConfig + if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil { + return err + } + if len(cfg.Services) == 0 || len(cfg.Services) > 64 { + return errors.New("workspace must list 1..64 services") + } + root := canonical(filepath.Dir(real)) + var options []Options + seen := map[string]bool{} + for _, service := range cfg.Services { + if service.Dir == "" || service.Dir == "." { + return errors.New("workspace requires distinct relative service directories") + } + if err = safePath(root, service.Dir+"/.goOmniWorkspaceCheck"); err != nil { + return err + } + dir := filepath.Join(root, filepath.FromSlash(service.Dir)) + key := strings.ToLower(dir) + for prior := range seen { + if prior == key || strings.HasPrefix(key, prior+string(filepath.Separator)) || strings.HasPrefix(prior, key+string(filepath.Separator)) { + return errors.New("workspace service directories overlap") + } + } + seen[key] = true + entry := service.API + if entry == "" { + entry, err = ProjectEntry(dir) + } else { + if err = safePath(dir, entry); err != nil { + return err + } + entry = filepath.Join(dir, entry) + } + if err != nil { + return err + } + options = append(options, Options{API: entry, Dir: dir, Module: service.Module, DryRun: true}) + } + var failures error + for _, o := range options { + if command == "format" { + spec, e := Parse(o.API) + if e == nil { + _, e = Compile(spec) + } + err = e + } else { + _, err = Generate(o) + } + if err != nil { + failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err)) + } + } + if failures != nil { + return failures + } + for _, o := range options { + switch command { + case "format": + _, err = FormatProject(o.API, false) + case "generate": + o.DryRun = false + _, err = Generate(o) + case "check": + _, fmtErr := FormatProject(o.API, true) + o.DryRun = false + o.Check = true + _, genErr := Generate(o) + err = errors.Join(fmtErr, genErr) + } + if err != nil { + failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err)) + fmt.Fprintf(w, "FAIL %s\n", o.Dir) + } else { + fmt.Fprintf(w, "OK %s\n", o.Dir) + } + } + return failures +} diff --git a/cmd/go-omni/apigen/workspace_test.go b/cmd/go-omni/apigen/workspace_test.go new file mode 100644 index 0000000..0eb7d35 --- /dev/null +++ b/cmd/go-omni/apigen/workspace_test.go @@ -0,0 +1,43 @@ +package apigen + +import ( + "io" + "path/filepath" + "strings" + "testing" +) + +func TestWorkspaceIndependentModules(t *testing.T) { + root := t.TempDir() + for _, name := range []string{"api", "id"} { + dir := filepath.Join(root, "services", name) + if _, err := New(Options{Dir: dir, Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil { + t.Fatal(err) + } + } + config := filepath.Join(root, "omni.workspace.json") + writeFile(t, config, `{"services":[{"dir":"services/api"},{"dir":"services/id"}]}`) + for _, op := range []string{"format", "generate", "check", "generate", "check"} { + if err := Workspace(config, op, io.Discard); err != nil { + t.Fatal(op, err) + } + } + // A bad second service must not leave the first service partially regenerated. + first := filepath.Join(root, "services/api/api/service.api") + writeFile(t, first, strings.Replace(readFile(t, first), "Greet a visitor", "Greet", 1)) + second := filepath.Join(root, "services/id/api/service.api") + writeFile(t, second, "invalid source") + before := snapshot(t, root) + if err := Workspace(config, "generate", io.Discard); err == nil { + t.Fatal("invalid workspace accepted") + } + if !sameFiles(before, snapshot(t, root)) { + t.Fatal("preflight failure changed project") + } + for _, bad := range []string{`{"services":[{"dir":"../outside"}]}`, `{"services":[{"dir":"services/api"},{"dir":"services/api/nested"}]}`, `{"services":[{"dir":"services/api","dir":"services/id"}]}`} { + writeFile(t, config, bad) + if err := Workspace(config, "check", io.Discard); err == nil { + t.Fatal("invalid workspace accepted") + } + } +} diff --git a/cmd/go-omni/apigen/write.go b/cmd/go-omni/apigen/write.go index 2237a54..336e8f6 100644 --- a/cmd/go-omni/apigen/write.go +++ b/cmd/go-omni/apigen/write.go @@ -16,12 +16,13 @@ import ( const manifestPath = ".goOmni/apiManifest.json" type manifest struct { - Profile *projectProfile `json:"profile,omitempty"` - Version int `json:"version"` - Module string `json:"module"` - Entry string `json:"entry"` - Generated map[string]string `json:"generated"` - Contract *contractSnapshot `json:"contract,omitempty"` + TemplateVersion string `json:"templateVersion,omitempty"` + Profile *projectProfile `json:"profile,omitempty"` + Version int `json:"version"` + Module string `json:"module"` + Entry string `json:"entry"` + Generated map[string]string `json:"generated"` + Contract *contractSnapshot `json:"contract,omitempty"` } type change struct { name string @@ -30,7 +31,7 @@ type change struct { mode os.FileMode } -func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile, contract *contractSnapshot) (Output, error) { +func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile, contract *contractSnapshot, lockHeld ...bool) (Output, error) { var result Output if info, err := os.Lstat(root); err == nil { if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() { @@ -61,7 +62,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check if err := safePath(root, manifestPath); err != nil { return result, err } - if !check && !dryRun { + if !check && !dryRun && !(len(lockHeld) > 0 && lockHeld[0]) { if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil { return result, err } @@ -108,7 +109,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check return result, err } } - next := manifest{Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract} + next := manifest{TemplateVersion: minimumVersion, Profile: &profile, Version: 2, Module: module, Entry: entry, Generated: map[string]string{}, Contract: contract} for name, data := range generated { next.Generated[name] = digest(data) } @@ -271,7 +272,10 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check } func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) } func ownedPath(name string) bool { - if name == "main.go" || name == "internal/handler/appGen.go" || name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" { + if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) { + return true + } + if name == "main.go" || name == "internal/svc/runtimeGen.go" || name == "internal/handler/appGen.go" || name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" { return true } if name == "internal/apierrors/errorsGen.go" || name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" { diff --git a/cmd/go-omni/main.go b/cmd/go-omni/main.go index a2c605a..8413d1a 100644 --- a/cmd/go-omni/main.go +++ b/cmd/go-omni/main.go @@ -18,6 +18,11 @@ func main() { } switch os.Args[1] { + case "workspace": + if err := runWorkspace(os.Args[2:]); err != nil { + fmt.Fprintln(os.Stderr, "go-omni workspace:", err) + os.Exit(1) + } case "api": if err := runAPI(os.Args[2:]); err != nil { fmt.Fprintln(os.Stderr, "go-omni api:", err) @@ -170,6 +175,9 @@ func usage() { fmt.Fprintln(os.Stderr, " go-omni new --api [--module ] ") fmt.Fprintln(os.Stderr, " go-omni api new [--module ] Create a runnable API project") fmt.Fprintln(os.Stderr, " go-omni api generate [--api ] [--dir ] [--check|--dry-run]") + fmt.Fprintln(os.Stderr, " go-omni api format [--api ] [--check]") + fmt.Fprintln(os.Stderr, " go-omni api upgrade [--dir ] [--dry-run|--preview |--apply]") + fmt.Fprintln(os.Stderr, " go-omni workspace [--file omni.workspace.json]") fmt.Fprintln(os.Stderr, " go-omni api inspect [--api ] Show routes, auth and business errors") fmt.Fprintln(os.Stderr, " go-omni api export [--api ] Write OpenAPI JSON to stdout") fmt.Fprintln(os.Stderr, " go-omni api validate --api Validate definitions and OpenAPI") diff --git a/cmd/go-omni/main_test.go b/cmd/go-omni/main_test.go index 1d360b0..ce46b99 100644 --- a/cmd/go-omni/main_test.go +++ b/cmd/go-omni/main_test.go @@ -8,7 +8,7 @@ import ( ) func TestNewWithRemoteFrameworkVersion(t *testing.T) { - for _, flag := range [][]string{{"--framework-version", "v0.13.0-preview"}, {"--framework-version=v0.13.0-preview"}} { + for _, flag := range [][]string{{"--framework-version", "v0.14.0-preview"}, {"--framework-version=v0.14.0-preview"}} { project := filepath.Join(t.TempDir(), "billing") args := append([]string{"--module", "example.com/billing"}, flag...) if err := runNew(append(args, project)); err != nil { @@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) { if err != nil { t.Fatal(err) } - if !strings.Contains(string(content), "go-omni v0.13.0-preview") || strings.Contains(string(content), "replace ") { + if !strings.Contains(string(content), "go-omni v0.14.0-preview") || strings.Contains(string(content), "replace ") { t.Fatalf("CLI did not pin remote dependency: %s", content) } } diff --git a/cmd/go-omni/scaffold/apiInfrastructure.go b/cmd/go-omni/scaffold/apiInfrastructure.go index ca119f6..95af63c 100644 --- a/cmd/go-omni/scaffold/apiInfrastructure.go +++ b/cmd/go-omni/scaffold/apiInfrastructure.go @@ -8,17 +8,37 @@ import ( // APIInfrastructure renders shared infrastructure without inventing business // tables or routes. The API generator applies these as developer-owned seeds. func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) { + return apiInfrastructure(module, database, storage, logOutput, logFormat, false) +} + +// LegacyAPIInfrastructure renders only the known v0.13 seed baseline for a safe upgrade. +func LegacyAPIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) { + return apiInfrastructure(module, database, storage, logOutput, logFormat, true) +} +func apiInfrastructure(module, database, storage, logOutput, logFormat string, legacy bool) (map[string][]byte, error) { data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true} files := map[string]string{ - "internal/svc/close.go": "templates/api-close.go.tmpl", + "internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl", "internal/svc/options.go": "templates/api-options.go.tmpl", - "internal/config/config.go": "templates/modules-settings.go.tmpl", + "internal/config/config.go": "templates/api-config.go.tmpl", "etc/config.yaml": "templates/modules-config.yaml.tmpl", "internal/svc/serviceContext.go": "templates/api-context.go.tmpl", } + if database == "mysql" { + files["internal/svc/databaseOptions.go"] = "templates/api-databaseOptions.go.tmpl" + } if storage != "none" { files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl" } + if legacy { + delete(files, "internal/svc/runtimeGen.go") + delete(files, "internal/svc/databaseOptions.go") + files["internal/svc/close.go"] = "templates/api-close.go.tmpl" + files["internal/config/config.go"] = "templates/modules-settings.go.tmpl" + for name, source := range files { + files[name] = strings.Replace(source, "templates/", "templates/legacyV013/", 1) + } + } result := make(map[string][]byte, len(files)) for name, source := range files { content, err := render(source, data) diff --git a/cmd/go-omni/scaffold/templates/api-config.go.tmpl b/cmd/go-omni/scaffold/templates/api-config.go.tmpl new file mode 100644 index 0000000..4daa7f7 --- /dev/null +++ b/cmd/go-omni/scaffold/templates/api-config.go.tmpl @@ -0,0 +1,104 @@ +package settings +import ( + "os" + "strings" + "time" + "git.yuntuo.hk/k2safe/go-omni/config" + "git.yuntuo.hk/k2safe/go-omni/logx" + {{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}} + {{if ne .Storage "none"}}"errors" + {{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}} + "git.yuntuo.hk/k2safe/go-omni/storage"{{end}} +) +type Config struct { + Server config.Config `json:"server"` + {{if eq .Database "mysql"}}Database mysqladapter.Config `json:"database" env:"GO_OMNI_DB"`{{end}} + {{if ne .Storage "none"}}Storage StorageConfig `json:"storage" env:"GO_OMNI_STORAGE"`{{end}} +} +func Default() Config { + server:=config.Default() + server.Address="127.0.0.1:8080" + server.EnableLogger=true + server.SlowRequestThreshold=500*time.Millisecond + server.Log.Service={{printf "%q" .Module}} + server.Log.Format={{printf "%q" .LogFormat}} + {{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log" + server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true} + {{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}} + return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:mysqladapter.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{ + Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object", + Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"}, + LinkTTL:5*time.Minute, RequestTimeout:30*time.Second, + {{if eq .Storage "s3"}}Region:"us-east-1",{{end}} + {{if eq .Storage "r2"}}Region:"auto",{{end}} + {{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}} + }{{end}}} +} + +// Load applies defaults -> file -> typed environment -> validation. +// New fields automatically use GO_OMNI_, or an explicit env tag. +func Load(path string)(Config,error){ + cfg:=Default();var err error + cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err} + if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}} + cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_"});if err!=nil{return Config{},err} + // Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc). + cfg.Server,err=config.ApplyEnv(cfg.Server,"");if err!=nil{return Config{},err} + if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}} + return cfg,cfg.Validate() +} +func(c Config)Validate()error{ + if err:=c.Server.Validate();err!=nil{return err} + {{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return err}{{end}} + {{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return err}{{end}} + return nil +} +{{if ne .Storage "none"}} +type StorageConfig struct { + Provider string `json:"provider"` + Root string `json:"root"` + PublicURL string `json:"publicUrl"` + Endpoint string `json:"endpoint"` + Region string `json:"region"` + Bucket string `json:"bucket"` + AccountID string `json:"accountId"` + Secure bool `json:"secure"` + MaxBytes int64 `json:"maxBytes"` + ContentTypes []string `json:"contentTypes"` + LinkTTL time.Duration `json:"linkTtl"` + RequestTimeout time.Duration `json:"requestTimeout"` + // Secrets are environment-only and cannot be populated by config files. + FilesToken string `json:"-" env:"GO_OMNI_FILES_TOKEN,secret"` + SigningKey string `json:"-" env:"GO_OMNI_STORAGE_SIGNING_KEY,secret"` + AccessKey string `json:"-" env:"GO_OMNI_STORAGE_ACCESS_KEY,secret"` + SecretKey string `json:"-" env:"GO_OMNI_STORAGE_SECRET_KEY,secret"` + SessionToken string `json:"-" env:"GO_OMNI_STORAGE_SESSION_TOKEN,secret"` +} +func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}} +func (s StorageConfig) Validate()error { + if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")} + if err:=s.Policy().Validate();err!=nil{return err} + if s.LinkTTLtime.Hour{return errors.New("storage.linkTtl must be 1s..1h")} + if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")} + {{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}} + {{if eq .Storage "local"}} + if s.Root=="" {return errors.New("storage.root is required")} + u,err:=url.Parse(s.PublicURL) + if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")} + if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")} + {{else}} + if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")} + {{if eq .Storage "r2"}} + if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")} + if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")} + {{else if eq .Storage "s3"}} + u,err:=url.Parse("https://"+s.Endpoint) + if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")} + {{else if eq .Storage "oss"}} + if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}} + if !s.Secure{return errors.New("OSS requires secure=true")} + {{end}} + {{end}} + return nil +} +{{end}} diff --git a/cmd/go-omni/scaffold/templates/api-context.go.tmpl b/cmd/go-omni/scaffold/templates/api-context.go.tmpl index 8a3b6b7..c44a0bf 100644 --- a/cmd/go-omni/scaffold/templates/api-context.go.tmpl +++ b/cmd/go-omni/scaffold/templates/api-context.go.tmpl @@ -1,26 +1,20 @@ package servicecontext import ( "context" - "errors" - "sync" {{if ne .Storage "none"}}"net/http" "git.yuntuo.hk/k2safe/go-omni/storage"{{end}} - {{if eq .Database "mysql"}}"time" - "git.yuntuo.hk/k2safe/go-omni/database" - "github.com/go-sql-driver/mysql"{{end}} + {{if eq .Database "mysql"}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}} "git.yuntuo.hk/k2safe/go-omni/health" "git.yuntuo.hk/k2safe/go-omni/httpx" "git.yuntuo.hk/k2safe/go-omni/logx" "git.yuntuo.hk/k2safe/go-omni/lifecycle" "{{.Module}}/internal/settings" - "{{.Module}}/internal/middleware" ) -// Context owns infrastructure, while Logger is borrowed from main. -// Add business repositories here; HTTP contracts do not define database tables. +// Context holds project dependencies. Runtime owns cleanup and shutdown. +// Logger is borrowed; the generated entry closes it after dependencies. type Context struct { - Lifecycle *lifecycle.Controller + *lifecycle.Runtime Config settings.Config - cleanups []func()error Logger *logx.Logger Health *health.Checker Middlewares map[string]httpx.Middleware @@ -31,38 +25,9 @@ type Context struct { Downloads storage.Presigner UploadLinks storage.UploadPresigner FileHTTP http.Handler - storageClose func()error {{end}} - closeMu sync.Mutex - closed bool - closeOnce sync.Once - closeErr error } -// Pass the settings loaded by main. The optional form supports lightweight -// tests without infrastructure; selected adapters still require valid settings. -func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){ - if ctx==nil{ctx=context.Background()} - if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))} - if len(configs)>1{return nil,errors.New("expected at most one configuration")} - cfg:=settings.Default();if len(configs)==1{cfg=configs[0]} - _=cfg - checker,err:=health.New();if err!=nil{return nil,err} - c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Lifecycle:lifecycle.New(ctx)} - ctx=c.Lifecycle.Context() - defer func(){if err!=nil{err=errors.Join(err,c.Close());if cause:=c.Lifecycle.Err();cause!=nil && !errors.Is(err,cause){err=errors.Join(err,cause)}}}() - {{if eq .Database "mysql"}} - if err=cfg.Validate();err!=nil{return nil,err} - dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN) - if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")} - dsn.ParseTime=true;dsn.Loc=time.UTC - cfg.Database.DSN=dsn.FormatDSN() - c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err} - if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err} - {{end}} - {{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}} - return c,nil -} - -// RequestShutdown reports a fatal dependency failure, or a normal stop when -// err is nil. It returns immediately; generated main drains and cleans up. -func(c *Context)RequestShutdown(err error){c.Lifecycle.RequestShutdown(err)} +// Init is project-owned. Add dependencies here and immediately register each +// cleanup with c.OnClose. Returning an error automatically rolls everything back. +// Generated infrastructure is ready here. Never create another App or logger. +func(c *Context)Init(ctx context.Context)error {return nil} diff --git a/cmd/go-omni/scaffold/templates/api-databaseOptions.go.tmpl b/cmd/go-omni/scaffold/templates/api-databaseOptions.go.tmpl new file mode 100644 index 0000000..fbc75b7 --- /dev/null +++ b/cmd/go-omni/scaffold/templates/api-databaseOptions.go.tmpl @@ -0,0 +1,5 @@ +package servicecontext +import mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql" +// DatabaseOptions supplies explicit driver settings or bounded startup checks. +// Keep schema/version requirements in this project; do not open another pool. +func(c *Context)DatabaseOptions()[]mysqladapter.Option{return nil} diff --git a/cmd/go-omni/scaffold/templates/api-runtime.go.tmpl b/cmd/go-omni/scaffold/templates/api-runtime.go.tmpl new file mode 100644 index 0000000..b875ab1 --- /dev/null +++ b/cmd/go-omni/scaffold/templates/api-runtime.go.tmpl @@ -0,0 +1,33 @@ +package servicecontext +import ( + "context" + "errors" + "git.yuntuo.hk/k2safe/go-omni/health" + "git.yuntuo.hk/k2safe/go-omni/lifecycle" + "git.yuntuo.hk/k2safe/go-omni/logx" + {{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}} + "{{.Module}}/internal/settings" + "{{.Module}}/internal/middleware" +) +// New is generator-owned. Customize dependencies in ServiceContext.Init. +func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){ + if ctx==nil{ctx=context.Background()} + if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))} + if logger==nil{return nil,errors.New("service context requires a logger")} + if len(configs)>1{return nil,errors.New("expected at most one configuration")} + cfg:=settings.Default();if len(configs)==1{cfg=configs[0]} + if err=cfg.Validate();err!=nil{return nil,err} + checker,err:=health.New();if err!=nil{return nil,err} + c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Runtime:lifecycle.NewRuntime(ctx)} + ctx=logx.WithLogger(c.Context(),logger) + defer func(){if p:=recover();p!=nil{_ = c.Close();panic(p)};if err!=nil{err=c.Finish(err)}}() + {{if eq .Database "mysql"}} + options:=append([]mysqladapter.Option{mysqladapter.WithLogger(logger)},c.DatabaseOptions()...) + c.DB,err=mysqladapter.Open(ctx,cfg.Database,options...);if err!=nil{return nil,err} + c.OnClose(c.DB.Close) + if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.Pool.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err} + {{end}} + {{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}} + if err=c.Init(ctx);err!=nil{return nil,err} + return c,nil +} diff --git a/cmd/go-omni/scaffold/templates/legacyV013/api-close.go.tmpl b/cmd/go-omni/scaffold/templates/legacyV013/api-close.go.tmpl new file mode 100644 index 0000000..542b96e --- /dev/null +++ b/cmd/go-omni/scaffold/templates/legacyV013/api-close.go.tmpl @@ -0,0 +1,23 @@ +package servicecontext +import "errors" +// Close is idempotent, including startup failure and graceful shutdown. +func(c *Context)Close()error{ + if c==nil{return nil} + c.closeOnce.Do(func(){ + if c.Lifecycle!=nil{c.Lifecycle.RequestShutdown(nil)} + c.closeMu.Lock();c.closed=true;cleanups:=c.cleanups;c.cleanups=nil;c.closeMu.Unlock() + for i:=len(cleanups)-1;i>=0;i--{c.closeErr=errors.Join(c.closeErr,cleanups[i]())} + {{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}} + {{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}} + }) + return c.closeErr +} + +// OnClose transfers a resource cleanup to this context immediately after creation. +// Hooks run once in reverse order, also when later initialization fails. +func(c *Context)OnClose(cleanup func()error) { + if cleanup==nil{return} + c.closeMu.Lock();defer c.closeMu.Unlock() + if c.closed{panic("svc: register cleanup after Close")} + c.cleanups=append(c.cleanups,cleanup) +} diff --git a/cmd/go-omni/scaffold/templates/legacyV013/api-context.go.tmpl b/cmd/go-omni/scaffold/templates/legacyV013/api-context.go.tmpl new file mode 100644 index 0000000..8a3b6b7 --- /dev/null +++ b/cmd/go-omni/scaffold/templates/legacyV013/api-context.go.tmpl @@ -0,0 +1,68 @@ +package servicecontext +import ( + "context" + "errors" + "sync" + {{if ne .Storage "none"}}"net/http" + "git.yuntuo.hk/k2safe/go-omni/storage"{{end}} + {{if eq .Database "mysql"}}"time" + "git.yuntuo.hk/k2safe/go-omni/database" + "github.com/go-sql-driver/mysql"{{end}} + "git.yuntuo.hk/k2safe/go-omni/health" + "git.yuntuo.hk/k2safe/go-omni/httpx" + "git.yuntuo.hk/k2safe/go-omni/logx" + "git.yuntuo.hk/k2safe/go-omni/lifecycle" + "{{.Module}}/internal/settings" + "{{.Module}}/internal/middleware" +) +// Context owns infrastructure, while Logger is borrowed from main. +// Add business repositories here; HTTP contracts do not define database tables. +type Context struct { + Lifecycle *lifecycle.Controller + Config settings.Config + cleanups []func()error + Logger *logx.Logger + Health *health.Checker + Middlewares map[string]httpx.Middleware + {{if eq .Database "mysql"}}DB *database.DB{{end}} + {{if ne .Storage "none"}} + Files storage.Store + Uploads *storage.Uploader + Downloads storage.Presigner + UploadLinks storage.UploadPresigner + FileHTTP http.Handler + storageClose func()error + {{end}} + closeMu sync.Mutex + closed bool + closeOnce sync.Once + closeErr error +} +// Pass the settings loaded by main. The optional form supports lightweight +// tests without infrastructure; selected adapters still require valid settings. +func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){ + if ctx==nil{ctx=context.Background()} + if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))} + if len(configs)>1{return nil,errors.New("expected at most one configuration")} + cfg:=settings.Default();if len(configs)==1{cfg=configs[0]} + _=cfg + checker,err:=health.New();if err!=nil{return nil,err} + c:=&Context{Config:cfg,Logger:logger,Health:checker,Middlewares:middleware.Defaults(),Lifecycle:lifecycle.New(ctx)} + ctx=c.Lifecycle.Context() + defer func(){if err!=nil{err=errors.Join(err,c.Close());if cause:=c.Lifecycle.Err();cause!=nil && !errors.Is(err,cause){err=errors.Join(err,cause)}}}() + {{if eq .Database "mysql"}} + if err=cfg.Validate();err!=nil{return nil,err} + dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN) + if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")} + dsn.ParseTime=true;dsn.Loc=time.UTC + cfg.Database.DSN=dsn.FormatDSN() + c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err} + if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err} + {{end}} + {{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}} + return c,nil +} + +// RequestShutdown reports a fatal dependency failure, or a normal stop when +// err is nil. It returns immediately; generated main drains and cleans up. +func(c *Context)RequestShutdown(err error){c.Lifecycle.RequestShutdown(err)} diff --git a/cmd/go-omni/scaffold/templates/legacyV013/api-options.go.tmpl b/cmd/go-omni/scaffold/templates/legacyV013/api-options.go.tmpl new file mode 100644 index 0000000..3e5f8c0 --- /dev/null +++ b/cmd/go-omni/scaffold/templates/legacyV013/api-options.go.tmpl @@ -0,0 +1,11 @@ +package servicecontext +import ( + omni "git.yuntuo.hk/k2safe/go-omni" + "git.yuntuo.hk/k2safe/go-omni/httpx" +) +// AppOptions runs before routes are registered. Register project error catalogs +// here with omni.WithErrors. It must not create resources or another App. +func(c *Context)AppOptions()[]omni.Option{return nil} +// GlobalMiddleware runs for all endpoints, including App 404/405 fallbacks. +// Named route middleware lives in Middlewares and is selected by the API contract. +func(c *Context)GlobalMiddleware()[]httpx.Middleware{return nil} diff --git a/cmd/go-omni/scaffold/templates/legacyV013/modules-config.yaml.tmpl b/cmd/go-omni/scaffold/templates/legacyV013/modules-config.yaml.tmpl new file mode 100644 index 0000000..3ab9bf8 --- /dev/null +++ b/cmd/go-omni/scaffold/templates/legacyV013/modules-config.yaml.tmpl @@ -0,0 +1,59 @@ +# Non-secret settings only. Environment variables override these values. +server: + address: '127.0.0.1:8080' + enableLogger: true + slowRequestThreshold: 500ms + shutdownTimeout: 10s +{{if .API}} endpoints: + liveness: + enabled: true + path: /health/live + responseMode: app + readiness: + enabled: true + path: /health/ready + responseMode: app + openapi: + enabled: true + path: /openapi.json +{{end}} + log: + service: {{printf "%q" .Module}} + level: info + format: {{.LogFormat}} +{{if eq .LogOutput "file"}} file: './var/log/server.log' + rotation: + enabled: true + maxSizeMb: 100 + maxBackups: 10 + maxAgeDays: 7 + compress: true +{{else}} output: {{.LogOutput}} +{{end}} redactKeys: [] +{{if ne .Storage "none"}}storage: + provider: {{.Storage}} + maxBytes: 33554432 + contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf'] + linkTtl: 5m + requestTimeout: 30s +{{if eq .Storage "local"}} root: './var/files' + publicUrl: 'http://127.0.0.1:8080/storage/object' +{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET + secure: true +{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID + region: auto +{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port]) + region: us-east-1 +{{else}} region: cn-hangzhou + endpoint: '' # empty uses the official regional endpoint +{{end}}{{end}}{{end}} + +{{if and .API (eq .Database "mysql")}}database: + driver: mysql + maxOpenConns: 10 + maxIdleConns: 5 + connMaxLifetime: 30m + connMaxIdleTime: 5m + pingTimeout: 5s +# GO_OMNI_DB_DSN supplies the secret connection string. +{{end}} diff --git a/cmd/go-omni/scaffold/templates/legacyV013/modules-settings.go.tmpl b/cmd/go-omni/scaffold/templates/legacyV013/modules-settings.go.tmpl new file mode 100644 index 0000000..41b8742 --- /dev/null +++ b/cmd/go-omni/scaffold/templates/legacyV013/modules-settings.go.tmpl @@ -0,0 +1,140 @@ +// Package settings owns this project's configuration; it adds no business identity model. +package settings + +import ( + "errors" + "os" + "strings" + "time" + {{if ne .Storage "none"}}"fmt" + {{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}} + "strconv" + "git.yuntuo.hk/k2safe/go-omni/storage"{{end}} + {{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}} + "git.yuntuo.hk/k2safe/go-omni/config" + "git.yuntuo.hk/k2safe/go-omni/logx" +) + +type Config struct { + Server config.Config `json:"server"` + {{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}} + {{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}} +} + +func Default() Config { + server:=config.Default() + server.Address="127.0.0.1:8080" + server.EnableLogger=true + server.SlowRequestThreshold=500*time.Millisecond + server.Log.Service={{printf "%q" .Module}} + server.Log.Format={{printf "%q" .LogFormat}} + {{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log" + server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true} + {{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}} + return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{ + Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object", + Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"}, + LinkTTL:5*time.Minute, RequestTimeout:30*time.Second, + {{if eq .Storage "s3"}}Region:"us-east-1",{{end}} + {{if eq .Storage "r2"}}Region:"auto",{{end}} + {{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}} + }{{end}}} +} + +// Load uses defaults -> strict YAML/JSON -> nonempty environment variables. +// Pass an empty path to use only defaults and environment. +func Load(path string)(Config,error) { + cfg:=Default() + var err error + if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} } + cfg.Server,err=config.ApplyEnv(cfg.Server,"") + if err!=nil{return Config{},err} + // A stream explicitly selected by environment overrides a file profile. + if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" { + cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{} + } + {{if ne .Storage "none"}} + if err=cfg.Storage.applyEnv();err!=nil{return Config{},err} + {{end}} + {{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}} + if err=cfg.Validate();err!=nil{return Config{},err} + return cfg,nil +} +func (c Config) Validate()error { + {{if and .API (eq .Database "mysql")}} + if err:=c.Database.Validate();err!=nil{return err} + if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")} + {{end}} + if err:=c.Server.Validate();err!=nil{return err} + if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")} + switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")} + {{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}} +} + +{{if ne .Storage "none"}} +type StorageConfig struct { + Provider string `json:"provider"` + Root string `json:"root"` + PublicURL string `json:"publicUrl"` + Endpoint string `json:"endpoint"` + Region string `json:"region"` + Bucket string `json:"bucket"` + AccountID string `json:"accountId"` + Secure bool `json:"secure"` + MaxBytes int64 `json:"maxBytes"` + ContentTypes []string `json:"contentTypes"` + LinkTTL time.Duration `json:"linkTtl"` + RequestTimeout time.Duration `json:"requestTimeout"` + // Secrets are environment-only and cannot be populated by config files. + FilesToken string `json:"-"` + SigningKey string `json:"-"` + AccessKey string `json:"-"` + SecretKey string `json:"-"` + SessionToken string `json:"-"` +} +func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}} +func (s *StorageConfig) applyEnv()error { + values:=map[string]*string{ + "GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL, + "GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region, + "GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID, + "GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken, + "GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey, + "GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken, + } + for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}} + for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} { + if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v} + } + if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v} + if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v} + if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}} + return nil +} +func (s StorageConfig) Validate()error { + if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")} + if err:=s.Policy().Validate();err!=nil{return err} + if s.LinkTTLtime.Hour{return errors.New("storage.linkTtl must be 1s..1h")} + if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")} + {{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}} + {{if eq .Storage "local"}} + if s.Root=="" {return errors.New("storage.root is required")} + u,err:=url.Parse(s.PublicURL) + if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")} + if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")} + {{else}} + if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")} + {{if eq .Storage "r2"}} + if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")} + if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")} + {{else if eq .Storage "s3"}} + u,err:=url.Parse("https://"+s.Endpoint) + if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")} + {{else if eq .Storage "oss"}} + if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}} + if !s.Secure{return errors.New("OSS requires secure=true")} + {{end}} + {{end}} + return nil +} +{{end}} diff --git a/cmd/go-omni/scaffold/templates/legacyV013/modules-storage.go.tmpl b/cmd/go-omni/scaffold/templates/legacyV013/modules-storage.go.tmpl new file mode 100644 index 0000000..fa772ef --- /dev/null +++ b/cmd/go-omni/scaffold/templates/legacyV013/modules-storage.go.tmpl @@ -0,0 +1,79 @@ +package servicecontext + +import ( + "context" + "errors" + {{if eq .Storage "local"}} + "git.yuntuo.hk/k2safe/go-omni/storage/local" + {{else}} + "net" + "net/http" + "time" + {{if eq .Storage "oss"}} + ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss" + ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss" + "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials" + {{else}} + "git.yuntuo.hk/k2safe/go-omni/storage/s3" + "github.com/minio/minio-go/v7/pkg/credentials" + {{end}}{{end}} + "git.yuntuo.hk/k2safe/go-omni/storage" + "{{.Module}}/internal/settings" +) + +// InitStorage is called once, before serving requests. The composition root +// owns provider clients; business services receive only capability interfaces. +func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error { + if c==nil || ctx==nil{return errors.New("storage initialization requires context")} + c.closeMu.Lock();defer c.closeMu.Unlock() + if c.closed{return errors.New("service dependencies are closed")} + if err:=ctx.Err();err!=nil{return err} + if c.Files!=nil{return errors.New("storage already initialized")} + if err:=cfg.Validate();err!=nil{return err} + {{if eq .Storage "local"}} + files,err:=local.New(cfg.Root,cfg.MaxBytes) + if err!=nil{return err} + signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy()) + if err!=nil{return err} + uploader,err:=storage.NewUploader(files,cfg.Policy()) + if err!=nil{return err} + c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer + {{else}} + // Dedicated bounded pool, never the process-wide default transport. + transport:=&http.Transport{ + Proxy:http.ProxyFromEnvironment, + DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext, + ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32, + IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second, + ResponseHeaderTimeout:cfg.RequestTimeout, + } + initialized:=false + defer func(){if !initialized{transport.CloseIdleConnections()}}() + {{if eq .Storage "oss"}} + sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region). + WithSignatureVersion(ossSDK.SignatureVersionV4). + WithAdditionalHeaders([]string{"content-length"}). + WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)). + WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout}) + if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)} + files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes) + {{else}} + creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken) + {{if eq .Storage "r2"}} + options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds) + if err!=nil{return err} + options.MaxBytes,options.Transport=cfg.MaxBytes,transport + {{else}} + options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport} + {{end}} + files,err:=s3.Open(options) + {{end}} + if err!=nil{return err} + uploader,err:=storage.NewUploader(files,cfg.Policy()) + if err!=nil{return err} + c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader + c.storageClose=func()error{transport.CloseIdleConnections();return nil} + initialized=true + {{end}} + return nil +} diff --git a/cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl b/cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl index 3ab9bf8..4a97af5 100644 --- a/cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl +++ b/cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl @@ -49,11 +49,16 @@ server: {{end}}{{end}}{{end}} {{if and .API (eq .Database "mysql")}}database: - driver: mysql - maxOpenConns: 10 - maxIdleConns: 5 - connMaxLifetime: 30m - connMaxIdleTime: 5m - pingTimeout: 5s -# GO_OMNI_DB_DSN supplies the secret connection string. + host: 127.0.0.1 + port: 3306 + database: '' # GO_OMNI_DB_DATABASE + user: '' # GO_OMNI_DB_USER; password via GO_OMNI_DB_PASSWORD + tls: + mode: required + pool: + maxOpenConns: 10 + maxIdleConns: 5 + connMaxLifetime: 30m + connMaxIdleTime: 5m + pingTimeout: 5s {{end}} diff --git a/cmd/go-omni/scaffold/templates/modules-storage.go.tmpl b/cmd/go-omni/scaffold/templates/modules-storage.go.tmpl index fa772ef..b555ec7 100644 --- a/cmd/go-omni/scaffold/templates/modules-storage.go.tmpl +++ b/cmd/go-omni/scaffold/templates/modules-storage.go.tmpl @@ -25,8 +25,8 @@ import ( // owns provider clients; business services receive only capability interfaces. func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error { if c==nil || ctx==nil{return errors.New("storage initialization requires context")} - c.closeMu.Lock();defer c.closeMu.Unlock() - if c.closed{return errors.New("service dependencies are closed")} + {{if .API}}if c.Runtime==nil || c.Context().Err()!=nil{return errors.New("service dependencies are closed")}{{else}}c.closeMu.Lock();defer c.closeMu.Unlock() + if c.closed{return errors.New("service dependencies are closed")}{{end}} if err:=ctx.Err();err!=nil{return err} if c.Files!=nil{return errors.New("storage already initialized")} if err:=cfg.Validate();err!=nil{return err} @@ -72,7 +72,7 @@ func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)err uploader,err:=storage.NewUploader(files,cfg.Policy()) if err!=nil{return err} c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader - c.storageClose=func()error{transport.CloseIdleConnections();return nil} + {{if .API}}c.OnClose(func()error{transport.CloseIdleConnections();return nil}){{else}}c.storageClose=func()error{transport.CloseIdleConnections();return nil}{{end}} initialized=true {{end}} return nil diff --git a/cmd/go-omni/scaffold/version.go b/cmd/go-omni/scaffold/version.go index 33c32bd..0c79a89 100644 --- a/cmd/go-omni/scaffold/version.go +++ b/cmd/go-omni/scaffold/version.go @@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" // Source builds can carry an unpublished VCS version (including +dirty). // Keep this fallback in sync with the next release; installed CLIs use their // downloaded module version, which also has a module checksum. -const defaultFrameworkVersion = "v0.13.0-preview" +const defaultFrameworkVersion = "v0.14.0-preview" // This target is not a claim that the release already exists. Source users // select a local framework checkout until the module templates are published. diff --git a/cmd/go-omni/upgrade.go b/cmd/go-omni/upgrade.go new file mode 100644 index 0000000..f1fb5fe --- /dev/null +++ b/cmd/go-omni/upgrade.go @@ -0,0 +1,44 @@ +package main + +import ( + "errors" + "flag" + "fmt" + "git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen" + "io" +) + +func runUpgrade(args []string) error { + flags := flag.NewFlagSet("upgrade", flag.ContinueOnError) + flags.SetOutput(io.Discard) + dir := flags.String("dir", ".", "service directory") + apply := flags.Bool("apply", false, "apply only recognized baseline changes") + preview := flags.String("preview", "", "write proposed infrastructure into an empty directory") + dry := flags.Bool("dry-run", false, "show the upgrade plan") + if err := flags.Parse(args); err != nil { + return err + } + if flags.NArg() != 0 { + return errors.New("unexpected upgrade argument") + } + if *apply && (*preview != "" || *dry) { + return errors.New("--apply cannot be combined with preview/dry-run") + } + var plan apigen.UpgradePlan + var err error + if *apply { + plan, err = apigen.ApplyUpgrade(*dir) + } else if *preview != "" { + plan, err = apigen.PreviewUpgrade(*dir, *preview) + } else { + plan, err = apigen.PlanUpgrade(*dir) + } + for _, c := range plan.Changes { + kind := "safe" + if c.Manual { + kind = "review" + } + fmt.Printf("%-6s %-6s %s\n", kind, c.Action, c.Path) + } + return err +} diff --git a/cmd/go-omni/workspace.go b/cmd/go-omni/workspace.go new file mode 100644 index 0000000..71c471e --- /dev/null +++ b/cmd/go-omni/workspace.go @@ -0,0 +1,25 @@ +package main + +import ( + "errors" + "flag" + "git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen" + "io" + "os" +) + +func runWorkspace(args []string) error { + if len(args) < 1 { + return errors.New("usage: go-omni workspace --file omni.workspace.json") + } + flags := flag.NewFlagSet("workspace", flag.ContinueOnError) + flags.SetOutput(io.Discard) + path := flags.String("file", "omni.workspace.json", "workspace configuration") + if err := flags.Parse(args[1:]); err != nil { + return err + } + if flags.NArg() != 0 { + return errors.New("unexpected workspace argument") + } + return apigen.Workspace(*path, args[0], os.Stdout) +} diff --git a/config/clone.go b/config/clone.go new file mode 100644 index 0000000..1c5eccf --- /dev/null +++ b/config/clone.go @@ -0,0 +1,81 @@ +package config + +import ( + "errors" + "reflect" + "strings" +) + +// Clone only configuration data. json:"-" capabilities remain borrowed and +// are never traversed (log writers, clients and mutex-bearing runtime objects). +// Unexported immutable implementation details such as time.Time are copied. +func cloneConfiguration(v reflect.Value, depth int) (reflect.Value, error) { + if depth > 64 { + return reflect.Value{}, errors.New("config: defaults contain cycles or excessive nesting") + } + result := reflect.New(v.Type()).Elem() + result.Set(v) + switch v.Kind() { + case reflect.Struct: + for i := 0; i < v.NumField(); i++ { + field := v.Type().Field(i) + if field.PkgPath != "" || strings.Split(field.Tag.Get("json"), ",")[0] == "-" { + continue + } + child, err := cloneConfiguration(v.Field(i), depth+1) + if err != nil { + return reflect.Value{}, err + } + result.Field(i).Set(child) + } + case reflect.Pointer: + if !v.IsNil() { + child, err := cloneConfiguration(v.Elem(), depth+1) + if err != nil { + return reflect.Value{}, err + } + result.Set(reflect.New(v.Type().Elem())) + result.Elem().Set(child) + } + case reflect.Interface: + if !v.IsNil() { + child, err := cloneConfiguration(v.Elem(), depth+1) + if err != nil { + return reflect.Value{}, err + } + result.Set(child) + } + case reflect.Slice: + if !v.IsNil() { + result = reflect.MakeSlice(v.Type(), v.Len(), v.Len()) + for i := 0; i < v.Len(); i++ { + child, err := cloneConfiguration(v.Index(i), depth+1) + if err != nil { + return reflect.Value{}, err + } + result.Index(i).Set(child) + } + } + case reflect.Array: + for i := 0; i < v.Len(); i++ { + child, err := cloneConfiguration(v.Index(i), depth+1) + if err != nil { + return reflect.Value{}, err + } + result.Index(i).Set(child) + } + case reflect.Map: + if !v.IsNil() { + result = reflect.MakeMapWithSize(v.Type(), v.Len()) + iter := v.MapRange() + for iter.Next() { + child, err := cloneConfiguration(iter.Value(), depth+1) + if err != nil { + return reflect.Value{}, err + } + result.SetMapIndex(iter.Key(), child) + } + } + } + return result, nil +} diff --git a/config/config.go b/config/config.go index cf2a845..6a9896a 100644 --- a/config/config.go +++ b/config/config.go @@ -4,6 +4,7 @@ import ( "strings" "time" + "git.yuntuo.hk/k2safe/go-omni/contract" apierrors "git.yuntuo.hk/k2safe/go-omni/errors" "git.yuntuo.hk/k2safe/go-omni/httpx" "git.yuntuo.hk/k2safe/go-omni/logx" @@ -26,20 +27,22 @@ type TLSConfig struct { // Config controls the default runtime behavior of an App. type Config struct { - Address string `json:"address"` - Transport string `json:"transport"` - TLS TLSConfig `json:"tls"` - HTTP HTTPConfig `json:"http"` - Endpoints EndpointsConfig `json:"endpoints"` - RequestIDHeader string `json:"requestIdHeader"` - EnableLogger bool `json:"enableLogger"` - SlowRequestThreshold time.Duration `json:"slowRequestThreshold"` - TrustedProxies []string `json:"trustedProxies"` - ShutdownTimeout time.Duration `json:"shutdownTimeout"` - Log logx.Config `json:"log"` - Errors apierrors.Catalog `json:"errors"` - Response response.Config `json:"response"` - ResponseEncoder response.Encoder `json:"-"` + Request contract.RequestPolicy `json:"request"` + Address string `json:"address"` + Transport string `json:"transport"` + TLS TLSConfig `json:"tls"` + HTTP HTTPConfig `json:"http"` + Endpoints EndpointsConfig `json:"endpoints"` + RequestIDHeader string `json:"requestIdHeader"` + EnableLogger bool `json:"enableLogger"` + SlowRequestThreshold time.Duration `json:"slowRequestThreshold"` + TrustedProxies []string `json:"trustedProxies"` + ShutdownTimeout time.Duration `json:"shutdownTimeout"` + Log logx.Config `json:"log"` + Errors apierrors.Catalog `json:"errors"` + Response response.Config `json:"response"` + ResponseEncoder response.Encoder `json:"-"` + LogInstance *logx.Logger `json:"-"` } func Default() Config { @@ -55,6 +58,12 @@ func Default() Config { // Validate checks configuration values before an application starts. func (c Config) Validate() error { + if err := c.Request.Validate(); err != nil { + return err + } + if err := c.Log.Validate(); err != nil { + return err + } if err := c.Endpoints.Validate(); err != nil { return err } @@ -221,3 +230,10 @@ func WithErrors(catalog apierrors.Catalog) Option { catalog = catalog.Clone() return func(cfg *Config) { cfg.Errors = catalog.Clone() } } + +// WithLogInstance borrows a logger for this App without changing process logging. +func WithLogInstance(logger *logx.Logger) Option { return func(c *Config) { c.LogInstance = logger } } + +func WithRequestPolicy(policy contract.RequestPolicy) Option { + return func(c *Config) { c.Request = policy } +} diff --git a/config/file.go b/config/file.go index c863398..624ec57 100644 --- a/config/file.go +++ b/config/file.go @@ -22,6 +22,23 @@ import ( // field names for both formats, rejects unknown fields and accepts durations // such as "3s". Application-specific validation remains with the caller. func DecodeFile[T any](path string, defaults T) (T, error) { + result, err := decodeFile(path, defaults) + if err != nil { + var zero T + return zero, &DecodeError{cause: err} + } + return result, nil +} + +// DecodeError keeps input values out of public diagnostics while preserving +// errors.Is/As for explicitly handled underlying I/O and syntax errors. +type DecodeError struct{ cause error } + +func (e *DecodeError) Error() string { + return "config: cannot decode file; check its path, format, field names and value types" +} +func (e *DecodeError) Unwrap() error { return e.cause } +func decodeFile[T any](path string, defaults T) (T, error) { var zero T f, err := os.Open(path) if err != nil { @@ -64,15 +81,13 @@ func DecodeFile[T any](path string, defaults T) (T, error) { if err != nil { return zero, err } - // Deep copy defaults so errors cannot partially mutate caller-owned maps. - original, err := json.Marshal(defaults) + // Clone configuration without serializing borrowed json:"-" capabilities. + var result T + original, err := cloneConfiguration(reflect.ValueOf(&defaults).Elem(), 0) if err != nil { return zero, err } - var result T - if err = json.Unmarshal(original, &result); err != nil { - return zero, err - } + reflect.ValueOf(&result).Elem().Set(original) dec := json.NewDecoder(bytes.NewReader(encoded)) dec.DisallowUnknownFields() if err = dec.Decode(&result); err != nil { @@ -115,6 +130,17 @@ func durations(v any, t reflect.Type) (any, error) { } } } + if t.Kind() == reflect.Map && t.Key().Kind() == reflect.String { + if values, ok := v.(map[string]any); ok { + for key, value := range values { + converted, err := durations(value, t.Elem()) + if err != nil { + return nil, err + } + values[key] = converted + } + } + } if t.Kind() == reflect.Slice { if items, ok := v.([]any); ok { for n, item := range items { diff --git a/config/project.go b/config/project.go new file mode 100644 index 0000000..74921ee --- /dev/null +++ b/config/project.go @@ -0,0 +1,290 @@ +package config + +import ( + "bytes" + "encoding" + "encoding/json" + "errors" + "fmt" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "math" + "os" + "reflect" + "strconv" + "strings" + "time" + "unicode" +) + +// EnvOptions controls typed project configuration overlays. Explicit env tags +// are absolute names; otherwise JSON names become PREFIX_NESTED_FIELD names. +// Empty is allow (default), ignore or reject. No input values appear in errors. +type EnvOptions struct { + Prefix string + Empty string + Lookup func(string) (string, bool) +} + +// ApplyDefaults applies envDefault tags to zero fields before file overlays. +func ApplyDefaults[T any](base T) (T, error) { return projectOverlay(base, EnvOptions{}, true) } + +// LoadProject applies code/tag defaults, a strict optional file, environment, +// then the project's Validate method, before any logger or dependency is opened. +// envDefault is only applied to zero-valued defaults before reading the file. +func LoadProject[T any](path string, defaults T, options EnvOptions) (T, error) { + var zero T + cfg, err := projectOverlay(defaults, options, true) + if err != nil { + return zero, err + } + if path != "" { + cfg, err = DecodeFile(path, cfg) + if err != nil { + return zero, errors.New("config: cannot decode file; check its path, format, field names and value types") + } + } + cfg, err = OverlayEnv(cfg, options) + if err != nil { + return zero, err + } + if valid, ok := any(&cfg).(interface{ Validate() error }); ok { + if err = valid.Validate(); err != nil { + return zero, err + } + } + return cfg, nil +} + +// OverlayEnv overlays the entire project struct without mutating the input. +// Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a +// subtree. required means nonempty string/non-nil pointer or collection; false +// and zero are valid scalars. Use project Validate for semantic constraints. +// Slices/maps use JSON syntax, avoiding ambiguous comma and empty semantics. +func OverlayEnv[T any](base T, options EnvOptions) (T, error) { + return projectOverlay(base, options, false) +} + +var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem() +var durationType = reflect.TypeOf(time.Duration(0)) + +func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error) { + var zero T + if options.Lookup == nil { + options.Lookup = os.LookupEnv + } + if options.Empty == "" { + options.Empty = "allow" + } + if options.Empty != "allow" && options.Empty != "ignore" && options.Empty != "reject" { + return zero, errors.New("config: empty policy must be allow, ignore or reject") + } + v := reflect.ValueOf(&base).Elem() + if v.Kind() != reflect.Struct { + return zero, errors.New("config: project configuration must be a struct") + } + result := reflect.New(v.Type()).Elem() + result.Set(v) + seen := map[string]bool{} + var walk func(reflect.Value, string, int) error + walk = func(value reflect.Value, prefix string, depth int) error { + if depth > 32 { + return errors.New("config: nesting exceeds 32") + } + for i := 0; i < value.NumField(); i++ { + field := value.Type().Field(i) + dst := value.Field(i) + if !dst.CanSet() || field.PkgPath != "" { + continue + } + tag := strings.Split(field.Tag.Get("env"), ",") + if tag[0] == "-" { + continue + } + jsonName := strings.Split(field.Tag.Get("json"), ",")[0] + if jsonName == "-" && tag[0] == "" { + continue + } + if jsonName == "" { + jsonName = field.Name + } + key := prefix + envName(jsonName) + if tag[0] != "" { + key = tag[0] + } + empty, required := options.Empty, false + for _, opt := range tag[1:] { + switch opt { + case "required": + required = true + case "secret", "": + case "empty=allow": + empty = "allow" + case "empty=ignore": + empty = "ignore" + case "empty=reject": + empty = "reject" + default: + return fmt.Errorf("config: invalid env tag for %s", key) + } + } + if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) { + if err := walk(dst, key+"_", depth+1); err != nil { + return err + } + continue + } + if dst.Kind() == reflect.Pointer && dst.Type().Elem().Kind() == reflect.Struct && !dst.Type().Implements(textType) { + // Clone before traversing so a failed overlay never edits defaults. + child := reflect.New(dst.Type().Elem()) + if !dst.IsNil() { + child.Elem().Set(dst.Elem()) + } + if err := walk(child.Elem(), key+"_", depth+1); err != nil { + return err + } + if !dst.IsNil() || !child.Elem().IsZero() { + dst.Set(child) + } + if required && !defaults && dst.IsNil() { + return fmt.Errorf("config: %s is required", key) + } + continue + } + // Unexported runtime capabilities are skipped unless explicitly tagged. + if dst.Kind() == reflect.Func || dst.Kind() == reflect.Chan || dst.Kind() == reflect.Interface { + if tag[0] != "" { + return fmt.Errorf("config: %s has unsupported environment type", key) + } + continue + } + if seen[key] { + return fmt.Errorf("config: duplicate environment key %s", key) + } + seen[key] = true + raw, present := options.Lookup(key) + if defaults { + raw, present = field.Tag.Lookup("envDefault") + present = present && dst.IsZero() + } + if present && raw == "" && !defaults { + if empty == "reject" { + return fmt.Errorf("config: %s must not be empty", key) + } + if empty == "ignore" { + present = false + } + } + if present { + parsed := reflect.New(dst.Type()).Elem() + if err := parseEnvValue(parsed, raw); err != nil { + return fmt.Errorf("config: %s must be a valid %s", key, dst.Type()) + } + dst.Set(parsed) + } + if required && !defaults && emptyRequired(dst) { + return fmt.Errorf("config: %s is required", key) + } + } + return nil + } + if err := walk(result, options.Prefix, 0); err != nil { + return zero, err + } + return result.Interface().(T), nil +} + +func emptyRequired(v reflect.Value) bool { + switch v.Kind() { + case reflect.String: + return v.Len() == 0 + case reflect.Pointer, reflect.Map, reflect.Slice: + return v.IsNil() + } + return false +} + +func parseEnvValue(v reflect.Value, raw string) (err error) { + defer func() { + if recover() != nil { + err = errors.New("invalid custom value") + } + }() + if v.Kind() == reflect.Pointer { + v.Set(reflect.New(v.Type().Elem())) + return parseEnvValue(v.Elem(), raw) + } + if v.CanAddr() && v.Addr().Type().Implements(textType) { + return v.Addr().Interface().(encoding.TextUnmarshaler).UnmarshalText([]byte(raw)) + } + if v.Type() == durationType { + d, err := time.ParseDuration(raw) + if err == nil { + v.SetInt(int64(d)) + } + return err + } + switch v.Kind() { + case reflect.String: + v.SetString(raw) + return nil + case reflect.Bool: + x, err := strconv.ParseBool(raw) + if err == nil { + v.SetBool(x) + } + return err + case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64: + x, err := strconv.ParseInt(raw, 10, v.Type().Bits()) + if err == nil { + v.SetInt(x) + } + return err + case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64: + x, err := strconv.ParseUint(raw, 10, v.Type().Bits()) + if err == nil { + v.SetUint(x) + } + return err + case reflect.Float32, reflect.Float64: + x, err := strconv.ParseFloat(raw, v.Type().Bits()) + if err == nil && (math.IsNaN(x) || math.IsInf(x, 0)) { + return errors.New("nonfinite value") + } + if err == nil { + v.SetFloat(x) + } + return err + case reflect.Slice, reflect.Map: + var value any + if err := apierrors.DecodeCatalogJSON([]byte(raw), &value); err != nil { + return err + } + value, err := durations(value, v.Type()) + if err != nil { + return err + } + encoded, err := json.Marshal(value) + if err != nil { + return err + } + decoder := json.NewDecoder(bytes.NewReader(encoded)) + decoder.DisallowUnknownFields() + return decoder.Decode(v.Addr().Interface()) + } + return errors.New("unsupported environment value") +} + +func envName(s string) string { + r := []rune(s) + var out strings.Builder + for i, c := range r { + if i > 0 && unicode.IsUpper(c) && (unicode.IsLower(r[i-1]) || unicode.IsDigit(r[i-1]) || (i+1 < len(r) && unicode.IsLower(r[i+1]))) { + out.WriteByte('_') + } + if c == '-' { + c = '_' + } + out.WriteRune(unicode.ToUpper(c)) + } + return out.String() +} diff --git a/config/projectAdvanced_test.go b/config/projectAdvanced_test.go new file mode 100644 index 0000000..fd5cbf7 --- /dev/null +++ b/config/projectAdvanced_test.go @@ -0,0 +1,61 @@ +package config + +import ( + "bytes" + "io" + "os" + "path/filepath" + "strings" + "testing" + "time" +) + +func TestConfigFilePreservesDefaultsWithoutSharingMutableData(t *testing.T) { + type cfg struct { + Writer io.Writer `json:"-"` + Secret string `json:"-"` + Values map[string]int64 `json:"values"` + Timers map[string]time.Duration `json:"timers"` + ID int64 `json:"id"` + } + var writer bytes.Buffer + base := cfg{Writer: &writer, Secret: "borrowedSecret", Values: map[string]int64{"old": 7}, ID: 9007199254740993} + file := filepath.Join(t.TempDir(), "config.json") + if err := os.WriteFile(file, []byte(`{"values":{"new":2},"timers":{"retry":"2s"}}`), 0600); err != nil { + t.Fatal(err) + } + next, err := DecodeFile(file, base) + if err != nil { + t.Fatal(err) + } + if next.Writer != &writer || next.Secret != base.Secret || next.ID != base.ID || next.Timers["retry"] != 2*time.Second || len(base.Values) != 1 { + t.Fatal("defaults lost or mutated") + } + next.Values["old"] = 9 + if base.Values["old"] != 7 { + t.Fatal("defaults shared map") + } + if err = os.WriteFile(file, []byte(`{"values":{"secretValue":"credential-super-secret"}}`), 0600); err != nil { + t.Fatal(err) + } + if _, err = DecodeFile(file, base); err == nil || strings.Contains(err.Error(), "credential-super-secret") { + t.Fatal("decode error leaked contents", err) + } +} +func TestEnvCollectionPrecisionAndDurationMaps(t *testing.T) { + type cfg struct { + IDs []int64 `json:"ids"` + Timers map[string]time.Duration `json:"timers"` + } + lookup := func(key string) (string, bool) { + v, ok := map[string]string{"IDS": "[9007199254740993]", "TIMERS": `{"backoff":"250ms"}`}[key] + return v, ok + } + c, err := OverlayEnv(cfg{}, EnvOptions{Lookup: lookup}) + if err != nil || c.IDs[0] != 9007199254740993 || c.Timers["backoff"] != 250*time.Millisecond { + t.Fatal(c, err) + } + if _, err = OverlayEnv(cfg{}, EnvOptions{Lookup: func(key string) (string, bool) { return `{"x":"1s","x":"2s"}`, key == "TIMERS" }}); err == nil { + t.Fatal("duplicate map key accepted") + } +} diff --git a/config/project_test.go b/config/project_test.go new file mode 100644 index 0000000..83119d9 --- /dev/null +++ b/config/project_test.go @@ -0,0 +1,124 @@ +package config + +import ( + "errors" + "os" + "path/filepath" + "reflect" + "strings" + "testing" + "time" +) + +type projectNested struct { + ReadTimeout time.Duration `json:"readTimeout"` + Names []string `json:"names"` +} +type projectConfig struct { + Nested *projectNested `json:"nested"` + Enabled bool `json:"enabled"` + Count int `json:"count" envDefault:"12"` + Password string `json:"-" env:"ETAX_PASSWORD,secret,required,empty=reject"` + Label string `json:"label" env:",empty=ignore"` +} + +func (c projectConfig) Validate() error { + if c.Count < 0 { + return errors.New("count must be nonnegative") + } + return nil +} +func TestProjectConfigurationOrderAndValues(t *testing.T) { + f := filepath.Join(t.TempDir(), "config.yaml") + if err := os.WriteFile(f, []byte("count: 3\nenabled: true\nnested:\n readTimeout: 2s\n names: [file]\nlabel: keep\n"), 0600); err != nil { + t.Fatal(err) + } + env := map[string]string{"ETAX_PASSWORD": " password ", "APP_ENABLED": "false", "APP_COUNT": "0", "APP_NESTED_READ_TIMEOUT": "7s", "APP_NESTED_NAMES": "[]", "APP_LABEL": ""} + options := EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }} + cfg, err := LoadProject(f, projectConfig{}, options) + if err != nil { + t.Fatal(err) + } + if cfg.Enabled || cfg.Count != 0 || cfg.Password != " password " || cfg.Nested.ReadTimeout != 7*time.Second || cfg.Nested.Names == nil || len(cfg.Nested.Names) != 0 || cfg.Label != "keep" { + t.Fatal("overlay mismatch") + } + delete(env, "APP_COUNT") + cfg, err = LoadProject("", projectConfig{}, options) + if err != nil || cfg.Count != 12 { + t.Fatal(cfg.Count, err) + } + env["APP_COUNT"] = "-1" + if _, err = LoadProject(f, projectConfig{}, options); err == nil { + t.Fatal("project Validate not called") + } +} + +func TestProjectOverlayIsTransactionalAndRedacted(t *testing.T) { + base := projectConfig{Nested: &projectNested{ReadTimeout: time.Second, Names: []string{"old"}}, Password: "original"} + env := map[string]string{"APP_NESTED_READ_TIMEOUT": "2s", "APP_NESTED_NAMES": "[\"new\"]", "APP_COUNT": "secret-value"} + _, err := OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}) + if err == nil || strings.Contains(err.Error(), "secret-value") { + t.Fatal(err) + } + if base.Nested.ReadTimeout != time.Second || !reflect.DeepEqual(base.Nested.Names, []string{"old"}) { + t.Fatal("default pointer modified") + } + delete(env, "APP_COUNT") + env["ETAX_PASSWORD"] = "" + if _, err = OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}); err == nil { + t.Fatal("empty secret accepted") + } + type required struct { + Token string `env:"TOKEN,required"` + } + if _, err = OverlayEnv(required{}, EnvOptions{Lookup: func(string) (string, bool) { return "", false }}); err == nil { + t.Fatal("required absent") + } + type duplicate struct { + A string `env:"SAME"` + B string `env:"SAME"` + } + if _, err = OverlayEnv(duplicate{}, EnvOptions{}); err == nil { + t.Fatal("duplicate env binding accepted") + } +} + +func TestEnvironmentCollectionAndEmptySemantics(t *testing.T) { + type sample struct { + Values []time.Duration + Flag bool + Text string + Optional *int + } + cases := []struct { + raw string + valid bool + }{{`["1s","2s"]`, true}, {`["invalid-secret"]`, false}} + for _, tc := range cases { + cfg, err := OverlayEnv(sample{}, EnvOptions{Lookup: func(k string) (string, bool) { + if k == "VALUES" { + return tc.raw, true + } + return "", false + }}) + if (err == nil) != tc.valid { + t.Fatal(err) + } + if tc.valid && !reflect.DeepEqual(cfg.Values, []time.Duration{time.Second, 2 * time.Second}) { + t.Fatal(cfg) + } + if err != nil && strings.Contains(err.Error(), "secret") { + t.Fatal(err) + } + } + for _, mode := range []string{"allow", "ignore", "reject"} { + cfg, err := OverlayEnv(sample{Text: "old"}, EnvOptions{Empty: mode, Lookup: func(k string) (string, bool) { return "", k == "TEXT" }}) + if mode == "reject" { + if err == nil { + t.Fatal("empty not rejected") + } + } else if err != nil || (mode == "allow" && cfg.Text != "") || (mode == "ignore" && cfg.Text != "old") { + t.Fatal(cfg, err) + } + } +} diff --git a/contract/bind.go b/contract/bind.go index 547c944..2db0cc3 100644 --- a/contract/bind.go +++ b/contract/bind.go @@ -1,6 +1,7 @@ package contract import ( + "bufio" "encoding/json" "errors" "fmt" @@ -11,6 +12,7 @@ import ( "net/url" "reflect" "strconv" + "strings" apierrors "git.yuntuo.hk/k2safe/go-omni/errors" ) @@ -19,9 +21,17 @@ import ( // strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query // parameters never overwrite path/body/header values. form means URL query only. func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error { + return b.BindWithPolicy(r, params, target, RequestPolicy{}, maxBytes...) +} + +// BindWithPolicy merges route overrides over the App request policy. +func (b *Binder) BindWithPolicy(r *http.Request, params map[string]string, target any, policy RequestPolicy, maxBytes ...int64) error { if b == nil || r == nil || b.schemaOnly { return apierrors.Internal(errors.New("contract: nil binder or request")) } + if b.HasMultipart() { + return apierrors.Internal(errors.New("multipart fields require BindMultipart")) + } v := reflect.ValueOf(target) if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ { return apierrors.Internal(errors.New("contract: target type mismatch")) @@ -33,54 +43,9 @@ func (b *Binder) Bind(r *http.Request, params map[string]string, target any, max if len(maxBytes) == 1 { limit = maxBytes[0] } - fresh := reflect.New(b.typ).Elem() - body := map[string]any{} - if len(b.root.schema.Properties) > 0 { - ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type")) - if err != nil || ct != "application/json" { - return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json") - } - if r.Body == nil { - return invalid(errors.New("body required")) - } - if r.ContentLength > limit { - return tooLarge() - } - reader := &io.LimitedReader{R: r.Body, N: limit + 1} - dec := json.NewDecoder(reader) - dec.UseNumber() - value, err := decodeValue(dec, 0) - if reader.N == 0 { - return tooLarge() - } - if err != nil { - var maxErr *http.MaxBytesError - if errors.As(err, &maxErr) { - return tooLarge() - } - return invalid(err) - } - _, err = dec.Token() - if reader.N == 0 { - return tooLarge() - } - if err != io.EOF { - var maxErr *http.MaxBytesError - if errors.As(err, &maxErr) { - return tooLarge() - } - return invalid(errors.New("exactly one JSON value required")) - } - var ok bool - body, ok = value.(map[string]any) - if !ok { - return invalid(errors.New("body must be an object")) - } - for key := range body { - if _, ok = b.root.schema.Properties[key]; !ok { - return invalid(fmt.Errorf("unknown body field %s", key)) - } - } + policy = MergePolicy(RequestPolicyFromContext(r.Context()), policy) + if err := policy.Validate(); err != nil { + return apierrors.Internal(err) } query := url.Values{} if r.URL != nil { @@ -90,10 +55,121 @@ func (b *Binder) Bind(r *http.Request, params map[string]string, target any, max return invalid(errors.New("malformed query")) } } + if policy.UnknownQuery == "reject" { + allowed := map[string]bool{} + for _, f := range b.root.fields { + if f.source == "form" { + allowed[f.name] = true + } + } + for key := range query { + if !allowed[key] { + return invalid(fmt.Errorf("unknown query field %s", key)) + } + } + } + fresh := reflect.New(b.typ).Elem() + body := map[string]any{} + bodyAbsent := false + bodyMode := policy.Body + if bodyMode == "" || bodyMode == "auto" { + if len(b.root.schema.Properties) > 0 { + bodyMode = "required" + } else { + bodyMode = "auto" + } + } + if (bodyMode == "none" || bodyMode == "emptyObject") && len(b.root.schema.Properties) > 0 { + return apierrors.Internal(errors.New("request body policy conflicts with JSON fields")) + } + if bodyMode == "none" { + if r.Body != nil { + var one [1]byte + n, err := io.ReadFull(r.Body, one[:]) + if n > 0 { + return invalid(errors.New("request body is forbidden")) + } + if err != io.EOF { + return invalid(err) + } + } + } + if bodyMode == "required" || bodyMode == "optional" || bodyMode == "emptyObject" { + var bodyReader io.Reader = r.Body + if bodyReader == nil { + bodyReader = strings.NewReader("") + } + if r.ContentLength > limit { + return tooLarge() + } + reader := &io.LimitedReader{R: bodyReader, N: limit + 1} + var checked io.Reader = reader + if policy.UTF8 == "reject" { + checked = &utf8Reader{r: checked} + } + buffered := bufio.NewReader(checked) + _, peekErr := buffered.Peek(1) + absent := peekErr == io.EOF + bodyAbsent = absent + if absent && bodyMode != "optional" { + return invalid(errors.New("body required")) + } + if peekErr != nil && !absent { + var maxErr *http.MaxBytesError + if errors.As(peekErr, &maxErr) { + return tooLarge() + } + return invalid(peekErr) + } + if !absent { + ct, mediaParams, err := mime.ParseMediaType(r.Header.Get("Content-Type")) + if err != nil || ct != "application/json" || (policy.Charset == "utf8" && mediaParams["charset"] != "" && !strings.EqualFold(mediaParams["charset"], "utf-8")) { + return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json with an allowed charset") + } + dec := json.NewDecoder(buffered) + dec.UseNumber() + value, err := decodeValue(dec, 0) + if reader.N == 0 { + return tooLarge() + } + if err != nil { + var maxErr *http.MaxBytesError + if errors.As(err, &maxErr) { + return tooLarge() + } + return invalid(err) + } + _, err = dec.Token() + if reader.N == 0 { + return tooLarge() + } + if err != io.EOF { + var maxErr *http.MaxBytesError + if errors.As(err, &maxErr) { + return tooLarge() + } + return invalid(errors.New("exactly one JSON value required")) + } + var ok bool + body, ok = value.(map[string]any) + if !ok { + return invalid(errors.New("body must be an object")) + } + for key := range body { + if _, ok = b.root.schema.Properties[key]; !ok { + return invalid(fmt.Errorf("unknown body field %s", key)) + } + } + } + } + for _, f := range b.root.fields { var value any var present bool if f.source == "json" { + if bodyAbsent && bodyMode == "optional" { + continue + } value, present = body[f.name] } else { var raw []string diff --git a/contract/contract.go b/contract/contract.go index d7bb7ef..1f32dba 100644 --- a/contract/contract.go +++ b/contract/contract.go @@ -36,6 +36,8 @@ type node struct { } type field struct { index int + maxBytes int64 + maxFiles int name, source string value *node optional bool @@ -113,6 +115,13 @@ func (c compiler) compile(t reflect.Type, root bool, stack map[reflect.Type]bool if t.Kind() != reflect.Pointer && (t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler)) { return nil, errors.New("contract: custom JSON marshalers require explicit Scalar registration") } + if t == uploadType { + if root { + return nil, errors.New("upload File cannot be a root DTO") + } + n.schema = &openapi.Schema{Type: "string", Format: "binary"} + return n, nil + } switch t.Kind() { case reflect.Interface: if t.NumMethod() != 0 { @@ -223,7 +232,7 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl var sourceTag string for key, value := range tags { switch key { - case "json", "path", "form", "header": + case "json", "path", "form", "header", "file", "multipart", "multipartJSON": if value == "-" { return f, errors.New("ignored fields are not part of an API contract") } @@ -232,6 +241,8 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl } f.source = key sourceTag = value + case "description", "example", "enumDescriptions": + // Documentation is validated after the field constraints. case "binding": f.binding = value default: @@ -254,6 +265,45 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl return f, err } f.value = n + if f.source == "file" { + if sf.Type != uploadType && (sf.Type.Kind() != reflect.Slice || sf.Type.Elem() != uploadType) { + return f, errors.New("file field requires File or []File") + } + seen := map[string]bool{} + for _, option := range parts[1:] { + key, v, _ := strings.Cut(option, "=") + if seen[key] { + return f, errors.New("duplicate file option") + } + seen[key] = true + switch key { + case "optional": + if option != key { + return f, errors.New("optional takes no value") + } + f.optional = true + case "maxBytes": + f.maxBytes, err = fieldLimit(v) + if err != nil { + return f, err + } + case "maxFiles": + f.maxFiles, err = strconv.Atoi(v) + if err != nil || f.maxFiles < 1 || f.maxFiles > 10000 || sf.Type.Kind() != reflect.Slice { + return f, errors.New("maxFiles requires []File and a limit 1..10000") + } + default: + return f, fmt.Errorf("unsupported file option %q", key) + } + } + if f.binding != "" { + return f, errors.New("file constraints use optional/maxBytes/maxFiles") + } + return f, applyDocumentation(&f, tags) + } + if containsUpload(n) { + return f, errors.New("File requires file source") + } opaque := base(n).scalar isList := base(n).typ.Kind() == reflect.Slice && opaque == nil if isList { @@ -263,25 +313,35 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl if isList && (f.source == "path" || f.source == "header") { return f, errors.New("path/header values must be scalars") } - if f.source != "json" && !opaque.Text { + if !jsonSource(f.source) && !opaque.Text { return f, errors.New("custom scalar needs text: true and UnmarshalText for path/query/header") } if f.binding != "" { return f, errors.New("custom scalar constraints belong in its decoder; use optional/pointer for presence and null") } for _, option := range parts[1:] { + if strings.HasPrefix(option, "maxBytes=") && multipartSource(f.source) { + f.maxBytes, err = fieldLimit(strings.TrimPrefix(option, "maxBytes=")) + if err != nil { + return f, err + } + continue + } if option != "optional" && option != "omitempty" { - return f, errors.New("custom scalar supports only optional/omitempty field options") + return f, errors.New("custom scalar supports optional/omitempty and multipart maxBytes") } f.optional = true } if f.source == "path" && (f.optional || sf.Type.Kind() == reflect.Pointer) { return f, errors.New("path fields must be required non-pointer values") } + if err := applyDocumentation(&f, tags); err != nil { + return f, err + } return f, nil } scalar := base(n).typ.Kind() - if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) { + if !jsonSource(f.source) && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) { return f, errors.New("path/query/header values must be scalars or scalar slices") } if f.source == "path" && !isScalar(scalar) { @@ -303,6 +363,14 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl return f, errors.New("optional takes no value") } f.optional = true + case "maxBytes": + if !multipartSource(f.source) { + return f, errors.New("maxBytes field option is only for multipart") + } + f.maxBytes, err = fieldLimit(value) + if err != nil { + return f, err + } case "default": if !isScalar(scalar) { return f, errors.New("default requires a scalar") @@ -354,6 +422,9 @@ func (c compiler) compileField(sf reflect.StructField, root bool, stack map[refl return f, fmt.Errorf("default violates validation: %w", err) } } + if err := applyDocumentation(&f, tags); err != nil { + return f, err + } return f, nil } @@ -432,7 +503,7 @@ func (b *Binder) ResponseSchema() (*openapi.Schema, error) { func (b *Binder) Parameters() []openapi.Parameter { var out []openapi.Parameter for _, f := range b.root.fields { - if f.source == "json" { + if f.source == "json" || multipartSource(f.source) { continue } in := f.source @@ -445,7 +516,7 @@ func (b *Binder) Parameters() []openapi.Parameter { } else if base(f.value).typ.Kind() == reflect.Slice && base(base(f.value).elem).scalar != nil { schema.Items = &openapi.Schema{Type: "string"} } - out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema}) + out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema, Description: f.value.schema.Description}) } return out } @@ -455,3 +526,10 @@ func copySchema(s *openapi.Schema) *openapi.Schema { _ = json.Unmarshal(data, &out) return &out } + +func containsUpload(n *node) bool { + if n.typ == uploadType { + return true + } + return n.elem != nil && containsUpload(n.elem) +} diff --git a/contract/documentation.go b/contract/documentation.go new file mode 100644 index 0000000..9f74ebe --- /dev/null +++ b/contract/documentation.go @@ -0,0 +1,174 @@ +package contract + +import ( + "encoding/json" + "errors" + "fmt" + "io" + "reflect" + "strings" +) + +// Documentation is part of the contract compiler, so CLI and runtime agree. +func applyDocumentation(f *field, tags map[string]string) error { + s := f.value.schema + if f.maxBytes > 0 { + v := f.maxBytes + s.MaxBytes = &v + } + if f.source == "file" && f.value.typ.Kind() == reflect.Slice { + v := f.maxFiles + if v == 0 { + v = 16 + } + s.MaxItems = &v + } + s.Description = tags["description"] + if raw, ok := tags["enumDescriptions"]; ok { + if err := json.Unmarshal([]byte(raw), &s.EnumDescriptions); err != nil { + return errors.New("enumDescriptions must be a JSON string array") + } + count := len(s.Enum) + if s.Nullable && count > 0 && s.Enum[count-1] == nil { + count-- + } + if count == 0 || len(s.EnumDescriptions) != count { + return errors.New("enumDescriptions must describe every declared enum value") + } + for _, v := range s.EnumDescriptions { + if strings.TrimSpace(v) == "" { + return errors.New("enum description cannot be empty") + } + } + } + if raw, ok := tags["example"]; ok { + var value any + if base(f.value).typ.Kind() == reflect.String && base(f.value).scalar == nil { + value = raw + } else { + decoder := json.NewDecoder(strings.NewReader(raw)) + decoder.UseNumber() + var err error + value, err = decodeValue(decoder, 0) + if err != nil { + return errors.New("invalid example JSON") + } + if _, err = decoder.Token(); err != io.EOF { + return errors.New("example requires one value") + } + } + // A schema-only custom scalar never executes project decoders. + if !documentationHasScalar(f.value) { + v := reflect.New(f.value.typ).Elem() + if err := setValue(f.value, v, value); err != nil { + return fmt.Errorf("example: %w", err) + } + if err := checkField(*f, v); err != nil { + return fmt.Errorf("example: %w", err) + } + } else if err := checkDocumentationScalarTree(f.value, value); err != nil { + return err + } + s.Example = value + } + return nil +} + +func documentationHasScalar(n *node) bool { + if n.scalar != nil { + return true + } + if n.elem != nil && documentationHasScalar(n.elem) { + return true + } + for _, f := range n.fields { + if documentationHasScalar(f.value) { + return true + } + } + return false +} + +// Metadata never invokes application decoders, including nested scalar types. +func checkDocumentationScalarTree(n *node, value any) error { + if value == nil { + if n.typ.Kind() == reflect.Pointer || n.typ.Kind() == reflect.Interface { + return nil + } + return errors.New("example null requires a nullable field") + } + if n.typ.Kind() == reflect.Pointer { + return checkDocumentationScalarTree(n.elem, value) + } + if n.scalar != nil { + for _, kind := range strings.Split(n.scalar.JSONInput, "|") { + if scalarAccepts(strings.TrimSpace(kind), value) { + return nil + } + } + return errors.New("example does not match scalar input") + } + if !documentationHasScalar(n) { + v := reflect.New(n.typ).Elem() + return setValue(n, v, value) + } + switch n.typ.Kind() { + case reflect.Slice: + values, ok := value.([]any) + if !ok { + return errors.New("example requires an array") + } + if n.schema.MinItems != nil && len(values) < *n.schema.MinItems || n.schema.MaxItems != nil && len(values) > *n.schema.MaxItems { + return errors.New("example violates item limits") + } + for _, v := range values { + if err := checkDocumentationScalarTree(n.elem, v); err != nil { + return err + } + } + case reflect.Map: + values, ok := value.(map[string]any) + if !ok { + return errors.New("example requires an object") + } + for _, v := range values { + if err := checkDocumentationScalarTree(n.elem, v); err != nil { + return err + } + } + case reflect.Struct: + values, ok := value.(map[string]any) + if !ok { + return errors.New("example requires an object") + } + declared := map[string]bool{} + for _, f := range n.fields { + declared[f.name] = true + v, ok := values[f.name] + if !ok { + if !f.optional && !f.hasDefault { + return errors.New("example missing field") + } + continue + } + if err := checkDocumentationScalarTree(f.value, v); err != nil { + return err + } + if !documentationHasScalar(f.value) { + typed := reflect.New(f.value.typ).Elem() + if err := setValue(f.value, typed, v); err != nil { + return err + } + if err := checkField(f, typed); err != nil { + return err + } + } + } + for name := range values { + if !declared[name] { + return errors.New("unknown example field") + } + } + } + return nil +} diff --git a/contract/documentation_test.go b/contract/documentation_test.go new file mode 100644 index 0000000..0d7f432 --- /dev/null +++ b/contract/documentation_test.go @@ -0,0 +1,55 @@ +package contract + +import ( + "encoding/json" + "reflect" + "testing" +) + +type documentationScalar string + +var documentationDecodeCalls int + +func (v documentationScalar) MarshalJSON() ([]byte, error) { return json.Marshal(string(v)) } +func (v *documentationScalar) UnmarshalJSON(raw []byte) error { + documentationDecodeCalls++ + var s string + err := json.Unmarshal(raw, &s) + *v = documentationScalar(s) + return err +} +func TestDocumentationNeverRunsNestedScalarDecoders(t *testing.T) { + type nested struct { + ID documentationScalar `json:"id"` + } + type input struct { + Value nested `json:"value" example:"{\"id\":\"123\"}"` + } + documentationDecodeCalls = 0 + scalar := ScalarOf[documentationScalar]("string", "string", false) + if _, err := New[input](scalar); err != nil { + t.Fatal(err) + } + if _, err := Describe(reflect.TypeOf(input{}), scalar); err != nil { + t.Fatal(err) + } + if documentationDecodeCalls != 0 { + t.Fatal("metadata invoked business decoder") + } + type bad struct { + Value nested `json:"value" example:"{\"id\":123}"` + } + if _, err := Describe(reflect.TypeOf(bad{}), scalar); err == nil { + t.Fatal("invalid scalar wire example accepted") + } + type constrained struct { + ID documentationScalar `json:"id"` + Count int `json:"count,range=[1:3]"` + } + type invalidConstraint struct { + Value constrained `json:"value" example:"{\"id\":\"123\",\"count\":9}"` + } + if _, err := Describe(reflect.TypeOf(invalidConstraint{}), scalar); err == nil { + t.Fatal("nested scalar bypassed another field's example constraints") + } +} diff --git a/contract/file.go b/contract/file.go new file mode 100644 index 0000000..19abb8c --- /dev/null +++ b/contract/file.go @@ -0,0 +1,80 @@ +package contract + +import ( + "errors" + "io" + "os" + "sync" +) + +// File is a request-scoped upload, staged on disk using a bounded copy buffer. +// Name and ContentType are untrusted client metadata. Open streams content; +// readers and the temporary file are closed when the generated handler returns. +// Persist wanted content before returning. Retaining File does not extend its lifetime. +type File struct { + Name string + ContentType string + Size int64 + state *uploadState +} +type uploadState struct { + mu sync.Mutex + path string + closed bool + readers map[*uploadReader]bool +} +type uploadReader struct { + file *os.File + once sync.Once + err error + state *uploadState +} + +// Present reports whether the optional upload was supplied. +func (f File) Present() bool { return f.state != nil } + +func (f File) Open() (io.ReadCloser, error) { + if f.state == nil { + return nil, errors.New("upload file is absent") + } + s := f.state + s.mu.Lock() + defer s.mu.Unlock() + if s.closed { + return nil, errors.New("upload file is closed") + } + file, err := os.Open(s.path) + if err != nil { + return nil, err + } + r := &uploadReader{file: file, state: s} + s.readers[r] = true + return r, nil +} +func (r *uploadReader) Read(p []byte) (int, error) { return r.file.Read(p) } +func (r *uploadReader) Close() error { + r.once.Do(func() { r.err = r.file.Close(); r.state.mu.Lock(); delete(r.state.readers, r); r.state.mu.Unlock() }) + return r.err +} +func (f File) cleanup() error { + if f.state == nil { + return nil + } + s := f.state + s.mu.Lock() + if s.closed { + s.mu.Unlock() + return nil + } + s.closed = true + readers := make([]*uploadReader, 0, len(s.readers)) + for r := range s.readers { + readers = append(readers, r) + } + s.mu.Unlock() + var err error + for _, r := range readers { + err = errors.Join(err, r.Close()) + } + return errors.Join(err, os.Remove(s.path)) +} diff --git a/contract/multipart.go b/contract/multipart.go new file mode 100644 index 0000000..1b1c31a --- /dev/null +++ b/contract/multipart.go @@ -0,0 +1,331 @@ +package contract + +import ( + "encoding/json" + "errors" + "fmt" + "io" + "mime" + "net/http" + "os" + "path" + "reflect" + "strconv" + "strings" + "sync" + "unicode/utf8" + + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "git.yuntuo.hk/k2safe/go-omni/httpx" + "git.yuntuo.hk/k2safe/go-omni/openapi" +) + +var uploadType = reflect.TypeOf(File{}) + +// MultipartOptions limits raw request bytes (including headers), part count, +// and individual decoded part sizes. Zero selects bounded defaults. Files are +// spooled to 0600 temporary files; text/JSON parts are buffered up to MaxFieldBytes. +type MultipartOptions struct { + MaxBodyBytes int64 + MaxParts int + MaxFieldBytes int64 + MaxFileBytes int64 + TempDir string + Policy RequestPolicy +} + +// Normalized returns validated effective protocol limits without allocating resources. +func (o MultipartOptions) Normalized() (MultipartOptions, error) { + if o.MaxBodyBytes == 0 { + o.MaxBodyBytes = DefaultMaxBodyBytes + } + if o.MaxParts == 0 { + o.MaxParts = 128 + } + if o.MaxFieldBytes == 0 { + o.MaxFieldBytes = 64 << 10 + } + if o.MaxFileBytes == 0 { + o.MaxFileBytes = o.MaxBodyBytes + } + if o.MaxBodyBytes < 1 || o.MaxBodyBytes > 1<<40 || o.MaxParts < 1 || o.MaxParts > 10000 || o.MaxFieldBytes < 1 || o.MaxFieldBytes > 1<<30 || o.MaxFileBytes < 1 || o.MaxFileBytes > 1<<40 { + return o, errors.New("invalid multipart limits") + } + return o, o.Policy.Validate() +} +func (b *Binder) HasMultipart() bool { + for _, f := range b.root.fields { + if multipartSource(f.source) { + return true + } + } + return false +} +func multipartSource(s string) bool { return s == "file" || s == "multipart" || s == "multipartJSON" } +func jsonSource(s string) bool { return s == "json" || s == "multipartJSON" } + +// MultipartSchema describes form parts. JSON fields get an encoding declaration +// in generated OpenAPI; they are decoded with the ordinary strict JSON binder. +func (b *Binder) MultipartSchema() *openapi.Schema { + s := &openapi.Schema{Type: "object", Properties: map[string]*openapi.Schema{}} + no := false + s.AdditionalPropertiesAllowed = &no + for _, f := range b.root.fields { + if multipartSource(f.source) { + s.Properties[f.name] = copySchema(f.value.schema) + if !f.optional && !f.hasDefault { + s.Required = append(s.Required, f.name) + } + } + } + return s +} +func (b *Binder) MultipartJSONFields() []string { + var out []string + for _, f := range b.root.fields { + if f.source == "multipartJSON" { + out = append(out, f.name) + } + } + return out +} + +// BindMultipart validates every part before publishing the DTO or entering Logic. +// On success the caller must defer cleanup; on failure all created files/readers +// are cleaned here. The request's Body remains owned by the HTTP server. +func (b *Binder) BindMultipart(r *http.Request, params map[string]string, target any, options MultipartOptions) (cleanup func() error, err error) { + if b == nil || r == nil || b.schemaOnly { + return nil, apierrors.Internal(errors.New("invalid multipart binder")) + } + dst := reflect.ValueOf(target) + if dst.Kind() != reflect.Pointer || dst.IsNil() || dst.Elem().Type() != b.typ { + return nil, apierrors.Internal(errors.New("multipart target mismatch")) + } + o, err := options.Normalized() + if err != nil { + return nil, apierrors.Internal(err) + } + if b.BodySchema() != nil { + return nil, apierrors.Internal(errors.New("multipart request cannot contain json source fields; use multipartJSON")) + } + if r.ContentLength > o.MaxBodyBytes { + return nil, tooLarge() + } + var files []File + var once sync.Once + var closeErr error + release := func() error { + once.Do(func() { + for i := len(files) - 1; i >= 0; i-- { + closeErr = errors.Join(closeErr, files[i].cleanup()) + } + }) + return closeErr + } + success := false + defer func() { + if !success { + err = errors.Join(err, release()) + } + }() + // Bind query/path/header once, independently of multipart body contents. + ordinary := *b + root := *b.root + root.fields = nil + ordinary.root = &root + for _, f := range b.root.fields { + if !multipartSource(f.source) { + root.fields = append(root.fields, f) + } + } + fresh := reflect.New(b.typ) + policy := o.Policy + policy.Body = "auto" + if err = ordinary.BindWithPolicy(r, params, fresh.Interface(), policy); err != nil { + return nil, err + } + request := *r + input := r.Body + if input == nil { + input = http.NoBody + } + request.Body = http.MaxBytesReader(nil, input, o.MaxBodyBytes) + body, err := httpx.NewBody(&request, "multipart") + if err != nil { + return nil, err + } + fields := map[string]field{} + for _, f := range b.root.fields { + if multipartSource(f.source) { + fields[f.name] = f + } + } + text := map[string][]string{} + uploaded := map[string][]File{} + seen := map[string]int{} + policy = MergePolicy(RequestPolicyFromContext(r.Context()), o.Policy) + for count := 0; ; count++ { + part, e := body.Parts.NextRawPart() + if e == io.EOF { + break + } + if e != nil { + return nil, multipartFailure(e) + } + if count >= o.MaxParts { + return nil, tooLarge() + } + f, ok := fields[part.FormName()] + if !ok { + return nil, invalid(errors.New("unknown multipart field")) + } + seen[f.name]++ + // Reject encodings that would make declared byte limits ambiguous. + if part.Header.Get("Content-Transfer-Encoding") != "" { + return nil, invalid(errors.New("encoded multipart parts are unsupported")) + } + _, disposition, e := mime.ParseMediaType(part.Header.Get("Content-Disposition")) + if e != nil { + return nil, invalid(e) + } + _, isFile := disposition["filename"] + if (f.source == "file") != isFile { + return nil, invalid(errors.New("multipart field kind mismatch")) + } + limit := o.MaxFieldBytes + if f.source == "file" { + limit = o.MaxFileBytes + if f.maxBytes > 0 && f.maxBytes < limit { + limit = f.maxBytes + } + maxFiles := f.maxFiles + if maxFiles == 0 { + maxFiles = 16 + } + if f.value.typ.Kind() != reflect.Slice { + maxFiles = 1 + } + if seen[f.name] > maxFiles { + return nil, invalid(errors.New("too many files for multipart field")) + } + name := path.Base(strings.ReplaceAll(part.FileName(), "\\", "/")) + if name == "" || name == "." || len(name) > 255 || !utf8.ValidString(name) || strings.ContainsAny(name, "\x00\r\n") { + return nil, invalid(errors.New("invalid upload filename")) + } + temp, e := os.CreateTemp(o.TempDir, "omni-upload-*") + if e != nil { + return nil, apierrors.Internal(e) + } + file := File{Name: name, ContentType: part.Header.Get("Content-Type"), state: &uploadState{path: temp.Name(), readers: map[*uploadReader]bool{}}} + files = append(files, file) + size, e := io.Copy(temp, io.LimitReader(part, limit+1)) + e = errors.Join(e, temp.Close()) + if size > limit { + return nil, tooLarge() + } + if e != nil { + return nil, multipartFailure(e) + } + file.Size = size + files[len(files)-1] = file + uploaded[f.name] = append(uploaded[f.name], file) + } else { + if f.maxBytes > 0 && f.maxBytes < limit { + limit = f.maxBytes + } + if seen[f.name] > 1 && (f.source == "multipartJSON" || base(f.value).typ.Kind() != reflect.Slice) { + return nil, invalid(errors.New("duplicate multipart field")) + } + data, e := io.ReadAll(io.LimitReader(part, limit+1)) + if int64(len(data)) > limit { + return nil, tooLarge() + } + if e != nil { + return nil, multipartFailure(e) + } + if policy.UTF8 == "reject" && !utf8.Valid(data) { + return nil, invalid(errUTF8) + } + if policy.Charset == "utf8" && part.Header.Get("Content-Type") != "" { + _, p, e := mime.ParseMediaType(part.Header.Get("Content-Type")) + if e != nil || (p["charset"] != "" && !strings.EqualFold(p["charset"], "utf-8")) { + return nil, invalid(errors.New("invalid multipart field charset")) + } + } + text[f.name] = append(text[f.name], strings.ToValidUTF8(string(data), "�")) + } + if err = part.Close(); err != nil { + return nil, multipartFailure(err) + } + } + // Consume epilogue as well; it counts against the total request cap. + if err = body.Complete(); err != nil { + return nil, multipartFailure(err) + } + for _, f := range b.root.fields { + if !multipartSource(f.source) { + continue + } + present := seen[f.name] > 0 + dest := fresh.Elem().Field(f.index) + if f.source == "file" { + if !present { + if !f.optional { + return nil, invalid(fmt.Errorf("%s: file required", f.name)) + } + continue + } + if f.value.typ.Kind() == reflect.Slice { + dest.Set(reflect.ValueOf(uploaded[f.name])) + } else { + dest.Set(reflect.ValueOf(uploaded[f.name][0])) + } + continue + } + var value any + if present { + if f.source == "multipartJSON" { + d := json.NewDecoder(strings.NewReader(text[f.name][0])) + d.UseNumber() + value, err = decodeValue(d, 0) + if err == nil { + _, err = d.Token() + if err == io.EOF { + err = nil + } else if err == nil { + err = errors.New("multiple JSON values") + } + } + } else { + value, err = fromStrings(f.value, text[f.name]) + } + if err != nil { + return nil, invalid(fmt.Errorf("%s: %w", f.name, err)) + } + } + if err = assignField(f, dest, value, present); err != nil { + return nil, invalid(fmt.Errorf("%s: %w", f.name, err)) + } + } + dst.Elem().Set(fresh.Elem()) + success = true + return release, nil +} +func multipartFailure(err error) error { + var max *http.MaxBytesError + if errors.As(err, &max) { + return tooLarge() + } + var api *apierrors.APIError + if errors.As(err, &api) { + return err + } + return invalid(err) +} +func fieldLimit(raw string) (int64, error) { + v, err := strconv.ParseInt(raw, 10, 64) + if err != nil || v < 1 || v > 1<<40 { + return 0, errors.New("maxBytes must be 1..1099511627776") + } + return v, nil +} diff --git a/contract/multipart_test.go b/contract/multipart_test.go new file mode 100644 index 0000000..1da7750 --- /dev/null +++ b/contract/multipart_test.go @@ -0,0 +1,152 @@ +package contract + +import ( + "bytes" + "errors" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "io" + "mime/multipart" + "net/http" + "net/http/httptest" + "os" + "strings" + "testing" +) + +type uploadInput struct { + Query string `form:"query,optional"` + Name string `multipart:"name" binding:"required,max=10"` + Meta struct { + Count int `json:"count"` + } `multipartJSON:"meta"` + Attachment File `file:"attachment,maxBytes=12"` + Extra []File `file:"extra,optional,maxBytes=8,maxFiles=2"` +} +type uploadPart struct{ name, filename, value string } + +func uploadRequest(t *testing.T, parts []uploadPart) (*http.Request, []byte) { + t.Helper() + var data bytes.Buffer + w := multipart.NewWriter(&data) + for _, part := range parts { + if part.filename != "" { + p, e := w.CreateFormFile(part.name, part.filename) + if e != nil { + t.Fatal(e) + } + _, _ = io.WriteString(p, part.value) + } else { + _ = w.WriteField(part.name, part.value) + } + } + _ = w.Close() + r := httptest.NewRequest("POST", "/upload?query=yes", bytes.NewReader(data.Bytes())) + r.Header.Set("Content-Type", w.FormDataContentType()) + return r, data.Bytes() +} +func goodParts() []uploadPart { + return []uploadPart{{"name", "", "合同"}, {"meta", "", `{"count":2}`}, {"attachment", "../contract.txt", "hello"}} +} +func assertUploadEmpty(t *testing.T, dir string) { + t.Helper() + files, err := os.ReadDir(dir) + if err != nil || len(files) != 0 { + t.Fatalf("temp files remain: %v %v", files, err) + } +} +func TestMultipartBindingAndLifetime(t *testing.T) { + b := MustNew[uploadInput]() + dir := t.TempDir() + r, _ := uploadRequest(t, append(goodParts(), uploadPart{"extra", "one", "first"}, uploadPart{"extra", "two", "second"})) + var input uploadInput + release, err := b.BindMultipart(r, nil, &input, MultipartOptions{TempDir: dir, Policy: RequestPolicy{UnknownQuery: "reject", UTF8: "reject"}}) + if err != nil { + t.Fatal(err) + } + if input.Name != "合同" || input.Query != "yes" || input.Meta.Count != 2 || input.Attachment.Size != 5 || input.Attachment.Name != "contract.txt" || len(input.Extra) != 2 { + t.Fatalf("%+v", input) + } + reader, err := input.Attachment.Open() + if err != nil { + t.Fatal(err) + } + data, err := io.ReadAll(reader) + if err != nil || string(data) != "hello" { + t.Fatal(string(data), err) + } + // An unclosed reader is owned by the upload cleanup as well. + if err = release(); err != nil { + t.Fatal(err) + } + if _, err = reader.Read(make([]byte, 1)); err == nil { + t.Fatal("reader remains open") + } + if _, err = input.Attachment.Open(); err == nil { + t.Fatal("expired upload reopened") + } + if err = release(); err != nil { + t.Fatal(err) + } + assertUploadEmpty(t, dir) + if err = b.Bind(r, nil, &input); err == nil { + t.Fatal("ordinary binding accepted multipart fields") + } +} +func TestMultipartFailuresAreTransactional(t *testing.T) { + b := MustNew[uploadInput]() + for _, tt := range []struct { + name string + parts []uploadPart + options MultipartOptions + status int + }{ + {"missing", goodParts()[1:], MultipartOptions{}, 400}, + {"unknown", append(goodParts(), uploadPart{"other", "", "x"}), MultipartOptions{}, 400}, + {"duplicate", append(goodParts(), uploadPart{"name", "", "x"}), MultipartOptions{}, 400}, + {"jsonDuplicate", []uploadPart{{"name", "", "n"}, {"attachment", "f", "x"}, {"meta", "", `{"count":1,"count":2}`}}, MultipartOptions{}, 400}, + {"badType", []uploadPart{{"name", "", "n"}, {"attachment", "f", "x"}, {"meta", "", `{"count":"x"}`}}, MultipartOptions{}, 400}, + {"badUTF8", append(goodParts()[:1], uploadPart{"meta", "", "{\"count\":\"\xff\"}"}, goodParts()[2]), MultipartOptions{Policy: RequestPolicy{UTF8: "reject"}}, 400}, + {"fileSize", []uploadPart{{"attachment", "x", strings.Repeat("x", 13)}}, MultipartOptions{}, 413}, + {"fieldSize", goodParts(), MultipartOptions{MaxFieldBytes: 2}, 413}, + {"partCount", goodParts(), MultipartOptions{MaxParts: 2}, 413}, + {"total", goodParts(), MultipartOptions{MaxBodyBytes: 32}, 413}, + {"fileKind", []uploadPart{{"attachment", "", "text"}}, MultipartOptions{}, 400}, + {"extraCount", append(goodParts(), uploadPart{"extra", "a", "a"}, uploadPart{"extra", "b", "b"}, uploadPart{"extra", "c", "c"}), MultipartOptions{}, 400}, + } { + t.Run(tt.name, func(t *testing.T) { + dir := t.TempDir() + tt.options.TempDir = dir + r, _ := uploadRequest(t, tt.parts) + r.ContentLength = -1 + input := uploadInput{Name: "untouched"} + release, err := b.BindMultipart(r, nil, &input, tt.options) + var api *apierrors.APIError + if !errors.As(err, &api) || api.Status != tt.status { + t.Fatalf("error %v", err) + } + if release != nil || input.Name != "untouched" { + t.Fatal("partial binding published") + } + assertUploadEmpty(t, dir) + }) + } +} +func TestMultipartTruncationAndEpilogueLimit(t *testing.T) { + b := MustNew[uploadInput]() + for _, epilogue := range []bool{false, true} { + dir := t.TempDir() + r, original := uploadRequest(t, goodParts()) + data := original[:len(original)-8] + limit := int64(len(original) + 16) + if epilogue { + data = append(original, bytes.Repeat([]byte("x"), 64)...) + } + r.Body = io.NopCloser(bytes.NewReader(data)) + r.ContentLength = -1 + var input uploadInput + if _, err := b.BindMultipart(r, nil, &input, MultipartOptions{TempDir: dir, MaxBodyBytes: limit}); err == nil { + t.Fatal("truncation or epilogue bypass accepted") + } + assertUploadEmpty(t, dir) + } +} diff --git a/contract/policy.go b/contract/policy.go new file mode 100644 index 0000000..a9ee8e0 --- /dev/null +++ b/contract/policy.go @@ -0,0 +1,105 @@ +package contract + +import ( + "context" + "errors" + "fmt" + "io" + "unicode/utf8" +) + +// RequestPolicy describes protocol strictness, independently of business +// fields. Empty values inherit App defaults, then the historical binder policy. +type RequestPolicy struct { + UnknownQuery string `json:"unknownQuery"` // ignore or reject + UTF8 string `json:"utf8"` // replace or reject + Charset string `json:"charset"` // any or utf8 + Body string `json:"body"` // auto, required, optional, none, emptyObject +} + +func (p RequestPolicy) Validate() error { + for _, item := range []struct { + name, value string + allowed []string + }{ + {"unknownQuery", p.UnknownQuery, []string{"ignore", "reject"}}, + {"utf8", p.UTF8, []string{"replace", "reject"}}, + {"charset", p.Charset, []string{"any", "utf8"}}, + {"body", p.Body, []string{"auto", "required", "optional", "none", "emptyObject"}}, + } { + if item.value == "" { + continue + } + ok := false + for _, value := range item.allowed { + ok = ok || value == item.value + } + if !ok { + return fmt.Errorf("request policy: invalid %s", item.name) + } + } + return nil +} +func MergePolicy(base, override RequestPolicy) RequestPolicy { + if override.UnknownQuery != "" { + base.UnknownQuery = override.UnknownQuery + } + if override.UTF8 != "" { + base.UTF8 = override.UTF8 + } + if override.Charset != "" { + base.Charset = override.Charset + } + if override.Body != "" { + base.Body = override.Body + } + return base +} + +type policyKey struct{} + +func WithRequestPolicy(ctx context.Context, p RequestPolicy) context.Context { + if ctx == nil { + ctx = context.Background() + } + return context.WithValue(ctx, policyKey{}, p) +} +func RequestPolicyFromContext(ctx context.Context) RequestPolicy { + if ctx == nil { + return RequestPolicy{} + } + p, _ := ctx.Value(policyKey{}).(RequestPolicy) + return p +} + +// utf8Reader checks original bytes in one pass, including runes crossing Read +// boundaries. It never replaces invalid bytes or buffers the complete body. +type utf8Reader struct { + r io.Reader + pending [4]byte + n int +} + +var errUTF8 = errors.New("invalid UTF-8 in JSON body") + +func (r *utf8Reader) Read(p []byte) (int, error) { + n, err := r.r.Read(p) + for _, b := range p[:n] { + if r.n == 0 && b < utf8.RuneSelf { + continue + } + r.pending[r.n] = b + r.n++ + if utf8.FullRune(r.pending[:r.n]) { + ch, size := utf8.DecodeRune(r.pending[:r.n]) + if ch == utf8.RuneError && size == 1 { + return n, errUTF8 + } + r.n = 0 + } + } + if err == io.EOF && r.n != 0 { + return n, errUTF8 + } + return n, err +} diff --git a/contract/policy_test.go b/contract/policy_test.go new file mode 100644 index 0000000..92aae29 --- /dev/null +++ b/contract/policy_test.go @@ -0,0 +1,104 @@ +package contract + +import ( + "bytes" + "context" + "errors" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "io" + "net/http" + "net/http/httptest" + "strings" + "testing" +) + +type byteReader struct{ r io.Reader } + +func (r byteReader) Read(p []byte) (int, error) { + if len(p) > 1 { + p = p[:1] + } + return r.r.Read(p) +} + +func TestBodyPolicyPropagatesOuterLimit(t *testing.T) { + for _, limit := range []int64{0, 1, 4} { + r := httptest.NewRequest("POST", "/", byteReader{strings.NewReader("{} ")}) + r.Header.Set("Content-Type", "application/json") + r.Body = http.MaxBytesReader(httptest.NewRecorder(), r.Body, limit) + err := MustNew[struct{}]().BindWithPolicy(r, nil, &struct{}{}, RequestPolicy{Body: "required", UTF8: "reject"}) + var e *apierrors.APIError + if !errors.As(err, &e) || e.Status != 413 { + t.Fatalf("limit %d: expected 413, got %v", limit, err) + } + } +} + +func TestStrictRequestPolicy(t *testing.T) { + type request struct { + Name string `json:"name"` + Tags []string `form:"tag,optional"` + } + b := MustNew[request]() + strict := RequestPolicy{UnknownQuery: "reject", UTF8: "reject", Charset: "utf8"} + for _, tc := range []struct { + name, url, ct, body string + valid bool + }{ + {"valid", "/?tag=a&tag=b", "application/json; charset=UTF-8", `{"name":"企业😀"}`, true}, + {"query", "/?typo=1", "application/json", `{"name":"ok"}`, false}, + {"charset", "/", "application/json; charset=gbk", `{"name":"ok"}`, false}, + {"utf8", "/", "application/json", "{\"name\":\"\xff\"}", false}, + {"truncatedRune", "/", "application/json", "{\"name\":\"\xe4\"}", false}, + {"tail", "/", "application/json", `{"name":"ok"} {}`, false}, + {"tailInvalidByte", "/", "application/json", "{\"name\":\"ok\"} \xff", false}, + } { + t.Run(tc.name, func(t *testing.T) { + r := httptest.NewRequest("POST", tc.url, byteReader{strings.NewReader(tc.body)}) + r.Header.Set("Content-Type", tc.ct) + var dst request + err := b.BindWithPolicy(r, nil, &dst, strict) + if (err == nil) != tc.valid { + t.Fatal(err) + } + if tc.valid && len(dst.Tags) != 2 { + t.Fatal("query slice rejected") + } + }) + } + // Explicit route overrides inherit the rest of the App policy. + r := httptest.NewRequest("POST", "/?typo=1", strings.NewReader(`{"name":"ok"}`)) + r.Header.Set("Content-Type", "application/json") + r = r.WithContext(WithRequestPolicy(context.Background(), strict)) + if err := b.BindWithPolicy(r, nil, &request{}, RequestPolicy{UnknownQuery: "ignore"}); err != nil { + t.Fatal(err) + } +} + +func TestBodyPresencePolicies(t *testing.T) { + b := MustNew[struct{}]() + for _, mode := range []string{"none", "emptyObject", "required", "optional"} { + for _, body := range []string{"", "{}", "null", `{"unexpected":1}`, "{} {}", " "} { + r := httptest.NewRequest("POST", "/", byteReader{strings.NewReader(body)}) + r.Header.Set("Content-Type", "application/json") + err := b.BindWithPolicy(r, nil, &struct{}{}, RequestPolicy{Body: mode, UTF8: "reject"}) + valid := (mode == "none" && body == "") || (mode != "none" && body == "{}") || (mode == "optional" && body == "") + if (err == nil) != valid { + t.Fatalf("mode=%s body=%q: %v", mode, body, err) + } + } + } + type object struct { + Value string `json:"value"` + } + r := httptest.NewRequest("POST", "/", bytes.NewReader(nil)) + if err := MustNew[object]().BindWithPolicy(r, nil, &object{}, RequestPolicy{Body: "optional"}); err != nil { + t.Fatal(err) + } + // Chunked/unknown size is still bounded under strict UTF-8 checking. + r = httptest.NewRequest("POST", "/", byteReader{strings.NewReader(`{"value":"1234567890"}`)}) + r.Header.Set("Content-Type", "application/json") + if err := MustNew[object]().BindWithPolicy(r, nil, &object{}, RequestPolicy{UTF8: "reject"}, 8); err == nil { + t.Fatal("chunked over-limit accepted") + } +} diff --git a/contract/scalar.go b/contract/scalar.go index 13d47a5..7eb619e 100644 --- a/contract/scalar.go +++ b/contract/scalar.go @@ -147,6 +147,10 @@ func responseSchema(n *node) *openapi.Schema { b := base(n) if b.scalar != nil { s := wireSchema(b.scalar.JSONOutput) + s.Description = n.schema.Description + if b.scalar.JSONInput == b.scalar.JSONOutput { + s.Example = n.schema.Example + } if n.typ.Kind() == reflect.Pointer { s = nullableSchema(s) } diff --git a/database/README.md b/database/README.md index a24aba4..86c6980 100644 --- a/database/README.md +++ b/database/README.md @@ -1,5 +1,7 @@ # database +结构化 MySQL 配置、独立 TLS/mTLS、驱动日志和生成项目接入见 [官方 MySQL 适配器](mysql/README.md)。已有 Open/OpenConnector 接口继续支持。 + `database` 包提供基于标准库 `database/sql` 的连接池生命周期，不绑定 ORM 或驱动： - `Config` / `ConfigFromEnv`：连接池参数和 `GO_OMNI_DB_*` 环境变量。 diff --git a/database/mysql/README.md b/database/mysql/README.md new file mode 100644 index 0000000..6b45af0 --- /dev/null +++ b/database/mysql/README.md @@ -0,0 +1,69 @@ +# 官方 MySQL 适配器 + +本包可选，使用 `database/sql` 和 `go-sql-driver/mysql`，复用框架的唯一连接池、Ping、事务与健康检查。不自动建库/迁移、不重试写操作，也不强制某个 MySQL 版本或业务表。 + +生成项目选择 `--database mysql` 即接入： + +```sh +go-omni new --database mysql --module example.com/billing ./billing +``` + +`etc/config.yaml` 使用结构化配置，环境覆盖优先： + +```yaml +database: + host: db.example.com + port: 3306 + user: app + database: billing + parseTime: true + location: UTC + charset: utf8mb4 + collation: utf8mb4_bin + connectTimeout: 5s + readTimeout: 5s + writeTimeout: 5s + tls: + mode: required + caFile: /etc/certs/database-ca.pem + serverName: db.example.com + pool: + maxOpenConns: 20 + maxIdleConns: 5 + connMaxLifetime: 30m + connMaxIdleTime: 5m + pingTimeout: 5s +``` + +通过 `GO_OMNI_DB_PASSWORD` 传密码；还有 HOST、PORT、USER、DATABASE、TLS_CA_FILE、TLS_CERT_FILE、TLS_KEY_FILE、POOL_MAX_OPEN_CONNS 等。业务也可自定义 env 标签。新生成服务不再读取 DSN；独立数据库迁移 CLI 和手写 `database.Open/OpenConnector` 的原接口保留。 + +默认启用 TLS 并校验系统信任链与主机名；私有 CA 用 caFile，双向 TLS 额外设置 certFile/keyFile。只有明确设置 `tls.mode: disabled` 才使用明文。Unix socket 用 socket，若在 socket 上使用 TLS，必须提供 serverName。不会注册全局 TLS 配置或全局驱动 Logger。 + +手写入口： + +```go +cfg := mysqladapter.DefaultConfig() +// 从项目配置覆盖 cfg；不要打印含密码的完整配置。 +db, err := mysqladapter.Open(ctx, cfg, mysqladapter.WithLogger(logger)) +if err != nil { return err } +defer db.Close() +// db.DB 是标准 *sql.DB；也可用框架 Health、InTx 等方法。 +``` + +生成项目的 `internal/svc/databaseOptions.go` 是保留的项目扩展位置： + +```go +func (c *ServiceContext) DatabaseOptions() []mysqladapter.Option { + return []mysqladapter.Option{ + mysqladapter.WithStartupCheck(func(ctx context.Context, db *database.DB) error { + return checkProjectSchema(ctx, db) + }), + } +} +``` + +检查在 Ping 后运行，共享启动时限；失败或 panic 会关闭新连接池。版本/表结构检查放在这些回调中，不放到框架。回调必须响应 context，并且不能自行关闭借用的 db。依赖尚未完成整体初始化时不要把连接池发布到外部。 + +`WithDriverConfig(func(*mysql.Config) error)` 支持高级拨号、驱动参数等，配置在回调后克隆；这是项目明确选择的底层扩展。`NewConnector` 返回标准 driver.Connector，供已有基础设施使用；它不接受启动检查选项。任意驱动仍可用核心 `database.OpenConnector`。 + +错误有安全摘要，errors.Is/As 保留 TLS/驱动/项目检查的原因。配置验证不读证书、不连接数据库；Open 才读取证书、创建池并握手。不要自行打印原始配置或展开可能包含内部信息的 driver 字段给用户。 diff --git a/database/mysql/integration_test.go b/database/mysql/integration_test.go new file mode 100644 index 0000000..1dbcf34 --- /dev/null +++ b/database/mysql/integration_test.go @@ -0,0 +1,115 @@ +package mysql_test + +import ( + "context" + "crypto/tls" + "errors" + "git.yuntuo.hk/k2safe/go-omni/database" + adapter "git.yuntuo.hk/k2safe/go-omni/database/mysql" + "net" + "os" + "strconv" + "testing" + "time" +) + +func TestStructuredMySQLAdapterTLS(t *testing.T) { + if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" { + t.Skip("use scripts/verifyConnector.py with an isolated local MySQL") + } + ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second) + defer cancel() + cfg := adapter.DefaultConfig() + host, port, err := net.SplitHostPort(os.Getenv("GO_OMNI_CONNECTOR_ADDR")) + if err != nil { + t.Fatal(err) + } + cfg.Host = host + cfg.Port, _ = strconv.Atoi(port) + cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER") + cfg.Password = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD") + cfg.Database = os.Getenv("GO_OMNI_CONNECTOR_DATABASE") + cfg.TLS.CAFile = os.Getenv("GO_OMNI_CONNECTOR_CA_FILE") + cfg.TLS.ServerName = "localhost" + cfg.Pool.MaxOpenConns = 2 + cfg.Pool.MaxIdleConns = 2 + cfg.Pool.PingTimeout = 2 * time.Second + cfg.Session = map[string]string{"time_zone": "'+00:00'"} + checked := false + first, err := adapter.Open(ctx, cfg, adapter.WithStartupCheck(func(ctx context.Context, db *database.DB) error { + var one int + err := db.QueryRowContext(ctx, "SELECT 1").Scan(&one) + checked = one == 1 + return err + })) + if err != nil { + t.Fatal(err) + } + defer first.Close() + if !checked { + t.Fatal("startup check not called") + } + cfg.Session = map[string]string{"time_zone": "'+08:00'"} + second, err := adapter.Open(ctx, cfg) + if err != nil { + t.Fatal(err) + } + defer second.Close() + for i, db := range []*database.DB{first, second} { + var key, cipher, zone string + if err = db.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&key, &cipher); err != nil || cipher == "" { + t.Fatal("not encrypted", err) + } + if err = db.QueryRowContext(ctx, "SELECT @@session.time_zone").Scan(&zone); err != nil || zone != []string{"+00:00", "+08:00"}[i] { + t.Fatal("session isolation failed", zone, err) + } + if db.Stats().MaxOpenConnections != 2 { + t.Fatal("pool limit ignored") + } + } + for _, bad := range []string{"untrusted", "hostname"} { + invalid := cfg + if bad == "untrusted" { + invalid.TLS.CAFile = "" + } else { + invalid.TLS.ServerName = "wrong.invalid" + } + db, e := adapter.Open(ctx, invalid) + if db != nil { + _ = db.Close() + t.Fatal("bad certificate accepted") + } + var verify *tls.CertificateVerificationError + if !errors.As(e, &verify) { + t.Fatal("lost TLS verification cause", e) + } + } + cause := errors.New("project schema check failed") + if db, e := adapter.Open(ctx, cfg, adapter.WithStartupCheck(func(context.Context, *database.DB) error { return cause })); db != nil || !errors.Is(e, cause) { + t.Fatal("failed startup escaped", e) + } + if err = first.Health(ctx); err != nil { + t.Fatal("failed pool affected healthy pool", err) + } + // The server requires a client certificate for this separate fixture user. + cfg.User = "omni_mtls" + cfg.Password = "" + cfg.Database = "" + cfg.Session = nil + if db, e := adapter.Open(ctx, cfg); e == nil { + db.Close() + t.Fatal("missing required client certificate accepted") + } + cfg.TLS.CertFile = os.Getenv("GO_OMNI_CONNECTOR_CLIENT_CERT") + cfg.TLS.KeyFile = os.Getenv("GO_OMNI_CONNECTOR_CLIENT_KEY") + mtls, err := adapter.Open(ctx, cfg) + if err != nil { + t.Fatal("mTLS failed", err) + } + if err = mtls.Close(); err != nil { + t.Fatal(err) + } + if err = mtls.Close(); err != nil { + t.Fatal("pool close is not idempotent", err) + } +} diff --git a/database/mysql/mysql.go b/database/mysql/mysql.go new file mode 100644 index 0000000..17a7cf2 --- /dev/null +++ b/database/mysql/mysql.go @@ -0,0 +1,322 @@ +// Package mysql provides an optional MySQL adapter over database.OpenConnector. +// Each pool owns independent driver settings; no global driver registry changes +// are made. It never migrates tables or retries application transactions. +package mysql + +import ( + "context" + "crypto/tls" + "crypto/x509" + "database/sql/driver" + "errors" + "fmt" + "io" + "net" + "os" + "regexp" + "strconv" + "strings" + "time" + + "git.yuntuo.hk/k2safe/go-omni/database" + "git.yuntuo.hk/k2safe/go-omni/logx" + mysqldriver "github.com/go-sql-driver/mysql" +) + +type Config struct { + Host string `json:"host"` + Port int `json:"port"` + Socket string `json:"socket"` + User string `json:"user"` + Password string `json:"password" env:",secret"` + Database string `json:"database"` + ParseTime bool `json:"parseTime"` + Location string `json:"location"` + Charset string `json:"charset"` + Collation string `json:"collation"` + // Session values are SQL expressions, as required by the driver, for + // example time_zone: "'+00:00'". They are trusted deployment configuration. + Session map[string]string `json:"session"` + ConnectTimeout time.Duration `json:"connectTimeout"` + ReadTimeout time.Duration `json:"readTimeout"` + WriteTimeout time.Duration `json:"writeTimeout"` + MultiStatements bool `json:"multiStatements"` + TLS TLSConfig `json:"tls"` + Pool database.PoolConfig `json:"pool"` +} +type TLSConfig struct { + Mode string `json:"mode"` // required (default) or disabled + CAFile string `json:"caFile"` + CertFile string `json:"certFile"` + KeyFile string `json:"keyFile"` + ServerName string `json:"serverName"` +} + +func DefaultConfig() Config { + return Config{Host: "127.0.0.1", Port: 3306, ParseTime: true, Location: "UTC", Charset: "utf8mb4", ConnectTimeout: 5 * time.Second, ReadTimeout: 5 * time.Second, WriteTimeout: 5 * time.Second, TLS: TLSConfig{Mode: "required"}, Pool: database.DefaultPoolConfig()} +} + +var identifier = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`) + +// Validate does not read certificate files or open connections. All defaults +// come from DefaultConfig; explicit zero I/O timeouts mean no driver deadline. +func (c Config) Validate() error { + if c.Socket == "" && (c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") || c.Port < 1 || c.Port > 65535) { + return errors.New("mysql: invalid host or port") + } + if strings.ContainsRune(c.Socket, 0) || strings.ContainsRune(c.Database, 0) || strings.ContainsRune(c.User, 0) { + return errors.New("mysql: invalid connection field") + } + if c.ConnectTimeout < 0 || c.ReadTimeout < 0 || c.WriteTimeout < 0 { + return errors.New("mysql: timeouts cannot be negative") + } + if err := c.Pool.Validate(); err != nil { + return err + } + if c.Location != "" { + if _, err := time.LoadLocation(c.Location); err != nil { + return errors.New("mysql: invalid time location") + } + } + if c.Charset != "" && !identifier.MatchString(c.Charset) { + return errors.New("mysql: invalid charset") + } + if c.Collation != "" && !identifier.MatchString(c.Collation) { + return errors.New("mysql: invalid collation") + } + for key := range c.Session { + if !identifier.MatchString(key) { + return errors.New("mysql: invalid session variable name") + } + } + if c.TLS.Mode != "" && c.TLS.Mode != "required" && c.TLS.Mode != "disabled" { + return errors.New("mysql: TLS mode must be required or disabled") + } + if (c.TLS.CertFile == "") != (c.TLS.KeyFile == "") { + return errors.New("mysql: TLS client certificate and key must be configured together") + } + if c.TLS.Mode == "disabled" && (c.TLS.CAFile != "" || c.TLS.CertFile != "" || c.TLS.ServerName != "") { + return errors.New("mysql: TLS fields cannot be used when disabled") + } + if c.Socket != "" && c.TLS.Mode != "disabled" && c.TLS.ServerName == "" { + return errors.New("mysql: TLS over a unix socket requires serverName") + } + return nil +} + +// Error has a safe summary; errors.Is/As retain driver/verification causes. +// Do not log raw configuration or a driver's manually extracted error fields. +type Error struct { + Operation string + cause error +} + +func (e *Error) Error() string { return "mysql: " + e.Operation + " failed" } +func (e *Error) Unwrap() error { return e.cause } + +type Check func(context.Context, *database.DB) error +type options struct { + logger *logx.Logger + checks []Check + configure func(*mysqldriver.Config) error +} +type Option func(*options) + +func WithLogger(l *logx.Logger) Option { return func(o *options) { o.logger = l } } + +// WithStartupCheck runs after Ping, before ownership reaches the caller. A +// failure closes the pool. Checks must honor ctx and must not close the pool. +func WithStartupCheck(check Check) Option { + return func(o *options) { + if check != nil { + o.checks = append(o.checks, check) + } + } +} + +// WithDriverConfig provides an advanced, per-pool escape hatch. Configuration +// is cloned after this callback; keep credentials out of callback errors/logs. +func WithDriverConfig(fn func(*mysqldriver.Config) error) Option { + return func(o *options) { o.configure = fn } +} + +func settings(opts []Option) options { + var o options + for _, opt := range opts { + if opt != nil { + opt(&o) + } + } + return o +} + +// Open returns the existing Omni pool; .DB is the same standard sql.DB for +// repositories or ORMs. Ping and startup checks share one bounded deadline. +func Open(ctx context.Context, cfg Config, opts ...Option) (db *database.DB, err error) { + if ctx == nil { + ctx = context.Background() + } + if err = ctx.Err(); err != nil { + return nil, err + } + o := settings(opts) + if o.logger == nil { + o.logger = logx.FromContext(ctx) + } + connector, err := connector(cfg, o) + if err != nil { + return nil, err + } + return openPool(ctx, connector, cfg.Pool, o.checks) +} + +func openPool(ctx context.Context, connector driver.Connector, pool database.PoolConfig, checks []Check) (db *database.DB, err error) { + checkCtx, cancel := context.WithTimeout(ctx, pool.PingTimeout) + defer cancel() + db, err = database.OpenConnector(checkCtx, connector, pool) + if err != nil { + return nil, &Error{Operation: "connect", cause: err} + } + defer func() { + if p := recover(); p != nil { + _ = db.Close() + panic(p) + } + if err != nil { + err = errors.Join(err, db.Close()) + db = nil + } + }() + for _, check := range checks { + if err = check(checkCtx, db); err != nil { + return db, &Error{Operation: "startup check", cause: err} + } + if err = checkCtx.Err(); err != nil { + return db, &Error{Operation: "startup check", cause: err} + } + } + return db, nil +} + +// NewConnector builds a standard connector for callers that own pool creation. +// Startup checks are only executed by Open. +func NewConnector(cfg Config, opts ...Option) (driver.Connector, error) { + o := settings(opts) + if len(o.checks) > 0 { + return nil, errors.New("mysql: startup checks require Open, not NewConnector") + } + return connector(cfg, o) +} +func connector(cfg Config, o options) (driver.Connector, error) { + d, err := driverConfig(cfg, o) + if err != nil { + return nil, err + } + c, err := mysqldriver.NewConnector(d) + if err != nil { + return nil, &Error{Operation: "connector configuration", cause: err} + } + return c, nil +} + +func driverConfig(cfg Config, o options) (*mysqldriver.Config, error) { + if err := cfg.Validate(); err != nil { + return nil, err + } + d := mysqldriver.NewConfig() + d.User, d.Passwd, d.DBName = cfg.User, cfg.Password, cfg.Database + d.Net, d.Addr = "tcp", net.JoinHostPort(cfg.Host, strconv.Itoa(cfg.Port)) + if cfg.Socket != "" { + d.Net, d.Addr = "unix", cfg.Socket + } + d.ParseTime = cfg.ParseTime + if cfg.Location != "" { + d.Loc, _ = time.LoadLocation(cfg.Location) + } + if cfg.Charset != "" { + if err := d.Apply(mysqldriver.Charset(cfg.Charset, cfg.Collation)); err != nil { + return nil, &Error{Operation: "charset", cause: err} + } + } else { + d.Collation = cfg.Collation + } + d.Params = make(map[string]string, len(cfg.Session)) + for k, v := range cfg.Session { + d.Params[k] = v + } + d.Timeout, d.ReadTimeout, d.WriteTimeout = cfg.ConnectTimeout, cfg.ReadTimeout, cfg.WriteTimeout + d.MultiStatements = cfg.MultiStatements + logger := o.logger + if logger == nil { + logger = logx.FromContext(nil) + } + d.Logger = driverLog{logger: logger, secret: cfg.Password} + if cfg.TLS.Mode != "disabled" { + name := cfg.TLS.ServerName + if name == "" { + name = cfg.Host + } + t := &tls.Config{MinVersion: tls.VersionTLS12, ServerName: name} + if cfg.TLS.CAFile != "" { + pem, err := readCertificate(cfg.TLS.CAFile) + if err != nil { + return nil, &Error{Operation: "read CA", cause: err} + } + t.RootCAs = x509.NewCertPool() + if !t.RootCAs.AppendCertsFromPEM(pem) { + return nil, errors.New("mysql: invalid CA certificate") + } + } + if cfg.TLS.CertFile != "" { + cert, err := readCertificate(cfg.TLS.CertFile) + if err != nil { + return nil, &Error{Operation: "read client certificate", cause: err} + } + key, err := readCertificate(cfg.TLS.KeyFile) + if err != nil { + return nil, &Error{Operation: "read client key", cause: err} + } + pair, err := tls.X509KeyPair(cert, key) + if err != nil { + return nil, &Error{Operation: "client key pair", cause: err} + } + t.Certificates = []tls.Certificate{pair} + } + d.TLS = t + } + if o.configure != nil { + if err := o.configure(d); err != nil { + return nil, &Error{Operation: "custom configuration", cause: err} + } + } + return d.Clone(), nil +} + +func readCertificate(path string) ([]byte, error) { + f, err := os.Open(path) + if err != nil { + return nil, err + } + defer f.Close() + data, err := io.ReadAll(io.LimitReader(f, (4<<20)+1)) + if err != nil { + return nil, err + } + if len(data) > 4<<20 { + return nil, errors.New("certificate file exceeds 4 MiB") + } + return data, nil +} + +type driverLog struct { + logger *logx.Logger + secret string +} + +func (l driverLog) Print(values ...any) { + message := fmt.Sprint(values...) + if l.secret != "" { + message = strings.ReplaceAll(message, l.secret, "[REDACTED]") + } + l.logger.Error("database.driver", "driver", "mysql", "error", message) +} diff --git a/database/mysql/mysql_test.go b/database/mysql/mysql_test.go new file mode 100644 index 0000000..8b435a1 --- /dev/null +++ b/database/mysql/mysql_test.go @@ -0,0 +1,146 @@ +package mysql + +import ( + "bytes" + "context" + "crypto/tls" + "database/sql/driver" + "errors" + "net" + "os" + "path/filepath" + "strings" + "sync/atomic" + "testing" + "time" + + "git.yuntuo.hk/k2safe/go-omni/database" + "git.yuntuo.hk/k2safe/go-omni/logx" + mysqldriver "github.com/go-sql-driver/mysql" +) + +func TestConfigurationTLSLoggingAndIsolation(t *testing.T) { + cfg := DefaultConfig() + cfg.User = "test" + cfg.Password = "do-not-print" + cfg.Database = "mixed-name" + cfg.Session = map[string]string{"time_zone": "'+00:00'"} + cfg.Collation = "utf8mb4_bin" + var out bytes.Buffer + logger, _ := logx.New(logx.Config{Writer: &out}) + d, err := driverConfig(cfg, settings([]Option{WithLogger(logger)})) + if err != nil { + t.Fatal(err) + } + if d.TLS == nil || d.TLS.MinVersion != tls.VersionTLS12 || d.TLS.ServerName != cfg.Host || d.MultiStatements || d.Params["time_zone"] != "'+00:00'" { + t.Fatal("driver policy differs") + } + cfg.Session["time_zone"] = "'+08:00'" + if d.Params["time_zone"] != "'+00:00'" { + t.Fatal("session map alias") + } + d.Logger.Print("connection ", cfg.Password) + if strings.Contains(out.String(), cfg.Password) || !strings.Contains(out.String(), "[REDACTED]") { + t.Fatal("secret leaked") + } + cfg.TLS.Mode = "disabled" + d2, err := driverConfig(cfg, options{}) + if err != nil || d2.TLS != nil || d.TLS == nil { + t.Fatal("TLS configuration crossed instances", err) + } + cfg.TLS.CAFile = "unused" + if err = cfg.Validate(); err == nil { + t.Fatal("disabled TLS accepted CA") + } + cfg = DefaultConfig() + cfg.TLS.CAFile = filepath.Join(t.TempDir(), "missing") + if err = cfg.Validate(); err != nil { + t.Fatal("Validate reads certificates", err) + } + if _, err = NewConnector(cfg); err == nil { + t.Fatal("missing CA accepted") + } + if err = os.WriteFile(cfg.TLS.CAFile, []byte("do-not-print"), 0600); err != nil { + t.Fatal(err) + } + if _, err = NewConnector(cfg); err == nil || strings.Contains(err.Error(), "do-not-print") { + t.Fatal(err) + } + if _, err = NewConnector(DefaultConfig(), WithStartupCheck(func(context.Context, *database.DB) error { return nil })); err == nil { + t.Fatal("unused startup check accepted") + } +} + +func TestDriverHandshakeErrorUsesIndependentLogger(t *testing.T) { + var a, b bytes.Buffer + la, _ := logx.New(logx.Config{Writer: &a}) + lb, _ := logx.New(logx.Config{Writer: &b}) + cfg := DefaultConfig() + cfg.TLS.Mode = "disabled" + cfg.Pool.PingTimeout = time.Second + for i, l := range []*logx.Logger{la, lb} { + before := []int{a.Len(), b.Len()} + db, err := Open(context.Background(), cfg, WithLogger(l), WithDriverConfig(func(d *mysqldriver.Config) error { + d.DialFunc = func(context.Context, string, string) (net.Conn, error) { + reader, writer := net.Pipe() + _ = writer.Close() + return reader, nil + } + return nil + })) + if db != nil || err == nil { + t.Fatal("broken handshake succeeded") + } + sizes := []int{a.Len(), b.Len()} + if sizes[i] <= before[i] || sizes[1-i] != before[1-i] { + t.Fatal("driver log isolation failed") + } + } +} + +type testConnector struct{ closed atomic.Int32 } + +func (c *testConnector) Connect(context.Context) (driver.Conn, error) { return &testConn{c}, nil } +func (c *testConnector) Driver() driver.Driver { return testDriver{} } + +type testDriver struct{} + +func (testDriver) Open(string) (driver.Conn, error) { return nil, errors.New("unexpected DSN") } + +type testConn struct{ owner *testConnector } + +func (c *testConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("unused") } +func (c *testConn) Begin() (driver.Tx, error) { return nil, errors.New("unused") } +func (c *testConn) Ping(context.Context) error { return nil } +func (c *testConn) Close() error { c.owner.closed.Add(1); return nil } + +func TestStartupCheckRollbackAndPanic(t *testing.T) { + for _, panics := range []bool{false, true} { + c := &testConnector{} + cause := errors.New("schema failed") + func() { + if panics { + defer func() { + if recover() == nil { + t.Error("panic lost") + } + }() + } + db, err := openPool(context.Background(), c, database.DefaultPoolConfig(), []Check{func(ctx context.Context, db *database.DB) error { + if _, ok := ctx.Deadline(); !ok { + t.Error("unbounded check") + } + if panics { + panic("failed check") + } + return cause + }}) + if db != nil || !errors.Is(err, cause) { + t.Error("failed pool escaped or cause lost", err) + } + }() + if c.closed.Load() != 1 { + t.Fatal("failed initialization leaked pool", c.closed.Load()) + } + } +} diff --git a/errors/export.go b/errors/export.go new file mode 100644 index 0000000..7f342ad --- /dev/null +++ b/errors/export.go @@ -0,0 +1,23 @@ +package apierrors + +import ( + "encoding/json" + "errors" + "io" +) + +// ExportCatalog writes normalized, public-only error metadata. Invoke this +// explicitly from project tooling, using the same Catalog function registered +// at runtime. The API CLI reads the artifact; it never executes project code. +func ExportCatalog(w io.Writer, c Catalog) error { + if w == nil { + return errors.New("errors: catalog export writer required") + } + registry, err := NewRegistry(c) + if err != nil { + return err + } + encoder := json.NewEncoder(w) + encoder.SetIndent("", " ") + return encoder.Encode(registry.Catalog()) +} diff --git a/errors/export_test.go b/errors/export_test.go new file mode 100644 index 0000000..4955bd0 --- /dev/null +++ b/errors/export_test.go @@ -0,0 +1,40 @@ +package apierrors + +import ( + "bytes" + "errors" + "strings" + "testing" +) + +type failedExport struct{ err error } + +func (w failedExport) Write([]byte) (int, error) { return 0, w.err } +func TestCatalogExport(t *testing.T) { + catalog := Catalog{DefaultStatus: 200, Definitions: map[string]Definition{"INVALID": {Code: int64(9007199254740993), Message: "invalid"}}} + var out bytes.Buffer + if err := ExportCatalog(&out, catalog); err != nil { + t.Fatal(err) + } + if !strings.Contains(out.String(), "9007199254740993") { + t.Fatal(out.String()) + } + var decoded Catalog + if err := DecodeCatalogJSON(out.Bytes(), &decoded); err != nil { + t.Fatal(err) + } + registry, err := NewRegistry(decoded) + if err != nil { + t.Fatal(err) + } + if registry.Resolve(ByCode("INVALID")).Status != 200 { + t.Fatal("status lost") + } + cause := errors.New("write failed") + if err = ExportCatalog(failedExport{cause}, catalog); !errors.Is(err, cause) { + t.Fatal("writer failure lost", err) + } + if err = ExportCatalog(nil, catalog); err == nil { + t.Fatal("nil writer accepted") + } +} diff --git a/examples/contracts/business.api b/examples/contracts/business.api new file mode 100644 index 0000000..893315c --- /dev/null +++ b/examples/contracts/business.api @@ -0,0 +1,61 @@ +syntax = "v1" +@app ( + defaultResponse: CommonResponse + request: {"unknownQuery":"reject","utf8":"reject","charset":"utf8"} +) +response CommonResponse ( + codeType: int32 + successCode: 0 + errorCode: 9999 + messageField: msg + dataMode: always + requestIdField: "-" +) +type Item { + Name string `json:"name"` +} +type Query { + Mode string `form:"mode,optional"` + Tags []string `form:"tags,optional"` +} +type Upload { + Title string `multipart:"title" binding:"required,max=20"` + Metadata Item `multipartJSON:"metadata"` + Attachment File `file:"attachment,maxBytes=1024"` + Extras []File `file:"extras,optional,maxFiles=2,maxBytes=512"` +} +@server (group: demo; middleware: RequireAccess) +service business { + @handler upload + @body (mode: multipart; maxBytes: 4096; maxParts: 6; maxFieldBytes: 128) + post /upload (Upload) returns (Item) + + @handler list + get /list (Query) returns ([]Item) + @handler nullable + get /nullable (Query) returns (*Item) + @handler nothing + post /nothing returns (null) + @handler rawNull + @response raw + get /rawNull returns (null) + @handler rawArray + @response raw + get /rawArray returns ([]Item) + @handler empty + @request (body: emptyObject) + post /empty returns (null) + @handler absent + @request (body: none) + post /absent returns (null) + @handler optional + @request (body: optional) + post /optional (Item) returns (Item) + @handler echo + post /echo (Item) returns (Item) + @handler tolerant + @request (unknownQuery: ignore; utf8: replace; charset: any) + post /tolerant (Item) returns (Item) + @handler noContent + delete /noContent +} diff --git a/examples/managedService/.goOmni/apiManifest.json b/examples/managedService/.goOmni/apiManifest.json index ecc30e9..f0715df 100644 --- a/examples/managedService/.goOmni/apiManifest.json +++ b/examples/managedService/.goOmni/apiManifest.json @@ -1,4 +1,5 @@ { + "templateVersion": "v0.14.0-preview", "profile": { "database": "none", "storage": "none", @@ -9,15 +10,17 @@ "module": "example.com/managed", "entry": "api/managed.api", "generated": { - "docs/openapi.json": "9eb707aefeefd3f4881c774534e2d143949c0edb1d8a4aabd762f978f6203df1", - "internal/handler/appGen.go": "c6f078932e1a598a46faebba397f4e753db9d8f4f29a02f9b6077a0557d6c14e", - "internal/handler/demo/handlersGen.go": "7a6f9d5e105e44b07196cc540cee3f0562d5f3d1c5f0bc094434db1064d0339c", - "internal/handler/openapiGen.go": "612920ed25e5dc7a4f673be56041d47b70aa3216707ff282e52dfe503c2d705a", + "docs/openapi.json": "68c7e9e0294bf62937aa953a44732288c51571108c922a36a72faa72c4360b5f", + "internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d", + "internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375", + "internal/handler/openapiGen.go": "ae9b794c63e0f3c8f9d200a5fc6977350139d4a6f00b42a26102c8630d907d03", "internal/handler/responseGen.go": "84397a502bba4777ebfa6fae3d3bc418297acab4f26b04b42e3c31b36c735399", "internal/handler/routesGen.go": "76c1a5b2644e75e56553dc6e9e862a63128d902dd94807ce7c8aee5491cd1e3e", "internal/middleware/registryGen.go": "e5f47efa57a1432765ef3419bae59dbf3b4d55b3aac0a6c935ce7a52ba6b47f4", - "internal/types/typesGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344", - "main.go": "a12db5b7df0f2fdca4939a7382daee04c85ddb63ef0bccd1a2ce26b718f42579" + "internal/svc/runtimeGen.go": "78ffd0b2a196fbc29e29473919d1341e8a2f508f6594ad148c0b399fbcc40018", + "internal/types/dtoManagedd28645a2af1cGen.go": "7586e86fa5d67051283178cc578c890f6c02fd2d5c383170a71a39c2e77ce344", + "internal/types/typesGen.go": "372d67f23a313b3e77d2a3b7a236315db9721bc63350bf1f71db1293e57570d1", + "main.go": "8a4051c0d51fc14ddf22c36d09de706f02499817accc937f0785e3d0fb5e2aa5" }, "contract": { "routes": { diff --git a/examples/managedService/README.md b/examples/managedService/README.md index 0f90c59..fd3bbc4 100644 --- a/examples/managedService/README.md +++ b/examples/managedService/README.md @@ -34,149 +34,18 @@ to fail on unresolved signature errors after generation. Run `go test ./...` too especially for project type aliases, build tags and changes inside method bodies. -This project is generated from an editable API contract. Business code uses -standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx. -main.go and HTTP composition are continuously generator-owned. +## Framework-managed runtime -## Run +`main.go`, `internal/svc/runtimeGen.go`, generated handlers, `internal/types/*Gen.go` +and OpenAPI are generator-owned. Add project dependencies in `svc.Init(ctx)` and +register each successful initialization with `OnClose`. Runtime owns reverse +cleanup and failure rollback; do not copy locks, constructors or Close methods +into this project. The logger is scoped to this App. -```sh -go mod tidy -go run . -``` - -GET /health/live and /health/ready check the running service. GET /openapi.json -serves the same document as docs/openapi.json. Bind address defaults to localhost. -Set GO_OMNI_TRANSPORT=native to select the native transport without changing services. -The greeting starter works immediately; other new operation stubs return 501 until -their business logic is implemented. Authentication hooks also fail closed with 501. - -## Develop - -1. Edit api/managed.api and its relative imports. -2. Run the same go-omni CLI version as the framework dependency: - -```sh -go-omni api validate -go-omni api inspect -go-omni api generate --dry-run -go-omni api generate -go-omni api generate --check -go-omni api export > openapi.json -go test ./... -``` - -Commands inside a generated project discover the recorded API entry automatically. ---dry-run previews create/update/remove actions without writing files. -The --check command is read-only and suitable for CI. It fails on stale or missing -generated files and missing business scaffolds. go.mod, configuration, Logic, -Repository, svc, middleware hook bodies and API files are created once and then -preserved byte for byte. main.go, Gen.go files and OpenAPI are owned -by the generator; manual edits cause a conflict instead of being overwritten. -Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it. -Avoid editing files while generation is running; api.lock serializes generators. - -Routes and types that disappear are removed from generated output. Handwritten -business files remain: remove/adapt obsolete implementations explicitly if an -API change makes their old types or signatures fail to compile. Every regeneration -must be followed by go test ./... . Do not delete service files to change contracts. - -## Layout - -```text -api/ # editable contract and imported types -main.go # GENERATED startup, lifecycle and route registration -etc/config.yaml # lowerCamelCase configuration -internal/config/ # defaults -> file -> environment -internal/svc/ # dependency initialization and cleanup -internal/types/typesGen.go # generated request/response types -internal/apierrors/errorsGen.go # generated fresh error constructors, when declared -internal/handler// # generated binding and Logic adapters -internal/logic// # developer-owned business operations -internal/repository// # developer-owned persistence contracts -internal/middleware/ # named hooks and generated registry -docs/openapi.json # generated OpenAPI 3.0.3 -.goOmni/apiManifest.json # ownership and content hashes -``` - -The API response type describes business data. Declare response Name (...) and -select the App default with @app(defaultResponse: Name) in the entry API. -@server(response: Name) and @response Name override it for groups/routes; -@response raw returns the business JSON directly. Without a template, the default -envelope includes code, message, data and requestId. Generated routes and OpenAPI -use the same contract. 404/405 and ordinary App routes use the declared App default. -No template is selected by declaration/import order. Without @app, existing framework -response defaults apply. An explicit API App default takes precedence over response -options in svc, keeping generated routes and OpenAPI consistent. -Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default. -Use @body(maxBytes: ...) for route limits. @body(mode: raw|multipart; maxBytes: ...) -adds a *httpx.Body argument to Logic, without handwritten handlers. Raw is the exact -limited request stream; Parts supports sequential multipart reading with no temp files. -@produces "application/octet-stream" (or another concrete media type) generates a -Logic result of *httpx.Stream; the handler closes Content and streams it unwrapped. -Signature verification and storage policy belong to Logic/middleware. A consumed -request stream cannot be reread automatically; use one reader owner or explicitly -bounded buffering. Stream reads respect configured server timeouts. Multipart parts -are not DTO form fields: form tags remain URL query parameters. The optional manual -handler/custom binder hooks are preserved, and are needed only for custom protocols. - -Fields require exactly one source tag: json, path, form (URL query only), header. -All are required by presence unless optional, omitempty or default is specified. -Zero and false are valid present values; binding:"required" rejects scalar zero. -Use *T with optional when business logic needs to distinguish missing from zero. -Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON -fields and duplicate scalar query/header values are rejected. Arrays in the query -use repeated keys. Enum/range/default and nested fields share the documentation -and binding compiler. Responses are expected to honor their declared constraints; -they are not validated on every response. Do not return nil slices/maps for a -non-nullable collection (initialize empty collections instead). - -Go-owned errors can be registered once through svc.ServiceContext.AppOptions with -omni.WithErrors(projecterrors.Catalog()). Generated routes inherit them unless they -declare an independent catalog. Do not repeat Go-owned errors in the API contract. -Alternatively, declare public business errors in error (...) and reference them with @errors. -Use internal/apierrors constructors to share status, code and message with OpenAPI. -Auth declarations connect a bearer/header/cookie scheme to a real named middleware. -The selected auth runs first; @auth none removes inherited declarative auth. -Validation policy remains in the middleware implementation. Public messages must -not contain secrets; error causes may be included in server logs. - -svc.New receives the loaded Config and borrowed Logger. Add dependencies there, -then immediately register each cleanup with OnClose. A later initialization failure -runs registered cleanups in reverse order. Close is idempotent. GlobalMiddleware -provides global hooks; Middlewares supplies named API hooks. AppOptions is evaluated -once before route registration; logging is configured earlier through Config. -The generated main runs HTTP draining before closing svc resources, then the logger. -Do not close borrowed resources or attach duplicate cleanup hooks to App. The startup -context cancels on signals; long-lived clients/workers that requests depend on need -their own lifetime, canceled by OnClose after HTTP draining. - -Named middleware is composed in declaration order (first is outermost). Implement -the generated hooks or replace entries in svc; never bypass an auth -hook merely to make the server start. Middleware names do not imply a JWT policy. - -This CLI supports a documented go-zero-style subset, not all goctl features. -Database tables and repositories cannot be inferred from an HTTP contract. Add -business repositories through svc. Database and storage options are -composed at creation; the original profile is retained on regeneration. - -## Infrastructure - -Selected database: none; storage: none; logging: stderr/json. -MySQL uses database/sql through deps.DB, including InTx and a readiness probe. -Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*. -No tables are invented or migrated automatically from request/response DTOs. -The none/memory selection opens no database; implement repositories per business. - -With storage enabled, services receive deps.Files, Uploads, Downloads and -UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object; -set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL. -Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint -or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate -and authorize before issuing them. No public upload-link route is added. -Storage failures during startup close an already-open database; Close is idempotent. - -Database/storage/logging selection is recorded in .goOmni/apiManifest.json. -Changing selection through generation flags is rejected on an existing project; -edit your owned configuration and initialization explicitly, or generate a fresh -project for comparison. New routes still regenerate without repeating these flags. +Configuration, API contracts, svc fields/Init/AppOptions, Logic, Repository and +middleware implementations remain developer-owned. Regeneration preserves them. +Use `go-omni api format` to format local contracts, then `api generate` and +`api generate --check`. See the [framework development guide](../../DEVELOPMENT.md) +for array/null responses, typed multipart, full configuration environment overrides, +shared contracts and v0.13 upgrade steps. This example keeps its raw callback, +streaming upload and download to exercise those existing extension points. diff --git a/examples/managedService/docs/openapi.json b/examples/managedService/docs/openapi.json index 50ab25b..e251a9f 100644 --- a/examples/managedService/docs/openapi.json +++ b/examples/managedService/docs/openapi.json @@ -195,6 +195,12 @@ "demo" ], "x-omni-max-body-bytes": 64, + "x-omni-request-policy": { + "unknownQuery": "", + "utf8": "", + "charset": "", + "body": "auto" + }, "x-omni-response": "CommonResponse" } }, @@ -410,6 +416,12 @@ "tags": [ "demo" ], + "x-omni-request-policy": { + "unknownQuery": "", + "utf8": "", + "charset": "", + "body": "" + }, "x-omni-response": "CommonResponse" } }, @@ -589,6 +601,12 @@ "tags": [ "demo" ], + "x-omni-request-policy": { + "unknownQuery": "", + "utf8": "", + "charset": "", + "body": "" + }, "x-omni-response": "CommonResponse" } }, @@ -804,6 +822,12 @@ "tags": [ "demo" ], + "x-omni-request-policy": { + "unknownQuery": "", + "utf8": "", + "charset": "", + "body": "" + }, "x-omni-response": "AlternateResponse" } }, @@ -992,6 +1016,12 @@ "tags": [ "demo" ], + "x-omni-request-policy": { + "unknownQuery": "", + "utf8": "", + "charset": "", + "body": "" + }, "x-omni-response": "raw" } }, @@ -1197,6 +1227,12 @@ "tags": [ "demo" ], + "x-omni-request-policy": { + "unknownQuery": "", + "utf8": "", + "charset": "", + "body": "" + }, "x-omni-response": "CommonResponse" } }, @@ -1414,9 +1450,27 @@ "demo" ], "x-omni-max-body-bytes": 2048, + "x-omni-multipart-limits": { + "maxBytes": 2048, + "maxFieldBytes": 65536, + "maxFileBytes": 2048, + "maxParts": 128 + }, + "x-omni-request-policy": { + "unknownQuery": "", + "utf8": "", + "charset": "", + "body": "auto" + }, "x-omni-response": "CommonResponse" } } }, + "x-omni-contract-sources": { + "files": { + "managed.api": "cec14fe0da47a83804bf461be047df3e004ee470cbc3ea330dabc1537d596672" + }, + "shared": {} + }, "x-omni-default-response": "CommonResponse" } diff --git a/examples/managedService/go.mod b/examples/managedService/go.mod index 10bb65a..dbdd160 100644 --- a/examples/managedService/go.mod +++ b/examples/managedService/go.mod @@ -2,7 +2,7 @@ module example.com/managed go 1.22.0 -require git.yuntuo.hk/k2safe/go-omni v0.13.0-preview +require git.yuntuo.hk/k2safe/go-omni v0.14.0-preview require ( github.com/gabriel-vasile/mimetype v1.4.3 // indirect diff --git a/examples/managedService/internal/config/config.go b/examples/managedService/internal/config/config.go index 5e83f81..1fee979 100644 --- a/examples/managedService/internal/config/config.go +++ b/examples/managedService/internal/config/config.go @@ -1,14 +1,11 @@ -// Package settings owns this project's configuration; it adds no business identity model. package config import ( - "errors" + "git.yuntuo.hk/k2safe/go-omni/config" + "git.yuntuo.hk/k2safe/go-omni/logx" "os" "strings" "time" - - "git.yuntuo.hk/k2safe/go-omni/config" - "git.yuntuo.hk/k2safe/go-omni/logx" ) type Config struct { @@ -26,44 +23,40 @@ func Default() Config { return Config{Server: server} } -// Load uses defaults -> strict YAML/JSON -> nonempty environment variables. -// Pass an empty path to use only defaults and environment. +// Load applies defaults -> file -> typed environment -> validation. +// New fields automatically use GO_OMNI_, or an explicit env tag. func Load(path string) (Config, error) { cfg := Default() var err error + cfg, err = config.ApplyDefaults(cfg) + if err != nil { + return Config{}, err + } if path != "" { cfg, err = config.DecodeFile(path, cfg) if err != nil { return Config{}, err } } + cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_"}) + if err != nil { + return Config{}, err + } + // Preserve the documented framework aliases (HTTP_ADDR, LOG_LEVEL, etc). cfg.Server, err = config.ApplyEnv(cfg.Server, "") if err != nil { return Config{}, err } - // A stream explicitly selected by environment overrides a file profile. if output := strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")); output != "" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE")) == "" { cfg.Server.Log.File = "" cfg.Server.Log.Rotation = logx.RotationConfig{} } - - if err = cfg.Validate(); err != nil { - return Config{}, err - } - return cfg, nil + return cfg, cfg.Validate() } func (c Config) Validate() error { - if err := c.Server.Validate(); err != nil { return err } - if c.Server.Log.Format != "json" && c.Server.Log.Format != "text" { - return errors.New("log.format must be json or text") - } - switch strings.ToLower(c.Server.Log.Level) { - case "debug", "info", "warn", "warning", "error": - default: - return errors.New("invalid log.level") - } + return nil } diff --git a/examples/managedService/internal/handler/appGen.go b/examples/managedService/internal/handler/appGen.go index 21aaef3..b2ded21 100644 --- a/examples/managedService/internal/handler/appGen.go +++ b/examples/managedService/internal/handler/appGen.go @@ -4,6 +4,7 @@ package handler import ( "fmt" omni "git.yuntuo.hk/k2safe/go-omni" + "git.yuntuo.hk/k2safe/go-omni/contract" "example.com/managed/internal/svc" "git.yuntuo.hk/k2safe/go-omni/config" @@ -16,11 +17,13 @@ func NewApp(deps *svc.ServiceContext) (*omni.App, error) { return nil, fmt.Errorf("service context and lifecycle controller required; see PROJECT.md upgrade instructions") } cfg := deps.Config.Server + cfg.LogInstance = deps.Logger for _, option := range deps.AppOptions() { if option != nil { option(&cfg) } } + cfg.Request = contract.MergePolicy(cfg.Request, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}) cfg.Response = newResponseTemplate("{\"body\":null,\"errorBody\":null,\"messageField\":\"msg\",\"requestIdField\":\"-\",\"codeType\":\"int32\",\"successCode\":0,\"errorCode\":9999,\"dataMode\":\"always\"}").Config() cfg.ResponseEncoder = nil if err := cfg.Validate(); err != nil { diff --git a/examples/managedService/internal/handler/demo/handlersGen.go b/examples/managedService/internal/handler/demo/handlersGen.go index 0534401..83dd6ad 100644 --- a/examples/managedService/internal/handler/demo/handlersGen.go +++ b/examples/managedService/internal/handler/demo/handlersGen.go @@ -36,7 +36,7 @@ var checkRequest = contract.MustNew[types.CheckReq]() func (h *Handler) Check(c *httpx.Context) error { var req types.CheckReq - if err := checkRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil { + if err := checkRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &req, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil { return err } resp, err := h.logic.Check(c.Context(), &req) @@ -53,7 +53,7 @@ var otherRequest = contract.MustNew[types.CheckReq]() func (h *Handler) Other(c *httpx.Context) error { var req types.CheckReq - if err := otherRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil { + if err := otherRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &req, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil { return err } resp, err := h.logic.Other(c.Context(), &req) @@ -65,7 +65,13 @@ func (h *Handler) Other(c *httpx.Context) error { } return c.OK(resp) } + +var rawJSONRequest = contract.MustNew[struct{}]() + func (h *Handler) RawJSON(c *httpx.Context) error { + if err := rawJSONRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil { + return err + } resp, err := h.logic.RawJSON(c.Context()) if err != nil { return err @@ -75,7 +81,13 @@ func (h *Handler) RawJSON(c *httpx.Context) error { } return c.OK(resp) } + +var uploadRequest = contract.MustNew[struct{}]() + func (h *Handler) Upload(c *httpx.Context) error { + if err := uploadRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: "auto"}, 2048); err != nil { + return err + } body, err := httpx.NewBody(c.Request(), "multipart") if err != nil { return err @@ -92,7 +104,13 @@ func (h *Handler) Upload(c *httpx.Context) error { } return c.OK(resp) } + +var callbackRequest = contract.MustNew[struct{}]() + func (h *Handler) Callback(c *httpx.Context) error { + if err := callbackRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: "auto"}, 64); err != nil { + return err + } body, err := httpx.NewBody(c.Request(), "raw") if err != nil { return err @@ -112,7 +130,13 @@ func (h *Handler) Callback(c *httpx.Context) error { } return c.WriteStream(200, "text/plain", resp) } + +var downloadRequest = contract.MustNew[struct{}]() + func (h *Handler) Download(c *httpx.Context) error { + if err := downloadRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil { + return err + } resp, err := h.logic.Download(c.Context()) if err != nil { if resp != nil { @@ -122,7 +146,13 @@ func (h *Handler) Download(c *httpx.Context) error { } return c.WriteStream(200, "application/octet-stream", resp) } + +var slowRequest = contract.MustNew[struct{}]() + func (h *Handler) Slow(c *httpx.Context) error { + if err := slowRequest.BindWithPolicy(c.Request(), httpx.RouteParams(c.Context()), &struct{}{}, contract.RequestPolicy{UnknownQuery: "", UTF8: "", Charset: "", Body: ""}); err != nil { + return err + } resp, err := h.logic.Slow(c.Context()) if err != nil { return err diff --git a/examples/managedService/internal/handler/openapiGen.go b/examples/managedService/internal/handler/openapiGen.go index bbc21c0..ff7521e 100644 --- a/examples/managedService/internal/handler/openapiGen.go +++ b/examples/managedService/internal/handler/openapiGen.go @@ -6,7 +6,7 @@ import ( "net/http" ) -const OpenAPIDocument = "{\n \"components\": {\n \"securitySchemes\": {}\n },\n \"info\": {\n \"title\": \"managed-api\",\n \"version\": \"1.0.0\"\n },\n \"openapi\": \"3.0.3\",\n \"paths\": {\n \"/callback\": {\n \"post\": {\n \"operationId\": \"demo_Callback\",\n \"requestBody\": {\n \"content\": {\n \"application/octet-stream\": {\n \"schema\": {\n \"format\": \"binary\",\n \"type\": \"string\"\n }\n }\n },\n \"required\": true\n },\n \"responses\": {\n \"200\": {\n \"content\": {\n \"text/plain\": {\n \"schema\": {\n \"type\": \"string\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-max-body-bytes\": 64,\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/check\": {\n \"get\": {\n \"operationId\": \"demo_Check\",\n \"parameters\": [\n {\n \"in\": \"query\",\n \"name\": \"kind\",\n \"required\": false,\n \"schema\": {\n \"type\": \"string\"\n }\n }\n ],\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/download\": {\n \"get\": {\n \"operationId\": \"demo_Download\",\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/octet-stream\": {\n \"schema\": {\n \"format\": \"binary\",\n \"type\": \"string\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/other\": {\n \"get\": {\n \"operationId\": \"demo_Other\",\n \"parameters\": [\n {\n \"in\": \"query\",\n \"name\": \"kind\",\n \"required\": false,\n \"schema\": {\n \"type\": \"string\"\n }\n }\n ],\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-response\": \"AlternateResponse\"\n }\n },\n \"/raw\": {\n \"get\": {\n \"operationId\": \"demo_RawJSON\",\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-response\": \"raw\"\n }\n },\n \"/slow\": {\n \"get\": {\n \"operationId\": \"demo_Slow\",\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/upload\": {\n \"post\": {\n \"operationId\": \"demo_Upload\",\n \"requestBody\": {\n \"content\": {\n \"multipart/form-data\": {\n \"schema\": {\n \"additionalProperties\": true,\n \"type\": \"object\"\n }\n }\n },\n \"required\": true\n },\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-max-body-bytes\": 2048,\n \"x-omni-response\": \"CommonResponse\"\n }\n }\n },\n \"x-omni-default-response\": \"CommonResponse\"\n}\n" +const OpenAPIDocument = "{\n \"components\": {\n \"securitySchemes\": {}\n },\n \"info\": {\n \"title\": \"managed-api\",\n \"version\": \"1.0.0\"\n },\n \"openapi\": \"3.0.3\",\n \"paths\": {\n \"/callback\": {\n \"post\": {\n \"operationId\": \"demo_Callback\",\n \"requestBody\": {\n \"content\": {\n \"application/octet-stream\": {\n \"schema\": {\n \"format\": \"binary\",\n \"type\": \"string\"\n }\n }\n },\n \"required\": true\n },\n \"responses\": {\n \"200\": {\n \"content\": {\n \"text/plain\": {\n \"schema\": {\n \"type\": \"string\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-max-body-bytes\": 64,\n \"x-omni-request-policy\": {\n \"unknownQuery\": \"\",\n \"utf8\": \"\",\n \"charset\": \"\",\n \"body\": \"auto\"\n },\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/check\": {\n \"get\": {\n \"operationId\": \"demo_Check\",\n \"parameters\": [\n {\n \"in\": \"query\",\n \"name\": \"kind\",\n \"required\": false,\n \"schema\": {\n \"type\": \"string\"\n }\n }\n ],\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-request-policy\": {\n \"unknownQuery\": \"\",\n \"utf8\": \"\",\n \"charset\": \"\",\n \"body\": \"\"\n },\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/download\": {\n \"get\": {\n \"operationId\": \"demo_Download\",\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/octet-stream\": {\n \"schema\": {\n \"format\": \"binary\",\n \"type\": \"string\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-request-policy\": {\n \"unknownQuery\": \"\",\n \"utf8\": \"\",\n \"charset\": \"\",\n \"body\": \"\"\n },\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/other\": {\n \"get\": {\n \"operationId\": \"demo_Other\",\n \"parameters\": [\n {\n \"in\": \"query\",\n \"name\": \"kind\",\n \"required\": false,\n \"schema\": {\n \"type\": \"string\"\n }\n }\n ],\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"detail\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"detail\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-request-policy\": {\n \"unknownQuery\": \"\",\n \"utf8\": \"\",\n \"charset\": \"\",\n \"body\": \"\"\n },\n \"x-omni-response\": \"AlternateResponse\"\n }\n },\n \"/raw\": {\n \"get\": {\n \"operationId\": \"demo_RawJSON\",\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-request-policy\": {\n \"unknownQuery\": \"\",\n \"utf8\": \"\",\n \"charset\": \"\",\n \"body\": \"\"\n },\n \"x-omni-response\": \"raw\"\n }\n },\n \"/slow\": {\n \"get\": {\n \"operationId\": \"demo_Slow\",\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-request-policy\": {\n \"unknownQuery\": \"\",\n \"utf8\": \"\",\n \"charset\": \"\",\n \"body\": \"\"\n },\n \"x-omni-response\": \"CommonResponse\"\n }\n },\n \"/upload\": {\n \"post\": {\n \"operationId\": \"demo_Upload\",\n \"requestBody\": {\n \"content\": {\n \"multipart/form-data\": {\n \"schema\": {\n \"additionalProperties\": true,\n \"type\": \"object\"\n }\n }\n },\n \"required\": true\n },\n \"responses\": {\n \"200\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"additionalProperties\": false,\n \"properties\": {\n \"bytes\": {\n \"format\": \"int64\",\n \"type\": \"integer\"\n }\n },\n \"required\": [\n \"bytes\"\n ],\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Success\"\n },\n \"400\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"413\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"415\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"500\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n },\n \"501\": {\n \"content\": {\n \"application/json\": {\n \"schema\": {\n \"properties\": {\n \"code\": {\n \"format\": \"int32\",\n \"type\": \"integer\"\n },\n \"data\": {\n \"enum\": [\n null\n ],\n \"nullable\": true,\n \"type\": \"object\"\n },\n \"msg\": {\n \"type\": \"string\"\n }\n },\n \"required\": [\n \"code\",\n \"msg\",\n \"data\"\n ],\n \"type\": \"object\"\n }\n }\n },\n \"description\": \"Error\"\n }\n },\n \"security\": [],\n \"summary\": \"\",\n \"tags\": [\n \"demo\"\n ],\n \"x-omni-max-body-bytes\": 2048,\n \"x-omni-multipart-limits\": {\n \"maxBytes\": 2048,\n \"maxFieldBytes\": 65536,\n \"maxFileBytes\": 2048,\n \"maxParts\": 128\n },\n \"x-omni-request-policy\": {\n \"unknownQuery\": \"\",\n \"utf8\": \"\",\n \"charset\": \"\",\n \"body\": \"auto\"\n },\n \"x-omni-response\": \"CommonResponse\"\n }\n }\n },\n \"x-omni-contract-sources\": {\n \"files\": {\n \"managed.api\": \"cec14fe0da47a83804bf461be047df3e004ee470cbc3ea330dabc1537d596672\"\n },\n \"shared\": {}\n },\n \"x-omni-default-response\": \"CommonResponse\"\n}\n" func OpenAPI(c *httpx.Context) error { return c.WriteResponse(200, http.Header{"Content-Type": {"application/json; charset=utf-8"}}, []byte(OpenAPIDocument)) diff --git a/examples/managedService/internal/svc/close.go b/examples/managedService/internal/svc/close.go deleted file mode 100644 index f030e55..0000000 --- a/examples/managedService/internal/svc/close.go +++ /dev/null @@ -1,39 +0,0 @@ -package svc - -import "errors" - -// Close is idempotent, including startup failure and graceful shutdown. -func (c *ServiceContext) Close() error { - if c == nil { - return nil - } - c.closeOnce.Do(func() { - if c.Lifecycle != nil { - c.Lifecycle.RequestShutdown(nil) - } - c.closeMu.Lock() - c.closed = true - cleanups := c.cleanups - c.cleanups = nil - c.closeMu.Unlock() - for i := len(cleanups) - 1; i >= 0; i-- { - c.closeErr = errors.Join(c.closeErr, cleanups[i]()) - } - - }) - return c.closeErr -} - -// OnClose transfers a resource cleanup to this context immediately after creation. -// Hooks run once in reverse order, also when later initialization fails. -func (c *ServiceContext) OnClose(cleanup func() error) { - if cleanup == nil { - return - } - c.closeMu.Lock() - defer c.closeMu.Unlock() - if c.closed { - panic("svc: register cleanup after Close") - } - c.cleanups = append(c.cleanups, cleanup) -} diff --git a/examples/managedService/internal/svc/runtimeGen.go b/examples/managedService/internal/svc/runtimeGen.go new file mode 100644 index 0000000..1765d59 --- /dev/null +++ b/examples/managedService/internal/svc/runtimeGen.go @@ -0,0 +1,56 @@ +// Code generated by go-omni api. DO NOT EDIT. +package svc + +import ( + "context" + "errors" + "git.yuntuo.hk/k2safe/go-omni/health" + "git.yuntuo.hk/k2safe/go-omni/lifecycle" + "git.yuntuo.hk/k2safe/go-omni/logx" + + "example.com/managed/internal/config" + "example.com/managed/internal/middleware" +) + +// New is generator-owned. Customize dependencies in ServiceContext.Init. +func New(ctx context.Context, logger *logx.Logger, configs ...config.Config) (result *ServiceContext, err error) { + if ctx == nil { + ctx = context.Background() + } + if err = ctx.Err(); err != nil { + return nil, errors.Join(err, lifecycle.Cause(ctx)) + } + if logger == nil { + return nil, errors.New("service context requires a logger") + } + if len(configs) > 1 { + return nil, errors.New("expected at most one configuration") + } + cfg := config.Default() + if len(configs) == 1 { + cfg = configs[0] + } + if err = cfg.Validate(); err != nil { + return nil, err + } + checker, err := health.New() + if err != nil { + return nil, err + } + c := &ServiceContext{Config: cfg, Logger: logger, Health: checker, Middlewares: middleware.Defaults(), Runtime: lifecycle.NewRuntime(ctx)} + ctx = logx.WithLogger(c.Context(), logger) + defer func() { + if p := recover(); p != nil { + _ = c.Close() + panic(p) + } + if err != nil { + err = c.Finish(err) + } + }() + + if err = c.Init(ctx); err != nil { + return nil, err + } + return c, nil +} diff --git a/examples/managedService/internal/svc/serviceContext.go b/examples/managedService/internal/svc/serviceContext.go index 9c21796..8876f91 100644 --- a/examples/managedService/internal/svc/serviceContext.go +++ b/examples/managedService/internal/svc/serviceContext.go @@ -2,66 +2,25 @@ package svc import ( "context" - "errors" - "sync" "example.com/managed/internal/config" - "example.com/managed/internal/middleware" "git.yuntuo.hk/k2safe/go-omni/health" "git.yuntuo.hk/k2safe/go-omni/httpx" "git.yuntuo.hk/k2safe/go-omni/lifecycle" "git.yuntuo.hk/k2safe/go-omni/logx" ) -// Context owns infrastructure, while Logger is borrowed from main. -// Add business repositories here; HTTP contracts do not define database tables. +// Context holds project dependencies. Runtime owns cleanup and shutdown. +// Logger is borrowed; the generated entry closes it after dependencies. type ServiceContext struct { - Lifecycle *lifecycle.Controller + *lifecycle.Runtime Config config.Config - cleanups []func() error Logger *logx.Logger Health *health.Checker Middlewares map[string]httpx.Middleware - - closeMu sync.Mutex - closed bool - closeOnce sync.Once - closeErr error } -// Pass the settings loaded by main. The optional form supports lightweight -// tests without infrastructure; selected adapters still require valid config. -func New(ctx context.Context, logger *logx.Logger, configs ...config.Config) (result *ServiceContext, err error) { - if ctx == nil { - ctx = context.Background() - } - if err = ctx.Err(); err != nil { - return nil, errors.Join(err, lifecycle.Cause(ctx)) - } - if len(configs) > 1 { - return nil, errors.New("expected at most one configuration") - } - cfg := config.Default() - if len(configs) == 1 { - cfg = configs[0] - } - _ = cfg - checker, err := health.New() - if err != nil { - return nil, err - } - c := &ServiceContext{Config: cfg, Logger: logger, Health: checker, Middlewares: middleware.Defaults(), Lifecycle: lifecycle.New(ctx)} - ctx = c.Lifecycle.Context() - defer func() { - if err != nil { - err = errors.Join(err, c.Close()) - if cause := c.Lifecycle.Err(); cause != nil && !errors.Is(err, cause) { - err = errors.Join(err, cause) - } - } - }() - - return c, nil -} - -func (c *ServiceContext) RequestShutdown(err error) { c.Lifecycle.RequestShutdown(err) } +// Init is project-owned. Add dependencies here and immediately register each +// cleanup with c.OnClose. Returning an error automatically rolls everything back. +// Generated infrastructure is ready here. Never create another App or logger. +func (c *ServiceContext) Init(ctx context.Context) error { return nil } diff --git a/examples/managedService/internal/types/dtoManagedd28645a2af1cGen.go b/examples/managedService/internal/types/dtoManagedd28645a2af1cGen.go new file mode 100644 index 0000000..3ed3c05 --- /dev/null +++ b/examples/managedService/internal/types/dtoManagedd28645a2af1cGen.go @@ -0,0 +1,9 @@ +// Code generated by go-omni api. DO NOT EDIT. +package types + +type CheckReq struct { + Kind string `form:"kind,optional"` +} +type Receipt struct { + Bytes int64 `json:"bytes"` +} diff --git a/examples/managedService/internal/types/typesGen.go b/examples/managedService/internal/types/typesGen.go index 3ed3c05..124e1a0 100644 --- a/examples/managedService/internal/types/typesGen.go +++ b/examples/managedService/internal/types/typesGen.go @@ -1,9 +1,2 @@ // Code generated by go-omni api. DO NOT EDIT. package types - -type CheckReq struct { - Kind string `form:"kind,optional"` -} -type Receipt struct { - Bytes int64 `json:"bytes"` -} diff --git a/examples/managedService/main.go b/examples/managedService/main.go index e8a62b6..7d38175 100644 --- a/examples/managedService/main.go +++ b/examples/managedService/main.go @@ -34,19 +34,13 @@ func run(ctx context.Context, path string) (runErr error) { return err } defer func() { runErr = errors.Join(runErr, logger.Close()) }() - logx.SetDefault(logger) deps, err := svc.New(ctx, logger, cfg) if err != nil { return err } // RunContext drains HTTP requests before returning. This scope owns dependencies // and closes them exactly once, before closing the borrowed logger. - defer func() { - runErr = errors.Join(runErr, deps.Close()) - if cause := deps.Lifecycle.Err(); cause != nil && !errors.Is(runErr, cause) { - runErr = errors.Join(runErr, cause) - } - }() + defer func() { runErr = deps.Finish(runErr) }() app, err := handler.NewApp(deps) if err != nil { return err diff --git a/httpx/metadata.go b/httpx/metadata.go new file mode 100644 index 0000000..b972c81 --- /dev/null +++ b/httpx/metadata.go @@ -0,0 +1,22 @@ +package httpx + +import "context" + +// RequestMetadata is a value snapshot for Logic. It excludes credentials, +// query strings, headers and bodies. ClientIP is resolved by App's proxy policy. +type RequestMetadata struct{ ClientIP, Method, Route string } +type metadataKey struct{} + +func WithRequestMetadata(ctx context.Context, value RequestMetadata) context.Context { + if ctx == nil { + ctx = context.Background() + } + return context.WithValue(ctx, metadataKey{}, value) +} +func Metadata(ctx context.Context) RequestMetadata { + if ctx == nil { + return RequestMetadata{} + } + value, _ := ctx.Value(metadataKey{}).(RequestMetadata) + return value +} diff --git a/lifecycle/runtime.go b/lifecycle/runtime.go new file mode 100644 index 0000000..ff54e9f --- /dev/null +++ b/lifecycle/runtime.go @@ -0,0 +1,81 @@ +package lifecycle + +import ( + "context" + "errors" + "fmt" + "sync" +) + +// Runtime owns resources registered during dependency initialization. A server +// must finish draining before calling Close. Embed *Runtime in the project's svc. +// The logger and any other borrowed resources must not be registered here. +type Runtime struct { + Lifecycle *Controller + mu sync.Mutex + cleanups []func() error + closed bool + once sync.Once + closeErr error +} + +func NewRuntime(ctx context.Context) *Runtime { return &Runtime{Lifecycle: New(ctx)} } +func (r *Runtime) Context() context.Context { return r.Lifecycle.Context() } +func (r *Runtime) RequestShutdown(err error) { r.Lifecycle.RequestShutdown(err) } + +// OnClose transfers cleanup ownership. Register immediately after creation, +// before exposing resources to other goroutines. Registration after Close is a +// programming error. Callbacks may request shutdown, but must not call Close. +func (r *Runtime) OnClose(cleanup func() error) { + if cleanup == nil { + return + } + r.mu.Lock() + defer r.mu.Unlock() + if r.closed { + panic("lifecycle: register cleanup after Close") + } + r.cleanups = append(r.cleanups, cleanup) +} + +// Close releases all resources in reverse order, including after a failed +// initialization. All callbacks are attempted even if one fails or panics. +func (r *Runtime) Close() error { + if r == nil { + return nil + } + r.once.Do(func() { + r.RequestShutdown(nil) + r.mu.Lock() + r.closed = true + cleanups := r.cleanups + r.cleanups = nil + r.mu.Unlock() + for i := len(cleanups) - 1; i >= 0; i-- { + r.closeErr = errors.Join(r.closeErr, runCleanup(cleanups[i])) + } + }) + return r.closeErr +} + +func runCleanup(fn func() error) (err error) { + defer func() { + if p := recover(); p != nil { + err = fmt.Errorf("lifecycle: cleanup panic: %v", p) + } + }() + return fn() +} + +// Finish closes resources and preserves the operation, cleanup and shutdown +// errors. Read the final cause only after monitors have been stopped by Close. +func (r *Runtime) Finish(err error) error { + if r == nil { + return err + } + err = errors.Join(err, r.Close()) + if cause := r.Lifecycle.Err(); cause != nil && !errors.Is(err, cause) { + err = errors.Join(err, cause) + } + return err +} diff --git a/lifecycle/runtime_test.go b/lifecycle/runtime_test.go new file mode 100644 index 0000000..00c9180 --- /dev/null +++ b/lifecycle/runtime_test.go @@ -0,0 +1,56 @@ +package lifecycle + +import ( + "context" + "errors" + "reflect" + "strings" + "sync" + "testing" +) + +func TestRuntimeRollbackAndConcurrentClose(t *testing.T) { + r := NewRuntime(context.Background()) + var order []int + failure, cleanup := errors.New("initialization failed"), errors.New("cleanup failed") + r.OnClose(func() error { order = append(order, 1); return cleanup }) + r.OnClose(func() error { order = append(order, 2); panic("broken dependency") }) + r.OnClose(func() error { order = append(order, 3); return nil }) + var wg sync.WaitGroup + for i := 0; i < 20; i++ { + wg.Add(1) + go func() { + defer wg.Done() + if err := r.Close(); !errors.Is(err, cleanup) { + t.Error(err) + } + }() + } + wg.Wait() + if !reflect.DeepEqual(order, []int{3, 2, 1}) { + t.Fatal(order) + } + err := r.Finish(failure) + if !errors.Is(err, failure) || !errors.Is(err, cleanup) || !strings.Contains(err.Error(), "cleanup panic") { + t.Fatal(err) + } + if r.Context().Err() == nil { + t.Fatal("runtime context was not stopped") + } +} + +func TestRuntimeRetainsMonitorFaultDuringCleanup(t *testing.T) { + r := NewRuntime(context.Background()) + fault := errors.New("monitor failed") + r.OnClose(func() error { r.RequestShutdown(fault); return nil }) + r.RequestShutdown(nil) + if err := r.Finish(nil); !errors.Is(err, fault) { + t.Fatal(err) + } + defer func() { + if recover() == nil { + t.Fatal("late ownership transfer accepted") + } + }() + r.OnClose(func() error { return nil }) +} diff --git a/logx/adapters.go b/logx/adapters.go new file mode 100644 index 0000000..5e0195f --- /dev/null +++ b/logx/adapters.go @@ -0,0 +1,67 @@ +package logx + +import ( + "context" + "errors" + "log" + "log/slog" + "strings" +) + +// Validate checks configuration without opening files or creating resources. +func (c Config) Validate() error { + c = normalizeConfig(c) + if _, err := parseLevel(c.Level); err != nil { + return errors.New("logx: invalid level") + } + if c.Format != "json" && c.Format != "text" { + return errors.New("logx: format must be json or text") + } + switch c.Output { + case "stdout", "stderr", "discard", "none": + default: + return errors.New("logx: invalid output") + } + return c.Rotation.validate(c) +} + +type loggerContextKey struct{} + +// WithLogger borrows a logger for a call tree; it does not alter the global +// default and does not transfer ownership. nil preserves the existing context. +func WithLogger(ctx context.Context, logger *Logger) context.Context { + ctx = normalizeContext(ctx) + if logger == nil { + return ctx + } + return context.WithValue(ctx, loggerContextKey{}, logger) +} + +// FromContext returns a call-scoped logger, falling back to package logging. +func FromContext(ctx context.Context) *Logger { + if ctx != nil { + if l, ok := ctx.Value(loggerContextKey{}).(*Logger); ok && l != nil { + return l + } + } + return defaultLogger.Load() +} + +// Standard adapts this instance to the standard logger (for http.Server etc). +// It borrows the writer and never closes it. Each Print call becomes one event. +func (l *Logger) Standard(level slog.Level) *log.Logger { + if l == nil { + l = defaultLogger.Load() + } + return log.New(standardWriter{logger: l, level: level}, "", 0) +} + +type standardWriter struct { + logger *Logger + level slog.Level +} + +func (w standardWriter) Write(p []byte) (int, error) { + w.logger.logger.Log(context.Background(), w.level, strings.TrimRight(string(p), "\r\n")) + return len(p), nil +} diff --git a/logx/adapters_test.go b/logx/adapters_test.go new file mode 100644 index 0000000..508989e --- /dev/null +++ b/logx/adapters_test.go @@ -0,0 +1,33 @@ +package logx + +import ( + "bytes" + "log/slog" + "os" + "path/filepath" + "strings" + "testing" +) + +func TestValidateBeforeOpeningWriter(t *testing.T) { + p := filepath.Join(t.TempDir(), "must-not-exist.log") + for _, cfg := range []Config{{File: p, Format: "typo"}, {File: p, Level: "typo"}, {File: p, Output: "typo"}} { + if cfg.Validate() == nil { + t.Fatal("invalid configuration accepted") + } + if _, err := New(cfg); err == nil { + t.Fatal("invalid logger created") + } + if _, err := os.Stat(p); !os.IsNotExist(err) { + t.Fatal("validation touched filesystem", err) + } + } +} +func TestStandardLoggerUsesBorrowedInstance(t *testing.T) { + var out bytes.Buffer + l, _ := New(Config{Writer: &out}) + l.Standard(slog.LevelWarn).Print("driver diagnostic") + if !strings.Contains(out.String(), `"level":"WARN"`) || !strings.Contains(out.String(), "driver diagnostic") { + t.Fatal(out.String()) + } +} diff --git a/logx/logx.go b/logx/logx.go index 0c6fc82..16d5b29 100644 --- a/logx/logx.go +++ b/logx/logx.go @@ -73,7 +73,7 @@ func init() { // call Close when the process is shutting down. func New(cfg Config) (*Logger, error) { cfg = normalizeConfig(cfg) - if err := cfg.Rotation.validate(cfg); err != nil { + if err := cfg.Validate(); err != nil { return nil, err } level, err := parseLevel(cfg.Level) @@ -225,7 +225,7 @@ func (l *Logger) ErrorStackContext(ctx context.Context, msg string, args ...any) } func ErrorStackContext(ctx context.Context, msg string, args ...any) { - defaultLogger.Load().ErrorStackContext(ctx, msg, args...) + FromContext(ctx).ErrorStackContext(ctx, msg, args...) } func (l *Logger) Debug(msg string, args ...any) { l.DebugContext(context.Background(), msg, args...) } @@ -234,19 +234,19 @@ func (l *Logger) Warn(msg string, args ...any) { l.WarnContext(context.Backgrou func (l *Logger) Error(msg string, args ...any) { l.ErrorContext(context.Background(), msg, args...) } func DebugContext(ctx context.Context, msg string, args ...any) { - defaultLogger.Load().DebugContext(ctx, msg, args...) + FromContext(ctx).DebugContext(ctx, msg, args...) } func InfoContext(ctx context.Context, msg string, args ...any) { - defaultLogger.Load().InfoContext(ctx, msg, args...) + FromContext(ctx).InfoContext(ctx, msg, args...) } func WarnContext(ctx context.Context, msg string, args ...any) { - defaultLogger.Load().WarnContext(ctx, msg, args...) + FromContext(ctx).WarnContext(ctx, msg, args...) } func ErrorContext(ctx context.Context, msg string, args ...any) { - defaultLogger.Load().ErrorContext(ctx, msg, args...) + FromContext(ctx).ErrorContext(ctx, msg, args...) } func Debug(msg string, args ...any) { DebugContext(context.Background(), msg, args...) } diff --git a/middleware/contextErrors.go b/middleware/contextErrors.go new file mode 100644 index 0000000..ac058a6 --- /dev/null +++ b/middleware/contextErrors.go @@ -0,0 +1,30 @@ +package middleware + +import ( + "context" + "errors" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "git.yuntuo.hk/k2safe/go-omni/httpx" +) + +// ContextErrors classifies unhandled context errors using project catalog keys. +// Empty keys leave that class unchanged. Explicit API errors always take +// precedence, even if they wrap a context failure. Causes stay available to logs. +func ContextErrors(canceledKey, deadlineKey string) httpx.Middleware { + return func(next httpx.Handler) httpx.Handler { + return func(c *httpx.Context) error { + err := next(c) + var public *apierrors.APIError + if err == nil || errors.As(err, &public) { + return err + } + if canceledKey != "" && errors.Is(err, context.Canceled) { + return apierrors.ByCode(canceledKey).WithCause(err) + } + if deadlineKey != "" && errors.Is(err, context.DeadlineExceeded) { + return apierrors.ByCode(deadlineKey).WithCause(err) + } + return err + } + } +} diff --git a/middleware/contextErrors_test.go b/middleware/contextErrors_test.go new file mode 100644 index 0000000..41cd8ac --- /dev/null +++ b/middleware/contextErrors_test.go @@ -0,0 +1,25 @@ +package middleware_test + +import ( + "context" + "errors" + "fmt" + apierrors "git.yuntuo.hk/k2safe/go-omni/errors" + "git.yuntuo.hk/k2safe/go-omni/httpx" + "git.yuntuo.hk/k2safe/go-omni/middleware" + "testing" +) + +func TestContextErrorsPreservesExplicitBusinessFailures(t *testing.T) { + business := apierrors.New(409, "CONFLICT", "conflict").WithCause(context.DeadlineExceeded) + for _, tc := range []struct { + err error + code string + }{{context.Canceled, "CANCELED"}, {fmt.Errorf("operation: %w", context.DeadlineExceeded), "TIMEOUT"}, {business, "CONFLICT"}} { + err := middleware.ContextErrors("CANCELED", "TIMEOUT")(func(*httpx.Context) error { return tc.err })(nil) + var result *apierrors.APIError + if !errors.As(err, &result) || result.Code != tc.code || !errors.Is(err, tc.err) { + t.Fatal(err) + } + } +} diff --git a/omni.go b/omni.go index 5ff185b..19555f5 100644 --- a/omni.go +++ b/omni.go @@ -97,3 +97,8 @@ func NotFound(code, message string) *APIError { return apierrors.NotFound(code, func Internal(cause error) *APIError { return apierrors.Internal(cause) } func PrincipalFromContext(ctx *Context) (Principal, bool) { return auth.FromContext(ctx) } + +// WithLogInstance borrows an independent logger for this application. +var WithLogInstance = config.WithLogInstance + +var WithRequestPolicy = config.WithRequestPolicy diff --git a/openapi/openapi.go b/openapi/openapi.go index 03ac68d..ad5d991 100644 --- a/openapi/openapi.go +++ b/openapi/openapi.go @@ -17,6 +17,9 @@ import ( ) type Schema struct { + MaxBytes *int64 `json:"x-omni-max-bytes,omitempty"` + Example any `json:"example,omitempty"` + EnumDescriptions []string `json:"x-enum-descriptions,omitempty"` Type string `json:"type,omitempty"` Format string `json:"format,omitempty"` Description string `json:"description,omitempty"` diff --git a/reports/businessDevelopmentAcceptance.md b/reports/businessDevelopmentAcceptance.md new file mode 100644 index 0000000..1f46b26 --- /dev/null +++ b/reports/businessDevelopmentAcceptance.md @@ -0,0 +1,58 @@ +# v0.14.0-preview 通用业务开发验收 + +验收日期：2026-09-24。范围为 go-omni 框架、生成消费者、隔离本地 MySQL；没有修改 eTax/ePay 业务，也没有连接既有业务服务器。本轮实现不内置业务字段、中文文案、表结构、权限或事务规则。 + +## 完成项与业务可移除的适配 + +| 能力 | 验收内容 | 接入项目可替换的通用代码 | +| --- | --- | --- | +| 根数组、可空对象、成功 null | 自动 Logic/Handler，nil 数组归一为 `[]`，可空对象与 null 保留，raw/统一公共体/204 分开验证 | 仅为响应形状存在的手写 Handler | +| 严格请求策略 | 未知 query、原始非法 UTF-8、charset、必填/可选/禁止 body、严格 `{}`、重复键、未知字段、分块限额及 413 | strictRequest、jsonEncoding、emptyJSON、query 白名单与从 OpenAPI 反读规则 | +| 声明式 multipart | 文件/文本/JSON DTO、普通字段校验先于 Logic、未知/重复/缺失/截断/各级超限、文件读者与临时文件清理 | FormFile/PostFormValue、手工 JSON 表单解码；仍保留资料内容检查 | +| 配置与校验 | 全结构体默认值→文件→环境、显式 0/false、空值策略、嵌套、duration/map/slice、敏感值不进入解析诊断、失败不污染默认配置 | 重复 LookupEnv/ParseBool/Atoi/ParseDuration 和框架日志校验 | +| 生命周期与日志 | 逆序清理、并发幂等关闭、失败回滚、清理 panic 后继续、依赖故障原因保留、排空先于释放、多 App Logger 隔离 | svc 的锁/once/清理栈、标准日志桥接、可信 ClientIP 搬运 | +| 依赖中间件与 context 错误 | 小接口工厂装配，标准 context 传递可信元数据；取消/超时映射尊重显式业务错误 | 为接入依赖而放进 context 的服务对象、重复 contextErrors | +| 字段文档与格式化 | 中文前置/同行注释、example/枚举说明；格式化保字面量与注释、字段对齐、幂等/只读 check；元数据不调用项目 scalar 解码器 | 常规接口的文档反射补丁、第二份手工维护的 OpenAPI | +| Go 错误目录文档 | 显式 ExportCatalog、严格元数据解码、参数/响应示例、独立路由目录优先，CLI 不执行项目 Go | 在 `.api` 重复维护 Go 错误表 | +| 多服务契约与升级 | roots/摘要、路径与符号链接边界、循环/深度/数量限制、分文件 DTO、独立 module 工作区；原模板升级、定制文件保护及失败回滚 | response.api 复制脚本、把整棵新生成目录手工拼回项目 | +| MySQL 接入 | 结构化配置、独立 TLS/双向 TLS、主机名与 CA 验证、连接池/会话/驱动日志隔离、启动检查、清理；继续复用标准 Connector | TLS 注册、驱动配置、建池/Ping/日志适配和通用失败清理；保留 SQL/版本和表结构策略 | + +以上是可替换的职责，不是已删除 eTax 文件的声明。业务方应迁入等价契约/配置并通过自己的权限、金额和事务回归后，再移除旧适配。 + +## 本地回归 + +以下命令已通过： + +```sh +GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./... -count=1 -timeout=300s +go vet ./... +go build ./... +go run ./cmd/go-omni api generate --dir examples/managedService --check +go run ./cmd/go-omni api format --api examples/contracts/business.api --check +git diff --check +python3 scripts/verifyManagedProject.py --report reports/businessDevelopmentManagedLocal.json +python3 scripts/verifyConnector.py --local --report reports/businessDevelopmentConnectorLocal.json +``` + +全量 race 包括框架既有网络层、错误继承/参数、桥接、上传下载回归，以及实际生成消费者。生成器集成包括 MySQL 与多种存储配置的编译/配置校验；不把这些编译测试称作云服务联调。 + +[生成进程报告](businessDevelopmentManagedLocal.json) 验证根目录 `go run .` / `go build .`、生成稳定性、业务保留、nethttp/native 的真实 HTTP、SIGINT/SIGTERM 和端口释放。消费者测试另外覆盖初始化/监听失败、依赖故障退出、排空后关闭、内置接口配置和 404/405。 + +[MySQL 报告](businessDevelopmentConnectorLocal.json) 使用仅绑定 localhost 的临时容器，限制 1 CPU、768 MiB、128 PIDs；临时凭据、CA、服务端与客户端证书在测试后移除。真实验证 TLS/mTLS、坏 CA/主机名拒绝、多个池隔离、会话设置与失败恢复，并复验 nethttp/native 读写超时。生成服务只改配置即可建池，并验证清理和借用 Logger 的所有权。 + +测试文件中的可选基础设施用例在普通 `go test` 下会跳过；上面的隔离数据库脚本显式开启它们，并拒绝测试级 skip。报告包含源码摘要用于追溯。 + +## 远程发布验收 + +代码发布标签后，使用以下命令在空 module 缓存验收同版远程 CLI/框架，不使用本地 replace；结果另行归档，发布标签不移动： + +```sh +python3 scripts/verifyManagedRelease.py --version v0.14.0-preview --report reports/businessDevelopmentManagedRemote.json +python3 scripts/verifyConnector.py --version v0.14.0-preview --report reports/businessDevelopmentConnectorRemote.json +``` + +## 维护边界 + +用户维护 `.api`、业务配置、svc 字段/Init/AppOptions/DatabaseOptions、错误目录、Logic、Repository、权限中间件及特殊协议扩展。生成器管理 main、runtimeGen、路由/Handler、DTO 和文档。升级方法和完整示例见 [业务开发指南](../DEVELOPMENT.md)。 + +回归通过不等于具体业务已迁移。声明式上传使用临时磁盘，部署仍需限制请求并发和容量；清理函数必须自行停止并等待业务后台任务。框架不会回滚 Logic 已提交的业务事务；纯流式接口仍遵守 Complete 后再提交副作用的约定。特殊 scalar 的业务语义和自定义 Encoder 文档仍需项目验证。 diff --git a/reports/businessDevelopmentConnectorLocal.json b/reports/businessDevelopmentConnectorLocal.json new file mode 100644 index 0000000..80fd7a5 --- /dev/null +++ b/reports/businessDevelopmentConnectorLocal.json @@ -0,0 +1,74 @@ +{ + "passed": true, + "frameworkVersion": "v0.14.0-preview", + "localSource": true, + "frameworkSum": null, + "freshModuleCache": false, + "noLocalReplace": false, + "mysqlImage": "sha256:1afd25fe334a1be64a3866495dd43a6bdba3c55dd1d6190ea12933163fe2843c", + "mysqlLimits": { + "cpus": 1, + "memoryMiB": 768, + "pids": 128 + }, + "durationSeconds": 12.64, + "tests": [ + "TestHTTPConfiguredStandardServer", + "TestHTTPInvalidStartupReleasesResources", + "TestHTTPConfiguredNetworkLimits/nethttp/slowHeaders", + "TestHTTPConfiguredNetworkLimits/nethttp/slowBody", + "TestHTTPConfiguredNetworkLimits/nethttp/idle", + "TestHTTPConfiguredNetworkLimits/nethttp/writeBackpressure", + "TestHTTPFileAndEnvironment/server.json", + "TestHTTPFileAndEnvironment/server.yaml", + "TestHTTPFileAndEnvironment", + "TestHTTPInvalidAndUnsupportedSettings/READ_HEADER_TIMEOUTinvalid", + "TestHTTPInvalidAndUnsupportedSettings/READ_HEADER_TIMEOUT-1", + "TestHTTPInvalidAndUnsupportedSettings/READ_TIMEOUTinvalid", + "TestHTTPInvalidAndUnsupportedSettings/READ_TIMEOUT-1", + "TestHTTPInvalidAndUnsupportedSettings/WRITE_TIMEOUTinvalid", + "TestHTTPInvalidAndUnsupportedSettings/WRITE_TIMEOUT-1", + "TestHTTPInvalidAndUnsupportedSettings/IDLE_TIMEOUTinvalid", + "TestHTTPInvalidAndUnsupportedSettings/IDLE_TIMEOUT-1", + "TestHTTPInvalidAndUnsupportedSettings/MAX_HEADER_BYTESinvalid", + "TestHTTPInvalidAndUnsupportedSettings/MAX_HEADER_BYTES-1", + "TestHTTPInvalidAndUnsupportedSettings", + "TestConnectorIndependentPoolsAndContext", + "TestConnectorRejectsInvalidInputWithoutTakingOwnership", + "TestConnectorStartupFailuresReleaseResources/connect", + "TestConnectorStartupFailuresReleaseResources/ping", + "TestConnectorStartupFailuresReleaseResources", + "TestConnectorTimeoutBoundsConnectAndPing/connect", + "TestConnectorTimeoutBoundsConnectAndPing/ping", + "TestConnectorTimeoutBoundsConnectAndPing", + "TestConnectorHealthBoundsPoolWaitAndRecovers", + "TestConnectorIdleZeroAndLifetime/noIdle", + "TestConnectorIdleZeroAndLifetime/expired", + "TestConnectorIdleZeroAndLifetime", + "TestConnectorTransactionsKeepCommitAndRollbackSemantics", + "TestMySQLConnectorDriverLoggerIsolation", + "TestMySQLConnectorTLSAndPoolIsolation/untrustedCA", + "TestMySQLConnectorTLSAndPoolIsolation/wrongHostname", + "TestMySQLConnectorTLSAndPoolIsolation", + "TestHTTPConfiguredNetworkLimits/nethttp/headerLimit", + "TestHTTPConfiguredNetworkLimits/nethttp", + "TestHTTPConfiguredNetworkLimits/native/slowHeaders", + "TestStructuredMySQLAdapterTLS", + "TestHTTPConfiguredNetworkLimits/native/slowBody", + "TestHTTPConfiguredNetworkLimits/native/idle", + "TestHTTPConfiguredNetworkLimits/native/writeBackpressure", + "TestHTTPConfiguredNetworkLimits/native/headerLimit", + "TestHTTPConfiguredNetworkLimits/native", + "TestHTTPConfiguredNetworkLimits", + "TestGeneratedMySQLConfigurationAndLifecycle" + ], + "sourceHashes": { + "database/connector_test.go": "0663dfc9794ba24130225fa3b6418a6d5f75d57d0f01686151f70c7ea3f592c0", + "database/mysqlConnector_test.go": "e1cdd11ef76b09f1332ba92bfdde5b1917d84c85cc42fac48f65a8b23f5e9720", + "app/httpConfig_test.go": "b060dc0b76fb8e5feebae7e3b12cb7bca845693fcc097e2a0496a5ba937c837a", + "config/http_test.go": "0cae3dcf213369cfaff5487bf630c2bc3ec097739ca1d40794124e69c3880db8", + "database/mysql/integration_test.go": "05b0a73d50363eef760e8a3324a7e9a68c598f7519c6360ad530503178b56f43", + "scripts/verifyConnector.py": "4d2b1da0a1fcf641e02600710789ced51a58cf6fa1726a00c194a01f8e557e1a", + "scripts/testdata/mysqlGenerated_test.go": "5630123add5ed17f261c1a1bf63ba773f7fa6ea613fd573d5c5cb72da3e42645" + } +} diff --git a/reports/businessDevelopmentManagedLocal.json b/reports/businessDevelopmentManagedLocal.json new file mode 100644 index 0000000..036087a --- /dev/null +++ b/reports/businessDevelopmentManagedLocal.json @@ -0,0 +1,57 @@ +{ + "passed": true, + "mode": "localSource", + "durationSeconds": 4.43, + "checks": [ + "rootGoRun", + "rootGoBuild", + "stableRegeneration", + "checkMainOwnership", + "contractChangePreview", + "logicSignatureCheckPersists", + "businessSourcePreserved", + "fixedConsumerBuild", + "nethttpSIGINT", + "nethttpSIGTERM", + "nativeSIGINT", + "nativeSIGTERM", + "listenerReleased", + "temporaryProcessesRemoved", + "nethttpGeneratedTransfers", + "nativeGeneratedTransfers", + "malformedMultipart400", + "goOwnedErrors", + "default404And405" + ], + "sourceHashes": { + "cmd/go-omni/apigen/generate.go": "e0290bc5b6fdd9404da7944f2697299548ca3ea3611cf0e4a3ad3db5be2f7554", + "cmd/go-omni/apigen/templates.go": "88b96812e2f9ee57e22b15842268bbd8396d90f950191a72f1d75e987254f3ff", + "cmd/go-omni/apigen/startup.go": "e3322fa3e2aff9f45d66427f44187535c7c0960edc0f5fdfd5d7b4943b349334", + "cmd/go-omni/apigen/managed_test.go": "2b6bdf62a06cf25b85cbf06d60a5284ba24256c6705611fad4c012f63661a03d", + "cmd/go-omni/apigen/contractChanges.go": "aeb83990e908738faaa3cdfa20c3b8938023db05b6e55f2d24352c44144056a8", + "cmd/go-omni/apigen/logicCheck.go": "ca1af4f876a3a0db7b7b9ba9b82c1d6399fd89d29f667dad39cfd3de526eb1cb", + "cmd/go-omni/apigen/write.go": "180c5f5daee1441fe936cb376b1645a3d0c802cc78bf4b451dbf7e7b3e4ac0f4", + "cmd/go-omni/api.go": "5203a17c5f66977a9eba838d61cf5a7356de53d5f72a1196fd4f2046aa43ac85", + "config/config.go": "d2dc0b98bcce154957d760775906ca734b836b60852e1db34c93dbcc6c4b0bc3", + "httpx/transfer.go": "c9fe4e98c0e599d367eaf19ebecf367db914edfca9ad3a65527496d1dea80a3d", + "httpx/multipart.go": "b942ede7a810b268cd43b395c3bddcfaaacad92812bbb60e653466b8d86bc875", + "scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18", + "lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31", + "health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866", + "config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d", + "httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb", + "app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0", + "app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73", + "cmd/go-omni/scaffold/templates/api-config.go.tmpl": "fa37bd0a71587e2f821ec1595eb1f1a371677c90428001ea75f61815243c3b7b", + "cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "aacde854edd12a35cd97de41a449a306ec7bb6f245a0d62391d44e9a2ee1486b", + "cmd/go-omni/scaffold/templates/api-close.go.tmpl": "3f22d272e92f3b64b62123b3d80ec1a6ce8834257d64752fd75facf12623902b", + "cmd/go-omni/scaffold/templates/api-databaseOptions.go.tmpl": "6e5395b4a787c77a6cabfd85f16049a39578d6ec751a9b78203f3a03baabdf28", + "cmd/go-omni/scaffold/templates/api-options.go.tmpl": "11fe34ce45e8c98f7b5c9c0a2c512eb0b3abf86f95b4b576e083bcb6f5d0f824", + "cmd/go-omni/scaffold/templates/api-context.go.tmpl": "ac64c9134bf6f68cbd21b1974a31e3100d6f1c2751f2c3b6f35d6807037f41fa", + "cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247", + "cmd/go-omni/apigen/testdata/managed/client.go": "db886c2d4dd27e67f7446265c20147335d4898d2d2091518e67403806ccd9b69", + "cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319", + "cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f", + "cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1" + } +} diff --git a/scripts/testdata/mysqlGenerated_test.go b/scripts/testdata/mysqlGenerated_test.go new file mode 100644 index 0000000..368ecd3 --- /dev/null +++ b/scripts/testdata/mysqlGenerated_test.go @@ -0,0 +1,76 @@ +package main + +import ( + "bytes" + "context" + "crypto/tls" + "errors" + "strings" + "testing" + "time" + + "example.com/mysqlAcceptance/internal/config" + "example.com/mysqlAcceptance/internal/svc" + "git.yuntuo.hk/k2safe/go-omni/logx" +) + +func TestGeneratedMySQLConfigurationAndLifecycle(t *testing.T) { + cfg, err := config.Load("etc/config.yaml") + if err != nil { + t.Fatal(err) + } + if cfg.Database.User != "omni_test" || cfg.Database.TLS.ServerName != "localhost" || cfg.Database.Pool.MaxOpenConns != 2 { + t.Fatal("structured environment overlay was not applied") + } + ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + var logs bytes.Buffer + logger, err := logx.New(logx.Config{Writer: &logs}) + if err != nil { + t.Fatal(err) + } + defer logger.Close() + first, err := svc.New(ctx, logger, cfg) + if err != nil { + t.Fatal(err) + } + defer first.Close() + second, err := svc.New(ctx, logger, cfg) + if err != nil { + t.Fatal(err) + } + defer second.Close() + if first.DB == second.DB { + t.Fatal("contexts share a pool") + } + if err = first.DB.Health(ctx); err != nil { + t.Fatal(err) + } + var key, cipher string + if err = first.DB.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&key, &cipher); err != nil || cipher == "" { + t.Fatal("generated pool has no TLS", err) + } + if err = first.Finish(nil); err != nil { + t.Fatal(err) + } + if err = first.Finish(nil); err != nil { + t.Fatal("cleanup not idempotent", err) + } + if err = first.DB.PingContext(ctx); err == nil { + t.Fatal("dependency pool remains open") + } + if err = second.DB.Health(ctx); err != nil { + t.Fatal("another context was closed", err) + } + bad := cfg + bad.Database.TLS.ServerName = "wrong.invalid" + failed, err := svc.New(ctx, logger, bad) + var verify *tls.CertificateVerificationError + if failed != nil || !errors.As(err, &verify) { + t.Fatal("TLS failure not propagated", err) + } + logger.Info("logger remains usable") + if !strings.Contains(logs.String(), "logger remains usable") { + t.Fatal("borrowed logger was closed") + } +} diff --git a/scripts/verifyConnector.py b/scripts/verifyConnector.py index e827700..fc7e1c6 100644 --- a/scripts/verifyConnector.py +++ b/scripts/verifyConnector.py @@ -20,7 +20,7 @@ import uuid ROOT = Path(__file__).resolve().parents[1] MODULE = "git.yuntuo.hk/k2safe/go-omni" SOURCES = ("database/connector_test.go", "database/mysqlConnector_test.go", - "app/httpConfig_test.go", "config/http_test.go") + "app/httpConfig_test.go", "config/http_test.go", "database/mysql/integration_test.go") def run(args, cwd, env, timeout=300): @@ -28,7 +28,7 @@ def run(args, cwd, env, timeout=300): text=True, capture_output=True, timeout=timeout) output = re.sub(r"https://[^/@\s]+:[^/@\s]+@", "https://", result.stdout + result.stderr) # Never print generated database credentials, including from a failed command. - for key in ("MYSQL_ROOT_PASSWORD", "MYSQL_PASSWORD", "GO_OMNI_CONNECTOR_PASSWORD"): + for key in ("MYSQL_ROOT_PASSWORD", "MYSQL_PASSWORD", "GO_OMNI_CONNECTOR_PASSWORD", "GO_OMNI_DB_PASSWORD"): if env.get(key): output = output.replace(env[key], "[redacted]") if result.returncode: @@ -62,12 +62,15 @@ def main(): run(["openssl", "req", "-newkey", "rsa:2048", "-nodes", "-keyout", certs / "server.key", "-out", temp / "server.csr", "-subj", "/CN=localhost"], temp, env) (temp / "extensions.cnf").write_text("subjectAltName=DNS:localhost,IP:127.0.0.1\nbasicConstraints=CA:FALSE\nextendedKeyUsage=serverAuth\n") run(["openssl", "x509", "-req", "-in", temp / "server.csr", "-CA", certs / "ca.pem", "-CAkey", certs / "ca.key", "-CAcreateserial", "-out", certs / "server.pem", "-days", "2", "-extfile", temp / "extensions.cnf"], temp, env) + run(["openssl", "req", "-newkey", "rsa:2048", "-nodes", "-keyout", certs / "client.key", "-out", temp / "client.csr", "-subj", "/CN=GoOmni Test Client"], temp, env) + (temp / "clientExtensions.cnf").write_text("basicConstraints=CA:FALSE\nextendedKeyUsage=clientAuth\n") + run(["openssl", "x509", "-req", "-in", temp / "client.csr", "-CA", certs / "ca.pem", "-CAkey", certs / "ca.key", "-CAcreateserial", "-out", certs / "client.pem", "-days", "2", "-extfile", temp / "clientExtensions.cnf"], temp, env) (certs / "ca.key").unlink() for path in certs.iterdir(): path.chmod(0o644) # Disposable fixtures, readable by the container's mysql uid. project = temp / "consumer" project.mkdir() - version = args.version or "v0.9.1-preview" + version = args.version or "v0.14.0-preview" (project / "go.mod").write_text(f"module example.com/connectorAcceptance\n\ngo 1.22.0\n\nrequire {MODULE} {version}\n") if args.local: run(["go", "mod", "edit", f"-replace={MODULE}={ROOT}"], project, env) @@ -75,13 +78,30 @@ def main(): env["GOMODCACHE"] = str(temp / "modcache") for name in SOURCES: target = project / name - target.parent.mkdir(exist_ok=True) + target.parent.mkdir(exist_ok=True, parents=True) shutil.copyfile(ROOT / name, target) print("Resolving consumer dependency", flush=True) run(["go", "mod", "tidy"], project, env) module = json.loads(run(["go", "list", "-m", "-json", MODULE], project, env)) if not args.local: assert module["Version"] == args.version and module.get("Sum") and not module.get("Replace"), module + cli = temp / "bin/go-omni" + cli.parent.mkdir() + if args.local: + run(["go", "build", "-o", cli, "./cmd/go-omni"], ROOT, env) + else: + env["GOBIN"] = str(cli.parent) + run(["go", "install", MODULE + "/cmd/go-omni@" + args.version], temp, env) + generated = temp / "generated" + command = [cli, "new", "--database", "mysql", "--module", "example.com/mysqlAcceptance"] + if args.local: + command += ["--framework-dir", ROOT] + run([*command, generated], temp, env) + shutil.copyfile(ROOT / "scripts/testdata/mysqlGenerated_test.go", generated / "mysql_test.go") + run(["go", "mod", "tidy"], generated, env) + generated_module = json.loads(run(["go", "list", "-m", "-json", MODULE], generated, env)) + if not args.local: + assert generated_module["Version"] == args.version and generated_module.get("Sum") == module["Sum"] and not generated_module.get("Replace") env.update(MYSQL_ROOT_PASSWORD=secrets.token_hex(24), MYSQL_PASSWORD=secrets.token_hex(24)) name = "go-omni-connector-" + uuid.uuid4().hex[:12] created = False @@ -103,17 +123,34 @@ def main(): time.sleep(1) if not ready: raise RuntimeError("isolated MySQL fixture did not become ready") + admin_env = dict(env, MYSQL_PWD=env["MYSQL_ROOT_PASSWORD"]) + run(["docker", "exec", "--env", "MYSQL_PWD", name, "mysql", "-uroot", "-e", "CREATE USER 'omni_mtls'@'%' REQUIRE X509"], temp, admin_env) addr = run(["docker", "port", name, "3306/tcp"], temp, env).strip() assert addr.startswith("127.0.0.1:"), addr env.update(GO_OMNI_CONNECTOR_MYSQL_TEST="1", GO_OMNI_CONNECTOR_ADDR=addr, GO_OMNI_CONNECTOR_USER="omni_test", GO_OMNI_CONNECTOR_PASSWORD=env["MYSQL_PASSWORD"], - GO_OMNI_CONNECTOR_DATABASE="omni_test", GO_OMNI_CONNECTOR_CA_FILE=str(certs / "ca.pem")) + GO_OMNI_CONNECTOR_DATABASE="omni_test", GO_OMNI_CONNECTOR_CA_FILE=str(certs / "ca.pem"), + GO_OMNI_CONNECTOR_CLIENT_CERT=str(certs / "client.pem"), GO_OMNI_CONNECTOR_CLIENT_KEY=str(certs / "client.key")) print("Testing TLS trust, per-connector options, driver logging, lifecycle and HTTP limits", flush=True) output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=60s", "./..."], project, env) events = [json.loads(line) for line in output.splitlines() if line.startswith("{")] passed = [e["Test"] for e in events if e.get("Action") == "pass" and "Test" in e] assert "TestMySQLConnectorTLSAndPoolIsolation" in passed + assert "TestStructuredMySQLAdapterTLS" in passed assert not any(e.get("Action") in ("fail", "skip") for e in events), output + env.update(GO_OMNI_DB_HOST="127.0.0.1", GO_OMNI_DB_PORT=addr.rsplit(":", 1)[1], + GO_OMNI_DB_USER="omni_test", GO_OMNI_DB_PASSWORD=env["MYSQL_PASSWORD"], + GO_OMNI_DB_DATABASE="omni_test", GO_OMNI_DB_TLS_CA_FILE=str(certs / "ca.pem"), + GO_OMNI_DB_TLS_SERVER_NAME="localhost", GO_OMNI_DB_POOL_MAX_OPEN_CONNS="2", + GO_OMNI_DB_POOL_MAX_IDLE_CONNS="2") + print("Testing generated MySQL service config, independent pools and cleanup", flush=True) + run([cli, "api", "generate", "--check"], generated, env) + output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=60s", "./..."], generated, env) + generated_events = [json.loads(line) for line in output.splitlines() if line.startswith("{")] + generated_passed = [e["Test"] for e in generated_events if e.get("Action") == "pass" and "Test" in e] + assert "TestGeneratedMySQLConfigurationAndLifecycle" in generated_passed, output + assert not any(e.get("Action") == "fail" or (e.get("Action") == "skip" and "Test" in e) for e in generated_events), output + passed += generated_passed image = run(["docker", "inspect", "--format", "{{.Image}}", name], temp, env).strip() finally: if created: @@ -123,7 +160,7 @@ def main(): "noLocalReplace": not bool(module.get("Replace")), "mysqlImage": image, "mysqlLimits": {"cpus": 1, "memoryMiB": 768, "pids": 128}, "durationSeconds": round(time.monotonic() - began, 2), "tests": passed, - "sourceHashes": {name: hashlib.sha256((ROOT / name).read_bytes()).hexdigest() for name in (*SOURCES, "scripts/verifyConnector.py")}} + "sourceHashes": {name: hashlib.sha256((ROOT / name).read_bytes()).hexdigest() for name in (*SOURCES, "scripts/verifyConnector.py", "scripts/testdata/mysqlGenerated_test.go")}} args.report.parent.mkdir(parents=True, exist_ok=True) args.report.write_text(json.dumps(report, ensure_ascii=False, indent=2) + "\n") print("PASS; isolated container, credentials, certificates and consumer removed", flush=True) diff --git a/scripts/verifyManagedRelease.py b/scripts/verifyManagedRelease.py index 589fa46..6c3d57a 100644 --- a/scripts/verifyManagedRelease.py +++ b/scripts/verifyManagedRelease.py @@ -72,7 +72,7 @@ def main(): checks += ["default404And405", "goOwnedErrorCatalog", "unwrappedDownloads", "rawCallback", "multipart400And413", "nethttpAndNativeTransfers"] env.update(GO_OMNI_SCAFFOLD_INTEGRATION="1", GO_OMNI_RELEASE_CLI=str(cli), GO_OMNI_RELEASE_VERSION=args.version) - tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer"] + tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject"] print("Running generated consumers against the remote framework, including lifecycle failure cleanup", flush=True) output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=240s", "./cmd/go-omni/apigen", "-run", "^(" + "|".join(tests) + ")$"], ROOT, env) events = [json.loads(line) for line in output.splitlines() if line.startswith("{")] @@ -81,10 +81,11 @@ def main(): evidence = "".join(e.get("Output", "") for e in events) sums = re.findall(r"checksum (h1:\S+); no local replace", evidence) assert len(sums) == len(tests) and all(s == dependency["Sum"] for s in sums), "remote consumer checksum mismatch" - checks += ["initializationAndListenFailureCleanup", "drainBeforeDependencyClose", "generatedConsumerRaceTests", "errorParamsAndInheritance", "manualHooksAndScalars", "dependencyFatalShutdown", "fatalProcessExitCode", "initializationShutdownNotification", "configuredBuiltinEndpoints", "projectHealthEnvelope", "endpointConflictCleanup"] + checks += ["initializationAndListenFailureCleanup", "drainBeforeDependencyClose", "generatedConsumerRaceTests", "errorParamsAndInheritance", "manualHooksAndScalars", "dependencyFatalShutdown", "fatalProcessExitCode", "initializationShutdownNotification", "configuredBuiltinEndpoints", "projectHealthEnvelope", "endpointConflictCleanup", "responseArrayNullableNull", "strictRequestPolicies", "declarativeMultipartBinding"] - sources = ["lifecycle/controller.go", "config/endpoints.go", "health/health.go", "httpx/response.go", "app/app.go", "app/transport.go", "scripts/verifyManagedRelease.py", "scripts/verifyManagedProject.py", "cmd/go-omni/apigen/managed_test.go"] + sources = ["lifecycle/runtime.go", "config/project.go", "contract/bind.go", "contract/policy.go", "contract/multipart.go", "contract/file.go", "cmd/go-omni/apigen/business_test.go", "cmd/go-omni/apigen/generate.go", "lifecycle/controller.go", "config/endpoints.go", "health/health.go", "httpx/response.go", "app/app.go", "app/transport.go", "scripts/verifyManagedRelease.py", "scripts/verifyManagedProject.py", "cmd/go-omni/apigen/managed_test.go"] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/managed").glob("*.go")] + sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/business").glob("*.go")] report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"], "freshModuleCache": True, "noLocalReplace": True, "durationSeconds": round(time.monotonic() - started, 2), "checks": checks,