Compare commits

..

11 Commits

Author SHA1 Message Date
20e84e5ff8 feat: support database connectors and HTTP server configuration 2026-09-24 02:18:18 +08:00
6200864a64 test: record remote migration release acceptance 2026-09-24 01:49:49 +08:00
a8c973d8a2 feat: add HTTP migration bridges and extensible API binding 2026-09-24 01:47:42 +08:00
7b8eca1a61 docs: make README the complete project integration entry point 2026-09-23 21:10:43 +08:00
35bd724326 test: isolate and record remote response release acceptance 2026-09-23 20:47:28 +08:00
7670b222ac feat: release configurable API response contracts
Define reusable response bodies in .api with typed code mappings, per-route overrides and matching OpenAPI schemas. Add application and callback encoders, early response policy selection, and isolated remote-consumer release verification.

Release v0.8.0-preview after full race, vet, build and local MySQL workflow acceptance.
2026-09-23 20:42:38 +08:00
bef7ea2641 docs: record remote API developer experience acceptance 2026-09-23 18:19:12 +08:00
2913e94a0a feat: declare API errors and authentication with developer tooling 2026-09-23 18:17:43 +08:00
ab8eac5fc4 docs: record remote preview acceptance and verify example error logging 2026-09-23 17:37:53 +08:00
0ce3efb6fc fix: normalize generated infrastructure file endings 2026-09-23 17:35:00 +08:00
2c74dc0859 feat: complete API-driven projects with infrastructure and business acceptance 2026-09-23 17:33:50 +08:00
210 changed files with 21412 additions and 380 deletions

View File

@@ -11,15 +11,17 @@
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别、轮转/脱敏/堆栈/慢请求 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
| `storage` | Local、S3 兼容(含 R2 配置)、OSS 官方 SDK 适配、临时下载链接 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
| `storage` | Local、S3/R2、OSS、临时下载、受限直传、云端分片 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
日志的配置、脱敏边界、轮转和慢请求记录见 [logx 使用说明](logx/README.md)。框架保留通用 Context/结构化字段扩展,不新增固定业务身份上下文。
## 对象存储
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
@@ -51,9 +53,9 @@ files, err := oss.New(client, bucket, 32<<20)
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,调用方保证与内容一致。默认单对象 32 MiB,流式读写;OSS 本适配器只做 simple upload,最大配置 5 GiB。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,并检查实际读取长度。默认单对象 32 MiB,流式读写;OSS 简单 Put 上限 5 GiB,更大使用独立分片能力。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。首版未封装前端直传、分片续传、生命周期、ACL、CDN 自定义域名和版本管理。
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。受限直传和云端分片现已提供可选接口,见 [上传与续传说明](storage/README.md)。生命周期、ACL、CDN 自定义域名和版本管理仍由平台/部署管理。
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。

287
API.md Normal file
View File

@@ -0,0 +1,287 @@
# API 契约与项目生成
首次接入请先看 [README](README.md):包含固定远程版本安装、可运行项目、完整 `.api` 示例和已有服务接入步骤。本页用于查阅语法与生成规则;下方带 `--framework-dir` 的命令用于框架源码开发。
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
## 与 go-zero 的对应关系
调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式:统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写,不引入 go-zero 或 Gin 运行时依赖。
| 使用体验 | go-omni 实现 |
| --- | --- |
| 新建接口项目 | `go-omni api new <directory>` |
| 按契约生成、更新 | `go-omni api generate --api user.api --dir .` |
| 熟悉的命令别名 | `api go -api user.api -dir .` |
| 校验契约 | `api validate --api user.api` |
| CI 检查生成产物是否过期 | `api generate --api user.api --dir . --check` |
| 业务逻辑目录 | `internal/service/<group>`,沿用 Omni 的 Service 分层 |
| 依赖装配 | `internal/servicecontext` |
| 接口文档 | 自动生成 `docs/openapi.json`,同时提供 `/openapi.json` |
这是明确限定的 go-zero 风格方言,**不是完整 goctl 兼容实现**。不支持的声明和校验标签会在生成前报错。
## 立即试用
源码开发时,从 go-omni 仓库运行,显式引用本地框架:
```sh
go run ./cmd/go-omni api new \
--module example.com/omniDemo --framework-dir . ../omniDemo
cd ../omniDemo
go mod tidy
go run ./cmd/server
```
访问 `http://127.0.0.1:8080/api/greet/Alice` 可得到带 `data.message` 的统一响应。`api new` 的问候示例包含可运行逻辑;自定义契约新生成的 Service 会明确返回 501,等待业务实现。
用现成的多分组定义生成项目,在 go-omni 仓库执行:
```sh
go run ./cmd/go-omni new --api examples/contracts/user.api \
--module example.com/userService --framework-dir . \
--log-output file --log-format json ../userService
```
`api generate` 在新目录中也会生成完整项目。默认只监听 localhost;`GO_OMNI_HTTP_ADDR` 修改地址,`GO_OMNI_TRANSPORT=native` 切换网络层。配置顺序为默认值 → `etc/config.yaml` → 环境变量。文件日志支持现有轮转配置。远程试用使用与 CLI 一致的固定版本,见 [发布说明](RELEASE.md)。
API 项目现在支持组合数据库、存储和日志:
```sh
go-omni new --api user.api --module example.com/userService \
--database mysql --storage local --log-output file ./userService
```
- `--database none|memory|mysql`:API 默认 `none`,`memory` 是不创建数据库连接的别名;MySQL 生成连接池、配置、事务入口和 readiness 探针。
- `--storage none|local|s3|r2|oss`:默认 none,生成统一 Files / Uploads / Downloads / UploadLinks 依赖;本地签名对象端点为 `/storage/object`。
- `--log-output stdout|stderr|file` 与 `--log-format json|text`:配置和无配置文件启动使用一致默认值。
初次选项记录在 `.goOmni/apiManifest.json`,以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错,因为初始化代码属于业务方;可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。旧式 CRUD 脚手架仍保留。
MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
HTTP 请求类型不能推导数据库表结构;迁移和 Repository 由业务自己维护。完整案例见 [用户管理示例](examples/userService/README.md),覆盖真实 MySQL、事务审计、鉴权、文件导出和日志。
## 接口定义
```go
syntax = "v1"
type UpdateUserReq {
UserID int64 `path:"userId,range=[1:1000000]"`
Notify bool `form:"notify,default=false"`
Name string `json:"name" binding:"required,max=50"`
Age *int `json:"age,optional,range=[0:150]"`
}
type UserResp {
UserID int64 `json:"userId"`
Name string `json:"name"`
}
@server (
group: user
prefix: /api/v1
)
service user-api {
@doc "Update a user"
@handler updateUser
put /users/:userId (UpdateUserReq) returns (UserResp)
}
```
类型和 Go 字段使用导出命名;JSON、query、path 名称使用 lowerCamelCase;Header 使用 HTTP 标准名称。响应类型定义业务返回值。未配置时使用默认公共体;当前源码支持在 `.api` 中声明公共响应模板,选择字段和结构,并按分组复用或由单接口覆盖,详见 [公共响应体](RESPONSE.md)。
支持:
- 可省略的 `syntax = "v1"`;`info` 中的 title、version 和描述性元数据。
- 单独的 `type X { ... }` 与 `type (...)`;基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
- 同一 service 名称的多个块;`@server` 的 group、prefix、middleware、auth、errors、response,group 可以写 `admin/user`。
- 每个路由必须有 `@handler`;可选单行/多行 `@doc`、`@auth`、`@errors`、`@response`;GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。
- 可省略请求或响应;有响应默认 200,无响应默认 204。Omni 扩展 `@status 201`、`@status 202`;204 不允许响应类型,HEAD 不允许响应体,GET/HEAD 不允许 JSON 请求体。
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query,不解析 multipart 或表单 body。query 切片用重复键;header/path 使用标量。
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
暂不支持匿名嵌入、别名、递归类型、未声明的外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
业务错误、具名鉴权、多行文档及下述开发命令从 CLI `v0.7.0-preview.1` 提供。公共响应模板从 `v0.8.0-preview` 提供,CLI 与业务项目中的框架依赖需一起升级。
## 业务错误:声明、代码和文档共用一份定义
```go
error (
Unauthenticated 401 "UNAUTHENTICATED" "authentication required"
Forbidden 403 "FORBIDDEN" "permission denied"
UserNotFound 404 "USER_NOT_FOUND" "user not found"
EmailTaken 409 "EMAIL_TAKEN" "email already in use"
)
```
也可写单条 `error EmailTaken 409 "EMAIL_TAKEN" "email already in use"`,或把错误目录放进相对导入的 `errors.api`。
名称用 UpperCamelCase,HTTP 状态范围 400–599,错误码用大写字母/数字/下划线且全局唯一,message 是可公开返回的稳定提示。
`@server (errors: Unauthenticated)` 声明组级错误;路由 `@errors UserNotFound, EmailTaken` 追加到组级列表并去重。
`@errors none` 清空继承的业务错误。未声明、重复声明或生成符号冲突会报告源码位置。
生成 `internal/apierrors/errorsGen.go`:
```go
return nil, apierrors.EmailTaken()
// 可用 apierrors.CodeEmailTaken 引用同一个错误码。
```
每次调用返回新错误,避免并发修改共享错误对象。生成器不会自动把任意数据库错误转成业务错误;由 Service/适配层调用这些函数。
内部 Cause 可能进入访问日志,敏感驱动报错应像 userService 示例一样先归类再转换。
OpenAPI 对每个状态按选定的响应模板生成结构和命名 examples,同一 409 下多个错误不会相互覆盖;平铺模式的纯业务状态还列出 code enum。
默认框架错误状态与自动添加的鉴权 401 保留开放的 code 字段,容纳校验器/中间件实际错误。
`x-omni-errors` 提供错误目录与路由关联。文档声明不拦截、改写或穷举运行时的所有错误。
## 鉴权:声明与路由注册一起变化
```go
auth UserAuth (
type: bearer
middleware: Auth
description: "Application access token"
)
@server (
group: user
prefix: /api/v1
auth: UserAuth
errors: Unauthenticated
)
service user-api {
@errors Forbidden, EmailTaken
@handler createUser
@status 201
post /users (CreateUserReq) returns (UserResp)
@auth none
@errors none
@handler publicStatus
get /status
}
```
`auth` 是具名契约,`middleware` 指向实际 hook;生成器把它放在普通 middleware 前面执行,并在注册任何路由前检查 hook 非空。
新 hook 未实现时返回 501,已有手写 hook 保留。需要在 hook 中验证凭证、失效时间、签发者、权限等;声明不会自动实现登录或 JWT 校验。
- `type: bearer`:对应 HTTP Bearer,可选 `bearerFormat: JWT`,这只是文档提示。
- `type: apiKey`:必须同时声明 `in: header` 或 `in: cookie` 和 `name`,例如 `X-API-Key` 或 `sessionId`。
- `auth: UserAuth` 可在组级使用;路由 `@auth OtherAuth` 替换,`@auth none` 撤销继承的声明鉴权。普通 middleware 继续保留。
- 不要再把该 auth 的 hook 重复写进普通 middleware。不支持 query token、多个方案 AND/OR、OAuth2 scopes 或隐含认证策略。
- 被保护路由自动描述 401;403 和具体错误码由 `@errors` 明确声明。未声明 auth 的旧项目仍按其原有 middleware 执行。
文档生成 `components.securitySchemes` 与逐接口 `security`;显式公共接口使用空数组。
结构遵循 [OpenAPI 3.0.3 Security Requirement](https://spec.openapis.org/oas/v3.0.3.html#security-requirement-object)。
## 文档和日常开发命令
保留单行 `@doc "摘要"`,也可以:
```go
@doc (
summary: "Create a user"
description: "Writes user data and audit in the same transaction"
tags: users, administration
deprecated: false
)
```
未设置 tags 时使用 group 分组。`info` 的 description/desc 也会进入 OpenAPI;两者不能同时声明。
在生成项目目录内,不必重复输入 API 路径:
```sh
go-omni api validate # 发现 manifest 中的入口并校验
go-omni api inspect # 方法、路径、Handler、鉴权与业务错误总览
go-omni api generate --dry-run # 预览新增/更新/移除,完全不写文件
go-omni api generate # 实际生成,保留手写业务
go-omni api generate --check # 只读检查,可用于 CI
go-omni api export > openapi.json # 独立导出纯 JSON,无需启动服务
go test ./...
```
`--dry-run` 和 `--check` 不能组合。新项目预览也不创建目录;预览不是审批或锁定,实际生成会重新检查文件状态。
在项目外可显式传 `--api`;生成时传 `--dir` 可定位其他项目。不会猜测多个 .api 中哪个是入口。
路由总览的 `(undeclared)` 仅表示没有声明 auth;旧式中间件仍可能做认证。
外发文档可用 export;项目内受生成器管理的 `docs/openapi.json` 请通过 generate 更新,以同步文件哈希。
## 迁移扩展(v0.9.0-preview)
- 动态 JSON 字段 `any` / `interface{}`,包含切片和 map;外围契约仍严格。
- `scalar Name (...)` 显式引用项目类型,声明 JSON 输入/输出类型及文本绑定;支持已有 JSON/Text 转换方法。
- `@body (mode: json/raw/multipart; maxBytes: ...)`:按路由限制请求体。
- `@binding custom` 生成保留的绑定 hook;`@handlerMode manual` 生成保留的整个 Handler,raw/multipart 使用这个入口。
- manual 可用 `@produces "text/plain"` 等声明无 DTO 成功输出。默认状态 200,初始手写 stub 返回 501。
完整语法、边界和示例见 [迁移扩展指南](MIGRATION.md)。新增生成代码要求 CLI 与框架依赖一起升级到 `v0.9.0-preview`;不兼容全部 goctl 语法。
## 参数语义
无 optional/default 的字段按“是否出现”判断必填,`false`、`0`、`""` 是已提供的值。额外声明 `binding:"required"` 才会拒绝标量零值。默认值只在字段缺失时使用,不替换显式的空值。
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;普通非指针字段拒绝 null(动态 `any` 允许);`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;非 nullable 集合请返回空集合而不是 nil。
## 生成目录和业务边界
```text
api/ # 开发者维护的契约及导入
cmd/server/main.go # 启动、日志、优雅退出
etc/config.yaml
internal/settings/
internal/servicecontext/ # 依赖初始化和释放
internal/types/typesGen.go
internal/apierrors/errorsGen.go # 声明错误后生成
internal/handler/routesGen.go
internal/handler/openapiGen.go
internal/handler/<group>/handlersGen.go
internal/service/<group>/service.go
internal/service/<group>/<operation>.go
internal/repository/<group>/repository.go
internal/middleware/registryGen.go
internal/middleware/<name>.go
docs/openapi.json
.goOmni/apiManifest.json
```
Service 方法接收标准 `context.Context` 与生成的请求 DTO;Handler 调用 Service 并完成响应包装。网络适配器仍经 Omni/httpx 接入。Repository 是业务所有的扩展位置,不自动生成虚构 CRUD 查询。
命名中间件按声明顺序组合,第一个在最外层。新 hook 初始返回 501,必须实现后才放行;registry 自动维护,hook 文件只创建一次。缺失 hook 在注册任何路由前报错。文档通过 `x-omni-middleware` 标出命名 hook,不把名称误当成已实现的认证协议。
## 安全更新与 CI
在生成项目中安装/使用与框架版本相符的 go-omni CLI:
```sh
go-omni api validate --api api/user.api
go-omni api generate --api api/user.api --dir .
go-omni api generate --api api/user.api --dir . --check
go test ./...
```
`Gen.go` 和 OpenAPI 由生成器管理,manifest 保存其 SHA-256。手动修改后更新会报冲突。Service、Repository、main、配置、依赖装配和中间件实现按字节保留;只补缺失文件。删除接口/分组时只移除旧的生成文件,保留业务代码;已删除类型仍被旧业务引用时,需要明确迁移该业务文件,编译器会提示。
`--check` 只读,检查产物是否过期,以及所需骨架是否缺失。契约复制到项目后,请编辑项目内的 API;外部定义与已存在副本不一致时会拒绝覆盖。一个项目由一个 API 入口管理,可以通过导入拆分文件。
生成器拒绝输出路径穿越、受管理路径上的符号链接、大小写路径冲突和无所有权的覆盖。生成前统一校验/格式化,锁文件防止两个生成器同时修改;文件以原子替换写入,普通写入失败会回滚。本版不是整个目录的崩溃原子事务:进程被强杀或机器断电后,应按 Git 恢复生成产物和 manifest,再重试。请将 manifest 与生成产物一并提交,也不要在生成过程中编辑这些文件。
生成器不自动安装依赖、不访问数据库、不推送仓库、不替业务实现权限规则。
## 验证
```sh
go test -race -p=2 ./contract ./openapi ./cmd/go-omni/...
python3 scripts/verifyAPI.py
```
Go 测试覆盖契约拒绝规则、多源绑定、零值/缺失、默认/范围/嵌套集合、文档约束、幂等更新、只读检查、业务保留、删除路由、路径边界及生成项目编译。Python 验收会在临时目录生成项目,分别启动 nethttp/native,验证真实 HTTP 请求、中间件、日志 requestId 和优雅退出;结束后清理自己创建的进程和临时文件。结果记录在 `performance/apiGenerationLocal.json`。

View File

@@ -1,6 +1,51 @@
# Changelog
## Unreleased
## v0.9.1-preview
- 新增标准 `database.OpenConnector` 和独立 `PoolConfig`;与 DSN 入口共享池参数、限时启动 Ping、失败清理、Health、事务,不注册全局驱动配置。
- 新增 `omni.WithHTTPConfig` / `Config.HTTP`:读头、读请求、写响应、空闲超时和头大小,贯通代码、JSON/YAML、环境变量、Run/Serve 与 NewServer;nethttp 支持 Go-only ErrorLog。
- 保持已有默认行为;负值和实验 http2 不支持的超时项在监听前报错;记录各网络层的具体语义。
- 增加真实 MySQL TLS/驱动日志隔离、连接池归属/取消/事务、nethttp/native 慢连接与写背压验收,以及无本地 replace 的远程消费者脚本。
- 兼容 v0.9.0-preview 生成代码;不修改业务项目的数据库策略或生成代码最低版本。
## v0.9.0-preview
- 标准 HTTP Handler/Middleware 桥接:跟踪 Raw 状态与提交,保留取消、流式能力和中间件跨请求状态;已提交响应不追加 JSON。
- `.api` 支持动态 JSON `any` / `interface{}`,保持重复键、大小、外围未知字段校验和数字精度。
- 路由级请求体限额、自定义绑定和 manual Handler;上传、原始 body、下载可保留手写实现,重生成受保护。
- 自定义 scalar 显式对接项目 JSON/Text 方法;支持可空和集合,分别生成请求/响应 OpenAPI;不放宽其他字段。
- 新增 [迁移扩展指南](MIGRATION.md)、可运行示例和真实生成项目回归。CLI 与框架依赖一起升级。
## v0.8.0-preview
- 公共响应 `codeType` 支持 `int32` / `int64` / `string`,校验默认码及映射范围,OpenAPI 标注整数位宽;JSON/YAML 和生成代码保留 int64 精度,旧 `integer` 兼容。
- 公共 JSON 响应体改为可配置模板:支持嵌套结构、独立错误体、错误码类型/映射、提示与空值行为;旧 DTO 仅保留源代码兼容。
- `.api` 支持 `response Name (...)`、分组引用和单路由覆盖;`@response raw` 输出原始业务 JSON。生成的模板、路由和 OpenAPI 使用同一份配置。
- 提供应用级/接口级编码器,以及 Text / XML / Bytes 直接输出,允许三方回调独立控制成功和失败的状态、Header、内容。
- 接口响应策略可在全局中间件之前挂载,覆盖提前鉴权/限流失败和 Context 克隆;内置路由 404/405 使用应用配置。
- 详见 [公共响应体](RESPONSE.md)。CLI 与项目框架依赖需要一起升级到本版。
## v0.7.0-preview.1
- `.api` 支持集中业务错误目录、组级与路由级错误声明,生成状态/错误码/提示一致的 `internal/apierrors` 函数,以及 OpenAPI 命名响应示例。
- 具名鉴权声明支持 Bearer、header/cookie API key,关联真实中间件;支持组级继承、单路由覆盖与公共接口,认证 hook 优先执行并在启动时校验。
- 新增多行 `@doc` 的摘要、描述、标签与弃用标记,`info.description` 同步文档。
- 新增 `api inspect`、`api export`、`api generate --dry-run`;项目内自动识别 manifest 的 API 入口,减少重复参数。
- 用户示例用真实 401/403/409 验证声明与行为,业务直接使用生成的错误函数;生成不会改写现有手写 hook 和 Service。
- 纠正旧文档误将事务 Outbox、分布式锁列为已实现的描述;这两个模块当前没有实现。
## v0.7.0-preview
- 新增 `.api` 契约驱动生成器:DTO、严格多来源绑定与校验、路由、分组 Handler / Service / Repository、命名中间件及 OpenAPI;生成文件哈希管理、业务文件保护和只读 `--check`。
- `new --api` / `api new` / `api generate` 可以组合 MySQL、日志以及 Local / S3 / R2 / OSS。保存初始基础设施选项,重生成自动沿用;显式切换已有项目选项会拒绝,避免静默丢失初始化代码。
- 新增 `examples/userService` 独立业务示例与 `verifyAPIWorkflow.py`:用户 CRUD / 分页、管理员 Token 鉴权、MySQL 事务审计、故障回滚、文件导出、requestId 日志、数据库 readiness 和优雅退出;支持从远程版本验收。
- 日志新增文件轮转、结构化敏感字段脱敏、异常堆栈及慢请求配置;存储新增统一上传策略、直传签名和云端分片能力。
- 新增缓存、HTTP 客户端、限流 / 幂等 / 超时、后台任务、Redis 队列、审计、迁移与 telemetry 等可选模块;业务身份与 Orbit 保持由应用自行组合。
- `database.ApplyEnv` 支持文件/默认配置叠加环境变量;数据库配置使用 lowerCamelCase,DSN 不参与 JSON/YAML 序列化。
- 修复 native 在业务层提前拒绝请求体为 413 时等待继续读取 body 的问题,拒绝后关闭未读连接。
以下是自上一版本以来的网络层变化与历史验收记录:
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。

182
MIGRATION.md Normal file
View File

@@ -0,0 +1,182 @@
# 已有 HTTP 项目逐步接入
从 `v0.9.0-preview` 开始,以下能力贯通运行时、`.api`、代码生成和 OpenAPI。CLI 和项目框架依赖一起升级;入口见 [README](README.md)。这些都是通用扩展点,框架不内置任何项目的 ID、支付回调、鉴权或数据库规则。
## 1. 保留标准 Handler 与中间件
```go
app := omni.New(omni.WithLogger(true))
app.Use(httpx.WrapMiddleware(existingMiddleware)) // func(http.Handler) http.Handler
// go-zero 风格的 func(http.HandlerFunc) http.HandlerFunc:
app.Use(httpx.WrapFuncMiddleware(existingFuncMiddleware))
app.POST("/legacy", httpx.WrapHandler(http.HandlerFunc(existingHandler)))
```
实际使用时先配置所有 `Use`,再注册路由。`WrapHandler` 也接受 `http.FileServer` 等标准 `http.Handler`。
- Handler 经过 App 的日志、requestId、恢复和应用中间件链,原来的 URL、状态码、Header、响应内容由它自己控制。
- `c.Raw()` 的写入也会更新 `c.Status()` / `c.Responded()`;404 会记为 404。提交响应后返回错误或 panic,不会再追加错误 JSON。103 等临时响应不算最终提交。
- 桥接器不缓存完整响应。下载继续流式复制;底层支持时保留 Flusher、Hijacker、Pusher、ReaderFrom、CloseNotifier。`http.ResponseController` 可沿 `Unwrap` 访问底层控制能力。能力仍取决于网络适配器和其他中间件。
- 请求 context、取消信号和已有值继续传递,Omni 的路径参数可用 Go 1.22 的 `r.PathValue("id")` 读取。其他框架私有的参数读取器仍需由项目适配。标准中间件应从原 `r.Context()` 派生上下文;不能换成无关的 Background,也不能在 ServeHTTP 返回后调用 next。异步 timeout middleware 不在此桥接器的契约内。
- 标准中间件在每条注册链构造一次,状态会跨请求保留。下游返回的错误在标准中间件退出前编码,因此压缩、审计等中间件能处理错误响应。它们自己的缓冲或改写行为仍由项目负责。
调用流使用 `App.Handle/GET/POST` 等入口即可,不要绕到原始 Router 注册。暂不使用 `.api` 的旧接口也能这样接入。
## 2. 动态 JSON:允许渐进补全 DTO
```go
type LegacyReply {
Data any `json:"data"`
}
```
`interface{}` 是 `any` 的等价写法,支持 `[]any`、`map[string]any` 和对应的 `interface{}` 写法。带方法的 interface 不支持,会提示明确的类型错误。
动态字段允许 object、array、string、number、bool、null;OpenAPI 字段 schema 为 `{}`,不假装知道内部结构。它放开的是**该字段的内容**,外围 DTO 未声明的字段仍被拒绝,JSON 重复键、深度、总大小限制仍生效。动态数字解析为 `json.Number`,避免大整数先转 float64 丢精度。原始 JSON 的空白、转义和字节布局不会保留;验签请走原始 body 入口。
这不会自动补齐缺少的业务路由,也不代表兼容全部 goctl 语法。能确定结构的新接口优先使用命名 DTO。
## 3. 按路由限制 body 与保留手写入口
普通 JSON 路由只声明需要的大小,不改变其他路由默认的 1 MiB:
```text
@body (mode: json
maxBytes: 2097152
)
@handler createItem
post /items (CreateItemReq) returns (ItemResp)
```
`maxBytes` 单位为字节,允许 1 至 1 TiB;不写时为 1 MiB。已知超长或读取未知长度/chunked body 超限都按 413 处理。内置 App/RouterGroup 在全局中间件读取前安装 reader 限额;同时保持接口响应策略。网络层若有更小限额,以更小值为准。手写 Handler 吞掉读取错误或已经提交响应后,框架不能重新改写 HTTP 状态,务必检查读取错误。
### 只替换绑定函数
```text
@binding custom
@body (mode: json
maxBytes: 65536
)
@handler createItem
post /items (CreateItemReq) returns (ItemResp)
```
首次生成 `internal/handler/<group>/createItemBind.go`:
```go
func (h *Handler) bindCreateItem(c *httpx.Context, req *types.CreateItemReq) error {
// 调用项目自己的解码器,填充 req;失败返回 error。
}
```
Handler 继续调用 Service 并使用统一响应;只把绑定过程交给这个函数。初始 hook 返回 501,不能忽略实现。自定义绑定需自己实现 Content-Type、未知字段、尾部数据等校验,并遵守 DTO 文档;框架负责路由总大小限制,不隐式套第二次 JSON 绑定。
### 上传、原文验签、下载
```text
@handlerMode manual
@body (mode: multipart
maxBytes: 10485760
)
@handler upload
post /files
@handlerMode manual
@body (mode: raw
maxBytes: 65536
)
@produces "text/plain"
@handler callback
post /callback
@handlerMode manual
@produces "application/octet-stream"
@handler download
get /download
```
首次生成 `<operation>Manual.go`,由项目实现整个 `func(h *Handler) Callback(c *httpx.Context) error`;依赖通过 `h.deps` 访问。生成器保留路由、中间件与文档管理,不会预读、JSON 解码或重新包装手写输出,也不生成这个路由的 Service 方法。需要调用 Service 时自行组合。
- manual 无返回类型默认成功状态 200;也可 `returns (UploadResp)` 描述 JSON DTO,手写实现按声明调用 `c.OK` 等。`@produces` 用于无 DTO 的手写输出,描述成功响应媒体类型,不会自动设置运行时 Header。
- raw 的文档为二进制请求体;multipart 为开放对象,具体文件字段与逐部分校验由项目实现。raw/multipart 请求 DTO 仅可含 path/form/header 字段,不能同时声明 JSON body 字段。`form` 始终指 URL query。
- 原文验签直接读取 `c.Request().Body`,在声明的小限额内 `io.ReadAll`;大文件用 `MultipartReader` 配合 `io.Copy` 流式处理,不能整个读入内存。
- 使用 `ParseMultipartForm` 时,其内存参数只是内存阈值,不是总上传限额。App/桥接器在请求结束(含错误和 panic)后删除临时文件;业务仍须关闭 `FormFile` 得到的文件。`MultipartReader` 不自动建临时文件,业务自行创建的文件由业务清理。
- 手写 Handler 的成功输出由它控制;返回的 error、鉴权拒绝仍走接口的错误策略。三方若要求错误也返回指定文本,使用 [自定义 Encoder](RESPONSE.md#手写路由配置文件与三方回调) 并维护对应错误文档;`@produces` 不会改变错误协议。
直接注册 Go 路由:
```go
app.HandleWithBodyLimit("POST", "/upload", 10<<20, nil, uploadHandler)
// nil encoder 沿用 App 策略,也可传项目自己的 response.Encoder。
```
通用注册器使用 `httpx.HandleWithBodyLimit(...)`;实现 `httpx.BodyLimitRegistrar` 可像 App 一样在全局中间件前限流量,未实现时只能在路由中间件入口限制。单独 `httpx.LimitBody` 同样受中间件顺序影响。
### 重新生成规则
`handlersGen.go`、`routesGen.go`、DTO、OpenAPI 继续由生成器管理;`*Bind.go`、`*Manual.go` 只首次创建,后续不会覆盖或删除。切换 manual 回自动模式时,先迁移实现并自行移除对应 Manual 文件,否则生成器明确拒绝,避免重复方法。修改 DTO 导致手写函数签名不匹配时,需要一起调整项目代码;`go test ./...` 是必要检查。
## 4. 自定义标量:项目定义行为,框架连接调用
例如输入 ID 兼容数字/字符串、输出统一为字符串:
```text
scalar BusinessID (
goImport: "example.com/billing/internal/ids"
goType: ID
jsonInput: string | int64
jsonOutput: string
text: true
)
type ItemReq {
ID BusinessID `path:"id"`
}
type ItemResp {
ID BusinessID `json:"id"`
}
```
`goImport`/`goType` 指向项目已有的具体 Go 类型,不是框架内置 ID。生成器创建类型别名和 `types.ContractScalars()`,然后把注册表显式传给每个 Binder。运行时使用标准接口:
| 项目类型的方法 | 用途 |
| --- | --- |
| 值接收者 `MarshalJSON() ([]byte, error)` | JSON 输出 |
| 指针接收者 `UnmarshalJSON([]byte) error` | JSON 输入 |
| 指针接收者 `UnmarshalText([]byte) error` | `text: true` 时的 path/query/header |
完整参考 [示例 ID 类型](examples/compatibility/ids/id.go)。CLI 校验声明并生成代码,不执行项目 Go 代码;启动时校验方法是否齐全,缺失会明确失败。
输入可选 `string`、`bool`、`int32`、`int64`、`float64`,用 `|` 声明多个允许的 JSON 类型;输出只声明一个。框架先校验 JSON 类型和数字范围,再调用项目转换方法。字符串是否是合法 ID、是否允许空串或负数由项目方法决定,不能把数字溢出静默截断。path/query/header 实际传输文本,OpenAPI 对这些位置描述 string;query 切片通过重复键传递。
普通字段不受注册影响,普通 `int64` 仍拒绝 JSON 字符串。没有全局注册表,也不会自动给所有 ID 放宽规则。标量可嵌套、放入切片/map,`*BusinessID` 允许 null;缺失默认必填,`optional`/`omitempty` 可选。标量的额外约束放在转换器中,目前不叠加 `binding`、default/options/range 等标签。
请求 OpenAPI 按 `string | int64` 生成 `anyOf`,响应按 `string` 生成,避免把输入兼容误描述为输出也有两种类型。输出方法必须遵守声明,框架不会每次重新验证响应;项目需为自定义类型测试边界行为。
不使用 `.api` 也可显式注册:
```go
binder := contract.MustNew[CreateReq](
contract.ScalarOf[ids.ID]("string|int64", "string", true),
)
// binder.Bind(c.Request(), httpx.RouteParams(c.Context()), &req)
```
## 可运行示例与验收
```sh
go run ./examples/compatibility
curl -i http://127.0.0.1:8080/legacy
curl -i -X POST http://127.0.0.1:8080/echo \
-H 'Content-Type: application/json' -d '{"id":9223372036854775807,"data":{"hello":true}}'
curl -i -F file=@README.md http://127.0.0.1:8080/upload
curl -OJ http://127.0.0.1:8080/download
```
完整契约:[compatibility.api](examples/contracts/compatibility.api)。生成项目起初的手写 hook 仍为 501;真实实现与原文 HMAC 验签回归见 [消费者 fixture](cmd/go-omni/apigen/testdata/compatibility)。
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./cmd/go-omni/apigen -run Compatibility
go test -race ./httpx ./contract
```
这些是框架验收,不代表任何业务项目已经迁移完毕;业务的路由覆盖、响应兼容、权限、资金事务和数据库行为需要业务自己的回归。

View File

@@ -2,7 +2,7 @@
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
这批能力位于 main,尚未包含在 `v0.6.0-preview` 标签中。统一示例见
这批能力从 `v0.7.0-preview` 提供。统一示例见
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。

550
README.md
View File

@@ -1,302 +1,400 @@
# GoOmni
GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用只依赖标准库的网络适配器,业务代码只依赖 GoOmni 的 transport-neutral API,后续可以替换为自研网络层。
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Service 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
**从这份 README 开始即可接入。** 当前版本为 **`v0.9.1-preview`**,Go **1.22+**;CLI 和项目依赖使用同一固定版本。框架不依赖 Gin,默认使用标准库网络层,提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置,可以替换。
## 选择接入方式
| 你的项目 | 从哪里开始 |
| --- | --- |
| 新建服务,先定义接口再开发 | [新项目:生成并运行](#新项目生成并运行) |
| 已有服务,逐步接入 HTTP 或基础模块 | [已有项目:按模块接入](#已有项目按模块接入) |
| 已有标准 Handler、动态 JSON、上传或自定义 ID 类型 | [迁移扩展指南](MIGRATION.md) |
| 想统一返回体、业务错误或三方回调 | [配置公共响应与独立协议](#配置公共响应与独立协议) |
| 要接日志、数据库、鉴权和存储 | [业务开发与依赖装配](#业务开发与依赖装配) |
| 要切换网络层、配置 TLS 和退出清理 | [配置与运行](#配置与运行) |
| 已接入,需要升级和检查生成代码 | [日常开发与升级](#日常开发与升级) |
## 准备环境与安装
module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可用,并具有仓库读取权限;使用自己的 Git 凭据管理器,不把账号或 Token 写进 module 地址。
以下命令在同一个终端执行。已有 `GOPRIVATE` 时,把 `git.yuntuo.hk` 追加到原来的逗号分隔列表。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
export GOPRIVATE=git.yuntuo.hk
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.9.1-preview
export PATH="$(go env GOPATH)/bin:$PATH"
go-omni --help
```
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
如果配置了自定义 `GOBIN`,将那个目录加入 PATH。只引用框架包、不生成项目时可以跳过 CLI 安装,直接使用下文的 `go get`。
创建新业务项目:
## 新项目:生成并运行
在准备存放业务项目的目录执行,`./hello` 必须是新目录或空目录;把 `example.com/hello` 换成自己的 Go module 名称。
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
go-omni new --module example.com/billing ./billing
cd billing
go-omni api new --framework-version v0.9.1-preview \
--module example.com/hello ./hello
cd hello
go mod tidy
go run ./cmd/server
```
远程安装的 CLI 默认引用自身版本,生成的项目不需要本地 `replace`。
具体版本和本地开发选项见 [CLI 文档](cmd/go-omni/README.md)。
另开终端验证:
需要数据库持久化时,使用 `go-omni new --database mysql --module example.com/billing ./billing`。
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
## 目录结构
```text
go-omni/
├── app/ # 应用启动和生命周期
├── httpx/ # 与网络框架无关的 HTTP 抽象
├── response/ # 统一响应
├── errors/ # 统一错误
├── health/ # 存活和就绪检查
├── middleware/ # 请求 ID 等基础中间件
├── logx/ # 业务层结构化日志
├── metrics/ # 请求指标与状态统计
├── config/ # 配置管理
├── validate/ # 参数校验
├── auth/ # 认证和权限接口
├── database/ # database/sql 生命周期和事务
├── cache/ # 有界内存缓存、Redis 适配与合并回源
├── httpc/ # 出站 HTTP 连接池、超时和重试
├── task/ # 有界后台任务与间隔调度
├── queue/ # 持久队列接口与 Redis Streams 适配
├── storage/ # Local、S3/R2、OSS 与临时下载链接
├── openapi/ # 路由和 OpenAPI 3.0.3 契约
├── telemetry/ # 可注入 OpenTelemetry 观测
├── resilience/ # 熔断与上游并发隔离
├── audit/ # 显式业务审计与可注入 Sink
├── pagination/ # 有界分页和统一结果
├── transport/nethttp/ # 标准库网络适配器
├── transport/native/ # 自研 HTTP/1.1 服务端预览
├── transport/http2/ # opt-in HTTP/2 实验适配器
├── examples/ # 示例项目
└── cmd/ # go-omni 项目脚手架
```sh
curl -i http://127.0.0.1:8080/api/greet/Alice
curl -i http://127.0.0.1:8080/health/ready
curl -sS http://127.0.0.1:8080/openapi.json
```
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
可以直接运行 [组合示例](examples/modules/README.md)。这些新增能力尚未包含在上面的历史版本标签中。
第一个接口返回 HTTP 200,业务数据为 `{"message":"Hello Alice"}`。初始模板未配置响应格式,使用默认公共体:
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
```json
{"code":"OK","message":"success","data":{"message":"Hello Alice"},"requestId":"本次请求编号"}
```
## 快速开始
按 Ctrl+C 停止服务。项目依赖已固定为远程版本,不需要克隆框架源码或添加本地 `replace`。
### 生成的项目放什么
```text
hello/
├── api/service.api # 维护接口定义,也可 import 其他 .api
├── cmd/server/main.go # 启动、路由注册、优雅退出
├── etc/config.yaml # 项目配置
├── internal/
│ ├── settings/ # 配置读取
│ ├── servicecontext/ # 数据库、日志、外部客户端等依赖
│ ├── types/typesGen.go # 生成的请求与业务返回类型
│ ├── handler/ # 生成的绑定、响应策略、路由及文档
│ ├── service/greet/ # 在这里实现业务逻辑
│ ├── repository/greet/ # 在这里实现数据访问
│ └── middleware/ # 声明鉴权等中间件后生成对应 hook
├── docs/openapi.json # 生成的 OpenAPI 3.0.3
└── .goOmni/apiManifest.json # 生成文件所有权与校验信息,一起提交 Git
```
修改 `api/service.api` 后重新生成,业务写在 `internal/service`,数据访问写在 `internal/repository`,依赖在 `internal/servicecontext` 装配。声明业务错误后还会生成 `internal/apierrors/errorsGen.go`。
`api new` 的问候接口可以直接运行;自定义接口新生成的 Service 和鉴权 hook 返回 **501**,需要实现后才可用。生成器不会根据 HTTP 字段猜数据库表或业务规则。
### 从自己的接口定义生成
保存下面的完整示例为 `catalog.api`,再执行后面的生成命令。这里的响应格式、错误码和提示都由这个业务项目定义。
```text
syntax = "v1"
response CommonResponse (
body: {"code":"${code}", "msg":"${message}", "data":"${data}"}
codeType: int32
successCode: 200
errorCode: 9999
successMessage: "操作成功"
dataMode: always
codes: {"ITEM_NOT_FOUND":404001}
)
error ItemNotFound 404 "ITEM_NOT_FOUND" "记录不存在"
type GetItemReq {
ItemID string `path:"itemId" binding:"required"`
}
type ItemResp {
ItemID string `json:"itemId"`
Name string `json:"name"`
}
@server (
group: item
prefix: /api/v1
response: CommonResponse
)
service catalog-api {
@doc "查询商品"
@errors ItemNotFound
@handler getItem
get /items/:itemId (GetItemReq) returns (ItemResp)
}
```
```sh
go-omni api validate --api catalog.api
go-omni new --api catalog.api --framework-version v0.9.1-preview \
--module example.com/catalog ./catalog
cd catalog
go mod tidy
```
将生成的 `internal/service/item/getItem.go` 替换为下面的演示实现。实际业务中换成 Repository 查询,Service 返回业务数据或错误,不再手工包装公共体:
```go
package item
import (
"context"
"example.com/catalog/internal/apierrors"
"example.com/catalog/internal/types"
)
func (s *Service) GetItem(ctx context.Context, req *types.GetItemReq) (*types.ItemResp, error) {
if req.ItemID != "1001" {
return nil, apierrors.ItemNotFound()
}
return &types.ItemResp{ItemID: req.ItemID, Name: "示例商品"}, nil
}
```
示例 import 对应上面的 `example.com/catalog`,使用其他 module 名称时同步替换。启动后,在另一个终端验证:
```sh
# catalog 项目目录中启动;先停掉前面的 hello 服务,释放 8080 端口。
go run ./cmd/server
# 另一个终端:
curl -i http://127.0.0.1:8080/api/v1/items/1001
curl -i http://127.0.0.1:8080/api/v1/items/missing
```
成功时得到 `{"code":200,"msg":"操作成功","data":{"itemId":"1001","name":"示例商品"}}`;不存在时为 HTTP 404、`{"code":404001,"msg":"记录不存在","data":null}`。
需要 MySQL、文件存储或文件日志时,**首次生成**可以使用:
```sh
go-omni new --api catalog.api --framework-version v0.9.1-preview \
--module example.com/catalogFull --database mysql --storage local \
--log-output file --log-format json ./catalogFull
```
该命令在 `catalog.api` 所在目录执行,目标仍需是新目录。数据库迁移、表结构和 Repository 由项目维护,API 模板不自动建表;MySQL DSN 通过 `GO_OMNI_DB_DSN` 提供。存储还可选 `s3`、`r2`、`oss`,按生成项目的配置说明提供桶、端点和凭据。完整装配例子见 [userService](examples/userService/README.md)。
不采用 `.api` 时,也可以用 `go-omni new --framework-version v0.9.1-preview --module example.com/billing ./billing` 创建手写接口的 CRUD 示例项目;两种生成方式选择一种即可。
## 已有项目:按模块接入
在业务项目的 **go.mod 所在目录**执行:
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.9.1-preview
```
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。
下面是可独立运行的 `main.go` 示例;迁入现有项目时,把路由调用接到自己的业务服务即可。
```go
package main
import omni "git.yuntuo.hk/k2safe/go-omni"
func main() {
app := omni.New()
app.GET("/health", func(ctx *omni.Context) error {
return ctx.OK(map[string]string{"status": "ok"})
})
app.Run(":8080")
}
```
成功响应默认使用统一结构:
```json
{
"code": "OK",
"message": "success",
"data": {"status": "ok"},
"requestId": "..."
}
```
业务错误可以直接返回:
```go
return omni.NotFound("USER_NOT_FOUND", "user was not found")
```
默认网络适配器位于 `transport/nethttp`。业务层不需要导入具体网络库;如果后续实现自己的网络层,只需要实现 `httpx.Router` / `httpx.RouterGroup`,然后通过 `omni.NewWithRouter` 启动应用:
```go
application := omni.NewWithRouter(myRouter)
```
仓库还提供了使用标准库路由的 `transport/nethttp` 参考实现。两种适配器已通过同一套路由、业务处理链和取消传播测试,完整用户 CRUD 复用同一份 Handler / Service。
自研适配器可以直接运行 `transport/transporttest.Run` 验收。具体规则、v0.4.0 的斜杠 / Any / 405 行为调整,以及自研网络内核与当前 HTTP 接口的边界,见 [适配层契约](transport/README.md)。
`app.Engine()` 返回标准 `http.Handler`,`ctx.Raw()` 返回适配器提供的 `http.ResponseWriter`,业务 API 不锁定具体网络实现。
需要试用第一版自研网络层时,业务注册代码无需改变,只替换启动器:
```go
import (
"context"
"log"
"os"
"os/signal"
"syscall"
"log"
"net/http"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/transport/native"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
)
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer cancel()
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
app := omni.New()
app.GET("/health", func(ctx *omni.Context) error {
return ctx.OK(map[string]string{"status": "ok"})
})
if err := app.RunWith(ctx, native.New(nil), ":8080"); err != nil {
log.Fatal(err)
func run() error {
closeLog, err := logx.Setup(logx.Config{
Service: "catalog", Level: "info", Format: "json", Output: "stdout",
})
if err != nil {
return err
}
defer closeLog()
app := omni.New(
omni.WithLogger(true),
omni.WithResponse(response.Config{
Body: map[string]any{
"code": "${code}", "msg": "${message}", "data": "${data}",
},
CodeType: "int32", SuccessCode: 200, ErrorCode: 9999,
SuccessMessage: "操作成功", DataMode: "always",
Codes: map[string]any{"ITEM_NOT_FOUND": 404001},
}),
)
app.GET("/api/v1/items/:itemId", func(c *omni.Context) error {
itemID := c.Param("itemId")
logx.InfoContext(c.Context(), "item.get", "itemId", itemID)
if itemID != "1001" {
return omni.NotFound("ITEM_NOT_FOUND", "记录不存在")
}
return c.OK(map[string]string{"itemId": itemID, "name": "示例商品"})
})
// 第三方协议可明确输出文本,不经过公共 JSON 包装。
app.GET("/ping", func(c *omni.Context) error {
return c.Text(http.StatusOK, "pong")
})
return app.Run("127.0.0.1:8080")
}
```
`transport/native` 当前支持 HTTP/1.1、Keep-Alive、`Content-Length` 和 chunked 请求体、`http.Flusher` 流式响应、可选 TLS、读写超时和优雅关闭;不支持的传输编码会被拒绝。它会复用连接状态和响应对象,并可通过 `MaxConnections`、`MaxConcurrentRequests` 限制内存峰值。它仍通过标准 HTTP 请求/响应接口连接业务层,适合在现有 Handler / Service 上做网络层验证。
运行 `go run .`,访问 `/api/v1/items/1001`、`/api/v1/items/missing` 和 `/ping`,可分别验证成功、业务错误及纯文本输出。`App` 实现标准 `http.Handler`,也可挂进现有 `http.Server` 或父路由;这时监听、TLS、信号和资源关闭由现有入口负责,结束时调用 `app.Close()`。
native TLS 当前只支持并宣告 ALPN `http/1.1`;配置 `h2` 会在启动时快速失败。需要 HTTP/2 时,可由 Nginx/Cloudflare 终止后回源 HTTP/1.1,或使用默认的 `transport/nethttp` 适配器。
现有 `http.HandlerFunc` 可直接通过 `app.POST("/legacy", httpx.WrapHandler(http.HandlerFunc(oldHandler)))` 接入;状态码与访问日志一致,下载和流式输出保持原协议。标准中间件、动态 JSON、上传/原文 body、手写绑定和自定义类型的完整用法见 [迁移扩展指南](MIGRATION.md)。
`transport/http2` 是独立的 opt-in 实验适配器,使用 `x/net/http2` 处理帧和流控,同时保留 GoOmni 的连接上限、全局请求并发闸门和生命周期边界。它不会改变 native 默认路径;实验入口、限制和验收结果见 [HTTP/2 进入条件](transport/native/HTTP2.md)。
接入现有业务时,逐项确认:
自研网络层还提供 `server.Stats()`,可读取 accepted、rejected、active connections 和 active requests,与上面的业务请求 metrics 组合后能区分连接堆积、过载拒绝和 Handler 慢请求。`rejectedConnections` 表示连接闸门拒绝数,`rejectedRequests` 表示并发请求闸门返回 503 的次数。
- **协议**:保留既有 URL、HTTP 状态、字段、分页和空值语义。已有业务错误类型需要在项目内转换为 Omni 错误,或使用 `WithResponseEncoder` 适配原错误;设置字段模板不会自动识别其他库的错误。
- **鉴权**:把现有登录态和权限校验接入中间件,继续使用原来的授权范围和业务规则。原来依赖 `gin.Context` 等类型的 HTTP 代码需要适配;标准 `context.Context` 的业务服务可以继续复用。
- **数据类型**:ID、金额、日期等按项目协议转换。用 `.api` 的 `scalar` 显式关联项目自己的类型与 JSON/Text 转换方法,分别声明输入和输出协议;普通字段保持严格绑定,见 [自定义标量](MIGRATION.md#4-自定义标量项目定义行为框架连接调用)。
- **生成器**:它不会自动接管任意已有 Go 项目。希望引入 `.api` 时,先在独立新目录生成对照,再规划路由与生成目录的接入,不要用脚手架覆盖现有业务目录。
业务代码可以直接使用 `git.yuntuo.hk/k2safe/go-omni/logx` 打结构化日志,不需要依赖具体网络库:
先通过一个接口的成功、参数错误、未登录、无权限及业务错误回归,再按模块迁移。框架不要求所有项目使用同一套业务目录或身份模型。
```go
func (s *UserService) Create(ctx *httpx.Context) error {
logx.InfoContext(ctx.Context(), "user.create", "email", input.Email)
return nil
}
```
## 配置公共响应与独立协议
需要基础运行指标时,可以注册 transport-neutral metrics middleware 和接口:
```go
requestMetrics := omni.NewMetrics()
app.Use(requestMetrics.Middleware())
app.GET("/metrics", requestMetrics.Handler())
```
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
```go
settings, err := omni.LoadConfig("")
if err != nil {
log.Fatal(err)
}
logger, err := logx.New(settings.Log)
if err != nil {
log.Fatal(err)
}
application := omni.New(omni.WithConfig(settings))
application.OnShutdown(logger.Close)
```
常用变量包括 `GO_OMNI_ENABLE_LOGGER`、`GO_OMNI_LOG_LEVEL`、
`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_OUTPUT`、`GO_OMNI_SERVICE`、
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR`、`GO_OMNI_SHUTDOWN_TIMEOUT` 和
`GO_OMNI_TRUSTED_PROXIES`。可信代理变量使用逗号分隔的 IP/CIDR,例如
`GO_OMNI_TRUSTED_PROXIES=127.0.0.1,10.0.0.0/8`;只有来自这些地址的连接才会解析
`X-Forwarded-For`,默认只记录 socket 对端地址。数据库连接仍由
`database.ConfigFromEnv` 从 `GO_OMNI_DB_*` 加载。
网络层也由同一份配置选择,Handler / Service / Repository 不需要导入具体适配器:
| 配置 | 默认值 / 用途 |
| 接口来源或需求 | 配置位置 |
| --- | --- |
| `GO_OMNI_TRANSPORT` | `nethttp`;可选 `native`、`http2` |
| `GO_OMNI_TLS_CERT_FILE` | TLS 证书 PEM 文件路径 |
| `GO_OMNI_TLS_KEY_FILE` | TLS 私钥 PEM 文件路径,必须与证书一起配置 |
| `GO_OMNI_SHUTDOWN_TIMEOUT` | `10s`,三种适配器共用的请求排空时间 |
| `.api` 生成的接口 | `response Name (...)` 定义,`@server (response: Name)` 引用 |
| 多个 `.api` 共用返回体 | 将声明放到 `common.api`,使用 `import "common.api"` |
| 某个生成接口用另一种 JSON 公共体 | 路由声明 `@response OtherResponse` |
| 只要业务 JSON、无成功包装 | 路由声明 `@response raw`;失败仍沿用组级格式 |
| 手写路由和应用级 404/405 | `omni.WithResponse(response.Config{...})`,或读取配置后传入 `WithConfig` |
| 成功和失败都要遵循特殊回调协议 | `HandleWithResponse` 绑定项目自己的 `response.Encoder` |
| 单次直接输出 JSON、文本、XML、字节或空响应 | `c.JSON` / `c.Text` / `c.XML` / `c.Bytes` / `c.NoContent` |
例如运行自研 HTTP/1.1,或显式试用独立 HTTP/2 适配器:
`${data}` 是业务返回值,`${message}` 是公开提示,`${code}` 是业务码;占位符保留原值类型,不是字符串拼接。还支持 `${requestId}`、`${ok}`、`${status}`,字段名和嵌套结构由项目决定。
```sh
GO_OMNI_TRANSPORT=native go run ./examples/reference
GO_OMNI_TRANSPORT=http2 GO_OMNI_HTTP_ADDR=127.0.0.1:8443 \
GO_OMNI_TLS_CERT_FILE=./server.crt GO_OMNI_TLS_KEY_FILE=./server.key \
go run ./examples/reference
```
- `codeType` 可以是 `string`、`int32`、`int64`;整数类型需要配置 `successCode` 和 `errorCode`。
- `codes` 把逻辑错误码映射成对外业务码;未单独映射的失败使用 `errorCode`。JSON 业务码不会改变 HTTP 状态,例如 `404001` 仍对应 HTTP 404。
- `successMessage` 设置成功提示;未设置 `errorMessage` 时沿用错误的公开提示。`messages` 可按逻辑错误码单独覆盖,优先级最高。全局 `errorMessage` 会替换未单独映射的所有失败提示,只有需要统一失败文案时才设置。
- `dataMode: always` 在无数据时输出 null;默认 `omitNil` 省略模板里相应对象字段。没有 `returns` 的 `.api` 路由默认 HTTP 204、无响应体,这与 `200 + data:null` 是不同的接口契约。
`http2` 必须配置 TLS;native TLS 只协商 `http/1.1`,nethttp TLS 支持标准库的 HTTP/1.1 和 HTTP/2。
未知适配器、缺少配对证书、无法加载证书时,`Run` / `RunContext` 在监听前返回错误并执行资源清理,不会退回其他协议。
HTTP/2 仍为实验适配器,尚未合并到 native 多协议监听器。
**生成接口的格式以 `.api` 为准**,修改后重新生成,运行时与 OpenAPI 一起更新;应用 YAML 不会覆盖它。404/405 没有匹配到某个 `.api` 接口,使用应用默认配置,需要统一时也配置应用默认值。
代码配置可写成 `omni.New(omni.WithTransport(omni.TransportNative), omni.WithTLS(certFile, keyFile))`。
`ServeContext(ctx, listener)` 使用相同配置并接管已有 listener;失败或退出时也会关闭它。
`RunWith(ctx, runner)` 保留自定义网络层入口,由 runner 自己管理 TLS、超时和网络参数;连接/请求上限等适配器专属选项仍通过这个入口设置。
`NewServer` 仅创建标准库明文服务器,配置选择应用于 `Run`、`RunContext` 和 `ServeContext`。
三种适配器的配置、真实 TLS 业务和关闭行为已通过 Linux 全量 race 及 10 次重复回归,见 [统一启动验收](performance/transportStartup.md)。
文本/XML 回调用 Go 编码器或直接输出,`.api` 可用 `@handlerMode manual` 与 `@produces` 声明成功媒体类型;特殊错误协议仍需项目维护。`@response raw` 仅表示原始 JSON。前面示例的 `c.Text` 控制这次正常返回;需要连鉴权失败、业务错误和 panic 都按回调协议响应时使用 `HandleWithResponse`。完整配置和回调示例见 [响应参考](RESPONSE.md)。
`App` 同时实现标准 `http.Handler`。`Run` 监听 `SIGINT` / `SIGTERM`;`RunContext` 则由调用方的 context 控制退出,生成项目已使用 `signal.NotifyContext` 接好信号。
退出时先停止接收新请求,等待活跃请求完成,再执行资源清理。超过 `ShutdownTimeout` 时强制关闭连接、取消请求 context,并返回超时错误。业务代码应响应 context 取消;框架无法强制结束忽略取消的业务 goroutine。数据库连接池由 `database.Open` 创建,在服务退出时调用 `Close`。
## 业务开发与依赖装配
应用资源可以直接挂到 `omni.New` 返回的生命周期上:
### 日志
业务包直接使用 `logx`,不需要 HTTP 框架对象:
```go
application := omni.New()
application.OnShutdown(db.Close)
// ctx 是 Service 接收的标准 context.Context。
logx.InfoContext(ctx, "order.created", "orderId", orderID)
logx.ErrorContext(ctx, "order.failed", "error", err)
```
清理函数会按注册顺序的逆序执行,`application.Close()` 和 `RunContext` 都会触发,且重复调用安全。
入口用 `logx.Setup` 初始化,退出时关闭;生成的 API 项目已经完成日志初始化和退出清理。Service、Repository 和外部请求持续传递请求的 `context.Context`,业务日志可关联 `requestId`,同时保留取消信号。框架不规定或自动注入 `userId`、`tenantId`。
服务可以把数据库、缓存等依赖注册为就绪探针:
`WithLogger(true)` 开启 HTTP 访问日志。格式、级别、文件轮转、脱敏键和慢请求阈值均可配置,见 [日志](logx/README.md)。
```go
checker, err := health.New(health.Probe{
Name: "database",
Check: db.PingContext,
})
if err != nil {
log.Fatal(err)
}
application.GET("/health/live", checker.LivenessHandler())
application.GET("/health/ready", checker.ReadinessHandler())
### 鉴权与业务错误
`.api` 可声明认证方式和关联中间件,例如把下面的声明放入契约,并在相应 `@server` 块中加上 `auth: UserAuth`:
```text
auth UserAuth (
type: bearer
middleware: Auth
)
```
存活检查只确认进程在线;就绪检查会执行全部探针,任意依赖失败时返回 HTTP 503。
生成器会创建 `internal/middleware/auth.go`,由项目实现凭据校验和身份注入。声明会生成路由绑定与 OpenAPI 的认证描述,**不会自动实现登录或授权规则**;未实现的 hook 返回 501。手写路由可使用 `auth.RequireAuth(authenticator)` 或自己的中间件。JWT、Session 和 Policy 按需选择。
新项目脚手架会生成项目自己的 `internal/servicecontext`。它集中保存 Logger、数据库、缓存和外部客户端,Service 只依赖这个项目上下文,不把基础设施初始化散落到各个 Handler:
业务错误在 `.api` 中用 `error` 声明,通过 `@errors` 关联路由后,生成可调用的错误函数和 OpenAPI 中对应的 401、403、409 等响应。手写路由可以直接返回 `omni.NotFound`、`omni.Forbidden` 等错误。已有项目在自己的适配层转换原错误,内部原因和公开提示分开处理。
```go
services, err := servicecontext.New(ctx, logger)
if err != nil {
return err
}
defer services.Close()
userService := service.NewUsers(services)
```
### 数据库、存储及其他模块
## 完整业务参考项目
依赖在启动入口或项目自己的 `internal/servicecontext` 中创建,通过构造函数传给 Service/Repository,避免每个请求都新建连接池。只注册自己拥有的资源,例如 `app.OnShutdown(db.Close)`;关闭按注册的逆序执行。
[`examples/reference`](examples/reference/README.md) 提供按 Handler / Service / Repository / Model 分层的用户 CRUD 示例。默认使用并发安全的内存存储,支持分页、邮箱唯一校验、结构化请求日志和统一错误,无需配置数据库。
已有项目使用 `driver.Connector` 时,直接 `database.OpenConnector(ctx, connector, database.DefaultPoolConfig())`。MySQL 等驱动的独立 TLS、Logger、时区与拨号参数由项目构造 Connector;框架负责连接池、启动 Ping、健康检查及事务。无需转成 DSN,也无需注册全局 TLS/Logger。具体示例和资源归属见 [Connector 接入](database/README.md#connector-接入v091-preview)。
从远程版本生成并升级一个独立业务模块,可运行:
| 能力 | 包与接入说明 |
| --- | --- |
| 数据库连接池、事务、显式迁移 | `database`,见 [数据库](database/README.md);表结构与驱动由项目选择 |
| 本地文件、S3、R2、OSS、直传与临时下载 | `storage`,见 [文件存储](storage/README.md);业务依赖接口,装配时选择适配器 |
| 缓存、出站 HTTP、限流、幂等、后台任务 | `cache`、`httpc`、`middleware`、`task`,见 [基础模块](MODULES.md) |
| JWT/Session、策略、Redis 队列、审计、分页、观测 | 见 [扩展模块](ADVANCED.md);按需注入实现 |
| 存活与就绪检查 | `health`,见 [健康检查](health/README.md);数据库等依赖作为 readiness 探针 |
API 项目的数据库迁移需要单独维护并显式执行;启动不会自动改表。完整示例 [userService](examples/userService/README.md) 展示 MySQL、事务、鉴权、日志和文件导出,可参考依赖装配方式后接入自己的业务规则。
## 配置与运行
生成项目从 `etc/config.yaml` 读取配置,优先级为 **默认值 → 配置文件 → 环境变量**。密钥通过环境变量提供。手写入口需显式调用配置加载函数并传给 `omni.New(omni.WithConfig(settings))`;单独调用 `omni.New()` 不会自动读取环境变量。
| 环境变量 | 用途 |
| --- | --- |
| `GO_OMNI_HTTP_ADDR` | 监听地址;生成项目默认 `127.0.0.1:8080` |
| `GO_OMNI_TRANSPORT` | `nethttp`(默认)、`native`、`http2` |
| `GO_OMNI_TLS_CERT_FILE` / `GO_OMNI_TLS_KEY_FILE` | 成对设置 TLS 证书和私钥路径 |
| `GO_OMNI_HTTP_READ_HEADER_TIMEOUT` / `GO_OMNI_HTTP_READ_TIMEOUT` | 请求头 / 请求读取超时,如 `5s` / `30s` |
| `GO_OMNI_HTTP_WRITE_TIMEOUT` / `GO_OMNI_HTTP_IDLE_TIMEOUT` | 响应写入 / Keep-Alive 空闲超时 |
| `GO_OMNI_HTTP_MAX_HEADER_BYTES` | 请求头大小限制,字节数 |
| `GO_OMNI_SHUTDOWN_TIMEOUT` | 优雅退出等待时间,默认 `10s` |
| `GO_OMNI_ENABLE_LOGGER` / `GO_OMNI_LOG_LEVEL` | 访问日志开关、日志级别 |
| `GO_OMNI_TRUSTED_PROXIES` | 可解析转发地址头的可信代理 IP/CIDR,逗号分隔 |
| `GO_OMNI_DB_DSN` | 选择 MySQL 初始化时的数据库连接配置 |
例如在生成项目中切换 native:
```sh
python3 scripts/verifyBusiness.py --version v0.6.0-preview
GO_OMNI_TRANSPORT=native go run ./cmd/server
```
该验收会在临时目录中先固定 `v0.2.0`,再升级到 `v0.6.0-preview`,检查无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
手写入口可用 `omni.WithTransport(omni.TransportNative)`;TLS 配置为 `omni.WithTLS(certFile, keyFile)`。native 提供 HTTP/1.1、Keep-Alive、TLS、超时、连接/请求并发上限和对象复用;HTTP/2 是单独的实验适配器,必须配置 TLS。默认 nethttp 的 TLS 支持 HTTP/1.1 与 HTTP/2。
标准库与 native 的并发、延迟和长连接内存对照见 [PERFORMANCE.md](PERFORMANCE.md)。`python3 scripts/verifyPerformance.py` 运行检查与 benchmark;增加 `--load --output /tmp/goOmniPerf.json` 可测量独立服务进程并保存报告。
读写超时也可以在生成项目的 `etc/config.yaml` 中声明(放在 `server.http` 下):
HTTP/2 仍是显式 opt-in 实验适配器,不会改变 native 的 HTTP/1.1 默认路径。Linux TLS 长测使用 `python3 scripts/verifyHTTP2.py --duration 30m`,默认使用普通 Go 运行模式;`--race` 默认改为 60 秒,用于并发检查,其额外内存开销不能当成生产运行基线。
```yaml
server:
http:
readHeaderTimeout: 5s
readTimeout: 30s
writeTimeout: 30s
idleTimeout: 60s
maxHeaderBytes: 1048576
```
多连接混合场景使用 `python3 scripts/verifyHTTP2.py --scenario mixed --duration 30m`:8 条独立 TLS 连接,每条连接 14 个普通请求 worker、1 个慢上传 worker、1 个慢下载 worker,共 128 个 worker。上传为 256 KiB、每 4 KiB 等待 2 ms;下载为 8 MiB、每次最多读取 64 KiB 后等待 20 ms。日志分别记录三类成功数、传输字节数、每条连接的覆盖情况,以及停止并释放测试数据后两次 GC 的采样。
代码配置使用 `omni.WithHTTPConfig(omni.HTTPConfig{ReadTimeout: 30*time.Second, WriteTimeout: 30*time.Second})`。`Run`、`RunContext`、`ServeContext` 会应用它;`NewServer` 会配置返回的标准 `http.Server`,包含可选的 Go 字段 `ErrorLog`。**正值覆盖、零值保留所选网络层的原有默认行为、负值报错**;不配置时升级行为不变。超时含义与支持范围见 [HTTP 配置](transport/README.md#http-配置v091-preview),不要把网络 I/O 超时当成业务执行时限。
该脚本默认把测试容器限制为 2 核、2 GiB 内存(禁用 swap)、256 个进程/线程,整个命令最多运行 40 分钟,中断和超时会清理本次容器。这些限制可用 `--cpus`、`--memory-mb`、`--pids-limit`、`--wall-timeout` 调整;共享服务器应先用 `--duration 60s` 短测,再逐步延长。每 15 秒输出的堆、RSS 和 FD 统计包含同一进程内的客户端与服务端。
自研网络层通过 `RunWith(ctx, runner)` 接入监听与连接生命周期,通过 `NewWithRouter(router)` 替换路由;业务 Handler/Service 不需要导入具体适配器。细节与验收契约见 [网络适配层](transport/README.md)。
Linux amd64 已完成 30 分钟、单 TLS 连接、128 个 worker 的普通模式长测:58,102,476 次请求,0 失败、0 协议错误,采样 RSS 峰值约 22.6 MiB,连接/活动请求归零且 FD 回到基线。完整采样和原始日志见 [HTTP/2 长测报告](performance/http2SoakLinux.json) 与 [原始日志](performance/http2SoakLinux.txt),此前的 race 对照见 [短测记录](performance/http2SmokeLinux.json)。这项结果覆盖单连接复用。
`Run` 处理 SIGINT/SIGTERM;`RunContext` 接收调用方控制的退出信号。服务先停止接收新请求,等待活跃请求完成,再清理资源;超时会取消请求并关闭连接,业务代码仍需响应 context 取消。
多连接慢客户端场景也已完成 Linux amd64 的 30 分钟长测:8 条 TLS 连接、128 个 worker,53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,共 53,191,687 次,0 失败与协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端、服务端和下载缓冲);退出后连接/请求归零,goroutine 回到 2、FD 回到 6;释放测试数据并两次 GC 后存活堆约 450 KiB,RSS 不要求立即归还操作系统。完整 120 份采样见 [混合长测报告](performance/http2MixedSoakLinux.json) 和 [原始日志](performance/http2MixedSoakLinux.txt)。连接级窗口耗尽、WINDOW_UPDATE 恢复及零窗口写阻塞的关闭回收也已有独立测试;Nginx 的 HTTP/2 回源验收结果见下方,适配器保持实验状态。
## 日常开发与升级
`python3 scripts/verifyHTTP2Proxy.py --count 3` 验证真实 Nginx 双段 TLS/h2、回源连接复用、证书拒绝、超时/取消恢复与新旧实例的 GOAWAY 排空;增加 `--race` 可运行并发检查。Linux amd64、Nginx 1.30.5 的普通与 race 模式各 3 轮均已通过。编译和测试容器顺序运行,均限制为 2 核、2 GiB;测试容器禁用外部网络,全部请求在容器内完成。使用方法、报告及当前边界见 [HTTP/2 代理回源验收](transport/http2/PROXY.md)。
在框架根目录运行:
在已生成的 API 项目目录执行:
```sh
go run ./examples/reference
go-omni api validate
go-omni api inspect
go-omni api generate --dry-run
go-omni api generate
go-omni api generate --check
go test ./...
```
默认地址为 `http://127.0.0.1:8080`。示例文档提供完整的 curl 操作流程。数据只保存在内存中,重启后清空。
`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Service、Repository、中间件实现、main、配置和依赖装配属于项目,重复生成保留手写内容;契约改变后仍需自行调整业务签名并通过编译。将 `.api`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 可运行 `generate --check` 检查是否同步。
框架的 `App` 也实现了标准 `http.Handler`,可以直接用于 `http.Server` 和 `httptest.NewServer`;业务路由可通过 `*omni.RouterGroup` 注册,无需导入网络库。
升级时先安装目标版本的 CLI,再在业务 module 中 `go get` **同一版本**;API 项目随后执行上述生成与测试步骤,手写项目只需依赖升级及业务回归。`v0.9.1-preview` 的变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
## 当前边界
## 按需深入
缓存、消息队列和定时任务暂时作为后续可选模块。认证层只定义 `Authenticator`、`Principal` 和路由保护中间件,具体 JWT、Session 或 OAuth 由项目实现。核心数据库层只依赖 `database/sql`,具体驱动、迁移和表结构由项目选择。
| 需要了解 | 文档 |
| --- | --- |
| `.api` 类型、参数绑定、错误、鉴权和生成规则 | [API 开发指南](API.md) |
| 公共响应体、提示优先级、独立回调协议 | [响应参考](RESPONSE.md) |
| CLI 完整参数、本地框架开发和模块脚手架 | [CLI](cmd/go-omni/README.md) |
| 完整 API 业务项目 | [userService](examples/userService/README.md) |
| 手写接口分层示例 | [reference](examples/reference/README.md) |
| 框架成熟度、历史变更 | [路线图](ROADMAP.md)、[更新记录](CHANGELOG.md) |
| 网络性能、测试条件与适用范围 | [性能说明](PERFORMANCE.md)、[HTTP/2 验收](transport/native/HTTP2.md) |
当前成熟度与后续验收标准见 [ROADMAP.md](ROADMAP.md),版本变更见 [CHANGELOG.md](CHANGELOG.md)。
当前版本可用于固定版本接入与业务验证,仍标记为 preview。`.api` 借鉴 go-zero 的接口开发方式,支持范围以本仓库文档为准,不是完整 goctl 语法兼容层;HTTP/2 保持实验状态,OSS/R2 云账号实网验收不属于本次发布范围。

45
RELEASE.md Normal file
View File

@@ -0,0 +1,45 @@
# v0.9.1-preview
本版补齐已有项目接入的两个框架缺口:标准数据库 Connector 注入,以及应用级 HTTP 读写超时配置。TLS、数据库驱动日志、字符集/时区、迁移与业务规则继续由项目决定。
## 安装与升级
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.9.1-preview
# 在各业务 module 中:
go get git.yuntuo.hk/k2safe/go-omni@v0.9.1-preview
```
私有仓库使用本机 Git 凭据管理器,并将 `git.yuntuo.hk` 追加到已有 `GOPRIVATE`。CLI 和项目依赖使用同一固定版本。v0.9.0-preview 生成代码继续兼容,本次无需为了这两个入口重新生成接口。
## 接入变化
- 将项目已有 `sql.OpenDB(connector)` 的池创建替换为 `database.OpenConnector(ctx, connector, poolConfig)`;不要同时保留两个池。返回对象可用于 Health/InTx;需要标准库指针时取 `.DB`。
- 项目原有 `mysql.NewConnector`、独立 TLS、driver Logger、驱动读写时限、MySQL 版本/时区/排序规则校验继续保留,配置不进入核心框架。
- 将原来逐项修改 `http.Server` 的读写/空闲/头大小配置放入 `omni.HTTPConfig`,通过 `WithHTTPConfig` 或 `WithConfig` 传入。Go-only `ErrorLog` 支持 nethttp。
- 原有 `database.Open(ctx, Config)`、手动组装 `http.Server`、自定义 Runner 均可继续使用。零 HTTP 配置保留所选网络层已有默认行为;正值覆盖,负值报错。
完整可复制示例:[数据库 Connector](database/README.md#connector-接入v091-preview)、[HTTP 配置及语义](transport/README.md#http-配置v091-preview)。其他迁移能力见 [迁移扩展指南](MIGRATION.md),从项目接入开始见 [README](README.md)。
## 边界
独立实验性 `TransportHTTP2` 本次支持 IdleTimeout/MaxHeaderBytes 的配置透传,但不支持读头、读请求和写响应超时,设置这些正值会在监听前明确报错。native 与标准库服务器的超时含义不完全相同,详见上表;网络超时不等于 Handler 执行时间限制。`RunWith` 自定义 Runner 的配置由项目管理。
Connector 必须遵守驱动 context 协议,资源所有权按标准 `database/sql` 管理。有资源的 Connector 不跨池共享;启动 Ping 失败关闭池,成功后应用负责关闭。没有把 eTax/ePay 的数据库策略写进框架,也未修改这些业务项目或宣称它们已完成迁移。
## 验证入口
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=1 -count=1 -timeout=300s ./...
go vet ./...
go build ./...
python3 scripts/verifyConnector.py --local --report performance/connectorLocal.json
python3 scripts/verifyConnector.py --version v0.9.1-preview --report performance/connectorRemote.json
python3 scripts/verifyCompatibilityRelease.py --version v0.9.1-preview --report performance/connectorCompatibilityRemote.json
```
Connector 脚本只使用本地独立 MySQL 8.4 容器,限制 1 CPU、768 MiB、128 PID,生成一次性 CA/证书及凭据,结束清理自己的容器/卷/临时文件。测试验证正确 TLS、错误 CA/主机名拒绝、每池时区/日志配置隔离、事务、关闭与失败回收。HTTP 测试使用回环地址验证慢头、慢 body、空闲连接、流式写背压及请求头上限。常规单测不依赖 Docker,显式脚本要求真实 TLS 测试通过,不能跳过。
历史发布及验收记录保留在 [CHANGELOG](CHANGELOG.md)、[v0.9.0 接入验收](performance/compatibilityAcceptance.md)。
全量 race(40 个测试包)、静态检查、构建及本地隔离验收已通过,详见 [验收记录](performance/connectorAcceptance.md)。

140
RESPONSE.md Normal file
View File

@@ -0,0 +1,140 @@
# 公共响应体与独立接口协议
这些功能从 `v0.8.0-preview` 提供。CLI 与项目框架依赖请固定到同一版本;源码开发可显式使用 `--framework-dir`。`v0.7.0-preview.1` 不包含响应模板声明。
go-omni 的响应编码由应用或接口配置。没有配置时保留旧版默认值;业务可以定义自己的字段、嵌套结构、错误码类型、提示和空值行为。`response.Response` 仅保留为兼容 DTO,运行时与生成器不再通过它固定响应结构。
## 在 .api 中定义一次、到处引用
```go
response CommonResponse (
body: {
"code": "${code}",
"msg": "${message}",
"data": "${data}"
}
codeType: int32
successCode: 200
errorCode: 9999
successMessage: "操作成功"
dataMode: always
codes: {"USER_NOT_FOUND":404001}
messages: {"USER_NOT_FOUND":"用户不存在"}
)
@server (
group: user
response: CommonResponse
)
service user-api {
@handler getUser
get /users/:userId (GetUserReq) returns (UserResp)
@response raw
@handler rawUser
get /rawUsers/:userId (GetUserReq) returns (UserResp)
}
```
请求/业务返回类型照常在 `type` 中定义。上例 `UserResp` 是业务数据,由模板中的 `${data}` 放入公共体。请求失败时,同一模板输出映射后的业务错误。错误本身的 HTTP 状态保持不变,业务 `code` 与 HTTP 状态相互独立。
- 公共声明可以放进 `common.api`,其他契约通过 `import "common.api"` 复用。
- `@server (response: CommonResponse)` 指定整个 service 块的公共体;不同块可以引用同一个模板。
- `@response PartnerResponse` 为单接口替换模板,成功和错误一起生效。
- `@response raw` 让成功 JSON 直接返回业务对象,错误沿用组级模板;组本身为 raw 时,错误使用默认模板。
- 模板名使用 UpperCamelCase;未知模板、未知配置、错误的占位符和冲突设置在生成前报错。
**生成接口以 `.api` 为响应契约来源。** 生成器把选定模板编译进 `internal/handler/responseGen.go`,在全局鉴权、限流等应用中间件之前选定,然后再执行路由中间件和业务。OpenAPI 与运行时共用配置和映射规则。即使 `.api` 没有声明模板,生成接口也明确使用默认契约,不会被应用的 YAML 设置静默改变。
业务只需返回业务数据或业务错误,无需每个 Handler 手工组装公共体。生成器维护模板、路由及文档文件,重复生成不覆盖手写 Service、Repository、middleware 和配置。
## 自定义结构与占位符
公共体没有规定必须出现的字段,也可以把成功和失败分开:
```go
response PartnerResponse (
body: {
"accepted": "${ok}",
"payload": "${data}",
"meta": {"trace":"${requestId}","httpStatus":"${status}"},
"version": 1
}
errorBody: {
"accepted": "${ok}",
"error": {"reason":"${message}","id":"${code}"}
}
)
```
`body` 是 JSON 对象,可嵌套对象、数组和常量;`errorBody` 可选,需要同时定义 `body`。空对象 `{}` 也是有效模板。
| 整值占位符 | 输出值 |
| --- | --- |
| `${data}` | 当前接口的业务返回值,保留对象/数组/数值等原类型 |
| `${code}` | 映射后的 string / int32 / int64 业务错误码 |
| `${message}` | 公开提示,经 message 配置映射 |
| `${requestId}` | 请求标识;为空时省略所在对象字段 |
| `${ok}` | 成功为 true;有错误或 HTTP 状态 ≥400 时为 false |
| `${status}` | HTTP 状态整数 |
占位符只替换整个值,不执行表达式或字符串拼接。字面量 `${data}` 写成 `$${data}`。数据为 nil 时,`dataMode: omitNil`(默认)省略所在对象字段;`always` 输出 null。数组中的占位符保留位置。模板最多嵌套 24 层。
`codeType` 使用与业务字段一致的类型名,默认 `string`:
| 类型 | 含义 | OpenAPI |
| --- | --- | --- |
| `string` | 字符串业务码,例如 `"OK"`、`"USER_NOT_FOUND"` | string |
| `int32` | 32 位有符号整数,−2147483648 到 2147483647 | integer / int32 |
| `int64` | 64 位有符号整数,−9223372036854775808 到 9223372036854775807 | integer / int64 |
整数类型必须提供 `successCode` 和 `errorCode`,支持 0 和负数;默认码和 `codes` 映射都会校验范围,越界或小数会报错。`int64` 的配置读取和生成代码保留完整整数精度,并按 JSON 数值输出;消费端也需要支持对应的整数精度。需要文本形式时可用 `string` 并配置字符串码。
旧写法 `integer` 保留兼容,沿用 ±(2^53−1) 的数值范围;新配置优先写 `int32` 或 `int64`。
`codes` 按逻辑错误码覆盖默认码;`messages` 优先于 `successMessage` / `errorMessage`。未配置的公开提示沿用原错误消息;内部 cause 不会被默认编码器序列化。
简单的平铺响应也可使用 `codeField` / `messageField` / `dataField` / `requestIdField` 快捷配置,`"-"` 禁用字段。空名称沿用旧默认值。这些字段名快捷配置不能和 `body` 混用,以免两个定义冲突。
## 手写路由、配置文件与三方回调
手写路由的应用默认配置可以放在 JSON/YAML 的 `response` 字段,也可用 Go 选项:
```go
app := omni.New(omni.WithResponse(response.Config{
Body: map[string]any{"code":"${code}", "msg":"${message}", "data":"${data}"},
CodeType: "int32", SuccessCode: 200, ErrorCode: 9999, DataMode: "always",
}))
```
应用之间的配置独立,模板在创建时验证并复制;没有全局可变的响应格式。应用配置快照也会复制模板和映射。响应配置属于静态配置,变更后需重启。
单次显式输出可使用 `c.JSON(status, value)`、`c.Text(status, text)`、`c.XML(status, value)`、`c.Bytes(status, contentType, bytes)` 或 `c.NoContent()`,都绕过公共体。已经提交的响应不会再追加错误 JSON。
三方回调可能要求成功与失败都返回 HTTP 200,并分别输出 `success` / `fail`。通过接口编码器统一处理成功、错误和 panic:
```go
ack := response.EncoderFunc(func(r response.Result) (response.Payload, error) {
if r.Err != nil || r.Status >= 400 {
return response.Text(200, "fail"), nil
}
return response.Text(200, "success"), nil
})
app.HandleWithResponse("POST", "/hooks/payment", ack, paymentHandler)
```
编码器也可以设置 HTTP 状态、Header、XML 或任意字节。`omni.WithResponseEncoder` 设置应用默认编码器;`httpx.WithResponse` 是普通路由中间件,按中间件顺序生效。需要覆盖全局中间件提前拒绝时,应使用 `HandleWithResponse`。自定义 Registrar 可以实现 `httpx.ResponseRegistrar` 提供相同的早期挂载能力;未实现时辅助函数退化为普通路由中间件。
`@response raw` 表示原始 JSON,不代表任意文本/XML协议。从 v0.9.0-preview 开始,`.api` 可用 `@handlerMode manual` 配合 `@produces` 描述文本/文件等成功输出,详见 [迁移扩展](MIGRATION.md)。自定义回调错误协议仍使用上述 Go 入口,并由应用维护相应文档;生成器不会猜测自定义 Go 编码器的输出。如果在命名中间件里替换编码器,业务也必须同步维护文档。
框架内置路由的 404/405 使用应用默认格式;它们不属于某个 `.api` 分组。TLS 握手、畸形 HTTP、连接级过载等发生在进入应用前,仍按网络层协议处理。自定义编码器失败、panic 或返回非法状态/204 body 时,最终返回空 500,避免递归编码或泄露另一种响应体。
## 运行示例与验证
```sh
go run ./cmd/go-omni api validate --api examples/contracts/responses.api
go run ./cmd/go-omni api generate --api examples/contracts/responses.api \
--module example.com/responseDemo --framework-dir . --dir ../responseDemo
```
示例业务方法起初为 501,完成 Service 后就会套用声明的模板。参考 [完整 .api](examples/contracts/responses.api)。验收测试会在临时目录生成真实项目,写入业务方法,执行已生成路由,检查数字错误码、原始 JSON、鉴权错误、路由覆盖与重复生成保护。

View File

@@ -88,4 +88,18 @@
- 已完成本地模块测试及 race 检查,Linux 全仓库 race/vet/build、生成项目 CRUD、三个 Runner 的 TLS 示例和独立 Redis/MinIO/MySQL 均通过。共享状态、重复提交、失败消息、未确认消息接管、旧确认拒绝、对象签名和迁移 dirty 阻断有对应证据,见 [扩展模块验收](performance/advancedAcceptance.md)。OSS/R2 尚无云账号实网验收。
- 契约、使用示例与限制见 [扩展模块](ADVANCED.md);验证入口为 `scripts/verifyModules.py --advanced`,继续限制单个测试容器 2 CPU / 2 GiB / 256 pids。
## 8. API 驱动项目与真实业务示例(v0.7.0-preview)
- `.api` → DTO / 绑定校验 / 路由 / Handler / Service / Repository / OpenAPI。
- MySQL、日志与五种存储选择可组合生成;manifest 管理生成文件,保留业务文件。
- 独立用户示例完成事务审计、鉴权、文件导出、数据库故障恢复与真实远程依赖验收。
- 证据:[API 业务闭环](performance/apiWorkflowAcceptance.md)。
## 9. 接口开发体验(v0.7.0-preview.1)
- 错误目录与调用函数、401/403/409 文档、具名鉴权和运行时 hook 关联。
- 组级继承/路由覆盖、多行接口描述、标签和弃用标记。
- 路由总览、独立 OpenAPI 导出、生成预览与项目入口自动识别。
- 证据:[接口开发体验验收](performance/apiDeveloperExperienceAcceptance.md)。
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,自研网络层已有可运行预览,稳定性与生产规模验证尚未完成。

View File

@@ -15,6 +15,7 @@ import (
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/middleware"
"git.yuntuo.hk/k2safe/go-omni/response"
netrouter "git.yuntuo.hk/k2safe/go-omni/transport/nethttp"
)
@@ -28,6 +29,7 @@ type App struct {
closeOnce sync.Once
closeErr error
clientIPResolver *httpx.ClientIPResolver
responseEncoder response.Encoder
}
// Runner is the small lifecycle boundary used by an HTTP server
@@ -64,17 +66,36 @@ func configure(options ...config.Option) config.Config {
}
func newApp(router httpx.Router, settings config.Config) *App {
settings.Response = settings.Response.Clone()
encoder, responseErr := response.New(settings.Response)
if responseErr != nil {
panic(responseErr)
}
var responseEncoder response.Encoder = encoder
if settings.ResponseEncoder != nil {
responseEncoder = settings.ResponseEncoder
}
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
if err != nil {
panic("go-omni: " + err.Error())
}
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
if settings.EnableLogger {
middlewares = append(middlewares, middleware.AccessLog())
middlewares = append(middlewares, middleware.AccessLog(middleware.WithSlowRequestThreshold(settings.SlowRequestThreshold)))
}
middlewares = append(middlewares, middleware.Recovery())
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
application := &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver, responseEncoder: responseEncoder}
if fallback, ok := router.(httpx.FallbackRouter); ok {
adaptError := func(status int, code, message string) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
application.adapt(func(*httpx.Context) error { return apierrors.New(status, code, message) }).ServeHTTP(w, r)
})
}
fallback.SetFallbacks(adaptError(404, "NOT_FOUND", "route not found"), adaptError(405, "METHOD_NOT_ALLOWED", "method not allowed"))
}
return application
}
// Router returns the transport contract used by this application.
@@ -92,7 +113,9 @@ func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
// Config returns the effective application configuration.
func (a *App) Config() config.Config {
settings := a.config
settings.Response = settings.Response.Clone()
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
return settings
}
@@ -133,16 +156,24 @@ func (a *App) Run(address ...string) error {
return a.RunContext(ctx, address...)
}
// NewServer creates a plain standard HTTP server around the application.
// NewServer creates a plain standard HTTP server with the configured HTTP options.
// Invalid HTTP options panic; RunContext/ServeContext return validation errors.
// Use RunContext or ServeContext to apply the configured transport and TLS.
func (a *App) NewServer(address string) *http.Server {
if err := a.config.HTTP.Validate(); err != nil {
panic(err)
}
if address == "" {
address = a.config.Address
if address == "" {
address = ":8080"
}
}
return &http.Server{Addr: address, Handler: a.router}
cfg := a.config.HTTP
return &http.Server{Addr: address, Handler: a.router,
ReadHeaderTimeout: cfg.ReadHeaderTimeout, ReadTimeout: cfg.ReadTimeout,
WriteTimeout: cfg.WriteTimeout, IdleTimeout: cfg.IdleTimeout,
MaxHeaderBytes: cfg.MaxHeaderBytes, ErrorLog: cfg.ErrorLog}
}
// RunContext serves until the context is canceled, then performs a graceful shutdown.
@@ -233,5 +264,5 @@ func (a *App) writeError(ctx *httpx.Context, err error) {
if err == nil || ctx.Responded() {
return
}
ctx.WriteError(apierrors.From(err))
ctx.WriteError(err)
}

32
app/body.go Normal file
View File

@@ -0,0 +1,32 @@
package app
import (
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/response"
"net/http"
)
func (a *App) bodyHandler(maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
checked := httpx.LimitBody(maxBytes)(handler)
if encoder == nil {
encoder = a.responseEncoder
}
next := a.adaptResponse(encoder, checked, middleware...)
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Body != nil {
r = r.WithContext(r.Context())
r.Body = http.MaxBytesReader(w, r.Body, maxBytes)
}
next.ServeHTTP(w, r)
})
}
// HandleWithBodyLimit caps the reader before application middleware and preserves
// the response policy, including auth failures and parameter errors.
func (a *App) HandleWithBodyLimit(method, path string, maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
a.router.Handle(method, path, a.bodyHandler(maxBytes, encoder, handler, middleware...))
}
func (g *RouterGroup) HandleWithBodyLimit(method, path string, maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
combined := append(append([]httpx.Middleware{}, g.middlewares...), middleware...)
g.group.Handle(method, path, g.app.bodyHandler(maxBytes, encoder, handler, combined...))
}

199
app/httpConfig_test.go Normal file
View File

@@ -0,0 +1,199 @@
package app_test
import (
"bufio"
"context"
"errors"
"io"
"log"
"net"
"net/http"
"strings"
"sync/atomic"
"testing"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
)
func TestHTTPConfiguredStandardServer(t *testing.T) {
cfg := omni.HTTPConfig{ReadHeaderTimeout: time.Second, ReadTimeout: 2 * time.Second, WriteTimeout: 3 * time.Second, IdleTimeout: 4 * time.Second, MaxHeaderBytes: 8192, ErrorLog: log.New(io.Discard, "", 0)}
a := omni.New(omni.WithHTTPConfig(cfg))
s := a.NewServer(":9000")
if s.ReadHeaderTimeout != cfg.ReadHeaderTimeout || s.ReadTimeout != cfg.ReadTimeout || s.WriteTimeout != cfg.WriteTimeout || s.IdleTimeout != cfg.IdleTimeout || s.MaxHeaderBytes != cfg.MaxHeaderBytes || s.ErrorLog != cfg.ErrorLog || s.Addr != ":9000" {
t.Fatalf("server options: %+v", s)
}
s = omni.New().NewServer("")
if s.ReadHeaderTimeout != 0 || s.ReadTimeout != 0 || s.WriteTimeout != 0 || s.IdleTimeout != 0 || s.MaxHeaderBytes != 0 || s.ErrorLog != nil {
t.Fatal("default behavior changed")
}
}
func TestHTTPInvalidStartupReleasesResources(t *testing.T) {
a := omni.New(omni.WithHTTPConfig(omni.HTTPConfig{ReadTimeout: -time.Second}))
called := 0
cleanupErr := errors.New("cleanup")
a.OnShutdown(func() error { called++; return cleanupErr })
err := a.RunContext(context.Background(), "bad address")
if err == nil || !strings.Contains(err.Error(), "http.readTimeout") || !errors.Is(err, cleanupErr) || called != 1 {
t.Fatalf("validation/cleanup: %v, %d", err, called)
}
defer func() {
if recover() == nil {
t.Error("NewServer accepted invalid config")
}
}()
a.NewServer("")
}
type httpConfigListener struct{ net.Listener }
func (l httpConfigListener) Accept() (net.Conn, error) {
c, err := l.Listener.Accept()
if err == nil {
if tcp, ok := c.(*net.TCPConn); ok {
_ = tcp.SetWriteBuffer(32 << 10)
}
}
return c, err
}
func httpConfigConnection(t *testing.T, transport string, cfg omni.HTTPConfig, handler omni.Handler) net.Conn {
t.Helper()
a := omni.New(omni.WithTransport(transport), omni.WithHTTPConfig(cfg), omni.WithShutdownTimeout(time.Second))
a.GET("/", handler)
a.POST("/", handler)
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- a.ServeContext(ctx, httpConfigListener{l}) }()
t.Cleanup(func() {
cancel()
select {
case err := <-done:
if err != nil {
t.Errorf("server shutdown: %v", err)
}
case <-time.After(3 * time.Second):
t.Error("server did not stop")
}
})
c, err := net.DialTimeout("tcp", l.Addr().String(), time.Second)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { c.Close() })
_ = c.SetDeadline(time.Now().Add(3 * time.Second))
return c
}
func TestHTTPConfiguredNetworkLimits(t *testing.T) {
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative} {
t.Run(transport, func(t *testing.T) {
empty := func(c *omni.Context) error { w := c.Raw(); w.WriteHeader(204); return nil }
t.Run("slowHeaders", func(t *testing.T) {
c := httpConfigConnection(t, transport, omni.HTTPConfig{ReadHeaderTimeout: 60 * time.Millisecond}, empty)
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\nX-Incomplete:"); err != nil {
t.Fatal(err)
}
_, err := io.ReadAll(c)
if e, ok := err.(net.Error); ok && e.Timeout() {
t.Fatal("server read-header timeout not applied")
}
})
t.Run("slowBody", func(t *testing.T) {
var accepted atomic.Bool
c := httpConfigConnection(t, transport, omni.HTTPConfig{ReadTimeout: 60 * time.Millisecond}, func(c *omni.Context) error {
_, err := io.ReadAll(c.Request().Body)
accepted.Store(err == nil)
return nil
})
if _, err := io.WriteString(c, "POST / HTTP/1.1\r\nHost: localhost\r\nConnection: close\r\nContent-Length: 10\r\n\r\nx"); err != nil {
t.Fatal(err)
}
_, err := io.ReadAll(c)
if e, ok := err.(net.Error); ok && e.Timeout() {
t.Fatal("read timeout not applied")
}
if accepted.Load() {
t.Fatal("incomplete body accepted")
}
})
t.Run("idle", func(t *testing.T) {
c := httpConfigConnection(t, transport, omni.HTTPConfig{IdleTimeout: 60 * time.Millisecond}, empty)
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"); err != nil {
t.Fatal(err)
}
r, err := http.ReadResponse(bufio.NewReader(c), nil)
if err != nil {
t.Fatal(err)
}
r.Body.Close()
if r.StatusCode != 204 {
t.Fatal(r.StatusCode)
}
_, err = c.Read(make([]byte, 1))
if e, ok := err.(net.Error); ok && e.Timeout() {
t.Fatal("idle timeout not applied")
}
if err == nil {
t.Fatal("idle connection not closed")
}
})
t.Run("writeBackpressure", func(t *testing.T) {
done := make(chan error, 1)
c := httpConfigConnection(t, transport, omni.HTTPConfig{WriteTimeout: 60 * time.Millisecond}, func(c *omni.Context) error {
w := c.Raw()
w.WriteHeader(200)
if err := http.NewResponseController(w).Flush(); err != nil {
done <- err
return nil
}
_, err := io.Copy(w, io.LimitReader(repeatingHTTPReader{}, 64<<20))
done <- err
return nil
})
if tcp, ok := c.(*net.TCPConn); ok {
_ = tcp.SetReadBuffer(1024)
}
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\n\r\n"); err != nil {
t.Fatal(err)
}
select {
case err := <-done:
if err == nil {
t.Fatal("write unexpectedly completed")
}
case <-time.After(2 * time.Second):
t.Fatal("write timeout not applied")
}
})
t.Run("headerLimit", func(t *testing.T) {
c := httpConfigConnection(t, transport, omni.HTTPConfig{MaxHeaderBytes: 1024}, empty)
if _, err := io.WriteString(c, "GET / HTTP/1.1\r\nHost: localhost\r\nX-Large: "+strings.Repeat("a", 8192)+"\r\n\r\n"); err != nil {
t.Fatal(err)
}
r, err := http.ReadResponse(bufio.NewReader(c), nil)
if err != nil {
t.Fatal(err)
}
r.Body.Close()
if r.StatusCode != 431 {
t.Fatalf("oversized header status: %d", r.StatusCode)
}
})
})
}
}
type repeatingHTTPReader struct{}
func (repeatingHTTPReader) Read(b []byte) (int, error) {
for i := range b {
b[i] = 'x'
}
return len(b), nil
}

View File

@@ -4,6 +4,7 @@ import (
"net/http"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/response"
)
func (a *App) handle(routes httpx.RouterGroup, method, path string, handler httpx.Handler, middleware []httpx.Middleware) {
@@ -11,6 +12,10 @@ func (a *App) handle(routes httpx.RouterGroup, method, path string, handler http
}
func (a *App) adapt(handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
return a.adaptResponse(a.responseEncoder, handler, middleware...)
}
func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
wrapped := handler
all := make([]httpx.Middleware, 0, len(a.middlewares)+len(middleware))
all = append(all, a.middlewares...)
@@ -25,6 +30,8 @@ func (a *App) adapt(handler httpx.Handler, middleware ...httpx.Middleware) http.
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
}
requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context()))
requestContext.SetResponseEncoder(encoder)
defer func() { httpx.CleanupMultipart(requestContext.Request()) }()
if err := wrapped(requestContext); err != nil {
a.writeError(requestContext, err)
}
@@ -37,6 +44,14 @@ func (a *App) Handle(method, path string, handler httpx.Handler, middleware ...h
a.handle(a.router, method, path, handler, middleware)
}
// HandleWithResponse selects the route policy before all application middleware.
func (a *App) HandleWithResponse(method, path string, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
if encoder == nil {
encoder = a.responseEncoder
}
a.router.Handle(method, path, a.adaptResponse(encoder, handler, middleware...))
}
func (a *App) GET(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
a.handle(a.router, http.MethodGet, path, handler, middleware)
}
@@ -94,6 +109,15 @@ func (g *RouterGroup) Handle(method, path string, handler httpx.Handler, middlew
g.route(method, path, handler, middleware)
}
func (g *RouterGroup) HandleWithResponse(method, path string, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
if encoder == nil {
encoder = g.app.responseEncoder
}
combined := append([]httpx.Middleware{}, g.middlewares...)
combined = append(combined, middleware...)
g.group.Handle(method, path, g.app.adaptResponse(encoder, handler, combined...))
}
func (g *RouterGroup) GET(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
g.route(http.MethodGet, path, handler, middleware)
}

View File

@@ -57,6 +57,22 @@ func (a *App) prepareTransport() (func(context.Context, net.Listener) error, err
case config.TransportNative:
server := native.New(a)
server.ShutdownTimeout = shutdownTimeout
cfg := a.config.HTTP
if cfg.ReadHeaderTimeout > 0 {
server.ReadHeaderTimeout = cfg.ReadHeaderTimeout
}
if cfg.ReadTimeout > 0 {
server.ReadTimeout = cfg.ReadTimeout
}
if cfg.WriteTimeout > 0 {
server.WriteTimeout = cfg.WriteTimeout
}
if cfg.IdleTimeout > 0 {
server.IdleTimeout = cfg.IdleTimeout
}
if cfg.MaxHeaderBytes > 0 {
server.MaxHeaderBytes = cfg.MaxHeaderBytes
}
if tlsConfig != nil {
return func(ctx context.Context, listener net.Listener) error {
return server.ServeTLS(ctx, listener, tlsConfig)
@@ -66,6 +82,12 @@ func (a *App) prepareTransport() (func(context.Context, net.Listener) error, err
case config.TransportHTTP2:
server := h2transport.New(a)
server.ShutdownTimeout = shutdownTimeout
if a.config.HTTP.IdleTimeout > 0 {
server.IdleTimeout = a.config.HTTP.IdleTimeout
}
if a.config.HTTP.MaxHeaderBytes > 0 {
server.MaxHeaderBytes = a.config.HTTP.MaxHeaderBytes
}
return func(ctx context.Context, listener net.Listener) error {
return server.ServeTLS(ctx, listener, tlsConfig)
}, nil

View File

@@ -1,17 +1,82 @@
# go-omni CLI
首次接入见 [README](../../README.md),无需克隆框架源码。本页说明命令参数;下方 `go run ./cmd/go-omni` 示例在框架仓库中执行,并显式引用本地源码。
## API 契约驱动项目
```sh
# 创建包含可运行问候接口的新项目
go run ./cmd/go-omni api new --module example.com/demo --framework-dir . ../demo
# 从已有定义生成对应目录、类型、路由和业务骨架
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
--dir ../userService --module example.com/userService --framework-dir .
# 校验定义;也接受 goctl 风格的 api go -api ... -dir ...
go run ./cmd/go-omni api validate --api examples/contracts/user.api
```
`new --api <file>` 是契约项目的另一个入口。可以组合 `--database mysql --storage local --log-output file`,生成配置、依赖初始化与资源关闭。API 默认无数据库、无对象存储;`--database memory` 等同于不打开连接。`--force` 不允许覆盖 API 项目。初始配置选择写入 manifest,重新生成自动沿用。
后续在生成项目内执行 `go-omni api generate --api api/user.api --dir .`。
项目内可省略 `--api` / `--dir`。`api inspect` 展示路由和鉴权/业务错误,`api export` 向 stdout 导出 OpenAPI,`api generate --dry-run` 预览文件变化。`--check` 只读检查生成产物是否过期。生成器记录所有权,保留 Service / Repository / 配置 / main 的手写内容,遇到生成文件被手动修改则报冲突。新 Service 与中间件占位实现返回 501。支持 `error (...)`、具名 `auth`、路由 `@errors` / `@auth` 和多行 `@doc`;错误目录同步生成 `internal/apierrors` 调用函数。
支持范围、参数语义、目录及安全更新策略见 [API 开发指南](../../API.md)。本版借鉴 go-zero 的工作流,不宣称兼容全部 goctl 语法;仍通过 Omni/httpx 保留网络层替换能力。
## 日志与对象存储按需装配
新增 `--log-output stdout|stderr|file`、`--log-format json|text`、`--storage none|local|s3|r2|oss`。
指定任意一个新选项后,生成严格的配置文件、初始化代码和项目自身的测试;不指定则保留原有轻量骨架。
对象存储独立于 `--database memory|mysql`。只有选择文件存储才会生成文件 Handler / Service 和对应适配器。
开发阶段可以使用本地源码;固定远程版本的安装和验收见 [发布说明](../../RELEASE.md):
```sh
# 在 go-omni 仓库中运行;目标目录须为空。
go run ./cmd/go-omni new --framework-dir . \
--module example.com/fileService --log-output file --log-format json \
--storage local ../fileService
```
生成项目包含 `etc/config.yaml`、`internal/settings`、`internal/servicecontext/storage.go`、
文件 Service/Handler、测试和 `MODULES.md`。按生成的 MODULES.md 配置示例 Token 和本地签名密钥后,
`go mod tidy && go run ./cmd/server` 即可体验。云存储分别选择 `--storage s3`、`r2`、`oss`,
再设置已有桶/凭据等环境变量;不自动创建桶或修改云资源。
配置优先级为默认值 → YAML/JSON → 环境变量;密钥仅允许环境变量。Service 依赖统一存储接口,
组合处负责创建客户端与有界连接池,退出时关闭自己创建的资源。文件接口提供服务端上传、直传签名、
临时下载链接及统一错误处理;日志示例携带 requestId,文件输出默认带轮转。没有固定业务身份上下文。
新模板要求 `v0.7.0-preview` 或更新版本;明确指定旧版本并启用新模块会被拒绝,
避免模板引用旧框架不存在的接口。本地 `--framework-dir` 验证不受此限制。`--force` 可重新生成兼容配置,
但如果切换配置会遗留引用已移除依赖的旧文件,则拒绝操作,要求使用新目录;不会自动删除修改过的代码。
验证当前生成器:
```sh
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./cmd/go-omni/...
python3 scripts/verifyScaffoldModules.py
```
前者覆盖两种数据库与五种文件存储选择、日志方式/格式,以及生成项目的配置、资源关闭和接口测试。
后者实际编译启动服务,验证日志、上传/下载/直传链接、错误状态、nethttp/native 下的 SIGINT/SIGTERM 和监听端口释放。
云端选项的测试使用本地替身,不代表 OSS/R2 实网验收;本批不访问业务服务器。报告见
[脚手架模块验收](../../performance/scaffoldModulesLocal.json)。
## 其他命令与远程安装
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
该命令尚未包含在下方历史标签中;当前源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
该命令从 `v0.7.0-preview` 提供;源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
边界和验证方式见 [业务基础模块](../../MODULES.md)。
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
go-omni new --module example.com/billing ./billing
GOPRIVATE=git.yuntuo.hk go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.9.1-preview
go-omni new --framework-version v0.9.1-preview --module example.com/billing ./billing
cd billing
go mod tidy
go run ./cmd/server

176
cmd/go-omni/api.go Normal file
View File

@@ -0,0 +1,176 @@
package main
import (
"fmt"
"os"
"strings"
"text/tabwriter"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
)
func runAPI(args []string) error {
if len(args) == 0 || args[0] == "help" || args[0] == "--help" || args[0] == "-h" {
usage()
return nil
}
command := args[0]
if command == "go" {
command = "generate"
}
if command == "check" {
command = "validate"
}
if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" {
return fmt.Errorf("unknown api command %q", command)
}
readOnly := command == "validate" || command == "inspect" || command == "export"
o := apigen.Options{}
for i := 1; i < len(args); i++ {
arg := args[i]
if arg == "--help" || arg == "-h" {
usage()
return nil
}
if arg == "--check" || arg == "--dry-run" {
if command != "generate" {
return fmt.Errorf("%s belongs to api generate", arg)
}
if arg == "--check" {
o.Check = true
} else {
o.DryRun = true
}
continue
}
if !strings.HasPrefix(arg, "-") {
if command == "new" && o.Dir == "" {
o.Dir = arg
continue
}
return fmt.Errorf("unexpected argument %q", arg)
}
name, value, inline := strings.Cut(arg, "=")
// goctl-style -api/-dir aliases ease migration; long forms are canonical.
if name == "-api" {
name = "--api"
}
if name == "-dir" {
name = "--dir"
}
switch name {
case "--api", "--dir", "--module", "--framework-dir", "--framework-version", "--log-output", "--log-format", "--database", "--storage":
default:
return fmt.Errorf("unknown api option %q", name)
}
if !inline {
if i+1 >= len(args) || strings.HasPrefix(args[i+1], "-") {
return fmt.Errorf("%s requires a value", name)
}
i++
value = args[i]
}
if strings.TrimSpace(value) == "" {
return fmt.Errorf("%s requires a value", name)
}
if readOnly && name != "--api" {
return fmt.Errorf("api %s does not accept %s", command, name)
}
switch name {
case "--api":
o.API = value
case "--dir":
o.Dir = value
case "--module":
o.Module = value
case "--framework-dir":
o.FrameworkDir = value
case "--framework-version":
o.FrameworkVersion = value
case "--database":
o.Database = value
case "--storage":
o.Storage = value
case "--log-output":
o.LogOutput = value
case "--log-format":
o.LogFormat = value
}
}
if command != "new" && o.API == "" {
dir := o.Dir
if dir == "" {
dir = "."
}
var err error
o.API, err = apigen.ProjectEntry(dir)
if err != nil {
return err
}
}
if readOnly {
spec, err := apigen.Parse(o.API)
if err != nil {
return err
}
compiled, err := apigen.Compile(spec)
if err != nil {
return err
}
switch command {
case "validate":
fmt.Printf("valid API: %d types, %d routes, %d errors, %d auth schemes\n", len(spec.Types), len(spec.Routes), len(spec.Errors), len(spec.Auth))
case "export":
fmt.Println(string(compiled.Document))
case "inspect":
writer := tabwriter.NewWriter(os.Stdout, 0, 4, 2, ' ', 0)
fmt.Fprintln(writer, "METHOD\tPATH\tHANDLER\tAUTH\tRESPONSE\tERRORS")
definitions := map[string]apigen.ErrorDefinition{}
for _, e := range spec.Errors {
definitions[e.Name] = e
}
for _, r := range compiled.Spec.Routes {
auth := r.Auth
if auth == "" {
auth = "(undeclared)"
}
var errs []string
for _, name := range r.Errors {
e := definitions[name]
errs = append(errs, fmt.Sprintf("%d:%s", e.Status, e.Code))
}
format := r.ResponseFormat
if format == "" {
format = "default"
}
fmt.Fprintf(writer, "%s\t%s\t%s/%s\t%s\t%s\t%s\n", strings.ToUpper(r.Method), r.Path, r.Group, r.Handler, auth, format, strings.Join(errs, ", "))
}
return writer.Flush()
}
return nil
}
var out apigen.Output
var err error
if command == "new" {
out, err = apigen.New(o)
} else {
if o.Dir == "" {
o.Dir = "."
}
out, err = apigen.Generate(o)
}
if err != nil {
return err
}
if o.DryRun {
for _, change := range out.Changes {
fmt.Printf("%-7s %s\n", change.Action, change.Path)
}
fmt.Printf("Preview: %d changes; %d existing developer-owned files preserved; no files written\n", len(out.Changes), len(out.Preserved))
} else if o.Check {
fmt.Println("API generated files are current")
} else {
fmt.Printf("API project ready: %d generated, %d created, %d preserved, %d removed\n", len(out.Generated), len(out.Created), len(out.Preserved), len(out.Removed))
}
return nil
}

107
cmd/go-omni/api_test.go Normal file
View File

@@ -0,0 +1,107 @@
package main
import (
"encoding/json"
"io"
"os"
"path/filepath"
"strings"
"testing"
)
func TestAPICommands(t *testing.T) {
root, err := filepath.Abs("../..")
if err != nil {
t.Fatal(err)
}
project := filepath.Join(t.TempDir(), "demo")
if err = runAPI([]string{"new", project, "--module=example.com/demo", "--framework-dir", root}); err != nil {
t.Fatal(err)
}
api := filepath.Join(project, "api/service.api")
if err = runAPI([]string{"validate", "-api", api}); err != nil {
t.Fatal(err)
}
if err = runAPI([]string{"go", "-api", api, "-dir", project, "--check"}); err != nil {
t.Fatal(err)
}
if err = runNew([]string{"--force", project}); err == nil {
t.Fatal("CRUD starter overwrote API project")
}
target := filepath.Join(t.TempDir(), "other")
if err = runNew([]string{"--api", api, "--module", "example.com/other", "--framework-dir", root, target}); err != nil {
t.Fatal(err)
}
if _, err = os.Stat(filepath.Join(target, "internal/service/greet/greet.go")); err != nil {
t.Fatal(err)
}
for _, args := range [][]string{{"validate"}, {"generate", "--api"}, {"new", "--api=x", "somewhere"}, {"generate", "--dir="}, {"generate", "--force"}, {"validate", "--api", api, "--dir", project}, {"validate", "--check"}} {
if err = runAPI(args); err == nil {
t.Fatalf("invalid args accepted: %v", args)
}
}
if err = runNew([]string{"--api", api, "--force", target}); err == nil || !strings.Contains(err.Error(), "does not accept --force") {
t.Fatalf("incompatible profile: %v", err)
}
}
func TestAPIDeveloperCommands(t *testing.T) {
root, err := filepath.Abs("../..")
if err != nil {
t.Fatal(err)
}
dir := filepath.Join(t.TempDir(), "project")
if err = runAPI([]string{"new", dir, "--framework-dir", root, "--module", "example.com/commands"}); err != nil {
t.Fatal(err)
}
// Generation can resolve the entry from an explicit target directory.
if err = runAPI([]string{"generate", "--dir", dir, "--dry-run"}); err != nil {
t.Fatal(err)
}
wd, err := os.Getwd()
if err != nil {
t.Fatal(err)
}
if err = os.Chdir(dir); err != nil {
t.Fatal(err)
}
defer os.Chdir(wd)
for _, args := range [][]string{{"validate"}, {"inspect"}, {"generate", "--check"}} {
if err = runAPI(args); err != nil {
t.Fatalf("%v: %v", args, err)
}
}
doc := captureAPIOutput(t, []string{"export"})
var parsed map[string]any
if err = json.Unmarshal([]byte(doc), &parsed); err != nil {
t.Fatalf("export must be pure JSON: %v", err)
}
if parsed["openapi"] != "3.0.3" {
t.Fatal("wrong export")
}
for _, args := range [][]string{{"generate", "--dry-run", "--check"}, {"new", "--dry-run", "other"}, {"inspect", "--dir", dir}, {"export", "--log-output", "file"}} {
if err = runAPI(args); err == nil {
t.Fatalf("accepted invalid flags: %v", args)
}
}
}
func captureAPIOutput(t *testing.T, args []string) string {
t.Helper()
r, w, err := os.Pipe()
if err != nil {
t.Fatal(err)
}
previous := os.Stdout
os.Stdout = w
defer func() { os.Stdout = previous; r.Close(); w.Close() }()
result := make(chan []byte, 1)
go func() { data, _ := io.ReadAll(r); result <- data }()
callErr := runAPI(args)
w.Close()
os.Stdout = previous
data := <-result
if callErr != nil {
t.Fatal(callErr)
}
return string(data)
}

View File

@@ -0,0 +1,360 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func frameworkRoot(t *testing.T) string {
t.Helper()
p, err := filepath.Abs("../../..")
if err != nil {
t.Fatal(err)
}
return p
}
func writeFile(t *testing.T, name, body string) {
t.Helper()
if err := os.MkdirAll(filepath.Dir(name), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(name, []byte(body), 0644); err != nil {
t.Fatal(err)
}
}
func readFile(t *testing.T, name string) string {
t.Helper()
b, err := os.ReadFile(name)
if err != nil {
t.Fatal(err)
}
return string(b)
}
func parseText(t *testing.T, source string) (*Spec, error) {
t.Helper()
file := filepath.Join(t.TempDir(), "service.api")
writeFile(t, file, source)
return Parse(file)
}
func TestCompileExample(t *testing.T) {
s, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/user.api"))
if err != nil {
t.Fatal(err)
}
c, err := Compile(s)
if err != nil {
t.Fatal(err)
}
if len(c.Groups) != 2 || len(s.Routes) != 3 || len(s.Sources) != 2 {
t.Fatalf("wrong spec: %+v", s)
}
var doc map[string]any
if err = json.Unmarshal(c.Document, &doc); err != nil {
t.Fatal(err)
}
route := doc["paths"].(map[string]any)["/api/v1/users/{userId}"].(map[string]any)["put"].(map[string]any)
if len(route["parameters"].([]any)) != 3 {
t.Fatal("missing mixed parameters")
}
schema := route["requestBody"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
if schema["additionalProperties"] != false || len(schema["required"].([]any)) != 2 {
t.Fatalf("wrong body schema: %+v", schema)
}
if strings.Contains(string(c.Document), `"user_id"`) {
t.Fatal("snake case wire name")
}
}
func TestInvalidDefinitions(t *testing.T) {
cases := []string{
`syntax="v2"`,
`@server ( jwt: Auth ) service user { @handler go get /go }`,
"type Request { Name string `json:\"name\" binding:\"typo\"` }\nservice user { @handler run\n post /go (Request) }",
"type A { Other *A `json:\"other,optional\"` }\nservice user { @handler run\n post /go (A) }",
"type A { Other Missing `json:\"other\"` }\nservice user { @handler run\n post /go (A) }",
"type A { UserID int `json:\"user_id\"` }\nservice user { @handler run\n post /go (A) }",
"service user { @handler go\n get /go\n @handler other\n get /go\n}",
"service user { @handler go\n get /:id\n}",
"service user { @handler service\n get /go\n}",
"type A { ID int `path:\"id\"` }\nservice user { @handler go\n get /go (A)\n}",
"type A { Name string `json:\"name\"` }\nservice user { @handler go\n get /go (A)\n}",
"service user { @handler go\n @status 201\n post /go\n}",
"type A { Name string `header:\"Name\"` }\nservice user { @handler go\n get /go returns (A)\n}",
"@server (group: ../../bad)\nservice user { @handler go\n get /go\n}",
"service user { @handler go\n get /health/ready\n}",
"type A { Name string `json:\"name\"`\n Name int `json:\"other\"` }\nservice user { @handler go\n post /go (A)\n}",
"service user { @handler go\n get /go\n @handler Go\n get /other\n}",
"type A { Value []byte `json:\"value\"` }\nservice user { @handler go\n post /go (A)\n}",
}
for i, source := range cases {
s, err := parseText(t, source)
if err == nil {
_, err = Compile(s)
}
if err == nil {
t.Fatalf("case %d accepted: %s", i, source)
}
}
}
func TestImports(t *testing.T) {
root := t.TempDir()
entry := filepath.Join(root, "api", "entry.api")
writeFile(t, entry, "import \"types.api\"\nservice user { @handler ping\n get /ping\n}")
writeFile(t, filepath.Join(root, "api/types.api"), `import "entry.api"`)
if _, err := Parse(entry); err == nil || !strings.Contains(err.Error(), "circular") {
t.Fatalf("cycle: %v", err)
}
writeFile(t, entry, "import \"../outside.api\"\nservice user { @handler ping\n get /ping\n}")
writeFile(t, filepath.Join(root, "outside.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
if _, err := Parse(entry); err == nil {
t.Fatal("escaped import root")
}
writeFile(t, entry, "import (\n \"shared.api\"\n \"./shared.api\"\n)\nservice user { @handler ping\n get /ping\n}")
writeFile(t, filepath.Join(root, "api/shared.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
s, err := Parse(entry)
if err != nil || len(s.Types) != 1 {
t.Fatalf("duplicate import: %+v %v", s, err)
}
}
func TestSharedPrefixConflict(t *testing.T) {
source := "type A { ID int `path:\"id\"` }\ntype B { Name string `path:\"name\"` }\nservice user {\n @handler first\n get /users/:id (A)\n @handler second\n get /users/:name/items (B)\n}"
s, err := parseText(t, source)
if err != nil {
t.Fatal(err)
}
if _, err = Compile(s); err == nil {
t.Fatal("shared prefix conflict was accepted")
}
}
func FuzzParser(f *testing.F) {
for _, s := range []string{starterAPI, declaredAPI, "error Broken", "auth Auth (type: bearer)", "syntax = \"v1\"", "type (", "@server (prefix: /api)"} {
f.Add(s)
}
f.Fuzz(func(t *testing.T, source string) {
if len(source) > 16384 {
t.Skip()
}
tokens, err := lex("fuzz.api", source)
if err != nil {
return
}
spec := &Spec{Title: "Fuzz", Version: "1"}
p := parser{tokens: tokens, spec: spec}
if p.parse() == nil {
_, _ = Compile(spec)
}
})
}
func TestRegenerationAndConflicts(t *testing.T) {
dir := filepath.Join(t.TempDir(), "app")
if _, err := New(Options{Dir: dir, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
manifestBefore := readFile(t, filepath.Join(dir, manifestPath))
typesBefore := readFile(t, filepath.Join(dir, "internal/types/typesGen.go"))
o.Check = true
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
servicePath := filepath.Join(dir, "internal/service/greet/greet.go")
service := readFile(t, servicePath) + "\n// user edit must survive\n"
writeFile(t, servicePath, service)
for _, file := range []string{"cmd/server/main.go", "etc/config.yaml", "internal/servicecontext/context.go", "internal/repository/greet/repository.go"} {
body := readFile(t, filepath.Join(dir, file))
if strings.HasSuffix(file, ".go") {
body += "\n// handwritten extension\n"
} else {
body += "\n# handwritten extension\n"
}
writeFile(t, filepath.Join(dir, file), body)
}
api := readFile(t, o.API)
api += "\n@server (group: extra)\nservice greet-api {\n @handler ping\n get /ping\n}\n"
writeFile(t, o.API, api)
if _, err := Generate(o); err == nil {
t.Fatal("check accepted stale output")
}
if readFile(t, filepath.Join(dir, manifestPath)) != manifestBefore {
t.Fatal("--check wrote manifest")
}
o.Check = false
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
if readFile(t, servicePath) != service {
t.Fatal("business code overwritten")
}
// Retiring a group removes only owned handlers, keeping its business files.
writeFile(t, o.API, strings.Split(api, "\n@server (group: extra)")[0])
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(dir, "internal/handler/extra/handlersGen.go")); !os.IsNotExist(err) {
t.Fatal("stale handler remains")
}
if _, err := os.Stat(filepath.Join(dir, "internal/service/extra/ping.go")); err != nil {
t.Fatal("business stub removed")
}
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), typesBefore+"\n// edited generated file\n")
before := snapshot(t, dir)
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "was edited") {
t.Fatalf("did not protect edited output: %v", err)
}
after := snapshot(t, dir)
if !sameFiles(before, after) {
t.Fatal("failed regeneration changed files")
}
}
func TestOutputBoundaries(t *testing.T) {
for _, kind := range []string{"symlink", "unowned", "lock", "caseCollision"} {
t.Run(kind, func(t *testing.T) {
root := t.TempDir()
dir := filepath.Join(root, "app")
api := filepath.Join(root, "input/service.api")
writeFile(t, api, starterAPI)
switch kind {
case "symlink":
if err := os.MkdirAll(dir, 0755); err != nil {
t.Fatal(err)
}
outside := filepath.Join(root, "outside")
os.MkdirAll(outside, 0755)
if err := os.Symlink(outside, filepath.Join(dir, "internal")); err != nil {
t.Fatal(err)
}
case "unowned":
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), "package types // mine\n")
case "lock":
writeFile(t, filepath.Join(dir, ".goOmni/api.lock"), "someone else")
case "caseCollision":
writeFile(t, api, starterAPI+"\n@server (group: gReet)\nservice greet-api {\n @handler extra\n get /extra\n}\n")
}
before := snapshot(t, root)
if _, err := Generate(Options{Dir: dir, API: api, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err == nil {
t.Fatal("unsafe output accepted")
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("conflict changed files")
}
})
}
}
func snapshot(t *testing.T, root string) map[string]string {
t.Helper()
out := map[string]string{}
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if d.Type().IsRegular() {
data, e := os.ReadFile(path)
if e != nil {
return e
}
rel, _ := filepath.Rel(root, path)
out[rel] = string(data)
}
return nil
})
if err != nil {
t.Fatal(err)
}
return out
}
func sameFiles(a, b map[string]string) bool {
if len(a) != len(b) {
return false
}
for k, v := range a {
if b[k] != v {
return false
}
}
return true
}
func TestGeneratedProjectsBuildAndRun(t *testing.T) {
if testing.Short() {
t.Skip("consumer module integration")
}
root := frameworkRoot(t)
for _, starter := range []bool{true, false} {
name := "contract"
if starter {
name = "starter"
}
t.Run(name, func(t *testing.T) {
dir := filepath.Join(t.TempDir(), "project")
o := Options{Dir: dir, Module: "example.com/consumer", FrameworkDir: root, LogOutput: "file", LogFormat: "text"}
var err error
if starter {
_, err = New(o)
} else {
o.API = filepath.Join(root, "examples/contracts/user.api")
_, err = Generate(o)
}
if err != nil {
t.Fatal(err)
}
source := consumerContractTest
if starter {
source = consumerStarterTest
} else {
writeFile(t, filepath.Join(dir, "internal/service/user/updateUser.go"), `package user
import("context";"example.com/consumer/internal/types")
func(s *Service) UpdateUser(ctx context.Context,req *types.UpdateUserReq)(*types.UserResp,error){return &types.UserResp{UserID:req.UserID,Name:req.Name,Active:req.Active},nil}
`)
}
writeFile(t, filepath.Join(dir, "contract_test.go"), source)
o.API = filepath.Join(dir, "api/service.api")
if !starter {
o.API = filepath.Join(dir, "api/user.api")
}
if _, err = Generate(o); err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
output, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("consumer: %v\n%s", err, output)
}
t.Log(string(output))
})
}
}
const consumerStarterTest = `package consumer_test
import("context";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext";"example.com/consumer/internal/settings")
func TestGreeting(t *testing.T){cfg,err:=settings.Load("etc/config.yaml");if err!=nil{t.Fatal(err)};cfg.Server.Log.Writer=io.Discard;cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{};logger,err:=logx.New(cfg.Server.Log);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)};w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,httptest.NewRequest("GET","/api/greet/Alice",nil));if w.Code!=200||!strings.Contains(w.Body.String(),"Hello Alice")||w.Header().Get("X-Request-ID")==""{t.Fatalf("%d %s",w.Code,w.Body.String())}}
`
const consumerContractTest = `package consumer_test
import("context";"encoding/json";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext")
func TestContract(t *testing.T){cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
for _,tc:=range []struct{method,path,body string;status int}{
{"PUT","/api/v1/users/7?notify=true",` + "`" + `{"name":"Alice","active":false,"age":0}` + "`" + `,200},
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice"}` + "`" + `,400},
{"PUT","/api/v1/users/7?notify=true&notify=false",` + "`" + `{"name":"Alice","active":false}` + "`" + `,400},
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice","active":false,"age":151}` + "`" + `,400},
{"GET","/api/v1/users?page=0","",400},
{"GET","/api/v1/users?page=1","",501},
{"DELETE","/admin/v1/users/7","",501},
}{req:=httptest.NewRequest(tc.method,tc.path,strings.NewReader(tc.body));req.Header.Set("Content-Type","application/json");w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,req);if w.Code!=tc.status{t.Fatalf("%s: %d %s",tc.path,w.Code,w.Body.String())};if tc.status==200{var v struct{Data struct{UserID int64 ` + "`json:\"userId\"`" + `;Name string ` + "`json:\"name\"`" + `}};if err:=json.Unmarshal(w.Body.Bytes(),&v);err!=nil||v.Data.UserID!=7||v.Data.Name!="Alice"{t.Fatal(w.Body.String())}}}
delete(deps.Middlewares,"Auth");if err:=handler.RegisterRoutes(omni.New(),deps);err==nil{t.Fatal("missing auth hook accepted")}
}
`

View File

@@ -0,0 +1,161 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestCompatibilityContract(t *testing.T) {
s, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/compatibility.api"))
if err != nil {
t.Fatal(err)
}
c, err := Compile(s)
if err != nil {
t.Fatal(err)
}
if len(s.Scalars) != 1 || len(s.Routes) != 6 {
t.Fatal("incomplete contract")
}
if !strings.Contains(string(c.Document), `"anyOf"`) || !strings.Contains(string(c.Document), `"multipart/form-data"`) {
t.Fatal("missing schemas")
}
for _, change := range [][2]string{{"jsonInput: string | int64", "jsonInput: mystery"}, {"text: true", "text: false"}, {"mode: multipart", "mode: invalid"}, {"maxBytes: 128", "maxBytes: 0"}, {"goType: ID", "goType: id"}, {"Data any", "Data interface{ Method() }"}} {
source := readFile(t, filepath.Join(frameworkRoot(t), "examples/contracts/compatibility.api"))
spec, err := parseText(t, strings.Replace(source, change[0], change[1], 1))
if err == nil {
_, err = Compile(spec)
}
if err == nil {
t.Fatalf("accepted %s", change[1])
}
}
}
func TestCompatibilityConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
t.Skip("consumer integration")
}
root := frameworkRoot(t)
dir := filepath.Join(t.TempDir(), "consumer")
o := Options{API: filepath.Join(root, "examples/contracts/compatibility.api"), Dir: dir, Module: "example.com/compatibility", FrameworkDir: root}
releaseCLI := os.Getenv("GO_OMNI_RELEASE_CLI")
generate := func(check bool) {
t.Helper()
if releaseCLI == "" {
options := o
options.Check = check
if _, err := Generate(options); err != nil {
t.Fatal(err)
}
return
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
args := []string{"api", "generate", "--api", o.API, "--dir", dir, "--module", o.Module}
if check {
args = append(args, "--check")
}
cmd := exec.CommandContext(ctx, releaseCLI, args...)
cmd.Dir = filepath.Dir(dir)
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("released CLI: %v\n%s", err, output)
}
}
generate(false)
files := map[string]string{
"internal/service/demo/echo.go": "echo.go",
"internal/handler/demo/customBind.go": "hooks.go",
"internal/handler/files/downloadManual.go": "download.go",
"consumer_test.go": "consumer_test.go",
}
// The fixture implements related methods together; remove only its generated
// one-time stubs, then fill every stub so regeneration can preserve all files.
for _, name := range []string{"internal/service/demo/item.go", "internal/service/demo/custom.go"} {
writeFile(t, filepath.Join(dir, name), "package demo\n")
}
for _, name := range []string{"internal/handler/demo/uploadManual.go", "internal/handler/demo/callbackManual.go"} {
writeFile(t, filepath.Join(dir, name), "package demo\n")
}
for target, source := range files {
writeFile(t, filepath.Join(dir, target), readFile(t, filepath.Join("testdata/compatibility", source)))
}
before := map[string]string{}
for target := range files {
before[target] = readFile(t, filepath.Join(dir, target))
}
generate(false)
for target, content := range before {
if readFile(t, filepath.Join(dir, target)) != content {
t.Fatalf("overwrote %s", target)
}
}
generate(true)
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("consumer: %v\n%s", err, output)
}
if releaseCLI != "" {
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
output, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("module: %v\n%s", err, output)
}
var module struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(output, &module); err != nil {
t.Fatal(err)
}
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
t.Fatalf("not a fixed remote dependency: %+v", module)
}
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
}
}
func TestNestedDynamicAliases(t *testing.T) {
source := readFile(t, filepath.Join(frameworkRoot(t), "examples/contracts/compatibility.api"))
for _, typ := range []string{"[]interface{}", "map[string]interface{}", "map[string][]interface{}"} {
s, err := parseText(t, strings.Replace(source, "Data any", "Data "+typ, 1))
if err == nil {
_, err = Compile(s)
}
if err != nil {
t.Fatalf("%s: %v", typ, err)
}
}
}
func TestSwitchFromManualRequiresExplicitMigration(t *testing.T) {
root := frameworkRoot(t)
o := Options{API: filepath.Join(root, "examples/contracts/compatibility.api"), Dir: filepath.Join(t.TempDir(), "consumer"), Module: "example.com/compatibility", FrameworkDir: root}
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
// The hand-written source may contain business code. Never delete it to make
// the newly automatic handler compile; fail before touching generated files.
o.API = filepath.Join(o.Dir, "api/compatibility.api")
source := readFile(t, o.API)
source = strings.Replace(source, "@handlerMode manual\n @produces \"application/octet-stream\"", "", 1)
writeFile(t, o.API, source)
before := snapshot(t, o.Dir)
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "migrate and remove") {
t.Fatalf("unsafe mode switch: %v", err)
}
if !sameFiles(before, snapshot(t, o.Dir)) {
t.Fatal("failed mode switch mutated project")
}
}

View File

@@ -0,0 +1,352 @@
package apigen
import (
"bytes"
"encoding/json"
"errors"
"fmt"
"go/ast"
goparser "go/parser"
gotoken "go/token"
"reflect"
"regexp"
"sort"
"strings"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/openapi"
)
type Compiled struct {
Spec *Spec
Bindings map[string]*contract.Binder
Document []byte
Groups []string
}
var exportedName = regexp.MustCompile(`^[A-Z][A-Za-z0-9]*$`)
var goName = regexp.MustCompile(`^[a-zA-Z][A-Za-z0-9]*$`)
var lowerName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
var serviceName = regexp.MustCompile(`^[a-zA-Z][a-zA-Z0-9-]*$`)
var staticSegment = regexp.MustCompile(`^[A-Za-z0-9._~-]+$`)
func Compile(spec *Spec) (*Compiled, error) {
if spec == nil || len(spec.Routes) == 0 || !serviceName.MatchString(spec.Service) {
return nil, errors.New("api: valid service and routes required")
}
var prepareErr error
spec, prepareErr = prepareMetadata(spec)
if prepareErr != nil {
return nil, prepareErr
}
definitions := map[string]Type{}
for _, t := range spec.Types {
if t.Name == "ContractScalars" && len(spec.Scalars) > 0 {
return nil, errors.New("ContractScalars is reserved when declaring scalars")
}
if !exportedName.MatchString(t.Name) {
return nil, fmt.Errorf("%s: type names must be UpperCamelCase", t.Position)
}
if _, ok := definitions[t.Name]; ok {
return nil, fmt.Errorf("%s: duplicate type %s", t.Position, t.Name)
}
definitions[t.Name] = t
}
scalarTypes, scalarDefs, err := compileScalars(spec, definitions)
if err != nil {
return nil, err
}
resolved := map[string]reflect.Type{}
resolving := map[string]bool{}
var resolve func(string, int) (reflect.Type, error)
var expression func(ast.Expr, int) (reflect.Type, error)
primitives := map[string]reflect.Type{"any": reflect.TypeOf((*any)(nil)).Elem(), "string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
if depth > 32 {
return nil, errors.New("type nesting exceeds 32")
}
switch e := e.(type) {
case *ast.Ident:
if t, ok := scalarTypes[e.Name]; ok {
return t, nil
}
if t, ok := primitives[e.Name]; ok {
return t, nil
}
return resolve(e.Name, depth+1)
case *ast.StarExpr:
if _, double := e.X.(*ast.StarExpr); double {
return nil, errors.New("multiple pointer levels unsupported")
}
t, err := expression(e.X, depth+1)
if err != nil {
return nil, err
}
return reflect.PointerTo(t), nil
case *ast.ArrayType:
if e.Len != nil {
return nil, errors.New("use slices, not fixed arrays")
}
t, err := expression(e.Elt, depth+1)
if err != nil {
return nil, err
}
return reflect.SliceOf(t), nil
case *ast.MapType:
key, ok := e.Key.(*ast.Ident)
if !ok || key.Name != "string" {
return nil, errors.New("map keys must be string")
}
t, err := expression(e.Value, depth+1)
if err != nil {
return nil, err
}
return reflect.MapOf(reflect.TypeOf(""), t), nil
default:
return nil, errors.New("unsupported type expression")
}
}
resolve = func(name string, depth int) (reflect.Type, error) {
if depth > 32 {
return nil, errors.New("type nesting exceeds 32")
}
if t, ok := resolved[name]; ok {
return t, nil
}
if resolving[name] {
return nil, fmt.Errorf("recursive type %s is unsupported", name)
}
typ, ok := definitions[name]
if !ok {
return nil, fmt.Errorf("unknown type %s", name)
}
resolving[name] = true
defer delete(resolving, name)
var fields []reflect.StructField
seen := map[string]bool{}
for _, f := range typ.Fields {
if !exportedName.MatchString(f.Name) || seen[f.Name] {
return nil, fmt.Errorf("%s: invalid or duplicate field %s", name, f.Name)
}
seen[f.Name] = true
expr, err := goparser.ParseExpr(f.Type)
if err != nil {
return nil, err
}
t, err := expression(expr, depth+1)
if err != nil {
return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err)
}
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)})
}
t := reflect.StructOf(fields)
resolved[name] = t
return t, nil
}
c := &Compiled{Spec: spec, Bindings: map[string]*contract.Binder{}}
for _, t := range spec.Types {
rt, err := resolve(t.Name, 0)
if err != nil {
return nil, err
}
b, err := contract.Describe(rt, scalarDefs...)
if err != nil {
return nil, fmt.Errorf("%s: %w", t.Position, err)
}
c.Bindings[t.Name] = b
}
api, err := openapi.New(spec.Title, spec.Version)
if err != nil {
return nil, err
}
handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{}
middlewareSymbols := map[string]string{}
for _, r := range spec.Routes {
if !goName.MatchString(r.Handler) {
return nil, fmt.Errorf("%s: invalid handler name", r.Position)
}
if strings.EqualFold(r.Handler, "service") {
return nil, errors.New("handler name service conflicts with the service constructor file")
}
if r.Group == "" {
return nil, errors.New("group required")
}
for _, part := range strings.Split(r.Group, "/") {
if !lowerName.MatchString(part) || gotoken.Lookup(part).IsKeyword() {
return nil, fmt.Errorf("invalid lowerCamelCase group %q", r.Group)
}
}
h := r.Group + "/" + strings.ToLower(r.Handler)
if handlers[h] {
return nil, fmt.Errorf("duplicate handler %s in group %s", r.Handler, r.Group)
}
handlers[h] = true
groups[r.Group] = true
mseen := map[string]bool{}
for _, m := range r.Middleware {
if !goName.MatchString(m) || mseen[m] {
return nil, fmt.Errorf("invalid or duplicate middleware %q", m)
}
mseen[m] = true
if strings.EqualFold(m, "defaults") || strings.EqualFold(m, "registryGen") {
return nil, fmt.Errorf("reserved middleware name %q", m)
}
if prior, ok := middlewareSymbols[strings.ToLower(m)]; ok && prior != m {
return nil, fmt.Errorf("middleware names %s and %s collide", prior, m)
}
middlewareSymbols[strings.ToLower(m)] = m
}
shape, err := routeShape(r.Path)
if err != nil {
return nil, fmt.Errorf("%s: %w", r.Position, err)
}
for _, prior := range spec.Routes {
if prior.Method != r.Method || prior.Path == r.Path {
continue
}
if conflictPrefix(prior.Path, r.Path) {
return nil, fmt.Errorf("conflicting parameter names at a shared route prefix: %s and %s", prior.Path, r.Path)
}
}
if prior, exists := shapes[shape]; exists && prior != r.Path {
return nil, fmt.Errorf("ambiguous route parameter names: %s and %s", prior, r.Path)
}
shapes[shape] = r.Path
pathKey := r.Method + " " + shape
if paths[pathKey] {
return nil, fmt.Errorf("duplicate route %s", pathKey)
}
paths[pathKey] = true
if r.Path == "/health/live" || r.Path == "/health/ready" || r.Path == "/openapi.json" {
return nil, fmt.Errorf("route %s is reserved by the generated server", r.Path)
}
if r.Method != strings.ToLower(r.Method) {
return nil, errors.New("HTTP methods must be lowercase")
}
if r.Status != 200 && r.Status != 201 && r.Status != 202 && r.Status != 204 {
return nil, errors.New("@status supports 200, 201, 202 or 204")
}
if !r.Manual && (r.Status == 204) != (r.Response == "") {
return nil, errors.New("204 requires no response; other success statuses require a response type")
}
if r.Method == "head" && r.Response != "" {
return nil, errors.New("HEAD must not declare a response body")
}
if err := validateRouteExtensions(r); err != nil {
return nil, fmt.Errorf("%s: %w", r.Position, err)
}
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
if r.Request != "" {
b, ok := c.Bindings[r.Request]
if !ok {
return nil, fmt.Errorf("unknown request %s", r.Request)
}
op.Parameters = b.Parameters()
op.Request = b.BodySchema()
if op.Request != nil && (r.BodyMode == "raw" || r.BodyMode == "multipart") {
return nil, errors.New("raw/multipart request DTOs may declare only path/form/header fields; body binding is manual")
}
if op.Request != nil && (r.Method == "get" || r.Method == "head") {
return nil, errors.New("GET/HEAD requests cannot declare JSON bodies; use form/path/header tags")
}
}
var response *openapi.Schema
if r.Response != "" {
b, ok := c.Bindings[r.Response]
if !ok {
return nil, fmt.Errorf("unknown response %s", r.Response)
}
s, err := b.ResponseSchema()
if err != nil {
return nil, err
}
if r.ResponseFormat == "raw" {
response = s
} else {
response, err = openapi.EnvelopeWith(s, responseConfig(spec, r))
if err != nil {
return nil, err
}
}
}
op.Responses[r.Status] = openapi.Response{Description: "Success", Schema: response}
errorSchema, err := openapi.EnvelopeWith(nil, responseConfig(spec, r))
if err != nil {
return nil, err
}
for _, code := range []int{400, 413, 415, 500, 501} {
op.Responses[code] = openapi.Response{Description: "Error", Schema: errorSchema}
}
if err = api.Handle(discardRoutes{}, op, func(*httpx.Context) error { return nil }); err != nil {
return nil, fmt.Errorf("%s: %w", r.Position, err)
}
}
for group := range groups {
c.Groups = append(c.Groups, group)
}
sort.Strings(c.Groups)
c.Document, err = api.JSON()
if err != nil {
return nil, err
}
// Decorate the document with explicit auth/error contracts. Credential
// validation remains in the linked application middleware.
var doc map[string]any
decoder := json.NewDecoder(bytes.NewReader(c.Document))
decoder.UseNumber()
if err = decoder.Decode(&doc); err != nil {
return nil, err
}
decorateDocument(doc, spec)
c.Document, err = json.MarshalIndent(doc, "", " ")
return c, err
}
type discardRoutes struct{}
func (discardRoutes) Handle(string, string, httpx.Handler, ...httpx.Middleware) {}
func operationID(r Route) string {
return strings.ReplaceAll(r.Group, "/", "_") + "_" + upper(r.Handler)
}
func upper(s string) string { return strings.ToUpper(s[:1]) + s[1:] }
func lower(s string) string { return strings.ToLower(s[:1]) + s[1:] }
func routeShape(path string) (string, error) {
if path == "/" {
return path, nil
}
if !strings.HasPrefix(path, "/") {
return "", errors.New("route must start with /")
}
parts := strings.Split(path[1:], "/")
seen := map[string]bool{}
for i, part := range parts {
if strings.HasPrefix(part, ":") {
name := part[1:]
if !lowerName.MatchString(name) || seen[name] {
return "", errors.New("invalid/duplicate path parameter")
}
seen[name] = true
parts[i] = ":"
} else if !staticSegment.MatchString(part) || part == "." || part == ".." {
return "", errors.New("invalid route segment")
}
}
return "/" + strings.Join(parts, "/"), nil
}
func conflictPrefix(left, right string) bool {
a, b := strings.Split(left, "/"), strings.Split(right, "/")
for i := 0; i < len(a) && i < len(b); i++ {
if strings.HasPrefix(a[i], ":") && strings.HasPrefix(b[i], ":") {
if a[i] != b[i] {
return true
}
continue
}
if a[i] != b[i] {
return false
}
}
return false
}

View File

@@ -0,0 +1,221 @@
package apigen
import (
"encoding/json"
"fmt"
"net/http"
"regexp"
"sort"
"strings"
"git.yuntuo.hk/k2safe/go-omni/openapi"
"git.yuntuo.hk/k2safe/go-omni/response"
)
var errorCode = regexp.MustCompile(`^[A-Z][A-Z0-9_]*$`)
var credentialName = regexp.MustCompile("^[!#$%&'*+.^_`|~0-9A-Za-z-]+$")
// prepareMetadata returns a detached spec; repeated compilation must not prepend
// authentication middleware more than once or mutate the parser's output.
func prepareMetadata(original *Spec) (*Spec, error) {
spec := *original
spec.Routes = append([]Route(nil), original.Routes...)
if err := validateResponses(&spec); err != nil {
return nil, err
}
definitions := map[string]ErrorDefinition{}
codes, symbols := map[string]bool{}, map[string]bool{}
for _, e := range spec.Errors {
if !exportedName.MatchString(e.Name) || e.Status < 400 || e.Status > 599 || !errorCode.MatchString(e.Code) || strings.TrimSpace(e.Message) == "" {
return nil, fmt.Errorf("%s: error requires UpperCamelCase name, status 400..599, UPPER_SNAKE_CASE code and a nonempty public message", e.Position)
}
if codes[e.Code] {
return nil, fmt.Errorf("%s: duplicate error code %s", e.Position, e.Code)
}
for _, symbol := range []string{e.Name, "Code" + e.Name} {
if symbols[symbol] {
return nil, fmt.Errorf("%s: duplicate error symbol %s", e.Position, symbol)
}
symbols[symbol] = true
}
codes[e.Code] = true
definitions[e.Name] = e
}
auth := map[string]AuthDefinition{}
for _, a := range spec.Auth {
if !exportedName.MatchString(a.Name) || !goName.MatchString(a.Middleware) || strings.EqualFold(a.Middleware, "defaults") || strings.EqualFold(a.Middleware, "registryGen") {
return nil, fmt.Errorf("%s: auth requires an UpperCamelCase name and a valid named middleware", a.Position)
}
if _, ok := auth[a.Name]; ok {
return nil, fmt.Errorf("%s: duplicate auth %s", a.Position, a.Name)
}
switch a.Type {
case "bearer":
if a.In != "" || a.KeyName != "" {
return nil, fmt.Errorf("%s: bearer auth does not accept in/name", a.Position)
}
case "apiKey":
if (a.In != "header" && a.In != "cookie") || !credentialName.MatchString(a.KeyName) || a.BearerFormat != "" {
return nil, fmt.Errorf("%s: apiKey requires in header/cookie and a valid name; bearerFormat is not supported", a.Position)
}
if a.In == "header" && (strings.EqualFold(a.KeyName, "Authorization") || strings.EqualFold(a.KeyName, "Cookie")) {
return nil, fmt.Errorf("%s: use bearer for Authorization or in cookie for a session", a.Position)
}
default:
return nil, fmt.Errorf("%s: auth type must be bearer or apiKey", a.Position)
}
auth[a.Name] = a
}
for i := range spec.Routes {
r := &spec.Routes[i]
r.Middleware = append([]string(nil), r.Middleware...)
if r.Auth != "" && r.Auth != "none" {
a, ok := auth[r.Auth]
if !ok {
return nil, fmt.Errorf("%s: unknown auth %s; declare auth %s (...) first", r.Position, r.Auth, r.Auth)
}
for _, m := range r.Middleware {
if strings.EqualFold(m, a.Middleware) {
return nil, fmt.Errorf("%s: auth %s already installs %s; remove it from middleware", r.Position, r.Auth, a.Middleware)
}
}
r.Middleware = append([]string{a.Middleware}, r.Middleware...)
}
seen := map[string]bool{}
for _, name := range r.Errors {
if _, ok := definitions[name]; !ok {
return nil, fmt.Errorf("%s: unknown error %s; declare it in error (...)", r.Position, name)
}
if seen[name] {
return nil, fmt.Errorf("%s: duplicate error %s", r.Position, name)
}
seen[name] = true
}
seen = map[string]bool{}
for _, tag := range r.Tags {
if strings.TrimSpace(tag) == "" || seen[tag] {
return nil, fmt.Errorf("%s: @doc tags must be nonempty and unique", r.Position)
}
seen[tag] = true
}
}
return &spec, nil
}
func decorateDocument(doc map[string]any, spec *Spec) {
if spec.Description != "" {
doc["info"].(map[string]any)["description"] = spec.Description
}
schemes := map[string]any{}
for _, a := range spec.Auth {
scheme := map[string]any{}
if a.Type == "bearer" {
scheme["type"], scheme["scheme"] = "http", "bearer"
if a.BearerFormat != "" {
scheme["bearerFormat"] = a.BearerFormat
}
} else {
scheme["type"], scheme["in"], scheme["name"] = "apiKey", a.In, a.KeyName
}
if a.Description != "" {
scheme["description"] = a.Description
}
schemes[a.Name] = scheme
}
doc["components"].(map[string]any)["securitySchemes"] = schemes
definitions := map[string]ErrorDefinition{}
for _, e := range spec.Errors {
definitions[e.Name] = e
}
if len(spec.Errors) > 0 {
doc["x-omni-errors"] = spec.Errors
}
for _, r := range spec.Routes {
policy := response.MustNew(responseConfig(spec, r))
cfg := policy.Config()
parts := strings.Split(r.Path, "/")
for i, part := range parts {
if strings.HasPrefix(part, ":") {
parts[i] = "{" + part[1:] + "}"
}
}
op := doc["paths"].(map[string]any)[strings.Join(parts, "/")].(map[string]any)[r.Method].(map[string]any)
decorateRequestOptions(op, r)
if r.ResponseFormat != "" {
op["x-omni-response"] = r.ResponseFormat
}
if len(r.Middleware) > 0 {
op["x-omni-middleware"] = r.Middleware
}
if r.Auth != "" && r.Auth != "none" {
op["security"] = []map[string][]string{{r.Auth: {}}}
} else {
op["security"] = []any{}
}
if r.Description != "" {
op["description"] = r.Description
}
if len(r.Tags) > 0 {
op["tags"] = r.Tags
} else {
op["tags"] = []string{r.Group}
}
if r.Deprecated {
op["deprecated"] = true
}
responses := op["responses"].(map[string]any)
if r.Auth != "" && r.Auth != "none" {
responses["401"] = genericErrorResponse("Authentication required", cfg)
}
grouped := map[int][]ErrorDefinition{}
for _, name := range r.Errors {
e := definitions[name]
grouped[e.Status] = append(grouped[e.Status], e)
}
if len(r.Errors) > 0 {
op["x-omni-errors"] = r.Errors
}
for status, errs := range grouped {
key := fmt.Sprint(status)
response, exists := responses[key].(map[string]any)
if !exists {
response = genericErrorResponse(http.StatusText(status), cfg)
responses[key] = response
}
media := response["content"].(map[string]any)["application/json"].(map[string]any)
examples := map[string]any{}
codes := []any{}
seenCodes := map[string]bool{}
description := []string{}
sort.Slice(errs, func(i, j int) bool { return errs[i].Name < errs[j].Name })
for _, e := range errs {
value := policy.Value(responseResult(e))
examples[e.Name] = map[string]any{"summary": e.Message, "value": value}
codeKey := fmt.Sprint(value[cfg.CodeField])
if !seenCodes[codeKey] {
codes = append(codes, value[cfg.CodeField])
seenCodes[codeKey] = true
}
description = append(description, e.Code+": "+e.Message)
}
media["examples"] = examples
if !exists && cfg.Body == nil && cfg.CodeField != "-" {
media["schema"].(map[string]any)["properties"].(map[string]any)[cfg.CodeField].(map[string]any)["enum"] = codes
}
response["description"] = strings.Join(description, "\n")
}
}
}
func responseResult(e ErrorDefinition) response.Result {
return response.Result{Status: e.Status, Code: e.Code, Message: e.Message, RequestID: "exampleRequest"}
}
func genericErrorResponse(description string, cfg response.Config) map[string]any {
if description == "" {
description = "Error"
}
schema, _ := openapi.EnvelopeWith(nil, cfg)
data, _ := json.Marshal(schema)
var value map[string]any
_ = json.Unmarshal(data, &value)
return map[string]any{"description": description, "content": map[string]any{"application/json": map[string]any{"schema": value}}}
}

View File

@@ -0,0 +1,165 @@
package apigen
import (
"fmt"
"strconv"
"strings"
)
// ErrorDefinition declares a public, stable error. Generated constructors return
// a fresh error each time so WithCause cannot mutate a shared singleton.
type ErrorDefinition struct {
Name string `json:"name"`
Status int `json:"status"`
Code string `json:"code"`
Message string `json:"message"`
Position string `json:"-"`
}
type AuthDefinition struct {
Name, Type, In, KeyName, Middleware, Description, BearerFormat, Position string
}
func (p *parser) parseErrors() error {
p.next()
block := p.peek() == "("
if block {
p.next()
p.skip()
}
for {
pos := p.tokens[p.at].position
name, err := p.ident()
if err != nil {
return err
}
status, err := strconv.Atoi(p.next().text)
if err != nil {
return p.err("error HTTP status must be an integer")
}
code, err := p.quoted()
if err != nil {
return err
}
message, err := p.quoted()
if err != nil {
return err
}
p.spec.Errors = append(p.spec.Errors, ErrorDefinition{Name: name, Status: status, Code: code, Message: message, Position: pos})
if p.peek() != "\n" && p.peek() != ";" && !(block && p.peek() == ")") && p.peek() != "" {
return p.err("expected end of error declaration")
}
if !block {
return nil
}
p.skip()
if p.peek() == ")" {
p.next()
return nil
}
}
}
func (p *parser) parseAuth() error {
pos := p.next().position
name, err := p.ident()
if err != nil {
return err
}
p.skip()
props, err := p.metadata()
if err != nil {
return err
}
a := AuthDefinition{Name: name, Position: pos}
for key, value := range props {
switch key {
case "type":
a.Type = value
case "in":
a.In = value
case "name":
a.KeyName = value
case "middleware":
a.Middleware = value
case "description":
a.Description = value
case "bearerFormat":
a.BearerFormat = value
default:
return fmt.Errorf("%s: unsupported auth property %q", pos, key)
}
}
p.spec.Auth = append(p.spec.Auth, a)
return nil
}
func (p *parser) names() ([]string, error) {
var names []string
for {
name, err := p.ident()
if err != nil {
return nil, err
}
for _, prior := range names {
if prior == name {
return nil, p.err("duplicate name " + name)
}
}
names = append(names, name)
if p.peek() != "," {
return names, nil
}
p.next()
}
}
func commaList(value string) []string {
if value == "" {
return nil
}
result := strings.Split(value, ",")
for i := range result {
result[i] = strings.TrimSpace(result[i])
}
return result
}
func mergeNames(base, extra []string) []string {
result := append([]string(nil), base...)
for _, name := range extra {
found := false
for _, prior := range result {
found = found || prior == name
}
if !found {
result = append(result, name)
}
}
return result
}
func (p *parser) parseDoc(route *Route) error {
if p.peek() != "(" {
value, err := p.quoted()
route.Summary = value
return err
}
pos := p.tokens[p.at].position
props, err := p.metadata()
if err != nil {
return err
}
for key, value := range props {
switch key {
case "summary":
route.Summary = value
case "description":
route.Description = value
case "tags":
route.Tags = commaList(value)
case "deprecated":
if value != "true" && value != "false" {
return fmt.Errorf("%s: deprecated must be true or false", pos)
}
route.Deprecated = value == "true"
default:
return fmt.Errorf("%s: unsupported @doc property %q", pos, key)
}
}
return nil
}

View File

@@ -0,0 +1,545 @@
package apigen
import (
"encoding/json"
"errors"
"fmt"
"go/format"
"os"
"path/filepath"
"runtime/debug"
"sort"
"strconv"
"strings"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
"golang.org/x/mod/modfile"
"golang.org/x/mod/module"
"golang.org/x/mod/semver"
)
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
const minimumVersion = "v0.9.0-preview"
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
type Options struct {
Database, Storage string
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
Check bool
DryRun bool
}
type FileChange struct {
Path string
Action string
}
type Output struct {
Generated, Created, Preserved, Removed []string
Changes []FileChange
}
// Generate creates or safely updates one contract-driven project. It renders
// and validates every output before applying an ownership-checked transaction.
func Generate(o Options) (Output, error) {
if o.Check && o.DryRun {
return Output{}, errors.New("--check and --dry-run cannot be combined")
}
if o.API == "" || o.Dir == "" {
return Output{}, errors.New("--api and --dir are required")
}
spec, err := Parse(o.API)
if err != nil {
return Output{}, err
}
compiled, err := Compile(spec)
if err != nil {
return Output{}, err
}
return generate(o, compiled, false)
}
const starterAPI = "syntax = \"v1\"\n\ninfo (\n title: \"Greeting API\"\n version: \"1.0.0\"\n)\n\ntype GreetReq {\n Name string `path:\"name\" binding:\"required,max=80\"`\n}\ntype GreetResp {\n Message string `json:\"message\"`\n}\n\n@server (\n group: greet\n prefix: /api\n)\nservice greet-api {\n @doc \"Greet a visitor\"\n @handler greet\n get /greet/:name (GreetReq) returns (GreetResp)\n}\n"
// New creates a runnable greeting project, including its editable API file.
func New(o Options) (Output, error) {
if o.API != "" || o.Check || o.DryRun {
return Output{}, errors.New("api new does not accept --api, --check or --dry-run")
}
if o.Dir == "" {
return Output{}, errors.New("project directory required")
}
if entries, err := os.ReadDir(o.Dir); err == nil && len(entries) > 0 {
return Output{}, errors.New("api new requires an empty directory")
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return Output{}, err
}
tokens, err := lex("service.api", starterAPI)
if err != nil {
return Output{}, err
}
spec := &Spec{Version: "1.0.0", Entry: "service.api", Sources: map[string][]byte{"service.api": []byte(starterAPI)}}
p := parser{tokens: tokens, spec: spec}
if err = p.parse(); err != nil {
return Output{}, err
}
c, err := Compile(spec)
if err != nil {
return Output{}, err
}
return generate(o, c, true)
}
func generate(o Options, c *Compiled, starter bool) (Output, error) {
directory, err := filepath.Abs(o.Dir)
if err != nil {
return Output{}, err
}
o.Dir = directory
for _, r := range c.Spec.Routes {
if !r.Manual {
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
if _, err := os.Lstat(path); err == nil {
return Output{}, fmt.Errorf("manual handler %s exists; migrate and remove it explicitly before switching to automatic generation", path)
} else if !errors.Is(err, os.ErrNotExist) {
return Output{}, err
}
}
}
if err := resolveProfile(&o); err != nil {
return Output{}, err
}
seeds := map[string][]byte{}
moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod"))
if readErr == nil {
mf, err := modfile.Parse("go.mod", moduleData, nil)
if err != nil || mf.Module == nil {
return Output{}, errors.New("invalid project go.mod")
}
if o.Module != "" && o.Module != mf.Module.Mod.Path {
return Output{}, errors.New("--module differs from existing go.mod")
}
o.Module = mf.Module.Mod.Path
found := false
requiredVersion := ""
for _, r := range mf.Require {
if r.Mod.Path == frameworkModule {
found = true
requiredVersion = r.Mod.Version
if o.FrameworkVersion != "" && o.FrameworkVersion != r.Mod.Version {
return Output{}, errors.New("existing framework version differs; update go.mod explicitly")
}
}
}
if !found {
return Output{}, errors.New("existing go.mod must require go-omni")
}
if semver.Compare(requiredVersion, minimumVersion) < 0 {
localCurrent := false
for _, r := range mf.Replace {
if r.Old.Path == frameworkModule && r.New.Version == "" {
path := r.New.Path
if !filepath.IsAbs(path) {
path = filepath.Join(directory, path)
}
if _, err := os.Stat(filepath.Join(path, "contract", "scalar.go")); err == nil {
localCurrent = true
}
}
}
if !localCurrent {
return Output{}, fmt.Errorf("existing framework dependency predates migration extensions; use %s or a current local replacement", minimumVersion)
}
}
if o.FrameworkDir != "" {
matched := false
for _, r := range mf.Replace {
if r.Old.Path == frameworkModule && r.New.Version == "" {
replacement := r.New.Path
if !filepath.IsAbs(replacement) {
replacement = filepath.Join(directory, replacement)
}
matched = canonical(replacement) == canonical(o.FrameworkDir)
}
}
if !matched {
return Output{}, errors.New("existing framework replacement differs; update go.mod explicitly")
}
}
} else if !errors.Is(readErr, os.ErrNotExist) {
return Output{}, readErr
} else {
if o.Module == "" {
o.Module = filepath.Base(directory)
}
data, err := moduleFile(o)
if err != nil {
return Output{}, err
}
seeds["go.mod"] = data
}
if err = module.CheckImportPath(o.Module); err != nil {
return Output{}, err
}
generated, base, err := renderProject(o, c, starter)
if err != nil {
return Output{}, err
}
for name, data := range base {
seeds[name] = data
}
// The API source remains developer-owned. A relocated project regenerates
// from its own api/ tree, including all imports.
entry := "api/" + c.Spec.Entry
for name, data := range c.Spec.Sources {
seeds["api/"+name] = data
}
if o.API != "" {
source, _ := filepath.Abs(o.API)
sourceReal, _ := filepath.EvalSymlinks(source)
targetReal := canonical(directory)
rel, e := filepath.Rel(targetReal, sourceReal)
if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
entry = filepath.ToSlash(rel)
for name := range c.Spec.Sources {
delete(seeds, "api/"+name)
}
}
}
// Source collisions must never leave a copied contract different from the
// generated code. On updates, use the developer-owned API in the project.
for name, data := range seeds {
if strings.HasSuffix(name, ".api") {
current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name)))
if e == nil && string(current) != string(data) {
return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name)
}
}
}
seeds["README.md"] = []byte(projectREADME(o.Module, entry) + infrastructureREADME(o))
for _, files := range []map[string][]byte{generated, seeds} {
for name, data := range files {
if strings.HasSuffix(name, ".go") {
data, err = format.Source(data)
if err != nil {
return Output{}, fmt.Errorf("format %s: %w", name, err)
}
files[name] = data
}
}
}
return apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile())
}
func moduleFile(o Options) ([]byte, error) {
if err := module.CheckImportPath(o.Module); err != nil {
return nil, err
}
root := o.FrameworkDir
version := o.FrameworkVersion
if root == "" && version == "" {
wd, _ := os.Getwd()
for wd != "" {
data, err := os.ReadFile(filepath.Join(wd, "go.mod"))
if err == nil {
mf, e := modfile.Parse("go.mod", data, nil)
if e == nil && mf.Module != nil && mf.Module.Mod.Path == frameworkModule {
root = wd
}
break
}
parent := filepath.Dir(wd)
if parent == wd {
break
}
wd = parent
}
}
if version == "" {
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
version = info.Main.Version
} else {
version = minimumVersion
}
}
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {
return nil, errors.New("invalid framework version")
}
if root == "" && semver.Compare(version, minimumVersion) < 0 {
return nil, fmt.Errorf("API generation needs %s or newer; use --framework-dir for unpublished source", minimumVersion)
}
text := "module " + o.Module + "\n\ngo 1.22.0\n\nrequire " + frameworkModule + " " + version + "\n"
if root != "" {
root = canonical(root)
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
if err != nil {
return nil, err
}
mf, err := modfile.Parse("go.mod", data, nil)
if err != nil || mf.Module == nil || mf.Module.Mod.Path != frameworkModule {
return nil, errors.New("--framework-dir must contain go-omni")
}
if _, err = os.Stat(filepath.Join(root, "contract", "contract.go")); err != nil {
return nil, errors.New("local framework lacks API contract support")
}
relative, err := filepath.Rel(canonical(o.Dir), root)
if err != nil {
return nil, err
}
text += "\nreplace " + frameworkModule + " => " + strconv.Quote(filepath.ToSlash(relative)) + "\n"
}
return []byte(text), nil
}
func canonical(path string) string {
path, _ = filepath.Abs(path)
candidate := path
var tail []string
for {
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
for i := len(tail) - 1; i >= 0; i-- {
resolved = filepath.Join(resolved, tail[i])
}
return resolved
}
parent := filepath.Dir(candidate)
if parent == candidate {
return path
}
tail = append(tail, filepath.Base(candidate))
candidate = parent
}
}
func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map[string][]byte, error) {
generated, seeds := map[string][]byte{}, map[string][]byte{}
put := func(files map[string][]byte, name, content string) { files[name] = []byte(content) }
if len(c.Spec.Errors) > 0 {
content := "package apierrors\nimport omnierrors " + strconv.Quote(frameworkModule+"/errors") + "\n"
for _, e := range c.Spec.Errors {
content += fmt.Sprintf("const Code%s = %q\n// %s returns a fresh public error declared in the API contract.\nfunc %s()*omnierrors.APIError{return omnierrors.New(%d,Code%s,%q)}\n", e.Name, e.Code, e.Name, e.Name, e.Status, e.Name, e.Message)
}
put(generated, "internal/apierrors/errorsGen.go", generatedHeader+content)
}
types := "package types\n\n"
if len(c.Spec.Scalars) > 0 {
types += "import("
for i, d := range c.Spec.Scalars {
types += fmt.Sprintf("scalar%d %q;", i, d.GoImport)
}
types += ")\n"
registry := fmt.Sprintf("package types\nimport %q\nfunc ContractScalars()[]contract.Scalar{return []contract.Scalar{", frameworkModule+"/contract")
for i, d := range c.Spec.Scalars {
types += fmt.Sprintf("type %s = scalar%d.%s\n", d.Name, i, d.GoType)
registry += fmt.Sprintf("contract.ScalarOf[%s](%q,%q,%t),", d.Name, d.JSONInput, d.JSONOutput, d.Text)
}
registry += "}}\n"
put(generated, "internal/types/scalarsGen.go", generatedHeader+registry)
}
for _, t := range c.Spec.Types {
types += "type " + t.Name + " struct {\n"
for _, f := range t.Fields {
types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag)
}
types += "}\n"
}
put(generated, "internal/types/typesGen.go", generatedHeader+types)
put(generated, "docs/openapi.json", string(c.Document)+"\n")
put(generated, "internal/handler/openapiGen.go", generatedHeader+fmt.Sprintf("package handler\nimport (\"net/http\"; %q)\nconst OpenAPIDocument = %q\nfunc OpenAPI(c *httpx.Context) error { return c.WriteResponse(200,http.Header{\"Content-Type\": {\"application/json; charset=utf-8\"}},[]byte(OpenAPIDocument)) }\n", frameworkModule+"/httpx", string(c.Document)+"\n"))
policySource := fmt.Sprintf("package handler\nimport(\"encoding/json\";\"strings\";%q)\nvar responseTemplates=map[string]response.Encoder{\"\":response.Default(),\n", frameworkModule+"/response")
for _, definition := range c.Spec.Responses {
data, err := json.Marshal(definition.Config)
if err != nil {
return nil, nil, err
}
policySource += fmt.Sprintf("%q: newResponseTemplate(%q),\n", definition.Name, string(data))
}
policySource += "}\nfunc newResponseTemplate(raw string)response.Encoder{var cfg response.Config;decoder:=json.NewDecoder(strings.NewReader(raw));decoder.UseNumber();if err:=decoder.Decode(&cfg);err!=nil{panic(err)};return response.MustNew(cfg)}\nfunc responseEncoder(name string,raw bool)response.Encoder{encoder:=responseTemplates[name];if encoder==nil{panic(\"unknown response template\")};if raw{encoder=response.RawJSON(encoder)};return encoder}\n"
put(generated, "internal/handler/responseGen.go", generatedHeader+policySource)
imports := fmt.Sprintf("import (\"fmt\"; %q; %q; %q;", frameworkModule+"/httpx", frameworkModule+"/openapi", o.Module+"/internal/servicecontext")
routes := "func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {\n if router==nil || deps==nil {return fmt.Errorf(\"router and dependencies required\")}\n"
if len(c.Spec.Scalars) > 0 {
imports += fmt.Sprintf("%q;%q;", frameworkModule+"/contract", o.Module+"/internal/types")
routes += "if err:=contract.ValidateScalars(types.ContractScalars()...);err!=nil{return err}\n"
}
middlewareSet := map[string]bool{}
for _, r := range c.Spec.Routes {
for _, m := range r.Middleware {
middlewareSet[m] = true
}
}
var middlewareNames []string
for m := range middlewareSet {
middlewareNames = append(middlewareNames, m)
}
sort.Strings(middlewareNames)
for _, m := range middlewareNames {
routes += fmt.Sprintf("if deps.Middlewares[%q]==nil {return fmt.Errorf(%q)}\n", m, "missing middleware "+m)
}
for i, group := range c.Groups {
imports += fmt.Sprintf("h%d %q; s%d %q;", i, o.Module+"/internal/handler/"+group, i, o.Module+"/internal/service/"+group)
routes += fmt.Sprintf("h%d := h%d.New(s%d.New(deps),deps)\n", i, i, i)
groupRoutes := []Route{}
for _, r := range c.Spec.Routes {
if r.Group == group {
groupRoutes = append(groupRoutes, r)
}
}
packageName := filepath.Base(group)
text := "package " + packageName + "\nimport(\"context\"; \"fmt\";"
needsTypes, needsBinder, hasAutomatic := false, false, false
for _, r := range groupRoutes {
if r.Manual {
continue
}
hasAutomatic = true
needsTypes = needsTypes || r.Request != "" || r.Response != ""
needsBinder = needsBinder || (r.Request != "" && !r.CustomBinding)
}
text += fmt.Sprintf("%q;%q;", frameworkModule+"/httpx", o.Module+"/internal/servicecontext")
if needsTypes {
text += fmt.Sprintf("%q;", o.Module+"/internal/types")
}
if needsBinder {
text += fmt.Sprintf("%q;", frameworkModule+"/contract")
}
text += ")\n"
text += "type Service interface {\n"
for _, r := range groupRoutes {
if !r.Manual {
text += signature(r) + "\n"
}
}
text += "}\ntype Handler struct {service Service;deps *servicecontext.Context}\nfunc New(s Service,deps ...*servicecontext.Context)*Handler {h:=&Handler{service:s};if len(deps)>0{h.deps=deps[0]};return h}\n"
for _, r := range groupRoutes {
name := upper(r.Handler)
// The response policy and body reader cap are installed before App middleware.
if r.MaxBodyBytes > 0 {
routes += fmt.Sprintf("httpx.HandleWithBodyLimit(router,%q,%q,%d,responseEncoder(%q,%t),h%d.%s", strings.ToUpper(r.Method), r.Path, r.MaxBodyBytes, responseName(r), r.ResponseFormat == "raw", i, name)
} else {
routes += fmt.Sprintf("httpx.HandleWithResponse(router,%q,%q,responseEncoder(%q,%t),h%d.%s", strings.ToUpper(r.Method), r.Path, responseName(r), r.ResponseFormat == "raw", i, name)
}
for _, m := range r.Middleware {
routes += fmt.Sprintf(",deps.Middlewares[%q]", m)
}
routes += ")\n"
if r.Manual {
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Manual.go", fmt.Sprintf("package %s\nimport(%q;apierrors %q)\n// %s owns raw/multipart binding and response output. h.deps holds project dependencies.\n// This file is preserved on regeneration.\nfunc(h *Handler)%s(c *httpx.Context)error{return apierrors.New(501,\"NOT_IMPLEMENTED\",\"manual handler is not implemented\")}\n", packageName, frameworkModule+"/httpx", frameworkModule+"/errors", name, name))
continue
}
if r.Request != "" && !r.CustomBinding {
scalarArgs := ""
if len(c.Spec.Scalars) > 0 {
scalarArgs = "types.ContractScalars()..."
}
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s](%s)\n", lower(r.Handler), r.Request, scalarArgs)
}
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
call := "c.Context()"
if r.Request != "" {
if r.CustomBinding {
text += fmt.Sprintf("var req types.%s\nif err:=h.bind%s(c,&req);err!=nil{return err}\n", r.Request, name)
put(seeds, "internal/handler/"+group+"/"+lower(r.Handler)+"Bind.go", fmt.Sprintf("package %s\nimport(%q;%q;apierrors %q)\n// bind%s is project-owned; repeated generation preserves it.\nfunc(h *Handler)bind%s(c *httpx.Context,req *types.%s)error{return apierrors.New(501,\"BINDING_NOT_IMPLEMENTED\",\"custom binding is not implemented\")}\n", packageName, frameworkModule+"/httpx", o.Module+"/internal/types", frameworkModule+"/errors", name, name, r.Request))
} else {
limit := ""
if r.MaxBodyBytes > 0 {
limit = fmt.Sprintf(",%d", r.MaxBodyBytes)
}
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req%s);err!=nil{return err}\n", r.Request, lower(r.Handler), limit)
}
call += ", &req"
}
if r.Response != "" {
text += fmt.Sprintf("resp,err:=h.service.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name)
method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status]
text += "return c." + method + "(resp)\n}\n"
} else {
text += fmt.Sprintf("if err:=h.service.%s(%s);err!=nil{return err};return c.NoContent()\n}\n", name, call)
}
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
if r.Request != "" || r.Response != "" {
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
}
service += ")\n"
service += "// " + name + " implements the business operation. This file is never overwritten.\nfunc(s *Service) " + signature(r) + " {\n"
if starter && r.Handler == "greet" {
service = strings.Replace(service, "apierrors "+strconv.Quote(frameworkModule+"/errors")+";", "\"fmt\";", 1)
service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}"
} else {
service += "// TODO: implement business behavior and use s.deps for dependencies.\n"
if r.Response != "" {
service += "return nil, "
} else {
service += "return "
}
service += "apierrors.New(501,\"NOT_IMPLEMENTED\",\"operation is not implemented\")\n}"
}
put(seeds, "internal/service/"+group+"/"+lower(r.Handler)+".go", service)
}
if !hasAutomatic {
text = strings.Replace(text, strconv.Quote(frameworkModule+"/httpx")+";", "", 1)
text = strings.Replace(text, "\"context\";", "", 1)
}
if !hasResponse(groupRoutes) {
text = strings.Replace(text, "\"fmt\";", "", 1)
}
put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text)
put(seeds, "internal/service/"+group+"/service.go", fmt.Sprintf("package %s\nimport %q\ntype Service struct {deps *servicecontext.Context}\nfunc New(deps *servicecontext.Context)*Service{return &Service{deps:deps}}\n", packageName, o.Module+"/internal/servicecontext"))
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
}
put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n")
registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n"
for _, m := range middlewareNames {
registry += fmt.Sprintf("%q:%s,\n", m, upper(m))
put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m)))
}
put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n")
for name, source := range baseTemplates {
content := strings.ReplaceAll(source, "MODULE", o.Module)
content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat)
content = strings.ReplaceAll(content, "LOGOUTPUT", o.LogOutput)
if name == "etc/config.yaml" && o.LogOutput == "file" {
content = strings.Replace(content, "output: file", "output: stderr\n file: ./var/log/server.log\n rotation:\n enabled: true\n maxSizeMb: 100\n maxBackups: 10\n maxAgeDays: 7\n compress: true", 1)
}
put(seeds, name, content)
}
infra, err := scaffold.APIInfrastructure(o.Module, o.Database, o.Storage, o.LogOutput, o.LogFormat)
if err != nil {
return nil, nil, err
}
for name, data := range infra {
seeds[name] = data
}
if o.Storage == "local" {
for _, r := range c.Spec.Routes {
if r.Path == "/storage/object" {
return nil, nil, errors.New("/storage/object is reserved by local storage")
}
}
main := string(seeds["cmd/server/main.go"])
main = strings.Replace(main, " if err=handler.RegisterRoutes", ` for _,method:=range []string{"GET","HEAD","PUT"}{app.Handle(method,"/storage/object",func(c *httpx.Context)error{deps.FileHTTP.ServeHTTP(c.Raw(),c.Request());return nil})}
if err=handler.RegisterRoutes`, 1)
main = strings.Replace(main, `"git.yuntuo.hk/k2safe/go-omni/logx"`, `"git.yuntuo.hk/k2safe/go-omni/logx"; "git.yuntuo.hk/k2safe/go-omni/httpx"`, 1)
seeds["cmd/server/main.go"] = []byte(main)
}
return generated, seeds, nil
}
func hasResponse(routes []Route) bool {
for _, r := range routes {
if r.Response != "" && !r.Manual {
return true
}
}
return false
}
func signature(r Route) string {
s := upper(r.Handler) + "(ctx context.Context"
if r.Request != "" {
s += ",req *types." + r.Request
}
s += ") "
if r.Response != "" {
s += "(*types." + r.Response + ",error)"
} else {
s += "error"
}
return s
}

View File

@@ -0,0 +1,305 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
const declaredAPI = `syntax = "v1"
info (
title: "Declared API"
description: "Errors and authentication share one contract"
)
error (
Unauthenticated 401 "UNAUTHENTICATED" "authentication required"
Forbidden 403 "FORBIDDEN" "permission denied"
EmailTaken 409 "EMAIL_TAKEN" "email already in use"
NameTaken 409 "NAME_TAKEN" "name already in use"
)
auth UserAuth (
type: bearer
middleware: Auth
description: "Verified application token"
)
auth PartnerAuth (
type: apiKey
in: header
name: X-API-Key
middleware: Partner
)
auth SessionAuth (
type: apiKey
in: cookie
name: sessionId
middleware: Session
)
@server (
group: user
auth: UserAuth
errors: Unauthenticated
middleware: Audit
)
service declared-api {
@doc (
summary: "Create a user"
description: "Uniqueness is enforced in a transaction"
tags: users, administration
deprecated: true
)
@errors Forbidden, EmailTaken, NameTaken
@handler create
post /users
@auth none
@errors none
@handler public
get /public
@auth PartnerAuth
@handler partner
get /partner
@auth SessionAuth
@handler session
get /session
}
`
func document(t *testing.T, c *Compiled) map[string]any {
t.Helper()
var doc map[string]any
if err := json.Unmarshal(c.Document, &doc); err != nil {
t.Fatal(err)
}
return doc
}
func operation(t *testing.T, doc map[string]any, path, method string) map[string]any {
t.Helper()
return doc["paths"].(map[string]any)[path].(map[string]any)[method].(map[string]any)
}
func TestDeclaredErrorsAndAuthentication(t *testing.T) {
spec, err := parseText(t, declaredAPI)
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
again, err := Compile(spec)
if err != nil || string(c.Document) != string(again.Document) {
t.Fatalf("compile is not repeatable: %v", err)
}
if strings.Join(spec.Routes[0].Middleware, ",") != "Audit" {
t.Fatal("parser output mutated")
}
if strings.Join(c.Spec.Routes[0].Middleware, ",") != "Auth,Audit" {
t.Fatal("authentication must be first")
}
if strings.Join(c.Spec.Routes[1].Middleware, ",") != "Audit" {
t.Fatal("public override retained automatic auth")
}
doc := document(t, c)
op := operation(t, doc, "/users", "post")
if op["deprecated"] != true || op["description"] == nil || len(op["tags"].([]any)) != 2 {
t.Fatalf("documentation missing: %v", op)
}
security := op["security"].([]any)
if _, ok := security[0].(map[string]any)["UserAuth"]; !ok {
t.Fatal("security requirement absent")
}
schemes := doc["components"].(map[string]any)["securitySchemes"].(map[string]any)
if len(schemes) != 3 || schemes["UserAuth"].(map[string]any)["scheme"] != "bearer" || schemes["SessionAuth"].(map[string]any)["in"] != "cookie" {
t.Fatalf("wrong security schemes: %v", schemes)
}
responses := op["responses"].(map[string]any)
for _, status := range []string{"401", "403", "409"} {
if responses[status] == nil {
t.Fatal("missing status " + status)
}
}
media := responses["409"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
if len(media["examples"].(map[string]any)) != 2 {
t.Fatal("same-status errors were overwritten")
}
codes := media["schema"].(map[string]any)["properties"].(map[string]any)["code"].(map[string]any)["enum"].([]any)
if len(codes) != 2 {
t.Fatal("business code enum absent")
}
public := operation(t, doc, "/public", "get")
if len(public["security"].([]any)) != 0 || public["responses"].(map[string]any)["401"] != nil || public["x-omni-errors"] != nil {
t.Fatal("public overrides did not clear group declarations")
}
}
func TestInvalidDeclarations(t *testing.T) {
replacements := [][2]string{
{`"EMAIL_TAKEN"`, `"UNAUTHENTICATED"`},
{`EmailTaken 409`, `EmailTaken 200`},
{`EmailTaken 409`, `emailTaken 409`},
{`NameTaken 409`, `CodeEmailTaken 409`},
{`auth: UserAuth`, `auth: Unknown`},
{`errors: Unauthenticated`, `errors: Unknown`},
{`middleware: Audit`, `middleware: Auth`},
{`type: bearer`, `type: oauth2`},
{`in: header`, `in: query`},
{`name: X-API-Key`, `name: Authorization`},
{`deprecated: true`, `deprecated: yes`},
{`tags: users, administration`, `tags: users, users`},
{`@errors Forbidden, EmailTaken, NameTaken`, `@errors Forbidden, Forbidden`},
{`middleware: Auth`, `middleware: Defaults`},
}
for _, change := range replacements {
spec, err := parseText(t, strings.Replace(declaredAPI, change[0], change[1], 1))
if err == nil {
_, err = Compile(spec)
}
if err == nil {
t.Fatalf("accepted invalid %q", change[1])
}
if !strings.Contains(err.Error(), "service.api:") {
t.Fatalf("source position missing: %v", err)
}
}
}
func TestImportedErrorCatalog(t *testing.T) {
root := t.TempDir()
writeFile(t, filepath.Join(root, "errors.api"), `error Conflict 409 "CONFLICT" "conflict"`)
writeFile(t, filepath.Join(root, "service.api"), "import \"errors.api\"\nservice imported-api { @errors Conflict\n @handler create\n post /users\n }")
spec, err := Parse(filepath.Join(root, "service.api"))
if err != nil {
t.Fatal(err)
}
if _, err = Compile(spec); err != nil {
t.Fatal(err)
}
}
func TestDryRunAndEntryDiscovery(t *testing.T) {
root := t.TempDir()
source := filepath.Join(root, "input/service.api")
writeFile(t, source, declaredAPI)
dir := filepath.Join(root, "project")
o := Options{API: source, Dir: dir, Module: "example.com/dx", FrameworkDir: frameworkRoot(t), DryRun: true}
out, err := Generate(o)
if err != nil || len(out.Changes) == 0 {
t.Fatalf("empty plan: %v", err)
}
if _, err = os.Stat(dir); !os.IsNotExist(err) {
t.Fatal("dry-run created directory")
}
o.DryRun = false
if _, err = Generate(o); err != nil {
t.Fatal(err)
}
entry, err := ProjectEntry(dir)
if err != nil || entry != filepath.Join(dir, "api/service.api") {
t.Fatalf("entry: %s %v", entry, err)
}
o = Options{API: entry, Dir: dir, DryRun: true}
before := snapshot(t, dir)
out, err = Generate(o)
if err != nil || len(out.Changes) != 0 {
t.Fatalf("unchanged project plan: %v %+v", err, out.Changes)
}
writeFile(t, entry, strings.Replace(declaredAPI, "email already in use", "email is already registered", 1))
before = snapshot(t, dir)
out, err = Generate(o)
if err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("dry-run changed project")
}
actions := map[string]string{}
for _, c := range out.Changes {
actions[c.Path] = c.Action
}
if actions["internal/apierrors/errorsGen.go"] != "update" || actions["docs/openapi.json"] != "update" {
t.Fatalf("missing plan: %v", actions)
}
o.Check = true
if _, err = Generate(o); err == nil {
t.Fatal("check and dry-run combined")
}
// A malformed manifest cannot redirect the shorthand outside its project.
writeFile(t, filepath.Join(dir, manifestPath), `{"version":1,"entry":"../input/service.api"}`)
if _, err = ProjectEntry(dir); err == nil {
t.Fatal("unsafe manifest entry accepted")
}
}
func TestDeclaredContractConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
t.Skip("consumer integration")
}
root := t.TempDir()
source := filepath.Join(root, "input/service.api")
writeFile(t, source, declaredAPI)
dir := filepath.Join(root, "project")
if _, err := Generate(Options{API: source, Dir: dir, Module: "example.com/dx", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
writeFile(t, filepath.Join(dir, "contract_test.go"), declaredConsumerTest)
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("consumer: %v\n%s", err, out)
}
}
const declaredConsumerTest = `package dx_test
import (
"context";"errors";"io";"net/http/httptest";"strings";"testing"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"example.com/dx/internal/apierrors"
"example.com/dx/internal/handler"
"example.com/dx/internal/servicecontext"
)
func TestRuntimeAuthAndFreshErrors(t *testing.T){
first:=apierrors.EmailTaken().WithCause(errors.New("private"));second:=apierrors.EmailTaken()
if first==second||second.Cause!=nil||second.Code!=apierrors.CodeEmailTaken||second.Status!=409{t.Fatal("shared error instance")}
cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close()
deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close()
app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,httptest.NewRequest("POST","/users",nil))
if w.Code!=501||!strings.Contains(w.Body.String(),"MIDDLEWARE_NOT_IMPLEMENTED"){t.Fatal("unimplemented auth did not fail closed")}
var called []string
deps.Middlewares["Auth"]=func(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error{called=append(called,"auth");if c.Header("Authorization")!="Bearer valid"{return apierrors.Unauthenticated()};return next(c)}}
deps.Middlewares["Audit"]=func(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error{called=append(called,"audit");return next(c)}}
app2:=omni.New();defer app2.Close();if err=handler.RegisterRoutes(app2,deps);err!=nil{t.Fatal(err)}
w=httptest.NewRecorder();app2.Engine().ServeHTTP(w,httptest.NewRequest("POST","/users",nil))
if w.Code!=401||strings.Join(called,",")!="auth"{t.Fatal("unauthenticated request reached other middleware")}
called=nil;req:=httptest.NewRequest("POST","/users",nil);req.Header.Set("Authorization","Bearer valid")
w=httptest.NewRecorder();app2.Engine().ServeHTTP(w,req)
if w.Code!=501||strings.Join(called,",")!="auth,audit"{t.Fatal("middleware order changed")}
called=nil;w=httptest.NewRecorder();app2.Engine().ServeHTTP(w,httptest.NewRequest("GET","/public",nil))
if w.Code!=501||strings.Join(called,",")!="audit"||strings.Contains(w.Body.String(),"MIDDLEWARE_NOT_IMPLEMENTED"){t.Fatal("public override retained auth")}
delete(deps.Middlewares,"Auth");app3:=omni.New();defer app3.Close()
if err=handler.RegisterRoutes(app3,deps);err==nil{t.Fatal("missing auth registration accepted")}
w=httptest.NewRecorder();app3.Engine().ServeHTTP(w,httptest.NewRequest("GET","/public",nil))
if w.Code!=404{t.Fatal("failed auth preflight registered partial routes")}
}
`
func TestAuthMetadataWithParameterNamePrefixes(t *testing.T) {
source := "type Req {\n ID int `path:\"id\"`\n IDLong int `path:\"idLong\"`\n}\nauth Gate (\n type: bearer\n middleware: Auth\n)\n@server (auth: Gate)\nservice prefix-api {\n @handler read\n get /items/:id/children/:idLong (Req)\n}"
spec, err := parseText(t, source)
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
op := operation(t, document(t, c), "/items/{id}/children/{idLong}", "get")
if len(op["security"].([]any)) != 1 {
t.Fatal("auth metadata lost")
}
}

570
cmd/go-omni/apigen/parse.go Normal file
View File

@@ -0,0 +1,570 @@
// Package apigen implements go-omni's deliberately bounded go-zero-style API
// dialect. It generates Omni handlers, never go-zero or transport dependencies.
package apigen
import (
"errors"
"fmt"
"io"
"os"
"path/filepath"
"strconv"
"strings"
"text/scanner"
)
type Spec struct {
Title, Version, Service, Description string
Errors []ErrorDefinition
Auth []AuthDefinition
Responses []ResponseDefinition
Scalars []ScalarDefinition
Types []Type
Routes []Route
Sources map[string][]byte // relative to the entry API's directory
Entry string
}
type Type struct {
Name string
Fields []Field
Position string
}
type Field struct{ Name, Type, Tag string }
type Route struct {
Manual, CustomBinding bool
BodyMode, Produces string
MaxBodyBytes int64
Method, Path, Request, Response, Handler, Group, Summary string
Middleware []string
Auth string
ResponseFormat string
ErrorResponseFormat string
Errors []string
Description string
Tags []string
Deprecated bool
Status int
Position string
}
type token struct {
text, position string
kind rune
space bool
}
type parser struct {
tokens []token
at int
spec *Spec
imports []string
}
func Parse(path string) (*Spec, error) {
entry, err := filepath.Abs(path)
if err != nil {
return nil, err
}
entry, err = filepath.EvalSymlinks(entry)
if err != nil {
return nil, err
}
root := filepath.Dir(entry)
spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
visiting, visited := map[string]bool{}, map[string]bool{}
total := 0
var visit func(string, int) error
visit = func(name string, depth int) error {
if depth > 32 || len(visited)+len(visiting) >= 64 {
return errors.New("api: import limit exceeded")
}
real, err := filepath.EvalSymlinks(name)
if err != nil {
return err
}
rel, err := filepath.Rel(root, real)
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
return errors.New("api: imports must stay within the entry API directory")
}
if filepath.Ext(real) != ".api" {
return errors.New("api: expected .api file")
}
if visiting[real] {
return fmt.Errorf("api: circular import %s", rel)
}
if visited[real] {
return nil
}
visiting[real] = true
defer delete(visiting, real)
f, err := os.Open(real)
if err != nil {
return err
}
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
_ = f.Close()
if err != nil {
return err
}
total += len(data)
if len(data) > 1<<20 || total > 8<<20 {
return errors.New("api: source size limit exceeded")
}
tokens, err := lex(rel, string(data))
if err != nil {
return err
}
p := &parser{tokens: tokens, spec: spec}
if err = p.parse(); err != nil {
return err
}
spec.Sources[filepath.ToSlash(rel)] = data
for _, imp := range p.imports {
if filepath.IsAbs(imp) || strings.Contains(imp, "\\") {
return errors.New("api: imports must use relative slash paths")
}
if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil {
return err
}
}
visited[real] = true
return nil
}
if err = visit(entry, 0); err != nil {
return nil, err
}
if spec.Service == "" || len(spec.Routes) == 0 {
return nil, errors.New("api: at least one service route required")
}
if spec.Title == "" {
spec.Title = spec.Service
}
return spec, nil
}
func lex(name, source string) ([]token, error) {
var s scanner.Scanner
s.Init(strings.NewReader(source))
s.Filename = name
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments
s.Whitespace ^= 1 << '\n'
var scanErr error
s.Error = func(s *scanner.Scanner, message string) {
if scanErr == nil {
scanErr = fmt.Errorf("%s: %s", s.Position, message)
}
}
var out []token
end := 0
for {
kind := s.Scan()
out = append(out, token{s.TokenText(), s.Position.String(), kind, s.Position.Offset > end})
end = s.Position.Offset + len(s.TokenText())
if kind == scanner.EOF {
break
}
}
return out, scanErr
}
func (p *parser) peek() string { return p.tokens[p.at].text }
func (p *parser) next() token {
t := p.tokens[p.at]
if t.kind != scanner.EOF {
p.at++
}
return t
}
func (p *parser) skip() {
for p.peek() == "\n" || p.peek() == ";" {
p.next()
}
}
func (p *parser) err(message string) error {
return fmt.Errorf("%s: %s (got %q)", p.tokens[p.at].position, message, p.peek())
}
func (p *parser) expect(s string) error {
if p.peek() != s {
return p.err("expected " + strconv.Quote(s))
}
p.next()
return nil
}
func (p *parser) ident() (string, error) {
if p.tokens[p.at].kind != scanner.Ident {
return "", p.err("expected identifier")
}
return p.next().text, nil
}
func (p *parser) quoted() (string, error) {
if p.tokens[p.at].kind != scanner.String {
return "", p.err("expected quoted string")
}
return strconv.Unquote(p.next().text)
}
func (p *parser) parse() error {
server := map[string]string{}
syntaxSeen := false
for p.skip(); p.tokens[p.at].kind != scanner.EOF; p.skip() {
switch p.peek() {
case "scalar":
if err := p.parseScalar(); err != nil {
return err
}
case "syntax":
if syntaxSeen {
return p.err("duplicate syntax")
}
syntaxSeen = true
p.next()
if err := p.expect("="); err != nil {
return err
}
v, err := p.quoted()
if err != nil {
return err
}
if v != "v1" {
return p.err("only syntax v1 supported")
}
case "info":
p.next()
m, err := p.metadata()
if err != nil {
return err
}
if _, a := m["desc"]; a {
if _, b := m["description"]; b {
return p.err("use only one of info desc and description")
}
}
for k, v := range m {
switch k {
case "title":
p.spec.Title = v
case "version":
p.spec.Version = v
case "desc", "description":
p.spec.Description = v
case "author", "email":
default:
return p.err("unsupported info property " + k)
}
}
case "import":
p.next()
if p.peek() == "(" {
p.next()
for p.skip(); p.peek() != ")"; p.skip() {
v, err := p.quoted()
if err != nil {
return err
}
p.imports = append(p.imports, v)
}
p.next()
} else {
v, err := p.quoted()
if err != nil {
return err
}
p.imports = append(p.imports, v)
}
case "error":
if err := p.parseErrors(); err != nil {
return err
}
case "auth":
if err := p.parseAuth(); err != nil {
return err
}
case "response":
if err := p.parseResponse(); err != nil {
return err
}
case "type":
p.next()
if p.peek() == "(" {
p.next()
for p.skip(); p.peek() != ")"; p.skip() {
if err := p.parseType(); err != nil {
return err
}
}
p.next()
} else {
if err := p.parseType(); err != nil {
return err
}
}
case "@":
p.next()
if err := p.expect("server"); err != nil {
return err
}
if len(server) != 0 {
return p.err("duplicate server block")
}
m, err := p.metadata()
if err != nil {
return err
}
server = m
for k := range m {
if k != "prefix" && k != "group" && k != "middleware" && k != "auth" && k != "errors" && k != "response" {
return p.err("unsupported @server option " + k + "; wire authentication through named middleware")
}
}
case "service":
if err := p.parseService(server); err != nil {
return err
}
server = map[string]string{}
default:
return p.err("unsupported declaration")
}
}
if len(server) > 0 {
return p.err("@server must precede a service")
}
return nil
}
func (p *parser) metadata() (map[string]string, error) {
if err := p.expect("("); err != nil {
return nil, err
}
out := map[string]string{}
for p.skip(); p.peek() != ")"; p.skip() {
key, err := p.ident()
if err != nil {
return nil, err
}
if err = p.expect(":"); err != nil {
return nil, err
}
value := ""
if p.tokens[p.at].kind == scanner.String {
value, err = p.quoted()
if err != nil {
return nil, err
}
} else if p.peek() == "{" {
depth := 0
var object strings.Builder
for {
t := p.next()
if t.kind == scanner.EOF {
return nil, p.err("unterminated metadata object")
}
if t.text == "{" {
depth++
}
if t.text == "}" {
depth--
}
if t.space {
object.WriteByte(' ')
}
object.WriteString(t.text)
if depth == 0 {
break
}
}
value = object.String()
} else {
for p.peek() != "\n" && p.peek() != ";" && p.peek() != ")" && p.tokens[p.at].kind != scanner.EOF {
value += p.next().text
}
}
if value == "" {
return nil, p.err("metadata value required")
}
if _, exists := out[key]; exists {
return nil, p.err("duplicate metadata " + key)
}
out[key] = value
}
p.next()
return out, nil
}
func (p *parser) parseType() error {
pos := p.tokens[p.at].position
name, err := p.ident()
if err != nil {
return err
}
p.skip()
if err = p.expect("{"); err != nil {
return err
}
typ := Type{Name: name, Position: pos}
for p.skip(); p.peek() != "}"; p.skip() {
name, err := p.ident()
if err != nil {
return err
}
expression := ""
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" && p.tokens[p.at].kind != scanner.RawString && p.tokens[p.at].kind != scanner.EOF {
if p.peek() == "interface" {
p.next()
if p.expect("{") != nil || p.expect("}") != nil {
return p.err("only empty interface{} is supported; use any for dynamic JSON")
}
expression += "any"
} else {
expression += p.next().text
}
}
if expression == "" {
return p.err("field type required; embedded types are not supported")
}
if p.tokens[p.at].kind != scanner.RawString {
return p.err("field source tag required")
}
tag, err := strconv.Unquote(p.next().text)
if err != nil {
return err
}
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag})
}
p.next()
p.spec.Types = append(p.spec.Types, typ)
return nil
}
func (p *parser) parseService(server map[string]string) error {
p.next()
name, err := p.ident()
if err != nil {
return err
}
for p.peek() == "-" {
p.next()
part, e := p.ident()
if e != nil {
return e
}
name += "-" + part
}
if p.spec.Service != "" && p.spec.Service != name {
return p.err("one service name per entry API is supported")
}
p.spec.Service = name
p.skip()
if err = p.expect("{"); err != nil {
return err
}
for p.skip(); p.peek() != "}"; p.skip() {
route := Route{ResponseFormat: server["response"], Auth: server["auth"], Errors: commaList(server["errors"]), Group: server["group"], Status: 200, Position: p.tokens[p.at].position}
if route.Group == "" {
route.Group = "api"
}
if server["middleware"] != "" {
route.Middleware = strings.Split(server["middleware"], ",")
}
seen := map[string]bool{}
for p.peek() == "@" {
p.next()
annotation, e := p.ident()
if e != nil {
return e
}
if seen[annotation] {
return p.err("duplicate route annotation")
}
seen[annotation] = true
switch annotation {
case "handlerMode":
mode, err := p.ident()
if err != nil {
return err
}
if mode != "manual" {
return p.err("handlerMode must be manual")
}
route.Manual = true
case "binding":
mode, err := p.ident()
if err != nil {
return err
}
if mode != "custom" {
return p.err("binding must be custom")
}
route.CustomBinding = true
case "produces":
route.Produces, e = p.quoted()
case "body":
e = p.parseBody(&route)
case "handler":
route.Handler, e = p.ident()
case "doc":
e = p.parseDoc(&route)
case "auth":
route.Auth, e = p.ident()
case "response":
var format string
format, e = p.ident()
if format == "raw" && route.ResponseFormat != "raw" {
route.ErrorResponseFormat = route.ResponseFormat
}
route.ResponseFormat = format
case "errors":
var names []string
names, e = p.names()
if len(names) == 1 && names[0] == "none" {
route.Errors = nil
} else {
route.Errors = mergeNames(route.Errors, names)
}
case "status":
route.Status, e = strconv.Atoi(p.next().text)
default:
return p.err("unsupported route annotation " + annotation)
}
if e != nil {
return e
}
p.skip()
}
if route.Handler == "" {
return p.err("@handler required")
}
route.Method, err = p.ident()
if err != nil {
return err
}
path := ""
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "(" && p.peek() != "returns" && p.peek() != "}" && p.tokens[p.at].kind != scanner.EOF {
path += p.next().text
}
route.Path = server["prefix"] + path
if p.peek() == "(" {
route.Request, err = p.routeType()
if err != nil {
return err
}
}
if p.peek() == "returns" {
p.next()
route.Response, err = p.routeType()
if err != nil {
return err
}
}
if p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" {
return p.err("unexpected route suffix")
}
if route.Response == "" && !seen["status"] && !route.Manual {
route.Status = 204
}
p.spec.Routes = append(p.spec.Routes, route)
}
p.next()
return nil
}
func (p *parser) routeType() (string, error) {
if err := p.expect("("); err != nil {
return "", err
}
name, err := p.ident()
if err != nil {
return "", err
}
return name, p.expect(")")
}

View File

@@ -0,0 +1,73 @@
package apigen
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
)
type projectProfile struct {
Database string `json:"database"`
Storage string `json:"storage"`
LogOutput string `json:"logOutput"`
LogFormat string `json:"logFormat"`
}
func (o Options) profile() projectProfile {
return projectProfile{o.Database, o.Storage, o.LogOutput, o.LogFormat}
}
// Infrastructure files belong to the developer. Persist the original selection
// so a plain regeneration cannot silently change adapters or repair a missing
// file with settings from another profile.
func resolveProfile(o *Options) error {
defaults := projectProfile{"none", "none", "stderr", "json"}
data, err := os.ReadFile(filepath.Join(o.Dir, manifestPath))
if err != nil && !errors.Is(err, os.ErrNotExist) {
return err
}
existing := err == nil
if existing {
var previous manifest
if err = json.Unmarshal(data, &previous); err != nil {
return err
}
if previous.Profile != nil {
defaults = *previous.Profile
}
}
if o.Database == "memory" {
o.Database = "none"
}
for _, item := range []struct {
name string
value *string
fallback string
}{
{"database", &o.Database, defaults.Database}, {"storage", &o.Storage, defaults.Storage},
{"log-output", &o.LogOutput, defaults.LogOutput}, {"log-format", &o.LogFormat, defaults.LogFormat},
} {
if *item.value == "" {
*item.value = item.fallback
} else if existing && *item.value != item.fallback {
return fmt.Errorf("--%s differs from the original project profile; infrastructure is developer-owned, generate a new project to compare and migrate it explicitly", item.name)
}
}
if o.Database != "none" && o.Database != "mysql" {
return errors.New("--database must be none, memory or mysql")
}
switch o.Storage {
case "none", "local", "s3", "r2", "oss":
default:
return errors.New("--storage must be none, local, s3, r2 or oss")
}
if o.LogOutput != "stderr" && o.LogOutput != "stdout" && o.LogOutput != "file" {
return errors.New("--log-output must be stdout, stderr or file")
}
if o.LogFormat != "json" && o.LogFormat != "text" {
return errors.New("--log-format must be json or text")
}
return nil
}

View File

@@ -0,0 +1,98 @@
package apigen
import (
"context"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestInfrastructureProfileOwnership(t *testing.T) {
dir := filepath.Join(t.TempDir(), "project")
o := Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: "mysql", Storage: "local", LogOutput: "file"}
if _, err := New(o); err != nil {
t.Fatal(err)
}
o = Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
before := snapshot(t, dir)
for _, changed := range []Options{
{Database: "none"}, {Storage: "s3"}, {LogOutput: "stdout"}, {LogFormat: "text"},
} {
changed.Dir, changed.API = dir, o.API
if _, err := Generate(changed); err == nil {
t.Fatal("profile switch accepted")
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("profile rejection changed project")
}
}
// Restoring a missing seed must use the saved profile, not current defaults.
if err := os.Remove(filepath.Join(dir, "internal/servicecontext/storage.go")); err != nil {
t.Fatal(err)
}
if _, err := Generate(o); err == nil {
t.Fatal("check accepted missing infrastructure")
}
o.Check = false
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("profile reconstruction changed project")
}
}
func TestAPIInfrastructureConsumers(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
t.Skip("consumer module integration")
}
for _, db := range []string{"none", "mysql"} {
for _, storage := range []string{"none", "local", "s3", "r2", "oss"} {
t.Run(db+"/"+storage, func(t *testing.T) {
dir := filepath.Join(t.TempDir(), "project")
if _, err := New(Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: db, Storage: storage, LogOutput: "file"}); err != nil {
t.Fatal(err)
}
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
t.Fatal(err)
}
test := `package settings
import("testing";"os")
func TestConfiguration(t *testing.T){
t.Setenv("GO_OMNI_DB_DSN","test:test@tcp(127.0.0.1:1)/test")
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY","01234567890123456789012345678901")
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","test");t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","test")
t.Setenv("GO_OMNI_STORAGE_BUCKET","test");t.Setenv("GO_OMNI_STORAGE_ENDPOINT","ENDPOINT")
t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID","01234567890123456789012345678901")
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
_=os.Getenv("unused")
}`
endpoint := ""
if storage == "s3" {
endpoint = "127.0.0.1:9000"
}
if storage == "oss" {
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
}
writeFile(t, filepath.Join(dir, "internal/settings/profile_test.go"), strings.ReplaceAll(test, `"ENDPOINT"`, `"`+endpoint+`"`))
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%v\n%s", err, out)
}
})
}
}
}

View File

@@ -0,0 +1,35 @@
package apigen
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
)
// ProjectEntry discovers the API entry recorded by generation, never guessing
// which of several .api files is the service entry.
func ProjectEntry(directory string) (string, error) {
root, err := filepath.Abs(directory)
if err != nil {
return "", err
}
if err = safePath(root, manifestPath); err != nil {
return "", err
}
data, err := os.ReadFile(filepath.Join(root, manifestPath))
if err != nil {
return "", fmt.Errorf("--api is required outside a generated project (no readable %s)", manifestPath)
}
var m manifest
if err = json.Unmarshal(data, &m); err != nil {
return "", fmt.Errorf("invalid API project manifest: %w", err)
}
if m.Version != 1 || filepath.Ext(m.Entry) != ".api" {
return "", fmt.Errorf("invalid API entry in %s; pass --api explicitly", manifestPath)
}
if err = safePath(root, m.Entry); err != nil {
return "", err
}
return filepath.Join(root, filepath.FromSlash(m.Entry)), nil
}

View File

@@ -0,0 +1,95 @@
package apigen
import (
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"mime"
"strconv"
"strings"
)
func (p *parser) parseBody(r *Route) error {
props, err := p.metadata()
if err != nil {
return err
}
for key, value := range props {
switch key {
case "mode":
r.BodyMode = value
case "maxBytes":
r.MaxBodyBytes, err = strconv.ParseInt(value, 10, 64)
if err != nil || r.MaxBodyBytes <= 0 || r.MaxBodyBytes > 1<<40 {
return p.err("maxBytes must be 1..1099511627776 bytes")
}
default:
return fmt.Errorf("unknown body property %s", key)
}
}
if r.MaxBodyBytes == 0 {
r.MaxBodyBytes = contract.DefaultMaxBodyBytes
}
return nil
}
func validateRouteExtensions(r Route) error {
if r.BodyMode != "" && r.BodyMode != "json" && r.BodyMode != "raw" && r.BodyMode != "multipart" {
return errors.New("body mode must be json, raw or multipart")
}
if (r.BodyMode == "raw" || r.BodyMode == "multipart") && !r.Manual {
return errors.New("raw/multipart body requires @handlerMode manual")
}
if r.CustomBinding && (r.Manual || r.Request == "") {
return errors.New("@binding custom requires an automatic handler with a request DTO")
}
if r.MaxBodyBytes > 0 && (r.Method == "get" || r.Method == "head") {
return errors.New("GET/HEAD cannot declare a body limit")
}
if r.Produces != "" {
if !r.Manual {
return errors.New("@produces requires @handlerMode manual")
}
media, params, err := mime.ParseMediaType(r.Produces)
if err != nil || len(params) != 0 || media != r.Produces || strings.Contains(media, "*") {
return errors.New("produces requires a concrete media type without parameters")
}
if r.Response != "" || r.Status == 204 || r.Method == "head" {
return errors.New("@produces uses a manual response without a DTO, 204 or HEAD")
}
}
if r.Manual && r.Status == 204 && r.Response != "" {
return errors.New("204 cannot declare a response")
}
return nil
}
func decorateRequestOptions(op map[string]any, r Route) {
if r.Manual {
op["x-omni-handler-mode"] = "manual"
}
if r.CustomBinding {
op["x-omni-binding"] = "custom"
}
if r.MaxBodyBytes > 0 {
op["x-omni-max-body-bytes"] = r.MaxBodyBytes
}
if r.BodyMode == "raw" || r.BodyMode == "multipart" {
media := "application/octet-stream"
schema := map[string]any{"type": "string", "format": "binary"}
if r.BodyMode == "multipart" {
media = "multipart/form-data"
schema = map[string]any{"type": "object", "additionalProperties": true}
}
op["requestBody"] = map[string]any{"required": true, "content": map[string]any{media: map[string]any{"schema": schema}}}
}
if r.Produces != "" {
schema := map[string]any{"type": "string"}
if r.Produces == "application/octet-stream" {
schema["format"] = "binary"
}
if r.Produces == "application/json" {
schema = map[string]any{}
}
op["responses"].(map[string]any)[strconv.Itoa(r.Status)] = map[string]any{"description": "Success", "content": map[string]any{r.Produces: map[string]any{"schema": schema}}}
}
}

View File

@@ -0,0 +1,114 @@
package apigen
import (
"bytes"
"encoding/json"
"fmt"
"io"
"strings"
"git.yuntuo.hk/k2safe/go-omni/response"
)
type ResponseDefinition struct {
Name string
Config response.Config
Position string
}
func (p *parser) parseResponse() error {
pos := p.next().position
name, err := p.ident()
if err != nil {
return err
}
p.skip()
props, err := p.metadata()
if err != nil {
return err
}
values := map[string]any{}
for key, value := range props {
switch key {
case "successCode", "errorCode":
if kind := props["codeType"]; kind == "integer" || kind == "int32" || kind == "int64" {
var number json.Number
if err = json.Unmarshal([]byte(value), &number); err != nil {
return fmt.Errorf("%s: %s must be an integer", pos, key)
}
values[key] = number
} else {
values[key] = value
}
case "codes", "messages", "body", "errorBody":
decoder := json.NewDecoder(strings.NewReader(value))
decoder.UseNumber()
var object map[string]any
if err = decoder.Decode(&object); err != nil || object == nil {
return fmt.Errorf("%s: %s must be a JSON object", pos, key)
}
var extra any
if decoder.Decode(&extra) != io.EOF {
return fmt.Errorf("%s: invalid %s mapping", pos, key)
}
values[key] = object
default:
values[key] = value
}
}
data, err := json.Marshal(values)
if err != nil {
return err
}
decoder := json.NewDecoder(bytes.NewReader(data))
decoder.UseNumber()
decoder.DisallowUnknownFields()
var cfg response.Config
if err = decoder.Decode(&cfg); err != nil {
return fmt.Errorf("%s: response %s: %w", pos, name, err)
}
if err = cfg.Validate(); err != nil {
return fmt.Errorf("%s: %w", pos, err)
}
p.spec.Responses = append(p.spec.Responses, ResponseDefinition{Name: name, Config: cfg, Position: pos})
return nil
}
func responseConfig(spec *Spec, route Route) response.Config {
name := responseName(route)
for _, definition := range spec.Responses {
if definition.Name == name {
return definition.Config
}
}
return response.Config{}
}
func responseName(route Route) string {
if route.ResponseFormat == "raw" {
return route.ErrorResponseFormat
}
return route.ResponseFormat
}
func validateResponses(spec *Spec) error {
seen := map[string]bool{}
for _, definition := range spec.Responses {
if !exportedName.MatchString(definition.Name) || seen[definition.Name] {
return fmt.Errorf("%s: response template requires a unique UpperCamelCase name", definition.Position)
}
if err := definition.Config.Validate(); err != nil {
return fmt.Errorf("%s: %w", definition.Position, err)
}
seen[definition.Name] = true
}
for _, r := range spec.Routes {
if r.ResponseFormat != "" && r.ResponseFormat != "raw" && !seen[r.ResponseFormat] {
return fmt.Errorf("%s: unknown response template %s", r.Position, r.ResponseFormat)
}
if r.ErrorResponseFormat != "" && !seen[r.ErrorResponseFormat] {
return fmt.Errorf("%s: unknown error response template %s", r.Position, r.ErrorResponseFormat)
}
}
return nil
}

View File

@@ -0,0 +1,360 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
const configurableResponseAPI = `syntax = "v1"
response CommonResponse (
codeField: code
messageField: msg
dataField: data
requestIdField: "-"
codeType: integer
successCode: 200
errorCode: 9999
successMessage: "操作成功"
errorMessage: "系统错误"
dataMode: always
codes: {
"UNAUTHENTICATED": 401001,
"EMAIL_TAKEN": 409001
}
messages: {"UNAUTHENTICATED":"请先登录","EMAIL_TAKEN":"邮箱已存在"}
)
response PartnerResponse (
codeField: result
messageField: notice
dataField: payload
requestIdField: trace
successCode: "PASS"
)
error (
Unauthenticated 401 "UNAUTHENTICATED" "authentication required"
EmailTaken 409 "EMAIL_TAKEN" "email already taken"
)
auth Access (type: bearer
middleware: Auth
)
type ItemResponse {
Name string ` + "`json:\"name\"`" + `
}
@server (
group: items
response: CommonResponse
auth: Access
errors: Unauthenticated
)
service response-api {
@errors EmailTaken
@handler item
get /item returns (ItemResponse)
@response raw
@handler callback
post /callback returns (ItemResponse)
@response PartnerResponse
@handler partner
get /partner returns (ItemResponse)
}
`
func TestResponseTemplatesDocumentAndValidation(t *testing.T) {
spec, err := parseText(t, configurableResponseAPI)
if err != nil {
t.Fatal(err)
}
c, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
doc := document(t, c)
getSchema := func(path, status string) map[string]any {
method := "get"
if path == "/callback" {
method = "post"
}
return operation(t, doc, path, method)["responses"].(map[string]any)[status].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
}
properties := getSchema("/item", "200")["properties"].(map[string]any)
if len(properties) != 3 || properties["code"].(map[string]any)["type"] != "integer" || properties["msg"] == nil || properties["requestId"] != nil {
t.Fatal(properties)
}
if getSchema("/callback", "200")["properties"].(map[string]any)["name"] == nil {
t.Fatal("raw JSON was wrapped")
}
if getSchema("/callback", "401")["properties"].(map[string]any)["msg"] == nil {
t.Fatal("raw override lost group error policy")
}
if getSchema("/partner", "200")["properties"].(map[string]any)["payload"] == nil {
t.Fatal("route override not applied")
}
conflict := operation(t, doc, "/item", "get")["responses"].(map[string]any)["409"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
value := conflict["examples"].(map[string]any)["EmailTaken"].(map[string]any)["value"].(map[string]any)
if value["code"] != float64(409001) || value["msg"] != "邮箱已存在" || len(value) != 3 {
t.Fatal(value)
}
if got := conflict["schema"].(map[string]any)["properties"].(map[string]any)["code"].(map[string]any)["enum"].([]any); got[0] != float64(409001) {
t.Fatal(got)
}
for _, change := range [][2]string{{"response: CommonResponse", "response: Missing"}, {"messageField: msg", "messageField: code"}, {"codeType: integer", "codeType: boolean"}, {"successCode: 200", "successCode: 2.5"}, {"requestIdField: \"-\"", "unknownOption: value"}, {"@response raw", "@response missing"}} {
invalid, err := parseText(t, strings.Replace(configurableResponseAPI, change[0], change[1], 1))
if err == nil {
_, err = Compile(invalid)
}
if err == nil || !strings.Contains(err.Error(), "service.api:") {
t.Fatalf("bad declaration %s: %v", change[1], err)
}
}
}
func TestImportedResponseTemplate(t *testing.T) {
dir := t.TempDir()
writeFile(t, filepath.Join(dir, "common.api"), "response Shared (messageField: msg)\n")
writeFile(t, filepath.Join(dir, "service.api"), "import \"common.api\"\n@server (response: Shared)\nservice shared-api {\n@handler ping\nget /ping\n}")
spec, err := Parse(filepath.Join(dir, "service.api"))
if err != nil {
t.Fatal(err)
}
if _, err = Compile(spec); err != nil {
t.Fatal(err)
}
}
func TestResponseNestedBodyDocument(t *testing.T) {
definition := `response Nested (
body: {"success":"${ok}","result":"${data}","meta":{"status":"${status}","trace":"${requestId}"},"version":1}
errorBody: {"success":"${ok}","error":{"id":"${code}","reason":"${message}"}}
codeType: integer
successCode: 0
errorCode: 9999
codes: {"EMAIL_TAKEN":409001}
)
`
spec, err := parseText(t, definition+strings.Replace(configurableResponseAPI, "response: CommonResponse", "response: Nested", 1))
if err != nil {
t.Fatal(err)
}
compiled, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
doc := document(t, compiled)
responses := operation(t, doc, "/item", "get")["responses"].(map[string]any)
success := responses["200"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)["properties"].(map[string]any)
if success["code"] != nil || success["result"] == nil || success["success"].(map[string]any)["type"] != "boolean" {
t.Fatal(success)
}
media := responses["409"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
schema := media["schema"].(map[string]any)["properties"].(map[string]any)
if schema["error"].(map[string]any)["properties"].(map[string]any)["id"].(map[string]any)["type"] != "integer" || schema["result"] != nil {
t.Fatal(schema)
}
example := media["examples"].(map[string]any)["EmailTaken"].(map[string]any)["value"].(map[string]any)
if example["success"] != false || example["error"].(map[string]any)["id"] != float64(409001) {
t.Fatal(example)
}
}
func TestResponseTemplateJSONTokens(t *testing.T) {
for _, tc := range []struct {
body string
valid bool
}{
{`{"scale":-1.5,"exponent":1e-3,"items":[true,null,"${data}"],"label":"a b"}`, true},
{`{"scale":1 2}`, false}, {`{"scale":- 12}`, false},
{`{"scale":tru e}`, false}, {`{"scale":1. 2}`, false},
} {
text := "response Shared (\nbody: " + tc.body + "\n)\n@server (response: Shared)\nservice test-api {\n@handler ping\nget /ping\n}\n"
spec, err := parseText(t, text)
if err == nil {
_, err = Compile(spec)
}
if (err == nil) != tc.valid {
t.Fatalf("%s: %v", tc.body, err)
}
}
}
func TestResponseCodeTypeDeclarations(t *testing.T) {
for _, kind := range []string{"int32", "int64"} {
source := strings.Replace(configurableResponseAPI, "codeType: integer", "codeType: "+kind, 1)
spec, err := parseText(t, source)
if err != nil {
t.Fatal(err)
}
compiled, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
doc := document(t, compiled)
for _, status := range []string{"200", "401", "409", "500"} {
responses := operation(t, doc, "/item", "get")["responses"].(map[string]any)
schema := responses[status].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
code := schema["properties"].(map[string]any)["code"].(map[string]any)
if code["type"] != "integer" || code["format"] != kind {
t.Fatal(code)
}
}
for _, bad := range []string{"2.5", "9223372036854775808", "-9223372036854775809"} {
invalid := strings.Replace(source, "successCode: 200", "successCode: "+bad, 1)
if _, err := parseText(t, invalid); err == nil || !strings.Contains(err.Error(), "service.api:") {
t.Fatalf("%s %s: %v", kind, bad, err)
}
}
}
invalid := strings.Replace(configurableResponseAPI, "codeType: integer", "codeType: int32", 1)
invalid = strings.Replace(invalid, `"EMAIL_TAKEN": 409001`, `"EMAIL_TAKEN": 2147483648`, 1)
if _, err := parseText(t, invalid); err == nil {
t.Fatal("overflowing int32 mapping accepted")
}
}
func TestResponseTemplateConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
t.Skip("consumer integration")
}
dir := filepath.Join(t.TempDir(), "project")
source := filepath.Join(t.TempDir(), "service.api")
// Use an explicitly declared public body, rather than field-name shortcuts.
sourceText := strings.Replace(configurableResponseAPI, " codeField: code\n messageField: msg\n dataField: data\n requestIdField: \"-\"", ` body: {"code":"${code}","msg":"${message}","data":"${data}"}`, 1)
sourceText = strings.Replace(sourceText, "codeType: integer", "codeType: int64", 1)
sourceText = strings.Replace(sourceText, "successCode: 200", "successCode: 9223372036854775807", 1)
writeFile(t, source, sourceText)
releaseCLI := os.Getenv("GO_OMNI_RELEASE_CLI")
generate := func(api string, initial bool) {
t.Helper()
if releaseCLI != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, releaseCLI, "api", "generate", "--api", api, "--dir", dir, "--module", "example.com/responseConsumer")
// The framework checkout intentionally enables local development mode.
// Run the installed CLI outside it to exercise remote-only consumption.
cmd.Dir = filepath.Dir(dir)
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("released CLI: %v\n%s", err, output)
}
return
}
o := Options{API: api, Dir: dir}
if initial {
o.Module = "example.com/responseConsumer"
o.FrameworkDir = frameworkRoot(t)
}
if _, err := Generate(o); err != nil {
t.Fatal(err)
}
}
generate(source, true)
writeFile(t, filepath.Join(dir, "response_test.go"), responseConsumerTest)
for _, method := range []string{"Item", "Callback", "Partner"} {
body := `return &types.ItemResponse{Name:"` + strings.ToLower(method) + `"},nil`
if method == "Partner" {
body = `return nil,apierrors.EmailTaken()`
}
imports := `"context";"example.com/responseConsumer/internal/types"`
if method == "Partner" {
imports += `;"example.com/responseConsumer/internal/apierrors"`
}
writeFile(t, filepath.Join(dir, "internal/service/items", strings.ToLower(method)+".go"), "package items\nimport("+imports+")\nfunc(s *Service)"+method+"(ctx context.Context)(*types.ItemResponse,error){"+body+"}\n")
}
before := snapshot(t, dir)
generate(filepath.Join(dir, "api/service.api"), false)
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("regeneration changed hand-written files")
}
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("consumer: %v\n%s", err, output)
}
if releaseCLI != "" {
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
output, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("released module: %v\n%s", err, output)
}
var module struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(output, &module); err != nil {
t.Fatal(err)
}
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
t.Fatalf("expected fixed remote module without replacement: %+v", module)
}
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
}
manifestBytes, err := os.ReadFile(filepath.Join(dir, manifestPath))
if err != nil {
t.Fatal(err)
}
var manifest manifest
if err = json.Unmarshal(manifestBytes, &manifest); err != nil {
t.Fatal(err)
}
if manifest.Generated["internal/handler/responseGen.go"] == "" {
t.Fatal("response policy not ownership-managed")
}
}
const responseConsumerTest = `package responseConsumer_test
import (
"bytes";"encoding/json";"io";"net/http/httptest";"testing"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
"example.com/responseConsumer/internal/apierrors"
"example.com/responseConsumer/internal/handler"
item "example.com/responseConsumer/internal/handler/items"
"example.com/responseConsumer/internal/types"
"context"
"example.com/responseConsumer/internal/servicecontext"
)
type service struct{}
func(service)Item(context.Context)(*types.ItemResponse,error){return &types.ItemResponse{Name:"item"},nil}
func(service)Callback(context.Context)(*types.ItemResponse,error){return &types.ItemResponse{Name:"callback"},nil}
func(service)Partner(context.Context)(*types.ItemResponse,error){return nil,apierrors.EmailTaken()}
func TestResponseContract(t *testing.T){
app:=omni.New();defer app.Close()
cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close()
deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close()
deps.Middlewares["Auth"]=func(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error{if c.Header("Authorization")!="Bearer valid"{return apierrors.Unauthenticated()};return next(c)}}
if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
// Registered routes use the compiled policy before even running authentication.
for _,path:=range []string{"/item","/callback"}{
method:="GET";if path=="/callback"{method="POST"};w:=httptest.NewRecorder();app.ServeHTTP(w,httptest.NewRequest(method,path,nil))
var value map[string]any;json.Unmarshal(w.Body.Bytes(),&value)
if w.Code!=401||value["code"]!=float64(401001)||value["msg"]!="请先登录"||len(value)!=3{t.Fatalf("%s: %s",path,w.Body.String())}
}
// Execute the real generated registration and user-owned service files.
for _,tc:=range []struct{method,path string;status int;key string}{{"GET","/item",200,"data"},{"POST","/callback",200,"name"},{"GET","/partner",409,"result"}}{
w:=httptest.NewRecorder();r:=httptest.NewRequest(tc.method,tc.path,nil);r.Header.Set("Authorization","Bearer valid");app.ServeHTTP(w,r)
var value map[string]any;decoder:=json.NewDecoder(bytes.NewReader(w.Body.Bytes()));decoder.UseNumber();if err:=decoder.Decode(&value);err!=nil{t.Fatal(err)}
if w.Code!=tc.status||value[tc.key]==nil{t.Fatalf("generated %s: %d %s",tc.path,w.Code,w.Body.String())}
if tc.path=="/item"&&value["code"]!=json.Number("9223372036854775807"){t.Fatalf("generated int64 lost precision: %s",w.Body.String())}
}
// Business handlers also honor the same runtime policy and raw override.
h:=item.New(service{});runtime:=omni.New();defer runtime.Close()
common:=response.MustNew(response.Config{CodeType:"integer",SuccessCode:200,ErrorCode:9999,MessageField:"msg",RequestIDField:"-",DataMode:"always"})
runtime.GET("/item",h.Item,httpx.WithResponse(common))
runtime.POST("/callback",h.Callback,httpx.WithResponse(response.RawJSON(common)))
for _,tc:=range []struct{method,path,key string}{{"GET","/item","data"},{"POST","/callback","name"}}{
w:=httptest.NewRecorder();runtime.ServeHTTP(w,httptest.NewRequest(tc.method,tc.path,nil));var value map[string]any;json.Unmarshal(w.Body.Bytes(),&value)
if w.Code!=200||value[tc.key]==nil{t.Fatal(w.Body.String())}
}
}
`

View File

@@ -0,0 +1,74 @@
package apigen
import (
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"golang.org/x/mod/module"
"reflect"
"strings"
)
type ScalarDefinition struct {
Name, GoImport, GoType, JSONInput, JSONOutput, Position string
Text bool
}
func (p *parser) parseScalar() error {
pos := p.next().position
name, err := p.ident()
if err != nil {
return err
}
p.skip()
props, err := p.metadata()
if err != nil {
return err
}
d := ScalarDefinition{Name: name, Position: pos}
for key, value := range props {
switch key {
case "goImport":
d.GoImport = value
case "goType":
d.GoType = value
case "jsonInput":
d.JSONInput = value
case "jsonOutput":
d.JSONOutput = value
case "text":
if value != "true" && value != "false" {
return p.err("text must be true or false")
}
d.Text = value == "true"
default:
return fmt.Errorf("%s: unknown scalar property %s", pos, key)
}
}
p.spec.Scalars = append(p.spec.Scalars, d)
return nil
}
func compileScalars(spec *Spec, types map[string]Type) (map[string]reflect.Type, []contract.Scalar, error) {
out := map[string]reflect.Type{}
var defs []contract.Scalar
goTypes := map[string]bool{}
for _, d := range spec.Scalars {
key := d.GoImport + "." + d.GoType
if d.Name == "ContractScalars" || !exportedName.MatchString(d.Name) || !exportedName.MatchString(d.GoType) || out[d.Name] != nil || types[d.Name].Name != "" || goTypes[key] {
return nil, nil, fmt.Errorf("%s: scalar names and Go types must be unique; names must be UpperCamelCase", d.Position)
}
if err := module.CheckImportPath(d.GoImport); err != nil || strings.Contains(d.GoImport, "\\") {
return nil, nil, fmt.Errorf("%s: invalid scalar goImport", d.Position)
}
// Unique stand-in types allow CLI schema checks without executing project code.
t := reflect.StructOf([]reflect.StructField{{Name: d.Name, Type: reflect.TypeOf(false)}})
def := contract.Scalar{Type: t, JSONInput: d.JSONInput, JSONOutput: d.JSONOutput, Text: d.Text}
probe := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: t, Tag: `json:"value"`}})
if _, err := contract.Describe(probe, def); err != nil {
return nil, nil, fmt.Errorf("%s: %w", d.Position, err)
}
out[d.Name] = t
defs = append(defs, def)
goTypes[key] = true
}
return out, defs, nil
}

View File

@@ -0,0 +1,176 @@
package apigen
import "fmt"
var baseTemplates = map[string]string{
"cmd/server/main.go": `package main
import (
"context"
"errors"
"flag"
"log"
"os"
"os/signal"
"syscall"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/logx"
"MODULE/internal/handler"
"MODULE/internal/servicecontext"
"MODULE/internal/settings"
)
func main(){
path:=flag.String("config","etc/config.yaml","configuration file; empty uses defaults and environment")
flag.Parse()
if err:=run(*path);err!=nil{log.Fatal(err)}
}
func run(path string)(runErr error){
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
defer stop()
cfg,err:=settings.Load(path);if err!=nil{return err}
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,logger.Close())}()
logx.SetDefault(logger)
deps,err:=servicecontext.New(ctx,logger,cfg);if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,deps.Close())}()
app:=omni.New(omni.WithConfig(cfg.Server))
app.OnShutdown(logger.Close)
app.OnShutdown(deps.Close)
if err=handler.RegisterRoutes(app,deps);err!=nil{return err}
app.GET("/openapi.json",handler.OpenAPI)
app.GET("/health/live",deps.Health.LivenessHandler())
app.GET("/health/ready",deps.Health.ReadinessHandler())
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
return app.RunContext(ctx,cfg.Server.Address)
}
`,
".gitignore": "/server\n/var/\n.env\n.env.*\n!.env.example\n/.goOmni/api.lock\n",
}
func projectREADME(module, entry string) string {
return fmt.Sprintf(`# %s
This project is generated from an editable API contract. Business code uses
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
## Run
`+"```sh"+`
go mod tidy
go run ./cmd/server
`+"```"+`
GET /health/live and /health/ready check the running service. GET /openapi.json
serves the same document as docs/openapi.json. Bind address defaults to localhost.
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
The greeting starter works immediately; other new operation stubs return 501 until
their business logic is implemented. Authentication hooks also fail closed with 501.
## Develop
1. Edit %s and its relative imports.
2. Run the same go-omni CLI version as the framework dependency:
`+"```sh"+`
go-omni api validate
go-omni api inspect
go-omni api generate --dry-run
go-omni api generate
go-omni api generate --check
go-omni api export > openapi.json
go test ./...
`+"```"+`
Commands inside a generated project discover the recorded API entry automatically.
--dry-run previews create/update/remove actions without writing files.
The --check command is read-only and suitable for CI. It fails on stale or missing
generated files and missing business scaffolds. go.mod, configuration, main,
Service, Repository, servicecontext, middleware hook bodies and API files are
created once and then preserved byte for byte. Gen.go files and OpenAPI are owned
by the generator; manual edits cause a conflict instead of being overwritten.
Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it.
Avoid editing files while generation is running; api.lock serializes generators.
Routes and types that disappear are removed from generated output. Handwritten
business files remain: remove/adapt obsolete implementations explicitly if an
API change makes their old types or signatures fail to compile. Every regeneration
must be followed by go test ./... . Do not delete service files to change contracts.
## Layout
`+"```text"+`
api/ # editable contract and imported types
cmd/server/main.go # startup, lifecycle and route registration
etc/config.yaml # lowerCamelCase configuration
internal/settings/ # defaults -> file -> environment
internal/servicecontext/ # dependency initialization and cleanup
internal/types/typesGen.go # generated request/response types
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
internal/handler/<group>/ # generated binding and service adapters
internal/service/<group>/ # developer-owned business operations
internal/repository/<group>/ # developer-owned persistence contracts
internal/middleware/ # named hooks and generated registry
docs/openapi.json # generated OpenAPI 3.0.3
.goOmni/apiManifest.json # ownership and content hashes
`+"```"+`
The API response type describes business data. Declare response Name (...) and
select it with @server(response: Name) or @response Name to share a public body;
@response raw returns the business JSON directly. Without a template, the default
envelope includes code, message, data and requestId. Generated routes and OpenAPI
use the API contract, independently of the app's default response configuration.
Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default.
Use contract.Binder.Bind's optional limit in a custom adapter for larger payloads.
Fields require exactly one source tag: json, path, form (URL query only), header.
All are required by presence unless optional, omitempty or default is specified.
Zero and false are valid present values; binding:"required" rejects scalar zero.
Use *T with optional when business logic needs to distinguish missing from zero.
Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON
fields and duplicate scalar query/header values are rejected. Arrays in the query
use repeated keys. Enum/range/default and nested fields share the documentation
and binding compiler. Responses are expected to honor their declared constraints;
they are not validated on every response. Do not return nil slices/maps for a
non-nullable collection (initialize empty collections instead).
Declare public business errors in error (...) and reference them with @errors.
Use internal/apierrors constructors to share status, code and message with OpenAPI.
Auth declarations connect a bearer/header/cookie scheme to a real named middleware.
The selected auth runs first; @auth none removes inherited declarative auth.
Validation policy remains in the middleware implementation. Public messages must
not contain secrets; error causes may be included in server logs.
Named middleware is composed in declaration order (first is outermost). Implement
the generated hooks or replace entries in servicecontext; never bypass an auth
hook merely to make the server start. Middleware names do not imply a JWT policy.
This CLI supports a documented go-zero-style subset, not all goctl features.
Database tables and repositories cannot be inferred from an HTTP contract. Add
business repositories through servicecontext. Database and storage options are
composed at creation; the original profile is retained on regeneration.
`, module, entry)
}
func infrastructureREADME(o Options) string {
return fmt.Sprintf(`
## Infrastructure
Selected database: %s; storage: %s; logging: %s/%s.
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
No tables are invented or migrated automatically from request/response DTOs.
The none/memory selection opens no database; implement repositories per business.
With storage enabled, services receive deps.Files, Uploads, Downloads and
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
and authorize before issuing them. No public upload-link route is added.
Storage failures during startup close an already-open database; Close is idempotent.
Database/storage/logging selection is recorded in .goOmni/apiManifest.json.
Changing selection through generation flags is rejected on an existing project;
edit your owned configuration and initialization explicitly, or generate a fresh
project for comparison. New routes still regenerate without repeating these flags.
`, o.Database, o.Storage, o.LogOutput, o.LogFormat)
}

View File

@@ -0,0 +1,117 @@
package compatibility_test
import (
"bytes"
"context"
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"example.com/compatibility/internal/handler"
"example.com/compatibility/internal/servicecontext"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/logx"
"io"
"mime/multipart"
"net/http/httptest"
"os"
"strings"
"testing"
)
func TestGeneratedCompatibility(t *testing.T) {
logger, err := logx.New(logx.Config{Writer: io.Discard})
if err != nil {
t.Fatal(err)
}
defer logger.Close()
deps, err := servicecontext.New(context.Background(), logger)
if err != nil {
t.Fatal(err)
}
defer deps.Close()
app := omni.New()
defer app.Close()
if err = handler.RegisterRoutes(app, deps); err != nil {
t.Fatal(err)
}
send := func(method, path, body, contentType string) *httptest.ResponseRecorder {
r := httptest.NewRequest(method, path, strings.NewReader(body))
r.Header.Set("Content-Type", contentType)
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
return w
}
for _, id := range []string{`9223372036854775807`, `"9223372036854775807"`} {
w := send("POST", "/echo", `{"id":`+id+`,"data":{"n":9223372036854775807}}`, "application/json")
if w.Code != 200 || !strings.Contains(w.Body.String(), `"id":"9223372036854775807"`) || !strings.Contains(w.Body.String(), `"n":9223372036854775807`) {
t.Fatalf("scalar/dynamic: %d %s", w.Code, w.Body)
}
}
for _, body := range []string{`{"id":true,"data":null}`, `{"id":9223372036854775808,"data":null}`, `{"id":1,"data":{"x":1,"x":2}}`, `{"id":1,"data":null,"extra":1}`} {
if w := send("POST", "/echo", body, "application/json"); w.Code != 400 {
t.Fatalf("accepted %s: %d", body, w.Code)
}
}
payload := `{"id":1,"data":"` + strings.Repeat("x", (1<<20)+1) + `"}`
if w := send("POST", "/echo", payload, "application/json"); w.Code != 200 {
t.Fatalf("route limit not passed to Binder: %d", w.Code)
}
if w := send("POST", "/echo", `{"id":1,"data":"`+strings.Repeat("x", 2<<20)+`"}`, "application/json"); w.Code != 413 {
t.Fatalf("large JSON: %d", w.Code)
}
if w := send("GET", "/items/9223372036854775807", "", ""); w.Code != 200 || !strings.Contains(w.Body.String(), `"id":"9223372036854775807"`) {
t.Fatalf("text scalar: %d %s", w.Code, w.Body)
}
if w := send("POST", "/custom", `{"value":"hello"}`, "application/json"); w.Code != 200 || !strings.Contains(w.Body.String(), "HELLO") {
t.Fatalf("custom binder: %d %s", w.Code, w.Body)
}
raw := `{ "event": 1, "id": "original bytes" }`
mac := hmac.New(sha256.New, []byte("fixture-only-key"))
mac.Write([]byte(raw))
r := httptest.NewRequest("POST", "/callback", strings.NewReader(raw))
r.Header.Set("X-Signature", hex.EncodeToString(mac.Sum(nil)))
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
if w.Code != 200 || w.Body.String() != "accepted" {
t.Fatalf("raw callback: %d %s", w.Code, w.Body)
}
r = httptest.NewRequest("POST", "/callback", strings.NewReader(strings.Repeat("x", 129)))
r.ContentLength = -1
w = httptest.NewRecorder()
app.ServeHTTP(w, r)
if w.Code != 413 {
t.Fatalf("chunked body limit: %d %s", w.Code, w.Body)
}
temp := t.TempDir()
t.Setenv("TMPDIR", temp)
var form bytes.Buffer
writer := multipart.NewWriter(&form)
part, err := writer.CreateFormFile("file", "sample.bin")
if err != nil {
t.Fatal(err)
}
part.Write(bytes.Repeat([]byte("x"), 8192))
writer.Close()
w = send("POST", "/upload", form.String(), writer.FormDataContentType())
if w.Code != 200 || !strings.Contains(w.Body.String(), `"bytes":8192`) {
t.Fatalf("upload: %d %s", w.Code, w.Body)
}
entries, err := os.ReadDir(temp)
if err != nil || len(entries) != 0 {
t.Fatalf("multipart temp files leaked: %v %v", entries, err)
}
w = send("GET", "/download", "", "")
if w.Code != 200 || w.Body.Len() != 128<<10 || w.Header().Get("Content-Disposition") == "" {
t.Fatalf("download wrapped: %d %d", w.Code, w.Body.Len())
}
var doc map[string]any
if err = json.Unmarshal([]byte(handler.OpenAPIDocument), &doc); err != nil {
t.Fatal(err)
}
paths := doc["paths"].(map[string]any)
media := paths["/download"].(map[string]any)["get"].(map[string]any)["responses"].(map[string]any)["200"].(map[string]any)["content"].(map[string]any)
if media["application/octet-stream"] == nil || media["application/json"] != nil {
t.Fatal("download schema is JSON")
}
}

View File

@@ -0,0 +1,16 @@
package files
import (
"git.yuntuo.hk/k2safe/go-omni/httpx"
"io"
"net/http"
"strings"
)
func (h *Handler) Download(c *httpx.Context) error {
return httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", `attachment; filename="demo.bin"`)
_, _ = io.Copy(w, strings.NewReader(strings.Repeat("x", 128<<10)))
}))(c)
}

View File

@@ -0,0 +1,16 @@
package demo
import (
"context"
"example.com/compatibility/internal/types"
)
func (s *Service) Echo(ctx context.Context, req *types.EchoReq) (*types.EchoResp, error) {
return &types.EchoResp{ID: req.ID, Data: req.Data}, nil
}
func (s *Service) Item(ctx context.Context, req *types.ItemReq) (*types.EchoResp, error) {
return &types.EchoResp{ID: req.ID, Data: map[string]any{"found": true}}, nil
}
func (s *Service) Custom(ctx context.Context, req *types.CustomReq) (*types.CustomResp, error) {
return &types.CustomResp{Value: req.Value}, nil
}

View File

@@ -0,0 +1,58 @@
package demo
import (
"crypto/hmac"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"example.com/compatibility/internal/types"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"io"
"net/http"
"strings"
)
func (h *Handler) bindCustom(c *httpx.Context, req *types.CustomReq) error {
decoder := json.NewDecoder(c.Request().Body)
decoder.DisallowUnknownFields()
if err := decoder.Decode(req); err != nil {
return apierrors.BadRequest("INVALID_REQUEST", "invalid request").WithCause(err)
}
var extra any
if err := decoder.Decode(&extra); err != io.EOF {
return apierrors.BadRequest("INVALID_REQUEST", "one JSON value required").WithCause(err)
}
req.Value = strings.ToUpper(req.Value)
return nil
}
func (h *Handler) Upload(c *httpx.Context) error {
// The small memory threshold deliberately exercises disk spill and cleanup.
if err := c.Request().ParseMultipartForm(8); err != nil {
return err
}
file, _, err := c.Request().FormFile("file")
if err != nil {
return err
}
defer file.Close()
count, err := io.Copy(io.Discard, file)
if err != nil {
return err
}
return c.OK(map[string]any{"bytes": count})
}
func (h *Handler) Callback(c *httpx.Context) error {
raw, err := io.ReadAll(c.Request().Body)
if err != nil {
return err
}
// This fixed key belongs only to the test fixture. Applications inject secrets.
mac := hmac.New(sha256.New, []byte("fixture-only-key"))
mac.Write(raw)
signature, err := hex.DecodeString(c.Header("X-Signature"))
if err != nil || !hmac.Equal(signature, mac.Sum(nil)) {
return c.Text(http.StatusUnauthorized, "invalid signature")
}
return c.Text(http.StatusOK, "accepted")
}

311
cmd/go-omni/apigen/write.go Normal file
View File

@@ -0,0 +1,311 @@
package apigen
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"sort"
"strings"
)
const manifestPath = ".goOmni/apiManifest.json"
type manifest struct {
Profile *projectProfile `json:"profile,omitempty"`
Version int `json:"version"`
Module string `json:"module"`
Entry string `json:"entry"`
Generated map[string]string `json:"generated"`
}
type change struct {
name string
before, after []byte
existed, remove bool
mode os.FileMode
}
func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile) (Output, error) {
var result Output
if info, err := os.Lstat(root); err == nil {
if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() {
return result, errors.New("api: output root must be a real directory")
}
} else if !errors.Is(err, os.ErrNotExist) {
return result, err
}
root = canonical(root)
names := map[string]bool{}
caseNames := map[string]string{}
for _, files := range []map[string][]byte{generated, seeds} {
for name := range files {
if names[name] {
return result, fmt.Errorf("duplicate output %s", name)
}
names[name] = true
folded := strings.ToLower(name)
if old, ok := caseNames[folded]; ok {
return result, fmt.Errorf("case-insensitive path collision: %s and %s", old, name)
}
caseNames[folded] = name
if err := safePath(root, name); err != nil {
return result, err
}
}
}
if err := safePath(root, manifestPath); err != nil {
return result, err
}
if !check && !dryRun {
if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
return result, err
}
lock := filepath.Join(root, ".goOmni", "api.lock")
f, err := os.OpenFile(lock, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600)
if err != nil {
return result, fmt.Errorf("api: acquire generation lock (remove stale api.lock only after checking no generator is running): %w", err)
}
_, _ = fmt.Fprintf(f, "pid=%d\n", os.Getpid())
_ = f.Close()
defer os.Remove(lock)
}
old := manifest{Generated: map[string]string{}}
oldData, err := os.ReadFile(filepath.Join(root, manifestPath))
if err == nil {
if err = json.Unmarshal(oldData, &old); err != nil {
return result, fmt.Errorf("invalid ownership manifest: %w", err)
}
if old.Profile != nil && *old.Profile != profile {
return result, errors.New("api: profile changed during generation; retry")
}
if old.Version != 1 || old.Module != module || old.Entry != entry {
return result, errors.New("api: manifest version/module/entry differs; one API entry owns this project")
}
} else if !errors.Is(err, os.ErrNotExist) {
return result, err
}
if oldData == nil {
if err = filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
if errors.Is(walkErr, os.ErrNotExist) {
return nil
}
if walkErr != nil {
return walkErr
}
if d.IsDir() && d.Name() == ".git" {
return filepath.SkipDir
}
if !d.IsDir() && strings.HasSuffix(d.Name(), ".go") {
return fmt.Errorf("api: %s belongs to an existing Go layout; generate into a new directory", path)
}
return nil
}); err != nil {
return result, err
}
}
next := manifest{Profile: &profile, Version: 1, Module: module, Entry: entry, Generated: map[string]string{}}
for name, data := range generated {
next.Generated[name] = digest(data)
}
var changes []change
// Check ALL previous outputs, even stale ones, before writing any new file.
for name, hash := range old.Generated {
if !ownedPath(name) {
return result, fmt.Errorf("api: invalid owned path %q", name)
}
if err = safePath(root, name); err != nil {
return result, err
}
data, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(name)))
if errors.Is(e, os.ErrNotExist) {
continue
}
if e != nil {
return result, e
}
if digest(data) != hash {
return result, fmt.Errorf("api: generated file %s was edited; move custom code into developer-owned files before regenerating", name)
}
if _, exists := generated[name]; !exists {
info, _ := os.Stat(filepath.Join(root, filepath.FromSlash(name)))
changes = append(changes, change{name: name, before: data, existed: true, remove: true, mode: info.Mode().Perm()})
result.Removed = append(result.Removed, name)
}
}
paths := make([]string, 0, len(names))
for name := range names {
paths = append(paths, name)
}
sort.Strings(paths)
for _, name := range paths {
target := filepath.Join(root, filepath.FromSlash(name))
current, err := os.ReadFile(target)
exists := err == nil
if err != nil && !errors.Is(err, os.ErrNotExist) {
return result, err
}
content, isGenerated := generated[name]
if !isGenerated {
content = seeds[name]
}
if exists && !isGenerated {
result.Preserved = append(result.Preserved, name)
continue
}
if exists && isGenerated {
if _, owned := old.Generated[name]; !owned {
return result, fmt.Errorf("api: refusing to adopt existing unowned file %s", name)
}
}
if isGenerated {
result.Generated = append(result.Generated, name)
} else {
result.Created = append(result.Created, name)
}
if exists && bytes.Equal(current, content) {
continue
}
mode := os.FileMode(0644)
if exists {
info, e := os.Stat(target)
if e != nil {
return result, e
}
mode = info.Mode().Perm()
}
changes = append(changes, change{name: name, before: current, after: content, existed: exists, mode: mode})
}
data, err := json.MarshalIndent(next, "", " ")
if err != nil {
return result, err
}
data = append(data, '\n')
if !bytes.Equal(oldData, data) {
changes = append(changes, change{name: manifestPath, before: oldData, after: data, existed: oldData != nil, mode: 0644})
}
sort.Strings(result.Removed)
for _, c := range changes {
action := "create"
if c.remove {
action = "remove"
} else if c.existed {
action = "update"
}
result.Changes = append(result.Changes, FileChange{Path: c.name, Action: action})
}
sort.Slice(result.Changes, func(i, j int) bool { return result.Changes[i].Path < result.Changes[j].Path })
if dryRun {
return result, nil
}
if check {
if len(changes) > 0 {
var stale []string
for _, c := range changes {
stale = append(stale, c.name)
}
sort.Strings(stale)
return result, fmt.Errorf("api: generated project is stale: %s", strings.Join(stale, ", "))
}
return result, nil
}
// A rollback restores bytes/modes if an ordinary filesystem write fails.
// File replacement is atomic; the whole directory is not crash-atomic.
var applied []change
for _, c := range changes {
if err = safePath(root, c.name); err == nil {
current, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(c.name)))
if c.existed {
if e != nil || !bytes.Equal(current, c.before) {
err = fmt.Errorf("file changed during generation: %s", c.name)
}
} else if !errors.Is(e, os.ErrNotExist) {
err = fmt.Errorf("file appeared during generation: %s", c.name)
}
}
if err == nil {
if c.remove {
err = os.Remove(filepath.Join(root, filepath.FromSlash(c.name)))
} else {
err = atomicWrite(root, c.name, c.after, c.mode)
}
}
if err != nil {
for i := len(applied) - 1; i >= 0; i-- {
previous := applied[i]
var rollback error
if previous.existed {
rollback = atomicWrite(root, previous.name, previous.before, previous.mode)
} else {
rollback = os.Remove(filepath.Join(root, filepath.FromSlash(previous.name)))
}
err = errors.Join(err, rollback)
}
return result, err
}
applied = append(applied, c)
}
return result, nil
}
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
func ownedPath(name string) bool {
if name == "internal/handler/responseGen.go" || name == "internal/types/scalarsGen.go" {
return true
}
if name == "internal/apierrors/errorsGen.go" || name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {
return true
}
return strings.HasPrefix(name, "internal/handler/") && strings.HasSuffix(name, "/handlersGen.go")
}
func safePath(root, name string) error {
if filepath.IsAbs(name) || strings.Contains(name, "\\") || filepath.ToSlash(filepath.Clean(name)) != name || name == "." || strings.HasPrefix(name, "../") {
return fmt.Errorf("api: unsafe output path %q", name)
}
path := root
parts := strings.Split(name, "/")
for i, part := range parts {
path = filepath.Join(path, part)
info, err := os.Lstat(path)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return err
}
if info.Mode()&os.ModeSymlink != 0 {
return fmt.Errorf("api: refusing symlink %s", path)
}
if i < len(parts)-1 && !info.IsDir() {
return fmt.Errorf("api: expected directory %s", path)
}
if i == len(parts)-1 && !info.Mode().IsRegular() {
return fmt.Errorf("api: expected regular file %s", path)
}
}
return nil
}
func atomicWrite(root, name string, data []byte, mode os.FileMode) error {
target := filepath.Join(root, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return err
}
f, err := os.CreateTemp(filepath.Dir(target), ".apiGen-*")
if err != nil {
return err
}
defer os.Remove(f.Name())
if err = f.Chmod(mode); err == nil {
_, err = f.Write(data)
}
if err == nil {
err = f.Sync()
}
err = errors.Join(err, f.Close())
if err != nil {
return err
}
return os.Rename(f.Name(), target)
}

View File

@@ -7,6 +7,7 @@ import (
"path/filepath"
"strings"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
)
@@ -17,6 +18,11 @@ func main() {
}
switch os.Args[1] {
case "api":
if err := runAPI(os.Args[2:]); err != nil {
fmt.Fprintln(os.Stderr, "go-omni api:", err)
os.Exit(1)
}
case "migrate":
if err := runMigrate(os.Args[2:]); err != nil {
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
@@ -44,9 +50,43 @@ func main() {
func runNew(args []string) error {
options := scaffold.Options{}
var target string
var apiFile string
for index := 0; index < len(args); index++ {
arg := args[index]
switch {
case arg == "--api" || strings.HasPrefix(arg, "--api="):
_, value, inline := strings.Cut(arg, "=")
if !inline {
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "-") {
return errors.New("--api requires a file")
}
index++
value = args[index]
}
if value == "" {
return errors.New("--api requires a file")
}
apiFile = value
case arg == "--log-output" || arg == "--log-format" || arg == "--storage" || strings.HasPrefix(arg, "--log-output=") || strings.HasPrefix(arg, "--log-format=") || strings.HasPrefix(arg, "--storage="):
name, value, hasValue := strings.Cut(arg, "=")
if !hasValue {
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "--") {
return fmt.Errorf("%s requires a value", name)
}
index++
value = args[index]
}
if strings.TrimSpace(value) == "" {
return fmt.Errorf("%s requires a value", name)
}
switch name {
case "--log-output":
options.LogOutput = value
case "--log-format":
options.LogFormat = value
case "--storage":
options.Storage = value
}
case arg == "--force":
options.Force = true
case arg == "--database":
@@ -98,10 +138,26 @@ func runNew(args []string) error {
if target == "" {
return errors.New("project directory is required")
}
if options.Module == "" {
if options.Module == "" && apiFile == "" {
options.Module = filepath.Base(filepath.Clean(target))
}
options.Dir = target
if apiFile != "" {
if options.Force {
return errors.New("--api protects existing business code and does not accept --force")
}
_, err := apigen.Generate(apigen.Options{API: apiFile, Dir: target, Module: options.Module, FrameworkDir: options.FrameworkDir, FrameworkVersion: options.FrameworkVersion, LogOutput: options.LogOutput, LogFormat: options.LogFormat, Database: options.Database, Storage: options.Storage})
if err != nil {
return err
}
fmt.Printf("created API project %s; implement internal/service operations\n", filepath.Clean(target))
return nil
}
if _, err := os.Lstat(filepath.Join(target, ".goOmni", "apiManifest.json")); err == nil {
return errors.New("this is an API project; use api generate instead of the CRUD/module starter")
} else if !errors.Is(err, os.ErrNotExist) {
return err
}
if err := scaffold.Generate(options); err != nil {
return err
}
@@ -115,10 +171,19 @@ func usage() {
fmt.Fprintln(os.Stderr, "Usage:")
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
fmt.Fprintln(os.Stderr, " go-omni api new [--module <module>] <directory> Create a runnable API project")
fmt.Fprintln(os.Stderr, " go-omni api generate [--api <file.api>] [--dir <project>] [--check|--dry-run]")
fmt.Fprintln(os.Stderr, " go-omni api inspect [--api <file.api>] Show routes, auth and business errors")
fmt.Fprintln(os.Stderr, " go-omni api export [--api <file.api>] Write OpenAPI JSON to stdout")
fmt.Fprintln(os.Stderr, " go-omni api validate --api <file.api> Validate definitions and OpenAPI")
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
fmt.Fprintln(os.Stderr, " --database <memory|mysql> Select generated storage (default memory)")
fmt.Fprintln(os.Stderr, " --database <none|memory|mysql> API defaults to none; CRUD defaults to memory")
fmt.Fprintln(os.Stderr, " --log-output <stdout|stderr|file> Generate configured logging (default stderr)")
fmt.Fprintln(os.Stderr, " --log-format <json|text> Select structured log format (default json)")
fmt.Fprintln(os.Stderr, " --storage <none|local|s3|r2|oss> Select optional storage adapter (default none)")
}
func runAdd(args []string) error {

View File

@@ -73,3 +73,28 @@ func TestNewMySQLDatabaseOptions(t *testing.T) {
}
}
}
func TestModuleOptionCLI(t *testing.T) {
for _, flags := range [][]string{
{"--log-output", "file", "--log-format", "json", "--storage", "local"},
{"--log-output=stdout", "--log-format=text", "--storage=none"},
} {
project := filepath.Join(t.TempDir(), "demo")
args := append([]string{"--module", "example.com/demo"}, flags...)
if err := runNew(append(args, project)); err != nil {
t.Fatal(err)
}
if _, err := os.Stat(filepath.Join(project, "etc/config.yaml")); err != nil {
t.Fatal(err)
}
}
for _, args := range [][]string{
{"--storage"}, {"--storage="}, {"--storage", "--force"},
{"--log-output"}, {"--log-output="}, {"--log-format="},
{"--storage", "unknown", "demo"}, {"--log-format", "xml", "demo"},
} {
if err := runNew(args); err == nil {
t.Fatal("invalid module option accepted", args)
}
}
}

View File

@@ -0,0 +1,29 @@
package scaffold
import (
"fmt"
"strings"
)
// APIInfrastructure renders shared infrastructure without inventing business
// tables or routes. The API generator applies these as developer-owned seeds.
func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true}
files := map[string]string{
"internal/settings/config.go": "templates/modules-settings.go.tmpl",
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
"internal/servicecontext/context.go": "templates/api-context.go.tmpl",
}
if storage != "none" {
files["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
}
result := make(map[string][]byte, len(files))
for name, source := range files {
content, err := render(source, data)
if err != nil {
return nil, fmt.Errorf("render %s: %w", name, err)
}
result[name] = []byte(strings.TrimRight(content, "\n") + "\n")
}
return result, nil
}

View File

@@ -0,0 +1,143 @@
package scaffold
import (
"context"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
)
type moduleProfile struct{ database, storage, output, format string }
var moduleProfiles = []moduleProfile{
{"memory", "none", "stdout", "json"},
{"memory", "local", "file", "json"},
{"memory", "s3", "stdout", "text"},
{"memory", "r2", "stderr", "json"},
{"memory", "oss", "stderr", "text"},
{"mysql", "none", "file", "text"},
{"mysql", "local", "stdout", "json"},
{"mysql", "s3", "file", "text"},
{"mysql", "r2", "stderr", "text"},
{"mysql", "oss", "file", "json"},
}
func generateProfile(t *testing.T, p moduleProfile) string {
t.Helper()
project := filepath.Join(t.TempDir(), "project")
if err := Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: p.database, Storage: p.storage, LogOutput: p.output, LogFormat: p.format}); err != nil {
t.Fatal(err)
}
return project
}
func TestGenerateModuleProfiles(t *testing.T) {
for _, p := range moduleProfiles {
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
project := generateProfile(t, p)
for _, name := range []string{"etc/config.yaml", "internal/settings/config.go", "MODULES.md", ".gitignore"} {
if _, err := os.Stat(filepath.Join(project, name)); err != nil {
t.Fatal(err)
}
}
_, err := os.Stat(filepath.Join(project, "internal/handler/files.go"))
if p.storage == "none" && !os.IsNotExist(err) {
t.Fatal("unselected storage was emitted")
}
if p.storage != "none" && err != nil {
t.Fatal(err)
}
module, err := os.ReadFile(filepath.Join(project, "go.mod"))
if err != nil {
t.Fatal(err)
}
pieces := strings.Split(string(module), "=> ")
if len(pieces) != 2 {
t.Fatal("local replace missing")
}
relative, err := strconv.Unquote(strings.TrimSpace(pieces[1]))
if err != nil {
t.Fatal(err)
}
if _, err = os.Stat(filepath.Join(project, relative, "go.mod")); err != nil {
t.Fatal("replacement does not resolve", err)
}
cfg, _ := os.ReadFile(filepath.Join(project, "etc/config.yaml"))
if strings.Contains(string(cfg), "secretKey:") || strings.Contains(string(cfg), "filesToken:") {
t.Fatal("secret defaults emitted")
}
})
}
}
func TestModuleOptionsRejectBeforeWriting(t *testing.T) {
for _, options := range []Options{
{LogOutput: "invalid"}, {LogFormat: "xml"}, {Storage: "ftp"},
{Storage: "local", FrameworkVersion: "v0.6.0-preview"},
{Module: "example.com/x\nrequire malicious v1.0.0"},
} {
options.Dir = filepath.Join(t.TempDir(), "project")
if options.Module == "" {
options.Module = "example.com/profile"
}
if err := Generate(options); err == nil {
t.Fatalf("invalid options accepted: %+v", options)
}
if _, err := os.Stat(options.Dir); !os.IsNotExist(err) {
t.Fatal("invalid options touched target")
}
}
}
func TestCanonicalPathBeforeDirectoryExists(t *testing.T) {
parent := t.TempDir()
alias := filepath.Join(t.TempDir(), "alias")
if err := os.Symlink(parent, alias); err != nil {
t.Skip(err)
}
path := filepath.Join(alias, "new", "nested")
if got, want := canonicalPath(path), filepath.Join(canonicalPath(parent), "new", "nested"); got != want {
t.Fatalf("%s != %s", got, want)
}
}
// Compile consuming modules, then run their generated tests. No provider account
// is used: cloud constructors are checked offline, file workflows use Local.
func TestModuleProfilesIntegration(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-profile tests")
}
for _, p := range moduleProfiles {
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
project := generateProfile(t, p)
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-p=2", "-count=1", "-timeout=60s", "./...")
cmd.Dir = project
cmd.Env = append(os.Environ(), "GOWORK=off", "GOMAXPROCS=2", "GO_OMNI_TEST_MYSQL_DSN=")
if output, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("generated %s/%s: %v\n%s", p.database, p.storage, err, output)
}
})
}
}
func TestForceRejectsRemovedDependenciesWithoutTouchingCode(t *testing.T) {
project := generateProfile(t, moduleProfile{"mysql", "local", "file", "json"})
path := filepath.Join(project, "cmd/server/main.go")
before, err := os.ReadFile(path)
if err != nil {
t.Fatal(err)
}
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Storage: "none"}); err == nil {
t.Fatal("incompatible force overwrite accepted")
}
after, _ := os.ReadFile(path)
if string(after) != string(before) {
t.Fatal("failed profile switch changed code")
}
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Database: "mysql", Storage: "local", LogOutput: "stdout"}); err != nil {
t.Fatal("compatible force overwrite rejected", err)
}
}

View File

@@ -12,6 +12,9 @@ import (
"strconv"
"strings"
"text/template"
"golang.org/x/mod/module"
"golang.org/x/mod/semver"
)
//go:embed templates moduleTemplates
@@ -24,6 +27,9 @@ type Options struct {
FrameworkDir string
FrameworkVersion string
Database string
LogOutput string
LogFormat string
Storage string
Force bool
}
@@ -48,6 +54,31 @@ func Generate(options Options) error {
if strings.TrimSpace(options.Module) == "" {
return errors.New("module path is required")
}
if err := module.CheckImportPath(options.Module); err != nil {
return fmt.Errorf("invalid module path: %w", err)
}
modules := options.LogOutput != "" || options.LogFormat != "" || options.Storage != ""
logOutput, logFormat, storage := options.LogOutput, options.LogFormat, options.Storage
if logOutput == "" {
logOutput = "stderr"
}
if logFormat == "" {
logFormat = "json"
}
if storage == "" {
storage = "none"
}
if logOutput != "stdout" && logOutput != "stderr" && logOutput != "file" {
return fmt.Errorf("unsupported log output %q: choose stdout, stderr or file", logOutput)
}
if logFormat != "json" && logFormat != "text" {
return fmt.Errorf("unsupported log format %q: choose json or text", logFormat)
}
switch storage {
case "none", "local", "s3", "r2", "oss":
default:
return fmt.Errorf("unsupported object storage %q: choose none, local, s3, r2 or oss", storage)
}
database := strings.ToLower(strings.TrimSpace(options.Database))
if database == "" {
database = "memory"
@@ -58,6 +89,9 @@ func Generate(options Options) error {
version := strings.TrimSpace(options.FrameworkVersion)
if version == "" {
version = currentFrameworkVersion()
if modules && version == defaultFrameworkVersion {
version = modulesFrameworkVersion
}
}
if !validFrameworkVersion(version) {
return fmt.Errorf("framework version %q must be a complete v0 or v1 semantic version", version)
@@ -70,20 +104,20 @@ func Generate(options Options) error {
if err := checkTarget(directory, options.Force); err != nil {
return err
}
if err := os.MkdirAll(directory, 0o755); err != nil {
return fmt.Errorf("create project directory: %w", err)
}
// An explicit remote version opts out of automatic local development mode.
replace := ""
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
replace = frameworkReplace(directory, options.FrameworkDir)
}
if modules && replace == "" && semver.Compare(version, modulesFrameworkVersion) < 0 && version != installedFrameworkVersion() {
return fmt.Errorf("module templates require %s or newer; use --framework-dir for unpublished source", modulesFrameworkVersion)
}
data := templateData{
Module: options.Module,
FrameworkVersion: version,
FrameworkReplace: replace,
Database: database,
Modules: modules, LogOutput: logOutput, LogFormat: logFormat, Storage: storage,
}
projectFiles := make(map[string]string, len(files)+3)
for path, source := range files {
@@ -94,11 +128,42 @@ func Generate(options Options) error {
projectFiles["schema.sql"] = "templates/schema.sql.tmpl"
projectFiles["internal/servicecontext/databaseIntegration_test.go"] = "templates/databaseIntegration_test.go.tmpl"
}
if modules {
projectFiles["cmd/server/main.go"] = "templates/modules-main.go.tmpl"
projectFiles["internal/settings/config.go"] = "templates/modules-settings.go.tmpl"
projectFiles["internal/settings/config_test.go"] = "templates/modules-settings_test.go.tmpl"
projectFiles["etc/config.yaml"] = "templates/modules-config.yaml.tmpl"
projectFiles[".gitignore"] = "templates/modules-gitignore.tmpl"
projectFiles["MODULES.md"] = "templates/modules-README.md.tmpl"
if storage != "none" {
projectFiles["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
projectFiles["internal/servicecontext/storage_test.go"] = "templates/modules-storage_test.go.tmpl"
projectFiles["internal/service/files.go"] = "templates/modules-filesService.go.tmpl"
projectFiles["internal/handler/files.go"] = "templates/modules-filesHandler.go.tmpl"
projectFiles["internal/handler/files_test.go"] = "templates/modules-filesHandler_test.go.tmpl"
}
}
if options.Force {
// Do not leave old generated Go files that reference dependencies omitted
// by this profile, and do not silently delete potentially edited code.
for _, path := range []string{"internal/repository/mysql.go", "internal/servicecontext/databaseIntegration_test.go", "schema.sql", "internal/settings/config.go", "internal/settings/config_test.go", "etc/config.yaml", "MODULES.md", "internal/servicecontext/storage.go", "internal/servicecontext/storage_test.go", "internal/service/files.go", "internal/handler/files.go", "internal/handler/files_test.go"} {
if _, emitted := projectFiles[path]; emitted {
continue
}
if _, err := os.Lstat(filepath.Join(directory, path)); err == nil {
return fmt.Errorf("existing %s conflicts with the selected profile; generate into a new directory instead of removing existing code", path)
} else if !errors.Is(err, os.ErrNotExist) {
return err
}
}
}
paths := make([]string, 0, len(projectFiles))
for path := range projectFiles {
paths = append(paths, path)
}
sort.Strings(paths)
// Render and format the whole project before touching the target directory.
contents := make(map[string]string, len(paths))
for _, path := range paths {
content, err := render(projectFiles[path], data)
if err != nil {
@@ -111,6 +176,9 @@ func Generate(options Options) error {
}
content = string(formatted)
}
contents[path] = content
}
for _, path := range paths {
target := filepath.Join(directory, filepath.FromSlash(path))
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
return fmt.Errorf("create %s: %w", filepath.Dir(path), err)
@@ -122,7 +190,7 @@ func Generate(options Options) error {
return fmt.Errorf("check %s: %w", target, err)
}
}
if err := os.WriteFile(target, []byte(content), 0o644); err != nil {
if err := os.WriteFile(target, []byte(contents[path]), 0o644); err != nil {
return fmt.Errorf("write %s: %w", target, err)
}
}
@@ -130,10 +198,15 @@ func Generate(options Options) error {
}
type templateData struct {
API bool
Module string
FrameworkVersion string
FrameworkReplace string
Database string
Modules bool
LogOutput string
LogFormat string
Storage string
}
func render(name string, data templateData) (string, error) {
@@ -202,11 +275,28 @@ func canonicalPath(path string) string {
if err != nil {
return path
}
resolved, err := filepath.EvalSymlinks(abs)
if err == nil {
return resolved
// Resolve the nearest existing ancestor too: generation now renders before
// creating directories, and macOS /var commonly resolves through /private.
candidate := abs
var tail []string
for {
resolved, err := filepath.EvalSymlinks(candidate)
if err == nil {
for i := len(tail) - 1; i >= 0; i-- {
resolved = filepath.Join(resolved, tail[i])
}
return resolved
}
if !errors.Is(err, os.ErrNotExist) {
return abs
}
parent := filepath.Dir(candidate)
if parent == candidate {
return abs
}
tail = append(tail, filepath.Base(candidate))
candidate = parent
}
return abs
}
func findFrameworkRoot() string {

View File

@@ -1,10 +1,13 @@
# GoOmni service
This service was generated with `go-omni new`.
{{if .Modules}}
**本项目启用了模块化配置,启动步骤和环境变量请先看 [MODULES.md](MODULES.md)。** 配置来自 `etc/config.yaml`,文件模块需要配置环境密钥后启动。
{{end}}
Framework dependency: `git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}`.
Storage: `{{.Database}}`.
Database: `{{.Database}}`.
{{if eq .Database "mysql"}}
先创建业务数据库,再执行项目的建表脚本(替换示例数据库名、账号和地址):

View File

@@ -0,0 +1,69 @@
package servicecontext
import (
"context"
"errors"
"sync"
{{if ne .Storage "none"}}"net/http"
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
{{if eq .Database "mysql"}}"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"github.com/go-sql-driver/mysql"{{end}}
"git.yuntuo.hk/k2safe/go-omni/health"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"{{.Module}}/internal/settings"
"{{.Module}}/internal/middleware"
)
// Context owns infrastructure, while Logger is borrowed from main.
// Add business repositories here; HTTP contracts do not define database tables.
type Context struct {
Logger *logx.Logger
Health *health.Checker
Middlewares map[string]httpx.Middleware
{{if eq .Database "mysql"}}DB *database.DB{{end}}
{{if ne .Storage "none"}}
Files storage.Store
Uploads *storage.Uploader
Downloads storage.Presigner
UploadLinks storage.UploadPresigner
FileHTTP http.Handler
storageClose func()error
{{end}}
closeMu sync.Mutex
closed bool
closeOnce sync.Once
closeErr error
}
// Pass the settings loaded by main. The optional form supports lightweight
// tests without infrastructure; selected adapters still require valid settings.
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
if ctx==nil{ctx=context.Background()}
if err=ctx.Err();err!=nil{return nil,err}
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
_=cfg
checker,err:=health.New();if err!=nil{return nil,err}
c:=&Context{Logger:logger,Health:checker,Middlewares:middleware.Defaults()}
defer func(){if err!=nil{err=errors.Join(err,c.Close())}}()
{{if eq .Database "mysql"}}
if err=cfg.Validate();err!=nil{return nil,err}
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
dsn.ParseTime=true;dsn.Loc=time.UTC
cfg.Database.DSN=dsn.FormatDSN()
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
{{end}}
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
return c,nil
}
// Close is idempotent, including startup failure and graceful shutdown.
func(c *Context)Close()error{
if c==nil{return nil}
c.closeMu.Lock();defer c.closeMu.Unlock();c.closed=true
c.closeOnce.Do(func(){
{{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}}
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
})
return c.closeErr
}

View File

@@ -23,7 +23,10 @@ func (s *Users) Create(ctx context.Context, input model.UserInput) (model.User,
if err != nil {
return model.User{}, err
}
return s.services.Users.Create(ctx, input)
user, err := s.services.Users.Create(ctx, input)
if err != nil { s.services.Logger.ErrorContext(ctx,"user.create.failed","error",err); return model.User{},err }
s.services.Logger.InfoContext(ctx,"user.created","userId",user.ID)
return user,nil
}
func (s *Users) Get(ctx context.Context, id int64) (model.User, error) {

View File

@@ -2,6 +2,10 @@ package servicecontext
import (
"context"
{{if and .Modules (ne .Storage "none")}}"errors"
"net/http"
"sync"
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
{{if eq .Database "mysql"}}"fmt"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
@@ -17,6 +21,18 @@ type Context struct {
Logger *logx.Logger
Users repository.UserRepository
Health *health.Checker
{{if and .Modules (ne .Storage "none")}}
Files storage.Store
Uploads *storage.Uploader
Downloads storage.Presigner
UploadLinks storage.UploadPresigner
FileHTTP http.Handler
storageClose func() error
closeMu sync.Mutex
closed bool
closeOnce sync.Once
closeErr error
{{end}}
{{if eq .Database "mysql"}}DB *database.DB{{end}}
}
@@ -61,6 +77,15 @@ func New(ctx context.Context, logger *logx.Logger) (*Context, error) {
// Close is safe to call on startup failure and again during application exit.
func (c *Context) Close() error {
{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}
return nil
{{if and .Modules (ne .Storage "none")}}
if c==nil{return nil}
c.closeMu.Lock();defer c.closeMu.Unlock()
c.closed=true
c.closeOnce.Do(func(){
if c.storageClose!=nil {c.closeErr=errors.Join(c.closeErr,c.storageClose())}
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
})
return c.closeErr
{{else}}{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}{{end}}
{{if not (and .Modules (ne .Storage "none"))}}return nil{{end}}
}

View File

@@ -0,0 +1,97 @@
# 按需装配的日志与文件模块
本项目选用日志输出 `{{.LogOutput}}`、格式 `{{.LogFormat}}`、对象存储 `{{.Storage}}`。数据库选择 `{{.Database}}` 独立于对象存储。
配置入口为 `etc/config.yaml`,通过 `go run ./cmd/server -config /path/to/config.yaml` 指定其他 JSON/YAML 文件;`-config ''` 只用代码默认值和环境变量。优先级:默认值 → 文件 → 非空环境变量。未知字段、无效参数会在监听前拒绝。
```sh
go mod tidy
{{if ne .Storage "none"}}# 示例文件接口使用单一共享 Token,不是用户/租户权限系统。
export GO_OMNI_FILES_TOKEN="$(openssl rand -hex 32)"
{{if eq .Storage "local"}}# 本地签名密钥需要稳定保存;换密钥后旧链接失效。
export GO_OMNI_STORAGE_SIGNING_KEY="$(openssl rand -hex 32)"
{{else}}# 配置当前存储平台已有的桶和凭据,不会自动创建桶。
export GO_OMNI_STORAGE_BUCKET='your-bucket'
export GO_OMNI_STORAGE_ACCESS_KEY='your-access-key'
export GO_OMNI_STORAGE_SECRET_KEY='your-secret-key'
{{if eq .Storage "s3"}}export GO_OMNI_STORAGE_ENDPOINT='s3.us-east-1.amazonaws.com'
export GO_OMNI_STORAGE_REGION='us-east-1'
{{else if eq .Storage "r2"}}export GO_OMNI_STORAGE_ACCOUNT_ID='your-32-character-account-id'
{{else}}export GO_OMNI_STORAGE_REGION='cn-hangzhou'
{{end}}{{end}}{{end}}
go run ./cmd/server
```
{{if eq .Database "mysql"}}还需按 README 配置独立 MySQL 数据库并应用 schema.sql;应用不自动迁移。{{end}}
默认仅监听 `127.0.0.1:8080`。`GO_OMNI_HTTP_ADDR` 覆盖地址;本地签名 URL 的外部地址通过 `GO_OMNI_STORAGE_PUBLIC_URL` 单独配置,始终以 `/storage/object` 结尾,不使用请求 Host 自动拼接可信链接。
## 日志
业务示例在创建用户/上传文件成功时通过 logx 写结构化日志。Context 原样传递 requestId,框架不自动增加用户、租户等业务字段。文件模式已配置轮转:100 MiB、10 份、7 天、压缩。调整 `server.log` 即可更改;`redactKeys` 可增加业务敏感键。
```sh
curl -H 'Content-Type: application/json' -H 'X-Request-ID: example-request' \
-d '{"name":"Example","email":"example@example.com"}' http://127.0.0.1:8080/users
```
`GO_OMNI_LOG_LEVEL`、`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_FILE` 覆盖对应项;显式设置 `GO_OMNI_LOG_OUTPUT=stdout` 且未设置 LOG_FILE 时,切换到控制台并关闭文件轮转。慢请求阈值为 500ms,可通过 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 调整。字段脱敏不扫描 msg/error 中的自由文本;不要把凭据或签名 URL 拼到日志中。
{{if ne .Storage "none"}}## 文件示例
Handler → Service → storage 接口;底层构造与连接池集中在 `internal/servicecontext/storage.go`。示例接口要求 `Authorization: Bearer $GO_OMNI_FILES_TOKEN`,它授权这个示例的全部文件,实际业务应替换认证和对象归属检查。存储适配器不承担业务授权。
```sh
printf 'hello go-omni' > /tmp/goOmniExample.txt
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" \
-H 'Content-Type: text/plain' -H 'X-Request-ID: file-example' \
--data-binary @/tmp/goOmniExample.txt http://127.0.0.1:8080/files
```
保存响应中 `data.key`,再请求下载链接(把示例 key 替换为返回值):
```sh
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
-d '{"key":"uploads/returned-key"}' http://127.0.0.1:8080/files/downloadLink
```
返回 `data.url`,在有效期内 GET 即可下载,无需再携带 Token。签发前会检查对象存在;URL 默认 5 分钟有效,不写日志,响应设置 no-store。
需要直传时先申请链接:
```sh
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
-d '{"size":13,"contentType":"text/plain"}' http://127.0.0.1:8080/files/uploadLink
```
响应包括随机 `data.key` 和 `data.request`(method/url/headers/expiresAt)。按返回的 method 和 headers 用原始文件字节上传到 URL;不能额外包 FormData。浏览器自动填写 Content-Length,其余签名 Header 原样发送;云桶需配置允许业务域名的 CORS。云端直传只绑定声明类型,实际内容需上传后验证;本地和 API 中转上传会检查允许类型及前缀嗅探结果。未完成内容验证前不要把暂存文件当作可信文件公开。
| 参数 | 默认值 / 环境变量 |
| --- | --- |
| 大小 | 32 MiB / GO_OMNI_STORAGE_MAX_BYTES |
| 类型 | text/plain、image/png、image/jpeg、application/pdf / GO_OMNI_STORAGE_CONTENT_TYPES(逗号分隔) |
| 链接有效期 | 5m / GO_OMNI_STORAGE_LINK_TTL(1s..1h) |
| 请求超时 | 30s / GO_OMNI_STORAGE_REQUEST_TIMEOUT(>0 且 <=5m) |
| 本地目录 | var/files / GO_OMNI_STORAGE_ROOT |
| 本地外部 URL | http://127.0.0.1:8080/storage/object / GO_OMNI_STORAGE_PUBLIC_URL |
| 云配置 | GO_OMNI_STORAGE_ENDPOINT、REGION、BUCKET、ACCOUNT_ID、SECURE |
| 环境密钥 | GO_OMNI_FILES_TOKEN、GO_OMNI_STORAGE_SIGNING_KEY、GO_OMNI_STORAGE_ACCESS_KEY、SECRET_KEY、SESSION_TOKEN |
全部 JSON/YAML 字段使用 lowerCamelCase。密钥字段仅允许环境变量,配置文件中写入会报未知字段;不要把运行环境文件提交到仓库。S3 的 endpoint 为 host[:port],OSS 的可选 endpoint 为 HTTPS origin;R2 从 accountId 派生官方 endpoint,region 为 auto。仅 S3/MinIO 的本地测试可设置 SECURE=false。
错误响应:未授权 401、无 Content-Length 411、超限 413、类型不允许 415、长度不符/非法 key 400、对象不存在 404、存储超时 504,其余内部错误统一隐藏细节。底层签名入口遵循存储协议返回 201/403 等状态,流式传输且访问日志保留实际状态。
云客户端只在组合处创建,使用专用有界连接池;配置正确不代表桶和网络已经可用,首次真实调用仍可能失败。不创建桶、不修改桶权限、不初始化任何业务数据库。更换平台时替换组合处适配器,业务 Service 接口保持不变。
{{end}}
## 生命周期与验证
启动失败会清理已创建资源;SIGINT/SIGTERM 后先等待 HTTP 请求退出,再关闭服务依赖,最后关闭日志。只关闭项目自己创建的资源,不关闭全局默认 HTTP Transport。
```sh
go test -race ./...
go build -o server ./cmd/server
./server
```
生成的测试覆盖配置优先级/校验{{if ne .Storage "none"}}、资源关闭幂等性和借用日志所有权、上传与临时链接、签名篡改、限额/类型错误。业务测试使用临时 Local,不访问云账号;云端配置需另做实网验收{{end}}。基础用户 CRUD 示例仍未启用认证,部署前按业务需要替换。

View File

@@ -0,0 +1,46 @@
# Non-secret settings only. Environment variables override these values.
server:
address: '127.0.0.1:8080'
enableLogger: true
slowRequestThreshold: 500ms
shutdownTimeout: 10s
log:
service: {{printf "%q" .Module}}
level: info
format: {{.LogFormat}}
{{if eq .LogOutput "file"}} file: './var/log/server.log'
rotation:
enabled: true
maxSizeMb: 100
maxBackups: 10
maxAgeDays: 7
compress: true
{{else}} output: {{.LogOutput}}
{{end}} redactKeys: []
{{if ne .Storage "none"}}storage:
provider: {{.Storage}}
maxBytes: 33554432
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
linkTtl: 5m
requestTimeout: 30s
{{if eq .Storage "local"}} root: './var/files'
publicUrl: 'http://127.0.0.1:8080/storage/object'
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
secure: true
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
region: auto
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
region: us-east-1
{{else}} region: cn-hangzhou
endpoint: '' # empty uses the official regional endpoint
{{end}}{{end}}{{end}}
{{if and .API (eq .Database "mysql")}}database:
driver: mysql
maxOpenConns: 10
maxIdleConns: 5
connMaxLifetime: 30m
connMaxIdleTime: 5m
pingTimeout: 5s
# GO_OMNI_DB_DSN supplies the secret connection string.
{{end}}

View File

@@ -0,0 +1,86 @@
package handler
import (
"context"
"crypto/subtle"
"errors"
"net/http"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/storage"
"{{.Module}}/internal/service"
)
type Files struct {service *service.Files; maxBytes int64; timeout time.Duration}
func NewFiles(service *service.Files,maxBytes int64,timeout time.Duration)*Files{return &Files{service:service,maxBytes:maxBytes,timeout:timeout}}
// RegisterRoutes protects example control endpoints with one shared demo token.
// Replace this gate with project authentication and object ownership checks.
// Signed object URLs carry their own short-lived authorization.
func (h *Files) RegisterRoutes(application *omni.App,token string,signed http.Handler)error {
if len(token)<32{return errors.New("file example requires a token of at least 32 bytes")}
bounded:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
ctx,cancel:=context.WithTimeout(c.Context(),h.timeout);defer cancel();c.WithContext(ctx);return next(c)
}}
authorized:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
if len(c.Request().Header.Values("Authorization"))!=1 || subtle.ConstantTimeCompare([]byte(c.Header("Authorization")),[]byte("Bearer "+token))!=1{return omni.Unauthorized("UNAUTHORIZED","invalid file access token")}
return next(c)
}}
group:=application.Group("/files",authorized,bounded)
group.POST("",h.Upload)
group.POST("/uploadLink",h.UploadLink)
group.POST("/downloadLink",h.DownloadLink)
if signed!=nil {application.Any("/storage/object",func(c *omni.Context)error{
signed.ServeHTTP(contextWriter{c},c.Request());return nil
},bounded)}
return nil
}
func (h *Files) Upload(c *omni.Context)error {
size:=c.Request().ContentLength
if size<0{return omni.NewError(http.StatusLengthRequired,"LENGTH_REQUIRED","Content-Length is required")}
if size>h.maxBytes{return omni.NewError(http.StatusRequestEntityTooLarge,"FILE_TOO_LARGE","file exceeds upload limit")}
body:=http.MaxBytesReader(c.Raw(),c.Request().Body,h.maxBytes)
defer body.Close()
info,err:=h.service.Upload(c.Context(),body,size,c.Header("Content-Type"))
if err!=nil{return fileError(err)}
return c.Created(info)
}
func (h *Files) UploadLink(c *omni.Context)error {
var input struct{Size int64 `json:"size" binding:"gte=0"`;ContentType string `json:"contentType" binding:"required,max=256"`}
if err:=c.BindJSON(&input);err!=nil{return err}
link,err:=h.service.UploadLink(c.Context(),input.Size,input.ContentType)
if err!=nil{return fileError(err)}
c.SetResponseHeader("Cache-Control","no-store")
return c.OK(link)
}
func (h *Files) DownloadLink(c *omni.Context)error {
var input struct{Key string `json:"key" binding:"required,max=1024"`}
if err:=c.BindJSON(&input);err!=nil{return err}
if err:=storage.Check(c.Context(),input.Key);err!=nil{return omni.BadRequest("INVALID_KEY","invalid object key")}
link,err:=h.service.DownloadLink(c.Context(),input.Key)
if err!=nil{return fileError(err)}
c.SetResponseHeader("Cache-Control","no-store")
return c.OK(map[string]string{"url":link})
}
func fileError(err error)error {
var limit *http.MaxBytesError
switch {
case errors.Is(err,storage.ErrTooLarge),errors.As(err,&limit):return omni.NewError(413,"FILE_TOO_LARGE","file exceeds upload limit").WithCause(err)
case errors.Is(err,storage.ErrContentType):return omni.NewError(415,"CONTENT_TYPE_NOT_ALLOWED","file content type is not allowed").WithCause(err)
case errors.Is(err,storage.ErrSizeMismatch):return omni.BadRequest("SIZE_MISMATCH","file size does not match Content-Length").WithCause(err)
case errors.Is(err,storage.ErrNotFound):return omni.NotFound("FILE_NOT_FOUND","file was not found").WithCause(err)
case errors.Is(err,context.DeadlineExceeded):return omni.NewError(504,"STORAGE_TIMEOUT","storage operation timed out").WithCause(err)
default:return err
}
}
// Track raw streaming handler responses through Omni so access logs retain the
// actual status (including failed signatures), without buffering object bodies.
type contextWriter struct{ctx *omni.Context}
func (w contextWriter) Header()http.Header{return w.ctx.Raw().Header()}
func (w contextWriter) WriteHeader(status int){_ = w.ctx.WriteResponse(status,nil,nil)}
func (w contextWriter) Write(data []byte)(int,error){
if !w.ctx.Responded(){if err:=w.ctx.WriteResponse(http.StatusOK,nil,nil);err!=nil{return 0,err}}
return w.ctx.Raw().Write(data)
}

View File

@@ -0,0 +1,75 @@
package handler_test
import (
"bytes"
"context"
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"net/url"
"strings"
"testing"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/storage"
"git.yuntuo.hk/k2safe/go-omni/storage/local"
"{{.Module}}/internal/handler"
"{{.Module}}/internal/service"
)
// Exercise business code using Local even when production selects a cloud
// adapter. These tests do not use cloud accounts or network credentials.
func TestFileUploadAndLinks(t *testing.T){
files,err:=local.New(t.TempDir(),64);if err!=nil{t.Fatal(err)}
policy:=storage.UploadPolicy{MaxBytes:64,ContentTypes:[]string{"text/plain"}}
signer,err:=storage.NewUploadSigner(files,"http://localhost/storage/object",bytes.Repeat([]byte{1},32),policy);if err!=nil{t.Fatal(err)}
uploader,err:=storage.NewUploader(files,policy);if err!=nil{t.Fatal(err)}
var logs bytes.Buffer
logger,err:=logx.New(logx.Config{Writer:&logs});if err!=nil{t.Fatal(err)}
logx.SetDefault(logger);defer logx.SetDefault(nil)
app:=omni.New(omni.WithLogger(true))
token:=strings.Repeat("t",32)
svc:=service.NewFiles(files,uploader,signer,signer,logger,time.Minute)
if err=handler.NewFiles(svc,64,time.Second).RegisterRoutes(app,token,signer);err!=nil{t.Fatal(err)}
request:=func(method,path,body,contentType string,authorized bool)*httptest.ResponseRecorder{
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type",contentType);r.Header.Set("X-Request-ID","upload-test")
if authorized{r.Header.Set("Authorization","Bearer "+token)}
w:=httptest.NewRecorder();app.ServeHTTP(w,r);return w
}
if w:=request("POST","/files","hello","text/plain",false);w.Code!=401{t.Fatal(w.Code)}
w:=request("POST","/files","hello","text/plain",true)
var saved struct{Data storage.Info}
if w.Code!=201 || json.Unmarshal(w.Body.Bytes(),&saved)!=nil || saved.Data.Size!=5{t.Fatal(w.Code,w.Body.String())}
raw,_:=json.Marshal(map[string]string{"key":saved.Data.Key})
w=request("POST","/files/downloadLink",string(raw),"application/json",true)
var download struct{Data struct{URL string}}
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&download)!=nil{t.Fatal(w.Code,w.Body.String())}
w=request("GET",download.Data.URL,"","",false)
if w.Code!=200 || w.Body.String()!="hello"{t.Fatal(w.Code,w.Body.String())}
u,_:=url.Parse(download.Data.URL);q:=u.Query();q.Set("signature","tampered");u.RawQuery=q.Encode()
if w=request("GET",u.String(),"","",false);w.Code!=403{t.Fatal(w.Code)}
for _,tc:=range []struct{path,body,contentType string;status int}{
{"/files",strings.Repeat("x",65),"text/plain",413},
{"/files","hello","image/png",415},
{"/files","<html>hello</html>","text/plain",415},
{"/files/downloadLink",`{"key":"../bad"}`,"application/json",400},
{"/files/downloadLink",`{"key":"missing"}`,"application/json",404},
{"/files/uploadLink",`{"size":65,"contentType":"text/plain"}`,"application/json",413},
{"/files/uploadLink",`{"size":5,"contentType":"image/png"}`,"application/json",415},
}{if w=request("POST",tc.path,tc.body,tc.contentType,true);w.Code!=tc.status{t.Fatal(tc.path,w.Code,w.Body.String())}}
w=request("POST","/files/uploadLink",`{"size":5,"contentType":"text/plain"}`,"application/json",true)
var direct struct{Data service.FileUploadLink}
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&direct)!=nil{t.Fatal(w.Code,w.Body.String())}
if w=request("PUT",direct.Data.Request.URL,"world","text/plain",false);w.Code!=201{t.Fatal(w.Code,w.Body.String())}
object,err:=files.Get(context.Background(),direct.Data.Key);if err!=nil{t.Fatal(err)};body,err:=io.ReadAll(object.Body);object.Body.Close();if err!=nil || string(body)!="world"{t.Fatal(err,string(body))}
decoder:=json.NewDecoder(&logs);sawBusiness,sawDenied:=false,false
for {var entry map[string]any;err:=decoder.Decode(&entry);if err==io.EOF{break};if err!=nil{t.Fatal(err)}
if entry["requestId"]!="upload-test"{t.Fatal("lost request correlation")}
if entry["msg"]=="file.uploaded"{sawBusiness=true}
if entry["msg"]=="http.request" && entry["path"]=="/storage/object" && entry["status"]==float64(http.StatusForbidden){sawDenied=true}
}
if !sawBusiness || !sawDenied{t.Fatal("business log or signed-handler status missing")}
}

View File

@@ -0,0 +1,61 @@
package service
import (
"context"
"crypto/rand"
"encoding/hex"
"io"
"time"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/storage"
)
// Files borrows provider-neutral capabilities; it never creates or closes clients.
// Authentication/ownership rules belong to this project, outside storage adapters.
type Files struct {
store storage.Store
uploader *storage.Uploader
downloads storage.Presigner
uploadLinks storage.UploadPresigner
logger *logx.Logger
ttl time.Duration
}
func NewFiles(store storage.Store,uploader *storage.Uploader,downloads storage.Presigner,uploadLinks storage.UploadPresigner,logger *logx.Logger,ttl time.Duration)*Files{
return &Files{store:store,uploader:uploader,downloads:downloads,uploadLinks:uploadLinks,logger:logger.With("component","files"),ttl:ttl}
}
func newFileKey()(string,error){
var id [16]byte
if _,err:=rand.Read(id[:]);err!=nil{return "",err}
return "uploads/"+hex.EncodeToString(id[:]),nil
}
func (s *Files) Upload(ctx context.Context,body io.Reader,size int64,contentType string)(storage.Info,error){
key,err:=newFileKey();if err!=nil{return storage.Info{},err}
info,err:=s.uploader.Put(ctx,key,body,size,contentType)
if err!=nil{s.logger.WarnContext(ctx,"file.upload.failed","error",err);return storage.Info{},err}
s.logger.InfoContext(ctx,"file.uploaded","key",info.Key,"size",info.Size,"contentType",info.ContentType)
return info,nil
}
type FileUploadLink struct {
Key string `json:"key"`
Request storage.UploadRequest `json:"request"`
}
func (s *Files) UploadLink(ctx context.Context,size int64,contentType string)(FileUploadLink,error){
key,err:=newFileKey();if err!=nil{return FileUploadLink{},err}
// Check the same policy before issuing a local or cloud capability.
// Cloud byte-level checks must happen in a post-upload validation workflow.
request,err:=s.uploadLinks.PresignPut(ctx,key,size,contentType,s.ttl)
if err!=nil{return FileUploadLink{},err}
s.logger.InfoContext(ctx,"file.uploadLink.issued","key",key)
return FileUploadLink{Key:key,Request:request},nil
}
func (s *Files) DownloadLink(ctx context.Context,key string)(string,error){
if err:=storage.Check(ctx,key);err!=nil{return "",err}
// Confirm the object exists without buffering its contents.
object,err:=s.store.Get(ctx,key);if err!=nil{return "",err}
if err=object.Body.Close();err!=nil{return "",err}
link,err:=s.downloads.PresignGet(ctx,key,s.ttl)
if err!=nil{return "",err}
s.logger.InfoContext(ctx,"file.downloadLink.issued","key",key)
return link.String(),nil
}

View File

@@ -0,0 +1,6 @@
/var/
/server
.env
.env.*
*.local.yaml
*.local.json

View File

@@ -0,0 +1,50 @@
package main
import (
"context"
"errors"
"flag"
"log"
"os"
"os/signal"
"syscall"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/logx"
"{{.Module}}/internal/handler"
"{{.Module}}/internal/service"
"{{.Module}}/internal/servicecontext"
"{{.Module}}/internal/settings"
)
func main(){
path:=flag.String("config","etc/config.yaml","JSON/YAML settings; empty uses defaults and environment")
flag.Parse()
if err:=run(*path);err!=nil{log.Fatal(err)}
}
func run(path string)(runErr error){
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
defer stop()
cfg,err:=settings.Load(path)
if err!=nil{return err}
logger,err:=logx.New(cfg.Server.Log)
if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,logger.Close())}()
logx.SetDefault(logger)
services,err:=servicecontext.New(ctx,logger)
if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,services.Close())}()
{{if ne .Storage "none"}}if err=services.InitStorage(ctx,cfg.Storage);err!=nil{return err}{{end}}
application:=omni.New(omni.WithConfig(cfg.Server))
application.OnShutdown(logger.Close)
application.OnShutdown(services.Close)
application.GET("/health/live",services.Health.LivenessHandler())
application.GET("/health/ready",services.Health.ReadinessHandler())
handler.NewUsers(service.NewUsers(services)).RegisterRoutes(application)
{{if ne .Storage "none"}}
files:=service.NewFiles(services.Files,services.Uploads,services.Downloads,services.UploadLinks,logger,cfg.Storage.LinkTTL)
if err=handler.NewFiles(files,cfg.Storage.MaxBytes,cfg.Storage.RequestTimeout).RegisterRoutes(application,cfg.Storage.FilesToken,services.FileHTTP);err!=nil{return err}
{{end}}
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
return application.RunContext(ctx,cfg.Server.Address)
}

View File

@@ -0,0 +1,140 @@
// Package settings owns this project's configuration; it adds no business identity model.
package settings
import (
"errors"
"os"
"strings"
"time"
{{if ne .Storage "none"}}"fmt"
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
"strconv"
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
type Config struct {
Server config.Config `json:"server"`
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
}
func Default() Config {
server:=config.Default()
server.Address="127.0.0.1:8080"
server.EnableLogger=true
server.SlowRequestThreshold=500*time.Millisecond
server.Log.Service={{printf "%q" .Module}}
server.Log.Format={{printf "%q" .LogFormat}}
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
{{if eq .Storage "r2"}}Region:"auto",{{end}}
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
}{{end}}}
}
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
// Pass an empty path to use only defaults and environment.
func Load(path string)(Config,error) {
cfg:=Default()
var err error
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
if err!=nil{return Config{},err}
// A stream explicitly selected by environment overrides a file profile.
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
}
{{if ne .Storage "none"}}
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
{{end}}
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
if err=cfg.Validate();err!=nil{return Config{},err}
return cfg,nil
}
func (c Config) Validate()error {
{{if and .API (eq .Database "mysql")}}
if err:=c.Database.Validate();err!=nil{return err}
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
{{end}}
if err:=c.Server.Validate();err!=nil{return err}
if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")}
switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")}
{{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}}
}
{{if ne .Storage "none"}}
type StorageConfig struct {
Provider string `json:"provider"`
Root string `json:"root"`
PublicURL string `json:"publicUrl"`
Endpoint string `json:"endpoint"`
Region string `json:"region"`
Bucket string `json:"bucket"`
AccountID string `json:"accountId"`
Secure bool `json:"secure"`
MaxBytes int64 `json:"maxBytes"`
ContentTypes []string `json:"contentTypes"`
LinkTTL time.Duration `json:"linkTtl"`
RequestTimeout time.Duration `json:"requestTimeout"`
// Secrets are environment-only and cannot be populated by config files.
FilesToken string `json:"-"`
SigningKey string `json:"-"`
AccessKey string `json:"-"`
SecretKey string `json:"-"`
SessionToken string `json:"-"`
}
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
func (s *StorageConfig) applyEnv()error {
values:=map[string]*string{
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
}
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
}
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
return nil
}
func (s StorageConfig) Validate()error {
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
if err:=s.Policy().Validate();err!=nil{return err}
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
{{if eq .Storage "local"}}
if s.Root=="" {return errors.New("storage.root is required")}
u,err:=url.Parse(s.PublicURL)
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
{{else}}
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
{{if eq .Storage "r2"}}
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
{{else if eq .Storage "s3"}}
u,err:=url.Parse("https://"+s.Endpoint)
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
{{else if eq .Storage "oss"}}
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
if !s.Secure{return errors.New("OSS requires secure=true")}
{{end}}
{{end}}
return nil
}
{{end}}

View File

@@ -0,0 +1,38 @@
package settings
import (
"os"
"path/filepath"
"strings"
"testing"
)
func environment(t *testing.T){
t.Helper()
for _,entry:=range os.Environ(){key,_,_:=strings.Cut(entry,"=");if strings.HasPrefix(key,"GO_OMNI_"){t.Setenv(key,"")}}
{{if ne .Storage "none"}}
t.Setenv("GO_OMNI_FILES_TOKEN",strings.Repeat("t",32))
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY",strings.Repeat("s",32))
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","fixtureAccess")
t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","fixtureSecret")
t.Setenv("GO_OMNI_STORAGE_BUCKET","testbucket")
{{if eq .Storage "s3"}}t.Setenv("GO_OMNI_STORAGE_ENDPOINT","127.0.0.1:9000"){{end}}
{{if eq .Storage "r2"}}t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID",strings.Repeat("a",32)){{end}}
{{end}}
}
func TestSettingsPrecedenceAndStrictness(t *testing.T){
environment(t)
path:=filepath.Join(t.TempDir(),"config.yaml")
if err:=os.WriteFile(path,[]byte("server:\n address: '127.0.0.1:8123'\n log:\n level: debug\n"),0600);err!=nil{t.Fatal(err)}
t.Setenv("GO_OMNI_HTTP_ADDR","127.0.0.1:9123")
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout")
cfg,err:=Load(path);if err!=nil{t.Fatal(err)}
if cfg.Server.Address!="127.0.0.1:9123" || cfg.Server.Log.Level!="debug" || cfg.Server.Log.Output!="stdout" || cfg.Server.Log.File!="" || cfg.Server.Log.Rotation.Enabled {t.Fatal("configuration precedence failed")}
for _,body:=range []string{"unexpected: true","server:\n slowRequestThreshold: -1s","server:\n log:\n format: invalid"{{if ne .Storage "none"}},"storage:\n filesToken: forbidden","storage:\n maxBytes: -1","storage:\n provider: unsupported"{{end}}}{
if err:=os.WriteFile(path,[]byte(body),0600);err!=nil{t.Fatal(err)}
if _,err:=Load(path);err==nil{t.Fatal("invalid configuration accepted")}
}
{{if ne .Storage "none"}}
t.Setenv("GO_OMNI_FILES_TOKEN","")
if _,err:=Load("");err==nil{t.Fatal("missing file token accepted")}
{{end}}
}

View File

@@ -0,0 +1,79 @@
package servicecontext
import (
"context"
"errors"
{{if eq .Storage "local"}}
"git.yuntuo.hk/k2safe/go-omni/storage/local"
{{else}}
"net"
"net/http"
"time"
{{if eq .Storage "oss"}}
ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss"
ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
{{else}}
"git.yuntuo.hk/k2safe/go-omni/storage/s3"
"github.com/minio/minio-go/v7/pkg/credentials"
{{end}}{{end}}
"git.yuntuo.hk/k2safe/go-omni/storage"
"{{.Module}}/internal/settings"
)
// InitStorage is called once, before serving requests. The composition root
// owns provider clients; business services receive only capability interfaces.
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
c.closeMu.Lock();defer c.closeMu.Unlock()
if c.closed{return errors.New("service dependencies are closed")}
if err:=ctx.Err();err!=nil{return err}
if c.Files!=nil{return errors.New("storage already initialized")}
if err:=cfg.Validate();err!=nil{return err}
{{if eq .Storage "local"}}
files,err:=local.New(cfg.Root,cfg.MaxBytes)
if err!=nil{return err}
signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy())
if err!=nil{return err}
uploader,err:=storage.NewUploader(files,cfg.Policy())
if err!=nil{return err}
c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer
{{else}}
// Dedicated bounded pool, never the process-wide default transport.
transport:=&http.Transport{
Proxy:http.ProxyFromEnvironment,
DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext,
ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32,
IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second,
ResponseHeaderTimeout:cfg.RequestTimeout,
}
initialized:=false
defer func(){if !initialized{transport.CloseIdleConnections()}}()
{{if eq .Storage "oss"}}
sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region).
WithSignatureVersion(ossSDK.SignatureVersionV4).
WithAdditionalHeaders([]string{"content-length"}).
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)).
WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout})
if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)}
files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes)
{{else}}
creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)
{{if eq .Storage "r2"}}
options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds)
if err!=nil{return err}
options.MaxBytes,options.Transport=cfg.MaxBytes,transport
{{else}}
options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport}
{{end}}
files,err:=s3.Open(options)
{{end}}
if err!=nil{return err}
uploader,err:=storage.NewUploader(files,cfg.Policy())
if err!=nil{return err}
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
initialized=true
{{end}}
return nil
}

View File

@@ -0,0 +1,38 @@
package servicecontext
import (
"bytes"
"context"
"strings"
"testing"
"git.yuntuo.hk/k2safe/go-omni/logx"
"{{.Module}}/internal/settings"
)
func TestStorageInitializationAndOwnership(t *testing.T){
cfg:=settings.Default().Storage
cfg.FilesToken=strings.Repeat("t",32)
cfg.SigningKey=strings.Repeat("s",32)
cfg.Root=t.TempDir()
cfg.AccessKey,cfg.SecretKey,cfg.Bucket="fixtureAccess","fixtureSecret","testbucket"
{{if eq .Storage "s3"}}cfg.Endpoint="127.0.0.1:9000"{{end}}
{{if eq .Storage "r2"}}cfg.AccountID=strings.Repeat("a",32){{end}}
var output bytes.Buffer
logger,err:=logx.New(logx.Config{Writer:&output});if err!=nil{t.Fatal(err)};defer logger.Close()
c:=&Context{Logger:logger}
canceled,cancel:=context.WithCancel(context.Background());cancel()
if err=c.InitStorage(canceled,cfg);err==nil || c.Files!=nil{t.Fatal("canceled startup initialized storage")}
invalid:=cfg;invalid.FilesToken=""
if err=c.InitStorage(context.Background(),invalid);err==nil || c.Files!=nil{t.Fatal("invalid settings initialized storage")}
if err=c.InitStorage(context.Background(),cfg);err!=nil{t.Fatal(err)}
if c.Files==nil || c.Uploads==nil || c.Downloads==nil || c.UploadLinks==nil{t.Fatal("dependencies not assembled")}
if err=c.InitStorage(context.Background(),cfg);err==nil{t.Fatal("duplicate initialization accepted")}
cleanup:=c.storageClose;count:=0
c.storageClose=func()error{count++;if cleanup!=nil{return cleanup()};return nil}
if err=c.Close();err!=nil{t.Fatal(err)};if err=c.Close();err!=nil{t.Fatal(err)}
logger.Info("still usable")
if count!=1 || !strings.Contains(output.String(),"still usable"){t.Fatal("resource ownership violated")}
closed:=&Context{Logger:logger}
if err=closed.Close();err!=nil{t.Fatal(err)}
if err=closed.InitStorage(context.Background(),cfg);err==nil || closed.Files!=nil{t.Fatal("closed context reopened resources")}
}

View File

@@ -10,7 +10,19 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Source builds can carry an unpublished VCS version (including +dirty).
// Keep this fallback in sync with the next release; installed CLIs use their
// downloaded module version, which also has a module checksum.
const defaultFrameworkVersion = "v0.6.0-preview"
const defaultFrameworkVersion = "v0.9.1-preview"
// This target is not a claim that the release already exists. Source users
// select a local framework checkout until the module templates are published.
const modulesFrameworkVersion = "v0.7.0-preview"
func installedFrameworkVersion() string {
info, _ := debug.ReadBuildInfo()
if info != nil && info.Main.Path == frameworkModule && info.Main.Sum != "" && validFrameworkVersion(info.Main.Version) {
return info.Main.Version
}
return ""
}
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)

View File

@@ -6,6 +6,7 @@ import (
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
)
const defaultRequestIDHeader = "X-Request-ID"
@@ -24,14 +25,18 @@ type TLSConfig struct {
// Config controls the default runtime behavior of an App.
type Config struct {
Address string `json:"address"`
Transport string `json:"transport"`
TLS TLSConfig `json:"tls"`
RequestIDHeader string `json:"requestIdHeader"`
EnableLogger bool `json:"enableLogger"`
TrustedProxies []string `json:"trustedProxies"`
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
Log logx.Config `json:"log"`
Address string `json:"address"`
Transport string `json:"transport"`
TLS TLSConfig `json:"tls"`
HTTP HTTPConfig `json:"http"`
RequestIDHeader string `json:"requestIdHeader"`
EnableLogger bool `json:"enableLogger"`
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
TrustedProxies []string `json:"trustedProxies"`
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
Log logx.Config `json:"log"`
Response response.Config `json:"response"`
ResponseEncoder response.Encoder `json:"-"`
}
func Default() Config {
@@ -46,6 +51,12 @@ func Default() Config {
// Validate checks configuration values before an application starts.
func (c Config) Validate() error {
if err := c.Response.Validate(); err != nil {
return err
}
if c.SlowRequestThreshold < 0 {
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
}
if strings.TrimSpace(c.Address) == "" {
return &ConfigError{Field: "address", Message: "is required"}
}
@@ -64,11 +75,20 @@ func (c Config) Validate() error {
// ValidateTransport checks built-in transport selection without opening files
// or sockets. Empty Transport preserves the standard net/http default.
func (c Config) ValidateTransport() error {
if err := c.HTTP.Validate(); err != nil {
return err
}
switch c.Transport {
case "", TransportNetHTTP, TransportNative, TransportHTTP2:
default:
return &ConfigError{Field: "transport", Message: "must be nethttp, native or http2"}
}
if c.Transport == TransportHTTP2 && (c.HTTP.ReadHeaderTimeout > 0 || c.HTTP.ReadTimeout > 0 || c.HTTP.WriteTimeout > 0) {
return &ConfigError{Field: "http", Message: "readHeaderTimeout, readTimeout and writeTimeout are not supported by the experimental http2 transport"}
}
if c.HTTP.ErrorLog != nil && c.Transport != "" && c.Transport != TransportNetHTTP {
return &ConfigError{Field: "http.errorLog", Message: "is supported only by the nethttp transport"}
}
cert, key := strings.TrimSpace(c.TLS.CertFile), strings.TrimSpace(c.TLS.KeyFile)
if (cert == "") != (key == "") {
return &ConfigError{Field: "tls", Message: "requires both certFile and keyFile"}
@@ -114,6 +134,11 @@ func WithLogger(enabled bool) Option {
return func(cfg *Config) { cfg.EnableLogger = enabled }
}
// WithSlowRequestThreshold enables access logs and warns after this duration.
func WithSlowRequestThreshold(threshold time.Duration) Option {
return func(cfg *Config) { cfg.SlowRequestThreshold = threshold; cfg.EnableLogger = true }
}
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
// An empty list disables forwarding-header trust. Invalid entries are rejected
// by Config.Validate and panic when passed directly to App.New.
@@ -125,16 +150,36 @@ func WithTrustedProxies(proxies ...string) Option {
// WithConfig applies a complete configuration loaded from a file or
// environment. Later options can still override individual fields.
func WithConfig(settings Config) Option {
settings.Response = settings.Response.Clone()
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
return func(cfg *Config) {
*cfg = settings
cfg.Response = settings.Response.Clone()
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
cfg.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
}
}
// WithResponse configures this application's default JSON envelope.
func WithResponse(settings response.Config) Option {
settings = settings.Clone()
return func(cfg *Config) { cfg.Response = settings.Clone(); cfg.ResponseEncoder = nil }
}
// WithResponseEncoder replaces the default encoder with application code.
// OpenAPI for custom encoders must describe their actual wire contract explicitly.
func WithResponseEncoder(encoder response.Encoder) Option {
return func(cfg *Config) { cfg.ResponseEncoder = encoder }
}
// WithLogConfig configures the logger used by the application runtime.
func WithLogConfig(settings logx.Config) Option {
return func(cfg *Config) { cfg.Log = settings }
settings.RedactKeys = append([]string(nil), settings.RedactKeys...)
return func(cfg *Config) {
cfg.Log = settings
cfg.Log.RedactKeys = append([]string(nil), settings.RedactKeys...)
}
}
// WithShutdownTimeout controls how long RunContext waits for active requests.

View File

@@ -14,6 +14,7 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
"GO_OMNI_SLOW_REQUEST_THRESHOLD",
} {
t.Setenv(key, "")
}
@@ -80,6 +81,7 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
t.Setenv("GO_OMNI_SLOW_REQUEST_THRESHOLD", "500ms")
cfg, err := LoadFromEnv("")
if err != nil {
@@ -91,6 +93,9 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
}
if cfg.SlowRequestThreshold != 500*time.Millisecond {
t.Fatal(cfg.SlowRequestThreshold)
}
}
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
@@ -142,3 +147,23 @@ func TestWithConfigAndWithLogConfig(t *testing.T) {
t.Fatalf("log config was not replaced: %+v", cfg.Log)
}
}
func TestApplyEnvPreservesProjectDefaults(t *testing.T) {
cfg := Default()
cfg.Address = "127.0.0.1:8123"
cfg.Log.RedactKeys = []string{"bankAccount"}
cfg.TrustedProxies = []string{"127.0.0.1"}
t.Setenv("OVERLAY_TEST_LOG_LEVEL", "debug")
got, err := ApplyEnv(cfg, "OVERLAY_TEST_")
if err != nil {
t.Fatal(err)
}
if got.Address != cfg.Address || got.Log.Level != "debug" {
t.Fatal(got)
}
got.Log.RedactKeys[0] = "changed"
got.TrustedProxies[0] = "192.0.2.1"
if cfg.Log.RedactKeys[0] != "bankAccount" || cfg.TrustedProxies[0] != "127.0.0.1" {
t.Fatal("caller-owned defaults mutated")
}
}

View File

@@ -16,6 +16,14 @@ func LoadFromEnv(prefix string) (Config, error) {
return applyEnv(Default(), prefix)
}
// ApplyEnv overlays nonempty environment values on a caller's defaults or file
// settings, then validates the result. Caller-owned slices are not retained.
func ApplyEnv(defaults Config, prefix string) (Config, error) {
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
return applyEnv(defaults, prefix)
}
func applyEnv(cfg Config, prefix string) (Config, error) {
prefix = strings.TrimSpace(prefix)
if prefix == "" {
@@ -40,12 +48,40 @@ func applyEnv(cfg Config, prefix string) (Config, error) {
cfg.Log.Environment = envString(prefix+"ENVIRONMENT", cfg.Log.Environment)
var err error
for _, field := range []struct {
name string
value *time.Duration
}{
{"HTTP_READ_HEADER_TIMEOUT", &cfg.HTTP.ReadHeaderTimeout},
{"HTTP_READ_TIMEOUT", &cfg.HTTP.ReadTimeout},
{"HTTP_WRITE_TIMEOUT", &cfg.HTTP.WriteTimeout},
{"HTTP_IDLE_TIMEOUT", &cfg.HTTP.IdleTimeout},
} {
if value := strings.TrimSpace(os.Getenv(prefix + field.name)); value != "" {
*field.value, err = time.ParseDuration(value)
if err != nil {
return Config{}, fmt.Errorf("%s%s must be a duration: %w", prefix, field.name, err)
}
}
}
if value := strings.TrimSpace(os.Getenv(prefix + "HTTP_MAX_HEADER_BYTES")); value != "" {
cfg.HTTP.MaxHeaderBytes, err = strconv.Atoi(value)
if err != nil {
return Config{}, fmt.Errorf("%sHTTP_MAX_HEADER_BYTES must be an integer: %w", prefix, err)
}
}
if cfg.EnableLogger, err = envBool(prefix+"ENABLE_LOGGER", cfg.EnableLogger); err != nil {
return Config{}, err
}
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
return Config{}, err
}
if value := strings.TrimSpace(os.Getenv(prefix + "SLOW_REQUEST_THRESHOLD")); value != "" {
cfg.SlowRequestThreshold, err = time.ParseDuration(value)
if err != nil {
return Config{}, fmt.Errorf("%sSLOW_REQUEST_THRESHOLD must be a duration: %w", prefix, err)
}
}
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
cfg.ShutdownTimeout, err = time.ParseDuration(value)
if err != nil {

View File

@@ -181,7 +181,9 @@ func (r *Reloader) Snapshot() Config {
r.mu.Lock()
defer r.mu.Unlock()
cfg := r.current
cfg.Response = r.current.Response.Clone()
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
cfg.Log.RedactKeys = append([]string(nil), cfg.Log.RedactKeys...)
return cfg
}
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {

45
config/http.go Normal file
View File

@@ -0,0 +1,45 @@
package config
import (
"log"
"time"
)
// HTTPConfig controls built-in HTTP servers. Positive values override the
// selected transport's defaults; zero keeps its existing behavior. Timeouts
// follow that transport's semantics, not a handler execution deadline.
type HTTPConfig struct {
ReadHeaderTimeout time.Duration `json:"readHeaderTimeout"`
ReadTimeout time.Duration `json:"readTimeout"`
WriteTimeout time.Duration `json:"writeTimeout"`
IdleTimeout time.Duration `json:"idleTimeout"`
MaxHeaderBytes int `json:"maxHeaderBytes"`
// ErrorLog is a Go-only hook for the nethttp server's internal diagnostics.
ErrorLog *log.Logger `json:"-"`
}
func (c HTTPConfig) Validate() error {
for _, field := range []struct {
name string
value time.Duration
}{
{"readHeaderTimeout", c.ReadHeaderTimeout},
{"readTimeout", c.ReadTimeout},
{"writeTimeout", c.WriteTimeout},
{"idleTimeout", c.IdleTimeout},
} {
if field.value < 0 {
return &ConfigError{Field: "http." + field.name, Message: "must be nonnegative"}
}
}
if c.MaxHeaderBytes < 0 {
return &ConfigError{Field: "http.maxHeaderBytes", Message: "must be nonnegative"}
}
return nil
}
// WithHTTPConfig sets the built-in server's HTTP options. RunWith uses a
// caller-owned Runner and does not apply these options to that runner.
func WithHTTPConfig(settings HTTPConfig) Option {
return func(cfg *Config) { cfg.HTTP = settings }
}

87
config/http_test.go Normal file
View File

@@ -0,0 +1,87 @@
package config_test
import (
"io"
"log"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
func TestHTTPFileAndEnvironment(t *testing.T) {
for _, file := range []struct{ name, body string }{
{"server.json", `{"http":{"readHeaderTimeout":"2s","readTimeout":"3s","writeTimeout":"4s","idleTimeout":"5s","maxHeaderBytes":8192}}`},
{"server.yaml", "http:\n readHeaderTimeout: 2s\n readTimeout: 3s\n writeTimeout: 4s\n idleTimeout: 5s\n maxHeaderBytes: 8192\n"},
} {
t.Run(file.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), file.name)
if err := os.WriteFile(path, []byte(file.body), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.LoadFile(path, "HTTP_TEST_")
if err != nil {
t.Fatal(err)
}
want := config.HTTPConfig{ReadHeaderTimeout: 2 * time.Second, ReadTimeout: 3 * time.Second, WriteTimeout: 4 * time.Second, IdleTimeout: 5 * time.Second, MaxHeaderBytes: 8192}
if cfg.HTTP != want {
t.Fatalf("HTTP: %+v", cfg.HTTP)
}
t.Setenv("HTTP_TEST_HTTP_READ_HEADER_TIMEOUT", "6s")
t.Setenv("HTTP_TEST_HTTP_READ_TIMEOUT", "7s")
t.Setenv("HTTP_TEST_HTTP_WRITE_TIMEOUT", "8s")
t.Setenv("HTTP_TEST_HTTP_IDLE_TIMEOUT", "9s")
t.Setenv("HTTP_TEST_HTTP_MAX_HEADER_BYTES", "16384")
got, err := config.ApplyEnv(cfg, "HTTP_TEST_")
if err != nil {
t.Fatal(err)
}
want = config.HTTPConfig{ReadHeaderTimeout: 6 * time.Second, ReadTimeout: 7 * time.Second, WriteTimeout: 8 * time.Second, IdleTimeout: 9 * time.Second, MaxHeaderBytes: 16384}
if got.HTTP != want {
t.Fatalf("overlay: %+v", got.HTTP)
}
})
}
}
func TestHTTPInvalidAndUnsupportedSettings(t *testing.T) {
for _, key := range []string{"READ_HEADER_TIMEOUT", "READ_TIMEOUT", "WRITE_TIMEOUT", "IDLE_TIMEOUT", "MAX_HEADER_BYTES"} {
for _, value := range []string{"invalid", "-1"} {
t.Run(key+value, func(t *testing.T) {
if value == "-1" && key != "MAX_HEADER_BYTES" {
value = "-1s"
}
t.Setenv("HTTP_INVALID_HTTP_"+key, value)
if _, err := config.LoadFromEnv("HTTP_INVALID_"); err == nil {
t.Fatal("invalid HTTP value accepted")
}
})
}
}
for _, http := range []config.HTTPConfig{{ReadHeaderTimeout: time.Second}, {ReadTimeout: time.Second}, {WriteTimeout: time.Second}} {
cfg := config.Default()
cfg.Transport = config.TransportHTTP2
cfg.TLS = config.TLSConfig{CertFile: "c", KeyFile: "k"}
cfg.HTTP = http
if err := cfg.ValidateTransport(); err == nil || !strings.Contains(err.Error(), "not supported") {
t.Fatalf("HTTP/2: %v", err)
}
}
cfg := config.Default()
cfg.Transport = config.TransportHTTP2
cfg.TLS = config.TLSConfig{CertFile: "c", KeyFile: "k"}
cfg.HTTP = config.HTTPConfig{IdleTimeout: time.Minute, MaxHeaderBytes: 8192}
if err := cfg.ValidateTransport(); err != nil {
t.Fatal(err)
}
for _, transport := range []string{config.TransportNative, config.TransportHTTP2} {
cfg.Transport = transport
cfg.HTTP = config.HTTPConfig{ErrorLog: log.New(io.Discard, "", 0)}
if err := cfg.ValidateTransport(); err == nil || !strings.Contains(err.Error(), "http.errorLog") {
t.Fatalf("ErrorLog silently ignored: %v", err)
}
}
}

30
contract/README.md Normal file
View File

@@ -0,0 +1,30 @@
# contract
Generated API projects use this package to compile Go field tags once and bind
each request without depending on a specific HTTP server. `New[T]` validates a
struct and returns an immutable Binder. `ForType` provides the same compiler to
the CLI. `BodySchema`, `Parameters`, and `ResponseSchema` return detached OpenAPI
snapshots. `MustNew[T]` is intended for generated, already checked declarations.
```go
type Request struct {
ID int64 `path:"id,range=[1:100]"`
Name string `json:"name" binding:"required,max=50"`
}
var request = contract.MustNew[Request]()
func Handle(c *httpx.Context) error {
var req Request
if err := request.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
return err
}
return c.OK(req.Name)
}
```
The default body limit is 1 MiB. An optional positive final argument to Bind
overrides it. Application transport read timeouts still control slow clients.
See [API.md](../API.md) for source tags, validation rules, null/presence behavior,
strict decoding and supported type limits. This API leaves the existing
`Context.BindJSON/BindQuery/BindURI` behavior unchanged.

331
contract/bind.go Normal file
View File

@@ -0,0 +1,331 @@
package contract
import (
"encoding/json"
"errors"
"fmt"
"io"
"math"
"mime"
"net/http"
"net/url"
"reflect"
"strconv"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
)
// Bind fills a fresh DTO transactionally. A JSON body, if declared, is required,
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
// parameters never overwrite path/body/header values. form means URL query only.
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
if b == nil || r == nil || b.schemaOnly {
return apierrors.Internal(errors.New("contract: nil binder or request"))
}
v := reflect.ValueOf(target)
if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ {
return apierrors.Internal(errors.New("contract: target type mismatch"))
}
limit := DefaultMaxBodyBytes
if len(maxBytes) > 1 || (len(maxBytes) == 1 && (maxBytes[0] <= 0 || maxBytes[0] == math.MaxInt64)) {
return apierrors.Internal(errors.New("contract: invalid body limit"))
}
if len(maxBytes) == 1 {
limit = maxBytes[0]
}
fresh := reflect.New(b.typ).Elem()
body := map[string]any{}
if len(b.root.schema.Properties) > 0 {
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
if err != nil || ct != "application/json" {
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json")
}
if r.Body == nil {
return invalid(errors.New("body required"))
}
if r.ContentLength > limit {
return tooLarge()
}
reader := &io.LimitedReader{R: r.Body, N: limit + 1}
dec := json.NewDecoder(reader)
dec.UseNumber()
value, err := decodeValue(dec, 0)
if reader.N == 0 {
return tooLarge()
}
if err != nil {
var maxErr *http.MaxBytesError
if errors.As(err, &maxErr) {
return tooLarge()
}
return invalid(err)
}
_, err = dec.Token()
if reader.N == 0 {
return tooLarge()
}
if err != io.EOF {
var maxErr *http.MaxBytesError
if errors.As(err, &maxErr) {
return tooLarge()
}
return invalid(errors.New("exactly one JSON value required"))
}
var ok bool
body, ok = value.(map[string]any)
if !ok {
return invalid(errors.New("body must be an object"))
}
for key := range body {
if _, ok = b.root.schema.Properties[key]; !ok {
return invalid(fmt.Errorf("unknown body field %s", key))
}
}
}
query := url.Values{}
if r.URL != nil {
var err error
query, err = url.ParseQuery(r.URL.RawQuery)
if err != nil {
return invalid(errors.New("malformed query"))
}
}
for _, f := range b.root.fields {
var value any
var present bool
if f.source == "json" {
value, present = body[f.name]
} else {
var raw []string
switch f.source {
case "path":
if s, ok := params[f.name]; ok {
raw = []string{s}
}
case "form":
raw = query[f.name]
case "header":
raw = r.Header.Values(f.name)
}
present = len(raw) > 0
if present {
var err error
value, err = fromStrings(f.value, raw)
if err != nil {
return invalid(fmt.Errorf("%s: %w", f.name, err))
}
}
}
if err := assignField(f, fresh.Field(f.index), value, present); err != nil {
return invalid(fmt.Errorf("%s: %w", f.name, err))
}
}
v.Elem().Set(fresh)
return nil
}
func invalid(err error) error {
return apierrors.BadRequest("INVALID_REQUEST", "request parameters are invalid").WithCause(err)
}
func tooLarge() error { return apierrors.New(413, "BODY_TOO_LARGE", "request body exceeds limit") }
func fromStrings(n *node, values []string) (any, error) {
n = base(n)
if n.scalar != nil {
if len(values) != 1 {
return nil, errors.New("duplicate scalar parameter")
}
return scalarText(values[0]), nil
}
if n.typ.Kind() == reflect.Slice {
out := make([]any, 0, len(values))
for _, s := range values {
v, err := fromStrings(n.elem, []string{s})
if err != nil {
return nil, err
}
out = append(out, v)
}
return out, nil
}
if len(values) != 1 {
return nil, errors.New("duplicate scalar parameter")
}
return parseScalar(values[0], n.typ)
}
func assignField(f field, dst reflect.Value, value any, present bool) error {
if !present {
if f.hasDefault {
value = f.fallback
} else if f.optional {
return nil
} else {
return errors.New("required field missing")
}
}
if err := setValue(f.value, dst, value); err != nil {
return err
}
return checkField(f, dst)
}
func setValue(n *node, dst reflect.Value, value any) error {
if n.scalar != nil {
return n.scalar.decode(dst, value)
}
if n.typ.Kind() == reflect.Interface && n.typ.NumMethod() == 0 {
if value == nil {
dst.SetZero()
} else {
dst.Set(reflect.ValueOf(value))
}
return nil
}
if value == nil {
if n.typ.Kind() == reflect.Pointer {
dst.SetZero()
return nil
}
return errors.New("null requires a pointer type")
}
if n.typ.Kind() == reflect.Pointer {
dst.Set(reflect.New(n.typ.Elem()))
return setValue(n.elem, dst.Elem(), value)
}
switch n.typ.Kind() {
case reflect.Struct:
object, ok := value.(map[string]any)
if !ok {
return errors.New("expected object")
}
for key := range object {
if _, ok = n.schema.Properties[key]; !ok {
return fmt.Errorf("unknown field %s", key)
}
}
for _, f := range n.fields {
item, exists := object[f.name]
if err := assignField(f, dst.Field(f.index), item, exists); err != nil {
return fmt.Errorf("%s: %w", f.name, err)
}
}
case reflect.Slice:
items, ok := value.([]any)
if !ok {
return errors.New("expected array")
}
dst.Set(reflect.MakeSlice(n.typ, len(items), len(items)))
for i, item := range items {
if err := setValue(n.elem, dst.Index(i), item); err != nil {
return fmt.Errorf("item %d: %w", i, err)
}
}
case reflect.Map:
items, ok := value.(map[string]any)
if !ok {
return errors.New("expected object")
}
dst.Set(reflect.MakeMapWithSize(n.typ, len(items)))
for key, item := range items {
v := reflect.New(n.typ.Elem()).Elem()
if err := setValue(n.elem, v, item); err != nil {
return fmt.Errorf("map value: %w", err)
}
dst.SetMapIndex(reflect.ValueOf(key).Convert(n.typ.Key()), v)
}
case reflect.String:
s, ok := value.(string)
if !ok {
return errors.New("expected string")
}
dst.SetString(s)
case reflect.Bool:
v, ok := value.(bool)
if !ok {
return errors.New("expected boolean")
}
dst.SetBool(v)
default:
num, ok := value.(json.Number)
if !ok {
return errors.New("expected number")
}
switch n.typ.Kind() {
case reflect.Float32, reflect.Float64:
v, err := strconv.ParseFloat(string(num), n.typ.Bits())
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
return errors.New("number out of range")
}
dst.SetFloat(v)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
v, err := strconv.ParseUint(string(num), 10, n.typ.Bits())
if err != nil {
return errors.New("invalid unsigned integer")
}
dst.SetUint(v)
default:
v, err := strconv.ParseInt(string(num), 10, n.typ.Bits())
if err != nil {
return errors.New("invalid integer")
}
dst.SetInt(v)
}
}
return nil
}
// Token decoding rejects duplicate keys at every level (encoding/json normally
// keeps the last value). Depth and body limits bound client-controlled work.
func decodeValue(dec *json.Decoder, depth int) (any, error) {
if depth > 64 {
return nil, errors.New("JSON nesting exceeds 64")
}
token, err := dec.Token()
if err != nil {
return nil, err
}
switch token {
case json.Delim('{'):
object := map[string]any{}
for dec.More() {
key, err := dec.Token()
if err != nil {
return nil, err
}
name, ok := key.(string)
if !ok {
return nil, errors.New("invalid object key")
}
if _, exists := object[name]; exists {
return nil, errors.New("duplicate JSON field")
}
value, err := decodeValue(dec, depth+1)
if err != nil {
return nil, err
}
object[name] = value
}
end, err := dec.Token()
if err != nil || end != json.Delim('}') {
return nil, errors.New("unterminated object")
}
return object, nil
case json.Delim('['):
items := []any{}
for dec.More() {
v, err := decodeValue(dec, depth+1)
if err != nil {
return nil, err
}
items = append(items, v)
}
end, err := dec.Token()
if err != nil || end != json.Delim(']') {
return nil, errors.New("unterminated array")
}
return items, nil
default:
if _, bad := token.(json.Delim); bad {
return nil, errors.New("unexpected JSON delimiter")
}
return token, nil
}
}

457
contract/contract.go Normal file
View File

@@ -0,0 +1,457 @@
// Package contract compiles API field tags into a reusable request binder and
// OpenAPI schemas. Generated services and the CLI use the same compiler.
package contract
import (
"encoding/json"
"errors"
"fmt"
"reflect"
"regexp"
"strconv"
"strings"
"git.yuntuo.hk/k2safe/go-omni/openapi"
"github.com/go-playground/validator/v10"
)
const DefaultMaxBodyBytes int64 = 1 << 20
var wireName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
var headerName = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9-]*$`)
var valuesValidator = validator.New()
// Binder is immutable after compilation and safe for concurrent requests.
type Binder struct {
root *node
typ reflect.Type
schemaOnly bool
}
type node struct {
scalar *Scalar
typ reflect.Type
schema *openapi.Schema
fields []field
elem *node
}
type field struct {
index int
name, source string
value *node
optional bool
fallback any
hasDefault bool
binding string
}
func New[T any](scalars ...Scalar) (*Binder, error) {
return ForType(reflect.TypeOf((*T)(nil)).Elem(), scalars...)
}
func MustNew[T any](scalars ...Scalar) *Binder {
b, err := New[T](scalars...)
if err != nil {
panic(err)
}
return b
}
// ForType also lets generators check the exact runtime contract before writing files.
// The root must be a struct; custom marshalers require explicit scalar registration.
func ForType(t reflect.Type, scalars ...Scalar) (*Binder, error) { return build(t, false, scalars) }
// Describe compiles schemas without loading or executing application Go types.
// The returned Binder cannot bind requests; it is for generators only.
func Describe(t reflect.Type, scalars ...Scalar) (*Binder, error) { return build(t, true, scalars) }
func build(t reflect.Type, schemaOnly bool, scalars []Scalar) (*Binder, error) {
if t == nil || t.Kind() != reflect.Struct {
return nil, errors.New("contract: root must be a struct")
}
c := compiler{scalars: map[reflect.Type]Scalar{}}
for _, scalar := range scalars {
if err := scalar.validate(schemaOnly); err != nil {
return nil, err
}
if _, exists := c.scalars[scalar.Type]; exists {
return nil, fmt.Errorf("contract: duplicate scalar type %s", scalar.Type)
}
c.scalars[scalar.Type] = scalar
}
budget := 10000
n, err := c.compile(t, true, map[reflect.Type]bool{}, 0, &budget)
if err != nil {
return nil, err
}
return &Binder{root: n, typ: t, schemaOnly: schemaOnly}, nil
}
func (c compiler) compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int, budget *int) (*node, error) {
*budget--
if *budget < 0 {
return nil, errors.New("contract: schema expansion exceeds 10000 nodes")
}
if depth > 32 {
return nil, errors.New("contract: type nesting exceeds 32")
}
if stack[t] {
return nil, errors.New("contract: recursive types are not supported")
}
stack[t] = true
defer delete(stack, t)
n := &node{typ: t, schema: &openapi.Schema{}}
if scalar, ok := c.scalars[t]; ok {
if root {
return nil, errors.New("contract: root cannot be a scalar")
}
n.scalar = &scalar
n.schema = scalar.inputSchema()
return n, nil
}
// Wire behavior must be described completely by these tags.
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
unmarshaler := reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
if t.Kind() != reflect.Pointer && (t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler)) {
return nil, errors.New("contract: custom JSON marshalers require explicit Scalar registration")
}
switch t.Kind() {
case reflect.Interface:
if t.NumMethod() != 0 {
return nil, errors.New("contract: only any / interface{} is supported")
}
case reflect.Pointer:
e, err := c.compile(t.Elem(), false, stack, depth+1, budget)
if err != nil {
return nil, err
}
n.elem = e
n.schema = nullableSchema(e.schema)
case reflect.String:
n.schema.Type = "string"
case reflect.Bool:
n.schema.Type = "boolean"
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
n.schema.Type = "integer"
if t.Bits() <= 32 {
n.schema.Format = "int32"
} else {
n.schema.Format = "int64"
}
if t.Kind() >= reflect.Uint && t.Kind() <= reflect.Uint64 {
v := float64(0)
n.schema.Minimum = &v
if t.Bits() < 64 {
max := float64((uint64(1) << uint(t.Bits())) - 1)
n.schema.Maximum = &max
if t.Bits() == 32 {
n.schema.Format = "int64"
}
} else {
n.schema.Format = ""
}
} else if t.Bits() < 32 {
min, max := -float64(int64(1)<<uint(t.Bits()-1)), float64((int64(1)<<uint(t.Bits()-1))-1)
n.schema.Minimum, n.schema.Maximum = &min, &max
}
case reflect.Float32, reflect.Float64:
n.schema.Type = "number"
if t.Kind() == reflect.Float32 {
n.schema.Format = "float"
} else {
n.schema.Format = "double"
}
case reflect.Slice, reflect.Map:
if t.Kind() == reflect.Map && t.Key().Kind() != reflect.String {
return nil, errors.New("contract: map keys must be strings")
}
if t.Kind() == reflect.Slice && t.Elem().Kind() == reflect.Uint8 && c.scalars[t.Elem()].Type == nil {
return nil, errors.New("contract: byte slices use base64 in Go; use string for binary data")
}
e, err := c.compile(t.Elem(), false, stack, depth+1, budget)
if err != nil {
return nil, err
}
n.elem = e
if t.Kind() == reflect.Slice {
n.schema.Type = "array"
n.schema.Items = e.schema
} else {
n.schema.Type = "object"
n.schema.AdditionalProperties = e.schema
}
case reflect.Struct:
n.schema.Type = "object"
allowed := false
n.schema.AdditionalPropertiesAllowed = &allowed
n.schema.Properties = map[string]*openapi.Schema{}
seen := map[string]bool{}
for i := 0; i < t.NumField(); i++ {
sf := t.Field(i)
if sf.PkgPath != "" || sf.Anonymous {
return nil, fmt.Errorf("contract: %s must be an exported named field", sf.Name)
}
f, err := c.compileField(sf, root, stack, depth+1, budget)
if err != nil {
return nil, fmt.Errorf("contract: %s: %w", sf.Name, err)
}
f.index = i
key := f.source + ":" + strings.ToLower(f.name)
if seen[key] {
return nil, fmt.Errorf("contract: duplicate field %s", f.name)
}
seen[key] = true
n.fields = append(n.fields, f)
if f.source == "json" {
n.schema.Properties[f.name] = f.value.schema
if !f.optional && !f.hasDefault {
n.schema.Required = append(n.schema.Required, f.name)
}
}
}
default:
return nil, fmt.Errorf("contract: unsupported type %s", t)
}
return n, nil
}
func (c compiler) compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool, depth int, budget *int) (field, error) {
f := field{}
// Reject typos and unsupported tags instead of silently losing validation.
tags, err := ParseTags(string(sf.Tag))
if err != nil {
return f, err
}
var sourceTag string
for key, value := range tags {
switch key {
case "json", "path", "form", "header":
if value == "-" {
return f, errors.New("ignored fields are not part of an API contract")
}
if f.source != "" {
return f, errors.New("exactly one source tag is required")
}
f.source = key
sourceTag = value
case "binding":
f.binding = value
default:
return f, fmt.Errorf("unsupported tag %q", key)
}
}
if f.source == "" {
return f, errors.New("json, path, form or header tag required")
}
if !root && f.source != "json" {
return f, errors.New("nested fields must use json tags")
}
parts := strings.Split(sourceTag, ",")
f.name = parts[0]
if (f.source == "header" && !headerName.MatchString(f.name)) || (f.source != "header" && !wireName.MatchString(f.name)) {
return f, errors.New("wire names must be lowerCamelCase (headers use HTTP names)")
}
n, err := c.compile(sf.Type, false, stack, depth, budget)
if err != nil {
return f, err
}
f.value = n
opaque := base(n).scalar
isList := base(n).typ.Kind() == reflect.Slice && opaque == nil
if isList {
opaque = base(base(n).elem).scalar
}
if opaque != nil {
if isList && (f.source == "path" || f.source == "header") {
return f, errors.New("path/header values must be scalars")
}
if f.source != "json" && !opaque.Text {
return f, errors.New("custom scalar needs text: true and UnmarshalText for path/query/header")
}
if f.binding != "" {
return f, errors.New("custom scalar constraints belong in its decoder; use optional/pointer for presence and null")
}
for _, option := range parts[1:] {
if option != "optional" && option != "omitempty" {
return f, errors.New("custom scalar supports only optional/omitempty field options")
}
f.optional = true
}
if f.source == "path" && (f.optional || sf.Type.Kind() == reflect.Pointer) {
return f, errors.New("path fields must be required non-pointer values")
}
return f, nil
}
scalar := base(n).typ.Kind()
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
return f, errors.New("path/query/header values must be scalars or scalar slices")
}
if f.source == "path" && !isScalar(scalar) {
return f, errors.New("path values must be scalars")
}
if f.source == "header" && !isScalar(scalar) {
return f, errors.New("header values must be scalars")
}
options := map[string]bool{}
for _, option := range parts[1:] {
key, value, _ := strings.Cut(option, "=")
if options[key] {
return f, fmt.Errorf("duplicate option %q", key)
}
options[key] = true
switch key {
case "optional", "omitempty":
if option != key {
return f, errors.New("optional takes no value")
}
f.optional = true
case "default":
if !isScalar(scalar) {
return f, errors.New("default requires a scalar")
}
v, e := parseScalar(value, base(n).typ)
if e != nil {
return f, fmt.Errorf("invalid default: %w", e)
}
f.fallback = v
f.hasDefault = true
n.schema.Default = v
case "options":
if !isScalar(scalar) || value == "" {
return f, errors.New("options requires scalar values")
}
for _, s := range strings.Split(value, "|") {
v, e := parseScalar(s, base(n).typ)
if e != nil {
return f, e
}
n.schema.Enum = append(n.schema.Enum, v)
}
case "range":
if !isNumber(scalar) {
return f, errors.New("range requires numeric type")
}
if err = applyRange(n.schema, value); err != nil {
return f, err
}
default:
return f, fmt.Errorf("unsupported field option %q", option)
}
}
if err = applyBinding(&f); err != nil {
return f, err
}
if n.schema.Nullable && len(n.schema.Enum) > 0 {
n.schema.Enum = append(n.schema.Enum, nil)
}
if f.source == "path" && (f.optional || f.hasDefault || sf.Type.Kind() == reflect.Pointer) {
return f, errors.New("path fields must be required non-pointer values")
}
if f.hasDefault {
v := reflect.New(sf.Type).Elem()
if err = setValue(n, v, f.fallback); err != nil {
return f, err
}
if err = checkField(f, v); err != nil {
return f, fmt.Errorf("default violates validation: %w", err)
}
}
return f, nil
}
// ParseTags strictly parses Go struct tags, including duplicate/unknown key checks
// performed by the compiler. It is also used to normalize generated Go JSON tags.
func ParseTags(raw string) (map[string]string, error) {
out := map[string]string{}
for strings.TrimSpace(raw) != "" {
raw = strings.TrimLeft(raw, " ")
i := strings.Index(raw, ":")
if i < 1 {
return nil, errors.New("invalid struct tag")
}
key := raw[:i]
raw = raw[i+1:]
if strings.ContainsAny(key, " \t\r\n\"`") || len(raw) == 0 || raw[0] != '"' {
return nil, errors.New("invalid struct tag")
}
end := 1
for end < len(raw) {
if raw[end] == '\\' {
end += 2
continue
}
if raw[end] == '"' {
break
}
end++
}
if end >= len(raw) {
return nil, errors.New("unterminated struct tag")
}
v, err := strconv.Unquote(raw[:end+1])
if err != nil {
return nil, err
}
if _, ok := out[key]; ok {
return nil, fmt.Errorf("duplicate tag %q", key)
}
out[key] = v
raw = raw[end+1:]
if raw != "" && raw[0] != ' ' {
return nil, errors.New("struct tags require spaces")
}
}
return out, nil
}
func base(n *node) *node {
for n.typ.Kind() == reflect.Pointer {
n = n.elem
}
return n
}
func isNumber(k reflect.Kind) bool {
return k >= reflect.Int && k <= reflect.Float64 && k != reflect.Uintptr
}
func isScalar(k reflect.Kind) bool { return k == reflect.String || k == reflect.Bool || isNumber(k) }
func (b *Binder) BodySchema() *openapi.Schema {
if len(b.root.schema.Properties) == 0 {
return nil
}
return copySchema(b.root.schema)
}
// ResponseSchema rejects request-only fields to keep response docs honest.
func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
for _, f := range b.root.fields {
if f.source != "json" {
return nil, errors.New("contract: response fields must use json tags")
}
}
return responseSchema(b.root), nil
}
func (b *Binder) Parameters() []openapi.Parameter {
var out []openapi.Parameter
for _, f := range b.root.fields {
if f.source == "json" {
continue
}
in := f.source
if in == "form" {
in = "query"
}
schema := copySchema(f.value.schema)
if base(f.value).scalar != nil {
schema = &openapi.Schema{Type: "string"}
} else if base(f.value).typ.Kind() == reflect.Slice && base(base(f.value).elem).scalar != nil {
schema.Items = &openapi.Schema{Type: "string"}
}
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: schema})
}
return out
}
func copySchema(s *openapi.Schema) *openapi.Schema {
data, _ := json.Marshal(s)
var out openapi.Schema
_ = json.Unmarshal(data, &out)
return &out
}

244
contract/contract_test.go Normal file
View File

@@ -0,0 +1,244 @@
package contract
import (
"encoding/json"
"net/http/httptest"
"reflect"
"strings"
"sync"
"testing"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
)
type address struct {
City string `json:"city" binding:"required,max=30"`
}
type mixed struct {
ID int64 `path:"userId,range=[1:100]"`
Page int `form:"page,default=1,range=[1:50]"`
Tags []string `form:"tag,optional" binding:"max=3"`
Client string `header:"X-Client"`
Name string `json:"name" binding:"required,max=20"`
Active bool `json:"active"`
Count *int `json:"count,optional"`
Contact address `json:"contact"`
Items []address `json:"items,optional"`
Labels map[string]address `json:"labels,optional"`
}
const validBody = `{"name":"小明","active":false,"count":0,"contact":{"city":"杭州"}}`
func bindMixed(body, query string) (mixed, error) {
b := MustNew[mixed]()
r := httptest.NewRequest("POST", "/users/7"+query, strings.NewReader(body))
r.Header.Set("Content-Type", "application/json")
r.Header.Set("X-Client", "web")
var v mixed
return v, b.Bind(r, map[string]string{"userId": "7"}, &v)
}
func TestMixedSourcesAndPresence(t *testing.T) {
b := MustNew[mixed]()
r := httptest.NewRequest("POST", "/users/7?tag=a&tag=b&userId=99&name=query", strings.NewReader(validBody))
r.Header.Set("Content-Type", "application/json; charset=utf-8")
r.Header.Set("X-Client", "web")
v := mixed{Name: "previous"}
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil {
t.Fatal(err)
}
if v.ID != 7 || v.Page != 1 || v.Name != "小明" || v.Active || v.Count == nil || *v.Count != 0 || len(v.Tags) != 2 || v.Client != "web" || v.Contact.City != "杭州" {
t.Fatalf("bad binding: %+v", v)
}
r = httptest.NewRequest("POST", "/", strings.NewReader(strings.Replace(validBody, `,"count":0`, "", 1)))
r.Header.Set("Content-Type", "application/json")
r.Header.Set("X-Client", "web")
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil || v.Count != nil {
t.Fatalf("missing != zero: %+v %v", v, err)
}
}
func TestRejectInvalidRequests(t *testing.T) {
cases := []struct{ name, body, query string }{
{"missing", `{"name":"a","contact":{"city":"杭州"}}`, ""},
{"empty", strings.Replace(validBody, `"小明"`, `""`, 1), ""},
{"caseSensitive", strings.Replace(validBody, `"name"`, `"Name"`, 1), ""},
{"unknown", strings.Replace(validBody, `"active":false`, `"active":false,"extra":1`, 1), ""},
{"duplicate", strings.Replace(validBody, `"active":false`, `"active":false,"active":true`, 1), ""},
{"nestedDuplicate", strings.Replace(validBody, `"city":"杭州"`, `"city":"杭州","city":"北京"`, 1), ""},
{"nestedRequired", strings.Replace(validBody, `{"city":"杭州"}`, `{}`, 1), ""},
{"nestedUnknown", strings.Replace(validBody, `{"city":"杭州"}`, `{"city":"杭州","other":0}`, 1), ""},
{"trailing", validBody + ` {}`, ""},
{"null", strings.Replace(validBody, `false`, `null`, 1), ""},
{"numberOverflow", strings.Replace(validBody, `"count":0`, `"count":99999999999999999999`, 1), ""},
{"arrayValidation", strings.TrimSuffix(validBody, "}") + `,"items":[{"city":""}]}`, ""},
{"mapValidation", strings.TrimSuffix(validBody, "}") + `,"labels":{"home":{"city":""}}}`, ""},
{"queryDuplicate", validBody, "?page=1&page=2"},
{"queryDefaultNotOnEmpty", validBody, "?page="},
{"range", validBody, "?page=51"},
{"malformedQuery", validBody, "?page=%zz"},
{"arrayLimit", validBody, "?tag=a&tag=b&tag=c&tag=d"},
}
b := MustNew[mixed]()
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
r := httptest.NewRequest("POST", "/"+tc.query, strings.NewReader(tc.body))
r.Header.Set("Content-Type", "application/json")
r.Header.Set("X-Client", "web")
v := mixed{Name: "unchanged"}
err := b.Bind(r, map[string]string{"userId": "7"}, &v)
if err == nil || apierrors.From(err).Status != 400 {
t.Fatalf("wanted 400: %v", err)
}
if v.Name != "unchanged" || v.Page != 0 {
t.Fatal("failed binding modified target")
}
})
}
}
func TestBodyLimitsAndMediaType(t *testing.T) {
type request struct {
Name string `json:"name"`
}
b := MustNew[request]()
for _, tc := range []struct {
body, ct string
limit int64
status int
}{
{`{"name":"x"}`, "text/plain", 100, 415},
{`{"name":"long"}`, "application/json", 10, 413},
{`{"name":"x"}` + strings.Repeat(" ", 2048), "application/json", 1024, 413},
{`{"name":"x"}`, "application/json", 12, 0},
} {
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
r.ContentLength = -1
r.Header.Set("Content-Type", tc.ct)
var req request
err := b.Bind(r, nil, &req, tc.limit)
if tc.status == 0 {
if err != nil {
t.Fatal(err)
}
} else if err == nil || apierrors.From(err).Status != tc.status {
t.Fatalf("status %d wanted: %v", tc.status, err)
}
}
}
func TestSchemaUsesSameRules(t *testing.T) {
b := MustNew[mixed]()
s := b.BodySchema()
if len(s.Required) != 3 || s.Properties["count"].Nullable != true || s.Properties["name"].MaxLength == nil || *s.Properties["name"].MaxLength != 20 {
t.Fatalf("wrong schema: %+v", s)
}
encoded, err := json.Marshal(s)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(encoded), `"additionalProperties":false`) {
t.Fatal(string(encoded))
}
params := b.Parameters()
if len(params) != 4 || params[1].Required || params[1].Schema.Default != json.Number("1") {
t.Fatalf("wrong parameters: %+v", params)
}
// Returned documentation snapshots do not mutate concurrent binders.
s.Properties["name"].MaxLength = nil
if b.BodySchema().Properties["name"].MaxLength == nil {
t.Fatal("shared schema")
}
if _, err = b.ResponseSchema(); err == nil {
t.Fatal("request sources in response accepted")
}
}
func TestRulesAndConcurrentReuse(t *testing.T) {
type req struct {
Score float64 `json:"score,options=1.0|2,range=(0:2]"`
Email string `json:"email" binding:"email"`
Value *int `json:"value" binding:"required"`
}
b := MustNew[req]()
var wg sync.WaitGroup
for i := 0; i < 24; i++ {
wg.Add(1)
go func() {
defer wg.Done()
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"score":1,"email":"a@example.com","value":3}`))
r.Header.Set("Content-Type", "application/json")
var v req
if err := b.Bind(r, nil, &v); err != nil {
t.Error(err)
}
}()
}
wg.Wait()
for _, body := range []string{`{"score":0,"email":"a@example.com","value":1}`, `{"score":2,"email":"bad","value":1}`, `{"score":2,"email":"a@example.com","value":null}`, `{"score":2,"email":"a@example.com","value":0}`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
r.Header.Set("Content-Type", "application/json")
var v req
if err := b.Bind(r, nil, &v); err == nil {
t.Fatal(body)
}
}
}
func TestInvalidContracts(t *testing.T) {
for _, tag := range []string{`json:"bad_name"`, `json:"a" form:"a"`, `json:"a,optionl"`, `json:"a,range=[5:2]"`, `json:"a,default=20,range=[1:10]"`, `json:"a,optional" binding:"required"`, `json:"a" binding:"unknown"`, `path:"a,optional"`, `json:"a" json:"b"`, `jsoon:"a"`, `json:"a,options=x"`} {
typ := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: reflect.TypeOf(0), Tag: reflect.StructTag(tag)}})
if _, err := ForType(typ); err == nil {
t.Fatalf("accepted %s", tag)
}
}
}
func TestExactLargeIntegerConstraints(t *testing.T) {
type req struct {
Value int64 `json:"value,range=[:9007199254740992]"`
}
b := MustNew[req]()
for _, tc := range []struct {
body string
valid bool
}{{`{"value":9007199254740992}`, true}, {`{"value":9007199254740993}`, false}} {
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
r.Header.Set("Content-Type", "application/json")
var v req
if err := b.Bind(r, nil, &v); (err == nil) != tc.valid {
t.Fatalf("%s: %v", tc.body, err)
}
}
type defaults struct {
Value uint64 `form:"value,default=18446744073709551615,options=18446744073709551615"`
}
d := MustNew[defaults]()
data, err := json.Marshal(d.Parameters())
if err != nil || !strings.Contains(string(data), "18446744073709551615") {
t.Fatalf("lost precision: %s %v", data, err)
}
}
func TestIntrinsicIntegerBounds(t *testing.T) {
type req struct {
Small uint8 `json:"small,range=[-1:500]"`
Wide uint64 `json:"wide"`
}
b := MustNew[req]()
s := b.BodySchema()
if *s.Properties["small"].Minimum != 0 || *s.Properties["small"].Maximum != 255 || s.Properties["wide"].Format != "" {
t.Fatalf("wrong integer bounds: %+v", s.Properties)
}
}
func FuzzStrictBody(f *testing.F) {
for _, seed := range []string{validBody, `{}`, `{"name":"a","name":"b"}`, `[null]`, `{"x":[]}`} {
f.Add(seed)
}
binder := MustNew[mixed]()
f.Fuzz(func(t *testing.T, body string) {
if len(body) > 16384 {
t.Skip()
}
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("X-Client", "fuzz")
var value mixed
_ = binder.Bind(req, map[string]string{"userId": "1"}, &value, 16384)
})
}

259
contract/rules.go Normal file
View File

@@ -0,0 +1,259 @@
package contract
import (
"encoding/json"
"errors"
"fmt"
"math"
"reflect"
"strconv"
"strings"
"unicode/utf8"
"git.yuntuo.hk/k2safe/go-omni/openapi"
)
func applyRange(s *openapi.Schema, value string) error {
if len(value) < 3 || (value[0] != '[' && value[0] != '(') || (value[len(value)-1] != ']' && value[len(value)-1] != ')') {
return errors.New("invalid range: use [min:max], (min:max), or mixed bounds")
}
lo, hi, ok := strings.Cut(value[1:len(value)-1], ":")
if !ok || (lo == "" && hi == "") {
return errors.New("invalid numeric range")
}
if lo != "" {
v, err := bound(lo)
if err != nil {
return err
}
if s.Minimum == nil || v > *s.Minimum || (v == *s.Minimum && value[0] == '(') {
s.Minimum = &v
s.ExclusiveMinimum = value[0] == '('
}
}
if hi != "" {
v, err := bound(hi)
if err != nil {
return err
}
if s.Maximum == nil || v < *s.Maximum || (v == *s.Maximum && value[len(value)-1] == ')') {
s.Maximum = &v
s.ExclusiveMaximum = value[len(value)-1] == ')'
}
}
return checkBounds(s)
}
func bound(s string) (float64, error) {
v, err := strconv.ParseFloat(s, 64)
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) || math.Abs(v) > 1<<53 {
return 0, errors.New("constraint bound must be finite and within ±2^53")
}
return v, nil
}
func checkBounds(s *openapi.Schema) error {
if s.Minimum != nil && s.Maximum != nil && (*s.Minimum > *s.Maximum || (*s.Minimum == *s.Maximum && (s.ExclusiveMinimum || s.ExclusiveMaximum))) {
return errors.New("empty numeric range")
}
if s.MinLength != nil && s.MaxLength != nil && *s.MinLength > *s.MaxLength {
return errors.New("empty string length range")
}
if s.MinItems != nil && s.MaxItems != nil && *s.MinItems > *s.MaxItems {
return errors.New("empty array length range")
}
return nil
}
func applyBinding(f *field) error {
if f.binding == "" {
return nil
}
n := base(f.value)
k := n.typ.Kind()
s := f.value.schema
seen := map[string]bool{}
for _, rule := range strings.Split(f.binding, ",") {
name, value, _ := strings.Cut(rule, "=")
if seen[name] {
return fmt.Errorf("duplicate binding rule %s", name)
}
seen[name] = true
switch name {
case "required":
if rule != name || f.optional || f.hasDefault {
return errors.New("required conflicts with optional/default")
}
if !isScalar(k) {
return errors.New("binding required supports scalars; other fields are already required by presence")
}
s.Nullable = false
switch k {
case reflect.String:
one := 1
s.MinLength = &one
case reflect.Bool:
s.Not = &openapi.Schema{Enum: []any{false}}
default:
s.Not = &openapi.Schema{Enum: []any{0}}
}
case "email":
if rule != name || k != reflect.String {
return errors.New("email requires a string")
}
s.Format = "email"
case "min", "max", "len":
if k == reflect.String || k == reflect.Slice {
v, err := strconv.Atoi(value)
if err != nil || v < 0 {
return errors.New("length constraints must be nonnegative integers")
}
if k == reflect.String {
if name == "min" || name == "len" {
if s.MinLength == nil || v > *s.MinLength {
s.MinLength = &v
}
}
if name == "max" || name == "len" {
s.MaxLength = &v
}
} else {
if name == "min" || name == "len" {
s.MinItems = &v
}
if name == "max" || name == "len" {
s.MaxItems = &v
}
}
} else if isNumber(k) && name != "len" {
v, err := bound(value)
if err != nil {
return err
}
if name == "min" {
if s.Minimum == nil || v > *s.Minimum {
s.Minimum = &v
s.ExclusiveMinimum = false
}
} else {
if s.Maximum == nil || v < *s.Maximum {
s.Maximum = &v
s.ExclusiveMaximum = false
}
}
} else {
return fmt.Errorf("%s requires a number, string or slice", name)
}
default:
return fmt.Errorf("unsupported binding rule %q; use required,min,max,len,email or source-tag options/range", name)
}
}
return checkBounds(s)
}
func parseScalar(raw string, t reflect.Type) (any, error) {
switch t.Kind() {
case reflect.String:
return raw, nil
case reflect.Bool:
if raw == "true" {
return true, nil
}
if raw == "false" {
return false, nil
}
return nil, errors.New("expected true or false")
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
v, err := strconv.ParseInt(raw, 10, t.Bits())
if err != nil {
return nil, errors.New("invalid integer")
}
return json.Number(strconv.FormatInt(v, 10)), nil
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
v, err := strconv.ParseUint(raw, 10, t.Bits())
if err != nil {
return nil, errors.New("invalid unsigned integer")
}
return json.Number(strconv.FormatUint(v, 10)), nil
case reflect.Float32, reflect.Float64:
v, err := strconv.ParseFloat(raw, t.Bits())
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
return nil, errors.New("invalid finite number")
}
return json.Number(strconv.FormatFloat(v, 'g', -1, t.Bits())), nil
}
return nil, errors.New("expected scalar")
}
func checkField(f field, v reflect.Value) error {
if v.Kind() == reflect.Pointer {
if v.IsNil() {
if strings.Contains(","+f.binding+",", ",required,") {
return errors.New("must not be null")
}
return nil
}
v = v.Elem()
}
if strings.Contains(","+f.binding+",", ",required,") && v.IsZero() {
return errors.New("must not be zero or empty")
}
s := f.value.schema
if s.Format == "email" && valuesValidator.Var(v.Interface(), "email") != nil {
return errors.New("invalid email")
}
if len(s.Enum) > 0 {
found := false
for _, item := range s.Enum {
option := reflect.New(v.Type()).Elem()
if setValue(base(f.value), option, item) == nil && reflect.DeepEqual(option.Interface(), v.Interface()) {
found = true
break
}
}
if !found {
return errors.New("not an allowed option")
}
}
if isNumber(v.Kind()) {
var number float64
switch v.Kind() {
case reflect.Float32, reflect.Float64:
number = v.Float()
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
number = float64(v.Uint())
if v.Uint() > 1<<53 {
number = math.Inf(1)
}
default:
number = float64(v.Int())
if v.Int() > 1<<53 {
number = math.Inf(1)
} else if v.Int() < -(1 << 53) {
number = math.Inf(-1)
}
}
if s.Minimum != nil && (number < *s.Minimum || (s.ExclusiveMinimum && number == *s.Minimum)) {
return errors.New("below minimum")
}
if s.Maximum != nil && (number > *s.Maximum || (s.ExclusiveMaximum && number == *s.Maximum)) {
return errors.New("above maximum")
}
}
if v.Kind() == reflect.String {
size := utf8.RuneCountInString(v.String())
if s.MinLength != nil && size < *s.MinLength {
return errors.New("string too short")
}
if s.MaxLength != nil && size > *s.MaxLength {
return errors.New("string too long")
}
}
if v.Kind() == reflect.Slice {
size := v.Len()
if s.MinItems != nil && size < *s.MinItems {
return errors.New("too few items")
}
if s.MaxItems != nil && size > *s.MaxItems {
return errors.New("too many items")
}
}
return nil
}

199
contract/scalar.go Normal file
View File

@@ -0,0 +1,199 @@
package contract
import (
"encoding"
"encoding/json"
"errors"
"fmt"
"reflect"
"strconv"
"strings"
"git.yuntuo.hk/k2safe/go-omni/openapi"
)
// Scalar explicitly opts one application Go type into custom wire conversion.
// Input is a | separated union of scalar wire types; Output is one wire type.
// The value must implement json.Marshaler, its pointer json.Unmarshaler, and
// Text=true additionally requires encoding.TextUnmarshaler on the pointer.
// No process-global registry or mutation of other binders is involved.
type Scalar struct {
Type reflect.Type
JSONInput, JSONOutput string
Text bool
}
func ScalarOf[T any](input, output string, text bool) Scalar {
return Scalar{Type: reflect.TypeOf((*T)(nil)).Elem(), JSONInput: input, JSONOutput: output, Text: text}
}
type compiler struct {
scalars map[reflect.Type]Scalar
}
type scalarText string
func (s Scalar) validate(schemaOnly bool) error {
if s.Type == nil || s.Type.Kind() == reflect.Pointer || s.Type.Kind() == reflect.Interface {
return errors.New("contract: scalar requires a concrete non-pointer type")
}
seen := map[string]bool{}
for _, kind := range strings.Split(s.JSONInput, "|") {
kind = strings.TrimSpace(kind)
if wireSchema(kind) == nil || seen[kind] {
return fmt.Errorf("contract: invalid/duplicate scalar input %q", kind)
}
seen[kind] = true
}
if wireSchema(s.JSONOutput) == nil {
return fmt.Errorf("contract: invalid scalar output %q", s.JSONOutput)
}
if !schemaOnly {
if !s.Type.Implements(reflect.TypeOf((*json.Marshaler)(nil)).Elem()) || !reflect.PointerTo(s.Type).Implements(reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()) {
return fmt.Errorf("contract: %s needs value MarshalJSON and pointer UnmarshalJSON", s.Type)
}
if s.Text && !reflect.PointerTo(s.Type).Implements(reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()) {
return fmt.Errorf("contract: %s needs UnmarshalText", s.Type)
}
}
return nil
}
func wireSchema(kind string) *openapi.Schema {
switch kind {
case "string":
return &openapi.Schema{Type: "string"}
case "bool":
return &openapi.Schema{Type: "boolean"}
case "int32", "int64":
return &openapi.Schema{Type: "integer", Format: kind}
case "float64":
return &openapi.Schema{Type: "number", Format: "double"}
}
return nil
}
func (s Scalar) inputSchema() *openapi.Schema {
var alternatives []*openapi.Schema
for _, kind := range strings.Split(s.JSONInput, "|") {
alternatives = append(alternatives, wireSchema(strings.TrimSpace(kind)))
}
if len(alternatives) == 1 {
return alternatives[0]
}
return &openapi.Schema{AnyOf: alternatives}
}
func (s Scalar) decode(dst reflect.Value, value any) (err error) {
// User decoders may fail; do not allow their panic to escape a binder.
defer func() {
if r := recover(); r != nil {
err = errors.New("custom scalar decoder panicked")
}
}()
if text, ok := value.(scalarText); ok {
decoder, ok := dst.Addr().Interface().(encoding.TextUnmarshaler)
if !s.Text || !ok {
return errors.New("text decoder not available")
}
return decoder.UnmarshalText([]byte(text))
}
allowed := false
for _, kind := range strings.Split(s.JSONInput, "|") {
if scalarAccepts(strings.TrimSpace(kind), value) {
allowed = true
break
}
}
if !allowed {
return errors.New("value does not match declared scalar input")
}
decoder, ok := dst.Addr().Interface().(json.Unmarshaler)
if !ok {
return errors.New("JSON decoder not available")
}
raw, err := json.Marshal(value) // json.Number preserves exact integer tokens.
if err != nil {
return err
}
return decoder.UnmarshalJSON(raw)
}
func scalarAccepts(kind string, value any) bool {
switch kind {
case "string":
_, ok := value.(string)
return ok
case "bool":
_, ok := value.(bool)
return ok
case "int32", "int64":
n, ok := value.(json.Number)
if !ok {
return false
}
bits := 64
if kind == "int32" {
bits = 32
}
_, err := strconv.ParseInt(string(n), 10, bits)
return err == nil
case "float64":
n, ok := value.(json.Number)
if !ok {
return false
}
_, err := strconv.ParseFloat(string(n), 64)
return err == nil
}
return false
}
func responseSchema(n *node) *openapi.Schema {
b := base(n)
if b.scalar != nil {
s := wireSchema(b.scalar.JSONOutput)
if n.typ.Kind() == reflect.Pointer {
s = nullableSchema(s)
}
return s
}
// Preserve the field's own constraints (including pointer range/enum and
// collection length) while changing only nested custom scalar wire types.
s := copySchema(n.schema)
if b.typ.Kind() == reflect.Slice {
s.Items = responseSchema(b.elem)
}
if b.typ.Kind() == reflect.Map {
s.AdditionalProperties = responseSchema(b.elem)
}
for _, f := range b.fields {
if f.source == "json" {
s.Properties[f.name] = responseSchema(f.value)
}
}
return s
}
// ValidateScalars checks the application's explicitly registered method contracts.
func ValidateScalars(scalars ...Scalar) error {
seen := map[reflect.Type]bool{}
for _, s := range scalars {
if err := s.validate(false); err != nil {
return err
}
if seen[s.Type] {
return fmt.Errorf("contract: duplicate scalar type %s", s.Type)
}
seen[s.Type] = true
}
return nil
}
// OpenAPI 3.0 nullable only applies alongside an explicit type. A union needs
// an explicit null alternative; an unconstrained dynamic schema already accepts null.
func nullableSchema(schema *openapi.Schema) *openapi.Schema {
s := copySchema(schema)
if s.Type != "" {
s.Nullable = true
return s
}
if len(s.AnyOf) > 0 {
s.AnyOf = append(s.AnyOf, &openapi.Schema{Type: "string", Nullable: true, Enum: []any{nil}})
}
return s
}

222
contract/scalar_test.go Normal file
View File

@@ -0,0 +1,222 @@
package contract
import (
"encoding/json"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"net/http"
"net/http/httptest"
"reflect"
"strconv"
"strings"
"testing"
)
type flexibleID int64
func (v flexibleID) MarshalJSON() ([]byte, error) {
return json.Marshal(strconv.FormatInt(int64(v), 10))
}
func (v *flexibleID) UnmarshalJSON(b []byte) error {
var s string
if len(b) > 0 && b[0] == '"' {
if err := json.Unmarshal(b, &s); err != nil {
return err
}
} else {
s = string(b)
}
return v.UnmarshalText([]byte(s))
}
func (v *flexibleID) UnmarshalText(b []byte) error {
n, err := strconv.ParseInt(string(b), 10, 64)
if err == nil {
*v = flexibleID(n)
}
return err
}
func TestDynamicJSONAndCustomScalars(t *testing.T) {
type request struct {
ID flexibleID `json:"id"`
Dynamic any `json:"dynamic"`
IDs []flexibleID `json:"ids,optional"`
}
scalar := ScalarOf[flexibleID]("string|int64", "string", true)
b, err := New[request](scalar)
if err != nil {
t.Fatal(err)
}
for _, id := range []string{`"9223372036854775807"`, `9223372036854775807`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"id":`+id+`,"dynamic":{"n":9223372036854775807},"ids":["1",2]}`))
r.Header.Set("Content-Type", "application/json")
var req request
if err = b.Bind(r, nil, &req); err != nil {
t.Fatal(err)
}
if req.ID != flexibleID(9223372036854775807) || req.Dynamic.(map[string]any)["n"] != json.Number("9223372036854775807") {
t.Fatalf("precision: %+v", req)
}
encoded, _ := json.Marshal(req)
if !strings.Contains(string(encoded), `"id":"9223372036854775807"`) {
t.Fatal(string(encoded))
}
}
for _, body := range []string{`{"id":true,"dynamic":null}`, `{"id":1.1,"dynamic":null}`, `{"id":9223372036854775808,"dynamic":null}`, `{"id":1,"dynamic":{"x":1,"x":2}}`, `{"id":1,"dynamic":null,"unexpected":1}`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
r.Header.Set("Content-Type", "application/json")
req := request{ID: 7}
if b.Bind(r, nil, &req) == nil || req.ID != 7 {
t.Fatalf("accepted or mutated %s", body)
}
}
for _, value := range []string{`null`, `[1,"x"]`, `false`, `"text"`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"id":1,"dynamic":`+value+`}`))
r.Header.Set("Content-Type", "application/json")
if err := b.Bind(r, nil, &request{}); err != nil {
t.Fatal(err)
}
}
if _, err := New[request](); err == nil {
t.Fatal("implicit custom type registration")
}
response, err := b.ResponseSchema()
if err != nil {
t.Fatal(err)
}
if response.Properties["id"].Type != "string" || response.Properties["ids"].Items.Type != "string" || len(b.BodySchema().Properties["id"].AnyOf) != 2 {
t.Fatal("input/output schemas conflated")
}
type path struct {
ID flexibleID `path:"id"`
}
p, err := New[path](scalar)
if err != nil {
t.Fatal(err)
}
var req path
if err := p.Bind(httptest.NewRequest("GET", "/", nil), map[string]string{"id": "9223372036854775807"}, &req); err != nil {
t.Fatal(err)
}
if p.Parameters()[0].Schema.Type != "string" {
t.Fatal("text parameters need text schema")
}
if _, err := New[path](ScalarOf[flexibleID]("string|int64", "string", false)); err == nil {
t.Fatal("text support not explicit")
}
schemaOnly, err := Describe(reflect.TypeOf(request{}), scalar)
if err != nil {
t.Fatal(err)
}
if schemaOnly.Bind(httptest.NewRequest("POST", "/", nil), nil, &request{}) == nil {
t.Fatal("schema-only binder executed")
}
}
func TestScalarPointersCollectionsAndRegistration(t *testing.T) {
scalar := ScalarOf[flexibleID]("string|int64", "string", true)
type nullable struct {
ID *flexibleID `json:"id,optional"`
Values map[string]*flexibleID `json:"values,optional"`
}
b, err := New[nullable](scalar)
if err != nil {
t.Fatal(err)
}
for _, body := range []string{`{}`, `{"id":null}`, `{"id":1,"values":{"a":"2","b":null}}`} {
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
r.Header.Set("Content-Type", "application/json")
if err := b.Bind(r, nil, &nullable{}); err != nil {
t.Fatal(err)
}
}
input := b.BodySchema().Properties["id"]
if len(input.AnyOf) != 3 || !input.AnyOf[2].Nullable || len(input.AnyOf[2].Enum) != 1 || input.AnyOf[2].Enum[0] != nil {
t.Fatal("nullable input union does not describe null")
}
out, err := b.ResponseSchema()
if err != nil {
t.Fatal(err)
}
if out.Properties["id"].Type != "string" || !out.Properties["id"].Nullable {
t.Fatal("nullable output type")
}
type query struct {
IDs []flexibleID `form:"ids"`
}
q, err := New[query](scalar)
if err != nil {
t.Fatal(err)
}
var req query
if err := q.Bind(httptest.NewRequest("GET", "/?ids=1&ids=2", nil), nil, &req); err != nil || !reflect.DeepEqual(req.IDs, []flexibleID{1, 2}) {
t.Fatalf("query scalar collection: %+v %v", req, err)
}
if q.Parameters()[0].Schema.Items.Type != "string" {
t.Fatal("text array schema")
}
if _, err := New[query](ScalarOf[flexibleID]("string|int64", "string", false)); err == nil {
t.Fatal("array bypassed text declaration")
}
if err := ValidateScalars(ScalarOf[int64]("int64", "string", true)); err == nil {
t.Fatal("missing methods accepted")
}
if err := ValidateScalars(scalar, scalar); err == nil {
t.Fatal("duplicate registration")
}
type strict struct {
ID int64 `json:"id"`
}
strictBinder := MustNew[strict]()
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"id":"1"}`))
r.Header.Set("Content-Type", "application/json")
if strictBinder.Bind(r, nil, &strict{}) == nil {
t.Fatal("other binder globally loosened")
}
}
type panicScalar string
func (v panicScalar) MarshalJSON() ([]byte, error) { return json.Marshal(string(v)) }
func (v *panicScalar) UnmarshalJSON([]byte) error { panic("decoder failure") }
func TestScalarPanicIsBindingError(t *testing.T) {
type req struct {
Value panicScalar `json:"value"`
}
b := MustNew[req](ScalarOf[panicScalar]("string", "string", false))
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"value":"bad"}`))
r.Header.Set("Content-Type", "application/json")
dst := req{Value: "original"}
if err := b.Bind(r, nil, &dst); err == nil || dst.Value != "original" {
t.Fatalf("decoder panic: %+v %v", dst, err)
}
}
func TestLimitedJSONTrailingWhitespace(t *testing.T) {
type req struct {
Value any `json:"value"`
}
b := MustNew[req]()
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"value":null}`+strings.Repeat(" ", 65)))
r.Header.Set("Content-Type", "application/json")
r.ContentLength = -1
r.Body = http.MaxBytesReader(httptest.NewRecorder(), r.Body, 64)
err := b.Bind(r, nil, &req{}, 64)
if apierrors.From(err).Status != 413 {
t.Fatalf("tail limit error: %v", err)
}
}
func TestResponseSchemaPreservesOrdinaryPointerConstraints(t *testing.T) {
type dto struct {
Count *int `json:"count,optional,range=[1:10]"`
Choice *string `json:"choice,options=a|b"`
Values *[]int `json:"values,optional" binding:"max=3"`
}
b := MustNew[dto]()
response, err := b.ResponseSchema()
if err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(response, b.BodySchema()) {
t.Fatalf("changed ordinary response constraints: %#v", response)
}
}

View File

@@ -3,7 +3,7 @@
`database` 包提供基于标准库 `database/sql` 的连接池生命周期,不绑定 ORM 或驱动:
- `Config` / `ConfigFromEnv`:连接池参数和 `GO_OMNI_DB_*` 环境变量。
- `Open`:创建连接池、设置连接上限并在返回前执行 `PingContext`。
- `Open` / `OpenConnector`:通过 DSN 或标准 Connector 创建连接池、设置连接上限并在返回前执行 `PingContext`。
- `Health`:用于就绪检查,等待连接和 Ping 均受 `PingTimeout` 限制。
- `InTx`:统一事务提交和回滚;工作函数返回错误或 panic 时回滚。
- `Close`:由嵌入的 `*sql.DB` 提供,应用退出时显式调用。
@@ -40,7 +40,7 @@ GO_OMNI_DB_CONN_MAX_IDLE_TIME=5m
GO_OMNI_DB_PING_TIMEOUT=5s
```
`DSN` 必填,驱动默认是 `mysql`。数据库建库、迁移、表结构和具体 Repository 属于业务项目或可选模块,不会被核心包自动执行。
使用 `ConfigFromEnv` / `Open` 时 `DSN` 必填,环境变量入口的驱动默认是 `mysql`。数据库建库、迁移、表结构和具体 Repository 属于业务项目或可选模块,不会被核心包自动执行。
`MaxIdleConns=0` 会禁用空闲连接。初始化 Ping 失败会关闭已创建的连接池。
@@ -56,3 +56,69 @@ python3 scripts/verifyDatabase.py --version v0.2.0
脚本仅操作本次创建的独立容器与数据库,验证 HTTP CRUD、服务重启持久化、事务提交与
错误/panic/取消回滚、连接池关闭、缺少表结构时启动失败,以及停库后的 readiness 503 和恢复后的 200。
结束时清理容器、匿名数据卷和临时项目。常规 `go test ./...` 不依赖 Docker。
## 项目配置叠加
`database.ApplyEnv(cfg, prefix)` 在已有文件/默认配置上覆盖非空环境变量,保留未覆盖的池参数。
`Config` 的 JSON/YAML 字段使用 lowerCamelCase,`DSN` 使用 `json:"-"`,连接密钥从环境变量加载。
API 脚手架 `--database mysql` 使用此流程,并要求正的 `maxOpenConns`,提供数据库 readiness 与幂等清理。
具体表结构、迁移和 Repository 留在业务项目,参考 `examples/userService`。
## Connector 接入(v0.9.1-preview)
项目已有 `database/sql/driver.Connector` 时使用 `OpenConnector`。这是标准驱动接口,核心包不增加具体驱动依赖。项目可以在同一进程创建多个使用不同证书、日志、拨号策略的连接池。
下面是 MySQL 的完整组装函数;连接地址、凭据、TLS 和驱动 Logger 都从调用方传入,不在框架中约定:
```go
package connection
import (
"context"
"crypto/tls"
"github.com/go-sql-driver/mysql"
"git.yuntuo.hk/k2safe/go-omni/database"
)
func Open(ctx context.Context, address, user, password, name string,
tlsConfig *tls.Config, logger mysql.Logger,
pool database.PoolConfig) (*database.DB, error) {
cfg := mysql.NewConfig()
cfg.Net, cfg.Addr = "tcp", address
cfg.User, cfg.Passwd, cfg.DBName = user, password, name
cfg.TLS = tlsConfig // 项目设置 RootCAs、ServerName、MinVersion 等
if logger != nil {
cfg.Logger = logger // 项目现有的 Print(...any) 日志适配器
}
connector, err := mysql.NewConnector(cfg)
if err != nil {
return nil, err
}
return database.OpenConnector(ctx, connector, pool)
}
```
调用时以 `database.DefaultPoolConfig()` 为起点,按项目需要调整。已有 `database.Config` 可调用 `.PoolConfig()` 提取池设置;`ConfigFromEnv` / `ApplyEnv` 仍是 DSN 入口,不用为 Connector 填假 DSN。`PoolConfig` 支持通过 `config.DecodeFile` 加载项目的 JSON/YAML 嵌套配置,环境覆盖由项目按自己的驱动配置组织。
| PoolConfig 字段 | 默认值 / 含义 |
| --- | --- |
| `MaxOpenConns` | `10`;`0` 表示不限制 |
| `MaxIdleConns` | `5`;`0` 禁止保留空闲连接 |
| `ConnMaxLifetime` | `30m`;`0` 不因连接年龄回收 |
| `ConnMaxIdleTime` | `5m`;`0` 不因空闲时长回收 |
| `PingTimeout` | `5s`;必须为正,约束启动 Ping 和 Health 等待 |
返回的 `*database.DB` 嵌入 `*sql.DB`,可使用 `QueryContext`、`ExecContext`、`Stats`,以及框架的 `Health`、`InTx`。需要标准数据库指针的 ORM/迁移器可取 `db.DB`;不要重复建立池。项目负责驱动支持的 context 取消与读写超时设置,框架不能强制中断不遵守 context 的自定义驱动。
**生命周期:** nil/带类型的 nil Connector、非法池参数、启动前已取消的 context 会在创建池前返回,Connector 仍归调用方。进入 `sql.OpenDB` 后由池持有;启动 Ping 失败自动关闭,成功后由调用方负责 `db.Close`,通常注册 `app.OnShutdown(db.Close)`。若 Connector 实现 `io.Closer`,遵循标准 `database/sql` 规则随池关闭;有资源的 Connector 不应跨多个池共享。资源初始化过程中、注册退出回调之前若业务校验失败,项目也须关闭池。
`db.PoolConfig()` 返回池配置快照。Connector 入口的 `db.Config()` 中 `Driver` / `DSN` 为空,不能用它重建 Connector。TLS、Logger、驱动版本检查、字符集、时区以及迁移规则保持由项目负责;不写入框架配置快照。
验收脚本使用独立本地 MySQL 8.4 容器(1 CPU / 768 MiB),自动生成测试 CA/证书与随机凭据,检查 TLS 信任/主机名、两池配置隔离、驱动日志、事务和 HTTP 超时,结束后清理本次资源:
```sh
python3 scripts/verifyConnector.py --local --report performance/connectorLocal.json
# 发布后通过全新 module cache 验证远程依赖,无本地 replace:
python3 scripts/verifyConnector.py --version v0.9.1-preview --report performance/connectorRemote.json
```

View File

@@ -0,0 +1,35 @@
package database
import (
"encoding/json"
"strings"
"testing"
"time"
)
func TestApplyEnvPreservesFilePoolAndHidesDSN(t *testing.T) {
cfg := DefaultConfig()
cfg.MaxOpenConns = 3
cfg.MaxIdleConns = 1
cfg.PingTimeout = time.Second
t.Setenv("OMNI_TEST_DSN", "user:secret@tcp(localhost:3306)/test")
t.Setenv("OMNI_TEST_MAX_IDLE_CONNS", "0")
got, err := ApplyEnv(cfg, "OMNI_TEST_")
if err != nil {
t.Fatal(err)
}
if got.MaxOpenConns != 3 || got.MaxIdleConns != 0 || got.PingTimeout != time.Second {
t.Fatalf("lost file defaults: %+v", got)
}
data, err := json.Marshal(got)
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(data), "secret") || !strings.Contains(string(data), "maxOpenConns") {
t.Fatalf("unsafe config encoding: %s", data)
}
t.Setenv("OMNI_TEST_MAX_OPEN_CONNS", "bad")
if _, err = ApplyEnv(cfg, "OMNI_TEST_"); err == nil {
t.Fatal("invalid override accepted")
}
}

82
database/connector.go Normal file
View File

@@ -0,0 +1,82 @@
package database
import (
"context"
"database/sql"
"database/sql/driver"
"errors"
"reflect"
"time"
)
// PoolConfig contains only driver-independent settings. Connection credentials,
// TLS, logging and dial behavior belong to the application's Connector.
// Start from DefaultPoolConfig; zero PingTimeout is invalid.
type PoolConfig struct {
MaxOpenConns int `json:"maxOpenConns"`
MaxIdleConns int `json:"maxIdleConns"`
ConnMaxLifetime time.Duration `json:"connMaxLifetime"`
ConnMaxIdleTime time.Duration `json:"connMaxIdleTime"`
PingTimeout time.Duration `json:"pingTimeout"`
}
func DefaultPoolConfig() PoolConfig {
return PoolConfig{MaxOpenConns: 10, MaxIdleConns: 5, ConnMaxLifetime: 30 * time.Minute, ConnMaxIdleTime: 5 * time.Minute, PingTimeout: 5 * time.Second}
}
func (c PoolConfig) Validate() error {
if c.MaxOpenConns < 0 || c.MaxIdleConns < 0 {
return errors.New("database pool limits cannot be negative")
}
if c.MaxOpenConns > 0 && c.MaxIdleConns > c.MaxOpenConns {
return errors.New("max idle connections cannot exceed max open connections")
}
if c.ConnMaxLifetime < 0 || c.ConnMaxIdleTime < 0 || c.PingTimeout <= 0 {
return errors.New("database timeouts are invalid")
}
return nil
}
func (c PoolConfig) config() Config {
return Config{MaxOpenConns: c.MaxOpenConns, MaxIdleConns: c.MaxIdleConns, ConnMaxLifetime: c.ConnMaxLifetime, ConnMaxIdleTime: c.ConnMaxIdleTime, PingTimeout: c.PingTimeout}
}
// OpenConnector opens an independent pool using the supplied standard Connector.
// No registered driver name or DSN is required. It applies cfg and completes a
// startup Ping bounded by ctx and PingTimeout before returning.
//
// Validation failures and an already-canceled ctx leave the Connector untouched.
// Once sql.OpenDB is called, the pool owns its lifecycle: a failed startup closes
// the pool; on success the caller must Close the returned DB. As in database/sql,
// this also closes a Connector that implements io.Closer. Use a separate Connector
// for each pool if it owns resources. Nil ctx means context.Background.
func OpenConnector(ctx context.Context, connector driver.Connector, cfg PoolConfig) (*DB, error) {
if nilConnector(connector) {
return nil, errors.New("database connector is required")
}
if err := cfg.Validate(); err != nil {
return nil, err
}
if ctx == nil {
ctx = context.Background()
}
if err := ctx.Err(); err != nil {
return nil, err
}
return initializePool(ctx, sql.OpenDB(connector), cfg.config())
}
func nilConnector(connector driver.Connector) bool {
if connector == nil {
return true
}
v := reflect.ValueOf(connector)
switch v.Kind() {
case reflect.Chan, reflect.Func, reflect.Interface, reflect.Map, reflect.Pointer, reflect.Slice:
return v.IsNil()
}
return false
}
// PoolConfig returns a copy of the settings used to create this pool.
func (db *DB) PoolConfig() PoolConfig { return db.config.PoolConfig() }

336
database/connector_test.go Normal file
View File

@@ -0,0 +1,336 @@
package database_test
import (
"context"
"database/sql"
"database/sql/driver"
"errors"
"io"
"sync"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
)
type fixtureConnector struct {
identity string
connects, closedConnections, closedConnector, commits, rollbacks atomic.Int32
connect func(context.Context) error
ping func(context.Context) error
query func(context.Context) error
closeErr error
}
func (c *fixtureConnector) Connect(ctx context.Context) (driver.Conn, error) {
c.connects.Add(1)
if c.connect != nil {
if err := c.connect(ctx); err != nil {
return nil, err
}
}
return &fixtureConn{owner: c}, nil
}
func (c *fixtureConnector) Driver() driver.Driver { return fixtureDriver{} }
func (c *fixtureConnector) Close() error { c.closedConnector.Add(1); return c.closeErr }
type fixtureDriver struct{}
func (fixtureDriver) Open(string) (driver.Conn, error) {
return nil, errors.New("unexpected driver name/DSN path")
}
type fixtureConn struct{ owner *fixtureConnector }
func (c *fixtureConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("not used") }
func (c *fixtureConn) Close() error { c.owner.closedConnections.Add(1); return nil }
func (c *fixtureConn) Begin() (driver.Tx, error) { return fixtureTx{c.owner}, nil }
func (c *fixtureConn) Ping(ctx context.Context) error {
if c.owner.ping != nil {
return c.owner.ping(ctx)
}
return nil
}
func (c *fixtureConn) QueryContext(ctx context.Context, _ string, _ []driver.NamedValue) (driver.Rows, error) {
if c.owner.query != nil {
if err := c.owner.query(ctx); err != nil {
return nil, err
}
}
return &fixtureRows{identity: c.owner.identity}, nil
}
type fixtureTx struct{ owner *fixtureConnector }
func (tx fixtureTx) Commit() error { tx.owner.commits.Add(1); return nil }
func (tx fixtureTx) Rollback() error { tx.owner.rollbacks.Add(1); return nil }
type fixtureRows struct {
identity string
sent bool
}
func (r *fixtureRows) Columns() []string { return []string{"identity"} }
func (r *fixtureRows) Close() error { return nil }
func (r *fixtureRows) Next(values []driver.Value) error {
if r.sent {
return io.EOF
}
r.sent = true
values[0] = r.identity
return nil
}
func TestConnectorIndependentPoolsAndContext(t *testing.T) {
type key struct{}
ctx := context.WithValue(context.Background(), key{}, "startup")
cfg := database.DefaultPoolConfig()
cfg.MaxOpenConns = 2
cfg.MaxIdleConns = 2
var owners []*fixtureConnector
var pools []*database.DB
for _, identity := range []string{"alpha", "beta"} {
owner := &fixtureConnector{identity: identity}
// Startup Connect/Ping and each query preserve their caller context.
// database/sql may also open replacement connections in the background.
owner.connect = func(ctx context.Context) error {
if owner.connects.Load() == 1 && ctx.Value(key{}) != "startup" {
return errors.New("lost request context")
}
return nil
}
owner.ping = func(ctx context.Context) error {
if ctx.Value(key{}) != "startup" {
return errors.New("lost ping context")
}
return nil
}
owner.query = func(ctx context.Context) error {
if ctx.Value(key{}) != "startup" {
return errors.New("lost query context")
}
return nil
}
pool, err := database.OpenConnector(ctx, owner, cfg)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { pool.Close() })
if pool.PoolConfig() != cfg || pool.Config().Driver != "" || pool.Config().DSN != "" || pool.Stats().MaxOpenConnections != 2 {
t.Fatal("pool settings or DSN boundary")
}
owners = append(owners, owner)
pools = append(pools, pool)
}
var wg sync.WaitGroup
for i, pool := range pools {
for n := 0; n < 16; n++ {
wg.Add(1)
go func(i int, pool *database.DB) {
defer wg.Done()
var identity string
if err := pool.QueryRowContext(ctx, "identity").Scan(&identity); err != nil {
t.Error(err)
} else if identity != owners[i].identity {
t.Error("crossed connectors")
}
}(i, pool)
}
}
wg.Wait()
for i, pool := range pools {
if pool.Stats().OpenConnections > cfg.MaxOpenConns {
t.Fatal("pool cap not applied")
}
if err := pool.Close(); err != nil {
t.Fatal(err)
}
if err := pool.Close(); err != nil {
t.Fatal(err)
}
if owners[i].closedConnector.Load() != 1 || owners[i].closedConnections.Load() != owners[i].connects.Load() {
t.Fatal("close did not release owned resources exactly once")
}
if pool.Health(ctx) == nil {
t.Fatal("closed pool is healthy")
}
}
}
func TestConnectorRejectsInvalidInputWithoutTakingOwnership(t *testing.T) {
var typedNil *fixtureConnector
for _, connector := range []driver.Connector{nil, typedNil} {
if db, err := database.OpenConnector(nil, connector, database.DefaultPoolConfig()); err == nil || db != nil {
t.Fatal("nil connector accepted")
}
}
for _, mutate := range []func(*database.PoolConfig){
func(c *database.PoolConfig) { c.MaxOpenConns = -1 }, func(c *database.PoolConfig) { c.MaxIdleConns = -1 },
func(c *database.PoolConfig) { c.MaxIdleConns = c.MaxOpenConns + 1 }, func(c *database.PoolConfig) { c.PingTimeout = 0 },
func(c *database.PoolConfig) { c.ConnMaxLifetime = -1 }, func(c *database.PoolConfig) { c.ConnMaxIdleTime = -1 },
} {
cfg := database.DefaultPoolConfig()
mutate(&cfg)
owner := &fixtureConnector{}
if db, err := database.OpenConnector(nil, owner, cfg); err == nil || db != nil {
t.Fatal("invalid config accepted")
}
if owner.connects.Load() != 0 || owner.closedConnector.Load() != 0 {
t.Fatal("validation consumed connector")
}
}
owner := &fixtureConnector{}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if db, err := database.OpenConnector(ctx, owner, database.DefaultPoolConfig()); !errors.Is(err, context.Canceled) || db != nil {
t.Fatalf("pre-canceled context: %v", err)
}
if owner.connects.Load() != 0 || owner.closedConnector.Load() != 0 {
t.Fatal("pre-cancellation consumed connector")
}
if err := database.DefaultConfig().Validate(); err == nil {
t.Fatal("legacy DSN requirements changed")
}
}
func TestConnectorStartupFailuresReleaseResources(t *testing.T) {
failure := errors.New("fixture unavailable")
closeFailure := errors.New("fixture close failed")
for _, phase := range []string{"connect", "ping"} {
t.Run(phase, func(t *testing.T) {
owner := &fixtureConnector{closeErr: closeFailure}
if phase == "connect" {
owner.connect = func(context.Context) error { return failure }
} else {
owner.ping = func(context.Context) error { return failure }
}
pool, err := database.OpenConnector(nil, owner, database.DefaultPoolConfig())
if pool != nil || !errors.Is(err, failure) || !errors.Is(err, closeFailure) {
t.Fatalf("lost failure or cleanup error: %v", err)
}
expectedConnections := int32(0)
if phase == "ping" {
expectedConnections = 1
}
if owner.closedConnector.Load() != 1 || owner.closedConnections.Load() != expectedConnections {
t.Fatal("startup failure leaked resources")
}
})
}
}
func TestConnectorTimeoutBoundsConnectAndPing(t *testing.T) {
for _, phase := range []string{"connect", "ping"} {
t.Run(phase, func(t *testing.T) {
owner := &fixtureConnector{}
cfg := database.DefaultPoolConfig()
cfg.PingTimeout = 20 * time.Millisecond
wait := func(ctx context.Context) error {
if _, ok := ctx.Deadline(); !ok {
return errors.New("missing deadline")
}
<-ctx.Done()
return ctx.Err()
}
if phase == "connect" {
owner.connect = wait
} else {
owner.ping = wait
}
pool, err := database.OpenConnector(context.Background(), owner, cfg)
if pool != nil || !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("%s timeout: %v", phase, err)
}
if owner.closedConnector.Load() != 1 {
t.Fatal("timeout leaked connector")
}
})
}
}
func TestConnectorHealthBoundsPoolWaitAndRecovers(t *testing.T) {
cfg := database.DefaultPoolConfig()
cfg.MaxOpenConns = 1
cfg.MaxIdleConns = 1
cfg.PingTimeout = 20 * time.Millisecond
owner := &fixtureConnector{}
pool, err := database.OpenConnector(nil, owner, cfg)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
conn, err := pool.Conn(context.Background())
if err != nil {
t.Fatal(err)
}
defer conn.Close()
if err = pool.Health(nil); !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("waiting for busy pool was not bounded: %v", err)
}
if err = conn.Close(); err != nil {
t.Fatal(err)
}
if err = pool.Health(nil); err != nil {
t.Fatal(err)
}
}
func TestConnectorIdleZeroAndLifetime(t *testing.T) {
for _, mode := range []string{"noIdle", "expired"} {
t.Run(mode, func(t *testing.T) {
cfg := database.DefaultPoolConfig()
if mode == "noIdle" {
cfg.MaxIdleConns = 0
} else {
cfg.ConnMaxLifetime = time.Nanosecond
}
owner := &fixtureConnector{}
pool, err := database.OpenConnector(nil, owner, cfg)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
if err = pool.Health(nil); err != nil {
t.Fatal(err)
}
if owner.connects.Load() < 2 || owner.closedConnections.Load() == 0 {
t.Fatal("pool did not retire connections")
}
if mode == "noIdle" && pool.Stats().Idle != 0 {
t.Fatal("zero idle not honored")
}
if mode == "expired" && pool.Stats().MaxLifetimeClosed == 0 {
t.Fatal("lifetime not honored")
}
})
}
}
func TestConnectorTransactionsKeepCommitAndRollbackSemantics(t *testing.T) {
owner := &fixtureConnector{}
pool, err := database.OpenConnector(nil, owner, database.DefaultPoolConfig())
if err != nil {
t.Fatal(err)
}
defer pool.Close()
ctx := context.Background()
if err = pool.InTx(ctx, nil, func(*sql.Tx) error { return nil }); err != nil {
t.Fatal(err)
}
failed := errors.New("business failed")
if err = pool.InTx(ctx, nil, func(*sql.Tx) error { return failed }); !errors.Is(err, failed) {
t.Fatal(err)
}
func() {
defer func() {
if recover() != failed {
t.Error("panic not preserved")
}
}()
_ = pool.InTx(ctx, nil, func(*sql.Tx) error { panic(failed) })
}()
if owner.commits.Load() != 1 || owner.rollbacks.Load() != 2 {
t.Fatal("transaction semantics changed")
}
}

View File

@@ -14,37 +14,47 @@ import (
)
type Config struct {
Driver string
DSN string
MaxOpenConns int
MaxIdleConns int
ConnMaxLifetime time.Duration
ConnMaxIdleTime time.Duration
PingTimeout time.Duration
Driver string `json:"driver"`
DSN string `json:"-"`
MaxOpenConns int `json:"maxOpenConns"`
MaxIdleConns int `json:"maxIdleConns"`
ConnMaxLifetime time.Duration `json:"connMaxLifetime"`
ConnMaxIdleTime time.Duration `json:"connMaxIdleTime"`
PingTimeout time.Duration `json:"pingTimeout"`
}
func DefaultConfig() Config {
return Config{
MaxOpenConns: 10,
MaxIdleConns: 5,
ConnMaxLifetime: 30 * time.Minute,
ConnMaxIdleTime: 5 * time.Minute,
PingTimeout: 5 * time.Second,
func DefaultConfig() Config { return DefaultPoolConfig().config() }
// PoolConfig returns driver-independent connection pool settings.
func (c Config) PoolConfig() PoolConfig {
return PoolConfig{
MaxOpenConns: c.MaxOpenConns, MaxIdleConns: c.MaxIdleConns,
ConnMaxLifetime: c.ConnMaxLifetime, ConnMaxIdleTime: c.ConnMaxIdleTime,
PingTimeout: c.PingTimeout,
}
}
// ConfigFromEnv reads GO_OMNI_DB_* style settings. DSN is required.
func ConfigFromEnv(prefix string) (Config, error) {
return ApplyEnv(DefaultConfig(), prefix)
}
// ApplyEnv overlays nonempty environment settings on file/default configuration.
// DSN is environment-only when Config is decoded from JSON or YAML.
func ApplyEnv(cfg Config, prefix string) (Config, error) {
prefix = strings.TrimSpace(prefix)
if prefix == "" {
prefix = "GO_OMNI_DB_"
}
cfg := DefaultConfig()
cfg.Driver = strings.TrimSpace(os.Getenv(prefix + "DRIVER"))
if value := strings.TrimSpace(os.Getenv(prefix + "DRIVER")); value != "" {
cfg.Driver = value
}
if cfg.Driver == "" {
cfg.Driver = "mysql"
}
cfg.DSN = strings.TrimSpace(os.Getenv(prefix + "DSN"))
if value := strings.TrimSpace(os.Getenv(prefix + "DSN")); value != "" {
cfg.DSN = value
}
if cfg.DSN == "" {
return Config{}, fmt.Errorf("%sDSN is required", prefix)
}
@@ -74,16 +84,7 @@ func (c Config) Validate() error {
if strings.TrimSpace(c.DSN) == "" {
return errors.New("database DSN is required")
}
if c.MaxOpenConns < 0 || c.MaxIdleConns < 0 {
return errors.New("database pool limits cannot be negative")
}
if c.MaxIdleConns > c.MaxOpenConns && c.MaxOpenConns > 0 {
return errors.New("max idle connections cannot exceed max open connections")
}
if c.ConnMaxLifetime < 0 || c.ConnMaxIdleTime < 0 || c.PingTimeout <= 0 {
return errors.New("database timeouts are invalid")
}
return nil
return c.PoolConfig().Validate()
}
// DB owns a connection pool and exposes the embedded database/sql methods.
@@ -103,23 +104,30 @@ func Open(ctx context.Context, cfg Config) (*DB, error) {
if err != nil {
return nil, fmt.Errorf("open database: %w", err)
}
if cfg.MaxOpenConns > 0 {
db.SetMaxOpenConns(cfg.MaxOpenConns)
}
// database/sql uses zero to disable idle connections; do not leave its
// default pool size in place when the caller explicitly chooses zero.
db.SetMaxIdleConns(cfg.MaxIdleConns)
return initializePool(ctx, db, cfg)
}
// initializePool is the shared ownership boundary for both creation paths.
// All configuration is validated before creating the sql.DB.
func initializePool(ctx context.Context, db *sql.DB, cfg Config) (*DB, error) {
db.SetMaxOpenConns(cfg.MaxOpenConns)
db.SetMaxIdleConns(cfg.MaxIdleConns) // Zero explicitly disables idle connections.
db.SetConnMaxLifetime(cfg.ConnMaxLifetime)
db.SetConnMaxIdleTime(cfg.ConnMaxIdleTime)
checkCtx, cancel := context.WithTimeout(ctx, cfg.PingTimeout)
defer cancel()
if err := db.PingContext(checkCtx); err != nil {
_ = db.Close()
return nil, fmt.Errorf("ping database: %w", err)
pingErr := fmt.Errorf("ping database: %w", err)
if closeErr := db.Close(); closeErr != nil {
return nil, errors.Join(pingErr, fmt.Errorf("close database after failed ping: %w", closeErr))
}
return nil, pingErr
}
return &DB{DB: db, config: cfg}, nil
}
// Config returns the original DSN configuration. Pools opened with a Connector
// have empty Driver/DSN fields; their connector cannot be reconstructed from Config.
func (db *DB) Config() Config { return db.config }
func (db *DB) Health(ctx context.Context) error {

View File

@@ -0,0 +1,180 @@
package database_test
import (
"context"
"crypto/tls"
"crypto/x509"
"database/sql"
"errors"
"net"
"os"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"github.com/go-sql-driver/mysql"
)
type connectorDriverLog struct{ count atomic.Int32 }
func (l *connectorDriverLog) Print(...any) { l.count.Add(1) }
func TestMySQLConnectorDriverLoggerIsolation(t *testing.T) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
stopped := make(chan struct{})
go func() {
defer close(stopped)
for {
c, err := listener.Accept()
if err != nil {
return
}
c.Close()
}
}()
defer func() { listener.Close(); <-stopped }()
logs := [2]connectorDriverLog{}
for i := range logs {
cfg := mysql.NewConfig()
cfg.Net = "tcp"
cfg.Addr = listener.Addr().String()
cfg.Timeout = time.Second
cfg.ReadTimeout = time.Second
cfg.Logger = &logs[i]
connector, err := mysql.NewConnector(cfg)
if err != nil {
t.Fatal(err)
}
before := [2]int32{logs[0].count.Load(), logs[1].count.Load()}
db, err := database.OpenConnector(context.Background(), connector, database.DefaultPoolConfig())
if err == nil {
db.Close()
t.Fatal("handshake to closed socket succeeded")
}
if logs[i].count.Load() <= before[i] || logs[1-i].count.Load() != before[1-i] {
t.Fatal("driver logger did not stay on its connector")
}
}
}
func TestMySQLConnectorTLSAndPoolIsolation(t *testing.T) {
if os.Getenv("GO_OMNI_CONNECTOR_MYSQL_TEST") != "1" {
t.Skip("use scripts/verifyConnector.py for isolated MySQL TLS verification")
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
ca, err := os.ReadFile(os.Getenv("GO_OMNI_CONNECTOR_CA_FILE"))
if err != nil {
t.Fatal(err)
}
roots := x509.NewCertPool()
if !roots.AppendCertsFromPEM(ca) {
t.Fatal("invalid test CA")
}
cfg := mysql.NewConfig()
cfg.Net = "tcp"
cfg.Addr = os.Getenv("GO_OMNI_CONNECTOR_ADDR")
cfg.User = os.Getenv("GO_OMNI_CONNECTOR_USER")
cfg.Passwd = os.Getenv("GO_OMNI_CONNECTOR_PASSWORD")
cfg.DBName = os.Getenv("GO_OMNI_CONNECTOR_DATABASE")
cfg.Timeout = time.Second
cfg.ReadTimeout = 2 * time.Second
cfg.WriteTimeout = 2 * time.Second
cfg.TLS = &tls.Config{MinVersion: tls.VersionTLS12, ServerName: "localhost", RootCAs: roots}
pool := database.DefaultPoolConfig()
pool.MaxOpenConns = 2
pool.MaxIdleConns = 2
pool.PingTimeout = 2 * time.Second
open := func(cfg *mysql.Config) (*database.DB, error) {
connector, err := mysql.NewConnector(cfg)
if err != nil {
return nil, err
}
return database.OpenConnector(ctx, connector, pool)
}
cfg.Params = map[string]string{"time_zone": "'+00:00'"}
first, err := open(cfg)
if err != nil {
t.Fatal(err)
}
defer first.Close()
other := cfg.Clone()
other.Params["time_zone"] = "'+08:00'"
second, err := open(other)
if err != nil {
t.Fatal(err)
}
defer second.Close()
for i, db := range []*database.DB{first, second} {
var name, cipher, zone string
if err := db.QueryRowContext(ctx, "SHOW SESSION STATUS LIKE 'Ssl_cipher'").Scan(&name, &cipher); err != nil {
t.Fatal(err)
}
if cipher == "" {
t.Fatal("connection is not encrypted")
}
if err := db.QueryRowContext(ctx, "SELECT @@session.time_zone").Scan(&zone); err != nil {
t.Fatal(err)
}
if zone != []string{"+00:00", "+08:00"}[i] {
t.Fatalf("session config leaked: %s", zone)
}
if db.Stats().MaxOpenConnections != 2 {
t.Fatal("pool configuration ignored")
}
}
for _, bad := range []string{"untrustedCA", "wrongHostname"} {
t.Run(bad, func(t *testing.T) {
invalid := cfg.Clone()
if bad == "untrustedCA" {
invalid.TLS.RootCAs = x509.NewCertPool()
} else {
invalid.TLS.ServerName = "wrong.invalid"
}
db, err := open(invalid)
if err == nil {
db.Close()
t.Fatal("invalid server certificate accepted")
}
var verification *tls.CertificateVerificationError
if !errors.As(err, &verification) {
t.Fatalf("not a TLS verification failure: %v", err)
}
if err := first.Health(ctx); err != nil {
t.Fatalf("failed connector affected existing pool: %v", err)
}
})
}
if _, err := first.ExecContext(ctx, "CREATE TABLE connector_check (id INT PRIMARY KEY)"); err != nil {
t.Fatal(err)
}
if err := first.InTx(ctx, nil, func(tx *sql.Tx) error {
_, err := tx.ExecContext(ctx, "INSERT INTO connector_check VALUES (1)")
return err
}); err != nil {
t.Fatal(err)
}
rollbackErr := errors.New("rollback requested")
if err := first.InTx(ctx, nil, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx, "INSERT INTO connector_check VALUES (2)"); err != nil {
return err
}
return rollbackErr
}); !errors.Is(err, rollbackErr) {
t.Fatal(err)
}
var count int
if err := second.QueryRowContext(ctx, "SELECT COUNT(*) FROM connector_check").Scan(&count); err != nil || count != 1 {
t.Fatalf("transaction count=%d err=%v", count, err)
}
if err := first.Close(); err != nil {
t.Fatal(err)
}
if err := second.Health(ctx); err != nil {
t.Fatal("closing first pool affected second:", err)
}
}

View File

@@ -0,0 +1,29 @@
// Package ids demonstrates an application-owned scalar. The framework does not
// impose this ID policy on any other type or project.
package ids
import (
"encoding/json"
"strconv"
)
type ID int64
func (id ID) MarshalJSON() ([]byte, error) { return json.Marshal(strconv.FormatInt(int64(id), 10)) }
func (id *ID) UnmarshalJSON(raw []byte) error {
value := string(raw)
if len(raw) > 0 && raw[0] == '"' {
if err := json.Unmarshal(raw, &value); err != nil {
return err
}
}
return id.UnmarshalText([]byte(value))
}
func (id *ID) UnmarshalText(raw []byte) error {
value, err := strconv.ParseInt(string(raw), 10, 64)
if err != nil {
return err
}
*id = ID(value)
return nil
}

View File

@@ -0,0 +1,75 @@
// This example only uses local HTTP and bounded streaming; it has no database.
package main
import (
"io"
"log"
"net/http"
"strings"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/examples/compatibility/ids"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
type echoRequest struct {
ID ids.ID `json:"id"`
Data any `json:"data"`
}
func main() {
app := omni.New(omni.WithLogger(true))
defer app.Close()
app.Use(httpx.WrapMiddleware(func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("X-Example", "compatibility")
next.ServeHTTP(w, r)
})
}))
app.GET("/legacy", httpx.WrapHandler(http.HandlerFunc(http.NotFound)))
binder := contract.MustNew[echoRequest](contract.ScalarOf[ids.ID]("string|int64", "string", true))
app.POST("/echo", func(c *httpx.Context) error {
var req echoRequest
if err := binder.Bind(c.Request(), nil, &req); err != nil {
return err
}
return c.OK(req)
})
app.HandleWithBodyLimit("POST", "/upload", 10<<20, nil, func(c *httpx.Context) error {
reader, err := c.Request().MultipartReader()
if err != nil {
return omni.BadRequest("INVALID_MULTIPART", "multipart body required")
}
var total int64
for {
part, err := reader.NextPart()
if err == io.EOF {
break
}
if err != nil {
return err
}
// Replace io.Discard with a streaming storage writer. Do not trust file names
// as filesystem paths. io.Copy never buffers the complete uploaded file.
n, copyErr := io.Copy(io.Discard, part)
closeErr := part.Close()
if copyErr != nil {
return copyErr
}
if closeErr != nil {
return closeErr
}
total += n
}
return c.OK(map[string]int64{"bytes": total})
})
app.GET("/download", httpx.WrapHandler(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/octet-stream")
w.Header().Set("Content-Disposition", `attachment; filename="example.txt"`)
_, _ = io.Copy(w, strings.NewReader("A standard Handler keeps its original download response.\n"))
})))
if err := app.Run("127.0.0.1:8080"); err != nil {
log.Fatal(err)
}
}

View File

@@ -0,0 +1,15 @@
# 接口定义示例
`user.api` 是入口,导入 `shared.api`。包含 JSON/path/query/header 混合请求、分页与重复 query 参数、嵌套对象、optional 指针、user 与 admin/user 分组以及 Auth/Audit 命名中间件。
在 go-omni 根目录执行:
```sh
go run ./cmd/go-omni api validate --api examples/contracts/user.api
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
--dir ../userService --module example.com/userService --framework-dir .
```
项目中补充 `internal/service` 的业务实现和 `internal/middleware` 的 Auth/Audit hook,再启动服务。初始占位实现返回 501。
需要立即能返回成功结果的最小项目,用 `go-omni api new` 创建问候示例。完整语法、目录、更新策略见 [API.md](../../API.md)。

View File

@@ -0,0 +1,64 @@
syntax = "v1"
scalar BusinessID (
goImport: "git.yuntuo.hk/k2safe/go-omni/examples/compatibility/ids"
goType: ID
jsonInput: string | int64
jsonOutput: string
text: true
)
type EchoReq {
ID BusinessID `json:"id"`
Data any `json:"data"`
Legacy interface{} `json:"legacy,optional"`
}
type EchoResp {
ID BusinessID `json:"id"`
Data any `json:"data"`
}
type ItemReq {
ID BusinessID `path:"id"`
}
type CustomReq {
Value string `json:"value"`
}
type CustomResp {
Value string `json:"value"`
}
@server (group: demo)
service compatibility-api {
@body (maxBytes: 2097152)
@handler echo
post /echo (EchoReq) returns (EchoResp)
@handler item
get /items/:id (ItemReq) returns (EchoResp)
@binding custom
@body (maxBytes: 512)
@handler custom
post /custom (CustomReq) returns (CustomResp)
@handlerMode manual
@body (mode: multipart
maxBytes: 65536)
@handler upload
post /upload
@handlerMode manual
@body (mode: raw
maxBytes: 128)
@produces "text/plain"
@handler callback
post /callback
}
@server (group: files)
service compatibility-api {
@handlerMode manual
@produces "application/octet-stream"
@handler download
get /download
}

View File

@@ -0,0 +1,58 @@
syntax = "v1"
info (
title: "Configurable response example"
version: "1.0.0"
)
// Define the public body once. No field names are required by the framework.
response CommonResponse (
body: {
"code": "${code}",
"msg": "${message}",
"data": "${data}"
}
codeType: int32
successCode: 200
errorCode: 9999
successMessage: "操作成功"
dataMode: always
codes: {"ITEM_NOT_FOUND":404001}
messages: {"ITEM_NOT_FOUND":"记录不存在"}
)
response PartnerResponse (
body: {"accepted":"${ok}","payload":"${data}","meta":{"trace":"${requestId}"}}
errorBody: {"accepted":"${ok}","error":{"reason":"${message}"}}
)
error ItemNotFound 404 "ITEM_NOT_FOUND" "item not found"
type ItemReq {
ItemID int64 `path:"itemId" binding:"min=1"`
}
type ItemResp {
ItemID int64 `json:"itemId"`
Name string `json:"name"`
}
@server (
group: items
prefix: /api
response: CommonResponse
)
service response-api {
@errors ItemNotFound
@handler getItem
get /items/:itemId (ItemReq) returns (ItemResp)
// Success is the ItemResp object itself; errors still use CommonResponse.
@response raw
@handler rawItem
get /rawItems/:itemId (ItemReq) returns (ItemResp)
@response PartnerResponse
@handler partnerItem
get /partnerItems/:itemId (ItemReq) returns (ItemResp)
}

View File

@@ -0,0 +1,11 @@
syntax = "v1"
type Address {
City string `json:"city" binding:"required,max=50"`
}
type UserResp {
UserID int64 `json:"userId"`
Name string `json:"name"`
Active bool `json:"active"`
}

View File

@@ -0,0 +1,58 @@
syntax = "v1"
info (
title: "User API"
version: "1.0.0"
)
import "shared.api"
type (
UpdateUserReq {
UserID int64 `path:"userId,range=[1:1000000]"`
Notify bool `form:"notify,default=false"`
Client string `header:"X-Client,optional"`
Name string `json:"name" binding:"required,max=50"`
Active bool `json:"active"`
Age *int `json:"age,optional,range=[0:150]"`
Address *Address `json:"address,optional"`
}
ListUsersReq {
Page int `form:"page,default=1,range=[1:1000]"`
PageSize int `form:"pageSize,default=20,range=[1:100]"`
Tags []string `form:"tag,optional" binding:"max=10"`
}
ListUsersResp {
Items []UserResp `json:"items"`
Page int `json:"page"`
Total int64 `json:"total"`
}
DeleteUserReq {
UserID int64 `path:"userId,range=[1:1000000]"`
}
)
@server (
group: user
prefix: /api/v1
)
service user-api {
@doc "Update a user"
@handler updateUser
put /users/:userId (UpdateUserReq) returns (UserResp)
@doc "List users"
@handler listUsers
get /users (ListUsersReq) returns (ListUsersResp)
}
@server (
group: admin/user
prefix: /admin/v1
middleware: Auth,Audit
)
service user-api {
@doc "Delete a user; business-owned authentication and auditing required"
@handler deleteUser
delete /users/:userId (DeleteUserReq)
}

6
examples/userService/.gitignore vendored Normal file
View File

@@ -0,0 +1,6 @@
/server
/var/
.env
.env.*
!.env.example
/.goOmni/api.lock

View File

@@ -0,0 +1,21 @@
{
"profile": {
"database": "mysql",
"storage": "local",
"logOutput": "file",
"logFormat": "json"
},
"version": 1,
"module": "example.com/userService",
"entry": "api/user.api",
"generated": {
"docs/openapi.json": "05bce8951699350cc1ebca5a626c39349395dd09e395b41560c129e530583204",
"internal/apierrors/errorsGen.go": "20e94bbc90ae9f4330de4c25b9a550026c4c352502719a643ede3e8912a6566e",
"internal/handler/openapiGen.go": "e4130f5c7494f7db4830bfef203fa8154a9cd087c95954f64c29d74693ead433",
"internal/handler/responseGen.go": "d742efc2a3d1d168fae7849e905297bd440c3d6b3802b6150f494c07e6097f4f",
"internal/handler/routesGen.go": "4e0c6879cea40e9c0155439fa1fce2cd19b5f83ea6e304523c41f8dc9c3a28e0",
"internal/handler/user/handlersGen.go": "1808afc8e4e8ca9785a8552eff64fad39bd5fe4e6e92971f35afc9006e23d76e",
"internal/middleware/registryGen.go": "a6122a5453dedc6a4415860108814c50e155128cfe5412b39dd33ce2d01b4b00",
"internal/types/typesGen.go": "3ab2ce9ffe467c3ea53ac3d1fec283ecebc87cef57fd592d08dca991fd964876"
}
}

View File

@@ -0,0 +1,82 @@
# API 驱动的用户管理示例
这个独立 Go module 从 `api/user.api` 生成,随后在开发者拥有的文件中实现业务。
使用 MySQL 持久化、事务审计、环境变量 Token 鉴权、文件日志和 local 存储;业务层只接收标准 context 和基础模块接口。
## 启动
先准备空的业务数据库,再设置环境变量(替换占位值,密钥不写进配置或 Git):
```sh
export GO_OMNI_DB_DSN='user:password@tcp(127.0.0.1:3306)/user_service?parseTime=true'
export USER_SERVICE_API_TOKEN='<至少32字节的随机值>'
export GO_OMNI_STORAGE_SIGNING_KEY='<另一个至少32字节的随机值>'
export GO_OMNI_STORAGE_PUBLIC_URL='http://127.0.0.1:8080/storage/object'
go mod tidy
# 在本目录运行与框架版本一致的 CLI;迁移必须显式执行。
go-omni migrate --dir migrations --apply
go run ./cmd/server
```
源码示例的 go.mod 使用 `replace ... => ../..`。生成代码使用 `v0.8.0-preview` 响应策略入口。
复制到独立仓库时删除 replace,保留同版本固定依赖,再执行 go mod tidy;CLI 也固定到同一版本。
远程版本验证脚本会检查没有本地替换。
`GO_OMNI_TRANSPORT=native` 可切换为 native,业务代码无需调整。
配置是默认值 → etc/config.yaml → 环境变量。日志默认写入 var/log/server.log 并轮转,
`GO_OMNI_LOG_OUTPUT=stderr` 可以切换为标准错误输出。MySQL 默认最多 10 条连接。
应用检查 users 和 user_audit 表,缺失时拒绝启动,不自动改表。
## 接口
除健康检查、OpenAPI 和持有效签名的对象下载外,业务接口需要 `Authorization: Bearer <Token>`。
这个 Token 代表示例的管理员;可选设置不同的 `USER_SERVICE_READ_TOKEN`,允许 GET/HEAD,写请求返回 403。可通过同一个命名中间件槽位替换为 JWT/OIDC 和业务授权策略。
框架不会推断租户模型或自动给接口放行。
| 接口 | 行为 |
| --- | --- |
| POST /api/v1/users | 创建,返回 201;邮箱规范化并唯一 |
| GET /api/v1/users/:id | 查询,缺失返回 404 |
| GET /api/v1/users?page=1&pageSize=20 | 分页,pageSize 最大 100,空结果返回 [] |
| PUT /api/v1/users/:id | 完整更新;邮箱冲突返回 409 |
| DELETE /api/v1/users/:id | 删除,返回 204 |
| POST /api/v1/users/:id/export | 私有文本导出,返回 5 分钟下载链接 |
| GET /health/ready | 数据库不可用时返回 503,恢复后重新就绪 |
| GET /openapi.json | 与 docs/openapi.json 一致 |
参数不合法返回 400,认证失败 401,只读身份写入 403,邮箱重复 409,内部故障只返回通用 500。
OpenAPI 的数据类型、鉴权和业务错误均来自契约;401/403/404/409 带具体命名示例。业务使用生成的 `apierrors.UserNotFound()`、`apierrors.EmailTaken()` 等函数。
创建、更新、删除都会在**同一个 MySQL 事务**写入 user_audit;审计写入失败时业务变更一起回滚。
事务提交后才写成功业务日志,requestId 同时出现在 HTTP 响应、业务日志和审计记录。
数据库异常日志不记录 SQL、DSN 或用户输入。
文件导出演示 `deps.Uploads` 和 `deps.Downloads`。对象使用随机 key,签名 URL 不进入日志。
签名过期只限制下载,不删除文件;实际项目应给 exports/ 配置保留期限或清理任务。
只签发下载链接,不对外开放任意对象上传或签名接口。
## 开发与验证
```sh
go-omni api inspect
go-omni api generate --dry-run
go-omni api generate
go-omni api generate --check
go test ./...
```
Gen.go 和 docs/openapi.json 由生成器维护;Service、Repository、迁移、鉴权、配置和 main 保留。
修改 .api 的类型/方法签名后,需要同步修改自己的业务实现并运行测试。
在框架仓库运行:
```sh
python3 scripts/verifyAPIWorkflow.py --report performance/responsePoliciesWorkflowLocal.json
# 使用本版远程 CLI 和模块验收,无本地替换。
python3 scripts/verifyAPIWorkflow.py --version v0.8.0-preview \
--report performance/responsePoliciesWorkflowRemote.json
```
验收创建只绑定 localhost 的临时 MySQL 容器(1 CPU / 768 MiB),使用随机密钥,结束时删除。
覆盖 nethttp/native 的 CRUD、校验、鉴权、事务错误/异常/取消回滚、数据库停机恢复、
重启持久化、签名文件下载、文件日志与优雅退出。不会连接业务服务器。

View File

@@ -0,0 +1,86 @@
syntax = "v1"
info (
title: "User service reference"
version: "1.0.0"
)
// Stable public errors are shared by Go code and the API document.
error (
Unauthenticated 401 "UNAUTHENTICATED" "authentication required"
Forbidden 403 "FORBIDDEN" "permission denied"
UserNotFound 404 "USER_NOT_FOUND" "user not found"
EmailTaken 409 "EMAIL_TAKEN" "email already in use"
InvalidUser 400 "INVALID_USER" "invalid user"
InvalidPage 400 "INVALID_PAGE" "invalid pagination"
)
auth AdminAuth (
type: bearer
middleware: Auth
description: "Use an administrator token for writes, or a read-only token for GET requests"
)
type (
UserResp {
ID int64 `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
CreatedAt string `json:"createdAt"`
UpdatedAt string `json:"updatedAt"`
}
CreateUserReq {
Name string `json:"name" binding:"required,max=100"`
Email string `json:"email" binding:"required,email,max=254"`
}
UserIDReq {
ID int64 `path:"id,range=[1:9007199254740991]"`
}
UpdateUserReq {
ID int64 `path:"id,range=[1:9007199254740991]"`
Name string `json:"name" binding:"required,max=100"`
Email string `json:"email" binding:"required,email,max=254"`
}
ListUsersReq {
Page int `form:"page,default=1,range=[1:1000000]"`
PageSize int `form:"pageSize,default=20,range=[1:100]"`
}
UserPageResp {
Items []UserResp `json:"items"`
Total int `json:"total"`
Page int `json:"page"`
PageSize int `json:"pageSize"`
}
ExportResp {
DownloadURL string `json:"downloadUrl"`
}
)
@server (
group: user
prefix: /api/v1
auth: AdminAuth
errors: Unauthenticated
)
service user-api {
@doc (
summary: "Create a user"
description: "Normalizes the email, then commits user data and audit in one transaction"
tags: users
)
@errors Forbidden, InvalidUser, EmailTaken
@handler createUser
@status 201
post /users (CreateUserReq) returns (UserResp)
@errors UserNotFound
@handler getUser
get /users/:id (UserIDReq) returns (UserResp)
@errors InvalidPage
@handler listUsers
get /users (ListUsersReq) returns (UserPageResp)
@errors Forbidden, InvalidUser, UserNotFound, EmailTaken
@handler updateUser
put /users/:id (UpdateUserReq) returns (UserResp)
@errors Forbidden, UserNotFound
@handler deleteUser
delete /users/:id (UserIDReq)
@doc "Export one user as a private text object with a short-lived download link"
@errors Forbidden, UserNotFound
@handler exportUser
post /users/:id/export (UserIDReq) returns (ExportResp)
}

View File

@@ -0,0 +1,58 @@
package main
import (
"context"
"errors"
"example.com/userService/internal/handler"
"example.com/userService/internal/servicecontext"
"example.com/userService/internal/settings"
"flag"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/logx"
"log"
"os"
"os/signal"
"syscall"
)
func main() {
path := flag.String("config", "etc/config.yaml", "configuration file; empty uses defaults and environment")
flag.Parse()
if err := run(*path); err != nil {
log.Fatal(err)
}
}
func run(path string) (runErr error) {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
cfg, err := settings.Load(path)
if err != nil {
return err
}
logger, err := logx.New(cfg.Server.Log)
if err != nil {
return err
}
defer func() { runErr = errors.Join(runErr, logger.Close()) }()
logx.SetDefault(logger)
deps, err := servicecontext.New(ctx, logger, cfg)
if err != nil {
return err
}
defer func() { runErr = errors.Join(runErr, deps.Close()) }()
app := omni.New(omni.WithConfig(cfg.Server))
app.OnShutdown(logger.Close)
app.OnShutdown(deps.Close)
for _, method := range []string{"GET", "HEAD", "PUT"} {
app.Handle(method, "/storage/object", func(c *httpx.Context) error { deps.FileHTTP.ServeHTTP(c.Raw(), c.Request()); return nil })
}
if err = handler.RegisterRoutes(app, deps); err != nil {
return err
}
app.GET("/openapi.json", handler.OpenAPI)
app.GET("/health/live", deps.Health.LivenessHandler())
app.GET("/health/ready", deps.Health.ReadinessHandler())
logger.InfoContext(ctx, "service.starting", "address", cfg.Server.Address)
return app.RunContext(ctx, cfg.Server.Address)
}

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,36 @@
# Non-secret settings only. Environment variables override these values.
server:
address: '127.0.0.1:8080'
enableLogger: true
slowRequestThreshold: 500ms
shutdownTimeout: 10s
log:
service: "example.com/userService"
level: info
format: json
file: './var/log/server.log'
rotation:
enabled: true
maxSizeMb: 100
maxBackups: 10
maxAgeDays: 7
compress: true
redactKeys: []
storage:
provider: local
maxBytes: 33554432
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
linkTtl: 5m
requestTimeout: 30s
root: './var/files'
publicUrl: 'http://127.0.0.1:8080/storage/object'
database:
driver: mysql
maxOpenConns: 10
maxIdleConns: 5
connMaxLifetime: 30m
connMaxIdleTime: 5m
pingTimeout: 5s
# GO_OMNI_DB_DSN supplies the secret connection string.

View File

@@ -0,0 +1,26 @@
module example.com/userService
go 1.22.0
require (
git.yuntuo.hk/k2safe/go-omni v0.8.0-preview
github.com/go-sql-driver/mysql v1.9.3
)
require (
filippo.io/edwards25519 v1.1.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.20.0 // indirect
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
golang.org/x/crypto v0.31.0 // indirect
golang.org/x/net v0.33.0 // indirect
golang.org/x/sys v0.28.0 // indirect
golang.org/x/text v0.21.0 // indirect
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
replace git.yuntuo.hk/k2safe/go-omni => ../..

View File

@@ -0,0 +1,38 @@
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

View File

@@ -0,0 +1,42 @@
// Code generated by go-omni api. DO NOT EDIT.
package apierrors
import omnierrors "git.yuntuo.hk/k2safe/go-omni/errors"
const CodeUnauthenticated = "UNAUTHENTICATED"
// Unauthenticated returns a fresh public error declared in the API contract.
func Unauthenticated() *omnierrors.APIError {
return omnierrors.New(401, CodeUnauthenticated, "authentication required")
}
const CodeForbidden = "FORBIDDEN"
// Forbidden returns a fresh public error declared in the API contract.
func Forbidden() *omnierrors.APIError { return omnierrors.New(403, CodeForbidden, "permission denied") }
const CodeUserNotFound = "USER_NOT_FOUND"
// UserNotFound returns a fresh public error declared in the API contract.
func UserNotFound() *omnierrors.APIError {
return omnierrors.New(404, CodeUserNotFound, "user not found")
}
const CodeEmailTaken = "EMAIL_TAKEN"
// EmailTaken returns a fresh public error declared in the API contract.
func EmailTaken() *omnierrors.APIError {
return omnierrors.New(409, CodeEmailTaken, "email already in use")
}
const CodeInvalidUser = "INVALID_USER"
// InvalidUser returns a fresh public error declared in the API contract.
func InvalidUser() *omnierrors.APIError { return omnierrors.New(400, CodeInvalidUser, "invalid user") }
const CodeInvalidPage = "INVALID_PAGE"
// InvalidPage returns a fresh public error declared in the API contract.
func InvalidPage() *omnierrors.APIError {
return omnierrors.New(400, CodeInvalidPage, "invalid pagination")
}

File diff suppressed because one or more lines are too long

View File

@@ -0,0 +1,30 @@
// Code generated by go-omni api. DO NOT EDIT.
package handler
import (
"encoding/json"
"git.yuntuo.hk/k2safe/go-omni/response"
"strings"
)
var responseTemplates = map[string]response.Encoder{"": response.Default()}
func newResponseTemplate(raw string) response.Encoder {
var cfg response.Config
decoder := json.NewDecoder(strings.NewReader(raw))
decoder.UseNumber()
if err := decoder.Decode(&cfg); err != nil {
panic(err)
}
return response.MustNew(cfg)
}
func responseEncoder(name string, raw bool) response.Encoder {
encoder := responseTemplates[name]
if encoder == nil {
panic("unknown response template")
}
if raw {
encoder = response.RawJSON(encoder)
}
return encoder
}

View File

@@ -0,0 +1,28 @@
// Code generated by go-omni api. DO NOT EDIT.
package handler
import (
h0 "example.com/userService/internal/handler/user"
s0 "example.com/userService/internal/service/user"
"example.com/userService/internal/servicecontext"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/openapi"
)
func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {
if router == nil || deps == nil {
return fmt.Errorf("router and dependencies required")
}
if deps.Middlewares["Auth"] == nil {
return fmt.Errorf("missing middleware Auth")
}
h0 := h0.New(s0.New(deps), deps)
httpx.HandleWithResponse(router, "POST", "/api/v1/users", responseEncoder("", false), h0.CreateUser, deps.Middlewares["Auth"])
httpx.HandleWithResponse(router, "GET", "/api/v1/users/:id", responseEncoder("", false), h0.GetUser, deps.Middlewares["Auth"])
httpx.HandleWithResponse(router, "GET", "/api/v1/users", responseEncoder("", false), h0.ListUsers, deps.Middlewares["Auth"])
httpx.HandleWithResponse(router, "PUT", "/api/v1/users/:id", responseEncoder("", false), h0.UpdateUser, deps.Middlewares["Auth"])
httpx.HandleWithResponse(router, "DELETE", "/api/v1/users/:id", responseEncoder("", false), h0.DeleteUser, deps.Middlewares["Auth"])
httpx.HandleWithResponse(router, "POST", "/api/v1/users/:id/export", responseEncoder("", false), h0.ExportUser, deps.Middlewares["Auth"])
return nil
}

View File

@@ -0,0 +1,130 @@
// Code generated by go-omni api. DO NOT EDIT.
package user
import (
"context"
"example.com/userService/internal/servicecontext"
"example.com/userService/internal/types"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/httpx"
)
type Service interface {
CreateUser(ctx context.Context, req *types.CreateUserReq) (*types.UserResp, error)
GetUser(ctx context.Context, req *types.UserIDReq) (*types.UserResp, error)
ListUsers(ctx context.Context, req *types.ListUsersReq) (*types.UserPageResp, error)
UpdateUser(ctx context.Context, req *types.UpdateUserReq) (*types.UserResp, error)
DeleteUser(ctx context.Context, req *types.UserIDReq) error
ExportUser(ctx context.Context, req *types.UserIDReq) (*types.ExportResp, error)
}
type Handler struct {
service Service
deps *servicecontext.Context
}
func New(s Service, deps ...*servicecontext.Context) *Handler {
h := &Handler{service: s}
if len(deps) > 0 {
h.deps = deps[0]
}
return h
}
var createUserRequest = contract.MustNew[types.CreateUserReq]()
func (h *Handler) CreateUser(c *httpx.Context) error {
var req types.CreateUserReq
if err := createUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
return err
}
resp, err := h.service.CreateUser(c.Context(), &req)
if err != nil {
return err
}
if resp == nil {
return fmt.Errorf("CreateUser returned a nil response")
}
return c.Created(resp)
}
var getUserRequest = contract.MustNew[types.UserIDReq]()
func (h *Handler) GetUser(c *httpx.Context) error {
var req types.UserIDReq
if err := getUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
return err
}
resp, err := h.service.GetUser(c.Context(), &req)
if err != nil {
return err
}
if resp == nil {
return fmt.Errorf("GetUser returned a nil response")
}
return c.OK(resp)
}
var listUsersRequest = contract.MustNew[types.ListUsersReq]()
func (h *Handler) ListUsers(c *httpx.Context) error {
var req types.ListUsersReq
if err := listUsersRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
return err
}
resp, err := h.service.ListUsers(c.Context(), &req)
if err != nil {
return err
}
if resp == nil {
return fmt.Errorf("ListUsers returned a nil response")
}
return c.OK(resp)
}
var updateUserRequest = contract.MustNew[types.UpdateUserReq]()
func (h *Handler) UpdateUser(c *httpx.Context) error {
var req types.UpdateUserReq
if err := updateUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
return err
}
resp, err := h.service.UpdateUser(c.Context(), &req)
if err != nil {
return err
}
if resp == nil {
return fmt.Errorf("UpdateUser returned a nil response")
}
return c.OK(resp)
}
var deleteUserRequest = contract.MustNew[types.UserIDReq]()
func (h *Handler) DeleteUser(c *httpx.Context) error {
var req types.UserIDReq
if err := deleteUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
return err
}
if err := h.service.DeleteUser(c.Context(), &req); err != nil {
return err
}
return c.NoContent()
}
var exportUserRequest = contract.MustNew[types.UserIDReq]()
func (h *Handler) ExportUser(c *httpx.Context) error {
var req types.UserIDReq
if err := exportUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
return err
}
resp, err := h.service.ExportUser(c.Context(), &req)
if err != nil {
return err
}
if resp == nil {
return fmt.Errorf("ExportUser returned a nil response")
}
return c.OK(resp)
}

View File

@@ -0,0 +1,50 @@
package middleware
import (
"crypto/sha256"
"crypto/subtle"
"errors"
"example.com/userService/internal/apierrors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"os"
"strings"
)
// Auth is the fail-closed registry default. Startup replaces it with configured
// authentication. This example uses an administrator key and an optional read key;
// production applications can inject JWT/OIDC through the same middleware slot.
func Auth(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
return apierrors.Unauthenticated()
}
}
func NewAuthFromEnv() (httpx.Middleware, error) {
token := os.Getenv("USER_SERVICE_API_TOKEN")
if len(token) < 32 || strings.ContainsAny(token, " \t\r\n") {
return nil, errors.New("USER_SERVICE_API_TOKEN requires at least 32 bytes without whitespace")
}
readToken := os.Getenv("USER_SERVICE_READ_TOKEN")
if readToken != "" && (len(readToken) < 32 || strings.ContainsAny(readToken, " \t\r\n") || readToken == token) {
return nil, errors.New("USER_SERVICE_READ_TOKEN must be a distinct token of at least 32 bytes without whitespace")
}
expectedRead := sha256.Sum256([]byte(readToken))
expected := sha256.Sum256([]byte(token))
return func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
parts := strings.Fields(c.Header("Authorization"))
if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") {
return apierrors.Unauthenticated()
}
actual := sha256.Sum256([]byte(parts[1]))
admin := subtle.ConstantTimeCompare(expected[:], actual[:]) == 1
read := subtle.ConstantTimeCompare(expectedRead[:], actual[:]) == 1 && readToken != ""
if !admin && !read {
return apierrors.Unauthenticated()
}
if !admin && c.Request().Method != "GET" && c.Request().Method != "HEAD" {
return apierrors.Forbidden()
}
return next(c)
}
}, nil
}

Some files were not shown because too many files have changed in this diff Show More