Compare commits

...

14 Commits

Author SHA1 Message Date
0d2c6040e2 feat: release v0.28.1 mixed API and generic project workspaces 2026-09-30 00:20:17 +08:00
1052f156bd test: record v0.28 remote generic and managed project acceptance 2026-09-30 00:01:38 +08:00
0b6c0043b6 feat: release v0.28 generic Go project scaffolding and lifecycle entry 2026-09-29 23:58:20 +08:00
f8e0dcc5a6 test: record v0.27 remote HTTP safety and generated consumer acceptance 2026-09-29 23:27:32 +08:00
3b00bb55d1 feat: release v0.27 unified HTTP limits and safe request shutdown 2026-09-29 23:24:19 +08:00
859f1851b1 test: record v0.26 remote i18n and generated consumer acceptance 2026-09-29 13:34:28 +08:00
d1a747c36f feat: release v0.26 portable i18n and presentation adapters 2026-09-29 13:31:42 +08:00
6a95cad184 test: record v0.25 remote transaction and MySQL acceptance 2026-09-26 20:32:23 +08:00
7a177744ab feat: expose transaction failure stages and conservative outcomes 2026-09-26 20:29:45 +08:00
8eddef9cc8 test: record v0.24 remote CORS and generated project acceptance 2026-09-26 19:55:41 +08:00
800974cfff feat: add configurable transport-independent CORS policy 2026-09-26 19:53:28 +08:00
9fade3bedb test: record v0.23 remote consumer and performance acceptance 2026-09-26 18:06:40 +08:00
3bbc6f60d7 feat: add precise diagnostics and optional client copy logging controls 2026-09-26 18:04:33 +08:00
69585dab61 test: record remote explicit configuration release acceptance 2026-09-26 16:16:37 +08:00
214 changed files with 15856 additions and 361 deletions

View File

@@ -1,5 +1,48 @@
# Changelog
## v0.27.0-preview(2026-09-29)
- HTTP 配置新增 defaultMaxBodyBytes、maxBodyBytes、shutdownCancelTimeout;三种内置服务统一默认/路由/硬请求体限额,配置冲突在监听前报告完整路由与限额。native/HTTP2 不再以隐含 8 MiB 阻挡已放宽的 App 上传路由。
- App 在所有服务入口跟踪完整请求处理链;关闭连接后有界等待处理器实际退出。超时仍有请求则返回 ErrResourcesRetained,跳过 App 清理;Runtime.Finish 继续停止后台工作但保留依赖,生成入口保留日志。
- native 提前拒绝上传后有界半关闭连接,减少 TCP reset 吞掉错误响应;业务 HTTP 200 策略不再触发对被拒绝请求体的继续读取。
- HTTP/2 Shutdown 同时等待连接和独立处理器,修复客户端断开后误判完成。增加真实 TLS/流式上传、取消收尾、拒绝晚到请求、资源保留和生成消费者回归,以及 httpSafety 示例。
- 迁移:nethttp 手写普通路由新增默认 8 MiB 请求体上限,大请求显式覆盖;生成契约默认 1 MiB 保持。HTTP 停机多一个默认 5s 取消收尾窗口;外部宿主须将运行错误传给 Runtime.Finish,并遵守资源保留信号。
## v0.26.0-preview(2026-09-29)
- 新增独立 i18n:不可变命名空间词典、源/默认/启用语言、BCP 47 匹配、显式回退、参数/条件/基数复数和有界资源加载;不依赖 HTTP、业务或数据库。
- Go/浏览器共用固定 CLDR 46.1 复数规则;JSON 资源版本、类型导出和按命名空间选择的语言包独立于业务错误格式。
- 新增可选 httpi18n/errori18n 适配;错误 translation 元数据、PublicMessage 展示入口,保留原业务码/状态/失败语义/原始原因和首次幂等响应。
- 新增 i18n check/export/types/extract;静态与动态键诊断、未知字段/重复定义/参数校验、显式覆盖;输入只读,不执行任意项目 Go。
- 新增无前端框架依赖的浏览器模块、按需加载/切换竞态/失败保留、CLI/HTTP/表单示例及通用验收。旧项目默认行为不变。
- 新 translation 生成代码要求 v0.26.0-preview 或具备能力的本地源码,未使用该元数据的模板最低版本保持 v0.22。
- 修复 JSON 对象/数组结束定界符读取失败时丢失原始错误的问题,保留请求体限额的 HTTP 413;共享 Go 目录测试工程改用声明支持的 Go 1.22。
## v0.25.0-preview(2026-09-26)
- database.InTx 新增 TxError、TxStage、TxOutcome,区分开启/回调/提交失败以及未执行回调、已确认回滚、结果不确定;原签名保留。
- 回调失败后只有显式回滚返回成功才标记 rolledBack;提交报错一律保守返回 unknown,不根据断网/超时或 ErrTxDone 推断回滚。不自动重试事务,不处理文件清理等业务决策。
- Err/RollbackErr 分开保留,errors.As/Is 可遍历两条原因链;业务错误目录和公开响应行为保持。nil context 接受,nil 回调在开启前报错,panic/Goexit 保持传播并尝试回滚。
- 新增阶段/取消竞争/确认丢失/并发/公开响应回归、真实 MySQL 驱动边界故障验收、性能比较和 examples/transactions;同步 README 与 database 指南。
- 错误现在有类型包装;直接错误相等、直接类型断言和错误文本依赖需改为 errors.Is/As。标准库异步回滚不公开的底层原因无法还原,结果保持 unknown;不改变 GORM 自己的 Transaction。
## v0.24.0-preview(2026-09-26)
- 新增可选 cors 包和 server.cors 配置;生成项目直接加载,手写项目使用 omni.WithCORS,独立 HTTP 服务可使用 Policy.Wrap。默认关闭,无新增第三方依赖。
- 精确来源白名单、方法/请求头、暴露头、凭据和预检缓存可配置;拒绝凭据与通配来源混用、非法来源/头及歧义的重复请求头,校验错误定位具体字段。
- 在路由和鉴权之前处理预检;实际请求仍执行原有中间件,统一响应、404/405、上传下载和流式能力保持。nethttp/native/HTTP2 共用策略。
- 补充真实生成项目、TLS 三种 transport、20 项真实浏览器检查、race/fuzz/性能基准;同步 README、配置指南、cors 指南及 examples/cors。
- 没有移除现有功能或修改默认跨域行为。代理已负责 CORS 的项目保持关闭,避免重复头;高级调用 Router().ServeHTTP 不经过应用外层 CORS。
## v0.23.0-preview(2026-09-26)
- MySQL/连接池、日志、HTTP 客户端、生命周期和请求策略返回字段级安全诊断,配置加载器保留文件/变量/位置与原始错误链。
- httpc.JSON 新增 WithSuccessStatuses,单次调用可限制允许的 2xx,默认兼容,拒绝响应继续关闭和有界回收。
- copyx.Check[S,T] / Copier.Check 预检嵌套类型与现有规则;修正空集合隐藏类型不匹配,动态接口和 converter 的具体行为继续运行时验证。
- logx.Config.TimeZone 绑定 Logger 实例,JSON/text/子 Logger/标准日志适配一致;不修改 time.Local 或业务时间属性。
- 无新映射文件、强制模块或第三方依赖;更新 README、各包指南和 developerTools 可运行示例。错误文本更精确,空集合中的确定类型错误现在会被拒绝。
## v0.22.0-preview(2026-09-26)
- 文件显式环境引用与备用值,字符串组合/转义,安全有界展开和类型绑定;变量值原样插入、不递归处理。

View File

@@ -141,6 +141,20 @@ config "etc/config.yaml":7:9 env "DB_PORT" field "database.port": expected a val
文件最多 1 MiB;单个环境值/展开字段最多 1 MiB,展开字符串合计/变量缓存分别最多 4 MiB;解析/展开限制 64 层、100000 个节点与引用计数。拒绝无效 UTF-8、多文档、重复字段、未知字段、非普通文件;YAML 不接受 alias/merge/custom tag,XML 不接受 DTD/外部实体。不执行 shell,不自动加载 `.env`。
## 框架模块的字段诊断
v0.23 起 MySQL(含连接池/TLS)、通用数据库池、日志(含轮转/时区)、HTTP 客户端、退出阶段和请求策略返回字段级安全原因。沿用项目已有的 `config.AtField("database", err)` / `config.AtField("server", err)`,自动组合嵌套字段,不需业务重复校验基础参数。
例如文件中 `database.port: "${DB_PORT}"` 的引用值为 70000:
```text
config "etc/config.yaml":3:9 env "DB_PORT" field "database.port": must be between 1 and 65535
```
日志级别非法指向 `server.log.level`;池参数指向 `database.pool.maxIdleConns` / `pingTimeout`。框架只公开已定义的安全原因,第三方/项目普通 error 仍显示 validation failed 并保留 errors.Is/As 原因链;业务使用 Invalid/AtField 提供公开原因。不要解析错误字符串;使用 `*config.Error` 字段。未提供字段的位置可能不可用,字段与原因仍保留。
生成项目现有 Load 和 `-check-config` 自动获得这些诊断。日志可在 `server.log.timeZone` 配置 IANA 名称,见 [实例时区](logx/README.md#实例时区)。
## 可选解析器与测试来源
内置格式无需项目手写加载器。需要额外格式时在已有 NewLoader 注册 Decoder,后续仍自动经过同一引用/绑定/校验流程:
@@ -175,3 +189,11 @@ Decoder 返回 Document(map/string/bool/json.Number/nil/[]any,XML 类标量
5. 升级匹配的 CLI 与框架,执行 generate、generate --check、go test、go build 和 `-check-config`,再使用真实部署配置启动。
低层 `DecodeFile(path, defaults[, loader])` 也解析显式引用,但不调用 Validate;默认值和原始 Document 都不会因失败而被部分修改。日志 Reloader 仍只允许 log.level 变化,其他静态字段变更要求重启。
## 国际化配置
通用国际化的 `i18n.Config` 可作为项目配置的字段显式加载并调用 Validate。正文词典使用 `i18n.LoadFS` / `DecodeJSON`,不经过环境引用展开。构建 Bundle 后通过 `WithLanguagePolicy` / `WithErrorLocalizer` 装配借用的只读实例;没有隐式环境变量或全局当前语言。见 [国际化指南](i18n/README.md) 和 [完整示例](examples/i18n/README.md)。
## 跨域配置
生成项目的 `server.cors` 由本加载流程直接绑定、校验并交给 App,不需要修改生成入口。支持固定来源数组或元素内显式环境引用;非法来源定位具体数组元素。默认关闭,完整配置与行为见 [CORS 指南](cors/README.md)。

25
COPY.md
View File

@@ -28,6 +28,31 @@ if err := copyx.Copy(&input, req); err != nil { return err }
新字段:普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时,两边各声明一次;同名同类型不增加拷贝配置。检查在运行时发生,实际类型对必须有测试,不能把运行时工具当作旧生成器的编译期字段覆盖检查。
## 类型预检
v0.23 新增可选预检,无需填写映射文件或构造测试值:
```go
// 参数方向是源 → 目标;与 Copy 使用同一组选项。
if err := copyx.Check[Request, BusinessInput](options...); err != nil {
return err
}
// 复用已有 Copier:传结果类型,不添加 Copy 所需的额外输出指针。
err := copier.Check(reflect.TypeFor[Request](), reflect.TypeFor[BusinessInput]())
```
适合 CI 测试或初始化检查;这是程序执行的类型检查,不是 Go 编译器自动证明。递归检查切片/数组元素、map 键值、指针、嵌套字段、改名、忽略和严格覆盖;用已访问类型对处理递归结构。不执行 converter,也不调用构造器,不读取实例值。错误路径如 `$.Rows[].Value: copyx: incompatible types: string -> int`。
Check 复用 Copy 的规则,包括浅拷贝对同类型容器的直接赋值:这种情况下内部 converter 不会自动逐项执行。需要处理内部字段时选择深拷贝或容器 converter。
- 未知具体值的源 interface 需要运行时验证;无法静态判定时返回 `ErrDynamicType`。项目可给该接口类型对提供精确 converter 并承担内容验证。
- 已注册 converter 的签名被接受,但函数不在 Check 中执行;其返回错误、panic、金额规则不能靠预检证明。
- nil 到非可空目标、map 键转换碰撞、深拷贝结果是否残留源引用继续在 Copy 时检查。Check 成功不会关闭这些保护。
- 实际 Copy 也修正了空集合的检查缺口:确定的嵌套类型不匹配不再因为切片/map 为空或数组长度为 0 而通过。需要这些转换时提供显式 converter;目标仍只在整体成功后更新。
- 不引入全局配置、映射语言或业务类型;Copier.Check 可并发调用,不共享每次检查的遍历状态。
示例:[developerTools](examples/developerTools/README.md)。普通同名同类型字段不需要额外配置。
## 深拷贝与构造器
```go

View File

@@ -1,4 +1,4 @@
# 通用业务开发指南(v0.22.0-preview)
# 通用业务开发指南(v0.27.0-preview)
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
@@ -199,6 +199,31 @@ go-omni workspace check --file omni.workspace.json
各服务保持独立 module;可显式填写各服务的 `api`(相对服务目录)和 `module`。所有服务预检通过后再写;每个服务独立提交文件事务,不承诺跨项目的崩溃原子性。check 汇总格式与生成漂移;业务编译/测试仍在各 module 执行。
### 混合 API 与通用工程
v0.28.1-preview 起,workspace 支持在同一份清单中管理 API、通用工程,也支持仅包含通用工程的工作区:
```json
{
"services": [
{ "dir": "apps/interface", "kind": "api" },
{ "dir": "apps/program", "kind": "project" }
]
}
```
`kind` 可省略:根据 `.goOmni/apiManifest.json` 或 `.goOmni/projectManifest.json` 识别。没有清单时,沿用旧 API 解释,需显式提供 api 入口才能首次生成。两种清单同时存在、显式 kind 与已有清单冲突、未知 kind 均报错。project 类型不能填写 api;可选 module 用来核对工程身份,不会重写 go.mod。
| 命令 | API 工程 | 通用工程 |
| --- | --- | --- |
| workspace generate | API 生成器 | project 生成器 |
| workspace check | 契约格式检查 + API 生成漂移检查 | project 生成漂移检查 |
| workspace format | 格式化 API 契约 | 显示 SKIP,不修改 Go 文件 |
检查会继续处理独立工程并汇总错误,任一失败返回非零。生成/格式化先预检全部目标;预检失败时输出 FAIL 与其他工程的 SKIP,不写任何生成结果。清单语法、目录越界/重叠或共享目录提供方失败属于工作区级错误,会直接中止。已有文件保护与路径规则保持,事务仍以单个工程为单位。
workspace 不创建或修改 go.work、go.mod、go.sum,不替代 Go 编译和业务测试。它只调用生成器;显式 errorCatalogs 导出仍按下节约定执行 Go 提供方,配置类型不会额外启动服务。完整 [混合工程与 go.work 示例](examples/mixedWorkspace/README.md)。
### 同一份 Go 错误目录供多个服务生成
在 workspace 清单显式声明提供目录的 Go 包和函数:

View File

@@ -22,7 +22,7 @@ app.GET("/metrics", measurements.Handler())
// 也可以直接读取 measurements.Snapshot()。
```
记录请求数、失败数、处理中数量、累计耗时与 HTTP 状态分布;返回的是 JSON 快照,并非 Prometheus 文本格式。指标接口是否暴露以及访问权限由项目配置。需要 HTTP/SQL/Redis spans 和标准 OTel exporter 时,使用 [telemetry 示例](examples/platform/main.go)。CORS 由项目中间件或代理配置,当前没有内置 CORS 模块。
记录请求数、失败数、处理中数量、累计耗时与 HTTP 状态分布;返回的是 JSON 快照,并非 Prometheus 文本格式。指标接口是否暴露以及访问权限由项目配置。需要 HTTP/SQL/Redis spans 和标准 OTel exporter 时,使用 [telemetry 示例](examples/platform/main.go)。需要跨域时配置 `server.cors`,由框架在路由与鉴权前处理预检;默认关闭。已有代理 CORS 可继续使用,避免两处重复设置。见 [CORS 配置与示例](cors/README.md)。
## 缓存:cache 与 cache/redis

81
PROGRAM.md Normal file
View File

@@ -0,0 +1,81 @@
# 通用 Go 项目模板
从 v0.28.0-preview 起,`project new/generate/check` 提供不绑定协议与业务的工程骨架。默认只有配置、日志、依赖装配、生命周期和普通 Go 运行入口;不生成 `.api`、HTTP Handler、Swagger、业务任务,也不监听端口、连接数据库或队列。不增加第三方依赖。
## 创建与运行
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview
go-omni project new demo --module example.com/demo
cd demo
go mod tidy
go run . -check-config
go run . # 默认等待 SIGINT / SIGTERM
go build -o program .
```
目标目录必须为空;未指定 module 时使用目录名。远程安装的 CLI 将自己的固定版本写入 go.mod,无本地替换。当前通用模板要求框架 v0.28.0-preview 或更新版本。源码开发可显式 `--framework-dir /path/to/go-omni`,不会从当前工作目录隐式选择本地框架。`--framework-version` 可指定兼容版本。框架版本、依赖更新由项目修改 go.mod,重新生成不会代为升级依赖。
常驻程序、一次性程序用同一份骨架:项目自己的 `Run` 决定何时完成。完整生成项目见 [examples/program](examples/program/README.md),不需要外部服务。
## 文件职责
| 路径 | 维护方与职责 |
| --- | --- |
| `main.go` | 生成器:信号、启动、运行入口监督、退出状态与日志收尾 |
| `internal/config/loadGen.go` | 生成器:统一配置加载 |
| `internal/svc/runtimeGen.go` | 生成器:依赖初始化及失败收尾 |
| `internal/config/config.go` | 项目:Config、Default、Validate;初始只含 log/lifecycle |
| `internal/config/loader.go` | 项目:按需注册额外文件格式 |
| `etc/config.yaml` | 项目:配置与显式环境引用 |
| `internal/svc/serviceContext.go` | 项目:依赖字段、Init、资源与后台工作登记 |
| `internal/run/run.go` | 项目:Run 的实际实现 |
| `go.mod`、README、.gitignore、其他 Go 文件 | 项目:生成器不覆盖 |
| `.goOmni/projectManifest.json` | 生成器:工程身份、模板版本、生成文件哈希;应提交 Git |
配置遵守 [CONFIG.md](CONFIG.md):默认值 → 文件显式 `${NAME}` / `${NAME:-fallback}` → 类型绑定 → 校验,没有隐式环境覆盖。JSON/YAML/XML 内置;`-config ""` 只用默认值。`-check-config` 不打开日志文件、不初始化依赖、不调用 Run。日志通过独立实例传给 svc,并注入 Context/WorkContext;不替换进程级默认日志实例。
## 运行与退出
```go
func Run(ctx context.Context, deps *svc.ServiceContext) error {
// 一次性程序可完成处理后直接 return nil。
// 常驻程序收到 ctx.Done() 后停止自己的主流程。
<-ctx.Done()
return nil
}
```
`nil` 表示正常完成,返回 error 表示运行失败。后台组件用 `RequestShutdown(err)` 报告致命故障,`RequestShutdown(nil)` 请求正常停止。框架不替项目重试、重新调度或重启任务。
依赖在 `svc.Init` 中装配:创建后立即用 `OnClose` 登记清理,后台组件同时登记同名 `OnStop` 和 `OnDrain`。所有登记应在 Init 返回前完成;Run 不得并发追加生命周期登记。组件何时接收工作由项目决定,停止后不得重新开始领取或启动未登记的工作。
- `Context()` 是提前停止通知,用来停止主流程或接收新工作。
- `WorkContext()` 在正常排空期间仍有效,用于已接受的工作。
- `OnStop` 停止接收新工作,不关闭处理中的工作仍依赖的资源。
- `OnDrain` 返回 nil,必须意味着组件处理、确认和 defer 收尾均已结束。仅返回 `ctx.Err()` 不能证明结束。
- 全部停止钩子先于全部排空钩子;同阶段并行。有先后依赖的组件在一个钩子内显式排序。
- 全部工作确认结束后,资源按 `OnClose` 登记的反序释放,最后关闭入口拥有的日志。
生成入口自动监督 Run 本身,以 `go-omni/project/run` 保留名称登记完成屏障。收到信号、Run 返回或报告故障后开始收尾,同时等待 Run 及其他已登记工作。Run 不应等待资源关闭:资源关闭发生在 Run 完成之后。
初始化失败同样收尾已创建的资源和已启动的工作。Init/Run 内 panic 被转为失败,安全收尾后进程以非零状态退出;任意自建 goroutine 的 panic 仍由项目负责处理。Run 不响应取消或其他工作无法确认结束时,`lifecycle.stopTimeout/drainTimeout/cancelTimeout` 限定等待窗口,返回 `ErrResourcesRetained`,跳过共享资源和日志关闭,以失败状态退出。取消后的工作不能被 Go 强制终止,保留资源不代表工作已经完成。SIGKILL 和进程崩溃不会执行退出钩子。
详见 [生命周期契约](lifecycle/README.md)。现成组件必须按其真实停止/完成语义适配,不能把某个 `Run/Close` 返回直接视为业务结束。
## 生成检查与兼容
```sh
go-omni project generate --dry-run
go-omni project generate
go-omni project check # 等价于 project generate --check
go test ./...
```
生成器不执行项目 Go 代码、不下载依赖。check/预览只读,不创建锁文件;生成时使用独立项目锁、路径和哈希预检。生成文件被手工修改会报冲突,不提供强制覆盖。删除的生成文件可补回;缺失的项目所有文件必须显式恢复,避免把删除的业务文件当成空白骨架重建。
每个文件通过原子替换写入,普通写入失败回滚此前变更;整个目录不具备崩溃事务性。生成期间不要同时手工编辑。发现遗留 project.lock 时,先确认没有生成进程再手工移除。
check 检查模板漂移和必要文件,不检查业务正确性,也不能替代 Go 编译与测试。新的模板版本会重新生成受管文件;不自动合并项目所有文件或更新 go.mod。未知的新清单版本会要求使用兼容 CLI。
原有 `new` / `api new` 及 API 工程行为保留。通用工程使用自己的清单和命令,不能交给 `api generate` 或 `add` 转换;新增业务包直接写普通 Go。从 v0.28.1-preview 起,`workspace generate/check` 可以同时处理 API 与通用工程。清单可填写 `kind: api | project`,省略时按生成清单识别;format 对通用工程显示 SKIP。见 [完整混合示例](examples/mixedWorkspace/README.md)。各 module 仍独立,workspace 命令不维护 go.work 或模块依赖。

View File

@@ -1,6 +1,8 @@
# 生成项目与文件维护边界
# API 生成项目与文件维护边界
v0.22.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
通用 Go 工程使用独立的 `project new/generate/check`,见 [PROGRAM.md](PROGRAM.md)。本文说明 API 契约工程。
v0.28.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
## 创建和日常开发
@@ -98,7 +100,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
最终配置校验包含自定义响应编码器与错误目录的组合。码类型/映射冲突、编码器已绑定另一份目录等问题在创建 App 前返回错误,进入入口的清理流程,不等到构造 App 时才 panic。
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后按框架规则强制关闭连接。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,使用 WorkContext 并登记成对的 OnStop/OnDrain,详见 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭)。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
日志在 AppOptions 前初始化,其配置应放在 Config 中。退出信号先触发 HTTP 排空;达到 shutdownTimeout 后取消请求并关闭连接,再按 `http.shutdownCancelTimeout` 等待处理函数收尾。仍未结束时保留依赖并返回 `ErrResourcesRetained`,不会提前关闭数据库。请求仍依赖的后台任务/客户端不能直接绑定信号 context 提前关闭,应持有独立生命周期,使用 WorkContext 并登记成对的 OnStop/OnDrain,详见 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭)。业务必须响应请求 context 取消;忽略取消的 goroutine 无法保证在超时前结束。
## 依赖主动申请退出
@@ -109,7 +111,7 @@ func (c *ServiceContext) AppOptions() []omni.Option {
c.RequestShutdown(err)
```
这只是通知,立即返回,不等待 HTTP 排空,也不直接关闭数据库。生成入口停止接收新请求,等待在途请求完成;超过 `shutdownTimeout` 后取消请求并关闭连接,再执行 svc.Close,最后关闭日志。非空故障原因保留在返回错误链中,进程退出码为 1;`RequestShutdown(nil)` 或正常 SIGINT/SIGTERM 在没有其他故障时退出码为 0。
这只是通知,立即返回,不等待 HTTP 排空,也不直接关闭数据库。生成入口停止接收新请求,等待在途请求完成;超过 `shutdownTimeout` 后取消请求并关闭连接,再等待 HTTP 取消收尾窗口。通过 `svc.Finish(runErr)` 执行后台停机;确认工作结束才关闭依赖与日志,无法确认时保留资源并非零退出。非空故障原因保留在返回错误链中,进程退出码为 1;`RequestShutdown(nil)` 或正常 SIGINT/SIGTERM 在没有其他故障时退出码为 0。
多个依赖可并发通知,关闭流程仅执行一次,首个非空故障原因保留。正常信号已开始排空时收到的故障也不会被清理动作覆盖。初始化期间的通知同样有效,不依赖 HTTP 监听已经启动。后台监控任务应通过 OnStop/OnDrain 停止并等待,再进入资源清理;关闭完成以后才报告的故障无法参与已经结束的进程结果。
@@ -215,7 +217,7 @@ get /download (DownloadReq)
## 已有服务接入步骤
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.22.0-preview:
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.28.0-preview:
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。

103
README.md
View File

@@ -1,8 +1,22 @@
# GoOmni
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
GoOmni 是面向多个项目复用的 Go 基础框架。通用工程使用 `project new` 生成配置、日志、依赖装配、生命周期和普通 Go 运行入口;需要 HTTP 接口时可选择 `.api` 工程生成,也可以在已有项目中按需引入独立模块。运行方式、业务类型和外部依赖由项目决定。
**从这份 README 开始接入。** 当前版本 **`v0.22.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**v0.28.1 改动**:`workspace generate/check` 支持 API 与通用工程混合处理,可选 `kind: api | project`,省略时按生成清单识别。`format` 对通用工程显示跳过;检查只读,生成前统一预检。见 [混合工作区示例](examples/mixedWorkspace/README.md)。
**v0.28 改动**:新增独立的 `project new/generate/check`,默认无 API、监听端口、业务任务或外部服务配置;保护手写代码,并等待主运行入口和登记的后台工作结束后释放依赖。见 [通用工程指南](PROGRAM.md)和[可运行示例](examples/program/README.md)。
**v0.27 改动**:统一三种内置服务实现的默认/路由/硬请求体限额,配置冲突在启动前报错;停机以处理函数实际退出为资源清理前提,超时仍有请求时保留共享依赖。见 [HTTP 策略](transport/README.md#统一请求体限额)、[停机规则](lifecycle/README.md#http-请求与依赖的关闭屏障)和[可运行示例](examples/httpSafety/README.md)。
**v0.26 改动**:通用 [i18n](i18n/README.md),提供独立词典、语言匹配、回退、命名参数/条件/复数、可选 HTTP 与错误适配、检查/导出 CLI 及前端运行时。文案由项目维护;[完整示例](examples/i18n/README.md)可直接运行。CLI 与框架使用同版 v0.28.1-preview。同时修复 Go 1.22 下请求体限额错误的识别。
**从这份 README 开始接入。** 当前版本 **`v0.28.1-preview`**,通用工程采用根目录入口和 `config / svc / run`;API 工程使用 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**v0.25 改动**:`database.InTx` 返回可识别的事务阶段与结果,保留业务/驱动及回滚错误链;提交报错保守标记为结果不确定,不等同于已回滚。[使用规则](database/README.md#事务阶段与结果)、[可运行故障示例](examples/transactions/README.md)。
**v0.24 改动**:新增可选 CORS,生成项目通过 `server.cors` 自动接入,支持预检、错误响应和明确的来源白名单;默认关闭。[配置与规则](cors/README.md)、[可运行示例](examples/cors/README.md)。
**v0.23 改动**:配置校验定位具体字段和安全原因;`httpc.JSON` 可选限制成功状态;`copyx.Check` 无需样本预检类型;`logx` 支持实例独立时区。[四项完整示例](examples/developerTools/README.md)。
**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。
@@ -28,32 +42,72 @@ Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可
| --- | --- | --- |
| 项目、接口与文档生成 | `go-omni new`;改 `.api` 后 `api generate`;`--check` 检查漂移 | [下方快速开始](#新项目生成并运行)、[CLI](cmd/go-omni/README.md)、[完整生成项目](examples/managedService/README.md) |
| 公共字段与跨服务共享契约 | `.api` 组合对象字段;`omni.imports.json` 声明共享根;workspace 批量生成 | [公共字段示例](GENERATION.md#公共字段组合)、[共享契约与 workspace](DEVELOPMENT.md) |
| 混合工程工作区 | `workspace generate/check` 按清单分发 API/project;format 仅处理 API | [可运行示例](examples/mixedWorkspace/README.md)、[工作区说明](DEVELOPMENT.md#混合-api-与通用工程) |
| 通用 Go 工程 | `go-omni project new/generate/check`;配置、日志、依赖、运行与退出 | [通用工程指南](PROGRAM.md)、[完整可运行示例](examples/program/README.md) |
| 中间件依赖工厂、可选 Repository | `omni.gen.json` 选择 factories / none;在 `New<Name>(deps)` 写一次实现 | [配置与完整工厂示例](GENERATION.md) |
| 路由、分组、请求上下文 | `omni.New()`、`app.GET` / `Group` / `Use`;业务使用 `context.Context` | [基本服务](examples/basic/main.go)、[已有项目接入](#已有项目按模块接入) |
| HTTP 网络与扩展适配 | 配置 transport 为 nethttp / native / http2;自定义 Router/Runner | [网络层配置与契约](transport/README.md)、[性能与限制](PERFORMANCE.md) |
| 浏览器跨域 CORS | `server.cors` / `omni.WithCORS`;独立 HTTP 用 `cors.New(...).Wrap(handler)` | [配置、默认值与协议边界](cors/README.md)、[鉴权与预检示例](examples/cors/README.md) |
| HTTP 老接口桥接 | 通过官方 Handler/Middleware 适配器接入标准 `net/http`,跟踪实际响应状态 | [接入说明](MIGRATION.md)、[桥接/上传/自定义 ID 示例](examples/compatibility/main.go) |
| 请求校验、动态 JSON、自定义标量 | `.api` 声明来源/校验/scalar;手写契约用 `contract.New[T]`;简单绑定可用 `validate.JSON/Query/URI` | [语法与例子](API.md)、[请求策略](DEVELOPMENT.md) |
| 数组、null、原始请求、上传下载 | `.api` 声明数组/可空返回、body/multipart 与大小限制;必要时使用 Bind/Manual 扩展 | [契约示例](examples/contracts)、[含上传/回调/下载的项目](examples/managedService/README.md) |
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
| 通用国际化 | `i18n.New/Render`;可选 `WithLanguagePolicy/WithErrorLocalizer`;`go-omni i18n check/export/types/extract` | [资源格式与边界](i18n/README.md)、[CLI/HTTP/前端示例](examples/i18n/README.md)、[生成契约](examples/contracts/i18n.api) |
| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`;支持独立时区、轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
| 文件配置与环境引用 | Default()→JSON/YAML/XML 字段引用→类型绑定→校验;无隐式覆盖,支持 `-check-config` | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;`InTx` 的 `TxError` 提供阶段、已知结果与原因链 | [通用数据库](database/README.md)、[事务失败示例](examples/transactions/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时;JSON 支持单次请求头/限额;`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时;JSON 支持单次成功状态限制/请求头/限额;`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
| 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;`Check[S,T]` 预检类型,Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
## 通用工程快速开始
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview
go-omni project new demo --module example.com/demo
cd demo
go mod tidy
go run . -check-config
go run .
# Ctrl+C / SIGTERM 退出;项目自己的 Run 也可以完成后直接返回。
go-omni project check
go test ./...
```
项目只修改配置、svc 依赖初始化和 Run;默认没有 HTTP 端口、数据库、队列或自动重试。`main.go`、加载和装配代码由生成器维护;手改受管文件会报冲突,手写文件保持。完整维护边界、异常退出和超时行为见 [PROGRAM.md](PROGRAM.md)。
## 浏览器跨域 CORS
同源部署保持默认关闭;需要跨域时,在生成项目的配置文件中加入:
```yaml
server:
cors:
enabled: true
allowedOrigins: [https://admin.example.com, https://merchant.example.com]
allowedMethods: [GET, HEAD, POST, PUT, PATCH, DELETE]
allowedHeaders: [Authorization, Content-Type]
exposedHeaders: [Content-Disposition, X-Request-ID]
allowCredentials: false
maxAge: 10m
```
不需要修改 main、Logic 或 `.api`。框架在鉴权前处理合法预检;实际请求仍鉴权,401/404/405/500 保持项目响应格式并附带允许来源的跨域头。没有 Origin 的服务端回调正常处理。配置只决定浏览器读取权限,不替代登录、验签或 CSRF 防护。
手写项目用 `omni.WithCORS(omni.CORSConfig{...})`,独立 HTTP 服务用 `cors.New(...).Wrap(handler)`;代理已经处理 CORS 时保持关闭。完整 [默认值与边界](cors/README.md)、[可运行示例](examples/cors/README.md)。
## 带认证的出站 HTTP
HTTP Client 在 svc 初始化一次,业务重复使用。普通内部请求可直接选择直连和连接参数,无需手写 Transport:
@@ -126,7 +180,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
```sh
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.22.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview
export PATH="$(go env GOPATH)/bin:$PATH"
go-omni --help
```
@@ -283,7 +337,7 @@ go-omni new --api catalog.api \
在业务项目的 **go.mod 所在目录**执行:
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.22.0-preview
go get git.yuntuo.hk/k2safe/go-omni@v0.28.1-preview
```
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
@@ -509,6 +563,28 @@ if err := copier.Copy(&view, input); err != nil { return err }
自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错;字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md),完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查,类型对需测试;深拷贝不适用于连接、锁等资源。
### 减少接入代码:四个常用入口
```yaml
server:
log:
timeZone: Asia/Shanghai
```
时区由 Logger 实例管理,不修改全局 `time.Local`。非法日志级别、数据库端口/连接池等配置会指出 `server.log.level`、`database.port` 等字段和原因,不打印输入值。详见 [配置诊断](CONFIG.md#框架模块的字段诊断)、[日志时区](logx/README.md#实例时区)。
```go
// 本次请求只接受 HTTP 200;默认仍接受所有 2xx。
err := client.JSON(ctx, http.MethodGet, url, nil, &result,
httpc.WithSuccessStatuses(http.StatusOK),
)
// CI/启动时检查类型,无需构造样本;仍使用 Copy 的同一组 Option。
err = copyx.Check[Request, BusinessInput](options...)
```
状态不符合要求返回 `*httpc.StatusError`,不会写入 result,响应体仍由框架关闭。Check 不执行 converter;动态 interface、具体 nil/转换失败和深拷贝引用检查有明确边界,见 [类型预检](COPY.md#类型预检)。完整可运行代码见 [developerTools](examples/developerTools/README.md)。
### 数据库、存储及其他模块
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
@@ -537,6 +613,9 @@ API 项目的数据库迁移需要单独维护并显式执行;启动不会自
| `server.http.readHeaderTimeout/readTimeout` | 请求头 / 请求读取超时 |
| `server.http.writeTimeout/idleTimeout` | 响应写入 / Keep-Alive 空闲超时 |
| `server.http.maxHeaderBytes` | 请求头大小限制,字节数 |
| `server.http.defaultMaxBodyBytes` | 普通 App 路由默认请求体上限,零默认 8 MiB;路由可覆盖 |
| `server.http.maxBodyBytes` | 服务请求体硬上限,路由不得超过;零不添加低于框架 1 TiB 的硬上限 |
| `server.http.shutdownCancelTimeout` | 请求取消后等待处理函数收尾,零默认 5s;未结束则保留资源 |
| `server.shutdownTimeout` | 优雅退出等待时间,默认 10s |
| `server.enableLogger` / `server.log.level` | 访问日志开关、日志级别 |
| `server.trustedProxies` | 可信代理 IP/CIDR,文件数组 |
@@ -564,13 +643,15 @@ server:
maxHeaderBytes: 1048576
```
代码配置使用 `omni.WithHTTPConfig(omni.HTTPConfig{ReadTimeout: 30*time.Second, WriteTimeout: 30*time.Second})`。`Run`、`RunContext`、`ServeContext` 会应用它;`NewServer` 会配置返回的标准 `http.Server`,包含可选的 Go 字段 `ErrorLog`。**正值覆盖、零值保留所选网络层的原有默认行为、负值报错**;不配置时升级行为不变。超时含义与支持范围见 [HTTP 配置](transport/README.md#http-配置v091-preview),不要把网络 I/O 超时当成业务执行时限。
代码配置使用 `omni.WithHTTPConfig(omni.HTTPConfig{ReadTimeout: 30*time.Second, WriteTimeout: 30*time.Second})`。`Run`、`RunContext`、`ServeContext` 会应用它;`NewServer` 会配置返回的标准 `http.Server`,包含可选的 Go 字段 `ErrorLog`。**网络 I/O 参数正值覆盖、零值保留所选网络层的原有默认行为、负值报错**。请求体与取消收尾字段按上表解释;网络超时含义与支持范围见 [HTTP 配置](transport/README.md#http-配置v091-preview),不要把网络 I/O 超时当成业务执行时限。
外部宿主在注册完成后可用 `app.ValidateHTTP()` 预检配置与路由限额。上传路由使用 `app.HandleWithBodyLimit("POST", "/upload", 16<<20, nil, handler)` 或 `.api @request(maxBytes: 16777216)`;限额涵盖整个请求体。配置硬上限时至少允许该值;multipart 单文件、字段数以及幂等中间件缓存限制仍独立配置。新增普通路由默认 8 MiB;原 nethttp 手写大请求请显式设置默认值或路由上限,生成路由既有 `maxBytes`(默认 1 MiB)保持。
自研网络层通过 `RunWith(ctx, runner)` 接入监听与连接生命周期,通过 `NewWithRouter(router)` 替换路由;业务 Handler/Service 不需要导入具体适配器。细节与验收契约见 [网络适配层](transport/README.md)。
生成项目的依赖可调用 `svcCtx.RequestShutdown(err)` 主动申请退出,框架排空请求、清理依赖后保留故障原因并以非零状态退出。健康检查与 OpenAPI 的开关、路径通过 `server.endpoints` 配置,健康响应默认沿用项目公共体;未就绪保持 HTTP 503。完整配置和 v0.12 项目的升级步骤见 [项目指南](PROJECT.md#依赖主动申请退出)。
`Run` 处理 SIGINT/SIGTERM;`RunContext` 接收调用方控制的退出信号。服务先停止接收新请求,等待活跃请求完成,再清理资源;超时会取消请求并关闭连接,业务代码仍需响应 context 取消。
`Run` 处理 SIGINT/SIGTERM;`RunContext` 接收调用方控制的退出信号。先停止接收新请求并等待排空,超时取消请求、关闭连接,再等待 `http.shutdownCancelTimeout` 收尾窗口。只有处理函数和它的 defer 全部退出才清理资源;仍有请求时返回 `lifecycle.ErrResourcesRetained`,跳过 App 清理钩子。生成入口通过 `deps.Finish(runErr)` 同步保留数据库等依赖及日志,非零退出。`App.Close`、`Engine`、`NewServer`、`RunWith` 共用这个完成屏障;业务自行启动的 goroutine、Hijack 后的工作需另行注册后台生命周期。
## 日常开发与升级
@@ -587,7 +668,7 @@ go test ./...
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
CLI 与业务 module 固定使用 `v0.22.0-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load,再升级依赖与 CLI 并重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
CLI 与业务 module 固定使用 `v0.28.1-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load,再升级依赖与 CLI 并重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
## 按需深入

View File

@@ -1,4 +1,162 @@
# v0.22.0-preview
# v0.28.1-preview
workspace generate/check 支持 API 与通用工程混合分发,也可处理纯通用工程工作区。services 新增可选 kind(api/project);省略时按生成清单识别,保留无清单时显式 api 入口的原有用法。类型与清单冲突、歧义和不合法选项会报错,项目身份、路径与文件归属继续由各生成器校验。
check 只读并汇总独立工程的失败;format 只处理 API 契约,对通用工程明确显示 SKIP。generate 预检所有工程后再写入,预检失败时不修改其他工程;不改变各模块依赖或 go.work,不增加业务逻辑和第三方依赖。模板与运行时接口未变,新 workspace 功能需要更新 CLI,旧生成工程可直接纳入。
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview
go-omni workspace generate
go-omni workspace check
```
[混合工作区示例](examples/mixedWorkspace/README.md)、[清单规则](DEVELOPMENT.md#混合-api-与通用工程)、[验收记录](reports/workspaceAcceptance.md)。56 个测试包全量 race、5 个 CLI 包的 Go 1.22 回归、实际生成的多模块构建、vet/build 均通过。
---
# v0.28.0-preview
新增通用 Go 工程生成命令 `project new/generate/check`。默认只有配置、日志、svc 依赖装配、生命周期与项目维护的 Run 入口,不生成 API、HTTP Handler、Swagger、业务任务或网络监听,也不自动连接数据库/队列或重试任务。常驻和一次性程序由普通 Go 代码决定。
入口监督 Run 的真实完成并复用现有 Runtime:信号、正常结束、运行错误或初始化失败都会进入收尾;先停止接收工作、等待已登记工作及 Run 结束,再反序释放依赖并关闭日志。Run/Init panic 转为失败;超时仍有工作则传播 ErrResourcesRetained,保留共享资源及日志,以非零状态退出。组件登记在 svc.Init 完成,业务语义和调度方式由项目负责。
配置遵守现有显式文件引用约定;支持 -check-config,在初始化日志和依赖前校验。生成器只管理 main/loadGen/runtimeGen,手写文件保留,生成文件被修改时拒绝覆盖;check/预览只读,独立 manifest/锁和共享文件事务保障生成。新模板要求框架 v0.28.0-preview;已有 new/api/workspace 行为保留,无强制迁移和新增第三方依赖。
```sh
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.0-preview
go-omni project new demo --module example.com/demo
cd demo
go mod tidy
go run . -check-config
go run .
go-omni project check
```
完整 [通用工程指南](PROGRAM.md)、[可运行示例](examples/program/README.md)、[回归记录](reports/programAcceptance.md)。56 个测试包全量 race 与 Go 1.22 回归、生成消费者、vet/build 和真实进程退出验证通过。
空缓存远程安装已完成:通用工程 30 项检查通过(26.45 秒),原有生成项目 98 项检查、10 组消费者通过(111.07 秒),均无本地 replace。见 [通用远程报告](reports/programRemote.json)、[原有能力远程报告](reports/programManagedRemote.json)。标签固定在 `0b6c0043b6c3b58648645aeee03bf80801e6aa3f`。
---
# v0.27.0-preview
新增统一请求体策略与请求完成屏障,无业务字段、数据库依赖或新第三方包。`server.http.defaultMaxBodyBytes` 控制普通路由默认值(零为 8 MiB),`maxBodyBytes` 为不可由路由突破的服务硬上限(零仅保留 1 TiB 框架上界)。已有显式路由和生成契约限额保持,冲突会在启动前报错;旧 nethttp 手写大请求需显式设置路由或默认上限。multipart 单文件、中间件缓存及代理限额仍各自配置。
HTTP 关闭连接后继续等待处理器实际退出,收尾窗口由 `server.http.shutdownCancelTimeout` 设置(零默认 5s)。超时仍有处理器运行时返回 `lifecycle.ErrResourcesRetained`,App/Runtime/生成入口保留共享依赖和日志;不再将连接关闭当成业务完成。`Runtime.Finish(runErr)` 会传播该信号,手写入口必须处理它,避免无条件关闭额外资源。该行为适用于全部 App 服务入口;HTTP/2 低层 Shutdown 同时等待独立处理器。
详见 [请求体策略](transport/README.md#统一请求体限额)、[停机规则](lifecycle/README.md#http-请求与依赖的关闭屏障)、[运行示例](examples/httpSafety/README.md)。CLI 与框架固定使用同版 v0.27.0-preview;模板最低兼容版本仍为 v0.22,HTTP 安全改进需要升级运行时。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.27.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.27.0-preview
go-omni api generate --check
go test ./...
```
54 个测试包全量 race、Go 1.22 兼容检查、生成项目集成测试、vet/build 及真实进程关闭检查通过,见 [验收记录](reports/httpSafetyAcceptance.md)。
空模块缓存安装远程固定版本后,10 组生成消费者和 98 项发布检查通过,耗时 107.57 秒,无本地 replace;覆盖上传限额、取消收尾、HTTP/2 断连、生成项目依赖与日志保留。标签固定在 `3b00bb55d189cbe480c7d15c8d14bcd09d3f2d52`,见 [远程验收记录](reports/httpSafetyRemote.json)。
---
# v0.26.0-preview
新增可独立使用的 i18n 底层模块及可选 HTTP/错误适配。项目维护文案和语言设置,框架提供不可变词典、语言匹配、回退、参数与分支、检查和导出;没有绑定业务身份、页面框架、数据库或响应字段名。
前后端共享版本化消息资源和固定 Unicode CLDR 46.1 基数复数规则。CLI 只读输入,支持源目录提取、缺译/参数/动态引用检查、按命名空间导出语言包和 TypeScript 类型。可选浏览器模块先加载并验证再切换,失败保留当前语言,旧请求不能覆盖新选择。
错误目录新增可选 `translation` 引用。内置响应使用展示译文,原目录文案继续参与契约校验;自定义 Encoder 通过 `PublicMessage()` 显式采用译文。业务码、HTTP 状态、成功/失败语义、cause、直接协议输出和幂等首次快照保持。旧配置与生成项目不启用时无需迁移。
完整 [使用指南](i18n/README.md)、[通用示例](examples/i18n/README.md)、[验收记录](reports/i18nAcceptance.md)。包含 translation 的新契约远程依赖最低版本为 v0.26.0-preview;未使用该元数据的模板最低版本仍为 v0.22。CLI 与框架使用同版。
同时修复 JSON 结束定界符读取失败时丢失原始错误的问题,使请求体超过外层限制时在 Go 1.22 中也保持 HTTP 413;共享目录测试工程使用 Go 1.22。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.26.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.26.0-preview
go-omni api generate --check
go test ./...
```
发布前 54 个测试包全量 race 与 Go 1.22(含生成消费者)、vet/build、真实进程检查通过。空缓存安装远程固定版本后,10 组生成消费者和 87 项发布检查通过,耗时 83.76 秒,无本地 replace;包含下载模块的国际化、CLDR/浏览器一致性、TypeScript 类型和 CLI 工具链验证。标签固定在 `d1a747c36f15d964dd445d661623b68a45892adf`,[远程验收记录](reports/i18nRemote.json)。
---
# v0.25.0-preview
事务失败可以按阶段和已知结果处理,避免把“提交错误”误判为“数据库没有保存”。
- `database.InTx` 签名不变,普通错误返回 `*database.TxError`;Stage 为 begin/work/commit,Outcome 为 notStarted/rolledBack/unknown。成功提交仍返回 nil。
- Commit 报错一律 unknown;回调失败且显式 Rollback 返回 nil 才标记 rolledBack。BeginTx 失败不执行回调。ErrTxDone 不作为回滚成功证据,框架不重试事务。
- `Err` 与 `RollbackErr` 保留两条原因链,可通过 errors.Is/As 识别;APIError、目录参数提示和公开响应继续工作。内部原因不自动进入 HTTP 提示。
- 提供无需外部数据库的 [故障演示](examples/transactions/README.md)。真实 MySQL 与性能结果见 [验收记录](reports/transactionAcceptance.md)。不引入新依赖,不包含文件删除、补偿任务或业务规则。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.25.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.25.0-preview
go-omni api generate --check
go test ./...
```
**兼容边界:** 既有 errors.Is/As 保持;直接 `err == sentinel`、直接类型断言或字符串匹配需调整。nil context 使用 Background,nil 回调在开启前返回 typed error。panic/Goexit 继续传播并尝试回滚,不转换为 error;标准库自动回滚不公开的原因无法还原,不能推断已回滚。结果仅覆盖按约定使用的当前事务,不覆盖非事务表、隐式提交 SQL、其他连接或外部副作用。GORM 原生 Transaction 没有被替换。
[database 使用规则](database/README.md#事务阶段与结果) 已同步。本轮 runtime 能力需要 v0.25,模板最低版本仍为 v0.22;没有新增必填配置,当前仍为 preview。
49 个测试包全量 race、Go 1.22、vet/build、取消竞争复测通过;成功事务没有新增分配。固定远程版本在空缓存下通过 77 项发布检查和 9 组生成消费者,耗时 71.93 秒。独立 MySQL TLS 容器的 53 条测试记录(含子用例)通过,包含真实提交/回滚后的确认丢失、可选 GORM、生成项目和后台关闭,耗时 28.49 秒。均无本地 replace,标签固定在 `7a177744ab2bb85f23efc0ec267fd905d349bd4b`。见 [远程发布记录](reports/transactionRemote.json)、[远程 MySQL 记录](reports/transactionMySQLRemote.json)。
---
# v0.24.0-preview(历史版本)
业务项目通过配置开启跨域,不需要手写 OPTIONS Handler 或逐条修改接口。
- 默认关闭,在 `server.cors` 显式配置来源、方法和请求头;统一文件加载、环境引用和字段级诊断自动生效。生成项目无需修改 main、svc 或 `.api`。
- 预检在鉴权前返回 204/403 空响应;实际请求继续鉴权、业务错误编码和路由处理。允许来源可读取 401/403/404/405/500,上传下载和流式能力保留。
- 只支持精确 HTTP(S) 来源,公开无凭据服务可配置单一 `*`;禁止 `*` 与凭据混用。配置初始化快照、请求并发隔离、Vary 缓存分流和有界请求头处理。
- 同一个 App 策略覆盖标准库、native 和 HTTP2;独立服务可使用 `cors.New(...).Wrap(handler)`。没有新增第三方依赖,没有移除现有功能。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.24.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.24.0-preview
go-omni api generate
go-omni api generate --check
go run . -config etc/config.yaml -check-config
```
现有项目不配置 CORS 时行为不变。代理已处理 CORS 的项目继续关闭;跨域不替代登录、权限或 CSRF 防护。通过 App/Engine/Run 接入,直接对外暴露底层 Router 会绕过应用外层策略。本轮新增 runtime 配置需 v0.24,未使用该配置的模板最低版本仍为 v0.22。
[配置与规则](cors/README.md)、[可运行示例](examples/cors/README.md)、[验收与性能](reports/corsAcceptance.md) 已同步。49 个测试包全量 race、Go 1.22、vet/build、实际生成项目、三种 transport 的 TLS 场景和 20 项真实浏览器检查通过;30 秒 fuzz 约 288 万次。当前仍为 preview。
空缓存安装远程版本后,9 组生成消费者和 70 项发布检查通过,耗时 66.78 秒,无本地 replace;包括下载模块的 cors/app race、三种 transport 的 TLS 检查和只改配置的生成项目。标签固定在 `800974cfff6228c2697373d9961d3c7c75611d08`,见 [远程记录](reports/corsRemote.json)。
---
# v0.23.0-preview(历史版本)
减少业务接入时重复维护的配置校验、HTTP 包装和类型检查。
- 配置诊断定位 database.port、database.pool.maxIdleConns、server.log.level/timeZone 等具体字段,保留文件、环境变量、行列和安全原因;凭据不进入公开提示,原始原因可 errors.Is/As。
- JSON 可传 `httpc.WithSuccessStatuses(200)`,默认仍接受全部 2xx;不匹配时不改目标,继续负责有界读取、关闭和连接回收,不添加新的重试规则。
- `copyx.Check[S,T](options...)` 或已有 Copier.Check 复用同一组 Go 选项,无需样本和映射文件;实际拷贝不再因集合为空而漏过确定的类型不匹配。动态 interface、nil 值、converter 失败及深拷贝共享引用仍有运行时检查。
- `server.log.timeZone: Asia/Shanghai` 配置实例时区;JSON/text/子日志一致,业务 time.Time 属性保持原样。极简镜像可用 `go build -tags timetzdata .` 内置时区数据,库不强制增加体积。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.23.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview
go-omni api generate
go-omni api generate --check
go run . -config etc/config.yaml -check-config
```
现有调用接口保留;配置错误文本现在更精确,业务应通过类型/字段识别错误。空集合里确定的类型错误会提前暴露,需要显式 converter 或调整实际类型,不应靠空值跳过校验。预检通过不代表 converter 对所有业务值必定成功。当前模板最低版本仍是 v0.22,本轮新增 runtime 能力需使用 v0.23。
[完整示例](examples/developerTools/README.md)、[配置](CONFIG.md)、[JSON 选项](httpc/README.md)、[类型预检](COPY.md#类型预检)、[日志时区](logx/README.md#实例时区)。验收与性能记录见 [本轮验收](reports/developerToolsAcceptance.md)。仍为 preview。
48 个测试包全量 race、Go 1.22、vet/build、实际生成消费者及约 392 万次 copyx fuzz 通过。空缓存安装远程版本后,66 项发布检查通过,包括下载模块的 config/httpc/copyx/logx race;耗时 57.06 秒,无本地 replace。标签固定在 `3bbc6f60d76cb9eae6f01fa6ca122eb1fbf481af`,见 [远程验收](reports/developerToolsRemote.json)。
---
# v0.22.0-preview(历史版本)
配置文件明确决定每个值的来源,项目只维护 Config、Default()、Validate() 和文件。
@@ -19,6 +177,8 @@ go run . -config etc/config.yaml -check-config
[完整规则与迁移](CONFIG.md)、[可运行示例](examples/configuration/README.md)、[场景与性能验收](reports/explicitConfigurationAcceptance.md) 已同步。48 个测试包全量 race、实际生成消费者、Go 1.22、vet/build 和约 28 万次引用 fuzz 通过;小配置固定值/带引用加载约 25 µs,属于本机启动阶段基准。当前仍为 preview。
空缓存远程 CLI 安装、下载模块 config/httpc race、8 组实际生成消费者与 `-check-config` 验收通过,无本地 replace。标签固定在 `d598b2ebee8cde36859021e882a6bc1e0bb4fd4c`;见 [远程验收记录](reports/explicitConfigurationRemote.json)。
---
# v0.21.0-preview(历史版本)

View File

@@ -1,5 +1,7 @@
# 公共响应体与独立接口协议
v0.26 新增的 [国际化适配](i18n/README.md#接入-go-omni-错误与-http)保留现有响应契约。`Result.Message` 是校验使用的目录原文;`MessageOverride` 是可选展示译文,`PublicMessage()` 返回最终文案。内置 Formatter 自动使用译文;自定义 Encoder 显式选择,固定文本/XML 回执和直接输出不自动改写。
这些功能从 `v0.8.0-preview` 提供。CLI 与项目框架依赖请固定到同一版本;源码开发可显式使用 `--framework-dir`。`v0.7.0-preview.1` 不包含响应模板声明。
go-omni 的响应编码由应用或接口配置。没有配置时保留旧版默认值;业务可以定义自己的字段、嵌套结构、错误码类型、提示和空值行为。`response.Response` 仅保留为兼容 DTO,运行时与生成器不再通过它固定响应结构。

View File

@@ -12,6 +12,7 @@ import (
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/cors"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
@@ -24,6 +25,7 @@ import (
// httpx.Router, leaving the network implementation replaceable.
type App struct {
router httpx.Router
handler http.Handler
config config.Config
middlewares []httpx.Middleware
closers []func() error
@@ -31,6 +33,10 @@ type App struct {
closeErr error
clientIPResolver *httpx.ClientIPResolver
responseEncoder response.Encoder
requests requestDrain
bodyLimits []routeBodyLimit
validateOnce sync.Once
validateErr error
}
// Runner is the small lifecycle boundary used by an HTTP server
@@ -67,6 +73,11 @@ func configure(options ...config.Option) config.Config {
}
func newApp(router httpx.Router, settings config.Config) *App {
settings.CORS = settings.CORS.Clone()
corsPolicy, corsErr := cors.New(settings.CORS)
if corsErr != nil {
panic(config.AtField("cors", corsErr))
}
settings.Response = settings.Response.Clone()
settings.Errors = settings.Errors.Clone()
encoder, responseErr := response.New(settings.Response)
@@ -93,6 +104,14 @@ func newApp(router httpx.Router, settings config.Config) *App {
}
middlewares = append(middlewares, middleware.Recovery())
application := &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver, responseEncoder: responseEncoder}
application.requests.done = make(chan struct{})
application.handler = corsPolicy.Wrap(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.ContentLength > application.hardBodyLimit() {
application.adapt(func(*httpx.Context) error { return apierrors.New(413, "BODY_TOO_LARGE", "request body exceeds limit") }).ServeHTTP(w, r)
return
}
router.ServeHTTP(w, r)
}))
if fallback, ok := router.(httpx.FallbackRouter); ok {
adaptError := func(status int, code, message string) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
@@ -104,21 +123,39 @@ func newApp(router httpx.Router, settings config.Config) *App {
return application
}
// Router returns the transport contract used by this application.
// Router exposes the underlying routing contract for advanced registration.
// Serve through App or Engine for CORS, hard body limits and shutdown tracking.
func (a *App) Router() httpx.Router { return a.router }
// Engine is kept as a compatibility name for the old API. It returns a
// standard http.Handler, so callers do not depend on a router type.
func (a *App) Engine() http.Handler { return a.router }
func (a *App) Engine() http.Handler { return a }
// ServeHTTP implements http.Handler for standard servers and HTTP tests.
func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
a.router.ServeHTTP(writer, request)
ctx, work := a.requests.begin(request.Context())
if work == nil {
// Do not invoke encoders, middleware or loggers after their owners close.
http.Error(writer, "server shutting down", http.StatusServiceUnavailable)
return
}
defer a.requests.end(work)
a.validateOnce.Do(func() { a.validateErr = a.ValidateHTTP() })
if a.validateErr != nil {
http.Error(writer, "invalid HTTP configuration", http.StatusServiceUnavailable)
return
}
request = request.WithContext(ctx)
if request.Body != nil {
request.Body = http.MaxBytesReader(writer, request.Body, a.hardBodyLimit())
}
a.handler.ServeHTTP(writer, request)
}
// Config returns the effective application configuration.
func (a *App) Config() config.Config {
settings := a.config
settings.CORS = settings.CORS.Clone()
settings.Response = settings.Response.Clone()
settings.Errors = settings.Errors.Clone()
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
@@ -133,7 +170,8 @@ func (a *App) Use(middleware ...httpx.Middleware) *App {
}
// OnShutdown registers a cleanup function owned by the application. Cleanup
// functions run in reverse registration order when Close or RunContext ends.
// functions run in reverse registration order after all handlers have returned.
// Incomplete draining skips these callbacks and returns ErrResourcesRetained.
// This keeps omni.New as the only public startup entry point while allowing
// projects to attach databases, loggers and other resources to its lifecycle.
func (a *App) OnShutdown(cleanup func() error) *App {
@@ -143,9 +181,21 @@ func (a *App) OnShutdown(cleanup func() error) *App {
return a
}
// Close releases application resources. It is safe to call more than once.
// Close stops admission, cancels active requests and waits for their handlers
// to return before releasing resources. It is idempotent. If cancellation grace
// expires, ErrResourcesRetained is permanent and no cleanup callbacks run.
// Use RunContext/ServeContext for graceful draining before cancellation. Do not
// call Close from a handler or cleanup callback; it would wait for itself.
func (a *App) Close() error {
a.closeOnce.Do(func() {
timeout := a.config.HTTP.ShutdownCancelTimeout
if timeout <= 0 {
timeout = 5 * time.Second
}
if err := a.requests.cancelAndWait(timeout); err != nil {
a.closeErr = err
return
}
for index := len(a.closers) - 1; index >= 0; index-- {
if err := a.closers[index](); err != nil {
a.closeErr = errors.Join(a.closeErr, err)
@@ -167,7 +217,7 @@ func (a *App) Run(address ...string) error {
// Invalid HTTP options panic; RunContext/ServeContext return validation errors.
// Use RunContext or ServeContext to apply the configured transport and TLS.
func (a *App) NewServer(address string) *http.Server {
if err := a.config.HTTP.Validate(); err != nil {
if err := a.ValidateHTTP(); err != nil {
panic(err)
}
if address == "" {
@@ -177,7 +227,7 @@ func (a *App) NewServer(address string) *http.Server {
}
}
cfg := a.config.HTTP
return &http.Server{Addr: address, Handler: a.router,
return &http.Server{Addr: address, Handler: a,
ReadHeaderTimeout: cfg.ReadHeaderTimeout, ReadTimeout: cfg.ReadTimeout,
WriteTimeout: cfg.WriteTimeout, IdleTimeout: cfg.IdleTimeout,
MaxHeaderBytes: cfg.MaxHeaderBytes, ErrorLog: cfg.ErrorLog}
@@ -192,6 +242,8 @@ func (a *App) RunContext(ctx context.Context, address ...string) (runErr error)
if ctx.Err() != nil {
return a.Close()
}
stopWatching := a.watchShutdown(ctx)
defer stopWatching()
serve, err := a.prepareTransport()
if err != nil {
return errors.Join(err, a.Close())
@@ -218,6 +270,11 @@ func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) (ru
if ctx.Err() != nil {
return a.Close()
}
if err := a.ValidateHTTP(); err != nil {
return errors.Join(err, a.Close())
}
stopWatching := a.watchShutdown(ctx)
defer stopWatching()
return errors.Join(runner.Run(ctx, a, a.listenAddress(address...)), a.Close())
}
@@ -251,12 +308,16 @@ func (a *App) serveContext(ctx context.Context, server *http.Server, listener ne
select {
case err := <-serveErr:
// Serve returning only proves the accept loop ended. Cancel any remaining
// connections before the application completion barrier and cleanup.
_ = server.Close()
closeErr := a.Close()
if err == http.ErrServerClosed {
return closeErr
}
return errors.Join(err, closeErr)
case <-ctx.Done():
a.requests.stop()
shutdownTimeout := a.config.ShutdownTimeout
if shutdownTimeout <= 0 {
shutdownTimeout = 10 * time.Second
@@ -265,8 +326,8 @@ func (a *App) serveContext(ctx context.Context, server *http.Server, listener ne
defer cancel()
shutdownErr := server.Shutdown(shutdownCtx)
if shutdownErr != nil {
// Shutdown leaves active connections open on timeout. Close them so
// request contexts are canceled before releasing shared resources.
// Closing sockets requests cancellation; App.Close separately waits
// for the actual handler return before releasing shared resources.
shutdownErr = errors.Join(shutdownErr, server.Close())
}
if err := <-serveErr; err != nil && err != http.ErrServerClosed {

View File

@@ -1,11 +1,68 @@
package app
import (
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/response"
"fmt"
"net/http"
"strings"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/internal/httplimits"
"git.yuntuo.hk/k2safe/go-omni/response"
)
type routeBodyLimit struct {
method, path string
maxBytes int64
}
func (a *App) hardBodyLimit() int64 {
if a.config.HTTP.MaxBodyBytes > 0 {
return a.config.HTTP.MaxBodyBytes
}
return httplimits.MaxBodyBytes
}
func (a *App) defaultBodyLimit() int64 {
limit := a.config.HTTP.DefaultMaxBodyBytes
if limit <= 0 {
limit = httplimits.DefaultBodyBytes
}
if limit > a.hardBodyLimit() {
limit = a.hardBodyLimit()
}
// Invalid configuration is rejected at startup or the first external request.
if limit > httplimits.MaxBodyBytes {
limit = httplimits.MaxBodyBytes
}
return limit
}
// ValidateHTTP checks HTTP settings and registered route body limits without
// opening a listener. Call after registration when hosting App/Engine externally.
// Built-in serving entry points and RunWith perform this check automatically.
func (a *App) ValidateHTTP() error {
if err := a.config.HTTP.Validate(); err != nil {
return err
}
for _, route := range a.bodyLimits {
if route.maxBytes > a.hardBodyLimit() {
return &config.ConfigError{Field: "http.maxBodyBytes", Message: fmt.Sprintf("route %s %s maxBytes=%d exceeds service hard limit=%d; raise http.maxBodyBytes or lower the route limit", route.method, route.path, route.maxBytes, a.hardBodyLimit())}
}
}
return nil
}
func joinRoutePath(prefix, path string) string {
if path == "" {
if prefix == "" {
return "/"
}
return prefix
}
return strings.TrimSuffix(prefix, "/") + "/" + strings.TrimPrefix(path, "/")
}
func (a *App) bodyHandler(maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
checked := httpx.LimitBody(maxBytes)(handler)
if encoder == nil {
@@ -25,8 +82,10 @@ func (a *App) bodyHandler(maxBytes int64, encoder response.Encoder, handler http
// the response policy, including auth failures and parameter errors.
func (a *App) HandleWithBodyLimit(method, path string, maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
a.router.Handle(method, path, a.bodyHandler(maxBytes, encoder, handler, middleware...))
a.bodyLimits = append(a.bodyLimits, routeBodyLimit{method, path, maxBytes})
}
func (g *RouterGroup) HandleWithBodyLimit(method, path string, maxBytes int64, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
combined := append(append([]httpx.Middleware{}, g.middlewares...), middleware...)
g.group.Handle(method, path, g.app.bodyHandler(maxBytes, encoder, handler, combined...))
g.app.bodyLimits = append(g.app.bodyLimits, routeBodyLimit{method, joinRoutePath(g.prefix, path), maxBytes})
}

224
app/bodyPolicy_test.go Normal file
View File

@@ -0,0 +1,224 @@
package app
import (
"bytes"
"context"
"errors"
"io"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
"git.yuntuo.hk/k2safe/go-omni/response"
)
func serveBodyPolicy(t *testing.T, a *App, transport string) (string, *http.Client) {
t.Helper()
cert, key, clientTLS := testutil.CertificateFiles(t)
a.config.Transport = transport
a.config.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- a.ServeContext(ctx, l) }()
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: clientTLS, ForceAttemptHTTP2: transport == config.TransportHTTP2}}
t.Cleanup(func() {
client.CloseIdleConnections()
cancel()
select {
case err := <-done:
if err != nil {
t.Error(err)
}
case <-time.After(5 * time.Second):
t.Error("server did not stop")
}
})
return "https://" + l.Addr().String(), client
}
func consumeBody(c *httpx.Context) error {
_, err := io.Copy(io.Discard, c.Request().Body)
if err != nil {
return err
}
return c.Text(200, "accepted")
}
func TestUploadPolicyAcrossTransports(t *testing.T) {
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
t.Run(transport, func(t *testing.T) {
a := New()
a.POST("/default", consumeBody)
a.HandleWithBodyLimit("POST", "/large", 16<<20, nil, consumeBody)
a.Group("/v1").Group("files").HandleWithBodyLimit("POST", "/large", 16<<20, nil, consumeBody)
base, client := serveBodyPolicy(t, a, transport)
payload := bytes.Repeat([]byte{'x'}, 9<<20)
for _, chunked := range []bool{false, true} {
for _, path := range []string{"/default", "/large", "/v1/files/large"} {
var body io.Reader = bytes.NewReader(payload)
if chunked {
body = struct{ io.Reader }{body}
}
req, _ := http.NewRequest("POST", base+path, body)
res, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
data, err := io.ReadAll(res.Body)
res.Body.Close()
if err != nil {
t.Fatal(err)
}
want := 200
if path == "/default" {
want = 413
}
if res.StatusCode != want || (want == 413 && !strings.Contains(string(data), "BODY_TOO_LARGE")) {
t.Fatalf("%s unknownLength=%v: %d %s", path, chunked, res.StatusCode, data)
}
if transport == config.TransportHTTP2 && res.ProtoMajor != 2 {
t.Fatal("HTTP/2 not negotiated")
}
}
}
})
}
}
func TestConfiguredBodyPolicyBeforeMiddleware(t *testing.T) {
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
t.Run(transport, func(t *testing.T) {
a := New(config.WithHTTPConfig(config.HTTPConfig{DefaultMaxBodyBytes: 32, MaxBodyBytes: 128}))
var calls atomic.Int32
a.Use(func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
data, err := io.ReadAll(c.Request().Body)
if err != nil {
return err
}
c.Request().Body = io.NopCloser(bytes.NewReader(data))
return next(c)
}
})
h := func(c *httpx.Context) error { calls.Add(1); return consumeBody(c) }
a.POST("/default", h)
a.HandleWithBodyLimit("POST", "/override", 64, nil, h)
a.HandleWithBodyLimit("POST", "/hard", 128, nil, h)
base, client := serveBodyPolicy(t, a, transport)
for _, tc := range []struct {
path string
size, status int
}{
{"/default", 32, 200}, {"/default", 33, 413}, {"/override", 64, 200}, {"/override", 65, 413}, {"/hard", 128, 200}, {"/hard", 129, 413},
} {
for _, unknown := range []bool{false, true} {
before := calls.Load()
var body io.Reader = strings.NewReader(strings.Repeat("a", tc.size))
if unknown {
body = struct{ io.Reader }{body}
}
res, err := client.Post(base+tc.path, "application/octet-stream", body)
if err != nil {
t.Fatal(err)
}
data, err := io.ReadAll(res.Body)
res.Body.Close()
if err != nil || res.StatusCode != tc.status {
t.Fatalf("%+v unknown=%v: %d %s %v", tc, unknown, res.StatusCode, data, err)
}
if tc.status == 413 && (calls.Load() != before || !strings.Contains(string(data), "BODY_TOO_LARGE")) {
t.Fatal("oversize reached business or lost error", string(data))
}
}
}
})
}
}
func TestBodyHardLimitConflictBeforeServing(t *testing.T) {
for _, entry := range []string{"RunContext", "ServeContext", "RunWith", "NewServer", "Engine"} {
t.Run(entry, func(t *testing.T) {
a := New(config.WithHTTPConfig(config.HTTPConfig{MaxBodyBytes: 8 << 20}))
a.Group("/v1").Group("files").HandleWithBodyLimit("POST", "/upload/", 16<<20, nil, consumeBody)
closed := 0
a.OnShutdown(func() error { closed++; return nil })
if err := a.ValidateHTTP(); err == nil {
t.Fatal("external preflight accepted conflicting route limit")
}
var err error
switch entry {
case "RunContext":
err = a.RunContext(context.Background(), "bad address")
case "ServeContext":
l, e := net.Listen("tcp", "127.0.0.1:0")
if e != nil {
t.Fatal(e)
}
err = a.ServeContext(context.Background(), l)
if _, e = l.Accept(); !errors.Is(e, net.ErrClosed) {
t.Fatal("listener leaked", e)
}
case "RunWith":
err = a.RunWith(context.Background(), runnerFunc(func(context.Context, http.Handler, string) error { t.Error("runner started"); return nil }))
case "NewServer":
func() {
defer func() {
if p := recover(); p != nil {
err, _ = p.(error)
}
}()
a.NewServer("")
}()
_ = a.Close()
case "Engine":
w := httptest.NewRecorder()
a.Engine().ServeHTTP(w, httptest.NewRequest("POST", "/v1/files/upload/", nil))
if w.Code != 503 {
t.Fatal(w.Code)
}
err = a.validateErr
_ = a.Close()
}
for _, part := range []string{"POST /v1/files/upload/", "16777216", "8388608", "http.maxBodyBytes"} {
if err == nil || !strings.Contains(err.Error(), part) {
t.Fatalf("missing %q: %v", part, err)
}
}
if closed != 1 {
t.Fatal("cleanup count", closed)
}
})
}
}
func TestDefaultBodyLimitRegistrationPaths(t *testing.T) {
a := New(config.WithHTTPConfig(config.HTTPConfig{DefaultMaxBodyBytes: 4, MaxBodyBytes: 16}))
h := func(*httpx.Context) error { t.Error("oversize reached business"); return nil }
encoder, _ := response.New(response.Config{})
a.Handle("POST", "/handle", h)
a.HandleWithResponse("POST", "/response", encoder, h)
a.Any("/any", h)
g := a.Group("/group")
g.POST("/post", h)
g.Any("/any", h)
g.HandleWithResponse("POST", "/response", encoder, h)
for _, path := range []string{"/handle", "/response", "/any", "/group/post", "/group/any", "/group/response"} {
w := httptest.NewRecorder()
r := httptest.NewRequest("POST", path, strings.NewReader("12345"))
a.ServeHTTP(w, r)
if w.Code != 413 {
t.Fatal(path, w.Code, w.Body.String())
}
}
}

187
app/cors_test.go Normal file
View File

@@ -0,0 +1,187 @@
package app_test
import (
"context"
"encoding/json"
"errors"
"io"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync/atomic"
"testing"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
"git.yuntuo.hk/k2safe/go-omni/response"
)
func corsApp() *omni.App {
return omni.New(omni.WithCORS(omni.CORSConfig{Enabled: true, AllowedOrigins: []string{"https://web.example"}, AllowedMethods: []string{"GET", "POST", "PUT"}, AllowedHeaders: []string{"Authorization", "Content-Type"}, ExposedHeaders: []string{"Content-Disposition"}}), omni.WithResponse(response.Config{CodeType: "int32", SuccessCode: 0, ErrorCode: 999, MessageField: "msg", DataMode: "always"}))
}
func TestAppCORSRoutingAndErrors(t *testing.T) {
a := corsApp()
defer a.Close()
var middlewareCalls int
a.Use(func(next omni.Handler) omni.Handler {
return func(c *omni.Context) error { middlewareCalls++; return next(c) }
})
a.GET("/secure", func(c *omni.Context) error { return c.OK("private") }, func(next omni.Handler) omni.Handler {
return func(c *omni.Context) error { return omni.Unauthorized("LOGIN_REQUIRED", "login required") }
})
a.GET("/failure", func(c *omni.Context) error { return omni.Internal(errors.New("private detail")) })
a.GET("/panic", func(c *omni.Context) error { panic("private panic") })
a.Handle("OPTIONS", "/options", func(c *omni.Context) error { return c.Text(200, "ordinary options") })
for name, entry := range map[string]http.Handler{"App": a, "Engine": a.Engine(), "NewServer": a.NewServer("").Handler} {
t.Run(name, func(t *testing.T) {
for _, tc := range []struct {
method, path string
status int
}{{"GET", "/secure", 401}, {"GET", "/failure", 500}, {"GET", "/panic", 500}, {"GET", "/absent", 404}, {"PUT", "/secure", 405}} {
r := httptest.NewRequest(tc.method, tc.path, nil)
r.Header.Set("Origin", "https://web.example")
w := httptest.NewRecorder()
entry.ServeHTTP(w, r)
var body map[string]any
err := json.Unmarshal(w.Body.Bytes(), &body)
if err != nil || w.Code != tc.status || w.Header().Get("Access-Control-Allow-Origin") != "https://web.example" || body["code"] != float64(999) || body["data"] != nil || body["msg"] == nil || strings.Contains(w.Body.String(), "private") {
t.Fatal(w.Code, w.Header(), w.Body.String(), err)
}
}
for _, path := range []string{"/secure", "/absent"} {
before := middlewareCalls
r := httptest.NewRequest("OPTIONS", path, nil)
r.Header.Set("Origin", "https://web.example")
r.Header.Set("Access-Control-Request-Method", "POST")
r.Header.Set("Access-Control-Request-Headers", "authorization,content-type")
w := httptest.NewRecorder()
entry.ServeHTTP(w, r)
if w.Code != 204 || w.Body.Len() != 0 || middlewareCalls != before {
t.Fatal(w.Code, w.Body.String(), middlewareCalls, before)
}
}
r := httptest.NewRequest("OPTIONS", "/options", nil)
w := httptest.NewRecorder()
entry.ServeHTTP(w, r)
if w.Code != 200 || w.Body.String() != "ordinary options" {
t.Fatal(w.Code, w.Body.String())
}
})
}
}
func TestAppCORSConfigIsolation(t *testing.T) {
cfg := config.Default()
cfg.CORS = omni.CORSConfig{Enabled: true, AllowedOrigins: []string{"https://one.example"}, AllowedHeaders: []string{"Authorization"}}
option := omni.WithConfig(cfg)
cfg.CORS.AllowedOrigins[0] = "https://evil.example"
a := omni.New(option)
defer a.Close()
b := omni.New(option)
defer b.Close()
a.Config().CORS.AllowedOrigins[0] = "https://evil.example"
for _, app := range []*omni.App{a, b} {
w := httptest.NewRecorder()
r := httptest.NewRequest("GET", "/absent", nil)
r.Header.Set("Origin", "https://one.example")
app.ServeHTTP(w, r)
if w.Header().Get("Access-Control-Allow-Origin") != "https://one.example" {
t.Fatal(w.Header())
}
}
defer func() {
if recover() == nil {
t.Error("invalid CORS passed direct New")
}
}()
omni.New(omni.WithCORS(omni.CORSConfig{Enabled: true}))
}
func TestCORSTransports(t *testing.T) {
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
t.Run(transport, func(t *testing.T) {
cert, key, tlsClient := testutil.CertificateFiles(t)
cfg := omni.CORSConfig{Enabled: true, AllowedOrigins: []string{"https://web.example"}, AllowedMethods: []string{"GET", "POST"}, AllowedHeaders: []string{"Content-Type"}, ExposedHeaders: []string{"Content-Disposition"}}
a := omni.New(omni.WithCORS(cfg), omni.WithTransport(transport), omni.WithTLS(cert, key))
var called atomic.Int32
a.Use(func(next omni.Handler) omni.Handler {
return func(c *omni.Context) error { called.Add(1); return next(c) }
})
a.POST("/upload", func(c *omni.Context) error {
b, err := io.ReadAll(c.Request().Body)
if err != nil {
return err
}
return c.Text(200, string(b))
})
a.GET("/download", func(c *omni.Context) error {
return c.WriteStream(200, "application/octet-stream", &httpx.Stream{Content: io.NopCloser(strings.NewReader("file bytes")), Name: "example.txt"})
})
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
done := make(chan error, 1)
go func() { done <- a.ServeContext(ctx, listener) }()
defer func() {
cancel()
select {
case err := <-done:
if err != nil {
t.Error(err)
}
case <-time.After(5 * time.Second):
t.Error("server did not close")
listener.Close()
}
}()
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
defer client.CloseIdleConnections()
url := "https://" + listener.Addr().String()
for _, tc := range []struct {
method, path, body string
status int
}{{"OPTIONS", "/upload", "", 204}, {"POST", "/upload", "raw upload", 200}, {"GET", "/download", "file bytes", 200}, {"GET", "/missing", "", 404}} {
var input io.Reader
if tc.method == "POST" {
input = strings.NewReader(tc.body)
}
r, _ := http.NewRequest(tc.method, url+tc.path, input)
r.Header.Set("Origin", "https://web.example")
if tc.method == "OPTIONS" {
r.Header.Set("Access-Control-Request-Method", "POST")
r.Header.Set("Access-Control-Request-Headers", "content-type")
}
res, err := client.Do(r)
if err != nil {
t.Fatal(err)
}
body, err := io.ReadAll(res.Body)
res.Body.Close()
if err != nil {
t.Fatal(err)
}
if res.StatusCode != tc.status || res.Header.Get("Access-Control-Allow-Origin") != "https://web.example" {
t.Fatal(res.StatusCode, res.Header)
}
if tc.status == 200 && string(body) != tc.body {
t.Fatal(string(body))
}
if tc.status == 204 && (len(body) != 0 || called.Load() != 0) {
t.Fatal("preflight reached middleware")
}
if tc.path == "/download" && (!strings.Contains(res.Header.Get("Content-Disposition"), "example.txt") || res.Header.Get("Access-Control-Expose-Headers") != "Content-Disposition") {
t.Fatal(res.Header)
}
if transport == omni.TransportHTTP2 && res.ProtoMajor != 2 {
t.Fatal(res.Proto)
}
}
})
}
}

241
app/i18n_test.go Normal file
View File

@@ -0,0 +1,241 @@
package app_test
import (
"context"
"encoding/json"
"errors"
"net/http"
"net/http/httptest"
"strings"
"sync"
"testing"
omni "git.yuntuo.hk/k2safe/go-omni"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/i18n"
"git.yuntuo.hk/k2safe/go-omni/i18n/errori18n"
"git.yuntuo.hk/k2safe/go-omni/i18n/httpi18n"
"git.yuntuo.hk/k2safe/go-omni/middleware"
"git.yuntuo.hk/k2safe/go-omni/response"
)
func localizedCatalog(namespace string, status int) apierrors.Catalog {
return errori18n.Bind(apierrors.Catalog{DefaultStatus: status, Definitions: map[string]apierrors.Definition{
"SAME": {Code: int64(9223372036854775807), Message: namespace + ":{value}", Params: []apierrors.Parameter{{Name: "value", Type: "string"}}},
"INTERNAL_ERROR": {Status: 500, Code: int64(500), Message: "internal"},
"FIXED": {Status: 401, Code: int64(401), Message: "fixed {literal}"},
}}, namespace)
}
func localizedOptions(t *testing.T, catalogs ...apierrors.Catalog) (*httpi18n.Policy, *errori18n.Localizer) {
t.Helper()
docs := []i18n.Document{}
for _, catalog := range catalogs {
source, err := errori18n.Source(catalog, "en")
if err != nil {
t.Fatal(err)
}
docs = append(docs, source...)
for _, d := range source {
docs = append(docs, i18n.Document{Version: 1, Namespace: d.Namespace, Locale: "zh-Hans", Messages: map[string]i18n.Message{
"SAME": {Value: i18n.Pattern{Text: d.Namespace + "译文:{value}"}}, "FIXED": {Value: i18n.Pattern{Text: "需要认证"}}, "INTERNAL_ERROR": {Value: i18n.Pattern{Text: "系统错误"}},
}})
}
}
b, err := i18n.New(i18n.Config{SourceLocale: "en", DefaultLocale: "en", Locales: []string{"en", "zh-Hans"}}, docs...)
if err != nil {
t.Fatal(err)
}
l := errori18n.New(b, nil)
for _, c := range catalogs {
if err := l.Validate(c); err != nil {
t.Fatal(err)
}
}
p, err := httpi18n.New(b)
if err != nil {
t.Fatal(err)
}
return p, l
}
func TestI18nRouteScopesEarlyFailuresAndReplay(t *testing.T) {
global, local := localizedCatalog("app", 200), localizedCatalog("route", 409)
policy, localizer := localizedOptions(t, global, local)
format := response.Config{CodeType: "int64", SuccessCode: int64(0), ErrorCode: int64(999), Body: map[string]any{"error": map[string]any{"reason": "${message}", "id": "${code}"}, "ok": "${ok}", "data": "${data}"}, DataMode: "always", Messages: map[string]string{"FIXED": "fixed {literal}"}}
app := omni.New(omni.WithErrors(global), omni.WithResponse(format), omni.WithLanguagePolicy(policy), omni.WithErrorLocalizer(localizer))
defer app.Close()
private := errors.New("private-value")
app.Use(func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error {
switch c.Header("X-Early") {
case "auth":
return apierrors.ByCode("FIXED")
case "panic":
panic(private)
}
return next(c)
}
})
handler := func(c *httpx.Context) error {
if c.Header("X-Params") == "missing" {
return apierrors.ByCode("SAME")
}
return apierrors.ByCode("SAME").WithParams(map[string]any{"value": "{literal}"}).WithCause(private)
}
app.GET("/global", handler)
bridge := httpx.WrapMiddleware(func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { next.ServeHTTP(w, r) })
})
app.HandleWithResponse("GET", "/route", response.RawJSON(response.MustWithErrors(local, response.MustNew(format))), handler, bridge)
for _, path := range []string{"/global", "/route"} {
for _, locale := range []string{"en", "zh-CN"} {
for _, mode := range []string{"", "auth", "panic", "missing"} {
r := httptest.NewRequest("GET", path, nil)
r.Header.Set("Accept-Language", locale)
r.Header.Set("X-Early", mode)
r.Header.Set("X-Params", mode)
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
var body struct {
Error struct {
Reason string `json:"reason"`
ID json.Number `json:"id"`
} `json:"error"`
OK bool `json:"ok"`
Data any `json:"data"`
}
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatal(err, w.Body.String())
}
ns, status := "app", 200
if path == "/route" {
ns, status = "route", 409
}
want := ns + ":{literal}"
if locale != "en" {
want = ns + "译文:{literal}"
}
if mode == "auth" {
status = 401
want = "fixed {literal}"
if locale != "en" {
want = "需要认证"
}
} else if mode == "panic" || mode == "missing" {
status = 500
want = "internal"
if locale != "en" {
want = "系统错误"
}
}
if w.Code != status || body.OK || body.Error.Reason != want || strings.Contains(w.Body.String(), "private-value") {
t.Fatal(path, locale, mode, w.Code, w.Body.String())
}
if mode == "" && body.Error.ID.String() != "9223372036854775807" {
t.Fatal("public code changed", body.Error)
}
}
}
}
for _, shared := range []bool{false, true} {
cfg := middleware.IdempotencyConfig{Scope: func(*httpx.Context) (string, error) { return "test", nil }}
if shared {
cfg.Store = &catalogStore{records: map[string]middleware.IdempotencyRecord{}}
}
idem, err := middleware.NewIdempotency(cfg)
if err != nil {
t.Fatal(err)
}
path := "/localReplay"
if shared {
path = "/sharedReplay"
}
calls := 0
app.POST(path, func(c *httpx.Context) error { calls++; return handler(c) }, idem.Middleware())
var first string
for _, locale := range []string{"en", "zh-CN"} {
r := httptest.NewRequest("POST", path, strings.NewReader(`{}`))
r.Header.Set("Idempotency-Key", "same")
r.Header.Set("Accept-Language", locale)
w := httptest.NewRecorder()
app.ServeHTTP(w, r)
if locale == "en" {
first = w.Body.String()
} else if w.Body.String() != first || w.Header().Get("Idempotency-Replayed") != "true" {
t.Fatal("replay changed", w.Body.String(), first)
}
}
if calls != 1 {
t.Fatal("language retriggered business", calls)
}
}
}
func TestI18nExplicitProtocolsAndConcurrentApps(t *testing.T) {
apps := []*omni.App{}
for _, ns := range []string{"one", "two"} {
c := localizedCatalog(ns, 200)
p, l := localizedOptions(t, c)
a := omni.New(omni.WithErrors(c), omni.WithResponse(response.Config{CodeType: "int64", SuccessCode: 0, ErrorCode: 999}), omni.WithLanguagePolicy(p), omni.WithErrorLocalizer(l))
defer a.Close()
a.GET("/", func(*httpx.Context) error { return apierrors.ByCode("FIXED") })
a.HandleWithResponse("GET", "/ack", response.EncoderFunc(func(r response.Result) (response.Payload, error) {
if !r.IsError() || r.Message != "fixed {literal}" || r.PublicMessage() != "需要认证" {
t.Error(r)
}
return response.Text(200, "ACK"), nil
}), func(*httpx.Context) error { return apierrors.ByCode("FIXED") })
a.GET("/direct", func(c *httpx.Context) error { return c.Text(200, "fixed-protocol") })
apps = append(apps, a)
}
var wg sync.WaitGroup
for n := 0; n < 60; n++ {
wg.Add(1)
go func(n int) {
defer wg.Done()
locale, want := "en", "fixed {literal}"
if n%2 == 0 {
locale, want = "zh-CN", "需要认证"
}
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set("Accept-Language", locale)
w := httptest.NewRecorder()
apps[n%len(apps)].ServeHTTP(w, r)
if w.Code != 401 || !strings.Contains(w.Body.String(), want) {
t.Error(w.Code, w.Body.String())
}
}(n)
}
wg.Wait()
for _, path := range []string{"/ack", "/direct"} {
r := httptest.NewRequest("GET", path, nil)
r.Header.Set("Accept-Language", "zh-CN")
w := httptest.NewRecorder()
apps[0].ServeHTTP(w, r)
want := "ACK"
if path == "/direct" {
want = "fixed-protocol"
}
if w.Body.String() != want {
t.Fatal(w.Body.String())
}
}
}
type panickingLocalizer struct{}
func (panickingLocalizer) Localize(context.Context, *apierrors.Registry, *apierrors.APIError) (string, bool) {
panic("presentation failed")
}
func TestI18nAdapterFailurePreservesBusiness(t *testing.T) {
c := localizedCatalog("test", 200)
a := omni.New(omni.WithErrors(c), omni.WithResponse(response.Config{CodeType: "int64", SuccessCode: 0, ErrorCode: 999}), omni.WithErrorLocalizer(panickingLocalizer{}))
defer a.Close()
a.GET("/", func(*httpx.Context) error { return apierrors.ByCode("FIXED") })
w := httptest.NewRecorder()
a.ServeHTTP(w, httptest.NewRequest("GET", "/", nil))
if w.Code != 401 || !strings.Contains(w.Body.String(), "fixed {literal}") {
t.Fatal(w.Code, w.Body.String())
}
}

88
app/requests.go Normal file
View File

@@ -0,0 +1,88 @@
package app
import (
"context"
"errors"
"fmt"
"sync"
"time"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
)
// Admission and completion share a lock: once stopped, no request can enter
// between observing zero active handlers and releasing their dependencies.
type requestDrain struct {
mu sync.Mutex
stopped bool
active map[*requestWork]struct{}
done chan struct{}
}
type requestWork struct{ cancel context.CancelFunc }
func (d *requestDrain) begin(ctx context.Context) (context.Context, *requestWork) {
d.mu.Lock()
defer d.mu.Unlock()
if d.stopped {
return ctx, nil
}
ctx, cancel := context.WithCancel(ctx)
w := &requestWork{cancel: cancel}
if d.active == nil {
d.active = make(map[*requestWork]struct{})
}
d.active[w] = struct{}{}
return ctx, w
}
func (d *requestDrain) end(w *requestWork) {
w.cancel()
d.mu.Lock()
defer d.mu.Unlock()
delete(d.active, w)
if d.stopped && len(d.active) == 0 {
close(d.done)
}
}
func (d *requestDrain) stop() {
d.mu.Lock()
defer d.mu.Unlock()
if d.stopped {
return
}
d.stopped = true
if len(d.active) == 0 {
close(d.done)
}
}
func (d *requestDrain) cancelAndWait(timeout time.Duration) error {
d.stop()
d.mu.Lock()
for w := range d.active {
w.cancel()
}
d.mu.Unlock()
timer := time.NewTimer(timeout)
defer timer.Stop()
select {
case <-d.done:
return nil
case <-timer.C:
select {
case <-d.done:
return nil
default:
}
return fmt.Errorf("go-omni: HTTP handlers did not finish within cancellation grace %s: %w", timeout,
errors.Join(context.DeadlineExceeded, lifecycle.ErrResourcesRetained))
}
}
// watchShutdown stops admission on the early signal without canceling accepted
// work. The transport retains its graceful drain budget before Close cancels.
func (a *App) watchShutdown(ctx context.Context) func() bool {
return context.AfterFunc(ctx, a.requests.stop)
}

167
app/requests_test.go Normal file
View File

@@ -0,0 +1,167 @@
package app
import (
"context"
"errors"
"io"
"net"
"net/http"
"net/http/httptest"
"sync"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
)
func TestShutdownWaitsForCancellationCleanup(t *testing.T) {
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
t.Run(transport, func(t *testing.T) {
cert, key, tlsClient := testutil.CertificateFiles(t)
a := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(30*time.Millisecond), config.WithHTTPConfig(config.HTTPConfig{ShutdownCancelTimeout: time.Second}))
started, canceled, release := make(chan struct{}), make(chan struct{}), make(chan struct{})
var once sync.Once
unblock := func() { once.Do(func() { close(release) }) }
defer unblock()
var closed atomic.Int32
a.OnShutdown(func() error { closed.Add(1); return nil })
a.GET("/work", func(c *httpx.Context) error {
close(started)
<-c.Context().Done()
close(canceled)
<-release
if closed.Load() != 0 {
t.Error("dependency closed during cancellation cleanup")
}
return nil
})
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
done := make(chan error, 1)
go func() { done <- a.ServeContext(ctx, l) }()
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
defer client.CloseIdleConnections()
requestDone := make(chan struct{})
go func() {
defer close(requestDone)
res, err := client.Get("https://" + l.Addr().String() + "/work")
if err == nil {
io.Copy(io.Discard, res.Body)
res.Body.Close()
}
}()
await(t, started)
cancel()
await(t, canceled)
if closed.Load() != 0 {
t.Fatal("dependency closed on cancellation")
}
select {
case err := <-done:
t.Fatal("returned before handler cleanup", err)
default:
}
// Even an in-memory invocation through Engine must now be rejected.
w := httptest.NewRecorder()
a.Engine().ServeHTTP(w, httptest.NewRequest("GET", "/work", nil))
if w.Code != 503 {
t.Fatal(w.Code)
}
unblock()
select {
case err := <-done:
if !errors.Is(err, context.DeadlineExceeded) || errors.Is(err, lifecycle.ErrResourcesRetained) {
t.Fatal(err)
}
case <-time.After(3 * time.Second):
t.Fatal("shutdown stuck")
}
await(t, requestDone)
if closed.Load() != 1 {
t.Fatal("cleanup count", closed.Load())
}
})
}
}
func TestExternalServingEntriesRetainActiveResources(t *testing.T) {
for _, entry := range []string{"App", "Engine", "NewServer", "RunWith"} {
t.Run(entry, func(t *testing.T) {
a := New(config.WithHTTPConfig(config.HTTPConfig{ShutdownCancelTimeout: 30 * time.Millisecond}))
started, release, returned := make(chan struct{}), make(chan struct{}), make(chan struct{})
var closed atomic.Int32
a.OnShutdown(func() error { closed.Add(1); return nil })
a.GET("/", func(*httpx.Context) error {
close(started)
<-release
if closed.Load() != 0 {
t.Error("closed while active")
}
return nil
})
var h http.Handler = a
if entry == "Engine" {
h = a.Engine()
}
if entry == "NewServer" {
h = a.NewServer("").Handler
}
invoke := func(h http.Handler) {
defer close(returned)
h.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", "/", nil))
}
defer func() { close(release); await(t, returned) }()
var err error
if entry == "RunWith" {
err = a.RunWith(context.Background(), runnerFunc(func(_ context.Context, h http.Handler, _ string) error { go invoke(h); await(t, started); return nil }))
} else {
go invoke(h)
await(t, started)
err = a.Close()
}
if !errors.Is(err, lifecycle.ErrResourcesRetained) || closed.Load() != 0 {
t.Fatal("retention failed", err, closed.Load())
}
if err = a.Close(); !errors.Is(err, lifecycle.ErrResourcesRetained) {
t.Fatal("repeat close lost retention", err)
}
})
}
}
func TestRequestAdmissionAndCloseRace(t *testing.T) {
for attempt := 0; attempt < 20; attempt++ {
a := New()
var closed atomic.Bool
a.OnShutdown(func() error { closed.Store(true); return nil })
a.GET("/", func(c *httpx.Context) error {
if closed.Load() {
t.Error("handler admitted after resource release")
}
return nil
})
start := make(chan struct{})
var wg sync.WaitGroup
for i := 0; i < 20; i++ {
wg.Add(1)
go func() {
defer wg.Done()
<-start
a.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", "/", nil))
}()
}
close(start)
if err := a.Close(); err != nil {
t.Fatal(err)
}
wg.Wait()
}
}

View File

@@ -10,7 +10,7 @@ import (
)
func (a *App) handle(routes httpx.RouterGroup, method, path string, handler httpx.Handler, middleware []httpx.Middleware) {
routes.Handle(method, path, a.adapt(handler, middleware...))
routes.Handle(method, path, a.bodyHandler(a.defaultBodyLimit(), nil, handler, middleware...))
}
func (a *App) adapt(handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
@@ -28,6 +28,9 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid
}
}
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
if a.config.LanguagePolicy != nil {
request = a.config.LanguagePolicy.Apply(writer, request)
}
request = request.WithContext(logx.WithLogger(contract.WithRequestPolicy(request.Context(), a.config.Request), a.config.LogInstance))
if len(a.config.TrustedProxies) > 0 {
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
@@ -40,6 +43,7 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid
requestContext.SetErrorRegistry(provider.ErrorRegistry())
}
requestContext.SetResponseEncoder(encoder)
requestContext.SetErrorLocalizer(a.config.ErrorLocalizer)
defer func() { httpx.CleanupMultipart(requestContext.Request()) }()
if err := wrapped(requestContext); err != nil {
a.writeError(requestContext, err)
@@ -58,7 +62,7 @@ func (a *App) HandleWithResponse(method, path string, encoder response.Encoder,
if encoder == nil {
encoder = a.responseEncoder
}
a.router.Handle(method, path, a.adaptResponse(encoder, handler, middleware...))
a.router.Handle(method, path, a.bodyHandler(a.defaultBodyLimit(), encoder, handler, middleware...))
}
func (a *App) GET(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
@@ -82,18 +86,19 @@ func (a *App) DELETE(path string, handler httpx.Handler, middleware ...httpx.Mid
}
func (a *App) Any(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
a.router.Any(path, a.adapt(handler, middleware...))
a.router.Any(path, a.bodyHandler(a.defaultBodyLimit(), nil, handler, middleware...))
}
// RouterGroup scopes a route prefix and optional middleware.
type RouterGroup struct {
prefix string
group httpx.RouterGroup
app *App
middlewares []httpx.Middleware
}
func (a *App) Group(path string, middleware ...httpx.Middleware) *RouterGroup {
return &RouterGroup{group: a.router.Group(path), app: a, middlewares: middleware}
return &RouterGroup{group: a.router.Group(path), app: a, middlewares: middleware, prefix: joinRoutePath("", path)}
}
func (g *RouterGroup) Use(middleware ...httpx.Middleware) *RouterGroup {
@@ -104,7 +109,7 @@ func (g *RouterGroup) Use(middleware ...httpx.Middleware) *RouterGroup {
func (g *RouterGroup) Group(path string, middleware ...httpx.Middleware) *RouterGroup {
combined := append([]httpx.Middleware{}, g.middlewares...)
combined = append(combined, middleware...)
return &RouterGroup{group: g.group.Group(path), app: g.app, middlewares: combined}
return &RouterGroup{group: g.group.Group(path), app: g.app, middlewares: combined, prefix: joinRoutePath(g.prefix, path)}
}
func (g *RouterGroup) route(method, path string, handler httpx.Handler, middleware []httpx.Middleware) {
@@ -124,7 +129,7 @@ func (g *RouterGroup) HandleWithResponse(method, path string, encoder response.E
}
combined := append([]httpx.Middleware{}, g.middlewares...)
combined = append(combined, middleware...)
g.group.Handle(method, path, g.app.adaptResponse(encoder, handler, combined...))
g.group.Handle(method, path, g.app.bodyHandler(g.app.defaultBodyLimit(), encoder, handler, combined...))
}
func (g *RouterGroup) GET(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
@@ -150,5 +155,5 @@ func (g *RouterGroup) DELETE(path string, handler httpx.Handler, middleware ...h
func (g *RouterGroup) Any(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
combined := append([]httpx.Middleware{}, g.middlewares...)
combined = append(combined, middleware...)
g.group.Any(path, g.app.adapt(handler, combined...))
g.group.Any(path, g.app.bodyHandler(g.app.defaultBodyLimit(), nil, handler, combined...))
}

View File

@@ -10,13 +10,14 @@ import (
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/internal/httplimits"
h2transport "git.yuntuo.hk/k2safe/go-omni/transport/http2"
"git.yuntuo.hk/k2safe/go-omni/transport/native"
)
// ServeContext serves a caller-created listener using the configured transport
// and TLS. App takes ownership of the listener, including on startup failure,
// and runs resource cleanup after the server has stopped.
// and runs resource cleanup only after admitted handlers have returned.
func (a *App) ServeContext(ctx context.Context, listener net.Listener) (runErr error) {
defer func() { runErr = shutdownResult(ctx, runErr) }()
if listener == nil {
@@ -30,6 +31,8 @@ func (a *App) ServeContext(ctx context.Context, listener net.Listener) (runErr e
_ = listener.Close()
return a.Close()
}
stopWatching := a.watchShutdown(ctx)
defer stopWatching()
serve, err := a.prepareTransport()
if err != nil {
_ = listener.Close()
@@ -39,6 +42,9 @@ func (a *App) ServeContext(ctx context.Context, listener net.Listener) (runErr e
}
func (a *App) prepareTransport() (func(context.Context, net.Listener) error, error) {
if err := a.ValidateHTTP(); err != nil {
return nil, err
}
if err := a.config.ValidateTransport(); err != nil {
return nil, err
}
@@ -57,6 +63,9 @@ func (a *App) prepareTransport() (func(context.Context, net.Listener) error, err
switch a.config.Transport {
case config.TransportNative:
server := native.New(a)
// App enforces the service/route policy after routing. Keep only the
// supported framing ceiling here, never a hidden smaller transport cap.
server.MaxBodyBytes = httplimits.MaxBodyBytes
server.ShutdownTimeout = shutdownTimeout
cfg := a.config.HTTP
if cfg.ReadHeaderTimeout > 0 {
@@ -82,6 +91,7 @@ func (a *App) prepareTransport() (func(context.Context, net.Listener) error, err
return server.Serve, nil
case config.TransportHTTP2:
server := h2transport.New(a)
server.MaxBodyBytes = httplimits.MaxBodyBytes
server.ShutdownTimeout = shutdownTimeout
if a.config.HTTP.IdleTimeout > 0 {
server.IdleTimeout = a.config.HTTP.IdleTimeout

View File

@@ -15,6 +15,7 @@ import (
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
)
func TestConfiguredTransportLifecycle(t *testing.T) {
@@ -269,7 +270,9 @@ func TestConfiguredShutdownDoesNotWaitForUncooperativeHandler(t *testing.T) {
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
t.Run(transport, func(t *testing.T) {
cert, key, tlsClient := testutil.CertificateFiles(t)
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(80*time.Millisecond))
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(80*time.Millisecond), config.WithHTTPConfig(config.HTTPConfig{ShutdownCancelTimeout: 40 * time.Millisecond}))
var cleaned atomic.Int32
application.OnShutdown(func() error { cleaned.Add(1); return nil })
started := make(chan context.Context, 1)
release, finished := make(chan struct{}), make(chan struct{})
application.GET("/blocked", func(ctx *httpx.Context) error {
@@ -304,11 +307,17 @@ func TestConfiguredShutdownDoesNotWaitForUncooperativeHandler(t *testing.T) {
close(release)
t.Fatal("handler did not start")
}
defer func() { close(release); await(t, finished) }()
defer func() {
close(release)
await(t, finished)
if err := application.Close(); !errors.Is(err, lifecycle.ErrResourcesRetained) || cleaned.Load() != 0 {
t.Fatalf("late handler return must not trigger cleanup: %v count=%d", err, cleaned.Load())
}
}()
cancel()
select {
case err := <-done:
if !errors.Is(err, context.DeadlineExceeded) {
if !errors.Is(err, context.DeadlineExceeded) || !errors.Is(err, lifecycle.ErrResourcesRetained) || cleaned.Load() != 0 {
t.Fatalf("shutdown: %v", err)
}
case <-time.After(2 * time.Second):

View File

@@ -1,5 +1,11 @@
# go-omni CLI
v0.28.1 的 `workspace generate/check` 支持 API 与通用工程混合处理。每个 services 项可指定 `kind: api | project`,省略时按工程清单识别,旧 API 配置继续有效。format 只处理 API 契约,通用工程显示 SKIP;check 只读并汇总各工程失败。见 [可运行示例](../../examples/mixedWorkspace/README.md)。
v0.28 新增 `project new/generate/check`:生成无 API、无监听端口和无外部服务依赖的通用 Go 程序,配置/日志/生命周期自动装配,项目维护 svc 与 Run。`project generate --dry-run` 预览,`project check` 只读检查;详见 [通用工程指南](../../PROGRAM.md)。
v0.26 新增 `i18n check|export|types|extract`:检查独立语言资源、提取 Go 静态消息引用或已导出的错误目录,导出指定命名空间的前端包和 TypeScript 类型。输入只读,结果写 stdout;动态引用明确报告。完整命令见 [国际化工具链](../../i18n/README.md#cli-提取检查与导出)。`.api errorCatalog` 可声明 translation 引用,旧 runtime 在生成前拒绝,接入时使用当前同版 CLI 与框架。
v0.19 移除结构映射生成:`omni.gen.json` 只保留 Repository 和中间件选项。普通接口直接使用生成 DTO;需要类型转换时用普通 Go 或可选 [copyx](../../COPY.md)。旧 `mappings` / `mappingFiles` 必须按迁移步骤清理,CLI 不会静默删除仍在调用的旧生成文件。
公共字段组合、中间件工厂和按需 Repository 继续支持,见 [生成能力指南](../../GENERATION.md)。
@@ -8,13 +14,13 @@ v0.16 增加基础类型/动态 JSON 数组返回、OpenAPI 公共引用和 work
v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目录>|--apply]`,以及 `workspace format|generate|check --file omni.workspace.json`。详见 [通用开发指南](../../DEVELOPMENT.md)。
新项目采用根目录 `main.go` 与 `internal/config、svc、logic、handler、types`。完整职责、启动扩展、流式接口和生成文件边界见 [项目指南](../../PROJECT.md)。
API 新项目采用根目录 `main.go` 与 `internal/config、svc、logic、handler、types`。完整职责、启动扩展、流式接口和生成文件边界见 [项目指南](../../PROJECT.md)。
## 创建项目
## 创建 API 项目
```sh
# 仓库读取权限与 PATH 配置见根目录 README
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.22.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.28.1-preview
go-omni new --module example.com/hello ./hello
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
@@ -49,14 +55,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
## 版本与迁移
CLI 与框架依赖固定使用 v0.22.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
CLI 与框架依赖固定使用 v0.28.1-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
```sh
# 在 go-omni 源码仓库中执行
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
```
当前模板与依赖以 v0.22.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
当前模板与依赖以 v0.28.1-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
## 数据库迁移命令

View File

@@ -55,6 +55,7 @@ func prepareMetadata(original *Spec) (*Spec, error) {
for i := range spec.Errors {
spec.Errors[i].BusinessCode = normalized.Definitions[spec.Errors[i].Code].Code
spec.Errors[i].Params = normalized.Definitions[spec.Errors[i].Code].Params
spec.Errors[i].Translation = normalized.Definitions[spec.Errors[i].Code].Translation
}
if err := validateErrorPolicies(&spec); err != nil {
return nil, err

View File

@@ -0,0 +1,59 @@
package apigen
import (
"encoding/json"
"os"
"os/exec"
"path/filepath"
"testing"
)
func TestGeneratedCORSConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "app")
api := filepath.Join(t.TempDir(), "service.api")
writeFile(t, api, "service corsdemo {\n@handler ping\nget /ping\n}\n")
o := Options{API: api, Dir: dir, Module: "example.com/corsdemo", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/cors/consumer_test.go"))
writeFile(t, filepath.Join(dir, "etc/config.yaml"), "server:\n log:\n output: discard\n cors:\n enabled: true\n allowedOrigins: [https://web.example]\n allowedMethods: [GET, POST, PUT]\n allowedHeaders: [Authorization, Content-Type]\n maxAge: 10m\n")
runConsumerGo(t, dir, "mod", "tidy")
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
runConsumerGo(t, dir, "build", ".")
o.API = filepath.Join(dir, "api/service.api")
before := snapshot(t, dir)
for _, mode := range []string{"dry", "generate", "check"} {
o.DryRun = mode == "dry"
o.Check = mode == "check"
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("regeneration changed project files")
}
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
cmd := exec.Command("go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
out, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(out))
}
var dependency struct {
Version, Sum string
Replace any
}
if err := json.Unmarshal(out, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a fixed remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
}

View File

@@ -11,14 +11,15 @@ import (
// ErrorDefinition declares a public, stable error. Generated constructors return
// a fresh error each time so WithCause cannot mutate a shared singleton.
type ErrorDefinition struct {
Name string `json:"name"`
BusinessCode any `json:"businessCode,omitempty"`
Params []apierrors.Parameter `json:"params,omitempty"`
Catalog string `json:"catalog,omitempty"`
Status int `json:"status"`
Code string `json:"code"`
Message string `json:"message"`
Position string `json:"-"`
Name string `json:"name"`
BusinessCode any `json:"businessCode,omitempty"`
Params []apierrors.Parameter `json:"params,omitempty"`
Translation *apierrors.Translation `json:"translation,omitempty"`
Catalog string `json:"catalog,omitempty"`
Status int `json:"status"`
Code string `json:"code"`
Message string `json:"message"`
Position string `json:"-"`
}
type AuthDefinition struct {
Name, Type, In, KeyName, Middleware, Description, BearerFormat, Position string

View File

@@ -49,11 +49,12 @@ func (p *parser) parseErrorCatalog() error {
}
}
type entry struct {
Key string `json:"key"`
Code any `json:"code"`
Message string `json:"message"`
Status int `json:"status"`
Params []apierrors.Parameter `json:"params"`
Key string `json:"key"`
Code any `json:"code"`
Message string `json:"message"`
Status int `json:"status"`
Params []apierrors.Parameter `json:"params"`
Translation *apierrors.Translation `json:"translation"`
}
var entries map[string]entry
if err := apierrors.DecodeCatalogJSON([]byte(props["errors"]), &entries); err != nil {
@@ -75,7 +76,7 @@ func (p *parser) parseErrorCatalog() error {
if e.Status == 0 {
e.Status = defaultStatus
}
p.spec.Errors = append(p.spec.Errors, ErrorDefinition{Name: fn, Status: e.Status, Code: e.Key, Message: e.Message, BusinessCode: e.Code, Params: e.Params, Catalog: name, Position: pos})
p.spec.Errors = append(p.spec.Errors, ErrorDefinition{Name: fn, Status: e.Status, Code: e.Key, Message: e.Message, BusinessCode: e.Code, Params: e.Params, Translation: e.Translation, Catalog: name, Position: pos})
}
return nil
}
@@ -90,7 +91,7 @@ func errorCatalog(spec *Spec, names []string) apierrors.Catalog {
if names != nil && !selected[e.Name] {
continue
}
c.Definitions[e.Code] = apierrors.Definition{Status: e.Status, Code: e.BusinessCode, Message: e.Message, Params: e.Params}
c.Definitions[e.Code] = apierrors.Definition{Status: e.Status, Code: e.BusinessCode, Message: e.Message, Params: e.Params, Translation: e.Translation}
}
return c
}

View File

@@ -12,6 +12,7 @@ import (
"strconv"
"strings"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/internal/genfs"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
"golang.org/x/mod/modfile"
"golang.org/x/mod/module"
@@ -22,7 +23,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Runtime-only releases need not raise the managed template compatibility floor.
const minimumVersion = "v0.22.0-preview"
const currentVersion = "v0.22.0-preview"
const currentVersion = "v0.28.1-preview"
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
type Options struct {
@@ -194,6 +195,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
}
seeds["go.mod"] = data
}
capabilityModule := moduleData
if seeded, ok := seeds["go.mod"]; ok {
capabilityModule = seeded
}
if err := checkI18nRuntime(c.Spec, capabilityModule, directory); err != nil {
return Output{}, err
}
if err = module.CheckImportPath(o.Module); err != nil {
return Output{}, err
}
@@ -338,25 +346,7 @@ func moduleFile(o Options) ([]byte, error) {
}
return []byte(text), nil
}
func canonical(path string) string {
path, _ = filepath.Abs(path)
candidate := path
var tail []string
for {
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
for i := len(tail) - 1; i >= 0; i-- {
resolved = filepath.Join(resolved, tail[i])
}
return resolved
}
parent := filepath.Dir(candidate)
if parent == candidate {
return path
}
tail = append(tail, filepath.Base(candidate))
candidate = parent
}
}
func canonical(path string) string { return genfs.Canonical(path) }
func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map[string][]byte, error) {
generated, seeds := map[string][]byte{}, map[string][]byte{}

View File

@@ -0,0 +1,57 @@
package apigen
import (
"fmt"
"os"
"path/filepath"
"strings"
"golang.org/x/mod/modfile"
"golang.org/x/mod/semver"
)
// Translation metadata was introduced in this release. Consumers of older
// runtimes must upgrade or use an explicitly supported local replacement.
const i18nMinimumVersion = "v0.26.0-preview"
func checkI18nRuntime(spec *Spec, moduleData []byte, directory string) error {
needed := false
for _, e := range spec.Errors {
needed = needed || e.Translation != nil
}
if !needed {
return nil
}
mf, err := modfile.Parse("go.mod", moduleData, nil)
if err != nil {
return err
}
version := ""
for _, r := range mf.Require {
if r.Mod.Path == frameworkModule {
version = r.Mod.Version
}
}
for _, r := range mf.Replace {
if r.Old.Path != frameworkModule || r.Old.Version != "" && r.Old.Version != version {
continue
}
if r.New.Version != "" {
version = r.New.Version
continue
}
root := r.New.Path
if !filepath.IsAbs(root) {
root = filepath.Join(directory, root)
}
data, e := os.ReadFile(filepath.Join(root, "errors", "localization.go"))
if e == nil && strings.Contains(string(data), "type Translation struct") {
return nil
}
return fmt.Errorf("translation metadata requires a local framework with errors.Translation support")
}
if semver.Compare(version, i18nMinimumVersion) < 0 {
return fmt.Errorf("translation metadata requires %s or newer; use --framework-dir for unpublished source", i18nMinimumVersion)
}
return nil
}

View File

@@ -0,0 +1,42 @@
package apigen
import (
"path/filepath"
"strings"
"testing"
)
func TestI18nTranslationMetadataAndVersionBoundary(t *testing.T) {
spec, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/i18n.api"))
if err != nil {
t.Fatal(err)
}
compiled, err := Compile(spec)
if err != nil {
t.Fatal(err)
}
ref := compiled.Spec.Errors[0].Translation
if ref == nil || ref.Namespace != "limits" || ref.Key != "exceeded" {
t.Fatal(ref)
}
if err := checkI18nRuntime(spec, []byte("module example.com/test\ngo 1.22\nrequire "+frameworkModule+" v0.25.0-preview\n"), t.TempDir()); err == nil {
t.Fatal("old runtime accepted")
}
generated, _, err := renderProject(Options{Module: "example.com/test"}, compiled, false)
if err != nil {
t.Fatal(err)
}
if !strings.Contains(string(generated["internal/apierrors/errorsGen.go"]), "translation") || !strings.Contains(string(generated["docs/openapi.json"]), "translation") {
t.Fatal("translation metadata lost")
}
bad := readFile(t, filepath.Join(frameworkRoot(t), "examples/contracts/i18n.api"))
bad = strings.Replace(bad, `"namespace":"limits"`, `"namespace":""`, 1)
parsed, err := parseText(t, bad)
if err == nil {
_, err = Compile(parsed)
}
if err == nil {
t.Fatal("invalid translation reference accepted")
}
}
func TestI18nGeneratedConsumer(t *testing.T) { testCatalogConsumer(t, "i18n.api", "i18n") }

View File

@@ -30,12 +30,12 @@ func main(){
func run(ctx context.Context,path string)(runErr error){
cfg,err:=config.Load(path);if err!=nil{return err}
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
// Unjoined background work may still log. The owning process exits on this
// Unjoined HTTP or background work may still log. The owning process exits on this
// error; do not close its borrowed logger before that work has finished.
defer func(){if !errors.Is(runErr,lifecycle.ErrResourcesRetained){runErr=errors.Join(runErr,logger.Close())}}()
deps,err:=svc.New(ctx,logger,cfg);if err!=nil{return err}
// RunContext drains HTTP requests before returning. This scope owns dependencies
// and closes them exactly once, before closing the borrowed logger.
// Finish propagates incomplete HTTP draining: stop background work but retain
// shared dependencies when any handler is still running. Otherwise close once.
defer func(){runErr=deps.Finish(runErr)}()
app,err:=handler.NewApp(deps);if err!=nil{return err}
defer func(){runErr=errors.Join(runErr,app.Close())}()
@@ -60,7 +60,12 @@ Unimplemented/nil factories fail startup and clean registered dependencies.
Existing projects retain middleware: handlers until explicitly migrated.
Ordinary Logic uses the generated Request/Response types directly. Where an
independent business type is needed, use ordinary Go or the optional copyx package.
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use explicit ${NAME} / ${NAME:-fallback} references; fixed values are never overridden by inferred environment names. Use -check-config to validate without opening dependencies. See the framework CONFIG.md.
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use explicit ${NAME} / ${NAME:-fallback} references; fixed values are never overridden by inferred environment names. Use -check-config to validate without opening dependencies. Framework validation
identifies fields such as server.log.level and database.port. server.log.timeZone
optionally selects a per-logger IANA zone. Optional server.cors is loaded and installed
automatically before routing/authentication; disabled by default. Configure allowedOrigins,
allowedMethods and allowedHeaders in the selected file; no main/handler edits.
See the framework CONFIG.md and cors/README.md.
There are no mappings/mappingFiles settings or generated mapping files in v0.19.
See COPY.md in the framework repository for copy policies and migration steps,

View File

@@ -91,3 +91,23 @@ func TestConfigurationCheckProcess(t *testing.T) {
t.Fatal(err, string(out))
}
}
func TestGeneratedModuleFieldDiagnostic(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.yaml")
t.Setenv("EXAMPLE_LOG_LEVEL", "invalid-private-level")
if err := os.WriteFile(path, []byte("server:\n log:\n level: '${EXAMPLE_LOG_LEVEL}'\n"), 0600); err != nil {
t.Fatal(err)
}
err := run(context.Background(), path)
var d *omniConfig.Error
if !errors.As(err, &d) || d.Field != "server.log.level" || d.Line != 3 || d.Env != "EXAMPLE_LOG_LEVEL" || !strings.Contains(d.Reason, "debug") || strings.Contains(err.Error(), "invalid-private-level") {
t.Fatal(err)
}
if err := os.WriteFile(path, []byte("server:\n log:\n timeZone: Asia/Shanghai\n"), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.Load(path)
if err != nil || cfg.Server.Log.TimeZone != "Asia/Shanghai" {
t.Fatal(err)
}
}

View File

@@ -0,0 +1,69 @@
package main
import (
"context"
"errors"
"net/http/httptest"
"os"
"path/filepath"
"strings"
"testing"
"example.com/corsdemo/internal/config"
"example.com/corsdemo/internal/handler"
"example.com/corsdemo/internal/svc"
omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
func TestGeneratedCORSConfiguration(t *testing.T) {
cfg, err := config.Load("etc/config.yaml")
if err != nil {
t.Fatal(err)
}
logger, err := logx.New(cfg.Server.Log)
if err != nil {
t.Fatal(err)
}
defer logger.Close()
deps, err := svc.New(context.Background(), logger, cfg)
if err != nil {
t.Fatal(err)
}
defer deps.Close()
a, err := handler.NewApp(deps)
if err != nil {
t.Fatal(err)
}
defer a.Close()
for _, tc := range []struct {
method, path string
status int
}{{"GET", "/ping", 501}, {"GET", "/missing", 404}, {"PUT", "/ping", 405}, {"GET", "/health/live", 200}, {"GET", "/health/ready", 200}, {"GET", "/openapi.json", 200}, {"OPTIONS", "/ping", 204}} {
r := httptest.NewRequest(tc.method, tc.path, nil)
r.Header.Set("Origin", "https://web.example")
if tc.method == "OPTIONS" {
r.Header.Set("Access-Control-Request-Method", "GET")
r.Header.Set("Access-Control-Request-Headers", "authorization")
}
w := httptest.NewRecorder()
a.ServeHTTP(w, r)
if w.Code != tc.status || w.Header().Get("Access-Control-Allow-Origin") != "https://web.example" {
t.Fatal(tc, w.Code, w.Header(), w.Body.String())
}
if tc.status == 204 && (w.Body.Len() != 0 || w.Header().Get("Access-Control-Max-Age") != "600") {
t.Fatal(w)
}
}
// Startup config validation uses the normal generated run/Load, before log
// files or dependencies. No changes to generated main/HTTP composition.
path := filepath.Join(t.TempDir(), "invalid.yaml")
if err := os.WriteFile(path, []byte("server:\n cors:\n enabled: true\n allowedOrigins: ['*']\n allowCredentials: true\n"), 0600); err != nil {
t.Fatal(err)
}
err = run(context.Background(), path)
var d *omniConfig.Error
if !errors.As(err, &d) || d.Field != "server.cors.allowedOrigins[0]" || !strings.Contains(d.Reason, "allowCredentials") {
t.Fatal(err)
}
}

View File

@@ -0,0 +1,70 @@
package main_test
import (
"context"
"encoding/json"
"io"
"net/http/httptest"
"testing"
"example.com/errorCatalog/internal/apierrors"
"example.com/errorCatalog/internal/handler"
"example.com/errorCatalog/internal/svc"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/i18n"
"git.yuntuo.hk/k2safe/go-omni/i18n/errori18n"
"git.yuntuo.hk/k2safe/go-omni/i18n/httpi18n"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
func TestGeneratedTranslationAndParameterCompatibility(t *testing.T) {
catalog := apierrors.Catalog()
source, err := errori18n.Source(catalog, "en")
if err != nil {
t.Fatal(err)
}
source = append(source, i18n.Document{Version: 1, Locale: "zh-Hans", Namespace: "limits", Messages: map[string]i18n.Message{"exceeded": {Value: i18n.Pattern{Text: "上限:{max}"}}}})
b, err := i18n.New(i18n.Config{SourceLocale: "en", DefaultLocale: "en", Locales: []string{"en", "zh-Hans"}}, source...)
if err != nil {
t.Fatal(err)
}
p, err := httpi18n.New(b)
if err != nil {
t.Fatal(err)
}
lc := logx.Default()
lc.Writer = io.Discard
logger, err := logx.New(lc)
if err != nil {
t.Fatal(err)
}
defer logger.Close()
deps, err := svc.New(context.Background(), logger)
if err != nil {
t.Fatal(err)
}
defer deps.Close()
a := omni.New(omni.WithLanguagePolicy(p), omni.WithErrorLocalizer(errori18n.New(b, nil)))
defer a.Close()
a.Use(func(next httpx.Handler) httpx.Handler {
return func(c *httpx.Context) error { return apierrors.LimitExceeded(9223372036854775807) }
})
if err := handler.RegisterRoutes(a, deps); err != nil {
t.Fatal(err)
}
r := httptest.NewRequest("GET", "/value", nil)
r.Header.Set("Accept-Language", "zh-CN")
w := httptest.NewRecorder()
a.ServeHTTP(w, r)
var body map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
if w.Code != 200 || body["detail"] != "上限:9223372036854775807" || body["code"] != float64(400001) || body["ok"] != false {
t.Fatal(w.Code, body)
}
if apierrors.LimitExceeded(5).Message != "Limit: 5" {
t.Fatal("non-HTTP constructor changed")
}
}

View File

@@ -7,6 +7,7 @@ import (
)
var Opened, Closed, DownloadClosed atomic.Int32
var SlowReturned atomic.Int32
var Started = make(chan struct{}, 1)
var Release = make(chan struct{})
var Current atomic.Pointer[ServiceContext]

View File

@@ -71,6 +71,7 @@ type countedReader struct{ io.Reader }
func (c *countedReader) Close() error { svc.DownloadClosed.Add(1); return nil }
func (l *Logic) Slow(context.Context) (*types.Receipt, error) {
defer svc.SlowReturned.Add(1)
before := svc.Closed.Load()
svc.Started <- struct{}{}
<-svc.Release

View File

@@ -19,6 +19,7 @@ import (
"example.com/managed/internal/handler"
"example.com/managed/internal/svc"
"git.yuntuo.hk/k2safe/go-omni/health"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
@@ -229,3 +230,80 @@ func TestGeneratedEndpointConflictCleansUp(t *testing.T) {
t.Fatal("collision leaked resources or started serving", err)
}
}
func TestGeneratedIncompleteHTTPDrainRetainsDependencies(t *testing.T) {
for _, transport := range []string{"nethttp", "native"} {
t.Run(transport, func(t *testing.T) {
l, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
address := l.Addr().String()
l.Close()
path := filepath.Join(t.TempDir(), "config.json")
logFile := filepath.Join(t.TempDir(), "service.log")
data, _ := json.Marshal(map[string]any{"server": map[string]any{"address": address, "transport": transport, "shutdownTimeout": "30ms", "http": map[string]string{"shutdownCancelTimeout": "30ms"}, "log": map[string]any{"file": logFile, "rotation": map[string]any{"enabled": true}}}})
if err = os.WriteFile(path, data, 0600); err != nil {
t.Fatal(err)
}
svc.Started = make(chan struct{}, 1)
svc.Release = make(chan struct{})
before := svc.Closed.Load()
returned := svc.SlowReturned.Load()
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
done := make(chan error, 1)
go func() { done <- run(ctx, path) }()
waitReady(t, "http://"+address, done)
client := &http.Client{Timeout: 3 * time.Second}
defer client.CloseIdleConnections()
requestDone := make(chan struct{})
go func() {
defer close(requestDone)
res, err := client.Get("http://" + address + "/slow")
if err == nil {
io.Copy(io.Discard, res.Body)
res.Body.Close()
}
}()
await(t, svc.Started)
// Capture the channel, because a later test installs its own fixture.
release := svc.Release
deps := svc.Current.Load()
defer func() {
close(release)
deadline := time.Now().Add(3 * time.Second)
for svc.SlowReturned.Load() == returned {
if time.Now().After(deadline) {
t.Error("handler did not exit after test release")
break
}
time.Sleep(time.Millisecond)
}
// The test owns the controlled handler and has now joined it.
_ = deps.Logger.Close()
}()
cancel()
select {
case err = <-done:
if !errors.Is(err, lifecycle.ErrResourcesRetained) || !errors.Is(err, context.DeadlineExceeded) {
t.Fatal(err)
}
case <-time.After(3 * time.Second):
t.Fatal("unbounded HTTP shutdown")
}
if err = deps.Close(); !errors.Is(err, lifecycle.ErrResourcesRetained) {
t.Fatal("deferred Close bypassed retention", err)
}
if svc.Closed.Load() != before {
t.Fatal("generated main closed client during active request")
}
deps.Logger.Info("retained.http.logger")
logs, err := os.ReadFile(logFile)
if err != nil || !strings.Contains(string(logs), "retained.http.logger") {
t.Fatal("generated main closed logger during active request", err, string(logs))
}
await(t, requestDone)
})
}
}

View File

@@ -3,15 +3,18 @@ package apigen
import (
"errors"
"fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"io"
"os"
"path/filepath"
"strings"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/projectgen"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
)
type WorkspaceService struct {
Dir string `json:"dir"`
Kind string `json:"kind,omitempty"`
API string `json:"api,omitempty"`
Module string `json:"module,omitempty"`
}
@@ -19,10 +22,14 @@ type WorkspaceConfig struct {
Services []WorkspaceService `json:"services"`
ErrorCatalogs []WorkspaceCatalog `json:"errorCatalogs,omitempty"`
}
type workspaceTarget struct {
dir, kind, module, entry string
err error
}
// Workspace operates independent Go modules listed in an explicit local file.
// All projects are preflighted before generate/format writes. Each generator
// transaction is atomic within its project; no cross-module transaction is promised.
// Workspace dispatches to each independent project's generator. Directory/schema
// validation precedes any providers; all targets are preflighted before writes.
// Transactions remain per-project, not crash-atomic across the workspace.
func Workspace(path, command string, w io.Writer) error {
if command != "generate" && command != "check" && command != "format" {
return errors.New("workspace command must be generate, check or format")
@@ -53,8 +60,8 @@ func Workspace(path, command string, w io.Writer) error {
return errors.New("workspace must list 1..64 services")
}
root := canonical(filepath.Dir(real))
var options []Options
seen := map[string]bool{}
// Validate the entire workspace boundary before resolving or running any target.
for _, service := range cfg.Services {
if service.Dir == "" || service.Dir == "." {
return errors.New("workspace requires distinct relative service directories")
@@ -62,27 +69,26 @@ func Workspace(path, command string, w io.Writer) error {
if err = safePath(root, service.Dir+"/.goOmniWorkspaceCheck"); err != nil {
return err
}
dir := filepath.Join(root, filepath.FromSlash(service.Dir))
key := strings.ToLower(dir)
key := strings.ToLower(filepath.Join(root, filepath.FromSlash(service.Dir)))
for prior := range seen {
if prior == key || strings.HasPrefix(key, prior+string(filepath.Separator)) || strings.HasPrefix(prior, key+string(filepath.Separator)) {
return errors.New("workspace service directories overlap")
}
}
seen[key] = true
entry := service.API
if entry == "" {
entry, err = ProjectEntry(dir)
} else {
if err = safePath(dir, entry); err != nil {
return err
}
targets := make([]workspaceTarget, 0, len(cfg.Services))
for _, service := range cfg.Services {
t := workspaceTarget{dir: filepath.Join(root, filepath.FromSlash(service.Dir)), module: service.Module}
t.kind, t.err = workspaceKind(t.dir, service.Kind, service.API)
if t.err == nil && t.kind == "api" {
if service.API == "" {
t.entry, t.err = ProjectEntry(t.dir)
} else if t.err = safePath(t.dir, service.API); t.err == nil {
t.entry = filepath.Join(t.dir, filepath.FromSlash(service.API))
}
entry = filepath.Join(dir, entry)
}
if err != nil {
return err
}
options = append(options, Options{API: entry, Dir: dir, Module: service.Module, DryRun: true})
targets = append(targets, t)
}
var catalogs map[string][]byte
if command != "format" {
@@ -90,26 +96,41 @@ func Workspace(path, command string, w io.Writer) error {
if err != nil {
return err
}
for i := range options {
options[i].documentCatalogs = catalogs
}
}
var failures error
for _, o := range options {
if command == "format" {
spec, e := Parse(o.API)
if e == nil {
_, e = Compile(spec)
}
err = e
report := func(t workspaceTarget, e error) {
if e != nil {
failures = errors.Join(failures, fmt.Errorf("%s: %w", t.dir, e))
fmt.Fprintf(w, "FAIL %s: %v\n", t.dir, e)
} else if command == "format" && t.kind == "project" {
fmt.Fprintf(w, "SKIP %s (project has no API contract to format)\n", t.dir)
} else {
_, err = Generate(o)
fmt.Fprintf(w, "OK %s\n", t.dir)
}
if err != nil {
failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err))
}
if command == "check" {
// Each generator checks its own manifest and managed outputs. An invalid
// project must not suppress diagnostics for another independent project.
for _, t := range targets {
report(t, runWorkspaceTarget(t, command, false, catalogs))
}
return failures
}
for i := range targets {
t := &targets[i]
t.err = runWorkspaceTarget(*t, command, true, catalogs)
}
for _, t := range targets {
if t.err != nil {
report(t, t.err)
}
}
if failures != nil {
for _, t := range targets {
if t.err == nil {
fmt.Fprintf(w, "SKIP %s (workspace preflight failed; no changes applied)\n", t.dir)
}
}
return failures
}
if command == "generate" {
@@ -119,26 +140,79 @@ func Workspace(path, command string, w io.Writer) error {
}
}
}
for _, o := range options {
switch command {
case "format":
_, err = FormatProject(o.API, false)
case "generate":
o.DryRun = false
_, err = Generate(o)
case "check":
_, fmtErr := FormatProject(o.API, true)
o.DryRun = false
o.Check = true
_, genErr := Generate(o)
err = errors.Join(fmtErr, genErr)
}
if err != nil {
failures = errors.Join(failures, fmt.Errorf("%s: %w", o.Dir, err))
fmt.Fprintf(w, "FAIL %s\n", o.Dir)
} else {
fmt.Fprintf(w, "OK %s\n", o.Dir)
}
for _, t := range targets {
report(t, runWorkspaceTarget(t, command, false, catalogs))
}
return failures
}
// An explicit API entry remains supported before the first API generation.
// Without manifests, omitted kind preserves the previous API interpretation.
// Manifests of both kinds or a conflicting declaration never silently choose one.
func workspaceKind(dir, kind, entry string) (string, error) {
if kind != "" && kind != "api" && kind != "project" {
return "", fmt.Errorf("unknown workspace kind %q; expected api or project", kind)
}
detected := ""
for _, candidate := range []struct{ kind, path string }{{"api", manifestPath}, {"project", ".goOmni/projectManifest.json"}} {
if err := safePath(dir, candidate.path); err != nil {
return "", err
}
if _, err := os.Lstat(filepath.Join(dir, candidate.path)); err == nil {
if detected != "" {
return "", errors.New("ambiguous project: both API and project manifests exist")
}
detected = candidate.kind
} else if !errors.Is(err, os.ErrNotExist) {
return "", err
}
}
if kind != "" && detected != "" && kind != detected {
return "", fmt.Errorf("workspace kind %q conflicts with %s manifest", kind, detected)
}
if kind == "" {
kind = detected
}
if kind == "" {
kind = "api"
}
if kind == "project" && entry != "" {
return "", errors.New("workspace project kind does not accept an api entry")
}
return kind, nil
}
func runWorkspaceTarget(t workspaceTarget, command string, preview bool, catalogs map[string][]byte) error {
if t.err != nil {
return t.err
}
if t.kind == "project" {
if command == "format" {
return nil
}
_, err := projectgen.Generate(projectgen.Options{Dir: t.dir, Module: t.module, DryRun: preview, Check: command == "check"})
return err
}
o := Options{API: t.entry, Dir: t.dir, Module: t.module, DryRun: preview, documentCatalogs: catalogs}
switch command {
case "format":
if preview {
spec, err := Parse(t.entry)
if err == nil {
_, err = Compile(spec)
}
return err
}
_, err := FormatProject(t.entry, false)
return err
case "generate":
_, err := Generate(o)
return err
case "check":
_, formatErr := FormatProject(t.entry, true)
o.Check = true
_, generateErr := Generate(o)
return errors.Join(formatErr, generateErr)
}
return errors.New("unsupported workspace command")
}

View File

@@ -4,6 +4,7 @@ import (
"bytes"
"encoding/json"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/projectgen"
"io"
"os"
"path/filepath"
@@ -13,7 +14,7 @@ import (
func TestWorkspaceSharedGoCatalog(t *testing.T) {
root := t.TempDir()
writeFile(t, filepath.Join(root, "go.mod"), fmt.Sprintf("module example.com/catalogs\n\ngo 1.24.0\nrequire %s v0.15.0-preview\nreplace %s => %s\n", frameworkModule, frameworkModule, strconvQuote(frameworkRoot(t))))
writeFile(t, filepath.Join(root, "go.mod"), fmt.Sprintf("module example.com/catalogs\n\ngo 1.22.0\nrequire %s v0.15.0-preview\nreplace %s => %s\n", frameworkModule, frameworkModule, strconvQuote(frameworkRoot(t))))
writeFile(t, filepath.Join(root, "internal/apierrors/catalog.go"), `package apierrors
import errors "git.yuntuo.hk/k2safe/go-omni/errors"
func Catalog() errors.Catalog {return errors.Catalog{DefaultStatus:400,Definitions:map[string]errors.Definition{
@@ -30,8 +31,11 @@ func Catalog() errors.Catalog {return errors.Catalog{DefaultStatus:400,Definitio
}
writeFile(t, filepath.Join(dir, "api/omni.imports.json"), `{"roots":{"common":{"path":"../../../contracts"}},"errors":{"source":"@common/errors.json","include":["INVALID"],"routes":{"GET /api/greet/:name":["MISSING"]}}}`)
}
if _, err := projectgen.New(projectgen.Options{Dir: filepath.Join(root, "services/program"), Module: "example.com/program", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
workspace := filepath.Join(root, "omni.workspace.json")
writeFile(t, workspace, `{"errorCatalogs":[{"dir":".","package":"example.com/catalogs/internal/apierrors","function":"Catalog","output":"contracts/errors.json"}],"services":[{"dir":"services/api"},{"dir":"services/id"}]}`)
writeFile(t, workspace, `{"errorCatalogs":[{"dir":".","package":"example.com/catalogs/internal/apierrors","function":"Catalog","output":"contracts/errors.json"}],"services":[{"dir":"services/api"},{"dir":"services/id"},{"dir":"services/program"}]}`)
if err := Workspace(workspace, "generate", io.Discard); err != nil {
t.Fatal(err)
}

View File

@@ -0,0 +1,226 @@
package apigen
import (
"bytes"
"encoding/json"
"io"
"os"
"path/filepath"
"strings"
"testing"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/projectgen"
)
func mixedWorkspace(t *testing.T) (string, string) {
t.Helper()
root := t.TempDir()
if _, err := New(Options{Dir: filepath.Join(root, "apps/interface"), Module: "example.com/interface", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
if _, err := projectgen.New(projectgen.Options{Dir: filepath.Join(root, "apps/program"), Module: "example.com/program", FrameworkDir: frameworkRoot(t)}); err != nil {
t.Fatal(err)
}
config := filepath.Join(root, "omni.workspace.json")
writeFile(t, config, `{"services":[{"dir":"apps/interface"},{"dir":"apps/program"}]}`)
if err := Workspace(config, "format", io.Discard); err != nil {
t.Fatal(err)
}
if err := Workspace(config, "generate", io.Discard); err != nil {
t.Fatal(err)
}
return root, config
}
func TestWorkspaceMixedKindsAndReadOnlyCheck(t *testing.T) {
for _, explicit := range []bool{false, true} {
t.Run(map[bool]string{false: "inferred", true: "explicit"}[explicit], func(t *testing.T) {
root, config := mixedWorkspace(t)
if explicit {
writeFile(t, config, `{"services":[{"dir":"apps/interface","kind":"api","module":"example.com/interface"},{"dir":"apps/program","kind":"project","module":"example.com/program"}]}`)
}
// Workspaces do not manage the Go toolchain's workspace or dependency files.
writeFile(t, filepath.Join(root, "go.work"), "go 1.22.0\nuse (\n ./apps/interface\n ./apps/program\n)\n")
writeFile(t, filepath.Join(root, "go.work.sum"), "workspace sums marker\n")
for _, dir := range []string{"apps/interface", "apps/program"} {
writeFile(t, filepath.Join(root, dir, "go.sum"), "module sums marker\n")
}
runFile := filepath.Join(root, "apps/program/internal/run/run.go")
writeFile(t, runFile, readFile(t, runFile)+"\n// project-owned work\n")
before := snapshot(t, root)
for _, op := range []string{"check", "generate", "check"} {
var output bytes.Buffer
if err := Workspace(config, op, &output); err != nil {
t.Fatal(op, err)
}
if strings.Count(output.String(), "OK ") != 2 {
t.Fatal(output.String())
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal(op, "changed clean workspace or project-owned files")
}
}
// Formatting only touches API contracts, even when project Go has drift.
main := filepath.Join(root, "apps/program/main.go")
writeFile(t, main, readFile(t, main)+"\n// drift intentionally not formatted\n")
before = snapshot(t, root)
var output bytes.Buffer
if err := Workspace(config, "format", &output); err != nil {
t.Fatal(err)
}
if !strings.Contains(output.String(), "SKIP ") || !strings.Contains(output.String(), "apps/program") {
t.Fatal(output.String())
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("format changed a plain Go project")
}
})
}
}
func TestWorkspaceMixedFailureAggregation(t *testing.T) {
root, config := mixedWorkspace(t)
for _, dir := range []string{"apps/interface", "apps/program"} {
path := filepath.Join(root, dir, "main.go")
writeFile(t, path, readFile(t, path)+"\n// manual drift\n")
}
before := snapshot(t, root)
var output bytes.Buffer
err := Workspace(config, "check", &output)
if err == nil || !strings.Contains(err.Error(), "apps/interface") || !strings.Contains(err.Error(), "apps/program") || strings.Count(output.String(), "FAIL ") != 2 {
t.Fatal(err, output.String())
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("failed check wrote files")
}
// Identity failures also keep unrelated project diagnostics.
writeFile(t, config, `{"services":[{"dir":"apps/interface","kind":"project"},{"dir":"apps/program"}]}`)
output.Reset()
before = snapshot(t, root)
err = Workspace(config, "check", &output)
if err == nil || !strings.Contains(err.Error(), "conflicts") || strings.Count(output.String(), "FAIL ") != 2 {
t.Fatal(err, output.String())
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("identity failure changed workspace")
}
}
func TestWorkspaceMixedPreflightBeforeAnyWrites(t *testing.T) {
for _, broken := range []string{"apps/interface", "apps/program"} {
t.Run(broken, func(t *testing.T) {
root, config := mixedWorkspace(t)
stale := "apps/interface"
if stale == broken {
stale = "apps/program"
}
// The first valid generator could repair its missing output, but must not
// write if another project's ownership check will fail.
if err := os.Remove(filepath.Join(root, stale, "main.go")); err != nil {
t.Fatal(err)
}
path := filepath.Join(root, broken, "main.go")
writeFile(t, path, readFile(t, path)+"\n// manual change\n")
before := snapshot(t, root)
var output bytes.Buffer
if err := Workspace(config, "generate", &output); err == nil {
t.Fatal("bad preflight accepted")
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("partial regeneration before failed preflight")
}
if !strings.Contains(output.String(), "SKIP ") || !strings.Contains(output.String(), "FAIL ") {
t.Fatal(output.String())
}
})
}
root, config := mixedWorkspace(t)
for _, dir := range []string{"apps/interface", "apps/program"} {
if err := os.Remove(filepath.Join(root, dir, "main.go")); err != nil {
t.Fatal(err)
}
}
if err := Workspace(config, "generate", io.Discard); err != nil {
t.Fatal(err)
}
if err := Workspace(config, "check", io.Discard); err != nil {
t.Fatal(err)
}
}
func TestWorkspaceKindGuardsAndLegacyEntry(t *testing.T) {
root, config := mixedWorkspace(t)
for _, tc := range []struct{ services, needle string }{
{`[{"dir":"apps/program","kind":"worker"}]`, "unknown workspace kind"},
{`[{"dir":"apps/program","kind":"api"}]`, "conflicts"},
{`[{"dir":"apps/interface","kind":"project"}]`, "conflicts"},
{`[{"dir":"apps/program","api":"api/service.api"}]`, "does not accept"},
{`[{"dir":"apps/program","kind":"project","api":"api/service.api"}]`, "does not accept"},
{`[{"dir":"apps/program","module":"example.com/other"}]`, "module differs"},
{`[{"dir":"../outside","kind":"project"}]`, "unsafe"},
{`[{"dir":"apps/program"},{"dir":"apps/program/nested","kind":"project"}]`, "overlap"},
{`[{"dir":"apps/program","kind":"project","kind":"api"}]`, "duplicate"},
} {
writeFile(t, config, `{"services":`+tc.services+`}`)
before := snapshot(t, root)
err := Workspace(config, "check", io.Discard)
if err == nil || !strings.Contains(err.Error(), tc.needle) {
t.Fatal(tc.services, err)
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("invalid kind wrote files")
}
}
writeFile(t, config, `{"services":[{"dir":"apps/program"}]}`)
other := filepath.Join(root, "apps/program/.goOmni/apiManifest.json")
writeFile(t, other, "{}")
if err := Workspace(config, "check", io.Discard); err == nil || !strings.Contains(err.Error(), "both API and project") {
t.Fatal(err)
}
if err := os.Remove(other); err != nil {
t.Fatal(err)
}
// A malformed manifest must be routed to its owner, never fall back to API.
writeFile(t, filepath.Join(root, "apps/program/.goOmni/projectManifest.json"), "invalid")
if err := Workspace(config, "check", io.Discard); err == nil || !strings.Contains(err.Error(), "invalid project manifest") {
t.Fatal(err)
}
// Explicit API entry outside a managed project preserves the original path.
legacy := filepath.Join(root, "legacy")
writeFile(t, filepath.Join(legacy, "service.api"), "service legacy {\n@handler ping\nget /ping\n}\n")
writeFile(t, config, `{"services":[{"dir":"legacy","api":"service.api","module":"example.com/legacy"}]}`)
if err := Workspace(config, "generate", io.Discard); err != nil {
t.Fatal("legacy API entry", err)
}
}
func TestWorkspaceManifestSymlinkRejected(t *testing.T) {
root, config := mixedWorkspace(t)
path := filepath.Join(root, "apps/program/.goOmni/projectManifest.json")
target := filepath.Join(t.TempDir(), "manifest.json")
writeFile(t, target, readFile(t, path))
if err := os.Remove(path); err != nil {
t.Fatal(err)
}
if err := os.Symlink(target, path); err != nil {
t.Fatal(err)
}
before := snapshot(t, root)
for _, op := range []string{"check", "format", "generate"} {
if err := Workspace(config, op, io.Discard); err == nil || !strings.Contains(err.Error(), "symlink") {
t.Fatal(op, err)
}
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("followed symlink")
}
}
func TestWorkspaceProjectOnly(t *testing.T) {
root, config := mixedWorkspace(t)
raw, _ := json.Marshal(WorkspaceConfig{Services: []WorkspaceService{{Dir: "apps/program"}}})
writeFile(t, config, string(raw))
before := snapshot(t, root)
for _, op := range []string{"format", "generate", "check"} {
if err := Workspace(config, op, io.Discard); err != nil {
t.Fatal(op, err)
}
}
if !sameFiles(before, snapshot(t, root)) {
t.Fatal("plain workspace changed")
}
}

View File

@@ -7,6 +7,7 @@ import (
"encoding/json"
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/internal/genfs"
"os"
"path/filepath"
"sort"
@@ -244,44 +245,13 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
}
return result, checkErr
}
// A rollback restores bytes/modes if an ordinary filesystem write fails.
// File replacement is atomic; the whole directory is not crash-atomic.
var applied []change
transaction := make([]genfs.Change, 0, len(changes))
for _, c := range changes {
if err = safePath(root, c.name); err == nil {
current, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(c.name)))
if c.existed {
if e != nil || !bytes.Equal(current, c.before) {
err = fmt.Errorf("file changed during generation: %s", c.name)
}
} else if !errors.Is(e, os.ErrNotExist) {
err = fmt.Errorf("file appeared during generation: %s", c.name)
}
}
if err == nil {
if c.remove {
err = os.Remove(filepath.Join(root, filepath.FromSlash(c.name)))
} else {
err = atomicWrite(root, c.name, c.after, c.mode)
}
}
if err != nil {
for i := len(applied) - 1; i >= 0; i-- {
previous := applied[i]
var rollback error
if previous.existed {
rollback = atomicWrite(root, previous.name, previous.before, previous.mode)
} else {
rollback = os.Remove(filepath.Join(root, filepath.FromSlash(previous.name)))
}
err = errors.Join(err, rollback)
}
return result, err
}
applied = append(applied, c)
transaction = append(transaction, genfs.Change{Name: c.name, Before: c.before, After: c.after, Existed: c.existed, Remove: c.remove, Mode: c.mode})
}
return result, nil
return result, genfs.Apply(root, transaction)
}
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
func ownedPath(name string) bool {
if name == "internal/config/loadGen.go" {
@@ -298,52 +268,7 @@ func ownedPath(name string) bool {
}
return strings.HasPrefix(name, "internal/handler/") && strings.HasSuffix(name, "/handlersGen.go")
}
func safePath(root, name string) error {
if filepath.IsAbs(name) || strings.Contains(name, "\\") || filepath.ToSlash(filepath.Clean(name)) != name || name == "." || strings.HasPrefix(name, "../") {
return fmt.Errorf("api: unsafe output path %q", name)
}
path := root
parts := strings.Split(name, "/")
for i, part := range parts {
path = filepath.Join(path, part)
info, err := os.Lstat(path)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return err
}
if info.Mode()&os.ModeSymlink != 0 {
return fmt.Errorf("api: refusing symlink %s", path)
}
if i < len(parts)-1 && !info.IsDir() {
return fmt.Errorf("api: expected directory %s", path)
}
if i == len(parts)-1 && !info.Mode().IsRegular() {
return fmt.Errorf("api: expected regular file %s", path)
}
}
return nil
}
func safePath(root, name string) error { return genfs.SafePath(root, name) }
func atomicWrite(root, name string, data []byte, mode os.FileMode) error {
target := filepath.Join(root, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return err
}
f, err := os.CreateTemp(filepath.Dir(target), ".apiGen-*")
if err != nil {
return err
}
defer os.Remove(f.Name())
if err = f.Chmod(mode); err == nil {
_, err = f.Write(data)
}
if err == nil {
err = f.Sync()
}
err = errors.Join(err, f.Close())
if err != nil {
return err
}
return os.Rename(f.Name(), target)
return genfs.AtomicWrite(root, name, data, mode)
}

195
cmd/go-omni/i18n.go Normal file
View File

@@ -0,0 +1,195 @@
package main
import (
"encoding/json"
"errors"
"flag"
"fmt"
"io"
"os"
"strings"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/i18n"
"git.yuntuo.hk/k2safe/go-omni/i18n/errori18n"
"git.yuntuo.hk/k2safe/go-omni/i18n/i18ntool"
)
type i18nFlags []string
func (v *i18nFlags) String() string { return strings.Join(*v, ",") }
func (v *i18nFlags) Set(s string) error {
if s == "" {
return errors.New("empty value")
}
*v = append(*v, s)
return nil
}
func runI18n(args []string) error { return runI18nTo(args, os.Stdout) }
func runI18nTo(args []string, w io.Writer) error {
if len(args) == 0 {
return errors.New("expected check, export, types or extract")
}
fs := flag.NewFlagSet("i18n "+args[0], flag.ContinueOnError)
fs.SetOutput(io.Discard)
var resources, namespaces, goFiles i18nFlags
configPath := fs.String("config", "", "i18n Config JSON file")
locale := fs.String("locale", "", "selected/source locale")
catalog := fs.String("catalog", "", "exported errors.Catalog JSON (extract only)")
fs.Var(&resources, "resource", "resource JSON file (repeatable)")
fs.Var(&namespaces, "namespace", "export namespace (repeatable)")
fs.Var(&goFiles, "go", "Go source file (repeatable)")
if err := fs.Parse(args[1:]); err != nil {
return err
}
if fs.NArg() != 0 {
return errors.New("unexpected positional arguments")
}
encode := func(v any) error { e := json.NewEncoder(w); e.SetIndent("", " "); return e.Encode(v) }
if args[0] == "extract" && *catalog != "" {
if len(goFiles) > 0 || len(resources) > 0 || *configPath != "" || len(namespaces) != 1 || *locale == "" {
return errors.New("catalog extraction requires --catalog, --locale and exactly one --namespace")
}
raw, err := readI18nFile(*catalog)
if err != nil {
return err
}
var c apierrors.Catalog
if err := apierrors.DecodeCatalogJSON(raw, &c); err != nil {
return err
}
docs, err := errori18n.Source(errori18n.Bind(c, namespaces[0]), *locale)
if err != nil {
return err
}
for _, doc := range docs {
if doc.Namespace == namespaces[0] {
return encode(doc)
}
}
return errors.New("catalog has no messages in the selected namespace")
}
if *catalog != "" {
return errors.New("--catalog is only supported by extract")
}
if args[0] == "extract" {
if len(goFiles) == 0 || len(resources) > 0 || len(namespaces) > 0 || *configPath != "" || *locale != "" {
return errors.New("Go extraction requires one or more --go files")
}
out := i18ntool.Extraction{References: []i18ntool.Reference{}, Findings: []i18ntool.Finding{}}
for _, path := range goFiles {
raw, err := readI18nFile(path)
if err != nil {
return err
}
result, err := i18ntool.ExtractGo(path, raw)
if err != nil {
return err
}
out.References = append(out.References, result.References...)
out.Findings = append(out.Findings, result.Findings...)
}
return encode(out)
}
if args[0] != "check" && args[0] != "export" && args[0] != "types" {
return errors.New("expected check, export, types or extract")
}
if *configPath == "" || len(resources) == 0 {
return errors.New("--config and at least one --resource are required")
}
data, err := readI18nFile(*configPath)
if err != nil {
return err
}
var cfg i18n.Config
if err := i18n.DecodeJSON(data, &cfg); err != nil {
return err
}
docs := []i18n.Document{}
total := 0
for _, path := range resources {
raw, err := readI18nFile(path)
if err != nil {
return err
}
total += len(raw)
if total > 4<<20 {
return errors.New("resources exceed 4 MiB")
}
var doc i18n.Document
if err := i18n.DecodeJSON(raw, &doc); err != nil {
return fmt.Errorf("%s: %w", path, err)
}
docs = append(docs, doc)
}
bundle, err := i18n.New(cfg, docs...)
if err != nil {
return err
}
switch args[0] {
case "export":
if len(goFiles) > 0 {
return errors.New("--go is only supported by check/extract")
}
return bundle.Export(w, *locale, namespaces...)
case "types":
if *locale != "" || len(goFiles) > 0 {
return errors.New("types does not accept --locale or --go")
}
return bundle.TypeScript(w, namespaces...)
case "check":
if *locale != "" || len(namespaces) > 0 {
return errors.New("check validates all configured locales and namespaces")
}
findings := bundle.Check()
for _, path := range goFiles {
raw, e := readI18nFile(path)
if e != nil {
return e
}
ex, e := i18ntool.ExtractGo(path, raw)
if e != nil {
return e
}
for _, r := range ex.References {
if _, ok := bundle.Schema(i18n.Ref{Namespace: r.Namespace, Key: r.Key}); !ok {
findings = append(findings, i18n.Diagnostic{Kind: "missing", Namespace: r.Namespace, Key: r.Key, Problem: fmt.Sprintf("referenced by %s:%d", r.File, r.Line)})
}
}
for _, f := range ex.Findings {
findings = append(findings, i18n.Diagnostic{Kind: "dynamic", Problem: fmt.Sprintf("%s:%d: %s", f.File, f.Line, f.Problem)})
}
}
if err := encode(findings); err != nil {
return err
}
if len(findings) > 0 {
return errors.New("internationalization checks failed")
}
return nil
}
return nil
}
func readI18nFile(path string) ([]byte, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
s, err := f.Stat()
if err != nil {
return nil, err
}
if !s.Mode().IsRegular() {
return nil, errors.New("expected a regular file")
}
b, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
if err != nil {
return nil, err
}
if len(b) > 1<<20 {
return nil, errors.New("file exceeds 1 MiB")
}
return b, nil
}

80
cmd/go-omni/i18n_test.go Normal file
View File

@@ -0,0 +1,80 @@
package main
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"strings"
"testing"
"git.yuntuo.hk/k2safe/go-omni/i18n"
)
func TestI18nCommandsAreReadOnlyAndCheckMissing(t *testing.T) {
dir := t.TempDir()
files := map[string]string{
"config.json": `{"sourceLocale":"en","defaultLocale":"en","locales":["en","fr"]}`,
"en.json": `{"version":1,"namespace":"ui","locale":"en","messages":{"hello":{"params":{"type":"string"},"value":{"text":"Hello {type}"}}}}`,
"fr.json": `{"version":1,"namespace":"ui","locale":"fr","messages":{"hello":{"value":{"text":"Bonjour {type}"}}}}`,
"use.go": "package demo\nimport words \"git.yuntuo.hk/k2safe/go-omni/i18n\"\nvar key=words.Ref{Namespace:\"ui\",Key:\"hello\"}\n",
}
for path, data := range files {
if err := os.WriteFile(filepath.Join(dir, path), []byte(data), 0600); err != nil {
t.Fatal(err)
}
}
base := []string{"--config", filepath.Join(dir, "config.json"), "--resource", filepath.Join(dir, "en.json"), "--resource", filepath.Join(dir, "fr.json")}
var out bytes.Buffer
args := append([]string{"check"}, base...)
args = append(args, "--go", filepath.Join(dir, "use.go"))
if err := runI18nTo(args, &out); err != nil || strings.TrimSpace(out.String()) != "[]" {
t.Fatal(err, out.String())
}
out.Reset()
args = append([]string{"export"}, base...)
args = append(args, "--locale", "fr", "--namespace", "ui")
if err := runI18nTo(args, &out); err != nil {
t.Fatal(err)
}
var pack i18n.Pack
if err := json.Unmarshal(out.Bytes(), &pack); err != nil || pack.Locale != "fr" || len(pack.Documents) != 2 {
t.Fatal(err, pack)
}
out.Reset()
args = append([]string{"types"}, base...)
args = append(args, "--namespace", "ui")
if err := runI18nTo(args, &out); err != nil || !strings.Contains(out.String(), `"type": string`) {
t.Fatal(err, out.String())
}
out.Reset()
if err := runI18nTo([]string{"extract", "--go", filepath.Join(dir, "use.go")}, &out); err != nil || !strings.Contains(out.String(), `"key": "hello"`) {
t.Fatal(err, out.String())
}
for path, want := range files {
got, _ := os.ReadFile(filepath.Join(dir, path))
if string(got) != want {
t.Fatal("command mutated", path)
}
}
out.Reset()
args = append([]string{"check"}, base[:4]...)
if err := runI18nTo(args, &out); err == nil || !strings.Contains(out.String(), "direct translation missing") {
t.Fatal(err, out.String())
}
}
func TestI18nCatalogExtractionPreservesLiteralBraces(t *testing.T) {
path := filepath.Join(t.TempDir(), "errors.json")
if err := os.WriteFile(path, []byte(`{"definitions":{"FIXED":{"code":1,"message":"${NAME}"}}}`), 0600); err != nil {
t.Fatal(err)
}
var out bytes.Buffer
if err := runI18nTo([]string{"extract", "--catalog", path, "--namespace", "errors", "--locale", "en"}, &out); err != nil {
t.Fatal(err)
}
var d i18n.Document
if err := i18n.DecodeJSON(out.Bytes(), &d); err != nil || !d.Messages["FIXED"].Value.Literal || d.Messages["FIXED"].Value.Text != "${NAME}" {
t.Fatal(err, d)
}
}

View File

@@ -0,0 +1,133 @@
// Package genfs shares guarded filesystem operations between project generators.
package genfs
import (
"bytes"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
)
func SafePath(root, name string) error {
if filepath.IsAbs(name) || strings.Contains(name, "\\") || filepath.ToSlash(filepath.Clean(name)) != name || name == "." || strings.HasPrefix(name, "../") {
return fmt.Errorf("generation: unsafe output path %q", name)
}
path := root
parts := strings.Split(name, "/")
for i, part := range parts {
path = filepath.Join(path, part)
info, err := os.Lstat(path)
if errors.Is(err, os.ErrNotExist) {
continue
}
if err != nil {
return err
}
if info.Mode()&os.ModeSymlink != 0 {
return fmt.Errorf("generation: refusing symlink %s", path)
}
if i < len(parts)-1 && !info.IsDir() {
return fmt.Errorf("generation: expected directory %s", path)
}
if i == len(parts)-1 && !info.Mode().IsRegular() {
return fmt.Errorf("generation: expected regular file %s", path)
}
}
return nil
}
func AtomicWrite(root, name string, data []byte, mode os.FileMode) error {
target := filepath.Join(root, filepath.FromSlash(name))
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
return err
}
f, err := os.CreateTemp(filepath.Dir(target), ".omniGen-*")
if err != nil {
return err
}
defer os.Remove(f.Name())
if err = f.Chmod(mode); err == nil {
_, err = f.Write(data)
}
if err == nil {
err = f.Sync()
}
err = errors.Join(err, f.Close())
if err != nil {
return err
}
return os.Rename(f.Name(), target)
}
// Change records the preflight snapshot used for conflict detection and rollback.
type Change struct {
Name string
Before, After []byte
Existed, Remove bool
Mode os.FileMode
}
// Apply replaces individual files atomically and rolls back ordinary write failures.
// Callers serialize generators with a project lock. This is not crash-atomic across files.
func Apply(root string, changes []Change) error {
var applied []Change
for _, c := range changes {
err := SafePath(root, c.Name)
if err == nil {
current, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(c.Name)))
if c.Existed {
if e != nil || !bytes.Equal(current, c.Before) {
err = fmt.Errorf("file changed during generation: %s", c.Name)
}
} else if !errors.Is(e, os.ErrNotExist) {
err = fmt.Errorf("file appeared during generation: %s", c.Name)
}
}
if err == nil {
if c.Remove {
err = os.Remove(filepath.Join(root, filepath.FromSlash(c.Name)))
} else {
err = AtomicWrite(root, c.Name, c.After, c.Mode)
}
}
if err != nil {
for i := len(applied) - 1; i >= 0; i-- {
previous := applied[i]
if e := SafePath(root, previous.Name); e != nil {
err = errors.Join(err, e)
continue
}
if previous.Existed {
err = errors.Join(err, AtomicWrite(root, previous.Name, previous.Before, previous.Mode))
} else {
err = errors.Join(err, os.Remove(filepath.Join(root, filepath.FromSlash(previous.Name))))
}
}
return err
}
applied = append(applied, c)
}
return nil
}
// Canonical resolves existing parent symlinks even for an output not yet created.
func Canonical(path string) string {
path, _ = filepath.Abs(path)
candidate := path
var tail []string
for {
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
for i := len(tail) - 1; i >= 0; i-- {
resolved = filepath.Join(resolved, tail[i])
}
return resolved
}
parent := filepath.Dir(candidate)
if parent == candidate {
return path
}
tail = append(tail, filepath.Base(candidate))
candidate = parent
}
}

View File

@@ -0,0 +1,47 @@
package genfs
import (
"os"
"path/filepath"
"testing"
)
func TestConflictRollsBackEarlierWrites(t *testing.T) {
root := t.TempDir()
if err := os.WriteFile(filepath.Join(root, "one"), []byte("before"), 0600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "two"), []byte("concurrent"), 0644); err != nil {
t.Fatal(err)
}
err := Apply(root, []Change{{Name: "one", Before: []byte("before"), After: []byte("after"), Existed: true, Mode: 0600}, {Name: "new", After: []byte("created"), Mode: 0644}, {Name: "two", Before: []byte("stale"), After: []byte("after"), Existed: true, Mode: 0644}})
if err == nil {
t.Fatal("conflict accepted")
}
data, _ := os.ReadFile(filepath.Join(root, "one"))
info, _ := os.Stat(filepath.Join(root, "one"))
if string(data) != "before" || info.Mode().Perm() != 0600 {
t.Fatal("rollback did not restore bytes/mode")
}
if _, err = os.Stat(filepath.Join(root, "new")); !os.IsNotExist(err) {
t.Fatal("new output survived rollback")
}
data, _ = os.ReadFile(filepath.Join(root, "two"))
if string(data) != "concurrent" {
t.Fatal("concurrent edit overwritten")
}
}
func TestUnsafePaths(t *testing.T) {
root := t.TempDir()
for _, name := range []string{"../escape", "a/../../escape", "/outside", "a\\b", ".", "a/../b"} {
if SafePath(root, name) == nil {
t.Fatal(name)
}
}
if err := os.Symlink(t.TempDir(), filepath.Join(root, "link")); err != nil {
t.Fatal(err)
}
if SafePath(root, "link/file") == nil {
t.Fatal("followed symlink")
}
}

View File

@@ -18,6 +18,16 @@ func main() {
}
switch os.Args[1] {
case "project":
if err := runProject(os.Args[2:]); err != nil {
fmt.Fprintln(os.Stderr, "go-omni project:", err)
os.Exit(1)
}
case "i18n":
if err := runI18n(os.Args[2:]); err != nil {
fmt.Fprintln(os.Stderr, "go-omni i18n:", err)
os.Exit(1)
}
case "workspace":
if err := runWorkspace(os.Args[2:]); err != nil {
fmt.Fprintln(os.Stderr, "go-omni workspace:", err)
@@ -170,6 +180,8 @@ func usage() {
fmt.Fprintln(os.Stderr, "GoOmni project tool")
fmt.Fprintln(os.Stderr, "")
fmt.Fprintln(os.Stderr, "Usage:")
fmt.Fprintln(os.Stderr, " go-omni project <new|generate|check> Generate a transport-independent Go program")
fmt.Fprintln(os.Stderr, " go-omni i18n <check|export|types|extract> Validate and export independent language resources")
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] <directory>")
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
@@ -211,6 +223,9 @@ func runAdd(args []string) error {
return fmt.Errorf("unexpected argument %q", arg)
}
}
if _, err := os.Lstat(filepath.Join(directory, ".goOmni", "projectManifest.json")); !errors.Is(err, os.ErrNotExist) {
return errors.New("plain Go projects add packages as ordinary Go code; use project generate for infrastructure")
}
if _, err := os.Stat(filepath.Join(directory, ".goOmni", "apiManifest.json")); err == nil {
return errors.New("managed API projects add modules through .api imports and api generate")
}

102
cmd/go-omni/project.go Normal file
View File

@@ -0,0 +1,102 @@
package main
import (
"errors"
"fmt"
"strings"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/projectgen"
)
func runProject(args []string) error {
if len(args) == 0 || args[0] == "help" || args[0] == "--help" || args[0] == "-h" {
projectUsage()
return nil
}
command := args[0]
if command != "new" && command != "generate" && command != "check" {
return fmt.Errorf("unknown project command %q", command)
}
o := projectgen.Options{Check: command == "check"}
for i := 1; i < len(args); i++ {
arg := args[i]
if arg == "--help" || arg == "-h" {
projectUsage()
return nil
}
if arg == "--check" || arg == "--dry-run" {
if command != "generate" {
return fmt.Errorf("%s belongs to project generate", arg)
}
if arg == "--check" {
o.Check = true
} else {
o.DryRun = true
}
continue
}
if !strings.HasPrefix(arg, "-") {
if command != "new" || o.Dir != "" {
return fmt.Errorf("unexpected argument %q", arg)
}
o.Dir = arg
continue
}
name, value, inline := strings.Cut(arg, "=")
switch name {
case "--dir", "--module", "--framework-dir", "--framework-version":
default:
return fmt.Errorf("unknown project option %q", name)
}
if command != "new" && name != "--dir" {
return fmt.Errorf("%s belongs to project new", name)
}
if !inline {
if i+1 >= len(args) || strings.HasPrefix(args[i+1], "-") {
return fmt.Errorf("%s requires a value", name)
}
i++
value = args[i]
}
if strings.TrimSpace(value) == "" {
return fmt.Errorf("%s requires a value", name)
}
switch name {
case "--dir":
if o.Dir != "" {
return errors.New("project directory specified more than once")
}
o.Dir = value
case "--module":
o.Module = value
case "--framework-dir":
o.FrameworkDir = value
case "--framework-version":
o.FrameworkVersion = value
}
}
var out projectgen.Output
var err error
if command == "new" {
out, err = projectgen.New(o)
} else {
out, err = projectgen.Generate(o)
}
if err != nil {
return err
}
if o.DryRun {
for _, change := range out.Changes {
fmt.Printf("%-7s %s\n", change.Action, change.Path)
}
} else {
fmt.Printf("project %s: %d file changes\n", command, len(out.Changes))
}
return nil
}
func projectUsage() {
fmt.Println("go-omni project new <directory> [--module <module>] [--framework-version <version>|--framework-dir <directory>]")
fmt.Println("go-omni project generate [--dir <project>] [--check|--dry-run]")
fmt.Println("go-omni project check [--dir <project>]")
fmt.Println("Creates a plain Go program with config, logging, dependency lifecycle and an editable Run entry.")
}

View File

@@ -0,0 +1,27 @@
package main
import (
"os"
"path/filepath"
"testing"
)
func TestProjectCLI(t *testing.T) {
root := filepath.Join(t.TempDir(), "program")
for _, args := range [][]string{{"new", root, "--module", "example.com/program"}, {"generate", "--dir", root, "--dry-run"}, {"generate", "--dir", root}, {"check", "--dir", root}, {"generate", "--dir", root, "--check"}} {
if err := runProject(args); err != nil {
t.Fatal(args, err)
}
}
if err := runAdd([]string{"item", "--dir", root}); err == nil {
t.Fatal("add adopted plain project")
}
if _, err := os.Stat(filepath.Join(root, "internal/item")); !os.IsNotExist(err) {
t.Fatal("business module generated")
}
for _, args := range [][]string{{"new"}, {"new", "a", "b"}, {"new", "--api", "x"}, {"check", "--dry-run"}, {"generate", "--check", "--dry-run"}, {"generate", "--module", "x"}, {"new", "--module"}, {"new", "--dir="}, {"new", "--dir", "x", "y"}, {"unknown"}} {
if err := runProject(args); err == nil {
t.Fatal("accepted", args)
}
}
}

View File

@@ -0,0 +1,190 @@
package projectgen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strconv"
"strings"
"syscall"
"testing"
"time"
)
func TestGeneratedProgramConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
root := filepath.Join(t.TempDir(), "program")
cli := os.Getenv("GO_OMNI_RELEASE_CLI")
if cli != "" {
runCommand(t, "", cli, "project", "new", root, "--module", "example.com/plain")
} else {
framework, err := filepath.Abs("../../..")
if err != nil {
t.Fatal(err)
}
if _, err = New(Options{Dir: root, Module: "example.com/plain", FrameworkDir: framework}); err != nil {
t.Fatal(err)
}
}
// Only project-owned files are customized. All entry/loading/lifecycle code
// remains exactly what the local or downloaded CLI generated.
svc := get(t, root, "internal/svc/serviceContext.go")
svc = strings.Replace(svc, "return nil", "return Initialize(ctx, c)", 1)
svc += "\nvar Initialize = func(context.Context, *ServiceContext) error { return nil }\n"
put(t, root, "internal/svc/serviceContext.go", svc)
put(t, root, "internal/run/run.go", `package run
import("context";"errors";"os";"example.com/plain/internal/svc")
var Execute=func(ctx context.Context,deps *svc.ServiceContext)error{
switch os.Getenv("GO_OMNI_PROGRAM_TEST_MODE") {case "error":return errors.New("process failure marker");case "panic":panic("process panic marker");case "unresponsive":select{}}
<-ctx.Done();return nil
}
func Run(ctx context.Context,deps *svc.ServiceContext)error{return Execute(ctx,deps)}
`)
put(t, root, "consumer_test.go", get(t, "testdata", "consumer_test.go"))
runCommand(t, root, "go", "mod", "tidy")
runCommand(t, root, "go", "test", "-race", "-count=1", "./...")
runCommand(t, root, "go", "vet", "./...")
binary := filepath.Join(root, "program")
runCommand(t, root, "go", "build", "-race", "-o", binary, ".")
before := snapshot(t, root)
for _, mode := range []string{"generate", "check", "dry-run"} {
if cli != "" {
args := []string{"project", mode, "--dir", root}
if mode == "dry-run" {
args = []string{"project", "generate", "--dir", root, "--dry-run"}
}
runCommand(t, root, cli, args...)
} else {
if _, err := Generate(Options{Dir: root, Check: mode == "check", DryRun: mode == "dry-run"}); err != nil {
t.Fatal(err)
}
}
}
after := snapshot(t, root)
for name, data := range before {
if after[name] != data {
t.Fatal("consumer changed during regeneration", name)
}
}
// Validation must not open even the configured log file.
logfile := filepath.Join(root, "check-must-not-open.log")
cfg := filepath.Join(root, "check.json")
raw, _ := json.Marshal(map[string]any{"log": map[string]any{"file": logfile}})
put(t, root, "check.json", string(raw))
runCommand(t, root, binary, "-config", cfg, "-check-config")
if _, err := os.Stat(logfile); !os.IsNotExist(err) {
t.Fatal("check-config opened logger", err)
}
put(t, root, "check.json", `{"log":{"level":"invalid"}}`)
cmd := exec.Command(binary, "-config", cfg, "-check-config")
cmd.Dir = root
if data, err := cmd.CombinedOutput(); err == nil || !strings.Contains(string(data), "log.level") {
t.Fatal("invalid config accepted", string(data), err)
}
for _, sig := range []os.Signal{os.Interrupt, syscall.SIGTERM} {
testProcess(t, root, binary, sig, "")
}
for _, mode := range []string{"error", "panic", "unresponsive"} {
var sig os.Signal
if mode == "unresponsive" {
sig = syscall.SIGTERM
}
testProcess(t, root, binary, sig, mode)
}
if cli != "" {
data := runCommand(t, root, "go", "list", "-m", "-json", frameworkModule)
var dep struct {
Version, Sum string
Replace any
}
if err := json.Unmarshal([]byte(data), &dep); err != nil {
t.Fatal(err)
}
if dep.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dep.Replace != nil || dep.Sum == "" {
t.Fatal("not a fixed remote consumer", dep)
}
t.Logf("remote framework %s checksum %s; no local replace", dep.Version, dep.Sum)
}
}
func runCommand(t *testing.T, dir, name string, args ...string) string {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
cmd := exec.CommandContext(ctx, name, args...)
if dir != "" {
cmd.Dir = dir
}
cmd.Env = append(os.Environ(), "GOWORK=off")
output, err := cmd.CombinedOutput()
if err != nil {
t.Fatalf("%s %v: %v\n%s", name, args, err, output)
}
t.Logf("%s %v\n%s", name, args, output)
return string(output)
}
func testProcess(t *testing.T, root, binary string, sig os.Signal, mode string) {
t.Helper()
logfile, err := os.CreateTemp(root, "signal-*.log")
if err != nil {
t.Fatal(err)
}
defer logfile.Close()
cfg := filepath.Join(root, "process.json")
put(t, root, "process.json", `{"lifecycle":{"stopTimeout":"100ms","drainTimeout":"30ms","cancelTimeout":"100ms"}}`)
cmd := exec.Command(binary, "-config", cfg)
cmd.Env = append(os.Environ(), "GO_OMNI_PROGRAM_TEST_MODE="+mode)
cmd.Dir = root
cmd.Stdout = logfile
cmd.Stderr = logfile
if err = cmd.Start(); err != nil {
t.Fatal(err)
}
done := make(chan error, 1)
go func() { done <- cmd.Wait() }()
defer func() { _ = cmd.Process.Kill() }()
deadline := time.Now().Add(10 * time.Second)
for {
data, _ := os.ReadFile(logfile.Name())
if strings.Contains(string(data), "program.starting") {
break
}
if time.Now().After(deadline) {
t.Fatal("program did not start", string(data))
}
time.Sleep(10 * time.Millisecond)
}
if path, err := exec.LookPath("lsof"); err == nil {
output, _ := exec.Command(path, "-nP", "-a", "-p", strconv.Itoa(cmd.Process.Pid), "-i").CombinedOutput()
if strings.Contains(string(output), "TCP") || strings.Contains(string(output), "UDP") {
t.Fatal("starter opened network socket", string(output))
}
}
if sig != nil {
if err = cmd.Process.Signal(sig); err != nil {
t.Fatal(err)
}
}
select {
case err = <-done:
data, _ := os.ReadFile(logfile.Name())
if mode == "" && err != nil {
t.Fatal("signal exit failed", err, string(data))
}
if mode != "" {
exit, ok := err.(*exec.ExitError)
if !ok || exit.ExitCode() != 1 {
t.Fatal("abnormal process exit status", err, string(data))
}
}
if mode == "unresponsive" && !strings.Contains(string(data), "shared resources retained") {
t.Fatal("timeout did not retain resources", string(data))
}
t.Logf("process mode=%q signal=%v passed", mode, sig)
case <-time.After(10 * time.Second):
t.Fatal("signal exit timed out")
}
}

View File

@@ -0,0 +1,359 @@
// Package projectgen generates transport-independent Go programs.
package projectgen
import (
"bytes"
"crypto/sha256"
"embed"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"go/format"
"io"
"os"
"path/filepath"
"runtime/debug"
"sort"
"strconv"
"strings"
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/internal/genfs"
"golang.org/x/mod/modfile"
"golang.org/x/mod/module"
"golang.org/x/mod/semver"
)
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
const currentVersion = "v0.28.1-preview"
const minimumVersion = "v0.28.0-preview"
const manifestPath = ".goOmni/projectManifest.json"
const lockPath = ".goOmni/project.lock"
const header = "// Code generated by go-omni project. DO NOT EDIT.\n"
//go:embed templates/*
var templates embed.FS
var managed = map[string]bool{"main.go": true, "internal/config/loadGen.go": true, "internal/svc/runtimeGen.go": true}
var seeds = []string{"go.mod", ".gitignore", "README.md", "etc/config.yaml", "internal/config/config.go", "internal/config/loader.go", "internal/svc/serviceContext.go", "internal/run/run.go"}
type Options struct {
Dir, Module, FrameworkDir, FrameworkVersion string
Check, DryRun bool
}
type FileChange struct{ Path, Action string }
type Output struct{ Changes []FileChange }
type manifest struct {
Version int `json:"version"`
Kind string `json:"kind"`
TemplateVersion string `json:"templateVersion"`
Module string `json:"module"`
Generated map[string]string `json:"generated"`
}
// New only accepts an empty directory. FrameworkDir is an explicit development opt-in.
func New(o Options) (Output, error) {
if o.Dir == "" || o.Check || o.DryRun {
return Output{}, errors.New("project new requires a directory and does not accept --check/--dry-run")
}
if entries, err := os.ReadDir(o.Dir); err == nil && len(entries) > 0 {
return Output{}, errors.New("project new requires an empty directory")
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
return Output{}, err
}
if o.Module == "" {
o.Module = filepath.Base(genfs.Canonical(o.Dir))
}
if err := module.CheckImportPath(o.Module); err != nil {
return Output{}, err
}
if _, err := moduleFile(o, genfs.Canonical(o.Dir), true); err != nil {
return Output{}, err
}
return generate(o, true)
}
func Generate(o Options) (Output, error) { return generate(o, false) }
func generate(o Options, create bool) (out Output, err error) {
if o.Check && o.DryRun {
return out, errors.New("--check and --dry-run cannot be combined")
}
if o.Dir == "" {
o.Dir = "."
}
root, err := filepath.Abs(o.Dir)
if err != nil {
return out, err
}
if info, e := os.Lstat(root); e == nil {
if !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
return out, errors.New("project directory must be a real directory")
}
} else if !create || !errors.Is(e, os.ErrNotExist) {
return out, e
}
root = genfs.Canonical(root)
for _, name := range append(append([]string{}, seeds...), manifestPath, lockPath) {
if err = genfs.SafePath(root, name); err != nil {
return out, err
}
}
for name := range managed {
if err = genfs.SafePath(root, name); err != nil {
return out, err
}
}
for _, other := range []string{".goOmni/apiManifest.json", "api/service.api"} {
if _, e := os.Lstat(filepath.Join(root, other)); !errors.Is(e, os.ErrNotExist) {
return out, errors.New("project generator requires its own project; existing API layout is not adopted")
}
}
if !o.Check && !o.DryRun {
if err = os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
return out, err
}
lock, e := os.OpenFile(filepath.Join(root, lockPath), os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600)
if e != nil {
return out, fmt.Errorf("acquire project.lock (check for a running generator before removing a stale lock): %w", e)
}
_, err = fmt.Fprintf(lock, "pid=%d\n", os.Getpid())
err = errors.Join(err, lock.Close())
defer os.Remove(filepath.Join(root, lockPath))
if err != nil {
return out, err
}
}
var old manifest
oldData, readErr := os.ReadFile(filepath.Join(root, manifestPath))
if create {
if !errors.Is(readErr, os.ErrNotExist) {
return out, errors.New("project manifest appeared during creation")
}
// Recheck after taking the lock; never adopt files created concurrently.
entries, e := os.ReadDir(root)
if e != nil {
return out, e
}
for _, entry := range entries {
if entry.Name() != ".goOmni" {
return out, errors.New("project directory changed during creation")
}
}
entries, e = os.ReadDir(filepath.Join(root, ".goOmni"))
if e != nil {
return out, e
}
for _, entry := range entries {
if entry.Name() != "project.lock" {
return out, errors.New("project metadata appeared during creation")
}
}
} else {
if readErr != nil {
return out, fmt.Errorf("read project manifest (create with project new first): %w", readErr)
}
decoder := json.NewDecoder(bytes.NewReader(oldData))
decoder.DisallowUnknownFields()
if err = decoder.Decode(&old); err != nil {
return out, fmt.Errorf("invalid project manifest: %w", err)
}
var extra any
if decoder.Decode(&extra) != io.EOF {
return out, errors.New("invalid trailing project manifest data")
}
if old.Version != 1 || old.Kind != "project" || old.Module == "" || len(old.Generated) != len(managed) || !semver.IsValid(old.TemplateVersion) || semver.Compare(old.TemplateVersion, minimumVersion) > 0 {
return out, errors.New("unsupported project manifest; use a compatible CLI")
}
if o.Module != "" && o.Module != old.Module {
return out, errors.New("module differs from project manifest")
}
o.Module = old.Module
for name := range managed {
if len(old.Generated[name]) != 64 {
return out, fmt.Errorf("invalid manifest ownership for %s", name)
}
}
}
if o.Module == "" {
o.Module = filepath.Base(root)
}
if err = module.CheckImportPath(o.Module); err != nil {
return out, err
}
moduleData, err := moduleFile(o, root, create)
if err != nil {
return out, err
}
rendered, err := render(o.Module, moduleData)
if err != nil {
return out, err
}
next := manifest{Version: 1, Kind: "project", TemplateVersion: minimumVersion, Module: o.Module, Generated: map[string]string{}}
var changes []genfs.Change
names := make([]string, 0, len(rendered))
for name := range rendered {
names = append(names, name)
}
sort.Strings(names)
for _, name := range names {
target := filepath.Join(root, filepath.FromSlash(name))
before, e := os.ReadFile(target)
exists := e == nil
if e != nil && !errors.Is(e, os.ErrNotExist) {
return out, e
}
generated := managed[name]
if !generated && !create {
if !exists {
return out, fmt.Errorf("project-owned file %s is missing; restore it explicitly", name)
}
continue
}
if generated {
next.Generated[name] = digest(rendered[name])
if exists && (create || digest(before) != old.Generated[name]) {
return out, fmt.Errorf("generated file %s was edited; move custom code to project-owned files", name)
}
} else if exists {
return out, fmt.Errorf("refusing to adopt %s", name)
}
if exists && bytes.Equal(before, rendered[name]) {
continue
}
mode := os.FileMode(0644)
if exists {
info, e := os.Stat(target)
if e != nil {
return out, e
}
mode = info.Mode().Perm()
}
changes = append(changes, genfs.Change{Name: name, Before: before, After: rendered[name], Existed: exists, Mode: mode})
}
data, err := json.MarshalIndent(next, "", " ")
if err != nil {
return out, err
}
data = append(data, '\n')
if !bytes.Equal(oldData, data) {
changes = append(changes, genfs.Change{Name: manifestPath, Before: oldData, After: data, Existed: oldData != nil, Mode: 0644})
}
for _, c := range changes {
action := "create"
if c.Existed {
action = "update"
}
out.Changes = append(out.Changes, FileChange{c.Name, action})
}
sort.Slice(out.Changes, func(i, j int) bool { return out.Changes[i].Path < out.Changes[j].Path })
if o.DryRun {
return out, nil
}
if o.Check {
if len(changes) > 0 {
return out, fmt.Errorf("project generation drift: %v", out.Changes)
}
return out, nil
}
return out, genfs.Apply(root, changes)
}
func moduleFile(o Options, root string, create bool) ([]byte, error) {
if !create {
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
if err != nil {
return nil, err
}
mf, err := modfile.Parse("go.mod", data, nil)
if err != nil {
return nil, err
}
if mf.Module == nil || mf.Module.Mod.Path != o.Module {
return nil, errors.New("go.mod module differs from project manifest")
}
if o.FrameworkDir != "" || o.FrameworkVersion != "" {
return nil, errors.New("framework options belong to project new; update go.mod explicitly")
}
for _, r := range mf.Require {
if r.Mod.Path == frameworkModule && semver.Compare(r.Mod.Version, minimumVersion) >= 0 {
return data, nil
}
}
return nil, fmt.Errorf("go.mod must require go-omni %s or newer", minimumVersion)
}
version := o.FrameworkVersion
if version == "" {
version = currentVersion
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) {
version = info.Main.Version
}
}
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" || semver.Compare(version, minimumVersion) < 0 {
return nil, fmt.Errorf("framework version must be %s or newer", minimumVersion)
}
mf := new(modfile.File)
if err := mf.AddModuleStmt(o.Module); err != nil {
return nil, err
}
if err := mf.AddGoStmt("1.22.0"); err != nil {
return nil, err
}
if err := mf.AddRequire(frameworkModule, version); err != nil {
return nil, err
}
if o.FrameworkDir != "" {
local, err := filepath.Abs(o.FrameworkDir)
if err != nil {
return nil, err
}
data, err := os.ReadFile(filepath.Join(local, "go.mod"))
if err != nil {
return nil, err
}
source, err := modfile.Parse("go.mod", data, nil)
if err != nil || source.Module == nil || source.Module.Mod.Path != frameworkModule {
return nil, errors.New("--framework-dir must contain go-omni")
}
local = genfs.Canonical(local)
rel, err := filepath.Rel(root, local)
if err != nil {
return nil, err
}
rel = filepath.ToSlash(rel)
if !strings.HasPrefix(rel, ".") {
rel = "./" + rel
}
if err = mf.AddReplace(frameworkModule, "", rel, ""); err != nil {
return nil, err
}
}
return mf.Format()
}
func render(moduleName string, moduleData []byte) (map[string][]byte, error) {
files := map[string][]byte{"go.mod": moduleData}
paths := append([]string{}, seeds[1:]...)
for name := range managed {
paths = append(paths, name)
}
for _, name := range paths {
raw, err := templates.ReadFile("templates/" + strings.ReplaceAll(name, "/", "_") + ".tmpl")
if err != nil {
return nil, err
}
s := strings.ReplaceAll(string(raw), "\"PROJECT_MODULE\"", strconv.Quote(moduleName))
s = strings.ReplaceAll(s, "PROJECT_MODULE/", moduleName+"/")
if managed[name] {
s = header + s
}
data := []byte(s)
if strings.HasSuffix(name, ".go") {
data, err = format.Source(data)
if err != nil {
return nil, fmt.Errorf("format %s: %w", name, err)
}
}
files[name] = data
}
return files, nil
}
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }

View File

@@ -0,0 +1,215 @@
package projectgen
import (
"encoding/json"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
)
func put(t *testing.T, root, name, content string) {
t.Helper()
p := filepath.Join(root, name)
if err := os.MkdirAll(filepath.Dir(p), 0755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(p, []byte(content), 0644); err != nil {
t.Fatal(err)
}
}
func get(t *testing.T, root, name string) string {
t.Helper()
data, err := os.ReadFile(filepath.Join(root, name))
if err != nil {
t.Fatal(err)
}
return string(data)
}
func snapshot(t *testing.T, root string) map[string]string {
t.Helper()
files := map[string]string{}
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() {
data, e := os.ReadFile(path)
if e != nil {
return e
}
rel, _ := filepath.Rel(root, path)
files[rel] = string(data)
}
return nil
})
if err != nil {
t.Fatal(err)
}
return files
}
func fresh(t *testing.T) string {
t.Helper()
root := filepath.Join(t.TempDir(), "program")
if _, err := New(Options{Dir: root, Module: "example.com/program"}); err != nil {
t.Fatal(err)
}
return root
}
func TestOwnershipAndDrift(t *testing.T) {
root := fresh(t)
files := snapshot(t, root)
for name, source := range files {
if strings.HasSuffix(name, ".api") || strings.Contains(name, "handler") || strings.Contains(name, "openapi") {
t.Fatal("unexpected API output", name)
}
if strings.HasSuffix(name, ".go") {
for _, forbidden := range []string{"net/http", "httpx", "middleware", "ListenAndServe", "database/", "queue/", "task.New"} {
if strings.Contains(source, forbidden) {
t.Fatal(name, forbidden)
}
}
}
}
if strings.Contains(files["go.mod"], "replace") || !strings.Contains(files["go.mod"], currentVersion) {
t.Fatal("default dependency must be remote and fixed")
}
for _, name := range seeds {
if name != "go.mod" {
put(t, root, name, get(t, root, name)+"\n// project edit\n")
}
}
before := snapshot(t, root)
for _, o := range []Options{{Dir: root}, {Dir: root, Check: true}, {Dir: root, DryRun: true}} {
out, err := Generate(o)
if err != nil || len(out.Changes) != 0 {
t.Fatal(out, err)
}
}
if !reflect.DeepEqual(before, snapshot(t, root)) {
t.Fatal("project files changed")
}
original := get(t, root, "main.go")
put(t, root, "main.go", original+"\n// manual edit\n")
for _, o := range []Options{{Dir: root}, {Dir: root, Check: true}, {Dir: root, DryRun: true}} {
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "was edited") {
t.Fatal(err)
}
}
put(t, root, "main.go", original)
if err := os.Remove(filepath.Join(root, "main.go")); err != nil {
t.Fatal(err)
}
before = snapshot(t, root)
if _, err := Generate(Options{Dir: root, Check: true}); err == nil {
t.Fatal("missing output accepted")
}
if !reflect.DeepEqual(before, snapshot(t, root)) {
t.Fatal("check wrote files")
}
out, err := Generate(Options{Dir: root, DryRun: true})
if err != nil || len(out.Changes) != 1 {
t.Fatal(out, err)
}
if !reflect.DeepEqual(before, snapshot(t, root)) {
t.Fatal("dry-run wrote files")
}
if _, err = Generate(Options{Dir: root}); err != nil {
t.Fatal(err)
}
if get(t, root, "main.go") != original {
t.Fatal("missing output not restored")
}
if err = os.Remove(filepath.Join(root, "internal/run/run.go")); err != nil {
t.Fatal(err)
}
if _, err = Generate(Options{Dir: root}); err == nil || !strings.Contains(err.Error(), "project-owned") {
t.Fatal("silently recreated handwritten file", err)
}
}
func TestRecordedTemplateUpgrade(t *testing.T) {
root := fresh(t)
var m manifest
if err := json.Unmarshal([]byte(get(t, root, manifestPath)), &m); err != nil {
t.Fatal(err)
}
original := get(t, root, "main.go")
older := original + "\n// previous generator baseline\n"
put(t, root, "main.go", older)
m.Generated["main.go"] = digest([]byte(older))
data, _ := json.Marshal(m)
put(t, root, manifestPath, string(data))
before := snapshot(t, root)
if _, err := Generate(Options{Dir: root, Check: true}); err == nil {
t.Fatal("stale baseline accepted")
}
if !reflect.DeepEqual(before, snapshot(t, root)) {
t.Fatal("check mutated baseline")
}
if _, err := Generate(Options{Dir: root}); err != nil {
t.Fatal(err)
}
if get(t, root, "main.go") != original {
t.Fatal("template was not updated")
}
}
func TestProjectGuards(t *testing.T) {
for _, tc := range []struct {
name, needle string
mutate func(*testing.T, string)
}{
{"module", "module differs", func(t *testing.T, r string) {
put(t, r, "go.mod", strings.Replace(get(t, r, "go.mod"), "example.com/program", "example.com/other", 1))
}},
{"oldFramework", "must require", func(t *testing.T, r string) {
put(t, r, "go.mod", strings.ReplaceAll(get(t, r, "go.mod"), currentVersion, "v0.22.0-preview"))
}},
{"manifest", "unsupported", func(t *testing.T, r string) { put(t, r, manifestPath, `{"version":1,"kind":"api"}`) }},
{"lock", "project.lock", func(t *testing.T, r string) { put(t, r, lockPath, "occupied") }},
{"otherGenerator", "existing API", func(t *testing.T, r string) { put(t, r, ".goOmni/apiManifest.json", "{}") }},
{"symlink", "symlink", func(t *testing.T, r string) {
if e := os.Remove(filepath.Join(r, "main.go")); e != nil {
t.Fatal(e)
}
target := filepath.Join(t.TempDir(), "outside.go")
put(t, filepath.Dir(target), "outside.go", "outside")
if e := os.Symlink(target, filepath.Join(r, "main.go")); e != nil {
t.Fatal(e)
}
}},
} {
t.Run(tc.name, func(t *testing.T) {
root := fresh(t)
tc.mutate(t, root)
before := snapshot(t, root)
if _, err := Generate(Options{Dir: root}); err == nil || !strings.Contains(err.Error(), tc.needle) {
t.Fatal(err)
}
if !reflect.DeepEqual(before, snapshot(t, root)) {
t.Fatal("failed generation changed files")
}
})
}
root := fresh(t)
before := snapshot(t, root)
if _, err := New(Options{Dir: root}); err == nil {
t.Fatal("new overwrote project")
}
if !reflect.DeepEqual(before, snapshot(t, root)) {
t.Fatal("new mutated project")
}
if _, err := Generate(Options{Dir: t.TempDir(), Check: true}); err == nil {
t.Fatal("adopted unmanaged directory")
}
for _, version := range []string{"v0.27.0-preview", "latest", "v2.0.0", "v0.28.0-preview\nreplace bad"} {
if _, err := New(Options{Dir: filepath.Join(t.TempDir(), "app"), FrameworkVersion: version}); err == nil {
t.Fatal("invalid version", version)
}
}
if _, err := New(Options{Dir: filepath.Join(t.TempDir(), "app"), Module: "example.com/a\nreplace bad"}); err == nil {
t.Fatal("invalid module")
}
}

View File

@@ -0,0 +1,6 @@
/program
/var/
.env
.env.*
!.env.example
/.goOmni/project.lock

View File

@@ -0,0 +1,55 @@
# Go program
This project has no HTTP listener, routes, tasks or external dependencies configured.
```sh
go mod tidy
go run . -check-config
go run .
# SIGINT/SIGTERM stops the program.
go build -o program .
go-omni project generate --dry-run
go-omni project generate
go-omni project check
go test ./...
```
Use the same fixed CLI and framework version. Local --framework-dir is only for
explicit source development; normal creation pins the installed CLI version.
Edit internal/config/config.go (Config/Default/Validate), etc/config.yaml and
internal/config/loader.go for optional decoders. JSON/YAML/XML resolve only
explicit ${NAME} / ${NAME:-fallback}; fixed values are never implicitly overridden.
-check-config validates without opening the logger or dependencies.
Add dependencies in internal/svc/serviceContext.go, registering OnClose immediately.
Register all OnStop/OnDrain hooks in Init, before starting work. Paired hooks use the
same name. OnStop stops acquisition; OnDrain returns nil only after all processing,
acknowledgements and defers finish. Runtime executes stop, then drain, then resource
cleanup in reverse registration order. Callbacks within a phase run concurrently;
combine components with ordering requirements in a single hook. Accepted work uses
WorkContext; Context is the early stop notification. No automatic retry or restart
policy is installed. Adapt external components according to their actual semantics.
Write program behavior in internal/run/run.go. Return nil when a one-shot run finishes;
return an error for fatal failure. The default Run waits for shutdown. Components may
call RequestShutdown(err) to report fatal failure. Signal or Run completion starts
shutdown; the entry waits for Run itself and registered background work. Do not make
Run wait for resource cleanup: cleanup follows Run completion. Do not register hooks
concurrently from Run. The drain name go-omni/project/run is reserved.
Run/Init panics are converted to startup/run failures and registered resources are
cleaned safely. Panics in arbitrary project goroutines must be handled by the project.
Unresponsive work is bounded by lifecycle timeouts: unfinished work causes
ErrResourcesRetained, skips shared resource/logger cleanup and exits unsuccessfully.
Go cannot forcibly stop a goroutine. SIGKILL/process crashes cannot run cleanup.
main.go, internal/config/loadGen.go and internal/svc/runtimeGen.go are generator-owned.
Manual edits cause a conflict. Other starter files are project-owned and preserved;
missing project-owned files require explicit restoration. Check is read-only and
reports missing or stale generated files. It does not compile or execute project code.
Commit .goOmni/projectManifest.json with the project. Generation uses a project lock,
preflight checks and per-file atomic replacement with rollback for ordinary write errors;
it is not crash-atomic across files. Avoid editing files during generation.
Full guide: https://git.yuntuo.hk/k2safe/go-omni/src/branch/main/PROGRAM.md

View File

@@ -0,0 +1,9 @@
# Project-owned. Environment references are explicit: ${NAME} or ${NAME:-fallback}.
log:
level: info
format: json
output: stderr
lifecycle:
stopTimeout: 5s
drainTimeout: 10s
cancelTimeout: 5s

View File

@@ -0,0 +1,22 @@
package config
import (
omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
// Config, Default and Validate are project-owned. Add project settings here.
type Config struct {
Log logx.Config `json:"log"`
Lifecycle lifecycle.ShutdownConfig `json:"lifecycle"`
}
func Default() Config {
log:=logx.Default();log.Service="PROJECT_MODULE"
return Config{Log:log,Lifecycle:lifecycle.DefaultShutdownConfig()}
}
func(c Config)Validate()error {
if err:=c.Log.Validate();err!=nil{return omniConfig.AtField("log",err)}
if err:=c.Lifecycle.Validate();err!=nil{return omniConfig.AtField("lifecycle",err)}
return nil
}

View File

@@ -0,0 +1,6 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
func Load(path string)(Config,error){
loader,err:=NewLoader();if err!=nil{return Config{},err}
return omniConfig.LoadProject(path,Default(),loader)
}

View File

@@ -0,0 +1,4 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// NewLoader is project-owned. Register additional decoders here if needed.
func NewLoader()(*omniConfig.Loader,error){return omniConfig.NewLoader()}

View File

@@ -0,0 +1,13 @@
package run
import (
"context"
"PROJECT_MODULE/internal/svc"
)
// Run is project-owned. Return nil for normal completion, or an error to fail.
// The starter waits for shutdown; a one-shot program can simply return when done.
// Register background stop/drain hooks in svc.Init, not concurrently here.
// Accepted background work uses deps.WorkContext(), not the early-stop ctx.
func Run(ctx context.Context,deps *svc.ServiceContext)error {
<-ctx.Done()
return nil
}

View File

@@ -0,0 +1,23 @@
package svc
import (
"context"
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
"PROJECT_MODULE/internal/config"
)
func New(ctx context.Context,logger *logx.Logger,cfg config.Config)(result *ServiceContext,err error){
if ctx==nil{ctx=context.Background()}
if err=ctx.Err();err!=nil{return nil,errors.Join(err,lifecycle.Cause(ctx))}
if logger==nil{return nil,errors.New("service context requires a logger")}
if err=cfg.Validate();err!=nil{return nil,err}
runtime,err:=lifecycle.NewRuntimeWithConfig(logx.WithLogger(ctx,logger),cfg.Lifecycle);if err!=nil{return nil,err}
c:=&ServiceContext{Runtime:runtime,Config:cfg,Logger:logger}
defer func(){
if p:=recover();p!=nil{err=fmt.Errorf("dependency initialization panic: %v",p)}
if err!=nil{result=nil;err=c.Finish(err)}
}()
if err=c.Init(c.Context());err!=nil{return nil,err}
return c,nil
}

View File

@@ -0,0 +1,21 @@
package svc
import (
"context"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
"PROJECT_MODULE/internal/config"
)
// ServiceContext is project-owned. Add dependencies without global variables.
// Runtime owns registered resources; Logger is borrowed from the entry point.
type ServiceContext struct {
*lifecycle.Runtime
Config config.Config
Logger *logx.Logger
}
// Init runs before Run. Register resources immediately with OnClose.
// Register paired OnStop/OnDrain hooks before exposing background work.
// Use WorkContext for accepted work; Context is the early stop notification.
// OnDrain returns nil only after processing and final cleanup have completed.
// Complete all registrations here, before Run can race with shutdown.
// The drain name "go-omni/project/run" is reserved for the generated entry.
func(c *ServiceContext)Init(ctx context.Context)error{return nil}

View File

@@ -0,0 +1,60 @@
package main
import (
"context"
"errors"
"flag"
"fmt"
"log"
"os"
"os/signal"
"syscall"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
"PROJECT_MODULE/internal/config"
program "PROJECT_MODULE/internal/run"
"PROJECT_MODULE/internal/svc"
)
func main() {
path:=flag.String("config","etc/config.yaml","configuration file; empty uses Default()")
check:=flag.Bool("check-config",false,"validate configuration without initializing dependencies")
flag.Parse()
if *check {if _,err:=config.Load(*path);err!=nil{log.Fatal(err)};return}
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
defer stop()
if err:=run(ctx,*path);err!=nil{log.Fatal(err)}
}
func run(ctx context.Context,path string)(runErr error) {
cfg,err:=config.Load(path);if err!=nil{return err}
logger,err:=logx.New(cfg.Log);if err!=nil{return err}
var deps *svc.ServiceContext
defer func(){
if p:=recover();p!=nil{runErr=fmt.Errorf("program initialization panic: %v",p)}
if deps!=nil{runErr=deps.Finish(runErr)}
// Unjoined work may still use its dependencies and logger. The process exits
// unsuccessfully; resource retention never means the work completed.
if !errors.Is(runErr,lifecycle.ErrResourcesRetained){runErr=errors.Join(runErr,logger.Close())}
}()
deps,err=svc.New(ctx,logger,cfg);if err!=nil{return err}
if deps.Context().Err()!=nil{return nil}
done:=make(chan struct{})
// This reserved drain proves Run has actually returned, including its defers.
// Runtime bounds the wait even when Run ignores cancellation.
deps.OnDrain("go-omni/project/run",func(context.Context)error{<-done;return nil})
logger.InfoContext(deps.Context(),"program.starting")
go func(){
var result error
returned:=false
defer func(){
if p:=recover();p!=nil{result=fmt.Errorf("program run panic: %v",p)} else if !returned{result=errors.New("program run exited without returning")}
deps.RequestShutdown(result)
close(done)
}()
result=program.Run(deps.Context(),deps)
returned=true
}()
<-deps.Context().Done()
return nil
}

View File

@@ -0,0 +1,283 @@
package main
import (
"context"
"errors"
"os"
"path/filepath"
"reflect"
"runtime"
"strings"
"sync/atomic"
"testing"
"time"
"example.com/plain/internal/config"
program "example.com/plain/internal/run"
"example.com/plain/internal/svc"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
)
func setup(t *testing.T) {
t.Helper()
svc.Initialize = func(context.Context, *svc.ServiceContext) error { return nil }
program.Execute = func(context.Context, *svc.ServiceContext) error { return nil }
t.Cleanup(func() {
svc.Initialize = func(context.Context, *svc.ServiceContext) error { return nil }
program.Execute = func(context.Context, *svc.ServiceContext) error { return nil }
})
}
func shortConfig(t *testing.T, logfile string) string {
t.Helper()
p := filepath.Join(t.TempDir(), "config.yaml")
text := "log:\n output: discard\nlifecycle:\n stopTimeout: 100ms\n drainTimeout: 30ms\n cancelTimeout: 100ms\n"
if logfile != "" {
text = "log:\n file: " + logfile + "\nlifecycle:\n stopTimeout: 100ms\n drainTimeout: 30ms\n cancelTimeout: 100ms\n"
}
if err := os.WriteFile(p, []byte(text), 0600); err != nil {
t.Fatal(err)
}
return p
}
func TestFiniteRunAndCleanupOrder(t *testing.T) {
setup(t)
var events []string
svc.Initialize = func(ctx context.Context, c *svc.ServiceContext) error {
events = append(events, "init")
c.OnClose(func() error { events = append(events, "first"); return nil })
c.OnClose(func() error { events = append(events, "second"); return nil })
return nil
}
program.Execute = func(context.Context, *svc.ServiceContext) error {
defer func() { events = append(events, "run defer") }()
events = append(events, "run")
return nil
}
if err := run(context.Background(), ""); err != nil {
t.Fatal(err)
}
if !reflect.DeepEqual(events, []string{"init", "run", "run defer", "second", "first"}) {
t.Fatal(events)
}
}
func TestInitializationAndRunFailures(t *testing.T) {
for _, stage := range []string{"init error", "init panic", "run error", "run panic", "run goexit", "component failure"} {
t.Run(stage, func(t *testing.T) {
setup(t)
boom := errors.New("failure marker")
var closed atomic.Int32
var started atomic.Bool
svc.Initialize = func(ctx context.Context, c *svc.ServiceContext) error {
c.OnClose(func() error { closed.Add(1); return nil })
if stage == "init error" {
return boom
}
if stage == "init panic" {
panic(boom)
}
return nil
}
program.Execute = func(ctx context.Context, c *svc.ServiceContext) error {
started.Store(true)
switch stage {
case "run error":
return boom
case "run panic":
panic(boom)
case "run goexit":
runtime.Goexit()
case "component failure":
c.RequestShutdown(boom)
<-ctx.Done()
}
return nil
}
err := run(context.Background(), "")
if err == nil || closed.Load() != 1 {
t.Fatal(stage, err, closed.Load())
}
if strings.HasPrefix(stage, "init") && started.Load() {
t.Fatal("run started after failed initialization")
}
if strings.Contains(stage, "error") || stage == "component failure" {
if !errors.Is(err, boom) {
t.Fatal("failure cause lost", err)
}
}
})
}
}
func TestGracefulStopDrainsAcknowledgementBeforeClose(t *testing.T) {
setup(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
ready := make(chan struct{})
stop := make(chan struct{})
done := make(chan struct{})
var closed, acked atomic.Bool
var canceledEarly atomic.Bool
svc.Initialize = func(ctx context.Context, c *svc.ServiceContext) error {
c.OnClose(func() error {
if !acked.Load() {
return errors.New("closed before acknowledgement")
}
closed.Store(true)
return nil
})
c.OnStop("component", func(context.Context) error { close(stop); return nil })
c.OnDrain("component", func(context.Context) error { <-done; return nil })
go func() {
defer close(done)
<-stop
if c.WorkContext().Err() != nil {
canceledEarly.Store(true)
}
time.Sleep(20 * time.Millisecond)
acked.Store(true)
}()
return nil
}
program.Execute = func(ctx context.Context, c *svc.ServiceContext) error { close(ready); <-ctx.Done(); return nil }
result := make(chan error, 1)
go func() { result <- run(ctx, "") }()
<-ready
cancel()
if err := <-result; err != nil || !closed.Load() || canceledEarly.Load() {
t.Fatal(err, closed.Load(), canceledEarly.Load())
}
}
func TestInitializationFailureDrainsStartedWork(t *testing.T) {
setup(t)
done := make(chan struct{})
stop := make(chan struct{})
var closed atomic.Bool
svc.Initialize = func(ctx context.Context, c *svc.ServiceContext) error {
c.OnClose(func() error {
select {
case <-done:
closed.Store(true)
return nil
default:
return errors.New("resource closed while starting work still active")
}
})
c.OnStop("partial", func(context.Context) error { close(stop); return nil })
c.OnDrain("partial", func(context.Context) error { <-done; return nil })
go func() { defer close(done); <-stop; time.Sleep(10 * time.Millisecond) }()
return errors.New("later dependency failed")
}
if err := run(context.Background(), ""); err == nil || !closed.Load() {
t.Fatal(err, closed.Load())
}
}
func TestCancellationGraceJoinsWork(t *testing.T) {
setup(t)
done := make(chan struct{})
var closed atomic.Bool
svc.Initialize = func(ctx context.Context, c *svc.ServiceContext) error {
c.OnClose(func() error { closed.Store(true); return nil })
c.OnStop("cooperative", func(context.Context) error { return nil })
c.OnDrain("cooperative", func(context.Context) error { <-done; return nil })
go func() { <-c.WorkContext().Done(); close(done) }()
return nil
}
err := run(context.Background(), shortConfig(t, ""))
if !errors.Is(err, context.DeadlineExceeded) || errors.Is(err, lifecycle.ErrResourcesRetained) || !closed.Load() {
t.Fatal(err, closed.Load())
}
}
func TestUnjoinedWorkRetainsDependenciesAndLogger(t *testing.T) {
for _, which := range []string{"run", "background", "init failure"} {
t.Run(which, func(t *testing.T) {
setup(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
release := make(chan struct{})
joined := make(chan struct{})
ready := make(chan struct{})
var closed atomic.Bool
var held *svc.ServiceContext
logfile := filepath.Join(t.TempDir(), "retained.log")
svc.Initialize = func(ctx context.Context, c *svc.ServiceContext) error {
held = c
c.OnClose(func() error { closed.Store(true); return nil })
if which != "run" {
c.OnStop("stuck", func(context.Context) error { return nil })
c.OnDrain("stuck", func(context.Context) error { <-joined; return nil })
go func() { defer close(joined); <-release; c.Logger.Info("after shutdown deadline") }()
}
if which == "init failure" {
return errors.New("initialization failed")
}
return nil
}
program.Execute = func(ctx context.Context, c *svc.ServiceContext) error {
close(ready)
if which == "run" {
defer close(joined)
<-release
c.Logger.Info("after shutdown deadline")
} else {
<-ctx.Done()
}
return nil
}
result := make(chan error, 1)
go func() { result <- run(ctx, shortConfig(t, logfile)) }()
if which != "init failure" {
<-ready
cancel()
}
var err error
select {
case err = <-result:
case <-time.After(3 * time.Second):
t.Fatal("shutdown hung")
}
if !errors.Is(err, lifecycle.ErrResourcesRetained) || closed.Load() {
t.Fatal(err, closed.Load())
}
close(release)
<-joined
// In the run case the entry's completion notification follows the user defer.
// Close is idempotent and must retain resources even after late completion.
if err = held.Close(); !errors.Is(err, lifecycle.ErrResourcesRetained) || closed.Load() {
t.Fatal("retention was lost", err)
}
data, _ := os.ReadFile(logfile)
if !strings.Contains(string(data), "after shutdown deadline") {
t.Fatal("logger closed while work still running", string(data))
}
_ = held.Logger.Close()
})
}
}
func TestExplicitConfigurationAndPreflight(t *testing.T) {
setup(t)
t.Setenv("PLAIN_LEVEL", "debug")
t.Setenv("LOG_LEVEL", "error")
p := filepath.Join(t.TempDir(), "config.yaml")
if err := os.WriteFile(p, []byte("log:\n level: ${PLAIN_LEVEL}\n"), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.Load(p)
if err != nil || cfg.Log.Level != "debug" {
t.Fatal(cfg.Log.Level, err)
}
if err = os.WriteFile(p, []byte("log:\n level: info\n"), 0600); err != nil {
t.Fatal(err)
}
cfg, err = config.Load(p)
if err != nil || cfg.Log.Level != "info" {
t.Fatal("implicit environment override", err)
}
var initCalled atomic.Bool
svc.Initialize = func(context.Context, *svc.ServiceContext) error { initCalled.Store(true); return nil }
if err = os.WriteFile(p, []byte("lifecycle:\n drainTimeout: -1s\n"), 0600); err != nil {
t.Fatal(err)
}
if err = run(context.Background(), p); err == nil || initCalled.Load() {
t.Fatal("invalid configuration initialized dependencies", err)
}
}

View File

@@ -1,5 +1,6 @@
# Only fields listed here override Default(). Environment names are explicit.
server:
# Optional cross-origin access: server.cors (disabled by default). See framework cors/README.md.
address: "${GO_OMNI_HTTP_ADDR:-127.0.0.1:8080}"
transport: "${GO_OMNI_TRANSPORT:-nethttp}"
{{if ne .Storage "none"}}storage:

View File

@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Source builds can carry an unpublished VCS version (including +dirty).
// Keep this fallback in sync with the next release; installed CLIs use their
// downloaded module version, which also has a module checksum.
const defaultFrameworkVersion = "v0.22.0-preview"
const defaultFrameworkVersion = "v0.28.1-preview"
// This target is not a claim that the release already exists. Source users
// select a local framework checkout until the module templates are published.

View File

@@ -0,0 +1,162 @@
package main
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
// The same consumer is exercised with a source-built CLI before release and a
// downloaded CLI after release. Project modules are always independently built.
func TestMixedWorkspaceConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
root := t.TempDir()
cli := os.Getenv("GO_OMNI_RELEASE_CLI")
local := cli == ""
framework, err := filepath.Abs("../..")
if err != nil {
t.Fatal(err)
}
if local {
cli = filepath.Join(t.TempDir(), "go-omni")
workspaceCommand(t, "", nil, false, "go", "build", "-o", cli, ".")
}
modulePath := "git.yuntuo.hk/k2safe/go-omni"
for _, kind := range []string{"api", "project"} {
dir := filepath.Join(root, kind)
args := []string{kind, "new", dir, "--module", "example.com/" + kind}
if local {
args = append(args, "--framework-dir", framework)
}
workspaceCommand(t, root, nil, false, cli, args...)
workspaceCommand(t, dir, nil, false, "go", "mod", "tidy")
data := workspaceCommand(t, dir, nil, false, "go", "list", "-m", "-json", modulePath)
if !local {
var dep struct {
Version, Sum string
Replace any
}
if err := json.Unmarshal([]byte(data), &dep); err != nil {
t.Fatal(err)
}
if dep.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dep.Replace != nil || dep.Sum == "" {
t.Fatal("consumer is not a fixed remote dependency", dep)
}
t.Logf("%s remote framework %s checksum %s; no local replace", kind, dep.Version, dep.Sum)
}
}
write := func(name, data string) {
t.Helper()
if err := os.WriteFile(filepath.Join(root, name), []byte(data), 0644); err != nil {
t.Fatal(err)
}
}
write("omni.workspace.json", `{"services":[{"dir":"api","kind":"api"},{"dir":"project","kind":"project"}]}`)
for _, op := range []string{"format", "generate", "check"} {
output := workspaceCommand(t, root, nil, false, cli, "workspace", op)
if op == "format" && !strings.Contains(output, "SKIP ") {
t.Fatal("missing project format skip", output)
}
}
// Run the actual Go multi-module toolchain independently of generation.
workspaceCommand(t, root, nil, false, "go", "work", "init", "./api", "./project")
work, err := filepath.EvalSymlinks(filepath.Join(root, "go.work"))
if err != nil {
t.Fatal(err)
}
workspaceCommand(t, root, []string{"GOWORK=" + work}, false, "go", "test", "-race", "./api/...", "./project/...")
workspaceCommand(t, root, []string{"GOWORK=" + work}, false, "go", "build", "./api/...", "./project/...")
before := workspaceSnapshot(t, root)
workspaceCommand(t, root, nil, false, cli, "workspace", "check")
if !reflect.DeepEqual(before, workspaceSnapshot(t, root)) {
t.Fatal("check modified Go workspace/module files")
}
// Omitted kinds discover both released manifests as well.
write("omni.workspace.json", `{"services":[{"dir":"api"},{"dir":"project"}]}`)
workspaceCommand(t, root, nil, false, cli, "workspace", "check")
originals := map[string]string{}
for _, kind := range []string{"api", "project"} {
name := kind + "/main.go"
data, err := os.ReadFile(filepath.Join(root, name))
if err != nil {
t.Fatal(err)
}
originals[name] = string(data)
write(name, string(data)+"\n// intentional drift\n")
}
before = workspaceSnapshot(t, root)
output := workspaceCommand(t, root, nil, true, cli, "workspace", "check")
if strings.Count(output, "FAIL ") != 2 {
t.Fatal("did not aggregate both projects", output)
}
if !reflect.DeepEqual(before, workspaceSnapshot(t, root)) {
t.Fatal("failed check wrote files")
}
// The healthy project's missing output must stay missing when another fails.
write("api/main.go", originals["api/main.go"])
if err := os.Remove(filepath.Join(root, "api/main.go")); err != nil {
t.Fatal(err)
}
before = workspaceSnapshot(t, root)
workspaceCommand(t, root, nil, true, cli, "workspace", "generate")
if !reflect.DeepEqual(before, workspaceSnapshot(t, root)) {
t.Fatal("partial workspace generation")
}
write("project/main.go", originals["project/main.go"])
workspaceCommand(t, root, nil, false, cli, "workspace", "generate")
workspaceCommand(t, root, nil, false, cli, "workspace", "check")
}
func workspaceCommand(t *testing.T, dir string, extra []string, wantFailure bool, name string, args ...string) string {
t.Helper()
ctx, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
cmd := exec.CommandContext(ctx, name, args...)
if dir != "" {
cmd.Dir = dir
}
cmd.Env = append(append(os.Environ(), "GOWORK=off"), extra...)
data, err := cmd.CombinedOutput()
if wantFailure {
exit, ok := err.(*exec.ExitError)
if !ok || exit.ExitCode() != 1 {
t.Fatalf("expected exit 1 from %s %v: %v\n%s", name, args, err, data)
}
} else if err != nil {
t.Fatalf("%s %v: %v\n%s", name, args, err, data)
}
t.Logf("%s %v\n%s", name, args, data)
return string(data)
}
func workspaceSnapshot(t *testing.T, root string) map[string]string {
t.Helper()
files := map[string]string{}
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
if err != nil {
return err
}
if !d.IsDir() {
data, e := os.ReadFile(path)
if e != nil {
return e
}
rel, e := filepath.Rel(root, path)
if e != nil {
return e
}
files[rel] = string(data)
}
return nil
})
if err != nil {
t.Fatal(err)
}
return files
}

View File

@@ -5,8 +5,10 @@ import (
"time"
"git.yuntuo.hk/k2safe/go-omni/contract"
"git.yuntuo.hk/k2safe/go-omni/cors"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/httpx"
"git.yuntuo.hk/k2safe/go-omni/i18n/httpi18n"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
"git.yuntuo.hk/k2safe/go-omni/logx"
"git.yuntuo.hk/k2safe/go-omni/response"
@@ -28,6 +30,7 @@ type TLSConfig struct {
// Config controls the default runtime behavior of an App.
type Config struct {
CORS cors.Config `json:"cors"`
Request contract.RequestPolicy `json:"request"`
Address string `json:"address"`
Transport string `json:"transport"`
@@ -45,6 +48,8 @@ type Config struct {
Response response.Config `json:"response"`
ResponseEncoder response.Encoder `json:"-"`
LogInstance *logx.Logger `json:"-"`
LanguagePolicy *httpi18n.Policy `json:"-"`
ErrorLocalizer apierrors.Localizer `json:"-"`
}
func Default() Config {
@@ -61,6 +66,9 @@ func Default() Config {
// Validate checks configuration values before an application starts.
func (c Config) Validate() error {
if err := c.CORS.Validate(); err != nil {
return AtField("cors", err)
}
if err := c.Lifecycle.Validate(); err != nil {
return AtField("lifecycle", err)
}
@@ -188,12 +196,14 @@ func WithTrustedProxies(proxies ...string) Option {
// WithConfig applies a complete configuration loaded from a file or
// environment. Later options can still override individual fields.
func WithConfig(settings Config) Option {
settings.CORS = settings.CORS.Clone()
settings.Response = settings.Response.Clone()
settings.Errors = settings.Errors.Clone()
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
return func(cfg *Config) {
*cfg = settings
cfg.CORS = settings.CORS.Clone()
cfg.Response = settings.Response.Clone()
cfg.Errors = settings.Errors.Clone()
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
@@ -201,6 +211,12 @@ func WithConfig(settings Config) Option {
}
}
// WithCORS installs an app-local CORS policy, before routing and authentication.
func WithCORS(settings cors.Config) Option {
settings = settings.Clone()
return func(cfg *Config) { cfg.CORS = settings.Clone() }
}
// WithResponse configures this application's default JSON envelope.
func WithResponse(settings response.Config) Option {
settings = settings.Clone()
@@ -222,7 +238,8 @@ func WithLogConfig(settings logx.Config) Option {
}
}
// WithShutdownTimeout controls how long RunContext waits for active requests.
// WithShutdownTimeout controls the graceful HTTP drain budget. After timeout,
// HTTP.ShutdownCancelTimeout separately bounds the wait for canceled handlers.
func WithShutdownTimeout(timeout time.Duration) Option {
return func(cfg *Config) {
if timeout > 0 {
@@ -240,6 +257,15 @@ func WithErrors(catalog apierrors.Catalog) Option {
// WithLogInstance borrows a logger for this App without changing process logging.
func WithLogInstance(logger *logx.Logger) Option { return func(c *Config) { c.LogInstance = logger } }
// WithLanguagePolicy borrows an immutable HTTP language policy. Resource loading
// remains explicit in the project; configuration strings are not dictionaries.
func WithLanguagePolicy(policy *httpi18n.Policy) Option {
return func(c *Config) { c.LanguagePolicy = policy }
}
func WithErrorLocalizer(localizer apierrors.Localizer) Option {
return func(c *Config) { c.ErrorLocalizer = localizer }
}
func WithRequestPolicy(policy contract.RequestPolicy) Option {
return func(c *Config) { c.Request = policy }
}

51
config/cors_test.go Normal file
View File

@@ -0,0 +1,51 @@
package config_test
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
type corsProject struct {
Server config.Config `json:"server"`
}
func (c corsProject) Validate() error { return config.AtField("server", c.Server.Validate()) }
func TestCORSConfiguration(t *testing.T) {
t.Setenv("WEB_ORIGIN", "https://web.example")
for ext, source := range map[string]string{
"yaml": "server:\n cors:\n enabled: true\n allowedOrigins:\n - '${WEB_ORIGIN}'\n allowedMethods: [GET, POST]\n allowedHeaders: [Authorization, Content-Type]\n maxAge: 10m\n",
"json": `{"server":{"cors":{"enabled":true,"allowedOrigins":["${WEB_ORIGIN}"],"allowedMethods":["GET","POST"],"allowedHeaders":["Authorization","Content-Type"],"maxAge":"10m"}}}`,
"xml": `<config><server><cors><enabled type="bool">true</enabled><allowedOrigins type="array"><item>${WEB_ORIGIN}</item></allowedOrigins><allowedMethods type="array"><item>GET</item><item>POST</item></allowedMethods><allowedHeaders type="array"><item>Authorization</item><item>Content-Type</item></allowedHeaders><maxAge>10m</maxAge></cors></server></config>`,
} {
t.Run(ext, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "config."+ext)
if err := os.WriteFile(path, []byte(source), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.LoadProject(path, corsProject{Server: config.Default()})
if err != nil {
t.Fatal(err)
}
if !cfg.Server.CORS.Enabled || cfg.Server.CORS.AllowedOrigins[0] != "https://web.example" || cfg.Server.CORS.MaxAge != 10*time.Minute {
t.Fatal(cfg.Server.CORS)
}
})
}
path := filepath.Join(t.TempDir(), "bad.yaml")
if err := os.WriteFile(path, []byte("server:\n cors:\n enabled: true\n allowedOrigins:\n - '${WEB_ORIGIN}'\n"), 0600); err != nil {
t.Fatal(err)
}
t.Setenv("WEB_ORIGIN", "https://user:secret@web.example")
_, err := config.LoadProject(path, corsProject{Server: config.Default()})
var d *config.Error
if !errors.As(err, &d) || d.Field != "server.cors.allowedOrigins[0]" || d.Env != "WEB_ORIGIN" || d.Line != 5 || strings.Contains(err.Error(), "secret") {
t.Fatal(err)
}
}

View File

@@ -4,6 +4,8 @@ import (
"errors"
"fmt"
"strings"
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
)
// Error is a value-free diagnostic. Cause may contain private parser details;
@@ -56,6 +58,10 @@ func validationError(err error) *Error {
out.cause = err
return &out
}
var nested *configdiag.Error
if errors.As(err, &nested) {
return &Error{Field: nested.Field, Reason: nested.Reason, cause: err}
}
var c *ConfigError
if errors.As(err, &c) {
return &Error{Field: c.Field, Reason: c.Message, cause: err}

View File

@@ -3,17 +3,28 @@ package config
import (
"log"
"time"
"git.yuntuo.hk/k2safe/go-omni/internal/httplimits"
)
// HTTPConfig controls built-in HTTP servers. Positive values override the
// selected transport's defaults; zero keeps its existing behavior. Timeouts
// follow that transport's semantics, not a handler execution deadline.
// HTTPConfig controls HTTP serving. Network timeouts follow the selected
// transport's semantics, not a handler execution deadline. Body limits and
// shutdown cancellation grace also apply to App used with an external server.
type HTTPConfig struct {
ReadHeaderTimeout time.Duration `json:"readHeaderTimeout"`
ReadTimeout time.Duration `json:"readTimeout"`
WriteTimeout time.Duration `json:"writeTimeout"`
IdleTimeout time.Duration `json:"idleTimeout"`
MaxHeaderBytes int `json:"maxHeaderBytes"`
// DefaultMaxBodyBytes applies to App routes without an explicit limit.
// Zero uses 8 MiB, or the hard limit if smaller. Route limits may override it.
DefaultMaxBodyBytes int64 `json:"defaultMaxBodyBytes"`
// MaxBodyBytes is an optional service hard limit. Zero adds no hard limit
// below the framework's supported maximum of 1 TiB.
MaxBodyBytes int64 `json:"maxBodyBytes"`
// ShutdownCancelTimeout bounds the wait for handlers after cancellation.
// Zero uses 5s. Resources are retained if handlers have not returned.
ShutdownCancelTimeout time.Duration `json:"shutdownCancelTimeout"`
// ErrorLog is a Go-only hook for the nethttp server's internal diagnostics.
ErrorLog *log.Logger `json:"-"`
}
@@ -27,6 +38,7 @@ func (c HTTPConfig) Validate() error {
{"readTimeout", c.ReadTimeout},
{"writeTimeout", c.WriteTimeout},
{"idleTimeout", c.IdleTimeout},
{"shutdownCancelTimeout", c.ShutdownCancelTimeout},
} {
if field.value < 0 {
return &ConfigError{Field: "http." + field.name, Message: "must be nonnegative"}
@@ -35,11 +47,22 @@ func (c HTTPConfig) Validate() error {
if c.MaxHeaderBytes < 0 {
return &ConfigError{Field: "http.maxHeaderBytes", Message: "must be nonnegative"}
}
for _, field := range []struct {
name string
value int64
}{{"defaultMaxBodyBytes", c.DefaultMaxBodyBytes}, {"maxBodyBytes", c.MaxBodyBytes}} {
if field.value < 0 || field.value > httplimits.MaxBodyBytes {
return &ConfigError{Field: "http." + field.name, Message: "must be 0..1 TiB (bytes)"}
}
}
if c.MaxBodyBytes > 0 && c.DefaultMaxBodyBytes > c.MaxBodyBytes {
return &ConfigError{Field: "http.defaultMaxBodyBytes", Message: "must not exceed http.maxBodyBytes"}
}
return nil
}
// WithHTTPConfig sets the built-in server's HTTP options. RunWith uses a
// caller-owned Runner and does not apply these options to that runner.
// WithHTTPConfig sets HTTP options. RunWith leaves network configuration to the
// caller-owned Runner, but App body limits and cancellation grace still apply.
func WithHTTPConfig(settings HTTPConfig) Option {
return func(cfg *Config) { cfg.HTTP = settings }
}

View File

@@ -93,3 +93,40 @@ func TestHTTPInvalidAndUnsupportedSettings(t *testing.T) {
}
}
}
func TestHTTPBodyAndCancellationSettings(t *testing.T) {
for _, cfg := range []config.HTTPConfig{
{DefaultMaxBodyBytes: -1}, {MaxBodyBytes: -1}, {DefaultMaxBodyBytes: 1<<40 + 1}, {MaxBodyBytes: 1<<40 + 1},
{DefaultMaxBodyBytes: 1024, MaxBodyBytes: 512}, {ShutdownCancelTimeout: -time.Second},
} {
if err := cfg.Validate(); err == nil {
t.Fatalf("invalid HTTP config accepted: %+v", cfg)
}
}
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
cfg := config.Default()
cfg.Transport = transport
cfg.TLS = config.TLSConfig{CertFile: "c", KeyFile: "k"}
cfg.HTTP = config.HTTPConfig{DefaultMaxBodyBytes: 1 << 20, MaxBodyBytes: 16 << 20, ShutdownCancelTimeout: time.Second}
if err := cfg.ValidateTransport(); err != nil {
t.Fatal(err)
}
}
t.Setenv("OMNI_BODY_HARD", "16777216")
for _, file := range []struct{ name, body string }{
{"config.json", `{"http":{"defaultMaxBodyBytes":1048576,"maxBodyBytes":"${OMNI_BODY_HARD}","shutdownCancelTimeout":"250ms"}}`},
{"config.yaml", "http:\n defaultMaxBodyBytes: 1048576\n maxBodyBytes: ${OMNI_BODY_HARD}\n shutdownCancelTimeout: 250ms\n"},
} {
path := filepath.Join(t.TempDir(), file.name)
if err := os.WriteFile(path, []byte(file.body), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.LoadFile(path)
if err != nil {
t.Fatal(err)
}
if cfg.HTTP.DefaultMaxBodyBytes != 1<<20 || cfg.HTTP.MaxBodyBytes != 16<<20 || cfg.HTTP.ShutdownCancelTimeout != 250*time.Millisecond {
t.Fatalf("loaded %+v", cfg.HTTP)
}
}
}

View File

@@ -0,0 +1,85 @@
package config_test
import (
"errors"
"os"
"path/filepath"
"strings"
"testing"
"git.yuntuo.hk/k2safe/go-omni/config"
mysql "git.yuntuo.hk/k2safe/go-omni/database/mysql"
"git.yuntuo.hk/k2safe/go-omni/httpc"
)
type moduleConfig struct {
Database mysql.Config `json:"database"`
Server config.Config `json:"server"`
Client httpc.Config `json:"client"`
}
func (c moduleConfig) Validate() error {
if err := c.Database.Validate(); err != nil {
return config.AtField("database", err)
}
if err := c.Server.Validate(); err != nil {
return config.AtField("server", err)
}
return config.AtField("client", c.Client.Validate())
}
func TestModuleFieldDiagnostics(t *testing.T) {
for _, tc := range []struct{ name, source, field, reason string }{
{"port", "database:\n port: 70000\n", "database.port", "1 and 65535"},
{"host", "database:\n host: 'invalid host'\n", "database.host", "hostname"},
{"pool", "database:\n pool:\n maxIdleConns: 999\n", "database.pool.maxIdleConns", "maxOpenConns"},
{"ping", "database:\n pool:\n pingTimeout: 0s\n", "database.pool.pingTimeout", "positive"},
{"timeout", "database:\n readTimeout: -1s\n", "database.readTimeout", "nonnegative"},
{"mysqlZone", "database:\n location: Secret/Not_A_Zone\n", "database.location", "IANA"},
{"tlsPair", "database:\n tls:\n certFile: secret-file\n keyFile: ''\n", "database.tls.keyFile", "certFile"},
{"level", "server:\n log:\n level: secret-level\n", "server.log.level", "debug"},
{"format", "server:\n log:\n format: secret-format\n", "server.log.format", "json or text"},
{"rotation", "server:\n log:\n rotation:\n maxBackups: -1\n", "server.log.rotation.maxBackups", "10000"},
{"zone", "server:\n log:\n timeZone: Secret/Not_A_Zone\n", "server.log.timeZone", "IANA"},
{"lifecycle", "server:\n lifecycle:\n stopTimeout: -1s\n", "server.lifecycle.stopTimeout", "nonnegative"},
{"policy", "server:\n request:\n body: secret-mode\n", "server.request.body", "required"},
{"clientPool", "client:\n maxIdleConns: -1\n", "client.maxIdleConns", "131072"},
{"proxy", "client:\n proxy:\n mode: url\n url: 'https://private:secret@localhost/path'\n", "client.proxy.url", "origin"},
} {
t.Run(tc.name, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "config.yaml")
if err := os.WriteFile(path, []byte(tc.source), 0600); err != nil {
t.Fatal(err)
}
_, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()})
var d *config.Error
if !errors.As(err, &d) || d.Field != tc.field || d.File != path || d.Line <= 1 || d.Column <= 0 || !strings.Contains(d.Reason, tc.reason) {
t.Fatalf("diagnostic: %v", err)
}
if strings.Contains(err.Error(), "secret") || strings.Contains(err.Error(), "Secret/") || strings.Contains(err.Error(), "private") {
t.Fatalf("value leaked: %v", err)
}
})
}
}
func TestModuleDiagnosticsKeepReferenceAndCause(t *testing.T) {
t.Setenv("CHECK_PORT", "70000")
path := filepath.Join(t.TempDir(), "config.yaml")
os.WriteFile(path, []byte("database:\n port: '${CHECK_PORT}'\n"), 0600)
_, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()})
var d *config.Error
if !errors.As(err, &d) || d.Field != "database.port" || d.Env != "CHECK_PORT" || d.Line != 2 {
t.Fatal(err)
}
original := mysql.DefaultConfig()
original.Port = 70000
cause := original.Validate()
wrapped := config.AtField("database", cause)
if !errors.Is(wrapped, cause) {
t.Fatal("lost original validation error")
}
// Unknown project errors continue to be redacted rather than echoed.
wrapped = config.AtField("custom", errors.New("private-password"))
if strings.Contains(wrapped.Error(), "private-password") {
t.Fatal(wrapped)
}
}

View File

@@ -380,7 +380,10 @@ func decodeValue(dec *json.Decoder, depth int) (any, error) {
object[name] = value
}
end, err := dec.Token()
if err != nil || end != json.Delim('}') {
if err != nil {
return nil, err
}
if end != json.Delim('}') {
return nil, errors.New("unterminated object")
}
return object, nil
@@ -394,7 +397,10 @@ func decodeValue(dec *json.Decoder, depth int) (any, error) {
items = append(items, v)
}
end, err := dec.Token()
if err != nil || end != json.Delim(']') {
if err != nil {
return nil, err
}
if end != json.Delim(']') {
return nil, errors.New("unterminated array")
}
return items, nil

View File

@@ -3,9 +3,11 @@ package contract
import (
"context"
"errors"
"fmt"
"io"
"strings"
"unicode/utf8"
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
)
// RequestPolicy describes protocol strictness, independently of business
@@ -35,7 +37,7 @@ func (p RequestPolicy) Validate() error {
ok = ok || value == item.value
}
if !ok {
return fmt.Errorf("request policy: invalid %s", item.name)
return configdiag.New(item.name, "must be one of "+strings.Join(item.allowed, ", "))
}
}
return nil

View File

@@ -3,6 +3,7 @@ package contract
import (
"bytes"
"context"
"encoding/json"
"errors"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"io"
@@ -10,6 +11,7 @@ import (
"net/http/httptest"
"strings"
"testing"
"testing/iotest"
)
type byteReader struct{ r io.Reader }
@@ -34,6 +36,17 @@ func TestBodyPolicyPropagatesOuterLimit(t *testing.T) {
}
}
func TestJSONClosingDelimiterPreservesReaderCause(t *testing.T) {
cause := errors.New("body reader failed")
for _, prefix := range []string{"{", "[", `{"value":[]`, `[{}`} {
reader := io.MultiReader(byteReader{strings.NewReader(prefix)}, iotest.ErrReader(cause))
_, err := decodeValue(json.NewDecoder(reader), 0)
if !errors.Is(err, cause) {
t.Fatalf("prefix %q: lost reader cause: %v", prefix, err)
}
}
}
func TestStrictRequestPolicy(t *testing.T) {
type request struct {
Name string `json:"name"`

192
copyx/check.go Normal file
View File

@@ -0,0 +1,192 @@
package copyx
import (
"errors"
"fmt"
"reflect"
"sort"
)
// ErrDynamicType means a source interface requires a concrete runtime value.
// An exact converter can explicitly own that dynamic boundary.
var ErrDynamicType = errors.New("copyx: dynamic source type requires runtime validation or an exact converter")
// Check validates a source/target type pair using the same options as Copy.
// It never executes converters and requires no example values. Nil policies,
// converter failures, map-key collisions and deep reference isolation still
// require runtime checks. Options remain local; no mapping files or registration.
func Check[S, T any](options ...Option) error {
c, err := New(options...)
if err != nil {
return err
}
return c.Check(reflect.TypeFor[S](), reflect.TypeFor[T]())
}
// Check validates types with an existing Copier's immutable settings. The target
// is the result type (not Copy's extra output pointer). Safe for concurrent use.
func (c *Copier) Check(source, target reflect.Type) error {
if c == nil {
return errors.New("copyx: nil Copier")
}
if source == nil || target == nil {
return errors.New("copyx: source and target types are required")
}
return newTypeChecker(&c.options, false).check(source, target, "$", 0)
}
type typeChecker struct {
options *settings
allowDynamic bool
seen map[pair]bool
nodes int
}
func newTypeChecker(options *settings, allowDynamic bool) *typeChecker {
return &typeChecker{options: options, allowDynamic: allowDynamic, seen: map[pair]bool{}}
}
// Absent elements can reveal type/field mismatches, but contain no actual
// resource handles or dynamic interface values to validate. Runtime checks stay
// responsible for those value-dependent cases.
func (s *state) checkAbsent(from, to reflect.Type, path string) error {
if s.checkedTypes == nil {
s.checkedTypes = map[pair]bool{}
}
checker := typeChecker{options: s.options, allowDynamic: true, seen: s.checkedTypes, nodes: s.checkedNodes}
err := checker.check(from, to, path, 0)
s.checkedNodes = checker.nodes
return err
}
func (c *typeChecker) check(from, to reflect.Type, path string, depth int) error {
c.nodes++
if depth > maxDepth || c.nodes > maxNodes {
return at(path, ErrLimit)
}
key := pair{from, to}
if c.seen[key] {
return nil
}
c.seen[key] = true
if !c.allowDynamic && !shallowSafe(to) {
return at(path, ErrUnsupported)
}
if _, ok := c.options.converters[key]; ok {
return nil
}
if from == to && len(c.options.converters) == 0 && len(c.options.fields) == 0 && !c.options.skipZero && c.options.nilPolicy != NilSkip {
if !c.options.deep || simpleValue(to) {
return nil
}
}
if from.Kind() == reflect.Interface {
if c.allowDynamic {
return nil
}
return at(path, ErrDynamicType)
}
if to.Kind() == reflect.Interface {
if !from.Implements(to) {
return at(path, ErrType)
}
return c.check(from, from, path, depth+1)
}
if !c.allowDynamic && c.options.deep && forbidden(to) {
return at(path, ErrUnsupported)
}
if from.Kind() == reflect.Pointer && to.Kind() != reflect.Pointer {
return c.check(from.Elem(), to, path, depth+1)
}
if to.Kind() == reflect.Pointer {
if !c.options.deep && from == to {
return nil
}
if from.Kind() == reflect.Pointer {
from = from.Elem()
}
return c.check(from, to.Elem(), path, depth+1)
}
if from.Kind() != to.Kind() {
return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to))
}
if to == timeType && from == to {
return nil
}
switch to.Kind() {
case reflect.Struct:
for i := 0; i < to.NumField(); i++ {
f := to.Field(i)
if !c.allowDynamic && f.PkgPath != "" && (from != to || c.options.deep && privateMutable(f.Type)) {
return at(path+"."+f.Name, ErrUnsupported)
}
}
sources, targets := exportedFields(from), exportedFields(to)
rules := c.options.fields[key]
if rules == nil {
rules = &fields{}
}
reverse, used := map[string]string{}, map[string]bool{}
for a, b := range rules.names {
reverse[b] = a
}
for _, name := range sortedFieldNames(targets) {
if rules.ignoreTarget[name] {
continue
}
sourceName := name
if n, ok := reverse[name]; ok {
sourceName = n
}
source, ok := sources[sourceName]
if !ok || rules.ignoreSource[sourceName] {
return at(path+"."+name, ErrField)
}
used[sourceName] = true
if err := c.check(source.Type, targets[name].Type, path+"."+name, depth+1); err != nil {
return err
}
}
if c.options.strict {
for _, name := range sortedFieldNames(sources) {
if !used[name] && !rules.ignoreSource[name] {
return at(path+"."+name, ErrField)
}
}
}
return nil
case reflect.Slice:
if !c.options.deep && from == to {
return nil
}
return c.check(from.Elem(), to.Elem(), path+"[]", depth+1)
case reflect.Array:
if from.Len() != to.Len() {
return at(path, ErrType)
}
return c.check(from.Elem(), to.Elem(), path+"[]", depth+1)
case reflect.Map:
if !c.options.deep && from == to {
return nil
}
if err := c.check(from.Key(), to.Key(), path+"[key]", depth+1); err != nil {
return err
}
return c.check(from.Elem(), to.Elem(), path+"[value]", depth+1)
case reflect.Func, reflect.Chan, reflect.UnsafePointer:
if c.options.deep && !c.allowDynamic {
return at(path, ErrUnsupported)
}
}
if from != to {
return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to))
}
return nil
}
func sortedFieldNames(fields map[string]reflect.StructField) []string {
names := make([]string, 0, len(fields))
for name := range fields {
names = append(names, name)
}
sort.Strings(names)
return names
}

192
copyx/check_test.go Normal file
View File

@@ -0,0 +1,192 @@
package copyx_test
import (
"errors"
"reflect"
"strings"
"sync"
"testing"
"git.yuntuo.hk/k2safe/go-omni/copyx"
)
type checkSource struct{ Rows []checkSourceRow }
type checkSourceRow struct{ Value string }
type checkTarget struct{ Rows []checkTargetRow }
type checkTargetRow struct{ Value int }
type recursiveSource struct {
Children []*recursiveSource
Value string
}
type recursiveTarget struct {
Children []*recursiveTarget
Value int
}
type recursiveCompatible struct {
Children []*recursiveCompatible
Value string
}
func TestCheckTypesAndEmptyCollections(t *testing.T) {
err := copyx.Check[checkSource, checkTarget]()
var field *copyx.Error
if !errors.Is(err, copyx.ErrType) || !errors.As(err, &field) || field.Path != "$.Rows[].Value" {
t.Fatal(err)
}
for _, rows := range [][]checkSourceRow{nil, {}, {{Value: "42"}}} {
out := checkTarget{Rows: []checkTargetRow{{Value: 99}}}
if err := copyx.Copy(&out, checkSource{rows}); !errors.Is(err, copyx.ErrType) {
t.Fatalf("rows %v: %v", rows, err)
}
if len(out.Rows) != 1 || out.Rows[0].Value != 99 {
t.Fatal("partial target committed")
}
}
if err := copyx.Check[map[string]checkSourceRow, map[string]checkTargetRow](); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
var dst map[string]checkTargetRow
if err := copyx.Copy(&dst, map[string]checkSourceRow{}); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
var array [0]checkTargetRow
if err := copyx.Copy(&array, [0]checkSourceRow{}); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
if err := copyx.Check[map[string]int, map[int]int](); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
if err := copyx.Check[[2]int, [3]int](); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
if err := copyx.Check[*recursiveSource, *recursiveTarget](); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
if err := copyx.Check[*recursiveSource, *recursiveCompatible](copyx.WithDeepCopy()); err != nil {
t.Fatal(err)
}
}
func TestCheckRulesAndConverterBoundaries(t *testing.T) {
type from struct {
Name string
Extra bool
}
type to struct {
Label string
Keep int
}
options := []copyx.Option{copyx.WithStrictFields(), copyx.WithFieldNames[from, to](map[string]string{"Name": "Label"}), copyx.WithIgnoreSource[from, to]("Extra"), copyx.WithIgnoreTarget[from, to]("Keep")}
if err := copyx.Check[from, to](options...); err != nil {
t.Fatal(err)
}
if err := copyx.Check[from, to](); !errors.Is(err, copyx.ErrField) {
t.Fatal(err)
}
type short struct{ Name string }
if err := copyx.Check[from, short](copyx.WithStrictFields()); !errors.Is(err, copyx.ErrField) {
t.Fatal(err)
}
if err := copyx.Check[checkSource, checkTarget](copyx.WithConverter(func(string) (int, error) { panic("check must not execute converters") })); err != nil {
t.Fatal(err)
}
if err := copyx.Check[struct{ Value any }, struct{ Value int }](); !errors.Is(err, copyx.ErrDynamicType) {
t.Fatal(err)
}
if err := copyx.Check[any, int](copyx.WithConverter(func(any) (int, error) { panic("not called") })); err != nil {
t.Fatal(err)
}
if err := copyx.Check[*int, int](); err != nil {
t.Fatal(err)
} // Actual nil remains a runtime error.
var src *int
dst := 1
if err := copyx.Copy(&dst, src); !errors.Is(err, copyx.ErrNil) || dst != 1 {
t.Fatal(err)
}
var empty []int
if err := copyx.Copy(&empty, []any{}); err != nil || empty == nil {
t.Fatal("dynamic emptiness changed", err)
}
if err := copyx.Check[sync.Mutex, sync.Mutex](); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
option := copyx.WithConverter(func(v []int) ([]int, error) { return v, nil })
if err := copyx.Check[[]int, []int](copyx.WithDeepCopy(), option); err != nil {
t.Fatal(err)
}
if _, err := copyx.Clone([]int{1}, option); !errors.Is(err, copyx.ErrSharedReference) {
t.Fatal("type check bypassed graph audit", err)
}
}
func TestCopierCheckConcurrentAndOptionsIsolation(t *testing.T) {
good, _ := copyx.New(copyx.WithConverter(func(string) (int, error) { return 42, nil }))
bad, _ := copyx.New()
var wg sync.WaitGroup
for i := 0; i < 24; i++ {
wg.Add(1)
go func() {
defer wg.Done()
for j := 0; j < 20; j++ {
if err := good.Check(reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()); err != nil {
t.Error(err)
}
if err := bad.Check(reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()); !errors.Is(err, copyx.ErrType) {
t.Error(err)
}
var target checkTarget
if err := good.Copy(&target, checkSource{Rows: []checkSourceRow{}}); err != nil || target.Rows == nil {
t.Error(err)
}
}
}()
}
wg.Wait()
if err := good.Check(nil, reflect.TypeFor[int]()); err == nil {
t.Fatal("nil type accepted")
}
var absent *copyx.Copier
if err := absent.Check(reflect.TypeFor[int](), reflect.TypeFor[int]()); err == nil {
t.Fatal("nil copier accepted")
}
if err := copyx.Check[int, string](); err == nil || !strings.Contains(err.Error(), "int -> string") {
t.Fatal(err)
}
}
func BenchmarkTypeCheck(b *testing.B) {
c, _ := copyx.New(copyx.WithConverter(func(string) (int, error) { return 1, nil }))
from, to := reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()
b.ReportAllocs()
for i := 0; i < b.N; i++ {
if err := c.Check(from, to); err != nil {
b.Fatal(err)
}
}
}
func TestEmptyTypeCheckPreservesAbsentValueSemantics(t *testing.T) {
var callback func()
if out, err := copyx.Clone(callback); err != nil || out != nil {
t.Fatal(err)
}
if out, err := copyx.Clone([]func(){}); err != nil || out == nil || len(out) != 0 {
t.Fatal(err)
}
if _, err := copyx.Clone([]func(){func() {}}); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal(err)
}
if err := copyx.Check[[]func(), []func()](copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrUnsupported) {
t.Fatal("preflight must report potentially non-nil handles", err)
}
type from struct {
Ignored *int
Value string
}
type to struct {
Ignored *int
Value int
}
var dst []to
if err := copyx.Copy(&dst, []from{}, copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrType) {
t.Fatal(err)
}
}

View File

@@ -89,12 +89,14 @@ type visit struct {
target reflect.Type
}
type state struct {
options *settings
seen map[visit]reflect.Value
graph *graph
nodes int
allocated uint64
valueRoot *reference
options *settings
seen map[visit]reflect.Value
graph *graph
nodes int
allocated uint64
valueRoot *reference
checkedTypes map[pair]bool
checkedNodes int
}
func (s *state) reserve(count int, size uintptr, path string) error {
@@ -239,6 +241,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth
case reflect.Struct:
return s.structure(src, base, t, path, depth)
case reflect.Slice:
if src.Len() == 0 {
if err := s.checkAbsent(src.Type(), t, path); err != nil {
return reflect.Value{}, err
}
}
if !s.options.deep && src.Type() == t {
return src, nil
}
@@ -264,6 +271,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth
}
return out, nil
case reflect.Array:
if src.Len() == 0 {
if err := s.checkAbsent(src.Type(), t, path); err != nil {
return reflect.Value{}, err
}
}
if src.Len() != t.Len() {
return reflect.Value{}, at(path, ErrType)
}
@@ -280,6 +292,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth
}
return out, nil
case reflect.Map:
if src.Len() == 0 {
if err := s.checkAbsent(src.Type(), t, path); err != nil {
return reflect.Value{}, err
}
}
if !s.options.deep && src.Type() == t {
return src, nil
}

82
cors/README.md Normal file
View File

@@ -0,0 +1,82 @@
# 可选 CORS
CORS 控制浏览器能否跨源读取 HTTP 响应。默认关闭;同源反向代理可以保持关闭。模块无 Gin 或具体业务依赖,标准库/native/HTTP2 与自定义 Runner 使用相同 App 接入策略。
## 生成项目只改配置
在既有 `etc/config.yaml` 的 `server` 中加入,无需修改 main、svc 或每条 `.api` 路由:
```yaml
server:
cors:
enabled: true
allowedOrigins:
- https://admin.example.com
- https://merchant.example.com
allowedMethods: [GET, HEAD, POST, PUT, PATCH, DELETE]
allowedHeaders: [Authorization, Content-Type]
exposedHeaders: [X-Request-ID, Content-Disposition]
allowCredentials: false
maxAge: 10m
```
文件也可显式引用环境变量:`allowedOrigins: ["${WEB_ORIGIN}"]`。一个数组元素代表一个来源,不把逗号分隔环境文本隐式拆成多个来源。JSON/YAML/XML 走同一加载和校验;非法配置定位 `server.cors.allowedOrigins[0]` 等字段、文件及可用环境名/行列,不回显值。
```sh
go run . -config etc/config.yaml -check-config
go run ./examples/cors # 在框架源码根目录运行独立示例
```
## 手写项目与独立 HTTP 服务
```go
app := omni.New(omni.WithCORS(omni.CORSConfig{
Enabled: true,
AllowedOrigins: []string{"https://web.example.com"},
AllowedMethods: []string{"GET", "POST"},
AllowedHeaders: []string{"Authorization", "Content-Type"},
}))
```
已有 net/http 服务也可单独使用,不需要创建 App:
```go
policy, err := cors.New(cors.Config{
Enabled: true,
AllowedOrigins: []string{"https://web.example.com"},
})
if err != nil { return err }
handler := policy.Wrap(existingHandler)
```
Config/New/WithCORS 在初始化时校验并快照配置;同一 Policy 可并发复用。`Config.Validate()` 返回 error,直接向 `omni.New` 传非法选项沿用框架初始化 panic 约定;生成项目在创建 App 前已校验。
## 配置规则
| 字段 | 规则 |
| --- | --- |
| enabled | 默认 false;关闭时直接使用原 Handler,不添加头、不拦截 OPTIONS |
| allowedOrigins | 启用时必填。完整 HTTP(S) origin(协议+主机+可选端口),无路径、尾斜杠、userinfo、查询或片段。配置中的协议/主机转小写、默认 80/443 端口省略;请求匹配浏览器序列化后的来源 |
| allowedMethods | 空列表默认 GET、HEAD、POST;填写后使用指定集合。大写方法名,拒绝浏览器禁止的 CONNECT/TRACE/TRACK,不提供方法通配符 |
| allowedHeaders | 空列表不批准额外预检请求头。Authorization、JSON Content-Type 等按需要列出;名称忽略大小写,不提供通配符 |
| exposedHeaders | 默认空;额外允许前端 JS 读取的响应头,例如下载文件名用 Content-Disposition。浏览器本来就可读取的安全列表不需重复填写 |
| allowCredentials | 默认 false;跨源 Cookie 等凭据模式需要显式 true 和具体来源。单纯手动设置 Authorization 头只需列入 allowedHeaders,不必因此开启 Cookie 凭据模式 |
| maxAge | 浏览器缓存预检结果的时长,必须非负且为整秒;0 不发送该头。浏览器可按自身上限截短 |
公开无凭据 API 可使用唯一来源 `"*"`;与 `allowCredentials: true` 或其他来源并存会报错。此版本只支持 HTTP(S) 来源,拒绝 `null`、子域通配符、正则、file:// 和不透明来源。国际化域名填写浏览器发送的 ASCII/Punycode 形式;IP 使用完整标准形式。不会为匹配来源查询 DNS。
## 预检、错误响应与安全边界
- CORS 位于路由/业务中间件外层。只有 OPTIONS 且同时带 Origin 和 Access-Control-Request-Method 才视为预检;普通 OPTIONS 继续执行项目逻辑。
- 合法预检返回 **204 空响应**,不进入业务中间件、鉴权、Logic 或业务响应包装。拒绝预检返回 **403 空响应**,不暴露内部原因;两者是协议响应,不受业务“失败也返回 HTTP 200”的设置影响。
- 预检验证全局来源、方法、请求头策略,不探测路由存在性;通过不代表实际接口一定存在。实际请求仍由路由处理,可返回 404/405。
- 实际请求继续正常鉴权;来源/方法获准时,正常响应、401/403、404/405、框架恢复的 500 都带 CORS 头,响应体、状态和错误目录不变。协议解析、TLS、网络层过载等在进入 App 前发生的拒绝不由本模块改写。
- 未获准或没有 Origin 的实际请求仍正常进入业务,只是不添加 CORS 许可头。CORS 不等于身份认证、服务端访问控制或 CSRF 防护;服务端回调仍需项目自己的验签/鉴权。
- 始终追加并去重 `Vary: Origin`;预检再追加请求方法/请求头的 Vary,保留原有 Vary 与 `*`。不要在下游代码覆盖这些头。代理与应用选择一处负责 CORS,避免重复头或冲突策略。
- 不包装 ResponseWriter、不读取/缓存 body、不克隆 Request。下载、上传、Flush、Hijack 和请求上下文保留;不增加响应体大小限制或改写统一返回格式。
- 使用 App、Engine、NewServer 或 Run/ServeContext/RunWith 接入。Router() 暴露底层路由供高级注册,单独拿 Router().ServeHTTP 服务会绕过应用的外层策略。
- App 内业务日志/指标不统计被提前结束的预检;如需记录全部流量,在 App 外层或代理配置访问日志。不要让全局登录校验挡在 CORS 预检之前。
配置最多 1024 个来源、每个最多 4096 字节;方法/请求头/暴露头各最多 128 个且名称最多 256 字节;预检请求头列表最多 128 项、8192 字节。拒绝 CRLF、歧义的重复 Origin/请求方法及非法 token。WebSocket Origin 校验、Private Network Access 扩展和云桶 CORS 不包含在本模块中,按对应服务配置。
规则依据:[Fetch CORS 协议](https://fetch.spec.whatwg.org/#http-cors-protocol)、[凭据规则](https://fetch.spec.whatwg.org/#cors-protocol-and-credentials)、[缓存规则](https://fetch.spec.whatwg.org/#cors-protocol-and-http-caches)。可运行示例:[examples/cors](../examples/cors/README.md)。

144
cors/config.go Normal file
View File

@@ -0,0 +1,144 @@
// Package cors implements optional, transport-independent browser CORS policy.
// It does not authenticate requests or provide CSRF protection.
package cors
import (
"net"
"net/netip"
"net/url"
"strconv"
"strings"
"time"
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
)
// Config is disabled by default. Origins are exact HTTP(S) origins, or a single
// "*" for public responses without credentials. Patterns and "null" are rejected.
// Empty methods default to GET, HEAD, POST; empty header lists grant no extra
// permissions. MaxAge is a nonnegative whole number of seconds; zero omits it.
type Config struct {
Enabled bool `json:"enabled"`
AllowedOrigins []string `json:"allowedOrigins"`
AllowedMethods []string `json:"allowedMethods"`
AllowedHeaders []string `json:"allowedHeaders"`
ExposedHeaders []string `json:"exposedHeaders"`
AllowCredentials bool `json:"allowCredentials"`
MaxAge time.Duration `json:"maxAge"`
}
func (c Config) Clone() Config {
c.AllowedOrigins = append([]string(nil), c.AllowedOrigins...)
c.AllowedMethods = append([]string(nil), c.AllowedMethods...)
c.AllowedHeaders = append([]string(nil), c.AllowedHeaders...)
c.ExposedHeaders = append([]string(nil), c.ExposedHeaders...)
return c
}
// Validate never echoes configured values in diagnostics. Invalid settings are
// rejected even when disabled; an empty origin list is legal only when disabled.
func (c Config) Validate() error {
if c.Enabled && len(c.AllowedOrigins) == 0 {
return configdiag.New("allowedOrigins", "is required when CORS is enabled")
}
if c.MaxAge < 0 || c.MaxAge%time.Second != 0 {
return configdiag.New("maxAge", "must be a nonnegative whole number of seconds")
}
if len(c.AllowedOrigins) > 1024 {
return configdiag.New("allowedOrigins", "must contain at most 1024 entries")
}
for i, origin := range c.AllowedOrigins {
field := "allowedOrigins[" + strconv.Itoa(i) + "]"
if origin == "*" {
if c.AllowCredentials || len(c.AllowedOrigins) != 1 {
return configdiag.New(field, "wildcard requires a single origin entry and allowCredentials=false")
}
} else if _, ok := normalizeOrigin(origin); !ok {
return configdiag.New(field, "must be an HTTP(S) origin without credentials, path, query, fragment or wildcard")
}
}
for _, list := range []struct {
name string
values []string
}{{"allowedMethods", c.AllowedMethods}, {"allowedHeaders", c.AllowedHeaders}, {"exposedHeaders", c.ExposedHeaders}} {
if len(list.values) > 128 {
return configdiag.New(list.name, "must contain at most 128 entries")
}
for i, value := range list.values {
field := list.name + "[" + strconv.Itoa(i) + "]"
if !token(value) || value == "*" {
return configdiag.New(field, "must be an explicit HTTP token, not a wildcard")
}
if list.name == "allowedMethods" && (value != strings.ToUpper(value) || value == "CONNECT" || value == "TRACE" || value == "TRACK") {
return configdiag.New(field, "must be an uppercase browser-allowed HTTP method")
}
}
}
return nil
}
// Normalize configured scheme/host case and default ports once. Request values
// match serialized browser origins; arbitrary URL prefixes/suffixes never match.
func normalizeOrigin(origin string) (string, bool) {
if len(origin) == 0 || len(origin) > 4096 {
return "", false
}
for _, b := range []byte(origin) {
if b <= 32 || b >= 127 || strings.ContainsRune("\\%?#*", rune(b)) {
return "", false
}
}
u, err := url.Parse(origin)
if err != nil || u.User != nil || u.Opaque != "" || u.Path != "" || u.RawQuery != "" || u.Fragment != "" || u.Host == "" {
return "", false
}
scheme, host := strings.ToLower(u.Scheme), strings.ToLower(u.Hostname())
if (scheme != "http" && scheme != "https") || host == "" || strings.HasSuffix(u.Host, ":") {
return "", false
}
if strings.Contains(host, ":") {
ip, err := netip.ParseAddr(host)
if err != nil || !ip.Is6() || ip.Zone() != "" {
return "", false
}
host = "[" + ip.String() + "]"
if ip.Is4In6() {
// Browsers serialize mapped IPv4 as hex IPv6 pieces; netip.String
// uses a dotted-decimal tail instead.
b := ip.As16()
host = "[::ffff:" + strconv.FormatUint(uint64(b[12])<<8|uint64(b[13]), 16) + ":" + strconv.FormatUint(uint64(b[14])<<8|uint64(b[15]), 16) + "]"
}
} else {
if strings.ContainsAny(u.Host, "[]") {
return "", false
}
for _, b := range []byte(host) {
if !(b >= 'a' && b <= 'z' || b >= '0' && b <= '9' || b == '.' || b == '-') {
return "", false
}
}
}
port := u.Port()
if port != "" {
n, err := strconv.Atoi(port)
if err != nil || n < 1 || n > 65535 {
return "", false
}
if !(scheme == "https" && n == 443 || scheme == "http" && n == 80) {
host = net.JoinHostPort(strings.Trim(host, "[]"), strconv.Itoa(n))
}
}
return scheme + "://" + host, true
}
func token(s string) bool {
if s == "" || len(s) > 256 {
return false
}
for _, b := range []byte(s) {
if !(b >= 'a' && b <= 'z' || b >= 'A' && b <= 'Z' || b >= '0' && b <= '9' || strings.ContainsRune("!#$%&'*+-.^_`|~", rune(b))) {
return false
}
}
return true
}

187
cors/cors.go Normal file
View File

@@ -0,0 +1,187 @@
package cors
import (
"net/http"
"strconv"
"strings"
"time"
)
// Policy is immutable after New and safe to share between concurrent requests.
type Policy struct {
enabled, wildcard, credentials bool
origins, methods, headers map[string]struct{}
allowMethods, allowHeaders, exposeHeaders, maxAge string
}
// New validates and snapshots policy. It performs no network or DNS operations.
func New(c Config) (*Policy, error) {
if err := c.Validate(); err != nil {
return nil, err
}
p := &Policy{enabled: c.Enabled, credentials: c.AllowCredentials}
if !c.Enabled {
return p, nil
}
p.origins = make(map[string]struct{}, len(c.AllowedOrigins))
for _, origin := range c.AllowedOrigins {
if origin == "*" {
p.wildcard = true
continue
}
normalized, _ := normalizeOrigin(origin)
p.origins[normalized] = struct{}{}
}
methods := c.AllowedMethods
if len(methods) == 0 {
methods = []string{http.MethodGet, http.MethodHead, http.MethodPost}
}
p.methods, p.allowMethods = compileList(methods, false)
p.headers, p.allowHeaders = compileList(c.AllowedHeaders, true)
_, p.exposeHeaders = compileList(c.ExposedHeaders, true)
if c.MaxAge > 0 {
p.maxAge = strconv.FormatInt(int64(c.MaxAge/time.Second), 10)
}
return p, nil
}
func compileList(values []string, header bool) (map[string]struct{}, string) {
set := make(map[string]struct{}, len(values))
list := make([]string, 0, len(values))
for _, value := range values {
key := value
if header {
key = strings.ToLower(value)
value = http.CanonicalHeaderKey(value)
}
if _, exists := set[key]; exists {
continue
}
set[key] = struct{}{}
list = append(list, value)
}
return set, strings.Join(list, ", ")
}
// Wrap installs CORS outside routing/authentication. Only actual CORS preflights
// are short-circuited. Actual requests always continue through normal handlers;
// an unlisted Origin receives no permission to read the response in a browser.
// The original ResponseWriter, Request and body pass through without buffering.
func (p *Policy) Wrap(next http.Handler) http.Handler {
if next == nil {
panic("cors: nil handler")
}
if p == nil {
panic("cors: nil policy")
}
if !p.enabled {
return next
}
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h := w.Header()
vary(h, "Origin")
origins := r.Header.Values("Origin")
// An OPTIONS request without Origin or Access-Control-Request-Method is
// ordinary application traffic, including user-defined OPTIONS handlers.
requestedMethods := r.Header.Values("Access-Control-Request-Method")
preflight := r.Method == http.MethodOptions && len(origins) > 0 && len(requestedMethods) > 0
if preflight {
vary(h, "Access-Control-Request-Method")
vary(h, "Access-Control-Request-Headers")
}
if len(origins) != 1 || !p.allowsOrigin(origins[0]) {
if preflight {
w.WriteHeader(http.StatusForbidden)
return
}
next.ServeHTTP(w, r)
return
}
if preflight {
if len(requestedMethods) != 1 || !p.allowsMethod(requestedMethods[0]) || !p.allowsHeaders(r.Header.Values("Access-Control-Request-Headers")) {
w.WriteHeader(http.StatusForbidden)
return
}
p.allowOrigin(h, origins[0])
h.Set("Access-Control-Allow-Methods", p.allowMethods)
if p.allowHeaders != "" {
h.Set("Access-Control-Allow-Headers", p.allowHeaders)
}
if p.maxAge != "" {
h.Set("Access-Control-Max-Age", p.maxAge)
}
w.WriteHeader(http.StatusNoContent)
return
}
// Match the actual method too: this grants response access, not permission
// to execute a side effect. Authentication/CSRF checks remain in the app.
if p.allowsMethod(r.Method) {
p.allowOrigin(h, origins[0])
if p.exposeHeaders != "" {
h.Set("Access-Control-Expose-Headers", p.exposeHeaders)
}
}
next.ServeHTTP(w, r)
})
}
func (p *Policy) allowsOrigin(origin string) bool {
if !p.wildcard {
_, ok := p.origins[origin]
return ok
}
normalized, ok := normalizeOrigin(origin)
return ok && normalized == origin
}
func (p *Policy) allowsMethod(method string) bool { _, ok := p.methods[method]; return ok }
func (p *Policy) allowOrigin(h http.Header, origin string) {
if p.wildcard {
origin = "*"
}
h.Set("Access-Control-Allow-Origin", origin)
if p.credentials {
h.Set("Access-Control-Allow-Credentials", "true")
}
}
func (p *Policy) allowsHeaders(lines []string) bool {
count, size := 0, 0
for _, line := range lines {
size += len(line)
if size > 8192 {
return false
}
// Parse a header list without allocating an attacker-sized split slice.
for {
name, rest, more := strings.Cut(line, ",")
name = strings.Trim(name, " \t")
count++
if count > 128 || !token(name) {
return false
}
if _, ok := p.headers[strings.ToLower(name)]; !ok {
return false
}
if !more {
break
}
line = rest
}
}
return true
}
func vary(h http.Header, name string) {
for _, line := range h.Values("Vary") {
for {
v, rest, more := strings.Cut(line, ",")
v = strings.TrimSpace(v)
if v == "*" || strings.EqualFold(v, name) {
return
}
if !more {
break
}
line = rest
}
}
h.Add("Vary", name)
}

303
cors/cors_test.go Normal file
View File

@@ -0,0 +1,303 @@
package cors
import (
"context"
"io"
"net/http"
"net/http/httptest"
"reflect"
"strings"
"sync"
"testing"
"time"
)
func exampleConfig() Config {
return Config{Enabled: true, AllowedOrigins: []string{"https://admin.example", "https://merchant.example"}, AllowedMethods: []string{"GET", "POST", "DELETE"}, AllowedHeaders: []string{"Authorization", "Content-Type"}, ExposedHeaders: []string{"X-Request-ID", "Content-Disposition"}, MaxAge: 10 * time.Minute}
}
func TestPolicyPreflightAndActual(t *testing.T) {
for _, tc := range []struct {
name, method, origin, requestedMethod, requestedHeaders string
status int
called, allowed bool
}{
{"preflight", "OPTIONS", "https://admin.example", "POST", "authorization, CONTENT-TYPE", 204, false, true},
{"no request headers", "OPTIONS", "https://admin.example", "DELETE", "", 204, false, true},
{"unknown origin", "OPTIONS", "https://other.example", "POST", "authorization", 403, false, false},
{"suffix attack", "OPTIONS", "https://admin.example.evil", "POST", "", 403, false, false},
{"scheme mismatch", "OPTIONS", "http://admin.example", "POST", "", 403, false, false},
{"port mismatch", "OPTIONS", "https://admin.example:444", "POST", "", 403, false, false},
{"opaque origin", "OPTIONS", "null", "POST", "", 403, false, false},
{"unknown method", "OPTIONS", "https://admin.example", "PATCH", "", 403, false, false},
{"case sensitive method", "OPTIONS", "https://admin.example", "post", "", 403, false, false},
{"unknown header", "OPTIONS", "https://admin.example", "POST", "X-Internal", 403, false, false},
{"trailing empty header", "OPTIONS", "https://admin.example", "POST", "Authorization,", 403, false, false},
{"control header", "OPTIONS", "https://admin.example", "POST", "Authorization\r\nX-Evil", 403, false, false},
{"duplicate allowed header", "OPTIONS", "https://admin.example", "POST", "Authorization, authorization", 204, false, true},
{"oversized list", "OPTIONS", "https://admin.example", "POST", strings.Repeat("Authorization,", 129) + "Authorization", 403, false, false},
{"actual error response", "GET", "https://merchant.example", "", "", 401, true, true},
{"actual denied origin still authenticates", "POST", "https://other.example", "", "", 401, true, false},
{"actual denied method still authenticates", "PUT", "https://admin.example", "", "", 401, true, false},
{"server callback", "POST", "", "", "", 401, true, false},
{"ordinary OPTIONS", "OPTIONS", "https://admin.example", "", "", 401, true, false},
{"OPTIONS without Origin", "OPTIONS", "", "POST", "", 401, true, false},
} {
t.Run(tc.name, func(t *testing.T) {
policy, err := New(exampleConfig())
if err != nil {
t.Fatal(err)
}
called := false
h := policy.Wrap(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
called = true
w.WriteHeader(401)
io.WriteString(w, "authentication required")
}))
r := httptest.NewRequest(tc.method, "http://api.example/resource", nil)
if tc.origin != "" {
r.Header.Set("Origin", tc.origin)
}
if tc.requestedMethod != "" {
r.Header.Set("Access-Control-Request-Method", tc.requestedMethod)
}
if tc.requestedHeaders != "" {
r.Header.Set("Access-Control-Request-Headers", tc.requestedHeaders)
}
w := httptest.NewRecorder()
w.Header().Add("Vary", "Accept-Encoding")
h.ServeHTTP(w, r)
if w.Code != tc.status || called != tc.called {
t.Fatalf("status=%d called=%v", w.Code, called)
}
if got := w.Header().Get("Access-Control-Allow-Origin"); (got != "") != tc.allowed || tc.allowed && got != tc.origin {
t.Fatal(w.Header())
}
if !strings.Contains(strings.Join(w.Header().Values("Vary"), ","), "Origin") || !strings.Contains(strings.Join(w.Header().Values("Vary"), ","), "Accept-Encoding") {
t.Fatal(w.Header())
}
if !tc.called {
if w.Body.Len() != 0 {
t.Fatal("preflight wrapped in business response")
}
if !strings.Contains(strings.Join(w.Header().Values("Vary"), ","), "Access-Control-Request-Headers") {
t.Fatal(w.Header())
}
if tc.allowed && (w.Header().Get("Access-Control-Max-Age") != "600" || !strings.Contains(w.Header().Get("Access-Control-Allow-Headers"), "Authorization")) {
t.Fatal(w.Header())
}
} else if w.Body.String() != "authentication required" {
t.Fatal(w.Body.String())
}
if tc.called && tc.allowed && w.Header().Get("Access-Control-Expose-Headers") != "X-Request-Id, Content-Disposition" {
t.Fatal(w.Header())
}
})
}
}
func TestPolicyValidation(t *testing.T) {
for _, origin := range []string{"https://admin.example/", "https://user:password@admin.example", "null", "https://*.example", "https://admin.example?secret", "https://admin.example#secret", "file://host", "https://host:", "https://host:0", "https://host:65536", "https://host%2eevil", "https://host\r\nX: value", "https://例子.example", "https://[localhost]", "https://[::1%25zone]"} {
c := exampleConfig()
c.AllowedOrigins = []string{origin}
err := c.Validate()
if err == nil || strings.Contains(err.Error(), origin) {
t.Fatalf("invalid origin accepted or exposed: %q %v", origin, err)
}
}
for _, change := range []func(*Config){
func(c *Config) { c.AllowedOrigins = nil },
func(c *Config) { c.AllowedOrigins = []string{"*"}; c.AllowCredentials = true },
func(c *Config) { c.AllowedOrigins = []string{"*", "https://admin.example"} },
func(c *Config) { c.AllowedHeaders = []string{"*"} },
func(c *Config) { c.ExposedHeaders = []string{"header\r\n"} },
func(c *Config) { c.AllowedMethods = []string{"get"} },
func(c *Config) { c.AllowedMethods = []string{"TRACE"} },
func(c *Config) { c.MaxAge = -time.Second },
func(c *Config) { c.MaxAge = time.Millisecond },
} {
c := exampleConfig()
change(&c)
if _, err := New(c); err == nil {
t.Fatal(c)
}
}
if err := (Config{}).Validate(); err != nil {
t.Fatal(err)
}
for input, want := range map[string]string{"https://[::ffff:192.0.2.1]": "https://[::ffff:c000:201]", "https://[::ffff:c000:201]": "https://[::ffff:c000:201]", "https://ADMIN.EXAMPLE:443": "https://admin.example", "http://localhost:80": "http://localhost", "http://localhost:8080": "http://localhost:8080", "https://[::1]:443": "https://[::1]", "http://[::1]:8080": "http://[::1]:8080"} {
got, ok := normalizeOrigin(input)
if !ok || got != want {
t.Fatalf("%q => %q %v", input, got, ok)
}
}
}
func TestPolicyCredentialsWildcardAndDuplicates(t *testing.T) {
for _, credentials := range []bool{false, true} {
c := exampleConfig()
c.AllowCredentials = credentials
if !credentials {
c.AllowedOrigins = []string{"*"}
}
p, _ := New(c)
for _, origin := range []string{"https://admin.example", "null", "https://admin.example/path", "https://bad\r\n"} {
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set("Origin", origin)
w := httptest.NewRecorder()
p.Wrap(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(200) })).ServeHTTP(w, r)
if origin != "https://admin.example" {
if w.Header().Get("Access-Control-Allow-Origin") != "" {
t.Fatal(w.Header())
}
continue
}
want := "*"
if credentials {
want = origin
}
if w.Header().Get("Access-Control-Allow-Origin") != want || (w.Header().Get("Access-Control-Allow-Credentials") == "true") != credentials {
t.Fatal(w.Header())
}
}
}
p, _ := New(exampleConfig())
for _, header := range []string{"Origin", "Access-Control-Request-Method"} {
r := httptest.NewRequest("OPTIONS", "/", nil)
r.Header.Set("Origin", "https://admin.example")
r.Header.Set("Access-Control-Request-Method", "POST")
r.Header.Add(header, r.Header.Get(header))
w := httptest.NewRecorder()
p.Wrap(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { t.Fatal("ambiguous preflight reached handler") })).ServeHTTP(w, r)
if w.Code != 403 || w.Header().Get("Access-Control-Allow-Origin") != "" {
t.Fatal(w.Code, w.Header())
}
}
}
func TestPolicyPreservesHTTPAndSnapshot(t *testing.T) {
c := exampleConfig()
p, _ := New(c)
c.AllowedOrigins[0] = "https://evil.example"
c.AllowedHeaders[0] = "X-Evil"
r := httptest.NewRequest("GET", "/", strings.NewReader("request"))
r.Header.Set("Origin", "https://admin.example")
type key struct{}
r = r.WithContext(context.WithValue(r.Context(), key{}, "kept"))
w := httptest.NewRecorder()
w.Header().Set("Vary", "Accept-Encoding, origin")
next := http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
if writer != w || request != r || request.Context().Value(key{}) != "kept" {
t.Fatal("HTTP capabilities replaced")
}
io.WriteString(writer, "chunk")
writer.(http.Flusher).Flush()
})
p.Wrap(next).ServeHTTP(w, r)
if !w.Flushed || w.Body.String() != "chunk" || w.Header().Get("Access-Control-Allow-Origin") != "https://admin.example" || len(w.Header().Values("Vary")) != 1 {
t.Fatal(w)
}
for _, prior := range []string{"*", "Accept-Encoding, Origin"} {
h := http.Header{"Vary": {prior}}
vary(h, "Origin")
if !reflect.DeepEqual(h.Values("Vary"), []string{prior}) {
t.Fatal(h)
}
}
disabled, _ := New(Config{})
if reflect.ValueOf(disabled.Wrap(next)).Pointer() != reflect.ValueOf(next).Pointer() {
t.Fatal("disabled policy not passthrough")
}
var wg sync.WaitGroup
h := p.Wrap(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(200) }))
for i := 0; i < 32; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
for j := 0; j < 100; j++ {
r := httptest.NewRequest("GET", "/", nil)
origin := "https://admin.example"
if i%2 == 1 {
origin = "https://merchant.example"
}
r.Header.Set("Origin", origin)
w := httptest.NewRecorder()
h.ServeHTTP(w, r)
if w.Header().Get("Access-Control-Allow-Origin") != origin {
t.Error("cross-request origin leak")
}
}
}(i)
}
wg.Wait()
}
func FuzzCORSHeaders(f *testing.F) {
f.Add("https://admin.example", "POST", "authorization, content-type")
f.Add("https://admin.example.evil", "POST", "")
f.Add("null", "GET", "\r\n")
p, _ := New(exampleConfig())
h := p.Wrap(http.HandlerFunc(func(http.ResponseWriter, *http.Request) { panic("preflight reached business") }))
f.Fuzz(func(t *testing.T, origin, method, headers string) {
r := httptest.NewRequest("OPTIONS", "/", nil)
r.Header.Set("Origin", origin)
r.Header.Set("Access-Control-Request-Method", method)
if headers != "" {
r.Header.Set("Access-Control-Request-Headers", headers)
}
w := httptest.NewRecorder()
h.ServeHTTP(w, r)
if w.Code != 204 && w.Code != 403 {
t.Fatal(w.Code)
}
if w.Body.Len() != 0 {
t.Fatal("unexpected body")
}
if value := w.Header().Get("Access-Control-Allow-Origin"); value != "" {
if value != "https://admin.example" && value != "https://merchant.example" {
t.Fatal("untrusted reflection")
}
}
})
}
type benchmarkWriter struct{ header http.Header }
func (w benchmarkWriter) Header() http.Header { return w.header }
func (w benchmarkWriter) Write(p []byte) (int, error) { return len(p), nil }
func (w benchmarkWriter) WriteHeader(int) {}
func BenchmarkCORS(b *testing.B) {
for _, mode := range []string{"disabled", "noOrigin", "allowed", "denied", "preflight", "wildcard"} {
b.Run(mode, func(b *testing.B) {
c := exampleConfig()
if mode == "disabled" {
c.Enabled = false
}
if mode == "wildcard" {
c.AllowedOrigins = []string{"*"}
}
p, _ := New(c)
h := p.Wrap(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(200) }))
r := httptest.NewRequest("GET", "/", nil)
r.Header.Set("Origin", "https://admin.example")
if mode == "noOrigin" {
r.Header.Del("Origin")
}
if mode == "denied" {
r.Header.Set("Origin", "https://other.example")
}
if mode == "preflight" {
r.Method = "OPTIONS"
r.Header.Set("Access-Control-Request-Method", "POST")
r.Header.Set("Access-Control-Request-Headers", "authorization, content-type")
}
w := benchmarkWriter{make(http.Header)}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
clear(w.header)
h.ServeHTTP(w, r)
}
})
}
}

View File

@@ -9,7 +9,7 @@
- `Config`:驱动、DSN 与连接池参数;嵌入项目配置,通过文件显式引用变量。
- `Open` / `OpenConnector`:通过 DSN 或标准 Connector 创建连接池、设置连接上限并在返回前执行 `PingContext`。
- `Health`:用于就绪检查,等待连接和 Ping 均受 `PingTimeout` 限制。
- `InTx`:统一事务提交和回滚;工作函数返回错误或 panic 时回滚。
- `InTx`:统一事务提交和回滚;工作函数返回错误或 panic 时尝试回滚。返回 `TxError` 区分阶段与结果,提交错误不代表已回滚。
- `Close`:由嵌入的 `*sql.DB` 提供,应用退出时显式调用。
框架不导入 MySQL、PostgreSQL 等具体驱动。应用负责注册驱动:
@@ -63,6 +63,55 @@ python3 scripts/verifyDatabase.py --version v0.2.0
配置结构作为项目 Config 的字段,Default() 初始化连接池,文件按需覆盖。DSN 可使用 `dsn: "${DB_DSN}"`;database.Config 的 JSON 序列化不输出 DSN 原值。不要打印原始配置对象。当前 API 脚手架使用结构化 MySQL 适配器,由 svc 统一初始化与关闭。
## 事务阶段与结果
v0.25 起,`db.InTx(ctx, options, work)` 签名不变,所有普通错误返回都是 `*database.TxError`。返回 nil 表示 Commit 返回成功;框架不重试回调、提交或整个事务。业务通过标准库 errors.As/Is 判断,不解析错误文本:
```go
err := db.InTx(ctx, nil, func(tx *sql.Tx) error {
_, err := tx.ExecContext(ctx, "INSERT INTO objects (id, storage_key) VALUES (?, ?)", id, key)
return err
})
if err != nil {
var txErr *database.TxError
if errors.As(err, &txErr) {
switch txErr.Outcome {
case database.TxOutcomeNotStarted, database.TxOutcomeRolledBack:
// 当前受管理事务未执行回调或已确认回滚。
// 业务结合文件归属、并发操作和其他副作用,决定是否清理。
default:
// 包括 TxOutcomeUnknown:保留文件,交给业务后续核查。
// 不因为收到提交错误就删除文件或直接重试整个操作。
}
// txErr.Stage 是 begin / work / commit。
// txErr.Err 是主要原因;txErr.RollbackErr 是回滚调用返回的原因。
}
if errors.Is(err, context.DeadlineExceeded) {
// 说明错误链包含超时,不意味着数据库没有提交。
}
return err
}
```
`errors` 为标准库包,业务自己的数据库驱动和 SQL 参数规则不变。完整可运行示例见 [examples/transactions](../examples/transactions/README.md)。
| 情况 | Stage | Outcome | RollbackErr |
| --- | --- | --- | --- |
| 未初始化数据库、nil 回调、BeginTx 报错 | TxStageBegin | TxOutcomeNotStarted | nil;框架没有拿到事务,不执行回调 |
| 回调报错,显式 Rollback 成功 | TxStageWork | TxOutcomeRolledBack | nil |
| 回调报错,Rollback 失败或返回 ErrTxDone | TxStageWork | TxOutcomeUnknown | 原始回滚错误 |
| Commit 报错,包括取消/断网/驱动错误 | TxStageCommit | TxOutcomeUnknown | 后续清理的 Rollback 返回值,可能为 ErrTxDone 或 nil |
`Stage` 描述主要失败发生的位置;`Outcome` 只描述框架掌握的结果,二者独立。提交调用返回错误后一律保守标为 unknown,即使某个驱动的某种错误意味着没有提交,通用层也不根据错误文本猜测。标准库在 Commit 后通常已经结束本地事务对象,后续 Rollback 返回 `sql.ErrTxDone`;它不能证明数据库已回滚,框架保留该原因。
`TxError.Err` 与 `RollbackErr` 分开保存;`Unwrap() []error` 按主要原因、回滚原因顺序暴露,所以 errors.Is/As 可查两条链。需要区分来源时对 `txErr.Err` 或 `txErr.RollbackErr` 单独判断。回调返回另一个事务的 TxError 时外层不把它压平:外层结果只属于外层事务,内部事务的结果仍在 Err 中。
**结果保证的范围:** 回调使用给定 `*sql.Tx` 完成事务操作,不自行 Commit/Rollback,不执行会隐式提交的 SQL,不将事务交给后台 goroutine。非事务表、其他连接、嵌套的独立事务、文件和第三方调用的副作用不在这份保证内。框架没有删除文件、入队补偿或分布式事务规则。
**取消与异常边界:** context 取消时 database/sql 可能自行异步回滚,且不公开那次驱动 Rollback 的原始错误。框架若只观察到 ErrTxDone,就返回 unknown,不能补造成功证据或不存在的底层原因。正常返回路径上,框架实际观察到的主要/回滚原因全部保留。panic/runtime.Goexit 保持原有传播行为并尝试回滚,不转换为普通 error;这类展开没有错误返回值,因此不能把回滚原因作为返回值交给调用方。nil context 使用 Background,nil 回调在开启事务前报错。
**兼容说明:** 已有 errors.Is/As 继续生效;以前直接用 `err == sentinel`、`err.(*SomeError)` 或匹配文本的调用要改成 errors.Is/As。业务 APIError、Go 错误目录和参数提示仍可穿过包装解析,驱动错误不会自动进入公开提示。TxError.Error 包含内部原因,仅供内部诊断。GORM 自己的 Transaction 不经过本方法,也不会自动变成 TxError;可选 gormx 的连接池接入不变。
## Connector 接入(v0.9.1-preview)
项目已有 `database/sql/driver.Connector` 时使用 `OpenConnector`。这是标准驱动接口,核心包不增加具体驱动依赖。项目可以在同一进程创建多个使用不同证书、日志、拨号策略的连接池。

View File

@@ -7,6 +7,8 @@ import (
"errors"
"reflect"
"time"
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
)
// PoolConfig contains only driver-independent settings. Connection credentials,
@@ -25,15 +27,29 @@ func DefaultPoolConfig() PoolConfig {
}
func (c PoolConfig) Validate() error {
if c.MaxOpenConns < 0 || c.MaxIdleConns < 0 {
return errors.New("database pool limits cannot be negative")
for _, f := range []struct {
name string
value int
}{{"maxOpenConns", c.MaxOpenConns}, {"maxIdleConns", c.MaxIdleConns}} {
if f.value < 0 {
return configdiag.New(f.name, "must be nonnegative")
}
}
if c.MaxOpenConns > 0 && c.MaxIdleConns > c.MaxOpenConns {
return errors.New("max idle connections cannot exceed max open connections")
return configdiag.New("maxIdleConns", "must not exceed maxOpenConns")
}
if c.ConnMaxLifetime < 0 || c.ConnMaxIdleTime < 0 || c.PingTimeout <= 0 {
return errors.New("database timeouts are invalid")
for _, f := range []struct {
name string
value time.Duration
}{{"connMaxLifetime", c.ConnMaxLifetime}, {"connMaxIdleTime", c.ConnMaxIdleTime}} {
if f.value < 0 {
return configdiag.New(f.name, "must be nonnegative")
}
}
if c.PingTimeout <= 0 {
return configdiag.New("pingTimeout", "must be positive")
}
return nil
}

View File

@@ -10,6 +10,8 @@ import (
"fmt"
"strings"
"time"
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
)
type Config struct {
@@ -43,10 +45,10 @@ func (c Config) PoolConfig() PoolConfig {
func (c Config) Validate() error {
if strings.TrimSpace(c.Driver) == "" {
return errors.New("database driver is required")
return configdiag.New("driver", "is required")
}
if strings.TrimSpace(c.DSN) == "" {
return errors.New("database DSN is required")
return configdiag.New("dsn", "is required")
}
return c.PoolConfig().Validate()
}
@@ -105,27 +107,3 @@ func (db *DB) Health(ctx context.Context) error {
defer cancel()
return db.PingContext(checkCtx)
}
// InTx commits only when work returns nil. Any work error or panic rolls back.
func (db *DB) InTx(ctx context.Context, options *sql.TxOptions, work func(*sql.Tx) error) (err error) {
if db == nil || db.DB == nil {
return errors.New("database is not initialized")
}
tx, err := db.BeginTx(ctx, options)
if err != nil {
return fmt.Errorf("begin transaction: %w", err)
}
defer func() {
rollbackErr := tx.Rollback()
if rollbackErr != nil && !errors.Is(rollbackErr, sql.ErrTxDone) {
err = errors.Join(err, fmt.Errorf("rollback transaction: %w", rollbackErr))
}
}()
if err = work(tx); err != nil {
return err
}
if err = tx.Commit(); err != nil {
return fmt.Errorf("commit transaction: %w", err)
}
return nil
}

View File

@@ -70,3 +70,7 @@ func (c *ServiceContext) DatabaseOptions() []mysqladapter.Option {
`WithDriverConfig(func(*mysql.Config) error)` 支持高级拨号、驱动参数等,配置在回调后克隆;这是项目明确选择的底层扩展。`NewConnector` 返回标准 driver.Connector,供已有基础设施使用;它不接受启动检查选项。任意驱动仍可用核心 `database.OpenConnector`。
错误有安全摘要,errors.Is/As 保留 TLS/驱动/项目检查的原因。配置验证不读证书、不连接数据库;Open 才读取证书、创建池并握手。不要自行打印原始配置或展开可能包含内部信息的 driver 字段给用户。
## 字段级配置错误
Validate 返回独立字段与安全原因,例如 port(1–65535)、readTimeout、pool.maxIdleConns、tls.keyFile。项目调用 `config.AtField("database", err)` 后,统一加载器补齐 database 前缀、文件、环境名和行列,无需重复写端口/连接池校验。TLS 成对配置会指出缺失的那个字段。连接失败、驱动和证书文件的内部原因仍通过 errors.Is/As 保留,不把凭据放进公开提示。见 [配置诊断](../../CONFIG.md#框架模块的字段诊断)。

View File

@@ -19,6 +19,7 @@ import (
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
"git.yuntuo.hk/k2safe/go-omni/logx"
mysqldriver "github.com/go-sql-driver/mysql"
)
@@ -61,45 +62,60 @@ var identifier = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
// Validate does not read certificate files or open connections. All defaults
// come from DefaultConfig; explicit zero I/O timeouts mean no driver deadline.
func (c Config) Validate() error {
if c.Socket == "" && (c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") || c.Port < 1 || c.Port > 65535) {
return errors.New("mysql: invalid host or port")
if c.Socket == "" {
if c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") {
return configdiag.New("host", "must be a nonempty hostname or IP address")
}
if c.Port < 1 || c.Port > 65535 {
return configdiag.New("port", "must be between 1 and 65535")
}
}
if strings.ContainsRune(c.Socket, 0) || strings.ContainsRune(c.Database, 0) || strings.ContainsRune(c.User, 0) {
return errors.New("mysql: invalid connection field")
for _, f := range []struct{ name, value string }{{"socket", c.Socket}, {"database", c.Database}, {"user", c.User}} {
if strings.ContainsRune(f.value, 0) {
return configdiag.New(f.name, "must not contain a NUL character")
}
}
if c.ConnectTimeout < 0 || c.ReadTimeout < 0 || c.WriteTimeout < 0 {
return errors.New("mysql: timeouts cannot be negative")
for _, f := range []struct {
name string
value time.Duration
}{{"connectTimeout", c.ConnectTimeout}, {"readTimeout", c.ReadTimeout}, {"writeTimeout", c.WriteTimeout}} {
if f.value < 0 {
return configdiag.New(f.name, "must be nonnegative")
}
}
if err := c.Pool.Validate(); err != nil {
return err
return configdiag.At("pool", err)
}
if c.Location != "" {
if _, err := time.LoadLocation(c.Location); err != nil {
return errors.New("mysql: invalid time location")
return configdiag.Wrap("location", "must be an available IANA time zone or UTC", err)
}
}
if c.Charset != "" && !identifier.MatchString(c.Charset) {
return errors.New("mysql: invalid charset")
return configdiag.New("charset", "must be a valid MySQL identifier")
}
if c.Collation != "" && !identifier.MatchString(c.Collation) {
return errors.New("mysql: invalid collation")
return configdiag.New("collation", "must be a valid MySQL identifier")
}
for key := range c.Session {
if !identifier.MatchString(key) {
return errors.New("mysql: invalid session variable name")
return configdiag.New("session", "keys must be valid MySQL identifiers")
}
}
if c.TLS.Mode != "" && c.TLS.Mode != "required" && c.TLS.Mode != "disabled" {
return errors.New("mysql: TLS mode must be required or disabled")
return configdiag.New("tls.mode", "must be required or disabled")
}
if (c.TLS.CertFile == "") != (c.TLS.KeyFile == "") {
return errors.New("mysql: TLS client certificate and key must be configured together")
if c.TLS.CertFile == "" {
return configdiag.New("tls.certFile", "is required when keyFile is configured")
}
return configdiag.New("tls.keyFile", "is required when certFile is configured")
}
if c.TLS.Mode == "disabled" && (c.TLS.CAFile != "" || c.TLS.CertFile != "" || c.TLS.ServerName != "") {
return errors.New("mysql: TLS fields cannot be used when disabled")
return configdiag.New("tls.mode", "must be required when TLS fields are configured")
}
if c.Socket != "" && c.TLS.Mode != "disabled" && c.TLS.ServerName == "" {
return errors.New("mysql: TLS over a unix socket requires serverName")
return configdiag.New("tls.serverName", "is required for TLS over a unix socket")
}
return nil
}

View File

@@ -5,6 +5,7 @@ import (
"crypto/tls"
"crypto/x509"
"database/sql"
"database/sql/driver"
"errors"
"net"
"os"
@@ -166,11 +167,64 @@ func TestMySQLConnectorTLSAndPoolIsolation(t *testing.T) {
return rollbackErr
}); !errors.Is(err, rollbackErr) {
t.Fatal(err)
} else {
var txErr *database.TxError
if !errors.As(err, &txErr) || txErr.Stage != database.TxStageWork || txErr.Outcome != database.TxOutcomeRolledBack {
t.Fatal("real MySQL rollback was not classified", err)
}
}
var count int
if err := second.QueryRowContext(ctx, "SELECT COUNT(*) FROM connector_check").Scan(&count); err != nil || count != 1 {
t.Fatalf("transaction count=%d err=%v", count, err)
}
// Commit/Rollback run against real MySQL, then a driver-boundary wrapper
// suppresses the acknowledgement. This is deterministic fault injection,
// not a claim to reproduce every wire-level disconnect timing.
for _, loseCommit := range []bool{true, false} {
name, id := "commitAcknowledgementLost", 3
if !loseCommit {
name, id = "rollbackAcknowledgementLost", 4
}
t.Run(name, func(t *testing.T) {
connector, err := mysql.NewConnector(cfg.Clone())
if err != nil {
t.Fatal(err)
}
ackErr, workErr := errors.New("injected lost acknowledgement"), errors.New("work rejected")
fault := &ackLossConnector{Connector: connector, loseCommit: loseCommit, ackErr: ackErr}
faultPool, err := database.OpenConnector(ctx, fault, pool)
if err != nil {
t.Fatal(err)
}
defer faultPool.Close()
err = faultPool.InTx(ctx, nil, func(tx *sql.Tx) error {
if _, err := tx.ExecContext(ctx, "INSERT INTO connector_check VALUES (?)", id); err != nil {
return err
}
if !loseCommit {
return workErr
}
return nil
})
var txErr *database.TxError
if !errors.As(err, &txErr) || txErr.Outcome != database.TxOutcomeUnknown || !errors.Is(err, ackErr) {
t.Fatal("unknown acknowledgement was treated as definite outcome", err)
}
wantCount := 0
if loseCommit {
wantCount = 1
if txErr.Stage != database.TxStageCommit || txErr.Err != ackErr || txErr.RollbackErr != sql.ErrTxDone || fault.commits.Load() != 1 || fault.rollbacks.Load() != 0 {
t.Fatal("commit failure was retried or misclassified", err)
}
} else if txErr.Stage != database.TxStageWork || !errors.Is(err, workErr) || txErr.RollbackErr != ackErr || fault.commits.Load() != 0 || fault.rollbacks.Load() != 1 {
t.Fatal("rollback failure lost its primary/secondary causes", err)
}
var stored int
if err := second.QueryRowContext(ctx, "SELECT COUNT(*) FROM connector_check WHERE id = ?", id).Scan(&stored); err != nil || stored != wantCount {
t.Fatal("independent connection observed unexpected persistence", stored, err)
}
})
}
if err := first.Close(); err != nil {
t.Fatal(err)
}
@@ -178,3 +232,58 @@ func TestMySQLConnectorTLSAndPoolIsolation(t *testing.T) {
t.Fatal("closing first pool affected second:", err)
}
}
type ackLossConnector struct {
driver.Connector
loseCommit bool
ackErr error
commits, rollbacks atomic.Int32
}
func (c *ackLossConnector) Connect(ctx context.Context) (driver.Conn, error) {
conn, err := c.Connector.Connect(ctx)
if err != nil {
return nil, err
}
return &ackLossConn{Conn: conn, owner: c}, nil
}
type ackLossConn struct {
driver.Conn
owner *ackLossConnector
}
func (c *ackLossConn) Ping(ctx context.Context) error { return c.Conn.(driver.Pinger).Ping(ctx) }
func (c *ackLossConn) BeginTx(ctx context.Context, opts driver.TxOptions) (driver.Tx, error) {
tx, err := c.Conn.(driver.ConnBeginTx).BeginTx(ctx, opts)
if err != nil {
return nil, err
}
return &ackLossTx{Tx: tx, owner: c.owner}, nil
}
type ackLossTx struct {
driver.Tx
owner *ackLossConnector
}
func (tx *ackLossTx) Commit() error {
tx.owner.commits.Add(1)
if err := tx.Tx.Commit(); err != nil {
return err
}
if tx.owner.loseCommit {
return tx.owner.ackErr
}
return nil
}
func (tx *ackLossTx) Rollback() error {
tx.owner.rollbacks.Add(1)
if err := tx.Tx.Rollback(); err != nil {
return err
}
if !tx.owner.loseCommit {
return tx.owner.ackErr
}
return nil
}

126
database/transaction.go Normal file
View File

@@ -0,0 +1,126 @@
package database
import (
"context"
"database/sql"
"errors"
)
// TxStage identifies the operation which produced the primary error.
type TxStage string
const (
TxStageBegin TxStage = "begin"
TxStageWork TxStage = "work"
TxStageCommit TxStage = "commit"
)
// TxOutcome describes only what InTx can confirm about its managed transaction.
// An absent/unrecognized value must be treated as unknown, never as rolled back.
type TxOutcome string
const (
TxOutcomeUnknown TxOutcome = "unknown"
TxOutcomeNotStarted TxOutcome = "notStarted"
TxOutcomeRolledBack TxOutcome = "rolledBack"
)
// TxError retains the primary failure and any rollback failure independently.
// Use errors.As to inspect it. errors.Is/As traverse Err first, then RollbackErr;
// inspect Err alone when the distinction between the two causes matters.
// Error text is for internal diagnostics and may include driver/business details.
type TxError struct {
Stage TxStage
Outcome TxOutcome
Err error
RollbackErr error
}
func (e *TxError) Error() string {
if e == nil {
return "transaction error"
}
message := string(e.Stage) + " transaction (" + string(e.Outcome) + ")"
if e.Err != nil {
message += ": " + e.Err.Error()
}
if e.RollbackErr != nil {
message += "; rollback transaction: " + e.RollbackErr.Error()
}
return message
}
// Unwrap preserves both causes, including sql.ErrTxDone. That sentinel means
// the local transaction is finished, not that the database confirmed rollback.
func (e *TxError) Unwrap() []error {
if e == nil {
return nil
}
if e.Err == nil {
if e.RollbackErr == nil {
return nil
}
return []error{e.RollbackErr}
}
if e.RollbackErr == nil {
return []error{e.Err}
}
return []error{e.Err, e.RollbackErr}
}
// InTx begins a transaction, calls work once, and commits only if work returns
// nil. Every returned error is a *TxError; a nil result confirms Commit succeeded.
// Commit failures always have an unknown outcome, even for cancellation or a
// driver error normally associated with a lost connection. InTx does not retry.
//
// work must use only the supplied transaction for transactional operations and
// must not Commit/Rollback it, issue transaction-control/implicit-commit SQL, or
// let it escape to background goroutines. Outcome describes this transaction,
// not files, other connections, nontransactional tables or external side effects.
//
// Panic and runtime.Goexit still attempt rollback and propagate unchanged; they
// do not return a TxError. Rollback errors during those unwinds cannot be returned.
// A nil context is treated as context.Background().
func (db *DB) InTx(ctx context.Context, options *sql.TxOptions, work func(*sql.Tx) error) error {
if db == nil || db.DB == nil {
return &TxError{Stage: TxStageBegin, Outcome: TxOutcomeNotStarted, Err: errors.New("database is not initialized")}
}
if work == nil {
return &TxError{Stage: TxStageBegin, Outcome: TxOutcomeNotStarted, Err: errors.New("transaction callback is nil")}
}
if ctx == nil {
ctx = context.Background()
}
tx, err := db.BeginTx(ctx, options)
if err != nil {
return &TxError{Stage: TxStageBegin, Outcome: TxOutcomeNotStarted, Err: err}
}
finished := false
defer func() {
if !finished {
// Only the panic/Goexit path reaches here without explicit cleanup.
_ = tx.Rollback()
}
}()
if err = work(tx); err != nil {
failure := txFailure(tx, TxStageWork, err)
finished = true
return failure
}
if err = tx.Commit(); err != nil {
failure := txFailure(tx, TxStageCommit, err)
finished = true
return failure
}
finished = true
return nil
}
func txFailure(tx *sql.Tx, stage TxStage, cause error) *TxError {
rollbackErr := tx.Rollback()
outcome := TxOutcomeUnknown
if stage == TxStageWork && rollbackErr == nil {
outcome = TxOutcomeRolledBack
}
return &TxError{Stage: stage, Outcome: outcome, Err: cause, RollbackErr: rollbackErr}
}

View File

@@ -0,0 +1,56 @@
package database_test
import (
"database/sql"
"encoding/json"
"errors"
"net/http/httptest"
"strings"
"testing"
omni "git.yuntuo.hk/k2safe/go-omni"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/response"
)
func TestInTxPublicErrorCompatibility(t *testing.T) {
private := errors.New("private database details")
for _, scenario := range []string{"business", "businessAndRollback", "commit"} {
t.Run(scenario, func(t *testing.T) {
f := &txFixture{}
if scenario == "businessAndRollback" {
f.rollbackErr = private
} else if scenario == "commit" {
f.commitErr = private
}
pool := txPool(t, f)
app := omni.New(omni.WithResponse(response.Config{CodeType: "int32", SuccessCode: 0, ErrorCode: 9999, MessageField: "msg", DataMode: "always"}),
omni.WithErrors(apierrors.Catalog{DefaultStatus: 200, Definitions: map[string]apierrors.Definition{
"INVALID": {Code: 400002, Message: "{field} is invalid", Params: []apierrors.Parameter{{Name: "field", Type: "string"}}},
}}))
defer app.Close()
app.POST("/record", func(c *omni.Context) error {
return pool.InTx(c.Context(), nil, func(*sql.Tx) error {
if scenario == "commit" {
return nil
}
return apierrors.ByCode("INVALID").WithParams(map[string]any{"field": "name"}).WithCause(private)
})
})
w := httptest.NewRecorder()
app.ServeHTTP(w, httptest.NewRequest("POST", "/record", nil))
var body map[string]any
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
t.Fatal(err)
}
wantStatus, wantCode, wantMessage := 200, float64(400002), "name is invalid"
if scenario == "commit" {
wantStatus, wantCode, wantMessage = 500, 9999, "internal server error"
}
data, hasData := body["data"]
if w.Code != wantStatus || body["code"] != wantCode || body["msg"] != wantMessage || !hasData || data != nil || strings.Contains(w.Body.String(), "private") {
t.Fatal("transaction wrapper changed public error response:", w.Code, w.Body.String())
}
})
}
}

View File

@@ -0,0 +1,441 @@
package database_test
import (
"context"
"database/sql"
"database/sql/driver"
"errors"
"fmt"
"runtime"
"sync"
"sync/atomic"
"testing"
"time"
"git.yuntuo.hk/k2safe/go-omni/database"
)
// The driver can persist a write and lose the commit acknowledgement, or fail
// before persisting it. Both must be indistinguishable to the generic wrapper.
type txFixture struct {
beginErr, commitErr, rollbackErr error
persistBeforeError bool
onBegin func(context.Context, driver.TxOptions)
onCommit func()
onRollback func()
begins, commits, rollbacks, saved atomic.Int64
}
func (f *txFixture) Connect(context.Context) (driver.Conn, error) { return &txConn{owner: f}, nil }
func (f *txFixture) Driver() driver.Driver { return f }
func (f *txFixture) Open(string) (driver.Conn, error) { return f.Connect(context.Background()) }
type txConn struct {
owner *txFixture
pending int64
}
func (c *txConn) Prepare(string) (driver.Stmt, error) { return nil, errors.New("not used") }
func (c *txConn) Close() error { return nil }
func (c *txConn) Begin() (driver.Tx, error) {
return c.BeginTx(context.Background(), driver.TxOptions{})
}
func (c *txConn) BeginTx(ctx context.Context, options driver.TxOptions) (driver.Tx, error) {
c.owner.begins.Add(1)
if c.owner.onBegin != nil {
c.owner.onBegin(ctx, options)
}
if c.owner.beginErr != nil {
return nil, c.owner.beginErr
}
c.pending = 0
return &txDriver{conn: c}, nil
}
func (c *txConn) ExecContext(context.Context, string, []driver.NamedValue) (driver.Result, error) {
c.pending++
return driver.RowsAffected(1), nil
}
type txDriver struct{ conn *txConn }
func (tx *txDriver) Commit() error {
f := tx.conn.owner
f.commits.Add(1)
if f.commitErr == nil || f.persistBeforeError {
f.saved.Add(tx.conn.pending)
}
if f.onCommit != nil {
f.onCommit()
}
return f.commitErr
}
func (tx *txDriver) Rollback() error {
f := tx.conn.owner
f.rollbacks.Add(1)
if f.onRollback != nil {
f.onRollback()
}
return f.rollbackErr
}
func txPool(t testing.TB, f *txFixture) *database.DB {
t.Helper()
pool := sql.OpenDB(f)
pool.SetMaxOpenConns(8)
t.Cleanup(func() { _ = pool.Close() })
return &database.DB{DB: pool}
}
type txCause struct{ name string }
func (e *txCause) Error() string { return e.name }
func checkTxError(t *testing.T, err error, stage database.TxStage, outcome database.TxOutcome, cause, rollback error) *database.TxError {
t.Helper()
var txErr *database.TxError
if !errors.As(fmt.Errorf("repository: %w", err), &txErr) {
t.Fatalf("missing typed transaction error: %v", err)
}
if txErr.Stage != stage || txErr.Outcome != outcome || txErr.Err != cause || txErr.RollbackErr != rollback {
t.Fatalf("unexpected transaction error: %+v", txErr)
}
if cause != nil && !errors.Is(err, cause) || rollback != nil && !errors.Is(err, rollback) {
t.Fatalf("cause chain lost: %v", err)
}
return txErr
}
func TestInTxStagesAndCauses(t *testing.T) {
primary, rollback := &txCause{"primary"}, &txCause{"rollback"}
for _, tc := range []struct {
name string
beginErr, workErr, commitErr, rb error
stage database.TxStage
outcome database.TxOutcome
wantWork, wantCommit, wantRB int64
}{
{"begin", primary, nil, nil, nil, database.TxStageBegin, database.TxOutcomeNotStarted, 0, 0, 0},
{"work", nil, primary, nil, nil, database.TxStageWork, database.TxOutcomeRolledBack, 1, 0, 1},
{"workAndRollback", nil, primary, nil, rollback, database.TxStageWork, database.TxOutcomeUnknown, 1, 0, 1},
{"commit", nil, nil, primary, nil, database.TxStageCommit, database.TxOutcomeUnknown, 1, 1, 0},
} {
t.Run(tc.name, func(t *testing.T) {
f := &txFixture{beginErr: tc.beginErr, commitErr: tc.commitErr, rollbackErr: tc.rb}
pool := txPool(t, f)
var calls int64
err := pool.InTx(context.Background(), nil, func(tx *sql.Tx) error {
calls++
_, err := tx.ExecContext(context.Background(), "write")
if err != nil {
t.Fatal(err)
}
return tc.workErr
})
rb := tc.rb
if tc.stage == database.TxStageCommit {
rb = sql.ErrTxDone // No driver rollback happened after Commit.
}
checkTxError(t, err, tc.stage, tc.outcome, primary, rb)
var typedCause *txCause
if !errors.As(err, &typedCause) || typedCause != primary {
t.Fatal("primary typed cause not first in chain")
}
if calls != tc.wantWork || f.commits.Load() != tc.wantCommit || f.rollbacks.Load() != tc.wantRB || f.saved.Load() != 0 {
t.Fatal("unexpected callback/commit/rollback calls or persisted write")
}
})
}
t.Run("rollbackTypedCause", func(t *testing.T) {
pool := txPool(t, &txFixture{rollbackErr: rollback})
cause := errors.New("work failed")
err := pool.InTx(nil, nil, func(*sql.Tx) error { return cause })
var found *txCause
if !errors.As(err, &found) || found != rollback {
t.Fatal("typed rollback cause lost")
}
})
}
func TestInTxCommitOutcomeUnknown(t *testing.T) {
for _, cause := range []error{&txCause{"lost acknowledgement"}, context.DeadlineExceeded, driver.ErrBadConn} {
for _, persisted := range []bool{false, true} {
t.Run(fmt.Sprintf("%s/persisted=%t", cause, persisted), func(t *testing.T) {
f := &txFixture{commitErr: cause, persistBeforeError: persisted}
pool := txPool(t, f)
calls := 0
err := pool.InTx(nil, nil, func(tx *sql.Tx) error {
calls++
_, err := tx.ExecContext(context.Background(), "register object")
return err
})
checkTxError(t, err, database.TxStageCommit, database.TxOutcomeUnknown, cause, sql.ErrTxDone)
wantSaved := int64(0)
if persisted {
wantSaved = 1
}
if f.saved.Load() != wantSaved || calls != 1 || f.commits.Load() != 1 || f.rollbacks.Load() != 0 {
t.Fatal("commit was retried, rolled back, or incorrectly modeled")
}
})
}
}
}
func TestInTxCancellationAndOwnership(t *testing.T) {
t.Run("beforeBegin", func(t *testing.T) {
f := &txFixture{}
pool := txPool(t, f)
ctx, cancel := context.WithCancel(context.Background())
cancel()
err := pool.InTx(ctx, nil, func(*sql.Tx) error { t.Fatal("work called"); return nil })
checkTxError(t, err, database.TxStageBegin, database.TxOutcomeNotStarted, context.Canceled, nil)
if f.begins.Load() != 0 {
t.Fatal("driver began a canceled transaction")
}
})
for _, returnedWorkError := range []bool{false, true} {
t.Run(fmt.Sprintf("automaticRollback/workError=%t", returnedWorkError), func(t *testing.T) {
rolledBack := make(chan struct{})
rollbackCause := errors.New("rollback connection lost")
f := &txFixture{rollbackErr: rollbackCause, onRollback: func() { close(rolledBack) }}
pool := txPool(t, f)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
err := pool.InTx(ctx, nil, func(*sql.Tx) error {
cancel()
select {
case <-rolledBack:
case <-time.After(3 * time.Second):
t.Fatal("automatic rollback did not run")
}
if returnedWorkError {
return context.Canceled
}
return nil
})
stage, primary := database.TxStageCommit, error(sql.ErrTxDone)
if returnedWorkError {
stage, primary = database.TxStageWork, context.Canceled
}
checkTxError(t, err, stage, database.TxOutcomeUnknown, primary, sql.ErrTxDone)
// database/sql owns this asynchronous rollback and does not expose
// its driver error. Do not pretend ErrTxDone means successful rollback.
if errors.Is(err, rollbackCause) || f.rollbacks.Load() != 1 || f.commits.Load() != 0 {
t.Fatal("unexpected automatic rollback accounting")
}
})
}
t.Run("cancelDuringCommit", func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
f := &txFixture{commitErr: context.Canceled, persistBeforeError: true, onCommit: cancel}
pool := txPool(t, f)
err := pool.InTx(ctx, nil, func(tx *sql.Tx) error { _, err := tx.ExecContext(ctx, "write"); return err })
checkTxError(t, err, database.TxStageCommit, database.TxOutcomeUnknown, context.Canceled, sql.ErrTxDone)
if f.saved.Load() != 1 || f.rollbacks.Load() != 0 {
t.Fatal("cancellation hid a committed write")
}
})
for _, action := range []string{"commit", "rollback"} {
t.Run("manual/"+action, func(t *testing.T) {
f := &txFixture{}
pool := txPool(t, f)
cause := errors.New("callback failed")
err := pool.InTx(nil, nil, func(tx *sql.Tx) error {
var err error
if action == "commit" {
err = tx.Commit()
} else {
err = tx.Rollback()
}
if err != nil {
t.Fatal(err)
}
return cause
})
checkTxError(t, err, database.TxStageWork, database.TxOutcomeUnknown, cause, sql.ErrTxDone)
})
}
}
func TestInTxSuccessPanicAndConcurrency(t *testing.T) {
t.Run("success", func(t *testing.T) {
type key struct{}
ctx := context.WithValue(context.Background(), key{}, "value")
f := &txFixture{onBegin: func(received context.Context, opts driver.TxOptions) {
if received.Value(key{}) != "value" || !opts.ReadOnly || opts.Isolation != driver.IsolationLevel(sql.LevelSerializable) {
t.Error("context or options changed")
}
}}
pool := txPool(t, f)
if err := pool.InTx(ctx, &sql.TxOptions{Isolation: sql.LevelSerializable, ReadOnly: true}, func(*sql.Tx) error { return nil }); err != nil {
t.Fatal(err)
}
if f.commits.Load() != 1 || f.rollbacks.Load() != 0 {
t.Fatal("unexpected successful transaction calls")
}
})
for _, rbErr := range []error{nil, errors.New("rollback failed during unwind")} {
t.Run(fmt.Sprintf("panic/%v", rbErr), func(t *testing.T) {
f := &txFixture{rollbackErr: rbErr}
pool := txPool(t, f)
value := &struct{ text string }{"original panic"}
func() {
defer func() {
if recover() != value {
t.Error("panic identity lost")
}
}()
_ = pool.InTx(nil, nil, func(*sql.Tx) error { panic(value) })
}()
if f.rollbacks.Load() != 1 || f.commits.Load() != 0 {
t.Fatal("panic did not roll back exactly once")
}
})
}
t.Run("goexit", func(t *testing.T) {
f := &txFixture{}
pool := txPool(t, f)
done := make(chan struct{})
go func() {
defer close(done)
_ = pool.InTx(nil, nil, func(*sql.Tx) error { runtime.Goexit(); return nil })
}()
select {
case <-done:
case <-time.After(3 * time.Second):
t.Fatal("Goexit did not finish")
}
if f.rollbacks.Load() != 1 || f.commits.Load() != 0 {
t.Fatal("Goexit did not roll back exactly once")
}
})
t.Run("concurrent", func(t *testing.T) {
f := &txFixture{}
pool := txPool(t, f)
var group sync.WaitGroup
for i := 0; i < 64; i++ {
group.Add(1)
go func(i int) {
defer group.Done()
cause := &txCause{fmt.Sprint(i)}
err := pool.InTx(nil, nil, func(*sql.Tx) error { return cause })
var txErr *database.TxError
if !errors.As(err, &txErr) || txErr.Err != cause || txErr.Outcome != database.TxOutcomeRolledBack || !errors.Is(err, cause) {
t.Error("cross-request error state")
}
}(i)
}
group.Wait()
if f.rollbacks.Load() != 64 || f.commits.Load() != 0 {
t.Fatal("incorrect concurrent calls")
}
})
}
func TestInTxPreconditionsAndNestedErrors(t *testing.T) {
for _, pool := range []*database.DB{nil, {}} {
err := pool.InTx(nil, nil, func(*sql.Tx) error { t.Fatal("unexpected callback"); return nil })
var txErr *database.TxError
if !errors.As(err, &txErr) || txErr.Stage != database.TxStageBegin || txErr.Outcome != database.TxOutcomeNotStarted || txErr.Err == nil {
t.Fatal("uninitialized database was not classified")
}
}
f := &txFixture{}
pool := txPool(t, f)
err := pool.InTx(nil, nil, nil)
var txErr *database.TxError
if !errors.As(err, &txErr) || txErr.Stage != database.TxStageBegin || f.begins.Load() != 0 {
t.Fatal("nil callback began a transaction")
}
inner := &database.TxError{Stage: database.TxStageCommit, Outcome: database.TxOutcomeUnknown, Err: errors.New("other transaction")}
err = pool.InTx(nil, nil, func(*sql.Tx) error { return inner })
outer := checkTxError(t, err, database.TxStageWork, database.TxOutcomeRolledBack, inner, nil)
if outer == inner || !errors.As(outer.Err, &txErr) || txErr != inner {
t.Fatal("nested error was flattened")
}
// Empty or typed-nil error values must not panic or expose nil unwrap entries.
for _, value := range []*database.TxError{nil, {}, {RollbackErr: inner}} {
_ = value.Error()
for _, child := range value.Unwrap() {
if child == nil {
t.Fatal("nil unwrap child")
}
}
}
}
func TestInTxCancellationRace(t *testing.T) {
const count = 128
rollbackCause := errors.New("rollback acknowledgement unavailable")
var rollbacks sync.WaitGroup
rollbacks.Add(count)
f := &txFixture{rollbackErr: rollbackCause, onRollback: rollbacks.Done}
pool := txPool(t, f)
for i := 0; i < count; i++ {
ctx, cancel := context.WithCancel(context.Background())
workFailure := i%2 == 0
err := pool.InTx(ctx, nil, func(*sql.Tx) error {
cancel()
if i%3 == 0 {
runtime.Gosched() // Let the automatic rollback race explicit cleanup.
}
if workFailure {
return context.Canceled
}
return nil
})
cancel()
var txErr *database.TxError
if !errors.As(err, &txErr) || txErr.Outcome != database.TxOutcomeUnknown || f.commits.Load() != 0 {
t.Fatal("cancellation was treated as confirmed rollback", err)
}
wantStage := database.TxStageCommit
if workFailure {
wantStage = database.TxStageWork
}
if txErr.Stage != wantStage || txErr.RollbackErr != rollbackCause && txErr.RollbackErr != sql.ErrTxDone {
t.Fatal("unexpected cancellation diagnostics", err)
}
if txErr.RollbackErr == rollbackCause && !errors.Is(err, rollbackCause) {
t.Fatal("observed rollback failure lost")
}
}
done := make(chan struct{})
go func() { rollbacks.Wait(); close(done) }()
select {
case <-done:
case <-time.After(3 * time.Second):
t.Fatal("canceled transactions did not finish rollback")
}
}
func BenchmarkInTx(b *testing.B) {
cause := errors.New("example failure")
for _, name := range []string{"success", "beginFailure", "workFailure", "rollbackFailure", "commitUnknown"} {
b.Run(name, func(b *testing.B) {
f := &txFixture{}
var workErr error
switch name {
case "beginFailure":
f.beginErr = cause
case "workFailure":
workErr = cause
case "rollbackFailure":
workErr, f.rollbackErr = cause, cause
case "commitUnknown":
f.commitErr = cause
}
pool := txPool(b, f)
work := func(*sql.Tx) error { return workErr }
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
err := pool.InTx(context.Background(), nil, work)
if (err == nil) != (name == "success") {
b.Fatal(err)
}
}
})
}
}

View File

@@ -1,5 +1,7 @@
# 统一错误目录
v0.26 新增的通用多语言适配见 [i18n](../i18n/README.md)。目录条目可显式引用 `translation: {namespace, key}`;`errori18n.Bind/Source` 支持从同一份 Go 目录生成源词典。翻译只影响公开展示,不改变业务码、HTTP 状态、失败语义、参数契约或 cause。没有引用或没有启用适配时沿用现有行为;幂等重放保留首次语言。该能力需要 v0.26.0-preview 或更新版本。
从 `v0.10.0-preview` 起,业务可以只返回逻辑错误标识,框架从项目目录取得 HTTP 状态、对外业务码与公开提示。目录不依赖具体业务、数据库、身份模型或网络层,也不定义固定返回体。
## 基本规则

View File

@@ -9,16 +9,19 @@ import (
"math"
"reflect"
"strings"
"unicode"
"unicode/utf8"
)
// Definition belongs to the project. Code is the public string/integer code;
// the Catalog map key is the distinct logical identifier used by ByCode.
// Nil Code retains legacy response code mapping (useful when migrating).
type Definition struct {
Status int `json:"status,omitempty"`
Code any `json:"code,omitempty"`
Message string `json:"message"`
Params []Parameter `json:"params,omitempty"`
Status int `json:"status,omitempty"`
Code any `json:"code,omitempty"`
Message string `json:"message"`
Params []Parameter `json:"params,omitempty"`
Translation *Translation `json:"translation,omitempty"`
}
type Catalog struct {
@@ -99,6 +102,10 @@ func (c Catalog) Clone() Catalog {
out := c
out.Definitions = make(map[string]Definition, len(c.Definitions))
for key, d := range c.Definitions {
if d.Translation != nil {
ref := *d.Translation
d.Translation = &ref
}
if d.Params != nil {
params := make([]Parameter, len(d.Params))
copy(params, d.Params)
@@ -126,6 +133,13 @@ func NewRegistry(c Catalog) (*Registry, error) {
}
messages := map[string][]messagePart{}
for key, d := range c.Definitions {
if d.Translation != nil {
for _, value := range []string{d.Translation.Namespace, d.Translation.Key} {
if value == "" || len(value) > 256 || !utf8.ValidString(value) || strings.TrimSpace(value) != value || strings.IndexFunc(value, unicode.IsControl) >= 0 {
return nil, fmt.Errorf("errors: %s has invalid translation reference", key)
}
}
}
if strings.TrimSpace(key) == "" || strings.TrimSpace(key) != key || strings.ContainsAny(key, "\r\n\t") {
return nil, fmt.Errorf("errors: invalid logical identifier %q", key)
}

46
errors/localization.go Normal file
View File

@@ -0,0 +1,46 @@
package apierrors
import "context"
// Translation is an explicit reference into an independently owned dictionary.
// It does not change the logical error key, status, public code or parameters.
type Translation struct {
Namespace string `json:"namespace"`
Key string `json:"key"`
}
// Localizer is an optional presentation adapter. false preserves the original
// public message. Implementations must support concurrent calls and must not
// mutate registry or failure, or expose causes/parameter values in diagnostics.
type Localizer interface {
Localize(context.Context, *Registry, *APIError) (string, bool)
}
// Definition returns detached metadata for the effective catalog scope.
func (r *Registry) Definition(key string) (Definition, bool) {
if r == nil {
return Definition{}, false
}
d, ok := r.catalog.Definitions[key]
if d.Params != nil {
d.Params = append([]Parameter{}, d.Params...)
}
if d.Translation != nil {
ref := *d.Translation
d.Translation = &ref
}
return d, ok
}
// Parameters returns detached public template arguments. This is for explicit
// presentation adapters, not automatic serialization into an API response.
func (e *APIError) Parameters() map[string]any {
if e == nil {
return nil
}
out := make(map[string]any, len(e.params))
for k, v := range e.params {
out[k] = v
}
return out
}

View File

@@ -0,0 +1,34 @@
syntax = "v1"
errorCatalog CommonErrors (
defaultStatus: 200
errors: {
"LimitExceeded": {
"key": "LIMIT_EXCEEDED", "code": 400001,
"message": "Limit: {max}",
"params": [{"name":"max","type":"int64","example":100}],
"translation": {"namespace":"limits","key":"exceeded"}
}
}
)
response CommonResponse (
codeType: int32
successCode: 0
errorCode: 9999
body: {"code":"${code}", "detail":"${message}", "ok":"${ok}"}
)
type ValueResponse {
Value string `json:"value"`
}
@server (
group: limits
response: CommonResponse
errors: LimitExceeded
)
service localized-api {
@handler value
get /value returns (ValueResponse)
}

17
examples/cors/README.md Normal file
View File

@@ -0,0 +1,17 @@
# CORS 与正常鉴权
在框架根目录运行:
```sh
go run ./examples/cors
WEB_ORIGIN=https://admin.example go run ./examples/cors
```
仅监听临时 localhost 端口,不依赖数据库或外部服务。配置由统一 LoadProject 加载,示例模拟浏览器 Origin/预检请求头,依次检查:
- OPTIONS 预检返回 204,无需登录。
- 实际 GET 未登录仍返回 401,前端可读取这条错误响应。
- 示例凭据通过后返回 200。
- 404 也保留允许来源的 CORS 响应头。
示例 Token 仅用于展示,项目使用自身鉴权实现。CORS 不验证 Token,也不保证其他客户端不能调用 API。完整规则与配置见 [cors](../../cors/README.md)。

10
examples/cors/config.yaml Normal file
View File

@@ -0,0 +1,10 @@
server:
cors:
enabled: true
allowedOrigins:
- "${WEB_ORIGIN:-https://web.example}"
allowedMethods: [GET, POST]
allowedHeaders: [Authorization, Content-Type]
exposedHeaders: [X-Request-ID]
allowCredentials: false
maxAge: 10m

76
examples/cors/main.go Normal file
View File

@@ -0,0 +1,76 @@
package main
import (
"errors"
"flag"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/config"
)
type Config struct {
Server config.Config `json:"server"`
}
func (c Config) Validate() error { return config.AtField("server", c.Server.Validate()) }
func run(path string) error {
cfg, err := config.LoadProject(path, Config{Server: config.Default()})
if err != nil {
return err
}
a := omni.New(omni.WithConfig(cfg.Server))
defer a.Close()
a.GET("/private", func(c *omni.Context) error {
if c.Header("Authorization") != "Bearer example-only" {
return omni.Unauthorized("LOGIN_REQUIRED", "login required")
}
return c.OK("authenticated")
})
server := httptest.NewServer(a)
defer server.Close()
client := &http.Client{Timeout: 3 * time.Second}
defer client.CloseIdleConnections()
for _, tc := range []struct {
method, path, auth string
status int
}{{"OPTIONS", "/private", "", 204}, {"GET", "/private", "", 401}, {"GET", "/private", "Bearer example-only", 200}, {"GET", "/missing", "", 404}} {
r, _ := http.NewRequest(tc.method, server.URL+tc.path, nil)
r.Header.Set("Origin", cfg.Server.CORS.AllowedOrigins[0])
if tc.method == "OPTIONS" {
r.Header.Set("Access-Control-Request-Method", "GET")
r.Header.Set("Access-Control-Request-Headers", "authorization")
}
if tc.auth != "" {
r.Header.Set("Authorization", tc.auth)
}
res, err := client.Do(r)
if err != nil {
return err
}
_, readErr := io.Copy(io.Discard, res.Body)
closeErr := res.Body.Close()
if err := errors.Join(readErr, closeErr); err != nil {
return err
}
if res.StatusCode != tc.status || res.Header.Get("Access-Control-Allow-Origin") == "" {
return fmt.Errorf("unexpected CORS response: %d", res.StatusCode)
}
fmt.Printf("%s %s -> %d, CORS allowed\n", tc.method, tc.path, res.StatusCode)
}
return nil
}
func main() {
path := flag.String("config", "examples/cors/config.yaml", "selected configuration file")
flag.Parse()
if err := run(*path); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

View File

@@ -0,0 +1,18 @@
# 配置诊断、严格 HTTP 状态、类型预检和日志时区
在框架根目录运行,不需要数据库、账号或外部服务。HTTP 示例只访问临时 localhost 服务:
```sh
go run ./examples/developerTools
DEMO_LOG_TIME_ZONE=UTC go run ./examples/developerTools
# 以下两条应非零退出,输出具体字段和环境变量名,不输出原值:
DEMO_DB_PORT=70000 go run ./examples/developerTools
DEMO_LOG_LEVEL=invalid go run ./examples/developerTools
```
- 配置只做校验,没有连接数据库。失败指向 `database.port` / `server.log.level`,并保留文件行列与变量名。
- Logger 使用选定时区;只改变记录的时间戳,不修改全局 time.Local 和业务时间字段。
- `copyx.Check[Input, Result]` 发现切片元素内的不兼容字段;加上显式 `strconv.Atoi` converter 后通过。预检不执行 converter,真实非法数字仍在 Copy 时返回错误。
- JSON 调用显式 `WithSuccessStatuses(200)`,拒绝 201,框架仍负责关闭响应体。
详见 [配置](../../CONFIG.md)、[日志](../../logx/README.md)、[HTTP 客户端](../../httpc/README.md)、[copyx](../../COPY.md)。

View File

@@ -0,0 +1,7 @@
# Validates database settings without connecting in this demonstration.
database:
port: "${DEMO_DB_PORT:-3306}"
server:
log:
level: "${DEMO_LOG_LEVEL:-info}"
timeZone: "${DEMO_LOG_TIME_ZONE:-Asia/Shanghai}"

View File

@@ -0,0 +1,104 @@
// Run from the framework root with: go run ./examples/developerTools
package main
import (
"context"
"errors"
"flag"
"fmt"
"io"
"net/http"
"net/http/httptest"
"os"
"strconv"
_ "time/tzdata" // This example embeds zones; production builds can use -tags timetzdata.
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/copyx"
mysql "git.yuntuo.hk/k2safe/go-omni/database/mysql"
"git.yuntuo.hk/k2safe/go-omni/httpc"
"git.yuntuo.hk/k2safe/go-omni/logx"
)
type Config struct {
Database mysql.Config `json:"database"`
Server config.Config `json:"server"`
}
func Default() Config {
c := config.Default()
c.Log.Output = "stdout"
return Config{Database: mysql.DefaultConfig(), Server: c}
}
func (c Config) Validate() error {
if err := c.Database.Validate(); err != nil {
return config.AtField("database", err)
}
return config.AtField("server", c.Server.Validate())
}
type Input struct{ Values []TextValue }
type TextValue struct{ Number string }
type Result struct{ Values []NumberValue }
type NumberValue struct{ Number int }
func run(path string) error {
cfg, err := config.LoadProject(path, Default())
if err != nil {
return err
}
// Validation only: this example never opens a database.
logger, err := logx.New(cfg.Server.Log)
if err != nil {
return err
}
defer logger.Close()
logger.InfoContext(context.Background(), "instance time zone", "configuredZone", cfg.Server.Log.TimeZone)
if err := copyx.Check[Input, Result](); !errors.Is(err, copyx.ErrType) {
return fmt.Errorf("expected a type mismatch, got %v", err)
}
// Explicit Go converter owns the number conversion; no JSON mapping file.
options := []copyx.Option{copyx.WithConverter(strconv.Atoi)}
if err := copyx.Check[Input, Result](options...); err != nil {
return err
}
var output Result
if err := copyx.Copy(&output, Input{Values: []TextValue{{Number: "42"}}}, options...); err != nil {
return err
}
fmt.Printf("type check and copy succeeded: %d\n", output.Values[0].Number)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/created" {
w.WriteHeader(http.StatusCreated)
}
io.WriteString(w, `{"value":"example"}`)
}))
defer server.Close()
client, err := httpc.New(httpc.Config{DisableLogging: true})
if err != nil {
return err
}
defer client.Close()
var reply struct {
Value string `json:"value"`
}
if err := client.JSON(context.Background(), http.MethodGet, server.URL, nil, &reply, httpc.WithSuccessStatuses(http.StatusOK)); err != nil {
return err
}
err = client.JSON(context.Background(), http.MethodGet, server.URL+"/created", nil, &reply, httpc.WithSuccessStatuses(http.StatusOK))
var status *httpc.StatusError
if !errors.As(err, &status) || status.StatusCode != http.StatusCreated {
return fmt.Errorf("expected status rejection, got %v", err)
}
fmt.Println("strict JSON: 200 accepted, 201 rejected")
return nil
}
func main() {
path := flag.String("config", "examples/developerTools/config.yaml", "selected configuration file")
flag.Parse()
if err := run(*path); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

View File

@@ -0,0 +1,15 @@
# HTTP 限额与停机
从仓库根目录运行 `go run ./examples/httpSafety`,可用 `-addr 127.0.0.1:18080` 指定端口。仅使用本机 HTTP 和模拟收尾,不连接数据库或其他服务。
上传验证:
```sh
dd if=/dev/zero bs=1048576 count=9 2>/dev/null | curl --data-binary @- http://127.0.0.1:8080/upload
```
普通路由默认 1 MiB,上传路由允许 16 MiB,服务硬上限 20 MiB;以上请求返回已读取的 9437184 字节。将上传路由改为 21 MiB 后,启动会报告该路由与服务硬上限冲突。实际文件上传须额外预留 multipart 字段/边界开销,并配置文件大小限制。
停机验证:另一个终端执行 `curl http://127.0.0.1:8080/work`,随后在服务终端按 Ctrl-C。先等待 1s 排空,再取消请求,处理器模拟 100ms 收尾;日志中先出现 `request cleanup completed`,再出现资源关闭记录。因为用尽排空预算,本次退出仍返回超时错误。
若处理器不在取消收尾窗口内退出,App 和 `deps.Finish(runErr)` 返回 `lifecycle.ErrResourcesRetained` 并跳过资源关闭。真实项目拥有的日志也必须在该错误下保留,生成入口已处理。进程应非零退出;库不自行退出进程,也不强杀 goroutine。

View File

@@ -0,0 +1,62 @@
// This example uses only local HTTP and simulated work; no external services.
package main
import (
"context"
"errors"
"flag"
"io"
"log"
"os"
"os/signal"
"syscall"
"time"
omni "git.yuntuo.hk/k2safe/go-omni"
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
)
func main() {
address := flag.String("addr", "127.0.0.1:8080", "local listen address")
flag.Parse()
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
if err := run(ctx, *address); err != nil {
log.Print(err)
os.Exit(1)
}
}
func run(ctx context.Context, address string) (runErr error) {
deps := lifecycle.NewRuntime(ctx)
deps.OnClose(func() error { log.Print("shared resource closed after requests completed"); return nil })
defer func() { runErr = deps.Finish(runErr) }()
app := omni.New(omni.WithHTTPConfig(omni.HTTPConfig{
DefaultMaxBodyBytes: 1 << 20,
MaxBodyBytes: 20 << 20,
ShutdownCancelTimeout: time.Second,
}), omni.WithShutdownTimeout(time.Second))
defer func() { runErr = errors.Join(runErr, app.Close()) }()
app.HandleWithBodyLimit("POST", "/upload", 16<<20, nil, func(c *omni.Context) error {
n, err := io.Copy(io.Discard, c.Request().Body)
if err != nil {
return err
}
return c.OK(map[string]int64{"bytes": n})
})
app.GET("/work", func(c *omni.Context) error {
log.Print("request accepted")
select {
case <-time.After(10 * time.Second):
return c.OK("finished")
case <-c.Context().Done():
// Simulate bounded rollback/cleanup after cancellation. Dependencies
// remain available until this handler and its defers have returned.
time.Sleep(100 * time.Millisecond)
log.Print("request cleanup completed")
return c.Context().Err()
}
})
log.Printf("listening on %s; POST /upload or GET /work, then Ctrl-C", address)
return app.RunContext(deps.Context(), address)
}

Some files were not shown because too many files have changed in this diff Show More