Compare commits

..

4 Commits

62 changed files with 4208 additions and 328 deletions

View File

@@ -1,5 +1,23 @@
# Changelog # Changelog
## v0.21.0-preview(2026-09-26)
- httpc 可配置环境/直连/固定代理、常用连接阶段超时和独立空闲池参数。
- JSON 新增逐请求 Header 快照与响应大小选项,保持有界读取、关闭和认证隔离。
- 成功请求少复制和退避计算,丢弃响应流式处理,错误体有界回收以改善连接复用。
- 增加真实本地代理、超时、连接限制/回收、并发、日志脱敏、gzip、fuzz 和修改前后基准。
- JSON 自定义方法接口签名需同步 variadic 参数;借用 Transport 的参数混配现在报错。详见 httpc/README.md。
## v0.20.0-preview(2026-09-26)
- 生成项目配置加载入口持续由生成器维护;保留业务字段、默认值、校验与 NewLoader 扩展文件。
- config.LoadProject 统一 JSON/YAML/XML、环境覆盖和校验;提供实例级 Decoder 注册。
- 文件/字段/环境/行列的安全诊断,Invalid/AtField 与原始错误链。
- 增加格式等价、精度、null/空数组、并发、资源限制、默认值隔离、真实生成项目与性能验收。
- 旧手写 Load 必须显式迁移;低层 API 保留。固定数组严格长度、YAML alias/merge/custom tag 拒绝。用法及限制见 CONFIG.md。
## v0.19.0-preview(2026-09-25) ## v0.19.0-preview(2026-09-25)
- 新增通用可选 copyx:构造器、浅/深拷贝、精确类型转换、字段规则、nil/零值策略、并发复用。 - 新增通用可选 copyx:构造器、浅/深拷贝、精确类型转换、字段规则、nil/零值策略、并发复用。

178
CONFIG.md Normal file
View File

@@ -0,0 +1,178 @@
# 统一配置加载
v0.20 起,新生成服务的 `main.go` 调用生成器维护的 `config.Load`。项目只需要定义业务字段、默认值和校验;日志、HTTP、数据库等配置使用同一流程,配置加载时不打开数据库、监听端口或创建日志文件。
```text
代码默认值 → envDefault(仅填零值)→ 可选配置文件
→ 完整结构体环境覆盖(含别名)→ 框架规范化 → 项目 Validate
→ 初始化日志及项目依赖 → 启动服务
```
## 生成项目要维护什么
| 文件 | 维护方 | 用途 |
| --- | --- | --- |
| `main.go` | 生成器 | 启动、信号、请求排空、依赖关闭 |
| `internal/config/loadGen.go` | 生成器 | 调用统一加载入口;纳入 `generate --check` |
| `internal/config/config.go` | 项目 | `Config`、`Default()`、`Validate()` |
| `internal/config/loader.go` | 项目 | `NewLoader()`,只有增加文件格式时才需修改 |
| `etc/config.yaml` | 项目 | 部署配置;可改用 JSON/XML 并传 `-config` |
| `internal/svc` | 项目与生成器分文件 | 初始化依赖、登记清理、AppOptions |
比如增加业务的批次数量:在 Config 增加一个字段,在 Validate 加范围校验;需要部署值时改配置文件或环境变量。不修改 main、Load 或 Handler。
```go
// 添加到项目 Config。
BatchSize int `json:"batchSize" envDefault:"100"`
// 添加到项目 Validate。
if c.BatchSize < 1 || c.BatchSize > 1000 {
return omniConfig.Invalid("batchSize", "must be between 1 and 1000")
}
```
自动支持 `GO_OMNI_BATCH_SIZE`。生成服务的空环境值策略为 `ignore`,0 和 false 是有效覆盖;范围由 Validate 决定。
重新生成保留 config.go、loader.go 和 svc 的手写扩展。
## 手写项目用法
```go
package main
import (
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
type Config struct {
Timeout time.Duration `json:"timeout" envDefault:"3s"`
Limit int `json:"limit" envDefault:"100"`
Token string `json:"-" env:"APP_TOKEN,secret"`
}
func (c Config) Validate() error {
if c.Timeout <= 0 {
return config.Invalid("timeout", "must be positive")
}
return nil
}
// cfg, err := config.LoadProject("etc/config.yaml", Config{},
// config.EnvOptions{Prefix: "APP_", Empty: "reject"})
```
`LoadProject` 接收结构体默认值,可实现值接收器或指针接收器的 `Validate() error`。空路径表示不读文件。
`DecodeFile` 只做文件覆盖,不做 envDefault、环境覆盖或业务 Validate;旧的两参数调用仍有效,可用结构体或结构体指针。低层 `ApplyDefaults`、`OverlayEnv`、`LoadFile` 继续存在。
嵌套校验用 `config.AtField("client", err)` 加路径前缀。公共原因通过 `config.Invalid` 提供,原因中不要包含密码、DSN、Token 或输入原值。
## JSON / YAML / XML
按文件扩展名选择解析器:`.json`、`.yaml`、`.yml`、`.xml`,大小写不敏感。所有格式使用 Go 结构体的 **json 字段名**,无需为 XML 复制一份结构或标签。
以下配置等价:
```json
{"client":{"timeout":"2s"},"enabled":false,"ports":[8080,8081],"optional":null}
```
```yaml
client:
timeout: 2s
enabled: false
ports: [8080, 8081]
optional: null
```
```xml
<config>
<client><timeout>2s</timeout></client>
<enabled>false</enabled>
<ports type="array"><item>8080</item><item>8081</item></ports>
<optional type="null"/>
</config>
```
XML 必须使用一个 `<config>` 根;普通叶节点按目标 Go 类型转换。数组显式 `type="array"` 加 `<item>`,不根据重复元素猜测类型。`type="array"` 空元素表示空数组,`type="object"` 表示空对象,`type="null"` 表示 null。`any` 中的 XML 叶节点默认是字符串,需要动态数字/布尔时标注 `type="number"` / `type="bool"`;也可用 `type="string"`。
类型和覆盖规则:
- 缺失字段保留默认值;显式 0、false、空字符串正常覆盖。
- null 清空指针、map、slice、interface;普通值字段保持默认值,沿用 Go JSON 解码语义。需要区分 null 时用指针。
- 数组整体替换;空 slice 保持非 nil;固定长度 Go 数组要求长度一致,避免丢弃多余配置。
- map 合并键,已提供的键整体覆盖对应值;map 中未提供的键保留。
- duration 支持 `250ms`、`3s`;整数保持精度,动态 JSON 数字使用 `json.Number`,不先转 float64。
- 自定义 `json.Unmarshaler` / `encoding.TextUnmarshaler` 仍可接入。这些回调是可信项目代码,必须自己保证并发和副作用安全;不要修改共享外部状态。
- JSON `,string` 字段、嵌入字段遵循 JSON 字段选择;未知字段、重复绑定、冲突均拒绝。
内置解析器限制:单文件最多 1 MiB,解析最多 64 层、100000 个节点;绑定和默认值复制也有深度限制。只接受普通文件,不读取设备或命名管道。不支持多文档、重复字段、无效 UTF-8;YAML 拒绝 alias/merge/custom tag,XML 拒绝 DTD/外部实体、命名空间、业务属性和混合内容。XML 仅支持配置所需的子集,不是通用 XML 文档映射器。
这些限制针对启动配置,避免意外超大文件、别名扩展和模糊覆盖。自定义值解码器的内部字段校验由该解码器负责;例如现有错误目录使用其专有解码规则。
## 环境覆盖
统一入口的 EnvOptions 控制前缀、空值策略、别名和 Lookup。完整 JSON 路径自动转换为 UPPER_SNAKE_CASE:`client.timeout` → `APP_CLIENT_TIMEOUT`。
```go
config.EnvOptions{
Prefix: "GO_OMNI_", Empty: "ignore",
Aliases: config.ServerEnvAliases("GO_OMNI_", "server"),
}
```
生成项目同时接受完整的 `GO_OMNI_SERVER_HTTP_READ_TIMEOUT` 与短别名 `GO_OMNI_HTTP_READ_TIMEOUT`。完整名存在时优先,即便其空值被 ignore,也不回退别名。不会再二次调用 ApplyEnv 覆盖业务字段。
显式 `env:"CUSTOM_NAME"` 是绝对名称;`env:"-"` 跳过子树。`json:"-"` 只在声明显式 env 名时参与,可让密钥只能从环境加载。`secret` 标签表达用途,解析错误对所有字段都不打印值。slice/map 的环境值采用 JSON。`required` 检查字符串非空、指针/集合非 nil;复杂范围由 Validate 负责。详见 [完整业务环境覆盖](DEVELOPMENT.md#完整业务配置的环境覆盖)。
显式环境选择日志 stdout/stderr 会清除文件 profile 的 File/Rotation,除非同时提供非空 File 覆盖。该兼容规则由框架处理。
## 扩展解析器
内置格式直接可用。新增格式只在 `internal/config/loader.go` 的 `NewLoader` 注册:
```go
func NewLoader() (*omniConfig.Loader, error) {
return omniConfig.NewLoader(
omniConfig.WithDecoder("custom", omniConfig.DecoderFunc(
func(data []byte) (omniConfig.Document, error) {
// 演示注册扩展名;实际可在这里调用项目选定的 TOML 等解析库。
return (omniConfig.JSONDecoder{}).Decode(data)
},
)),
)
}
```
Decoder 返回 Document:Value 是 map[string]any 根,子值为 string/bool/json.Number/nil/[]any/map[string]any;无自身类型的叶节点可返回 config.Text。Positions 可选,用 `client.timeout` 或 `servers[0].timeout` 标记 1-based 行列。不提供位置时仍有文件、字段、原因。
自己的语法错误可返回 `&config.Error{Field: "client.timeout", Line: 3, Column: 5, Reason: "expected a duration"}`。Reason 必须安全;普通第三方错误仅公开 `invalid configuration syntax`,内部 cause 通过 errors.Is/As 保留。
注册表属于 Loader 实例,无全局 init 注册;不能覆盖已有扩展名。Loader 可共享,但 Decoder 必须支持并发且不保留/修改输入。解析插件是可信 Go 代码,框架不是插件沙箱。新格式依赖由选择它的项目显式引入,核心不强制引入 TOML 等库。
## 定位配置错误
```text
config "etc/config.yaml":4:12 field "client.timeout": expected a duration such as 5s
config env "APP_CLIENT_TIMEOUT" field "client.timeout": expected a valid time.Duration
```
`errors.As(err, &diagnostic)` 可获取 `*config.Error` 的 File/Field/Env/Line/Column/Reason;原 `DecodeError` 名称保留为别名。`errors.Is` 可判断文件不存在等 I/O 原因。
解析失败显示文件和可获取的行列;Validate 使用 Invalid/AtField 才能定位业务字段。环境覆盖导致的错误优先标环境变量,不错误地归因到文件;数组元素验证也会追溯到覆盖它的集合环境变量。默认值未出现在文件里时没有行列。YAML 语法错误通常只有行号,列显示 1。
不打印底层错误原文或配置值;任意普通 `errors.New(...)` 校验失败的公开原因是 `validation failed`,内部原因仍在链中。项目应改用 Invalid 给出安全的原因。第三方子配置校验若没有字段元数据,只定位到对应子树,框架不会从任意错误文案猜字段。
不要把 Unwrap 后的原始错误、整个配置对象写入公开日志/HTTP;自定义解析器与校验器传入的 Reason 由项目负责脱敏。
## v0.19 及更早项目迁移
先提交现有代码,再升级匹配的 CLI 与框架。旧项目的 `internal/config/config.go` 含手写 Load,新生成器会明确阻止 generate/check/dry-run,不会直接覆盖它。
1. 保留 Config、Default、Validate、业务配置结构及辅助函数。
2. 删除旧 Load。普通的 defaults/file/env/Validate 流程已由 LoadProject 接管;移除由此不再使用的 import。将自定义默认值和校验分别迁入 Default / Validate,自定义文件格式迁入 NewLoader。
3. 框架只在 `loader.go` 不存在时创建 NewLoader;已有项目文件同名时需要保证它提供同签名函数。无需手写 loadGen.go。
4. 重新生成,执行 `generate --check`、`go test ./...`、`go build .`,再用真实配置验证启动。
保留业务需要的显式 env 标签/规则;如果原 Load 有解密或其他项目特殊步骤,不要直接删除,应先明确其归属,必要时在自定义 Decoder 中处理。生成器不自动搬动或猜测手写行为。`api upgrade` 仍可预览较早版本的基础设施差异,但已修改的配置必须按上述步骤迁移。
不会移除低层配置 API;本次移除的是**新项目里重复手写的加载流程**,不删除旧项目的业务文件。
完整可运行例子:[examples/configuration](examples/configuration/README.md)。

View File

@@ -1,4 +1,4 @@
# 通用业务开发指南(v0.19.0-preview) # 通用业务开发指南(v0.21.0-preview)
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。 从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
@@ -102,7 +102,9 @@ defer reader.Close()
## 完整业务配置的环境覆盖 ## 完整业务配置的环境覆盖
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML 文件 → 全结构体环境覆盖 → 既有框架环境别名 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。 生成器维护 `internal/config/loadGen.go`;项目维护 Config/Default/Validate 和可选 NewLoader。三种格式、可扩展解析器、字段错误定位及旧 Load 迁移见 [统一配置指南](CONFIG.md)。
生成项目默认执行:代码与 `envDefault` 默认值 → JSON/YAML/XML 文件 → 全结构体环境覆盖(含别名)→ 规范化 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
```go ```go
type ClientConfig struct { type ClientConfig struct {
@@ -278,7 +280,7 @@ go-omni workspace check --file omni.workspace.json
## v0.13 项目升级 ## v0.13 项目升级
先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.19.0-preview(或显式引用当前源码)。 先使用与目标框架匹配的 CLI;CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.20.0-preview(或显式引用当前源码)。
```sh ```sh
go-omni api upgrade --dir . --dry-run go-omni api upgrade --dir . --dry-run

View File

@@ -6,6 +6,24 @@
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖 [examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。 `httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
## 请求追踪与基础指标
`omni.New()` 的默认链包含 RequestID、访问日志(可配置开关)和 Recovery;无需重复挂载。业务通过 `c.Context()` 传递 requestId,`logx.FromContext(ctx)` 取得该 App 的 Logger。手动组装 httpx 链时,也可使用 `middleware.RequestID("X-Request-ID")`、`middleware.AccessLog()` 和 `middleware.Recovery()`。
轻量进程指标直接使用 `metrics`,无需 OTel collector:
```go
// import omni "git.yuntuo.hk/k2safe/go-omni"
// import "git.yuntuo.hk/k2safe/go-omni/metrics"
app := omni.New()
measurements := metrics.New()
app.Use(measurements.Middleware())
app.GET("/metrics", measurements.Handler())
// 也可以直接读取 measurements.Snapshot()。
```
记录请求数、失败数、处理中数量、累计耗时与 HTTP 状态分布;返回的是 JSON 快照,并非 Prometheus 文本格式。指标接口是否暴露以及访问权限由项目配置。需要 HTTP/SQL/Redis spans 和标准 OTel exporter 时,使用 [telemetry 示例](examples/platform/main.go)。CORS 由项目中间件或代理配置,当前没有内置 CORS 模块。
## 缓存:cache 与 cache/redis ## 缓存:cache 与 cache/redis
```go ```go
@@ -52,30 +70,26 @@ application.OnShutdown(client.Close)
```go ```go
client, err := httpc.New(httpc.Config{ client, err := httpc.New(httpc.Config{
Timeout: 3*time.Second, MaxRetries: 2, MaxConnsPerHost: 32, Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
MaxResponseBytes: 1 << 20, Timeout: 3*time.Second, DialTimeout: time.Second,
TLSHandshakeTimeout: time.Second, ResponseHeaderTimeout: time.Second,
MaxConnsPerHost: 32, MaxIdleConnsPerHost: 16, MaxIdleConns: 64,
IdleConnTimeout: 60*time.Second, MaxResponseBytes: 1<<20,
}) })
if err != nil { return err } if err != nil { return err }
application.OnShutdown(client.Close) application.OnShutdown(client.Close)
err = client.JSON(ctx, "GET", serviceURL, nil, &result) err = client.JSON(ctx, "GET", serviceURL, nil, &result,
httpc.WithHeader("Authorization", "Bearer "+token),
)
``` ```
客户端应在服务启动时创建并复用。默认使用独立的 `net/http.Transport` 连接池,每个目标主机 启动时创建并复用 Client。代理可选环境、直连、固定地址;不配置时保留标准库原代理策略。JSON 支持单次请求头和响应限额,不保存 Client 全局 Token。新阶段超时与空闲连接参数可选,0 沿用默认。
有连接上限;该上限不是整个进程的全局连接数上限。也可以注入自有 `http.RoundTripper`,
其连接限制和生命周期由调用方负责。这样出站实现同样可以替换。
总超时覆盖建连、重试、等待和读取响应体。`Do` 返回后调用方必须读取/关闭 `response.Body`; 成功响应读取有界;output=nil 流式丢弃,不整体缓存。小错误体只在大小/时间预算内尝试读完以复用连接;未知或大错误体及时关闭。Do 保留标准流式 Request/Response,由调用者限量读取并关闭 body。Close 拒绝新调用并关闭自有池空闲连接,项目先排空在途工作。
`JSON` 自动关闭,并限制解码前读取的字节数。`Do` 允许流式读取,本身不限制响应体大小。
`Close` 拒绝新请求、关闭自有池的空闲连接;在途请求应先排空。
`MaxRetries` 默认 0,最多 5。封装只对可重放的 GET/HEAD/OPTIONS 重试网络错误及 自定义 Transport 借用、不修改、不关闭;不能同时指定池/代理/阶段选项。总超时持续覆盖 HTTP 等待、重试和读取 body,但不抢占 JSON 编解码或不遵守 context 的自定义代码。MaxRetries 默认 0,只对可重放 GET/HEAD/OPTIONS 重试;3xx 不跟随。标准库和借用 Transport 自身仍可能重放,不能据此承诺业务只执行一次。
429/502/503/504;POST/PUT/PATCH/DELETE 不做应用层自动重试。标准库 Transport 和借用的
RoundTripper 自身的重放规则仍生效,不能据此承诺“只执行一次”。退避带抖动,尊重
`Retry-After`;服务器要求等待超过 `MaxBackoff` 时直接返回该响应,不提前重试。
重定向交给业务处理,3xx 不会自动跟随。
默认从 context 传递 `X-Request-ID`,日志只包含方法、主机、状态、尝试次数和收到响应头的耗时, 完整参数、NO_PROXY、TLS、安全日志、重试、签名兼容说明见 [httpc 指南](httpc/README.md),可运行 [HTTP Client 示例](examples/httpClient/README.md)。
不记录完整 URL、查询参数、凭证和响应体。非 2xx 的 JSON 调用返回只带状态码的 `StatusError`。
## 请求保护:middleware ## 请求保护:middleware

View File

@@ -1,6 +1,6 @@
# 生成项目与文件维护边界 # 生成项目与文件维护边界
v0.19.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 v0.21.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
## 创建和日常开发 ## 创建和日常开发
@@ -38,7 +38,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
这是轻量源码分析,不运行项目 init、不执行 Go 命令、不下载依赖。项目自定义类型别名、嵌入方法或 build tags 等无法可靠判断时,会给 REVIEW 并交给 Go 编译器验证;不把这些情况强行判为签名错误。它不检查方法体是否引用已删除字段,也不证明业务逻辑正确。CI 中应同时执行 `go-omni api generate --check` 和 `go test ./...`,按项目实际构建参数测试。当前版本首次接手缺少快照的 manifest v2 时,会明确提示建立基线,不猜测旧契约;Logic 签名检查仍会执行。 这是轻量源码分析,不运行项目 init、不执行 Go 命令、不下载依赖。项目自定义类型别名、嵌入方法或 build tags 等无法可靠判断时,会给 REVIEW 并交给 Go 编译器验证;不把这些情况强行判为签名错误。它不检查方法体是否引用已删除字段,也不证明业务逻辑正确。CI 中应同时执行 `go-omni api generate --check` 和 `go test ./...`,按项目实际构建参数测试。当前版本首次接手缺少快照的 manifest v2 时,会明确提示建立基线,不猜测旧契约;Logic 签名检查仍会执行。
本版新增生成选项、公共字段组合和映射,详见 [生成能力指南](GENERATION.md)。 生成选项、公共字段组合和可选 Go 拷贝工具,详见 [生成能力指南](GENERATION.md)。
## 哪些文件由用户维护 ## 哪些文件由用户维护
@@ -46,7 +46,8 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
| --- | --- | | --- | --- |
| `api/*.api` | 路由、DTO、具名中间件、响应模板、请求/响应模式 | | `api/*.api` | 路由、DTO、具名中间件、响应模板、请求/响应模式 |
| `etc/config.yaml` | 运行参数;密钥通过环境变量提供 | | `etc/config.yaml` | 运行参数;密钥通过环境变量提供 |
| `internal/config/config.go` | 自定义配置字段、加载、默认值与校验 | | `internal/config/config.go` | 自定义配置字段、默认值与校验 |
| `internal/config/loader.go` | 需要额外文件格式时注册 Decoder |
| `internal/svc/serviceContext.go` | 依赖字段和初始化,持有加载后的 Config | | `internal/svc/serviceContext.go` | 依赖字段和初始化,持有加载后的 Config |
| `internal/svc/options.go` | AppOptions、GlobalMiddleware;注册 Go 错误目录和全局扩展 | | `internal/svc/options.go` | AppOptions、GlobalMiddleware;注册 Go 错误目录和全局扩展 |
| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 | | `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 |
@@ -62,7 +63,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。 以上骨架只补缺失文件,既有内容按字节保留。删除已废弃接口后,对应业务文件也不会被自动删除;由开发者整理编译器指出的失效类型或签名。
生成器持续管理:`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。 生成器持续管理:`internal/config/loadGen.go`、`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误,还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT;手改会被拒绝覆盖,需先移走自定义内容。
Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。 Logic 实例在注册时创建并由请求并发复用;请求数据用方法参数和局部变量传递,不存入共享 Logic/svc 字段。
@@ -114,6 +115,8 @@ c.RequestShutdown(err)
`Lifecycle *lifecycle.Controller` 是 svc 持有的通用控制器,初始化时创建,生成入口使用它的 Context。该 Context 表示服务正在停止,不代表数据库已经关闭。不能把它当作在途请求的 Context,也不要在它取消时立刻关闭请求仍依赖的资源。发号器失去租约后拒绝发号等业务安全规则仍由项目实现。 `Lifecycle *lifecycle.Controller` 是 svc 持有的通用控制器,初始化时创建,生成入口使用它的 Context。该 Context 表示服务正在停止,不代表数据库已经关闭。不能把它当作在途请求的 Context,也不要在它取消时立刻关闭请求仍依赖的资源。发号器失去租约后拒绝发号等业务安全规则仍由项目实现。
配置加载入口 `internal/config/loadGen.go` 也由生成器维护。业务扩展 Config/Default/Validate;只有新格式才需要修改 NewLoader。JSON/YAML/XML、错误定位和旧 Load 迁移见 [配置指南](CONFIG.md)。
## 配置内置接口 ## 配置内置接口
生成项目在 `etc/config.yaml` 明确列出以下默认配置;可以独立关闭、修改路径,或让健康接口返回原始 JSON: 生成项目在 `etc/config.yaml` 明确列出以下默认配置;可以独立关闭、修改路径,或让健康接口返回原始 JSON:
@@ -212,7 +215,7 @@ get /download (DownloadReq)
## 已有服务接入步骤 ## 已有服务接入步骤
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.19.0-preview: 每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.21.0-preview:
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。 1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。 2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。

View File

@@ -2,7 +2,7 @@
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。 GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
**从这份 README 开始接入。** 当前版本 **`v0.19.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。 **从这份 README 开始接入。** 当前版本 **`v0.21.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。 **v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。
@@ -37,23 +37,71 @@ Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) | | 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) | | 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) | | OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
| 结构化业务日志 | `logx.Info/Error`、`logx.From(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) | | 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
| 文件配置与环境覆盖 | 默认值→JSON/YAML→环境;完整业务结构用 `config.LoadProject[T]` / 环境别名 | [配置详解与例子](DEVELOPMENT.md)、[动态级别](ADVANCED.md#配置与日志热更新) | | 文件配置与环境覆盖 | 默认值→JSON/YAML/XML→环境→校验;完整业务结构用 `config.LoadProject[T]` / 环境别名 | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) | | 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) | | 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) | | 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) | | 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) | | 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
| 出站 HTTP、熔断 | `httpc.New` 复用连接池与截止时间;`resilience.New` 隔离故障依赖 | [HTTP 用法](MODULES.md)、[熔断/观测](ADVANCED.md#熔断与观测) | | 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时;JSON 支持单次请求头/限额;`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
| CORS、日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md)、[分布式实现](ADVANCED.md#分布式限流与幂等) | | 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) | | 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) | | 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) | | 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [观测用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) | | 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) | | 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) | | 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) | | 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
## 带认证的出站 HTTP
HTTP Client 在 svc 初始化一次,业务重复使用。普通内部请求可直接选择直连和连接参数,无需手写 Transport:
```go
client, err := httpc.New(httpc.Config{
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
Timeout: 3*time.Second, DialTimeout: time.Second,
TLSHandshakeTimeout: time.Second, ResponseHeaderTimeout: time.Second,
MaxConnsPerHost: 32, MaxIdleConnsPerHost: 16, MaxIdleConns: 64,
})
if err != nil { return err }
// 在 svc 生命周期登记 client.Close;关闭前先排空使用它的工作。
err = client.JSON(ctx, "GET", endpoint, nil, &result,
httpc.WithHeader("Authorization", "Bearer "+token),
httpc.WithMaxResponseBytes(64<<10),
)
```
请求头只作用于本次调用,不把用户 Token 留在共享 Client。默认代理/参数保持原行为;也可选择环境代理或固定代理。JSON 自动关闭并限制响应,Do 保留下载/流式读取扩展。完整配置、重试边界和已有自定义 Transport 的迁移见 [httpc](httpc/README.md)。
```sh
go run ./examples/httpClient
```
## 统一配置加载
新生成项目的加载流程由框架管理。业务在 `internal/config/config.go` 增加字段、默认值和 `Validate`,无需改 main 或复制环境变量解析。JSON、YAML、XML 共用字段和校验;扩展格式在 `loader.go` 注册。
```go
// 项目 Config 中增加字段,自动支持 GO_OMNI_BATCH_SIZE:
BatchSize int `json:"batchSize" envDefault:"100"`
// 项目 Validate 中增加业务范围检查:
if c.BatchSize < 1 {
return omniConfig.Invalid("batchSize", "must be positive")
}
```
错误包含文件、字段、可用的行列或环境变量名;输入原值不自动写入错误。
```sh
go run ./examples/configuration -format xml
DEMO_CLIENT_TIMEOUT=500ms go run ./examples/configuration
```
详见 [统一加载、解析器、迁移](CONFIG.md) 和 [完整示例](examples/configuration/README.md)。**旧项目须先迁移手写 Load,再重新生成**;生成器会保留业务配置并拒绝冲突,不静默覆盖。
## 选择接入方式 ## 选择接入方式
| 你的项目 | 从哪里开始 | | 你的项目 | 从哪里开始 |
@@ -74,7 +122,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
```sh ```sh
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk" export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.21.0-preview
export PATH="$(go env GOPATH)/bin:$PATH" export PATH="$(go env GOPATH)/bin:$PATH"
go-omni --help go-omni --help
``` ```
@@ -231,7 +279,7 @@ go-omni new --api catalog.api \
在业务项目的 **go.mod 所在目录**执行: 在业务项目的 **go.mod 所在目录**执行:
```sh ```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview go get git.yuntuo.hk/k2safe/go-omni@v0.21.0-preview
``` ```
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。 已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
@@ -533,7 +581,7 @@ go test ./...
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。 `main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
CLI 与业务 module 固定使用 `v0.19.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 CLI 与业务 module 固定使用 `v0.21.0-preview`。v0.14–v0.19 项目先按 [配置入口迁移](CONFIG.md#v019-及更早项目迁移) 整理旧 Load,再升级依赖与 CLI 并重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
## 按需深入 ## 按需深入

View File

@@ -1,4 +1,49 @@
# v0.19.0-preview # v0.21.0-preview
让带认证的 HTTP 调用和内部直连可以直接使用 httpc,减少手写 Transport 与 JSON 响应处理。
- Proxy 支持环境、直连、固定代理;默认保留既有策略。
- 建连、TCP keepalive、TLS 握手、响应头、100-continue、空闲超时和独立空闲连接数量可选配置;不修改全局 Transport,不关闭借用实例。
- JSON 支持 WithHeader / WithHeaders / WithMaxResponseBytes;认证信息逐请求隔离,自动关闭和有界响应处理保留。
- 成功路径减少请求复制和无用退避;output=nil 按限额流式丢弃;小错误体在大小和时间预算内尝试复用连接。
- 不新增依赖、不增加业务规则;Do 保留标准流式扩展。CLI 最新版本与模板最低版本分开,本轮模板兼容门槛仍是 v0.20。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.21.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.21.0-preview
```
**兼容注意**:JSON 旧的直接调用仍有效,自定义 interface/函数值需要同步新增的 variadic 参数。自定义 Transport 与显式连接/代理/阶段参数混用现在报错,避免参数被忽略;迁移时在自己的 Transport 配置,或改用内置字段。没有删除原调用入口。
[参数与使用说明](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[场景与性能验收](reports/httpcAcceptance.md) 已同步到 README。48 个测试包全量 race、Go 1.22 兼容、vet/build、约 110 万次请求头 fuzz 通过。性能来自本机同条件基准,不能代替项目实际流量验证;仍为 preview。
---
# v0.20.0-preview(历史版本)
统一配置加载,减少生成项目重复维护的启动代码。
- 生成器维护 `internal/config/loadGen.go`,调用 `config.LoadProject`;项目只扩展 Config/Default/Validate,可选在 loader.go/NewLoader 注册新格式。入口、环境覆盖、日志输出规范化无需业务重复实现。
- JSON/YAML/XML 使用同一个加载与绑定流程;Decoder 通过实例注册,无全局 init 或额外强制依赖。内置格式、字段、文件大小与层数均有明确限制。
- `config.Error` 携带文件、字段、环境变量、可用行列及安全原因;`Invalid` / `AtField` 支持业务验证。保留内部错误链,不自动输出原值;大整数、空集合、null、默认值隔离和自定义反序列化有回归覆盖。
- 配置解析在日志、依赖和监听初始化之前;生成/check/dry-run 使用一致入口和文件所有权规则。
- **迁移要求:旧项目先整理手写 Load,再重新生成。** 不静默覆盖业务文件。低层配置 API 保留;新项目不再复制手写加载流程。固定 Go 数组严格检查长度,YAML alias/merge/custom tag 不再接受,避免模糊覆盖和扩展行为。
```sh
go get git.yuntuo.hk/k2safe/go-omni@v0.20.0-preview
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.20.0-preview
# 先按 CONFIG.md 迁移旧 Load,再执行:
go-omni api generate
go-omni api generate --check
go test ./...
go build .
```
完整用法、格式、脱敏边界和迁移见 [统一配置指南](CONFIG.md),可运行例子见 [examples/configuration](examples/configuration/README.md)。回归、模糊测试与启动加载性能见 [验收记录](reports/configurationAcceptance.md)。47 个测试包的全量 race、Go 1.22 兼容、vet/build、30 秒 fuzz 与 5 个基准场景三轮测量通过。空缓存远程安装和 8 组生成消费者回归通过,无本地 replace;标签固定在 `c60841f901a3bcc3e9582eece6cfb2cad008fb9e`,见 [远程记录](reports/configurationRemote.json)。仍为 preview;项目的实际配置和业务依赖应自行验收。
---
# v0.19.0-preview(历史版本)
将可选结构转换改为直接 Go 调用,减少业务项目需要维护的配置。普通接口继续使用 `.api` 生成 DTO,不要求增加独立模型或任何拷贝规则。 将可选结构转换改为直接 Go 调用,减少业务项目需要维护的配置。普通接口继续使用 `.api` 生成 DTO,不要求增加独立模型或任何拷贝规则。
@@ -19,7 +64,7 @@ go test ./...
迁移步骤和语义边界见 [copyx](COPY.md#从旧-mapping-迁移)。新工具做运行时检查,不能声称具备旧生成器的编译期字段覆盖保证;实际类型对必须有测试。独立三端工资示例验证金额精度、缺失/null/零值、空集合、错误链和字段新增;不修改或代替接入项目的完整业务回归。 迁移步骤和语义边界见 [copyx](COPY.md#从旧-mapping-迁移)。新工具做运行时检查,不能声称具备旧生成器的编译期字段覆盖保证;实际类型对必须有测试。独立三端工资示例验证金额精度、缺失/null/零值、空集合、错误链和字段新增;不修改或代替接入项目的完整业务回归。
46 个测试包的全量 race(含实际生成项目)、vet/build、Go 1.22 兼容测试通过;30 秒 fuzz 无失败,14 种基准场景各测量三轮。验收条件、性能数据及本轮未覆盖的外部环境见 [v0.19 验收记录](reports/copyxAcceptance.md)。本版仍为 preview。 46 个测试包的全量 race(含实际生成项目)、vet/build、Go 1.22 兼容测试通过;30 秒 fuzz 无失败,14 种基准场景各测量三轮。验收条件、性能数据及本轮未覆盖的外部环境见 [v0.19 验收记录](reports/copyxAcceptance.md)。本版仍为 preview。 空缓存远程 CLI 与 7 组真实生成消费者验收通过,无本地 replace;标签固定在 `fd83235ce7fc15ce88a482d385d70e61f25d7b32`,详见 [远程记录](reports/copyxRemote.json)。
--- ---

View File

@@ -14,7 +14,7 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目
```sh ```sh
# 仓库读取权限与 PATH 配置见根目录 README # 仓库读取权限与 PATH 配置见根目录 README
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.21.0-preview
go-omni new --module example.com/hello ./hello go-omni new --module example.com/hello ./hello
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在 # 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
@@ -49,14 +49,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
## 版本与迁移 ## 版本与迁移
CLI 与框架依赖固定使用 v0.19.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换: CLI 与框架依赖固定使用 v0.20.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
```sh ```sh
# 在 go-omni 源码仓库中执行 # 在 go-omni 源码仓库中执行
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
``` ```
当前模板与依赖以 v0.19.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。 当前模板与依赖以 v0.20.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
## 数据库迁移命令 ## 数据库迁移命令
@@ -69,3 +69,7 @@ go-omni migrate --dir migrations --apply
``` ```
SQL 命名示例 `0001_createUsers.up.sql`;预览只显示版本和 SHA-256,不连接数据库。失败保留 dirty 状态,不自动 down/force。细节见 [扩展模块](../../ADVANCED.md)。 SQL 命名示例 `0001_createUsers.up.sql`;预览只显示版本和 SHA-256,不连接数据库。失败保留 dirty 状态,不自动 down/force。细节见 [扩展模块](../../ADVANCED.md)。
### v0.20 配置入口
新生成 `internal/config/loadGen.go` 调用统一加载,纳入生成检查;项目 config.go 只维护字段、默认值和校验,loader.go 用于注册额外格式。JSON/YAML/XML 内置。旧手写 Load 会阻止生成,请先按 [迁移步骤](../../CONFIG.md#v019-及更早项目迁移) 整理,不会自动覆盖业务代码。

View File

@@ -0,0 +1,113 @@
package apigen
import (
"context"
"encoding/json"
"os"
"os/exec"
"path/filepath"
"strings"
"testing"
"time"
)
func TestGeneratedConfigurationConsumer(t *testing.T) {
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
}
dir := filepath.Join(t.TempDir(), "app")
api := filepath.Join(t.TempDir(), "service.api")
writeFile(t, api, "service configuration {\n@handler ping\nget /ping\n}\n")
o := Options{API: api, Dir: dir, Module: "example.com/configuration", FrameworkDir: frameworkRoot(t)}
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
main := readFile(t, filepath.Join(dir, "main.go"))
// The business only changes fields/defaults/validation, not the loading flow.
writeFile(t, filepath.Join(dir, "internal/config/config.go"), `package config
import("git.yuntuo.hk/k2safe/go-omni/config";"git.yuntuo.hk/k2safe/go-omni/logx")
type Config struct { Server config.Config `+"`json:\"server\"`"+`;Limit int `+"`json:\"limit\" envDefault:\"10\"`"+` }
func Default()Config{ c:=config.Default();c.Log.File="./var/log/app.log";c.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:10};return Config{Server:c} }
func(c Config)Validate()error{if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)};if c.Limit<0{return config.Invalid("limit","must not be negative")};return nil}
`)
writeFile(t, filepath.Join(dir, "internal/config/loader.go"), `package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
func NewLoader()(*omniConfig.Loader,error){return omniConfig.NewLoader(omniConfig.WithDecoder("custom",omniConfig.JSONDecoder{}))}
`)
svc := filepath.Join(dir, "internal/svc/serviceContext.go")
writeFile(t, svc, strings.Replace(readFile(t, svc), "return nil", "panic(\"dependencies must not initialize on invalid config\")", 1))
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/configuration/consumer_test.go"))
runConsumerGo(t, dir, "mod", "tidy")
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
runConsumerGo(t, dir, "build", ".")
o.API = filepath.Join(dir, "api/service.api")
before := snapshot(t, dir)
for _, mode := range []string{"dry", "generate", "check"} {
o.DryRun = mode == "dry"
o.Check = mode == "check"
if err := generateManagedTestProject(t, o); err != nil {
t.Fatal(err)
}
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("business files overwritten")
}
if main != readFile(t, filepath.Join(dir, "main.go")) {
t.Fatal("main changed")
}
load := filepath.Join(dir, "internal/config/loadGen.go")
original := readFile(t, load)
writeFile(t, load, original+"\n// drift\n")
if err := generateManagedTestProject(t, o); err == nil {
t.Fatal("loadGen drift accepted")
}
writeFile(t, load, original)
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
data, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(data))
}
var dependency struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(data, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
}
func TestConfigLoadMigrationGuard(t *testing.T) {
dir := filepath.Join(t.TempDir(), "app")
o := Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}
if _, err := New(o); err != nil {
t.Fatal(err)
}
o.API = filepath.Join(dir, "api/service.api")
matches, err := filepath.Glob(filepath.Join(dir, "api/*.api"))
if err != nil || len(matches) != 1 {
t.Fatal(matches, err)
}
o.API = matches[0]
writeFile(t, filepath.Join(dir, "internal/config/legacy.go"), "package config\nfunc Load(path string)(Config,error){return Default(),nil}\n")
before := snapshot(t, dir)
for _, mode := range []string{"dry", "generate", "check"} {
o.DryRun = mode == "dry"
o.Check = mode == "check"
_, err := Generate(o)
if err == nil || !strings.Contains(err.Error(), "CONFIG.md") {
t.Fatal("missing migration diagnostic", err)
}
if !sameFiles(before, snapshot(t, dir)) {
t.Fatal("migration guard changed files")
}
}
}

View File

@@ -2,6 +2,7 @@ package apigen
import ( import (
"context" "context"
"encoding/json"
"fmt" "fmt"
"os" "os"
"os/exec" "os/exec"
@@ -92,4 +93,26 @@ func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResp
} }
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...") runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
runConsumerGo(t, dir, "build", ".") runConsumerGo(t, dir, "build", ".")
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
cmd.Dir = dir
cmd.Env = append(os.Environ(), "GOWORK=off")
data, err := cmd.CombinedOutput()
if err != nil {
t.Fatal(err, string(data))
}
var dependency struct {
Version, Sum string
Replace any
}
if err = json.Unmarshal(data, &dependency); err != nil {
t.Fatal(err)
}
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
t.Fatal("not a fixed remote consumer")
}
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
}
} }

View File

@@ -19,7 +19,10 @@ import (
) )
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
const minimumVersion = "v0.19.0-preview"
// Runtime-only releases need not raise the managed template compatibility floor.
const minimumVersion = "v0.20.0-preview"
const currentVersion = "v0.21.0-preview"
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n" const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
type Options struct { type Options struct {
@@ -106,6 +109,9 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
if err = checkRuntimeExtension(directory); err != nil { if err = checkRuntimeExtension(directory); err != nil {
return Output{}, err return Output{}, err
} }
if err = checkConfigLoadExtension(directory); err != nil {
return Output{}, err
}
for _, r := range c.Spec.Routes { for _, r := range c.Spec.Routes {
if !r.Manual { if !r.Manual {
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go") path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
@@ -152,7 +158,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
if !filepath.IsAbs(path) { if !filepath.IsAbs(path) {
path = filepath.Join(directory, path) path = filepath.Join(directory, path)
} }
if source, err := os.ReadFile(filepath.Join(path, "config", "env.go")); err == nil && strings.Contains(string(source), "func ServerEnvAliases(") { if source, err := os.ReadFile(filepath.Join(path, "config", "loader.go")); err == nil && strings.Contains(string(source), "func NewLoader(") {
localCurrent = true localCurrent = true
} }
} }
@@ -262,7 +268,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
return Output{}, errors.New("generation configuration changed during generation; retry") return Output{}, errors.New("generation configuration changed during generation; retry")
} }
out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld) out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld)
expectedVersion := minimumVersion expectedVersion := currentVersion
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) { if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) {
expectedVersion = info.Main.Version expectedVersion = info.Main.Version
} }
@@ -301,7 +307,7 @@ func moduleFile(o Options) ([]byte, error) {
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" { if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
version = info.Main.Version version = info.Main.Version
} else { } else {
version = minimumVersion version = currentVersion
} }
} }
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" { if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {

View File

@@ -58,6 +58,8 @@ Unimplemented/nil factories fail startup and clean registered dependencies.
Existing projects retain middleware: handlers until explicitly migrated. Existing projects retain middleware: handlers until explicitly migrated.
Ordinary Logic uses the generated Request/Response types directly. Where an Ordinary Logic uses the generated Request/Response types directly. Where an
independent business type is needed, use ordinary Go or the optional copyx package. independent business type is needed, use ordinary Go or the optional copyx package.
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use one pipeline. See the framework CONFIG.md.
There are no mappings/mappingFiles settings or generated mapping files in v0.19. There are no mappings/mappingFiles settings or generated mapping files in v0.19.
See COPY.md in the framework repository for copy policies and migration steps, See COPY.md in the framework repository for copy policies and migration steps,
and GENERATION.md for field composition and middleware factories. and GENERATION.md for field composition and middleware factories.

View File

@@ -0,0 +1,59 @@
package main
import (
"context"
"errors"
"example.com/configuration/internal/config"
omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
"os"
"path/filepath"
"strings"
"testing"
)
func TestGeneratedConfiguration(t *testing.T) {
for ext, source := range map[string]string{
"json": `{"limit":7}`, "yaml": "limit: 7\n", "xml": "<config><limit>7</limit></config>", "custom": `{"limit":7}`,
} {
t.Run(ext, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "app."+ext)
if err := os.WriteFile(path, []byte(source), 0600); err != nil {
t.Fatal(err)
}
cfg, err := config.Load(path)
if err != nil {
t.Fatal(err)
}
if cfg.Limit != 7 {
t.Fatal(cfg.Limit)
}
t.Setenv("GO_OMNI_LIMIT", "0")
cfg, err = config.Load(path)
if err != nil || cfg.Limit != 0 {
t.Fatal(cfg.Limit, err)
}
})
}
t.Setenv("GO_OMNI_LOG_OUTPUT", "stderr")
cfg, err := config.Load("")
if err != nil {
t.Fatal(err)
}
if cfg.Limit != 10 || cfg.Server.Log.File != "" || cfg.Server.Log.Rotation.Enabled {
t.Fatal("defaults/log normalization", cfg, err)
}
path := filepath.Join(t.TempDir(), "bad.yaml")
os.WriteFile(path, []byte("limit: -1\n"), 0600)
// Validation must run before svc.Initialize; the project initializer panics
// below if reached, so this also detects accidentally starting dependencies.
err = run(context.Background(), path)
var diagnostic *omniConfig.Error
if !errors.As(err, &diagnostic) || diagnostic.Field != "limit" || diagnostic.File != path || diagnostic.Line != 1 {
t.Fatal(err)
}
t.Setenv("GO_OMNI_LIMIT", "private-not-a-number")
_, err = config.Load(path)
if !errors.As(err, &diagnostic) || diagnostic.Env != "GO_OMNI_LIMIT" || strings.Contains(err.Error(), "private-not-a-number") {
t.Fatal(err)
}
}

View File

@@ -30,6 +30,39 @@ type UpgradePlan struct {
Module, Entry string Module, Entry string
} }
// Handwritten old loaders must be migrated explicitly, never silently replaced
// or left competing with the generated entry point.
func checkConfigLoadExtension(root string) error {
dir := filepath.Join(root, "internal/config")
entries, err := os.ReadDir(dir)
if errors.Is(err, os.ErrNotExist) {
return nil
}
if err != nil {
return err
}
for _, entry := range entries {
name := entry.Name()
if entry.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") || name == "loadGen.go" {
continue
}
rel := "internal/config/" + name
if err := safePath(root, rel); err != nil {
return err
}
f, err := goparser.ParseFile(gotoken.NewFileSet(), filepath.Join(root, rel), nil, 0)
if err != nil {
return fmt.Errorf("parse %s: %w", rel, err)
}
for _, decl := range f.Decls {
if fn, ok := decl.(*ast.FuncDecl); ok && fn.Recv == nil && fn.Name.Name == "Load" {
return fmt.Errorf("%s: migrate the project-owned Load function before generating internal/config/loadGen.go; retain Config/Default/Validate and move decoder options to NewLoader; see CONFIG.md", rel)
}
}
}
return nil
}
// PlanUpgrade compares infrastructure against the exact previous release seeds. // PlanUpgrade compares infrastructure against the exact previous release seeds.
// A custom file is never classified as safe to replace. Logic, middleware, // A custom file is never classified as safe to replace. Logic, middleware,
// options and repositories outside the changed templates are left untouched. // options and repositories outside the changed templates are left untouched.

View File

@@ -39,7 +39,7 @@ func oldInfrastructureProject(t *testing.T) string {
raw, _ := json.Marshal(m) raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw)) writeFile(t, filepath.Join(dir, manifestPath), string(raw))
path := filepath.Join(dir, "go.mod") path := filepath.Join(dir, "go.mod")
writeFile(t, path, strings.ReplaceAll(readFile(t, path), minimumVersion, "v0.13.0-preview")) writeFile(t, path, strings.ReplaceAll(readFile(t, path), currentVersion, "v0.13.0-preview"))
return dir return dir
} }
@@ -74,7 +74,7 @@ func TestUpgradeV014PreservesExtensions(t *testing.T) {
raw, _ := json.Marshal(m) raw, _ := json.Marshal(m)
writeFile(t, filepath.Join(dir, manifestPath), string(raw)) writeFile(t, filepath.Join(dir, manifestPath), string(raw))
mod := filepath.Join(dir, "go.mod") mod := filepath.Join(dir, "go.mod")
writeFile(t, mod, strings.ReplaceAll(readFile(t, mod), minimumVersion, "v0.14.1-preview")) writeFile(t, mod, strings.ReplaceAll(readFile(t, mod), currentVersion, "v0.14.1-preview"))
owned := map[string]string{} owned := map[string]string{}
for _, name := range []string{"internal/svc/serviceContext.go", "internal/config/config.go", "etc/config.yaml"} { for _, name := range []string{"internal/svc/serviceContext.go", "internal/config/config.go", "etc/config.yaml"} {
path := filepath.Join(dir, name) path := filepath.Join(dir, name)

View File

@@ -284,6 +284,9 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
} }
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) } func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
func ownedPath(name string) bool { func ownedPath(name string) bool {
if name == "internal/config/loadGen.go" {
return true
}
if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) { if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) {
return true return true
} }

View File

@@ -8,7 +8,7 @@ import (
) )
func TestNewWithRemoteFrameworkVersion(t *testing.T) { func TestNewWithRemoteFrameworkVersion(t *testing.T) {
for _, flag := range [][]string{{"--framework-version", "v0.19.0-preview"}, {"--framework-version=v0.19.0-preview"}} { for _, flag := range [][]string{{"--framework-version", "v0.21.0-preview"}, {"--framework-version=v0.21.0-preview"}} {
project := filepath.Join(t.TempDir(), "billing") project := filepath.Join(t.TempDir(), "billing")
args := append([]string{"--module", "example.com/billing"}, flag...) args := append([]string{"--module", "example.com/billing"}, flag...)
if err := runNew(append(args, project)); err != nil { if err := runNew(append(args, project)); err != nil {
@@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
if !strings.Contains(string(content), "go-omni v0.19.0-preview") || strings.Contains(string(content), "replace ") { if !strings.Contains(string(content), "go-omni v0.21.0-preview") || strings.Contains(string(content), "replace ") {
t.Fatalf("CLI did not pin remote dependency: %s", content) t.Fatalf("CLI did not pin remote dependency: %s", content)
} }
} }

View File

@@ -21,6 +21,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
"internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl", "internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl",
"internal/svc/options.go": "templates/api-options.go.tmpl", "internal/svc/options.go": "templates/api-options.go.tmpl",
"internal/config/config.go": "templates/api-config.go.tmpl", "internal/config/config.go": "templates/api-config.go.tmpl",
"internal/config/loadGen.go": "templates/api-load.go.tmpl",
"internal/config/loader.go": "templates/api-loader.go.tmpl",
"etc/config.yaml": "templates/modules-config.yaml.tmpl", "etc/config.yaml": "templates/modules-config.yaml.tmpl",
"internal/svc/serviceContext.go": "templates/api-context.go.tmpl", "internal/svc/serviceContext.go": "templates/api-context.go.tmpl",
} }
@@ -31,6 +33,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl" files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl"
} }
if legacy { if legacy {
delete(files, "internal/config/loadGen.go")
delete(files, "internal/config/loader.go")
delete(files, "internal/svc/runtimeGen.go") delete(files, "internal/svc/runtimeGen.go")
delete(files, "internal/svc/databaseOptions.go") delete(files, "internal/svc/databaseOptions.go")
files["internal/svc/close.go"] = "templates/api-close.go.tmpl" files["internal/svc/close.go"] = "templates/api-close.go.tmpl"

View File

@@ -1,10 +1,9 @@
package settings package settings
import ( import (
"os"
{{if and (not .API) (ne .Storage "none")}}"strings"{{end}} {{if and (not .API) (ne .Storage "none")}}"strings"{{end}}
"time" "time"
"git.yuntuo.hk/k2safe/go-omni/config" "git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx" {{if eq .LogOutput "file"}}"git.yuntuo.hk/k2safe/go-omni/logx"{{end}}
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}} {{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
{{if ne .Storage "none"}}"errors" {{if ne .Storage "none"}}"errors"
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}} {{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
@@ -35,22 +34,10 @@ func Default() Config {
}{{end}}} }{{end}}}
} }
// Load applies defaults -> file -> typed environment -> validation.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string)(Config,error){
cfg:=Default();var err error
cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err}
if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}}
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_",Empty:"ignore",Aliases:config.ServerEnvAliases("GO_OMNI_","server")});if err!=nil{return Config{},err}
output,outputSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_OUTPUT");if !outputSet {output=os.Getenv("GO_OMNI_LOG_OUTPUT")}
file,fileSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_FILE");if !fileSet {file=os.Getenv("GO_OMNI_LOG_FILE")}
if output!="" && file=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
return cfg,cfg.Validate()
}
func(c Config)Validate()error{ func(c Config)Validate()error{
if err:=c.Server.Validate();err!=nil{return err} if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)}
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return err}{{end}} {{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return config.AtField("database",err)}{{end}}
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return err}{{end}} {{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return config.AtField("storage",err)}{{end}}
return nil return nil
} }
{{if ne .Storage "none"}} {{if ne .Storage "none"}}

View File

@@ -0,0 +1,13 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// Load is maintained by the generator. Extend Config/Default/Validate in
// config.go, or NewLoader in loader.go; do not duplicate the loading pipeline.
func Load(path string) (Config, error) {
loader,err:=NewLoader()
if err!=nil{return Config{},err}
return omniConfig.LoadProject(path,Default(),omniConfig.EnvOptions{
Prefix:"GO_OMNI_",Empty:"ignore",Aliases:omniConfig.ServerEnvAliases("GO_OMNI_","server"),
},loader)
}

View File

@@ -0,0 +1,7 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// NewLoader is project-owned. JSON/YAML/XML work without changes. Register
// additional format decoders here only when the project needs them.
func NewLoader() (*omniConfig.Loader,error) { return omniConfig.NewLoader() }

View File

@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
// Source builds can carry an unpublished VCS version (including +dirty). // Source builds can carry an unpublished VCS version (including +dirty).
// Keep this fallback in sync with the next release; installed CLIs use their // Keep this fallback in sync with the next release; installed CLIs use their
// downloaded module version, which also has a module checksum. // downloaded module version, which also has a module checksum.
const defaultFrameworkVersion = "v0.19.0-preview" const defaultFrameworkVersion = "v0.21.0-preview"
// This target is not a claim that the release already exists. Source users // This target is not a claim that the release already exists. Source users
// select a local framework checkout until the module templates are published. // select a local framework checkout until the module templates are published.

320
config/bind.go Normal file
View File

@@ -0,0 +1,320 @@
package config
import (
"encoding/json"
"fmt"
"reflect"
"sort"
"strings"
)
var jsonUnmarshalerType = reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
type documentBinder struct {
doc Document
nodes int
}
func (b *documentBinder) step(path string, depth int) error {
b.nodes++
if depth > maxConfigDepth || b.nodes > maxConfigNodes {
return b.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
}
return nil
}
func bindDocument[T any](doc Document, defaults T) (T, error) {
var zero T
base := reflect.ValueOf(&defaults).Elem()
if base.Kind() != reflect.Struct && (base.Kind() != reflect.Pointer || base.Type().Elem().Kind() != reflect.Struct) {
return zero, &Error{Reason: "project configuration must be a struct"}
}
result, err := cloneConfiguration(base, 0)
if err != nil {
return zero, &Error{Reason: "cannot clone configuration defaults", cause: err}
}
b := documentBinder{doc: doc}
if err = b.bind(doc.Value, result, "", 0, false); err != nil {
return zero, err
}
return result.Interface().(T), nil
}
func (b *documentBinder) decode(v any, dst reflect.Value, path string) (err error) {
defer func() {
if recover() != nil {
err = b.doc.diagnostic(path, "custom value decoder panicked", nil)
}
}()
raw, err := json.Marshal(v)
if err != nil {
return b.doc.diagnostic(path, "unsupported configuration value", err)
}
if err = json.Unmarshal(raw, dst.Addr().Interface()); err != nil {
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
}
return nil
}
func (b *documentBinder) bind(v any, dst reflect.Value, path string, depth int, quoted bool) error {
if err := b.step(path, depth); err != nil {
return err
}
if v == nil {
return b.decode(nil, dst, path)
} // Preserve encoding/json null semantics.
if dst.Kind() == reflect.Pointer {
if dst.IsNil() {
dst.Set(reflect.New(dst.Type().Elem()))
}
return b.bind(v, dst.Elem(), path, depth+1, quoted)
}
if dst.Type() == durationType {
var raw string
switch x := v.(type) {
case string:
raw = x
case Text:
raw = string(x)
default:
return b.decode(v, dst, path)
}
if err := parseEnvValue(dst, strings.TrimSpace(raw)); err != nil {
return b.doc.diagnostic(path, "expected a duration such as 5s", err)
}
return nil
}
if dst.CanAddr() && dst.Addr().Type().Implements(jsonUnmarshalerType) {
normalized, err := b.dynamic(v, path, depth+1)
if err != nil {
return err
}
return b.decode(normalized, dst, path)
}
if dst.CanAddr() && dst.Addr().Type().Implements(textType) {
if text, ok := v.(Text); ok {
v = string(text)
}
return b.decode(v, dst, path)
}
if text, ok := v.(Text); ok {
if dst.Kind() == reflect.String || dst.Kind() == reflect.Interface || (dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8) {
return b.decode(string(text), dst, path)
}
switch dst.Kind() {
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64:
if err := parseEnvValue(dst, strings.TrimSpace(string(text))); err != nil {
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
}
return nil
default:
return b.doc.diagnostic(path, "expected an object or array, not scalar text", nil)
}
}
if quoted {
s, ok := v.(string)
if !ok {
return b.doc.diagnostic(path, "expected a JSON-encoded string", nil)
}
if err := json.Unmarshal([]byte(s), dst.Addr().Interface()); err != nil {
return b.doc.diagnostic(path, "invalid string-encoded value", err)
}
return nil
}
switch dst.Kind() {
case reflect.Struct:
m, ok := v.(map[string]any)
if !ok {
return b.doc.diagnostic(path, "expected an object", nil)
}
fields := configurationFields(dst.Type())
used := map[string]bool{}
for _, name := range sortedKeys(m) {
child := joinField(path, name)
f, ok := fields[name]
canonical := name
if !ok {
for n, field := range fields {
if strings.EqualFold(n, name) {
if ok {
return b.doc.diagnostic(child, "ambiguous field name", nil)
}
f, ok, canonical = field, true, n
}
}
}
if !ok {
return b.doc.diagnostic(child, "unknown field", nil)
}
if used[canonical] {
return b.doc.diagnostic(child, "duplicate field binding", nil)
}
used[canonical] = true
target := dst
for _, index := range f.Index {
if target.Kind() == reflect.Pointer {
if target.IsNil() {
if !target.CanSet() {
return b.doc.diagnostic(child, "field cannot be assigned", nil)
}
target.Set(reflect.New(target.Type().Elem()))
}
target = target.Elem()
}
target = target.Field(index)
}
if !target.CanSet() {
return b.doc.diagnostic(child, "field cannot be assigned", nil)
}
if err := b.bind(m[name], target, child, depth+1, strings.Contains(","+f.Tag.Get("json")+",", ",string,")); err != nil {
return err
}
}
return nil
case reflect.Map:
m, ok := v.(map[string]any)
if !ok || dst.Type().Key().Kind() != reflect.String {
return b.doc.diagnostic(path, "expected an object with string keys", nil)
}
if dst.IsNil() {
dst.Set(reflect.MakeMap(dst.Type()))
}
for _, key := range sortedKeys(m) {
value := reflect.New(dst.Type().Elem()).Elem()
if err := b.bind(m[key], value, joinField(path, key), depth+1, false); err != nil {
return err
}
k := reflect.New(dst.Type().Key()).Elem()
k.SetString(key)
dst.SetMapIndex(k, value)
}
return nil
case reflect.Slice, reflect.Array:
if _, ok := v.(string); ok && dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8 {
return b.decode(v, dst, path)
}
items, ok := v.([]any)
if !ok {
return b.doc.diagnostic(path, "expected an array", nil)
}
if dst.Kind() == reflect.Array && len(items) != dst.Len() {
return b.doc.diagnostic(path, "array length does not match its declared type", nil)
}
if dst.Kind() == reflect.Slice {
dst.Set(reflect.MakeSlice(dst.Type(), len(items), len(items)))
}
for i, value := range items {
if err := b.bind(value, dst.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1, false); err != nil {
return err
}
}
return nil
case reflect.Interface:
v, err := b.dynamic(v, path, depth+1)
if err != nil {
return err
}
if dst.Type().NumMethod() == 0 {
dst.Set(reflect.ValueOf(v))
return nil
}
return b.decode(v, dst, path)
default:
return b.decode(v, dst, path)
}
}
func sortedKeys(m map[string]any) []string {
keys := make([]string, 0, len(m))
for k := range m {
keys = append(keys, k)
}
sort.Strings(keys)
return keys
}
func (b *documentBinder) dynamic(v any, path string, depth int) (any, error) {
if err := b.step(path, depth); err != nil {
return nil, err
}
switch x := v.(type) {
case Text:
return string(x), nil
case map[string]any:
out := make(map[string]any, len(x))
for _, k := range sortedKeys(x) {
c, err := b.dynamic(x[k], joinField(path, k), depth+1)
if err != nil {
return nil, err
}
out[k] = c
}
return out, nil
case []any:
out := make([]any, len(x))
for i, v := range x {
c, err := b.dynamic(v, fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return nil, err
}
out[i] = c
}
return out, nil
default:
return v, nil
}
}
// Match encoding/json's field promotion: the shallowest declaration wins;
// explicit tags win ties, and ambiguous fields remain unavailable.
func configurationFields(t reflect.Type) map[string]reflect.StructField {
type candidate struct {
field reflect.StructField
tagged bool
}
candidates := map[string][]candidate{}
var visit func(reflect.Type, []int, map[reflect.Type]bool)
visit = func(current reflect.Type, prefix []int, visiting map[reflect.Type]bool) {
if visiting[current] {
return
}
visiting[current] = true
defer delete(visiting, current)
for i := 0; i < current.NumField(); i++ {
f := current.Field(i)
name := strings.Split(f.Tag.Get("json"), ",")[0]
if name == "-" {
continue
}
child := f.Type
if child.Kind() == reflect.Pointer {
child = child.Elem()
}
if f.PkgPath != "" && (!f.Anonymous || child.Kind() != reflect.Struct) {
continue
}
f.Index = append(append([]int(nil), prefix...), i)
if name == "" && f.Anonymous && child.Kind() == reflect.Struct {
visit(child, f.Index, visiting)
continue
}
tagged := name != ""
if name == "" {
name = f.Name
}
candidates[name] = append(candidates[name], candidate{f, tagged})
}
}
visit(t, nil, map[reflect.Type]bool{})
fields := map[string]reflect.StructField{}
for name, list := range candidates {
sort.Slice(list, func(i, j int) bool {
if len(list[i].field.Index) != len(list[j].field.Index) {
return len(list[i].field.Index) < len(list[j].field.Index)
}
return list[i].tagged && !list[j].tagged
})
if len(list) > 1 && len(list[0].field.Index) == len(list[1].field.Index) && list[0].tagged == list[1].tagged {
continue
}
fields[name] = list[0].field
}
return fields
}

View File

@@ -62,19 +62,19 @@ func Default() Config {
// Validate checks configuration values before an application starts. // Validate checks configuration values before an application starts.
func (c Config) Validate() error { func (c Config) Validate() error {
if err := c.Lifecycle.Validate(); err != nil { if err := c.Lifecycle.Validate(); err != nil {
return err return AtField("lifecycle", err)
} }
if err := c.Request.Validate(); err != nil { if err := c.Request.Validate(); err != nil {
return err return AtField("request", err)
} }
if err := c.Log.Validate(); err != nil { if err := c.Log.Validate(); err != nil {
return err return AtField("log", err)
} }
if err := c.Endpoints.Validate(); err != nil { if err := c.Endpoints.Validate(); err != nil {
return err return err
} }
if err := c.Errors.Validate(); err != nil { if err := c.Errors.Validate(); err != nil {
return err return AtField("errors", err)
} }
encoder := c.ResponseEncoder encoder := c.ResponseEncoder
if encoder == nil { if encoder == nil {
@@ -90,7 +90,7 @@ func (c Config) Validate() error {
return err return err
} }
if err := c.Response.Validate(); err != nil { if err := c.Response.Validate(); err != nil {
return err return AtField("response", err)
} }
if c.SlowRequestThreshold < 0 { if c.SlowRequestThreshold < 0 {
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"} return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
@@ -99,13 +99,13 @@ func (c Config) Validate() error {
return &ConfigError{Field: "address", Message: "is required"} return &ConfigError{Field: "address", Message: "is required"}
} }
if strings.TrimSpace(c.RequestIDHeader) == "" { if strings.TrimSpace(c.RequestIDHeader) == "" {
return &ConfigError{Field: "request_id_header", Message: "is required"} return &ConfigError{Field: "requestIdHeader", Message: "is required"}
} }
if c.ShutdownTimeout <= 0 { if c.ShutdownTimeout <= 0 {
return &ConfigError{Field: "shutdown_timeout", Message: "must be positive"} return &ConfigError{Field: "shutdownTimeout", Message: "must be positive"}
} }
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil { if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
return &ConfigError{Field: "trusted_proxies", Message: err.Error()} return &Error{Field: "trustedProxies", Reason: "expected IP addresses or CIDRs", cause: err}
} }
return c.ValidateTransport() return c.ValidateTransport()
} }

View File

@@ -112,7 +112,7 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s") t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s")
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com") t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com")
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trusted_proxies") { if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trustedProxies") {
t.Fatalf("expected invalid trusted proxy error, got %v", err) t.Fatalf("expected invalid trusted proxy error, got %v", err)
} }
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "") t.Setenv("GO_OMNI_TRUSTED_PROXIES", "")
@@ -123,7 +123,7 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
} }
cfg = Default() cfg = Default()
cfg.RequestIDHeader = " " cfg.RequestIDHeader = " "
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "request_id_header") { if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "requestIdHeader") {
t.Fatalf("expected invalid request header error, got %v", err) t.Fatalf("expected invalid request header error, got %v", err)
} }
} }

398
config/decoders.go Normal file
View File

@@ -0,0 +1,398 @@
package config
import (
"bytes"
"encoding/json"
"encoding/xml"
"errors"
"fmt"
"io"
"regexp"
"sort"
"strconv"
"strings"
"unicode/utf8"
"gopkg.in/yaml.v3"
)
type JSONDecoder struct{}
type YAMLDecoder struct{}
type XMLDecoder struct{}
type parseState struct {
doc Document
nodes int
}
func newParseState() *parseState { return &parseState{doc: Document{Positions: map[string]Position{}}} }
func (p *parseState) enter(path string, pos Position, depth int) error {
p.nodes++
p.doc.Positions[path] = pos
if depth > maxConfigDepth || p.nodes > maxConfigNodes {
return p.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
}
return nil
}
type sourcePositions struct{ starts []int }
func newSourcePositions(data []byte) sourcePositions {
p := sourcePositions{starts: []int{0}}
for i, c := range data {
if c == '\n' {
p.starts = append(p.starts, i+1)
}
}
return p
}
func (p sourcePositions) at(offset int64) Position {
if offset < 0 {
offset = 0
}
i := sort.Search(len(p.starts), func(i int) bool { return int64(p.starts[i]) > offset }) - 1
return Position{Line: i + 1, Column: int(offset) - p.starts[i] + 1}
}
var yamlLine = regexp.MustCompile(`line ([0-9]+)`)
func yamlSyntax(err error) *Error {
d := &Error{Reason: "invalid YAML syntax", cause: err}
if m := yamlLine.FindStringSubmatch(err.Error()); len(m) == 2 {
d.Line, _ = strconv.Atoi(m[1])
d.Column = 1
}
return d
}
func (JSONDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
positions := newSourcePositions(data)
d := json.NewDecoder(bytes.NewReader(data))
d.UseNumber()
var parse func(string, int) (any, error)
parse = func(path string, depth int) (any, error) {
offset := d.InputOffset()
for offset < int64(len(data)) && strings.ContainsRune(" \t\r\n:,", rune(data[offset])) {
offset++
}
if err := p.enter(path, positions.at(offset), depth); err != nil {
return nil, err
}
token, err := d.Token()
if err != nil {
pos := positions.at(d.InputOffset())
var syntax *json.SyntaxError
if errors.As(err, &syntax) {
pos = positions.at(syntax.Offset - 1)
}
return nil, &Error{Field: path, Line: pos.Line, Column: pos.Column, Reason: "invalid JSON syntax", cause: err}
}
if delim, ok := token.(json.Delim); ok {
switch delim {
case '{':
out := map[string]any{}
for d.More() {
k, e := d.Token()
if e != nil {
return nil, p.doc.diagnostic(path, "invalid object key", e)
}
key, ok := k.(string)
if !ok {
return nil, p.doc.diagnostic(path, "object key must be a string", nil)
}
child := joinField(path, key)
if _, exists := out[key]; exists {
return nil, p.doc.diagnostic(child, "duplicate field", nil)
}
out[key], e = parse(child, depth+1)
if e != nil {
return nil, e
}
}
if _, err = d.Token(); err != nil {
return nil, p.doc.diagnostic(path, "unterminated object", err)
}
return out, nil
case '[':
out := []any{}
for d.More() {
v, e := parse(fmt.Sprintf("%s[%d]", path, len(out)), depth+1)
if e != nil {
return nil, e
}
out = append(out, v)
}
if _, err = d.Token(); err != nil {
return nil, p.doc.diagnostic(path, "unterminated array", err)
}
return out, nil
default:
return nil, p.doc.diagnostic(path, "unexpected JSON delimiter", nil)
}
}
return token, nil
}
var err error
p.doc.Value, err = parse("", 0)
if err == nil {
if _, e := d.Token(); e != io.EOF {
err = p.doc.diagnostic("", "exactly one JSON document required", e)
}
}
return p.doc, err
}
func (YAMLDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
d := yaml.NewDecoder(bytes.NewReader(data))
var root yaml.Node
if err := d.Decode(&root); err != nil {
return p.doc, yamlSyntax(err)
}
var extra yaml.Node
if err := d.Decode(&extra); err != io.EOF {
return p.doc, &Error{Reason: "exactly one YAML document required", cause: err}
}
var walk func(*yaml.Node, string, int) (any, error)
walk = func(n *yaml.Node, path string, depth int) (any, error) {
if err := p.enter(path, Position{n.Line, n.Column}, depth); err != nil {
return nil, err
}
switch n.Kind {
case yaml.MappingNode:
out := map[string]any{}
for i := 0; i < len(n.Content); i += 2 {
k, v := n.Content[i], n.Content[i+1]
if k.Tag != "!!str" {
return nil, p.doc.diagnostic(path, "mapping keys must be strings; merge keys are not supported", nil)
}
child := joinField(path, k.Value)
if _, ok := out[k.Value]; ok {
return nil, &Error{Field: child, Line: k.Line, Column: k.Column, Reason: "duplicate field"}
}
x, err := walk(v, child, depth+1)
if err != nil {
return nil, err
}
out[k.Value] = x
}
return out, nil
case yaml.SequenceNode:
out := make([]any, len(n.Content))
for i, c := range n.Content {
var err error
out[i], err = walk(c, fmt.Sprintf("%s[%d]", path, i), depth+1)
if err != nil {
return nil, err
}
}
return out, nil
case yaml.AliasNode:
return nil, p.doc.diagnostic(path, "YAML aliases are not supported", nil)
case yaml.ScalarNode:
switch n.Tag {
case "!!str", "!!timestamp":
return n.Value, nil
case "!!null":
return nil, nil
case "!!bool", "!!int", "!!float":
var v any
if err := n.Decode(&v); err != nil {
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
}
raw, err := json.Marshal(v)
if err != nil {
return nil, p.doc.diagnostic(path, "number must be finite", err)
}
var out any
dec := json.NewDecoder(bytes.NewReader(raw))
dec.UseNumber()
if err = dec.Decode(&out); err != nil {
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
}
return out, nil
default:
return nil, p.doc.diagnostic(path, "unsupported YAML tag", nil)
}
}
return nil, p.doc.diagnostic(path, "unsupported YAML node", nil)
}
if len(root.Content) != 1 {
return p.doc, &Error{Reason: "one nonempty YAML document required"}
}
var err error
p.doc.Value, err = walk(root.Content[0], "", 0)
return p.doc, err
}
// XML uses a config root, element field names, type="array" with item children,
// type="object" for empty objects, and type="null" for explicit null. Scalar
// text is bound using the destination type; optional number/bool/string types
// let dynamic any values preserve their intended JSON type.
func (XMLDecoder) Decode(data []byte) (Document, error) {
p := newParseState()
if len(data) > maxConfigBytes {
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
}
if !utf8.Valid(data) {
return p.doc, &Error{Reason: "configuration must be UTF-8"}
}
d := xml.NewDecoder(bytes.NewReader(data))
var parse func(xml.StartElement, string, int, Position) (any, error)
parse = func(start xml.StartElement, path string, depth int, pos Position) (any, error) {
if err := p.enter(path, pos, depth); err != nil {
return nil, err
}
if start.Name.Space != "" {
return nil, p.doc.diagnostic(path, "XML namespaces are not supported", nil)
}
kind := ""
attrSeen := false
for _, a := range start.Attr {
if a.Name.Space != "" || a.Name.Local != "type" || attrSeen {
return nil, p.doc.diagnostic(path, "only one XML type attribute is supported", nil)
}
kind = a.Value
attrSeen = true
}
if depth == 0 && kind == "" {
kind = "object"
}
switch kind {
case "", "object", "array", "null", "string", "number", "bool":
default:
return nil, p.doc.diagnostic(path, "unsupported XML type", nil)
}
children := map[string]any{}
array := []any{}
var text strings.Builder
for {
line, column := d.InputPos()
token, err := d.Token()
if err != nil {
return nil, &Error{Field: path, Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
}
switch x := token.(type) {
case xml.StartElement:
if kind != "" && kind != "object" && kind != "array" {
return nil, p.doc.diagnostic(path, "scalar XML element cannot contain children", nil)
}
child := joinField(path, x.Name.Local)
if kind == "array" {
if x.Name.Local != "item" {
return nil, p.doc.diagnostic(path, "XML array children must be item elements", nil)
}
child = fmt.Sprintf("%s[%d]", path, len(array))
} else if _, ok := children[x.Name.Local]; ok {
return nil, p.doc.diagnostic(child, "duplicate field; use type=\"array\" for lists", nil)
}
v, e := parse(x, child, depth+1, Position{line, column})
if e != nil {
return nil, e
}
if kind == "array" {
array = append(array, v)
} else {
children[x.Name.Local] = v
}
case xml.CharData:
text.Write([]byte(x))
case xml.Comment:
case xml.EndElement:
raw := text.String()
if len(children) > 0 || kind == "array" || kind == "object" {
if strings.TrimSpace(raw) != "" {
return nil, p.doc.diagnostic(path, "mixed XML text and elements are not supported", nil)
}
if kind == "array" {
return array, nil
}
return children, nil
}
switch kind {
case "null":
if strings.TrimSpace(raw) != "" {
return nil, p.doc.diagnostic(path, "null element must be empty", nil)
}
return nil, nil
case "string":
return raw, nil
case "number":
var n any
dec := json.NewDecoder(strings.NewReader(strings.TrimSpace(raw)))
dec.UseNumber()
if err := dec.Decode(&n); err != nil {
return nil, p.doc.diagnostic(path, "invalid number", err)
}
if _, ok := n.(json.Number); !ok {
return nil, p.doc.diagnostic(path, "expected number", nil)
}
if _, e := dec.Token(); e != io.EOF {
return nil, p.doc.diagnostic(path, "invalid number", e)
}
return n, nil
case "bool":
b, e := strconv.ParseBool(strings.TrimSpace(raw))
if e != nil {
return nil, p.doc.diagnostic(path, "expected boolean", e)
}
return b, nil
}
return Text(raw), nil
default:
return nil, p.doc.diagnostic(path, "XML directives and processing instructions are not supported", nil)
}
}
}
found := false
prolog := false
for {
line, column := d.InputPos()
token, err := d.Token()
if err == io.EOF {
break
}
if err != nil {
return p.doc, &Error{Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
}
switch x := token.(type) {
case xml.StartElement:
if found || x.Name.Local != "config" {
return p.doc, &Error{Line: line, Column: column, Reason: "exactly one config root required"}
}
found = true
p.doc.Value, err = parse(x, "", 0, Position{line, column})
if err != nil {
return p.doc, err
}
case xml.CharData:
if strings.TrimSpace(string(x)) != "" {
return p.doc, &Error{Reason: "unexpected text outside XML root"}
}
case xml.Comment:
case xml.ProcInst:
if found || prolog || x.Target != "xml" {
return p.doc, &Error{Reason: "XML processing instructions are not supported"}
}
prolog = true
default:
return p.doc, &Error{Reason: "XML directives are not supported"}
}
}
if !found {
return p.doc, &Error{Reason: "config root required"}
}
return p.doc, nil
}

94
config/diagnostic.go Normal file
View File

@@ -0,0 +1,94 @@
package config
import (
"errors"
"fmt"
"strings"
)
// Error is a value-free diagnostic. Cause may contain private parser details;
// errors.Is/As remain available, but Error never formats Cause automatically.
type Error struct {
File, Field, Env, Reason string
Line, Column int
cause error
}
func (e *Error) Error() string {
where := "config"
if e.File != "" {
where = fmt.Sprintf("config %q", e.File)
}
if e.Line > 0 {
where += fmt.Sprintf(":%d:%d", e.Line, e.Column)
}
if e.Env != "" {
where += fmt.Sprintf(" env %q", e.Env)
}
if e.Field != "" {
where += fmt.Sprintf(" field %q", e.Field)
}
return where + ": " + e.Reason
}
func (e *Error) Unwrap() error { return e.cause }
// DecodeError is retained for callers of errors.As from earlier releases.
type DecodeError = Error
// Invalid reports a public, value-free validation reason for a configuration field.
func Invalid(field, reason string) error { return &ConfigError{Field: field, Message: reason} }
// AtField prefixes a nested validation error without losing its original cause.
// Unstructured validation errors are deliberately not echoed; use Invalid when
// a custom validator needs a public reason rather than a private error message.
func AtField(field string, err error) error {
if err == nil {
return nil
}
d := validationError(err)
d.Field = joinField(field, d.Field)
return d
}
func validationError(err error) *Error {
var d *Error
if errors.As(err, &d) {
out := *d
out.cause = err
return &out
}
var c *ConfigError
if errors.As(err, &c) {
return &Error{Field: c.Field, Reason: c.Message, cause: err}
}
return &Error{Reason: "validation failed", cause: err}
}
func joinField(parent, child string) string {
if parent == "" {
return child
}
if child == "" {
return parent
}
if strings.HasPrefix(child, "[") {
return parent + child
}
return parent + "." + child
}
// Position uses one-based source line/column numbers. Zero means unavailable.
type Position struct{ Line, Column int }
// Document is the common decoder result. Values use JSON-compatible data:
// string, bool, json.Number, nil, []any, map[string]any. Text is available for
// formats such as XML whose scalar tokens have no inherent JSON type.
type Document struct {
Value any
Positions map[string]Position
}
type Text string
func (d Document) diagnostic(field, reason string, cause error) *Error {
p := d.Positions[field]
return &Error{Field: field, Reason: reason, Line: p.Line, Column: p.Column, cause: cause}
}

View File

@@ -1,100 +1,16 @@
package config package config
import ( import (
"bytes"
"context" "context"
"encoding/json"
"errors" "errors"
"io"
"os"
"path/filepath"
"reflect" "reflect"
"strings" "strings"
"sync" "sync"
"time" "time"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/logx" "git.yuntuo.hk/k2safe/go-omni/logx"
"gopkg.in/yaml.v3"
) )
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
// field names for both formats, rejects unknown fields and accepts durations
// such as "3s". Application-specific validation remains with the caller.
func DecodeFile[T any](path string, defaults T) (T, error) {
result, err := decodeFile(path, defaults)
if err != nil {
var zero T
return zero, &DecodeError{cause: err}
}
return result, nil
}
// DecodeError keeps input values out of public diagnostics while preserving
// errors.Is/As for explicitly handled underlying I/O and syntax errors.
type DecodeError struct{ cause error }
func (e *DecodeError) Error() string {
return "config: cannot decode file; check its path, format, field names and value types"
}
func (e *DecodeError) Unwrap() error { return e.cause }
func decodeFile[T any](path string, defaults T) (T, error) {
var zero T
f, err := os.Open(path)
if err != nil {
return zero, err
}
defer f.Close()
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
if err != nil {
return zero, err
}
if len(data) > 1<<20 {
return zero, errors.New("config: file exceeds 1 MiB")
}
var value any
switch strings.ToLower(filepath.Ext(path)) {
case ".yaml", ".yml":
dec := yaml.NewDecoder(bytes.NewReader(data))
if err := dec.Decode(&value); err != nil {
return zero, err
}
var extra any
if err := dec.Decode(&extra); err != io.EOF {
return zero, errors.New("config: one YAML document required")
}
case ".json":
if err := apierrors.DecodeCatalogJSON(data, &value); err != nil {
return zero, err
}
default:
return zero, errors.New("config: expected .json, .yaml or .yml")
}
if _, ok := value.(map[string]any); !ok {
return zero, errors.New("config: root must be an object")
}
value, err = durations(value, reflect.TypeOf(defaults))
if err != nil {
return zero, err
}
encoded, err := json.Marshal(value)
if err != nil {
return zero, err
}
// Clone configuration without serializing borrowed json:"-" capabilities.
var result T
original, err := cloneConfiguration(reflect.ValueOf(&defaults).Elem(), 0)
if err != nil {
return zero, err
}
reflect.ValueOf(&result).Elem().Set(original)
dec := json.NewDecoder(bytes.NewReader(encoded))
dec.DisallowUnknownFields()
if err = dec.Decode(&result); err != nil {
return zero, err
}
return result, nil
}
func durations(v any, t reflect.Type) (any, error) { func durations(v any, t reflect.Type) (any, error) {
if t == nil { if t == nil {
return v, nil return v, nil

175
config/loader.go Normal file
View File

@@ -0,0 +1,175 @@
package config
import (
"errors"
"io"
"os"
"path/filepath"
"reflect"
"strings"
)
const maxConfigBytes = 1 << 20
const maxConfigDepth = 64
const maxConfigNodes = 100000
// Decoder must not retain or mutate input and must support concurrent calls
// when its Loader is shared. It is trusted application code, not a sandbox.
type Decoder interface {
Decode([]byte) (Document, error)
}
type DecoderFunc func([]byte) (Document, error)
func (f DecoderFunc) Decode(data []byte) (Document, error) { return f(data) }
// Loader owns an immutable per-instance decoder registry. No global registration.
type Loader struct {
decoders map[string]Decoder
sealed bool
}
type LoaderOption func(*Loader) error
// WithDecoder adds a new extension; builtin/duplicate extensions cannot be replaced.
func WithDecoder(extension string, decoder Decoder) LoaderOption {
return func(l *Loader) error {
if l == nil || l.decoders == nil || l.sealed {
return errors.New("config: decoder registry cannot be changed after construction")
}
ext := strings.ToLower(strings.TrimPrefix(extension, "."))
if ext == "" || strings.ContainsAny(ext, "./\\ \t\r\n") {
return errors.New("config: invalid decoder extension")
}
if decoder == nil {
return errors.New("config: nil decoder")
}
v := reflect.ValueOf(decoder)
if (v.Kind() == reflect.Pointer || v.Kind() == reflect.Func || v.Kind() == reflect.Map || v.Kind() == reflect.Slice || v.Kind() == reflect.Interface) && v.IsNil() {
return errors.New("config: nil decoder")
}
if _, exists := l.decoders[ext]; exists {
return errors.New("config: decoder extension already registered")
}
l.decoders[ext] = decoder
return nil
}
}
func NewLoader(options ...LoaderOption) (*Loader, error) {
l := &Loader{decoders: map[string]Decoder{"json": JSONDecoder{}, "yaml": YAMLDecoder{}, "yml": YAMLDecoder{}, "xml": XMLDecoder{}}}
for _, o := range options {
if o == nil {
return nil, errors.New("config: nil loader option")
}
if err := o(l); err != nil {
return nil, err
}
}
l.sealed = true
return l, nil
}
var defaultLoader, _ = NewLoader()
func selectLoader(loaders []*Loader) (*Loader, error) {
if len(loaders) == 0 {
return defaultLoader, nil
}
if len(loaders) != 1 || loaders[0] == nil {
return nil, errors.New("config: expected one non-nil loader")
}
return loaders[0], nil
}
func (l *Loader) read(path string) (doc Document, err error) {
defer func() {
if err != nil {
var d *Error
if errors.As(err, &d) {
c := *d
c.File = path
err = &c
} else {
reason := "cannot read configuration file"
if errors.Is(err, os.ErrNotExist) {
reason = "configuration file does not exist"
} else if errors.Is(err, os.ErrPermission) {
reason = "permission denied reading configuration file"
}
err = &Error{File: path, Reason: reason, cause: err}
}
}
}()
decoder := l.decoders[strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")]
if decoder == nil {
return doc, &Error{Reason: "unsupported configuration format"}
}
info, e := os.Stat(path)
if e != nil {
return doc, e
}
if !info.Mode().IsRegular() {
return doc, &Error{Reason: "configuration must be a regular file"}
}
f, e := os.Open(path)
if e != nil {
return doc, e
}
defer f.Close()
info, e = f.Stat()
if e != nil {
return doc, e
}
if !info.Mode().IsRegular() {
return doc, &Error{Reason: "configuration must be a regular file"}
}
data, e := io.ReadAll(io.LimitReader(f, maxConfigBytes+1))
if e != nil {
return doc, e
}
if len(data) > maxConfigBytes {
return doc, &Error{Reason: "file exceeds 1 MiB"}
}
doc, err = decodeSafely(decoder, data)
if err != nil {
return doc, err
}
if _, ok := doc.Value.(map[string]any); !ok {
return doc, doc.diagnostic("", "root must be an object", nil)
}
return doc, nil
}
func decodeSafely(decoder Decoder, data []byte) (doc Document, err error) {
defer func() {
if recover() != nil {
err = &Error{Reason: "decoder panicked"}
}
}()
doc, err = decoder.Decode(data)
if err != nil {
var d *Error
if !errors.As(err, &d) {
err = &Error{Reason: "invalid configuration syntax", cause: err}
}
}
return
}
// DecodeFile overlays a bounded document onto independent defaults. With no
// loader argument it accepts JSON/YAML/XML. It never opens dependencies.
func DecodeFile[T any](path string, defaults T, loaders ...*Loader) (T, error) {
var zero T
l, err := selectLoader(loaders)
if err != nil {
return zero, err
}
doc, err := l.read(path)
if err != nil {
return zero, err
}
result, err := bindDocument(doc, defaults)
if err != nil {
d := validationError(err)
d.File = path
return zero, d
}
return result, nil
}

397
config/loader_test.go Normal file
View File

@@ -0,0 +1,397 @@
package config
import (
"encoding/json"
"errors"
"fmt"
"os"
"path/filepath"
"reflect"
"strings"
"sync"
"testing"
"time"
)
type loaderExample struct {
Server Config `json:"server"`
Upload struct {
MaxBytes int64 `json:"maxBytes" envDefault:"1024"`
} `json:"upload"`
Values []int64 `json:"values"`
Optional *string `json:"optional"`
Enabled bool `json:"enabled"`
Dynamic any `json:"dynamic"`
}
func (c loaderExample) Validate() error {
if err := c.Server.Validate(); err != nil {
return AtField("server", err)
}
if c.Upload.MaxBytes < 0 {
return Invalid("upload.maxBytes", "must be nonnegative")
}
return nil
}
func noEnvironment(string) (string, bool) { return "", false }
func configFile(t *testing.T, ext, raw string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "service."+ext)
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
t.Fatal(err)
}
return path
}
func TestUnifiedFormatAndPrecedence(t *testing.T) {
formats := map[string]string{
"json": `{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":7},"values":[9007199254740993],"optional":null,"enabled":true,"dynamic":{"id":9007199254740993}}`,
"yaml": "server:\n http:\n readTimeout: 2s\nupload:\n maxBytes: 7\nvalues: [9007199254740993]\noptional: null\nenabled: true\ndynamic:\n id: 9007199254740993\n",
"xml": `<config><server><http><readTimeout>2s</readTimeout></http></server><upload><maxBytes>7</maxBytes></upload><values type="array"><item>9007199254740993</item></values><optional type="null"/><enabled>true</enabled><dynamic><id type="number">9007199254740993</id></dynamic></config>`,
}
var reference loaderExample
for _, ext := range []string{"json", "yaml", "xml"} {
t.Run(ext, func(t *testing.T) {
old := "old"
base := loaderExample{Server: Default(), Optional: &old}
cfg, err := LoadProject(configFile(t, ext, formats[ext]), base, EnvOptions{Prefix: "APP_", Lookup: func(key string) (string, bool) {
v, ok := map[string]string{"APP_UPLOAD_MAX_BYTES": "0", "APP_ENABLED": "false"}[key]
return v, ok
}})
if err != nil {
t.Fatal(err)
}
if cfg.Upload.MaxBytes != 0 || cfg.Enabled || cfg.Optional != nil || cfg.Values[0] != 9007199254740993 || cfg.Server.HTTP.ReadTimeout != 2*time.Second {
t.Fatal("values changed", cfg)
}
if cfg.Dynamic.(map[string]any)["id"] != json.Number("9007199254740993") {
t.Fatal("dynamic integer precision lost")
}
if ext == "json" {
reference = cfg
} else if !reflect.DeepEqual(reference, cfg) {
t.Fatal("format-dependent semantics")
}
if base.Optional == nil || *base.Optional != "old" {
t.Fatal("defaults mutated")
}
})
}
cfg, err := LoadProject("", loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
if err != nil || cfg.Upload.MaxBytes != 1024 {
t.Fatal(cfg, err)
}
}
func TestConfigDiagnosticsAndCauses(t *testing.T) {
for _, tc := range []struct {
ext, body, field string
line bool
}{
{"json", "{\n\"upload\": {\"maxBytes\":\"private-value\"}}", "upload.maxBytes", true},
{"yaml", "server:\n http:\n readTimeout: private-value", "server.http.readTimeout", true},
{"xml", "<config>\n<upload><maxBytes>private-value</maxBytes></upload></config>", "upload.maxBytes", true},
{"json", `{"upload":{"maxBytes":-1}}`, "upload.maxBytes", true},
{"yaml", "server:\n http:\n readTimeout: -1s", "server.http.readTimeout", true},
{"json", `{"upload":{"typo":3}}`, "upload.typo", true},
} {
t.Run(tc.ext+tc.field, func(t *testing.T) {
path := configFile(t, tc.ext, tc.body)
_, err := LoadProject(path, loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
var d *Error
if !errors.As(err, &d) || d.File != path || d.Field != tc.field || d.Reason == "" || tc.line && d.Line < 1 {
t.Fatalf("missing location: %#v %v", d, err)
}
if strings.Contains(err.Error(), "private-value") {
t.Fatal("secret leaked", err)
}
})
}
path := filepath.Join(t.TempDir(), "missing.json")
_, err := LoadProject(path, loaderExample{}, EnvOptions{})
if !errors.Is(err, os.ErrNotExist) {
t.Fatal("I/O cause discarded", err)
}
_, err = DecodeFile(configFile(t, "json", `{"count":"private-value"}`), struct {
Count int `json:"count"`
}{})
var typed *json.UnmarshalTypeError
if !errors.As(err, &typed) {
t.Fatal("type cause discarded", err)
}
_, err = LoadProject(configFile(t, "yaml", "upload:\n maxBytes: 7"), loaderExample{Server: Default()}, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { return "-1", k == "APP_UPLOAD_MAX_BYTES" }})
var d *Error
if !errors.As(err, &d) || d.Env != "APP_UPLOAD_MAX_BYTES" || d.Field != "upload.maxBytes" || d.File != "" || d.Line != 0 {
t.Fatal("environment validation wrongly blamed file", err)
}
}
func TestMalformedFormatsAndLimits(t *testing.T) {
for ext, bodies := range map[string][]string{
"json": {`{"enabled":true,"enabled":false}`, `{} {}`, `{"upload":`, `{"enabled":true,"Enabled":false}`, strings.Repeat(`{"a":`, 70) + `0` + strings.Repeat(`}`, 70)},
"yaml": {"enabled: true\nenabled: false", "---\n{}\n---\n{}", "a: &x [1]\nb: *x", "a:\n <<: {}", "upload: [\n"},
"xml": {`<!DOCTYPE config><config/>`, `<config>&private-value;</config>`, `<config/><config/>`, `<config xmlns="x"/>`, `<config type="" type=""/>`, `<config><enabled>true</enabled><enabled>false</enabled></config>`, `<config>text<enabled>true</enabled></config>`, `<config><values type="array"><bad>1</bad></values></config>`},
} {
for i, raw := range bodies {
t.Run(fmt.Sprintf("%s%d", ext, i), func(t *testing.T) {
_, err := LoadProject(configFile(t, ext, raw), loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment})
if err == nil || strings.Contains(err.Error(), "private-value") {
t.Fatal(err)
}
})
}
}
if _, err := DecodeFile(configFile(t, "json", strings.Repeat(" ", maxConfigBytes+1)), loaderExample{}); err == nil {
t.Fatal("unbounded file")
}
if _, err := DecodeFile(configFile(t, "json", string([]byte{'{', '"', 0xff, '"', ':', '0', '}'})), loaderExample{}); err == nil {
t.Fatal("invalid UTF-8")
}
}
func TestDecoderRegistryAndConcurrentUse(t *testing.T) {
decoder := DecoderFunc(func([]byte) (Document, error) {
return Document{Value: map[string]any{"upload": map[string]any{"maxBytes": json.Number("42")}}}, nil
})
l, err := NewLoader(WithDecoder("custom", decoder))
if err != nil {
t.Fatal(err)
}
path := configFile(t, "custom", "ignored")
var wg sync.WaitGroup
for i := 0; i < 24; i++ {
wg.Add(1)
go func() {
defer wg.Done()
cfg, e := LoadProject(path, loaderExample{Server: Default()}, EnvOptions{Lookup: noEnvironment}, l)
if e != nil || cfg.Upload.MaxBytes != 42 {
t.Error(e)
}
}()
}
wg.Wait()
if _, err := DecodeFile(path, loaderExample{}); err == nil {
t.Fatal("decoder registration leaked globally")
}
for _, options := range [][]LoaderOption{{WithDecoder("json", decoder)}, {WithDecoder("x", decoder), WithDecoder("x", decoder)}, {WithDecoder("x", DecoderFunc(nil))}} {
if _, err = NewLoader(options...); err == nil {
t.Fatal("invalid registration")
}
}
l, _ = NewLoader(WithDecoder("panic", DecoderFunc(func([]byte) (Document, error) { panic("private-value") })))
_, err = DecodeFile(configFile(t, "panic", ""), loaderExample{}, l)
if err == nil || strings.Contains(err.Error(), "private-value") {
t.Fatal(err)
}
cycle := map[string]any{}
cycle["self"] = cycle
l, _ = NewLoader(WithDecoder("cycle", DecoderFunc(func([]byte) (Document, error) { return Document{Value: map[string]any{"dynamic": cycle}}, nil })))
if _, err = DecodeFile(configFile(t, "cycle", ""), loaderExample{}, l); err == nil {
t.Fatal("decoder bypassed traversal limit")
}
}
func TestUnifiedLoggerNormalization(t *testing.T) {
type project struct {
Server Config `json:"server"`
}
base := project{Server: Default()}
base.Server.Log.File = "old.log"
base.Server.Log.Rotation.Enabled = true
for _, full := range []bool{false, true} {
name := "GO_OMNI_LOG_OUTPUT"
if full {
name = "GO_OMNI_SERVER_LOG_OUTPUT"
}
cfg, err := LoadProject("", base, EnvOptions{Prefix: "GO_OMNI_", Empty: "ignore", Aliases: ServerEnvAliases("GO_OMNI_", "server"), Lookup: func(k string) (string, bool) { return "stdout", k == name }})
if err != nil || cfg.Server.Log.File != "" || cfg.Server.Log.Rotation.Enabled || base.Server.Log.File != "old.log" {
t.Fatal(cfg, err)
}
}
}
func FuzzConfigurationDecoders(f *testing.F) {
f.Add(uint8(0), []byte(`{"count":1}`))
f.Add(uint8(1), []byte("count: 1"))
f.Add(uint8(2), []byte("<config><count>1</count></config>"))
f.Fuzz(func(t *testing.T, format uint8, data []byte) {
if len(data) > 8192 {
data = data[:8192]
}
ds := []Decoder{JSONDecoder{}, YAMLDecoder{}, XMLDecoder{}}
doc, err := ds[int(format)%len(ds)].Decode(data)
if err == nil {
_, _ = bindDocument(doc, struct {
Count int `json:"count"`
}{})
}
})
}
func BenchmarkConfiguration(b *testing.B) {
for name, tc := range map[string]struct {
d Decoder
raw string
}{"json": {JSONDecoder{}, `{"count":9007199254740993,"timeout":"5s","values":[1,2,3]}`}, "yaml": {YAMLDecoder{}, "count: 9007199254740993\ntimeout: 5s\nvalues: [1,2,3]"}, "xml": {XMLDecoder{}, `<config><count>9007199254740993</count><timeout>5s</timeout><values type="array"><item>1</item><item>2</item><item>3</item></values></config>`}} {
b.Run(name, func(b *testing.B) {
b.ReportAllocs()
raw := []byte(tc.raw)
for i := 0; i < b.N; i++ {
doc, err := tc.d.Decode(raw)
if err != nil {
b.Fatal(err)
}
_, err = bindDocument(doc, struct {
Count int64 `json:"count"`
Timeout time.Duration `json:"timeout"`
Values []int `json:"values"`
}{})
if err != nil {
b.Fatal(err)
}
}
})
}
}
// Text-only scalars and embedded config fields use the same behavior in each format.
type testText struct{ Value string }
func (v *testText) UnmarshalText(data []byte) error { v.Value = string(data); return nil }
func TestConfigScalarAndEmbedding(t *testing.T) {
type Fields struct {
Name string `json:"name"`
}
type C struct {
Fields
Nested Fields `json:"nested"`
Scalar testText `json:"scalar"`
Bytes []byte `json:"bytes"`
}
cases := map[string]string{
"json": `{"name":"outer","nested":{"name":"inner"},"scalar":"text","bytes":"aGk="}`,
"yaml": "name: outer\nnested:\n name: inner\nscalar: text\nbytes: aGk=\n",
"xml": `<config><name>outer</name><nested><name>inner</name></nested><scalar>text</scalar><bytes>aGk=</bytes></config>`,
}
for ext, data := range cases {
t.Run(ext, func(t *testing.T) {
path := filepath.Join(t.TempDir(), "config."+ext)
os.WriteFile(path, []byte(data), 0600)
got, err := DecodeFile(path, &C{})
if err != nil {
t.Fatal(err)
}
if got.Name != "outer" || got.Nested.Name != "inner" || got.Scalar.Value != "text" || string(got.Bytes) != "hi" {
t.Fatal(got)
}
})
}
type Tagged struct {
Fields `json:"nested"`
}
doc, _ := (JSONDecoder{}).Decode([]byte(`{"name":"wrong"}`))
if _, err := bindDocument(doc, Tagged{}); err == nil {
t.Fatal("tagged embed incorrectly flattened")
}
}
func TestConfigDefaultsIsolationAndEmptyValues(t *testing.T) {
type Child struct {
Name string `json:"name"`
}
type C struct {
Child *Child `json:"child"`
Items []int `json:"items"`
Labels map[string]string `json:"labels"`
Enabled bool `json:"enabled"`
}
base := C{Child: &Child{Name: "old"}, Items: []int{1}, Labels: map[string]string{"old": "yes"}, Enabled: true}
doc, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"new"},"labels":{"new":"yes"},"items":[],"enabled":false}`))
got, err := bindDocument(doc, base)
if err != nil {
t.Fatal(err)
}
if got.Child.Name != "new" || got.Items == nil || len(got.Items) != 0 || got.Enabled {
t.Fatal(got)
}
if base.Child.Name != "old" || len(base.Items) != 1 || len(base.Labels) != 1 {
t.Fatal("mutated defaults")
}
bad, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"changed"},"items":"bad"}`))
if _, err := bindDocument(bad, base); err == nil {
t.Fatal("invalid input accepted")
}
if base.Child.Name != "old" {
t.Fatal("failed load changed default pointer")
}
null, _ := (JSONDecoder{}).Decode([]byte(`{"child":null,"items":null,"labels":null,"enabled":null}`))
got, err = bindDocument(null, base)
if err != nil || got.Child != nil || got.Items != nil || got.Labels != nil || !got.Enabled {
t.Fatal(got, err)
}
}
func TestDefaultStageDoesNotReadEnvironment(t *testing.T) {
reads := 0
type C struct {
Value int `json:"value" envDefault:"3"`
}
got, err := LoadProject("", C{}, EnvOptions{Lookup: func(string) (string, bool) { reads++; return "", false }})
if err != nil || got.Value != 3 || reads != 1 {
t.Fatal(got, err, reads)
}
d := locateError(Invalid("servers[0].timeout", "must be positive"), "a.json", Document{}, map[string]string{"servers": "APP_SERVERS"})
if d.Env != "APP_SERVERS" || d.File != "" {
t.Fatal(d)
}
}
func TestDecoderRegistryFrozen(t *testing.T) {
l, err := NewLoader()
if err != nil {
t.Fatal(err)
}
if err = WithDecoder("extra", JSONDecoder{})(l); err == nil {
t.Fatal("mutable registry")
}
}
func BenchmarkConfigurationStartup(b *testing.B) {
path := filepath.Join(b.TempDir(), "app.json")
if err := os.WriteFile(path, []byte(`{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":4096}}`), 0600); err != nil {
b.Fatal(err)
}
options := EnvOptions{Prefix: "BENCH_", Lookup: func(name string) (string, bool) {
if name == "BENCH_UPLOAD_MAX_BYTES" {
return "8192", true
}
return "", false
}}
defaults := loaderExample{Server: Default()}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if _, err := LoadProject(path, defaults, options); err != nil {
b.Fatal(err)
}
}
}
func BenchmarkConfigurationLarge(b *testing.B) {
type Entry struct {
Timeout time.Duration `json:"timeout"`
Count int `json:"count"`
}
type Large struct {
Entries []Entry `json:"entries"`
}
raw := []byte(`{"entries":[` + strings.TrimSuffix(strings.Repeat(`{"timeout":"3s","count":10},`, 1000), ",") + `]}`)
b.SetBytes(int64(len(raw)))
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
doc, err := (JSONDecoder{}).Decode(raw)
if err != nil {
b.Fatal(err)
}
if _, err = bindDocument(doc, Large{}); err != nil {
b.Fatal(err)
}
}
}

View File

@@ -7,6 +7,7 @@ import (
"errors" "errors"
"fmt" "fmt"
apierrors "git.yuntuo.hk/k2safe/go-omni/errors" apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
"git.yuntuo.hk/k2safe/go-omni/logx"
"math" "math"
"os" "os"
"reflect" "reflect"
@@ -34,30 +35,114 @@ func ApplyDefaults[T any](base T) (T, error) { return projectOverlay(base, EnvOp
// LoadProject applies code/tag defaults, a strict optional file, environment, // LoadProject applies code/tag defaults, a strict optional file, environment,
// then the project's Validate method, before any logger or dependency is opened. // then the project's Validate method, before any logger or dependency is opened.
// envDefault is only applied to zero-valued defaults before reading the file. // envDefault is only applied to zero-valued defaults before reading the file.
func LoadProject[T any](path string, defaults T, options EnvOptions) (T, error) { func LoadProject[T any](path string, defaults T, options EnvOptions, loaders ...*Loader) (T, error) {
var zero T var zero T
loader, err := selectLoader(loaders)
if err != nil {
return zero, err
}
cfg, err := projectOverlay(defaults, options, true) cfg, err := projectOverlay(defaults, options, true)
if err != nil { if err != nil {
return zero, err return zero, err
} }
var doc Document
if path != "" { if path != "" {
cfg, err = DecodeFile(path, cfg) doc, err = loader.read(path)
if err != nil { if err != nil {
return zero, errors.New("config: cannot decode file; check its path, format, field names and value types") return zero, err
} }
cfg, err = bindDocument(doc, cfg)
if err != nil {
return zero, locateError(err, path, doc, nil)
}
} else {
value, e := cloneConfiguration(reflect.ValueOf(&cfg).Elem(), 0)
if e != nil {
return zero, &Error{Reason: "cannot clone defaults", cause: e}
}
cfg = value.Interface().(T)
} }
cfg, err = OverlayEnv(cfg, options) origins := map[string]string{}
cfg, err = projectOverlay(cfg, options, false, origins)
if err != nil { if err != nil {
return zero, err return zero, err
} }
normalizeProject(reflect.ValueOf(&cfg).Elem(), "", origins, 0)
if valid, ok := any(&cfg).(interface{ Validate() error }); ok { if valid, ok := any(&cfg).(interface{ Validate() error }); ok {
if err = valid.Validate(); err != nil { if err = valid.Validate(); err != nil {
return zero, err return zero, locateError(err, path, doc, origins)
} }
} }
return cfg, nil return cfg, nil
} }
func locateError(err error, path string, doc Document, origins map[string]string) *Error {
d := validationError(err)
field := d.Field
for field != "" && origins[field] == "" {
index := strings.LastIndexAny(field, ".[")
if index < 0 {
field = ""
} else {
field = field[:index]
}
}
if env := origins[field]; env != "" {
d.Env = env
d.File = ""
d.Line = 0
d.Column = 0
return d
}
d.File = path
if d.Line == 0 {
p := doc.Positions[d.Field]
d.Line = p.Line
d.Column = p.Column
}
return d
}
// Framework-owned normalization stays inside the common pipeline. Explicit
// environment stream selection clears a file profile unless a file override
// was also provided; project code never needs to repeat this rule.
func normalizeProject(v reflect.Value, path string, origins map[string]string, depth int) {
if depth > 32 {
return
}
if v.Kind() == reflect.Pointer {
if v.IsNil() {
return
}
v = v.Elem()
}
if v.Kind() != reflect.Struct {
return
}
if v.Type() == reflect.TypeOf(Config{}) && v.CanAddr() {
c := v.Addr().Interface().(*Config)
if origins[joinField(path, "log.output")] != "" && c.Log.Output != "" && (origins[joinField(path, "log.file")] == "" || c.Log.File == "") {
c.Log.File = ""
c.Log.Rotation = logx.RotationConfig{}
}
return
}
for i := 0; i < v.NumField(); i++ {
f := v.Type().Field(i)
if f.PkgPath != "" {
continue
}
name := strings.Split(f.Tag.Get("json"), ",")[0]
if name == "-" {
continue
}
if name == "" {
name = f.Name
}
normalizeProject(v.Field(i), joinField(path, name), origins, depth+1)
}
}
// OverlayEnv overlays the entire project struct without mutating the input. // OverlayEnv overlays the entire project struct without mutating the input.
// Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a // Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a
// subtree. required means nonempty string/non-nil pointer or collection; false // subtree. required means nonempty string/non-nil pointer or collection; false
@@ -70,7 +155,7 @@ func OverlayEnv[T any](base T, options EnvOptions) (T, error) {
var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem() var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
var durationType = reflect.TypeOf(time.Duration(0)) var durationType = reflect.TypeOf(time.Duration(0))
func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error) { func projectOverlay[T any](base T, options EnvOptions, defaults bool, sourceMaps ...map[string]string) (T, error) {
var zero T var zero T
if options.Lookup == nil { if options.Lookup == nil {
options.Lookup = os.LookupEnv options.Lookup = os.LookupEnv
@@ -88,8 +173,8 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
result := reflect.New(v.Type()).Elem() result := reflect.New(v.Type()).Elem()
result.Set(v) result.Set(v)
seen := map[string]bool{} seen := map[string]bool{}
var walk func(reflect.Value, string, int) error var walk func(reflect.Value, string, string, int) error
walk = func(value reflect.Value, prefix string, depth int) error { walk = func(value reflect.Value, prefix, path string, depth int) error {
if depth > 32 { if depth > 32 {
return errors.New("config: nesting exceeds 32") return errors.New("config: nesting exceeds 32")
} }
@@ -110,6 +195,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if jsonName == "" { if jsonName == "" {
jsonName = field.Name jsonName = field.Name
} }
fieldPath := joinField(path, jsonName)
key := prefix + envName(jsonName) key := prefix + envName(jsonName)
if tag[0] != "" { if tag[0] != "" {
key = tag[0] key = tag[0]
@@ -131,7 +217,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
} }
} }
if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) { if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) {
if err := walk(dst, key+"_", depth+1); err != nil { if err := walk(dst, key+"_", fieldPath, depth+1); err != nil {
return err return err
} }
continue continue
@@ -142,14 +228,14 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if !dst.IsNil() { if !dst.IsNil() {
child.Elem().Set(dst.Elem()) child.Elem().Set(dst.Elem())
} }
if err := walk(child.Elem(), key+"_", depth+1); err != nil { if err := walk(child.Elem(), key+"_", fieldPath, depth+1); err != nil {
return err return err
} }
if !dst.IsNil() || !child.Elem().IsZero() { if !dst.IsNil() || !child.Elem().IsZero() {
dst.Set(child) dst.Set(child)
} }
if required && !defaults && dst.IsNil() { if required && !defaults && dst.IsNil() {
return fmt.Errorf("config: %s is required", key) return &Error{Env: key, Field: fieldPath, Reason: "is required"}
} }
continue continue
} }
@@ -164,7 +250,11 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
return fmt.Errorf("config: duplicate environment key %s", key) return fmt.Errorf("config: duplicate environment key %s", key)
} }
seen[key] = true seen[key] = true
raw, present := options.Lookup(key) var raw string
var present bool
if !defaults {
raw, present = options.Lookup(key)
}
selectedKey := key selectedKey := key
if !present && !defaults { if !present && !defaults {
for _, alias := range options.Aliases[key] { for _, alias := range options.Aliases[key] {
@@ -180,7 +270,7 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
} }
if present && raw == "" && !defaults { if present && raw == "" && !defaults {
if empty == "reject" { if empty == "reject" {
return fmt.Errorf("config: %s must not be empty", selectedKey) return &Error{Env: selectedKey, Field: fieldPath, Reason: "must not be empty"}
} }
if empty == "ignore" { if empty == "ignore" {
present = false present = false
@@ -189,17 +279,24 @@ func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error)
if present { if present {
parsed := reflect.New(dst.Type()).Elem() parsed := reflect.New(dst.Type()).Elem()
if err := parseEnvValue(parsed, raw); err != nil { if err := parseEnvValue(parsed, raw); err != nil {
return fmt.Errorf("config: %s must be a valid %s", selectedKey, dst.Type()) origin := selectedKey
if defaults {
origin = ""
}
return &Error{Env: origin, Field: fieldPath, Reason: "expected a valid " + dst.Type().String(), cause: err}
} }
dst.Set(parsed) dst.Set(parsed)
if !defaults && len(sourceMaps) > 0 {
sourceMaps[0][fieldPath] = selectedKey
}
} }
if required && !defaults && emptyRequired(dst) { if required && !defaults && emptyRequired(dst) {
return fmt.Errorf("config: %s is required", key) return &Error{Env: key, Field: fieldPath, Reason: "is required"}
} }
} }
return nil return nil
} }
if err := walk(result, options.Prefix, 0); err != nil { if err := walk(result, options.Prefix, "", 0); err != nil {
return zero, err return zero, err
} }
return result.Interface().(T), nil return result.Interface().(T), nil

View File

@@ -0,0 +1,21 @@
# 统一配置加载示例
从框架根目录运行,无需数据库或外部服务:
```sh
go run ./examples/configuration
go run ./examples/configuration -format json
go run ./examples/configuration -format xml
DEMO_CLIENT_TIMEOUT=500ms DEMO_MAX_ITEMS=0 go run ./examples/configuration
DEMO_MAX_ITEMS=wrong go run ./examples/configuration
```
前三次都输出 `timeout=2s maxItems=100 allowed=[json xml]`。环境覆盖允许显式零值。
最后一条输出定位 `maxItems` / `DEMO_MAX_ITEMS` 的安全错误,不打印输入原值。
也可 `-config /path/to/app.yaml` 读取自己的文件。示例没有建立网络连接。
`Config` 定义字段,`envDefault` 定义缺省值,`Validate` 用 `config.Invalid` 返回字段与公开原因。
`newLoader` 额外注册 `.custom`(演示复用 JSON decoder);同一个 `load` 可读取所有格式。
JSON/YAML/XML 直接可用,无需注册。新增格式只修改解析器注册,不改业务结构或校验。
只打印选定非敏感字段,不输出整个配置。完整格式、错误定位和旧项目迁移见 [CONFIG.md](../../CONFIG.md)。

View File

@@ -0,0 +1,5 @@
{
"client": {"endpoint": "https://example.com", "timeout": "2s"},
"maxItems": 100,
"allowed": ["json", "xml"]
}

View File

@@ -0,0 +1,8 @@
<config>
<client>
<endpoint>https://example.com</endpoint>
<timeout>2s</timeout>
</client>
<maxItems>100</maxItems>
<allowed type="array"><item>json</item><item>xml</item></allowed>
</config>

View File

@@ -0,0 +1,5 @@
client:
endpoint: https://example.com
timeout: 2s
maxItems: 100
allowed: [json, xml]

View File

@@ -0,0 +1,85 @@
// Run with go run ./examples/configuration -format xml, or -config /path/to/app.yaml.
package main
import (
"embed"
"flag"
"fmt"
"os"
"path/filepath"
"time"
"git.yuntuo.hk/k2safe/go-omni/config"
)
//go:embed config.json config.yaml config.xml
var examples embed.FS
type ClientConfig struct {
Endpoint string `json:"endpoint"`
Timeout time.Duration `json:"timeout" envDefault:"3s"`
// This field is deliberately unavailable to configuration files.
Token string `json:"-" env:"DEMO_CLIENT_TOKEN,secret"`
}
type Config struct {
Client ClientConfig `json:"client"`
MaxItems int `json:"maxItems" envDefault:"20"`
Allowed []string `json:"allowed"`
}
func (c Config) Validate() error {
if c.Client.Timeout <= 0 {
return config.Invalid("client.timeout", "must be positive")
}
if c.MaxItems < 0 {
return config.Invalid("maxItems", "must not be negative")
}
return nil
}
func newLoader() (*config.Loader, error) {
// Example of an instance-local extension. A TOML/etc. adapter can implement
// Decoder and return Document using that parser instead; no global registry.
return config.NewLoader(config.WithDecoder("custom", config.DecoderFunc(func(data []byte) (config.Document, error) {
return (config.JSONDecoder{}).Decode(data)
})))
}
func load(path string) (Config, error) {
loader, err := newLoader()
if err != nil {
return Config{}, err
}
return config.LoadProject(path, Config{}, config.EnvOptions{Prefix: "DEMO_", Empty: "reject"}, loader)
}
func run() error {
path := flag.String("config", "", "external configuration file")
format := flag.String("format", "yaml", "embedded example: json, yaml, xml")
flag.Parse()
if *path == "" {
data, err := examples.ReadFile("config." + *format)
if err != nil {
return fmt.Errorf("choose json, yaml or xml")
}
dir, err := os.MkdirTemp("", "goOmniConfiguration-")
if err != nil {
return err
}
defer os.RemoveAll(dir)
*path = filepath.Join(dir, "config."+*format)
if err = os.WriteFile(*path, data, 0600); err != nil {
return err
}
}
cfg, err := load(*path)
if err != nil {
return err
}
// Print selected non-sensitive fields, never the whole configuration.
fmt.Printf("timeout=%s maxItems=%d allowed=%v\n", cfg.Client.Timeout, cfg.MaxItems, cfg.Allowed)
return nil
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

View File

@@ -0,0 +1,34 @@
package main
import (
"os"
"path/filepath"
"reflect"
"testing"
)
func TestExamples(t *testing.T) {
var previous Config
for i, ext := range []string{"json", "yaml", "xml", "custom"} {
source := ext
if source == "custom" {
source = "json"
}
data, err := examples.ReadFile("config." + source)
if err != nil {
t.Fatal(err)
}
path := filepath.Join(t.TempDir(), "app."+ext)
if err = os.WriteFile(path, data, 0600); err != nil {
t.Fatal(err)
}
got, err := load(path)
if err != nil {
t.Fatal(err)
}
if i > 0 && !reflect.DeepEqual(previous, got) {
t.Fatal("format difference")
}
previous = got
}
}

View File

@@ -0,0 +1,12 @@
# HTTP Client 示例
```sh
go run ./examples/httpClient
```
只创建本机临时 HTTP 服务,不访问业务系统、数据库或公网;结束后关闭连接和监听器。
示例创建并复用一个 Client,显式直连,配置连接池/阶段超时,通过 JSON 选项传入 Authorization 和 Accept,并按单次请求收紧响应上限。
输出 `authenticated request succeeded`。示例 token 是固定演示字符串,实际项目从配置或请求上下文取得认证信息。
完整参数、代理规则、流式调用、生命周期、重试边界和兼容说明见 [httpc](../../httpc/README.md)。

View File

@@ -0,0 +1,57 @@
// Run with go run ./examples/httpClient. All HTTP traffic stays on loopback.
package main
import (
"context"
"encoding/json"
"fmt"
"net/http"
"net/http/httptest"
"os"
"time"
"git.yuntuo.hk/k2safe/go-omni/httpc"
)
type result struct {
Message string `json:"message"`
}
func run() error {
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer example-token" {
w.WriteHeader(http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(result{Message: "authenticated request succeeded"})
}))
defer upstream.Close()
client, err := httpc.New(httpc.Config{
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
Timeout: 3 * time.Second, DialTimeout: time.Second, TLSHandshakeTimeout: time.Second,
ResponseHeaderTimeout: time.Second, IdleConnTimeout: 30 * time.Second,
MaxConnsPerHost: 16, MaxIdleConnsPerHost: 8, MaxIdleConns: 32,
MaxResponseBytes: 1 << 20, DisableLogging: true,
})
if err != nil {
return err
}
defer client.Close()
var output result
if err = client.JSON(context.Background(), http.MethodGet, upstream.URL, nil, &output,
httpc.WithHeader("Authorization", "Bearer example-token"),
httpc.WithHeaders(http.Header{"Accept": []string{"application/json"}}),
httpc.WithMaxResponseBytes(64<<10),
); err != nil {
return err
}
fmt.Println(output.Message)
return nil
}
func main() {
if err := run(); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}

View File

@@ -0,0 +1,9 @@
package main
import "testing"
func TestExample(t *testing.T) {
if err := run(); err != nil {
t.Fatal(err)
}
}

View File

@@ -1,6 +1,6 @@
{ {
"generatorConfig": true, "generatorConfig": true,
"templateVersion": "v0.19.0-preview", "templateVersion": "v0.20.0-preview",
"profile": { "profile": {
"database": "none", "database": "none",
"storage": "none", "storage": "none",
@@ -12,6 +12,7 @@
"entry": "api/managed.api", "entry": "api/managed.api",
"generated": { "generated": {
"docs/openapi.json": "0370fd6bd6a387e2ef21f6b42b009ce66749846b8bea3d481009a34f3af039a5", "docs/openapi.json": "0370fd6bd6a387e2ef21f6b42b009ce66749846b8bea3d481009a34f3af039a5",
"internal/config/loadGen.go": "5d630b894abdbbd66ddfa7d7d30f7573fc4ff995020a32a9f9157cd4af3a5655",
"internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d", "internal/handler/appGen.go": "f5251b4dc33e2312b0bdb71ead70d2171977d851265e124f640458d705a2cc9d",
"internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375", "internal/handler/demo/handlersGen.go": "3c76d85dbe89eb5127d9d134dc625203ecf9719dd11d174c1f335691b99e1375",
"internal/handler/openapiGen.go": "f24cab3abbb8557a3fc4dd93fba23cc0b093f3e0c944d3ba1bd5b0eeff2d96f3", "internal/handler/openapiGen.go": "f24cab3abbb8557a3fc4dd93fba23cc0b093f3e0c944d3ba1bd5b0eeff2d96f3",

View File

@@ -2,7 +2,7 @@ module example.com/managed
go 1.22.0 go 1.22.0
require git.yuntuo.hk/k2safe/go-omni v0.19.0-preview require git.yuntuo.hk/k2safe/go-omni v0.21.0-preview
require ( require (
github.com/gabriel-vasile/mimetype v1.4.3 // indirect github.com/gabriel-vasile/mimetype v1.4.3 // indirect

View File

@@ -1,10 +1,9 @@
package config package config
import ( import (
"git.yuntuo.hk/k2safe/go-omni/config"
"git.yuntuo.hk/k2safe/go-omni/logx"
"os"
"time" "time"
"git.yuntuo.hk/k2safe/go-omni/config"
) )
type Config struct { type Config struct {
@@ -22,42 +21,10 @@ func Default() Config {
return Config{Server: server} return Config{Server: server}
} }
// Load applies defaults -> file -> typed environment -> validation. // Validate checks configuration before logs or dependencies open.
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
func Load(path string) (Config, error) {
cfg := Default()
var err error
cfg, err = config.ApplyDefaults(cfg)
if err != nil {
return Config{}, err
}
if path != "" {
cfg, err = config.DecodeFile(path, cfg)
if err != nil {
return Config{}, err
}
}
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{Prefix: "GO_OMNI_", Empty: "ignore", Aliases: config.ServerEnvAliases("GO_OMNI_", "server")})
if err != nil {
return Config{}, err
}
output, outputSet := os.LookupEnv("GO_OMNI_SERVER_LOG_OUTPUT")
if !outputSet {
output = os.Getenv("GO_OMNI_LOG_OUTPUT")
}
file, fileSet := os.LookupEnv("GO_OMNI_SERVER_LOG_FILE")
if !fileSet {
file = os.Getenv("GO_OMNI_LOG_FILE")
}
if output != "" && file == "" {
cfg.Server.Log.File = ""
cfg.Server.Log.Rotation = logx.RotationConfig{}
}
return cfg, cfg.Validate()
}
func (c Config) Validate() error { func (c Config) Validate() error {
if err := c.Server.Validate(); err != nil { if err := c.Server.Validate(); err != nil {
return err return config.AtField("server", err)
} }
return nil return nil

View File

@@ -0,0 +1,16 @@
// Code generated by go-omni api. DO NOT EDIT.
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// Load is maintained by the generator. Extend Config/Default/Validate in
// config.go, or NewLoader in loader.go; do not duplicate the loading pipeline.
func Load(path string) (Config, error) {
loader, err := NewLoader()
if err != nil {
return Config{}, err
}
return omniConfig.LoadProject(path, Default(), omniConfig.EnvOptions{
Prefix: "GO_OMNI_", Empty: "ignore", Aliases: omniConfig.ServerEnvAliases("GO_OMNI_", "server"),
}, loader)
}

View File

@@ -0,0 +1,7 @@
package config
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
// NewLoader is project-owned. JSON/YAML/XML work without changes. Register
// additional format decoders here only when the project needs them.
func NewLoader() (*omniConfig.Loader, error) { return omniConfig.NewLoader() }

153
httpc/README.md Normal file
View File

@@ -0,0 +1,153 @@
# httpc:复用连接的 HTTP 客户端
`httpc` 在标准 `net/http` 上提供独立连接池、网络阶段总超时、保守重试、请求追踪和有界 JSON 调用。普通接口可以直接带认证头调用;上传、下载、特殊协议仍使用标准 Request/Response。没有业务身份模型或全局认证信息。
## 创建一次,业务重复使用
```go
client, err := httpc.New(httpc.Config{
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
Timeout: 3 * time.Second,
DialTimeout: time.Second,
TLSHandshakeTimeout: time.Second,
ResponseHeaderTimeout: time.Second,
MaxConnsPerHost: 32,
MaxIdleConnsPerHost: 16,
MaxIdleConns: 64,
IdleConnTimeout: 60 * time.Second,
MaxResponseBytes: 1 << 20,
})
if err != nil { return err }
// 在 svc 初始化并持有 client,向生命周期登记 client.Close。
// 普通独立程序也可 defer client.Close();不要每次请求重新 New。
err = client.JSON(ctx, http.MethodGet, serviceURL, nil, &result,
httpc.WithHeader("Authorization", "Bearer "+token),
httpc.WithHeaders(http.Header{"Accept": {"application/json"}}),
)
```
Client 可并发使用。每个调用的 input/output 及其读写同步由调用方负责,不要让多个请求同时写同一个 output。
直接运行 [examples/httpClient](../examples/httpClient/README.md),只使用本地临时服务,无需业务 Token 或外部基础设施。
## 代理规则
| 配置 | 行为 |
| --- | --- |
| 不填写 Proxy | 保留被克隆 DefaultTransport 的代理策略;未改动的标准库默认使用环境代理 |
| `Mode: httpc.ProxyEnvironment` | 明确使用 HTTP_PROXY / HTTPS_PROXY / NO_PROXY(及小写形式) |
| `Mode: httpc.ProxyDirect` | 设置 Proxy=nil,明确直连,不受环境代理影响 |
| `Mode: httpc.ProxyURL, URL: "http://proxy.example:8080"` | 固定代理地址,不通过 NO_PROXY 排除 |
支持 http、https、socks5、socks5h;底层行为沿用 Go Transport。固定地址必须是代理 origin,不接受业务路径、查询、fragment 或无效端口。凭证可由 URL userinfo 提供,但配置错误不输出 URL/密码;不要打印整个 Config。环境代理规则由标准库读取并缓存,不提供运行时环境代理热更新。
内部服务要直连,只设置 `proxy.mode: direct`,无需创建 Transport。需要自定义路由、代理认证握手等复杂逻辑时注入自己的 Transport。
## 超时和连接池
除原有字段外,新字段都可选:**0 保留默认行为,正数覆盖,负数拒绝**。需要明确关闭某个阶段超时、关闭 TCP keepalive 或特殊 dialer 时,使用自定义 Transport。
| 字段 | 默认/含义 |
| --- | --- |
| Timeout | 10s;从 Do 开始,覆盖 DNS/建连、等待连接、重试、退避、响应头和读取响应体 |
| DialTimeout | 不填时保留原 dialer;标准库默认 30s;配置后使用标准 net.Dialer |
| DialKeepAlive | 标准 net.Dialer 的 TCP keepalive 探测间隔,标准默认 30s;不是 HTTP 空闲连接超时 |
| TLSHandshakeTimeout | 沿用克隆 Transport;标准库默认 10s |
| ResponseHeaderTimeout | 沿用 Transport,标准默认不单独限制;从完整发送请求后等响应头,总 Timeout 仍生效 |
| ExpectContinueTimeout | 沿用 Transport;标准默认 1s,等待 `100-continue` 的时间 |
| IdleConnTimeout | 沿用 Transport;标准默认 90s,只回收空闲连接 |
| MaxConnsPerHost | 默认 64;每个目标的建连中、活动与空闲连接总上限 |
| MaxIdleConnsPerHost | 默认等于 MaxConnsPerHost,可单独配置 |
| MaxIdleConns | 默认 MaxConnsPerHost×2,可单独配置;所有目标合计的空闲连接上限 |
| MaxResponseBytes | JSON 成功响应默认最多 1 MiB,最大允许配置 1 GiB;应按实际接口保持较小限额 |
每主机上限最多 65536,总空闲连接最多 131072。总空闲上限可以小于各主机上限之和。MaxConnsPerHost 不等于整个进程的连接上限,也不等于 HTTP/2 的请求并发上限。
配置 DialTimeout / DialKeepAlive 会创建标准 net.Dialer;未填的另一个值使用标准默认 30s。不要通过全局修改 DefaultTransport 来塞入项目的特殊拨号逻辑;请显式使用借用的 Transport。若 DefaultTransport 有自定义 TLS dialer,相关阶段覆盖会报错,避免 TLS timeout 被悄悄忽略。
调用者 context 的更早截止时间优先。总超时不能抢占 Go 的 JSON 编解码或不遵守 context 的自定义 RoundTripper;计时从 Do 开始,不包括调用前的 json.Marshal。TLS 校验默认保持开启,没有新增 InsecureSkipVerify 配置。
## JSON 的单次请求选项
```go
err := client.JSON(ctx, "POST", endpoint, input, &output,
httpc.WithHeader("Authorization", "Bearer "+token),
httpc.WithHeader("X-Tenant", tenant),
httpc.WithMaxResponseBytes(64<<10),
)
```
- `WithHeader` 设置一个头;`WithHeaders` 支持多值 http.Header。后出现的选项覆盖同名头,不影响其他头。
- WithHeaders 在创建选项时快照原 map,每次调用复制值。构建后修改原 map 不影响选项,选项可以并发复用;构建时原 map 的读写同步由调用方负责。
- 不提供 Client 全局 Authorization,避免跨用户调用共享凭证。框架不会把请求头写进日志。
- 有 input 时默认 Content-Type 为 application/json;显式请求头可覆盖。请求头名称/值会校验,拒绝 CRLF 等控制字符和歧义的大小写重复键。
- Host、Content-Length、Transfer-Encoding、Trailer 属于请求目标/分帧控制,请用标准 Request 配合 Do;JSON 选项中不允许伪装成普通头。
- `WithMaxResponseBytes` 是本次 JSON 的正数限额,不修改 Client 默认值,不改变 Do 的流式语义。
JSON 自动关闭响应体。仅 2xx 解码;空成功响应(例如 204)保留 output 原值。非空输入使用标准 json.Unmarshal,拒绝多个 JSON 文档,类型错误保留 Go 的错误信息与行为。
`output == nil` 按限额流式丢弃成功响应,不把整个 body 读进内存,也不检查 JSON 语法。响应限额按实际读取的 body 计算;标准库自动 gzip 解压后仍受上限约束。input 会先 json.Marshal 到内存,大文件上传不适合用 JSON helper。
非 2xx 返回 `*httpc.StatusError`,只包含状态码,不包含错误响应体。为连接复用,只尝试读完**声明长度已知且不超过 min(32 KiB, 本次响应限额)** 的小错误体,最多等待 25ms(或更早总截止时间);失败仍返回原 HTTP 状态。未知长度/大错误体立即关闭,优先及时返回。重试过程的错误体另受 32 KiB 大小及整体请求截止时间约束,不无界读取。
## 配置文件与生成项目
在项目 Config 中加一个字段:
```go
Outbound httpc.Config `json:"outbound"`
```
项目 Validate 中调用 `c.Outbound.Validate()`;创建时 New 也会验证。svc 初始化一次并通过生命周期登记 Close。文件例子:
```yaml
outbound:
timeout: 3s
proxy:
mode: direct
dialTimeout: 500ms
tlsHandshakeTimeout: 1s
responseHeaderTimeout: 1s
maxConnsPerHost: 32
maxIdleConnsPerHost: 16
maxIdleConns: 64
idleConnTimeout: 60s
maxResponseBytes: 1048576
```
现有统一加载入口可读取 JSON/YAML/XML,并自动覆盖 `GO_OMNI_OUTBOUND_PROXY_MODE`、`GO_OMNI_OUTBOUND_DIAL_TIMEOUT` 等环境变量。无需为每个参数手写环境解析。详见 [统一配置](../CONFIG.md)。
## 标准请求、特殊 TLS 与生命周期
```go
req, err := http.NewRequestWithContext(ctx, "GET", downloadURL, nil)
if err != nil { return err }
req.Header.Set("Authorization", "Bearer "+token)
res, err := client.Do(req)
if err != nil { return err }
defer res.Body.Close()
// 自行判断状态,按项目要求限流/限制大小,流式写入目标;不要无界 ReadAll 大文件。
_, err = io.Copy(destination, res.Body)
```
Do 不自动限制 body,允许下载/流式处理,调用方必须关闭。响应体消费/关闭前网络超时持续有效。
`Config.Transport` 可注入任意 http.RoundTripper,适配自定义 dialer、mTLS、根证书、监控包装或测试替身。框架借用它,**不修改、不调用其 CloseIdleConnections**。代理、连接池、TLS/建连阶段配置不能同时填入,避免无效配置;总 Timeout、JSON 限额、框架重试/日志仍有效。借用实现必须遵守标准请求和 body 所有权、并发及 context 约定。
`Close` 幂等:拒绝新调用,关闭自有池的空闲连接,不主动打断已开始的请求。服务关闭应先排空 HTTP/后台调用,再关闭 Client;它不是后台 drain 的替代品。生成项目可通过 svc 的生命周期登记清理。
## 重试、日志与兼容
MaxRetries 默认 0,最多 5。仅可重放的 GET/HEAD/OPTIONS 对网络错误、429/502/503/504 做框架重试。POST/PUT/PATCH/DELETE 不自动做应用层重试。标准库或自定义 Transport 自己的重放规则仍可能生效,不能承诺业务“恰好执行一次”。
Backoff 默认 100ms,MaxBackoff 默认 2s,采用指数退避与抖动。Retry-After 要求等待超过 MaxBackoff 时返回当前响应,不提前请求;总截止时间限制全部尝试。成功或无重试的调用不计算退避。
不跟随重定向,避免自动向其他地址转发认证信息;3xx 由调用者判断。默认传递 context 中的 X-Request-ID,显式请求头优先。日志只包含 method、hostname、status、attempt、收到响应头的耗时等,不打印路径、查询、用户名密码、认证头或响应体;标准 net/http 返回的 URL 错误可能携带地址,项目不要把 URL 中的密钥当作安全日志字段。
v0.21 兼容说明:
- 原来的 JSON 五参数直接调用继续有效;新方法末尾增加 `...httpc.RequestOption`,自行定义的 interface/方法值类型需要同步签名。
- 原默认代理、总超时、重试、池大小和成功响应含义保留;新字段未设置时不改变它们。
- `Transport` 与显式池/代理/阶段参数混用现在报错;此前 MaxConnsPerHost 在自定义 Transport 下实际被忽略。迁移时把参数放到自己的 Transport,或删掉 Transport 改用内置配置。
- 没有移除 Do、JSON、StatusError 或自定义 Transport,没有引入新第三方依赖。

100
httpc/benchmark_test.go Normal file
View File

@@ -0,0 +1,100 @@
package httpc
import (
"context"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
type benchmarkTransport struct {
payload string
status int
}
func (t benchmarkTransport) RoundTrip(r *http.Request) (*http.Response, error) {
if r.Body != nil {
_ = r.Body.Close()
}
return &http.Response{StatusCode: t.status, Header: make(http.Header), Body: io.NopCloser(strings.NewReader(t.payload)), ContentLength: int64(len(t.payload)), Request: r}, nil
}
func BenchmarkHTTPClient(b *testing.B) {
for _, size := range []int{128, 4096, 65536} {
for _, discard := range []bool{false, true} {
label := map[int]string{128: "128B", 4096: "4KiB", 65536: "64KiB"}[size]
if discard {
label += "/discard"
} else {
label += "/decode"
}
b.Run(label, func(b *testing.B) {
c, err := New(Config{Transport: benchmarkTransport{`{"value":"` + strings.Repeat("x", size) + `"}`, 200}, DisableLogging: true})
if err != nil {
b.Fatal(err)
}
defer c.Close()
var output struct {
Value string `json:"value"`
}
var target any = &output
if discard {
target = nil
}
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := c.JSON(context.Background(), "GET", "http://example.invalid/item", nil, target); err != nil {
b.Fatal(err)
}
}
})
}
}
b.Run("Do", func(b *testing.B) {
c, _ := New(Config{Transport: benchmarkTransport{"", 200}, DisableLogging: true})
defer c.Close()
r, _ := http.NewRequest("GET", "http://example.invalid/item", nil)
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
res, err := c.Do(r)
if err != nil {
b.Fatal(err)
}
res.Body.Close()
}
})
}
func BenchmarkHTTPPool(b *testing.B) {
for _, status := range []int{200, 401} {
b.Run(http.StatusText(status), func(b *testing.B) {
s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { w.WriteHeader(status); io.WriteString(w, `{"ok":true}`) }))
defer s.Close()
c, _ := New(Config{DisableLogging: true})
defer c.Close()
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
err := c.JSON(context.Background(), "GET", s.URL, nil, nil)
if status == 200 && err != nil {
b.Fatal(err)
}
}
})
}
}
func BenchmarkHTTPHeaders(b *testing.B) {
c, _ := New(Config{Transport: benchmarkTransport{`{"ok":true}`, 200}, DisableLogging: true})
defer c.Close()
option := WithHeaders(http.Header{"Authorization": []string{"Bearer example"}, "X-Request-Id": []string{"example-request"}})
b.ReportAllocs()
b.ResetTimer()
for i := 0; i < b.N; i++ {
if err := c.JSON(context.Background(), "GET", "http://example.invalid", nil, nil, option); err != nil {
b.Fatal(err)
}
}
}

View File

@@ -3,9 +3,7 @@
package httpc package httpc
import ( import (
"bytes"
"context" "context"
"encoding/json"
"errors" "errors"
"fmt" "fmt"
"io" "io"
@@ -22,18 +20,6 @@ import (
var ErrClosed = errors.New("httpc: closed") var ErrClosed = errors.New("httpc: closed")
var ErrResponseTooLarge = errors.New("httpc: response exceeds limit") var ErrResponseTooLarge = errors.New("httpc: response exceeds limit")
type Config struct {
Timeout time.Duration `json:"timeout"`
MaxRetries int `json:"maxRetries"`
Backoff time.Duration `json:"backoff"`
MaxBackoff time.Duration `json:"maxBackoff"`
MaxConnsPerHost int `json:"maxConnsPerHost"`
MaxResponseBytes int64 `json:"maxResponseBytes"`
DisableRequestID bool `json:"disableRequestId"`
DisableLogging bool `json:"disableLogging"`
Transport http.RoundTripper `json:"-"` // Borrowed. The caller owns its lifecycle.
}
type Client struct { type Client struct {
config Config config Config
client *http.Client client *http.Client
@@ -41,43 +27,19 @@ type Client struct {
closed atomic.Bool closed atomic.Bool
} }
// New creates a reusable client. Close it after callers and response streams drain.
func New(cfg Config) (*Client, error) { func New(cfg Config) (*Client, error) {
if cfg.Timeout < 0 || cfg.Backoff < 0 || cfg.MaxBackoff < 0 || cfg.MaxRetries < 0 || cfg.MaxRetries > 5 || cfg.MaxConnsPerHost < 0 || cfg.MaxResponseBytes < 0 { normalized, err := normalizeConfig(cfg)
return nil, errors.New("httpc: invalid configuration") if err != nil {
return nil, err
} }
if cfg.Timeout == 0 { c := &Client{config: normalized}
cfg.Timeout = 10 * time.Second
}
if cfg.Backoff == 0 {
cfg.Backoff = 100 * time.Millisecond
}
if cfg.MaxBackoff == 0 {
cfg.MaxBackoff = 2 * time.Second
}
if cfg.Backoff > cfg.MaxBackoff {
return nil, errors.New("httpc: backoff exceeds maxBackoff")
}
if cfg.MaxConnsPerHost == 0 {
cfg.MaxConnsPerHost = 64
}
if cfg.MaxResponseBytes == 0 {
cfg.MaxResponseBytes = 1 << 20
}
if cfg.MaxResponseBytes > 1<<30 || cfg.MaxConnsPerHost > 65536 {
return nil, errors.New("httpc: response or connection limit exceeds supported capacity")
}
c := &Client{config: cfg}
transport := cfg.Transport transport := cfg.Transport
if transport == nil { if transport == nil {
// Own a private pool; never mutate the process-wide DefaultTransport. c.owned, err = ownedTransport(normalized)
base, ok := http.DefaultTransport.(*http.Transport) if err != nil {
if !ok { return nil, err
return nil, errors.New("httpc: provide Transport when DefaultTransport is replaced")
} }
c.owned = base.Clone()
c.owned.MaxConnsPerHost = cfg.MaxConnsPerHost
c.owned.MaxIdleConnsPerHost = cfg.MaxConnsPerHost
c.owned.MaxIdleConns = cfg.MaxConnsPerHost * 2
transport = c.owned transport = c.owned
} }
c.client = &http.Client{Transport: transport, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }} c.client = &http.Client{Transport: transport, CheckRedirect: func(*http.Request, []*http.Request) error { return http.ErrUseLastResponse }}
@@ -114,7 +76,10 @@ func (c *Client) Do(request *http.Request) (*http.Response, error) {
} }
replayable := (base.Method == "" || base.Method == "GET" || base.Method == "HEAD" || base.Method == "OPTIONS") && (base.Body == nil || base.Body == http.NoBody || base.GetBody != nil) replayable := (base.Method == "" || base.Method == "GET" || base.Method == "HEAD" || base.Method == "OPTIONS") && (base.Body == nil || base.Body == http.NoBody || base.GetBody != nil)
for attempt := 0; ; attempt++ { for attempt := 0; ; attempt++ {
req := base.Clone(ctx) req := base
if attempt > 0 {
req = base.Clone(ctx)
}
if attempt > 0 && base.Body != nil && base.Body != http.NoBody { if attempt > 0 && base.Body != nil && base.Body != http.NoBody {
body, err := base.GetBody() body, err := base.GetBody()
if err != nil { if err != nil {
@@ -133,23 +98,9 @@ func (c *Client) Do(request *http.Request) (*http.Response, error) {
logx.InfoContext(ctx, "http.client", "method", req.Method, "host", req.URL.Hostname(), "status", status, "attempt", attempt+1, "durationMs", time.Since(started).Seconds()*1000, "failed", err != nil) logx.InfoContext(ctx, "http.client", "method", req.Method, "host", req.URL.Hostname(), "status", status, "attempt", attempt+1, "durationMs", time.Since(started).Seconds()*1000, "failed", err != nil)
} }
retry := replayable && attempt < c.config.MaxRetries && ctx.Err() == nil && (err != nil || status == 429 || status == 502 || status == 503 || status == 504) retry := replayable && attempt < c.config.MaxRetries && ctx.Err() == nil && (err != nil || status == 429 || status == 502 || status == 503 || status == 504)
delay := c.config.Backoff var delay time.Duration
for i := 0; i < attempt && delay < c.config.MaxBackoff; i++ { if retry {
if delay > c.config.MaxBackoff/2 { delay, retry = c.retryDelay(attempt, response)
delay = c.config.MaxBackoff
} else {
delay *= 2
}
}
if response != nil && response.Header.Get("Retry-After") != "" {
if value, ok := retryAfter(response.Header.Get("Retry-After")); ok {
delay = value
}
} else if delay > 1 {
delay = delay/2 + time.Duration(rand.Int63n(int64(delay/2)+1))
}
if delay > c.config.MaxBackoff {
retry = false
} }
if !retry { if !retry {
if err != nil { if err != nil {
@@ -160,9 +111,7 @@ func (c *Client) Do(request *http.Request) (*http.Response, error) {
return response, nil return response, nil
} }
if response != nil { if response != nil {
if response.ContentLength >= 0 && response.ContentLength <= 32<<10 { drainSmallBody(response, 32<<10)
_, _ = io.Copy(io.Discard, response.Body)
}
_ = response.Body.Close() _ = response.Body.Close()
} }
timer := time.NewTimer(delay) timer := time.NewTimer(delay)
@@ -204,48 +153,6 @@ type StatusError struct{ StatusCode int }
func (e *StatusError) Error() string { return fmt.Sprintf("httpc: HTTP status %d", e.StatusCode) } func (e *StatusError) Error() string { return fmt.Sprintf("httpc: HTTP status %d", e.StatusCode) }
// JSON bounds response memory, closes the body, and decodes only 2xx responses.
// Error bodies, URLs, credentials and query strings are not included in logs.
func (c *Client) JSON(ctx context.Context, method, url string, input, output any) error {
if ctx == nil {
return errors.New("httpc: nil context")
}
var body io.Reader
if input != nil {
data, err := json.Marshal(input)
if err != nil {
return err
}
body = bytes.NewReader(data)
}
request, err := http.NewRequestWithContext(ctx, method, url, body)
if err != nil {
return err
}
if input != nil {
request.Header.Set("Content-Type", "application/json")
}
response, err := c.Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode < 200 || response.StatusCode >= 300 {
return &StatusError{StatusCode: response.StatusCode}
}
data, err := io.ReadAll(io.LimitReader(response.Body, c.config.MaxResponseBytes+1))
if err != nil {
return err
}
if int64(len(data)) > c.config.MaxResponseBytes {
return ErrResponseTooLarge
}
if output == nil || len(bytes.TrimSpace(data)) == 0 {
return nil
}
return json.Unmarshal(data, output)
}
func (c *Client) Close() error { func (c *Client) Close() error {
c.closed.Store(true) c.closed.Store(true)
if c.owned != nil { if c.owned != nil {
@@ -253,3 +160,34 @@ func (c *Client) Close() error {
} }
return nil return nil
} }
// retryDelay is deliberately outside the success path: no jitter/random work
// for calls that cannot or do not need to retry.
func (c *Client) retryDelay(attempt int, response *http.Response) (time.Duration, bool) {
delay := c.config.Backoff
for i := 0; i < attempt && delay < c.config.MaxBackoff; i++ {
if delay > c.config.MaxBackoff/2 {
delay = c.config.MaxBackoff
} else {
delay *= 2
}
}
if response != nil {
if value, ok := retryAfter(response.Header.Get("Retry-After")); ok {
return value, value <= c.config.MaxBackoff
}
}
if delay > 1 {
delay = delay/2 + time.Duration(rand.Int63n(int64(delay/2)+1))
}
return delay, delay <= c.config.MaxBackoff
}
// Drain only a known small error body. Unknown/large bodies are closed promptly;
// never allocate or consume an unbounded error document just to retain a pool slot.
func drainSmallBody(response *http.Response, limit int64) {
limit = min(limit, 32<<10)
if response.Body != nil && response.ContentLength >= 0 && response.ContentLength <= limit {
_, _ = io.Copy(io.Discard, io.LimitReader(response.Body, limit+1))
}
}

205
httpc/config.go Normal file
View File

@@ -0,0 +1,205 @@
package httpc
import (
"errors"
"fmt"
"net"
"net/http"
"net/url"
"reflect"
"strconv"
"strings"
"time"
)
// ProxyMode controls the owned transport's proxy selection. Empty preserves
// DefaultTransport's policy (normally environment proxies).
type ProxyMode string
const (
ProxyEnvironment ProxyMode = "environment"
ProxyDirect ProxyMode = "direct"
ProxyURL ProxyMode = "url"
)
type ProxyConfig struct {
Mode ProxyMode `json:"mode"`
URL string `json:"url"` // May contain credentials; never log the configuration.
}
// Config owns one private connection pool unless Transport is supplied.
// New duration/pool fields are optional: zero retains existing defaults, positive
// values override them. Use a custom Transport for disabling timeouts, custom
// dialers, mTLS and other protocol-specific settings.
type Config struct {
Timeout time.Duration `json:"timeout"`
MaxRetries int `json:"maxRetries"`
Backoff time.Duration `json:"backoff"`
MaxBackoff time.Duration `json:"maxBackoff"`
MaxConnsPerHost int `json:"maxConnsPerHost"`
MaxIdleConns int `json:"maxIdleConns"`
MaxIdleConnsPerHost int `json:"maxIdleConnsPerHost"`
DialTimeout time.Duration `json:"dialTimeout"`
DialKeepAlive time.Duration `json:"dialKeepAlive"`
TLSHandshakeTimeout time.Duration `json:"tlsHandshakeTimeout"`
ResponseHeaderTimeout time.Duration `json:"responseHeaderTimeout"`
ExpectContinueTimeout time.Duration `json:"expectContinueTimeout"`
IdleConnTimeout time.Duration `json:"idleConnTimeout"`
Proxy ProxyConfig `json:"proxy"`
MaxResponseBytes int64 `json:"maxResponseBytes"`
DisableRequestID bool `json:"disableRequestId"`
DisableLogging bool `json:"disableLogging"`
Transport http.RoundTripper `json:"-"` // Borrowed; never modified or closed.
}
const maxResponseBytes int64 = 1 << 30
// Validate checks values without creating pools or making network requests.
func (cfg Config) Validate() error { _, err := normalizeConfig(cfg); return err }
func normalizeConfig(cfg Config) (Config, error) {
for _, field := range []struct {
name string
value time.Duration
}{
{"timeout", cfg.Timeout}, {"backoff", cfg.Backoff}, {"maxBackoff", cfg.MaxBackoff},
{"dialTimeout", cfg.DialTimeout}, {"dialKeepAlive", cfg.DialKeepAlive}, {"tlsHandshakeTimeout", cfg.TLSHandshakeTimeout},
{"responseHeaderTimeout", cfg.ResponseHeaderTimeout}, {"expectContinueTimeout", cfg.ExpectContinueTimeout}, {"idleConnTimeout", cfg.IdleConnTimeout},
} {
if field.value < 0 {
return Config{}, fmt.Errorf("httpc: %s must not be negative", field.name)
}
}
if cfg.MaxRetries < 0 || cfg.MaxRetries > 5 {
return Config{}, errors.New("httpc: maxRetries must be 0..5")
}
if cfg.MaxConnsPerHost < 0 || cfg.MaxConnsPerHost > 65536 || cfg.MaxIdleConnsPerHost < 0 || cfg.MaxIdleConnsPerHost > 65536 || cfg.MaxIdleConns < 0 || cfg.MaxIdleConns > 131072 {
return Config{}, errors.New("httpc: invalid connection pool limits")
}
if cfg.MaxResponseBytes < 0 || cfg.MaxResponseBytes > maxResponseBytes {
return Config{}, errors.New("httpc: maxResponseBytes must be 0..1 GiB")
}
if _, err := proxyFunction(cfg.Proxy); err != nil {
return Config{}, err
}
if cfg.Transport != nil {
value := reflect.ValueOf(cfg.Transport)
switch value.Kind() {
case reflect.Pointer, reflect.Map, reflect.Func, reflect.Slice, reflect.Interface, reflect.Chan:
if value.IsNil() {
return Config{}, errors.New("httpc: nil Transport")
}
}
if cfg.hasTransportOptions() {
return Config{}, errors.New("httpc: Transport cannot be combined with proxy, dial, TLS or connection pool options; configure the borrowed transport directly")
}
}
if cfg.Timeout == 0 {
cfg.Timeout = 10 * time.Second
}
if cfg.Backoff == 0 {
cfg.Backoff = 100 * time.Millisecond
}
if cfg.MaxBackoff == 0 {
cfg.MaxBackoff = 2 * time.Second
}
if cfg.Backoff > cfg.MaxBackoff {
return Config{}, errors.New("httpc: backoff exceeds maxBackoff")
}
if cfg.MaxConnsPerHost == 0 {
cfg.MaxConnsPerHost = 64
}
if cfg.MaxIdleConnsPerHost == 0 {
cfg.MaxIdleConnsPerHost = cfg.MaxConnsPerHost
}
if cfg.MaxIdleConns == 0 {
cfg.MaxIdleConns = cfg.MaxConnsPerHost * 2
}
if cfg.MaxResponseBytes == 0 {
cfg.MaxResponseBytes = 1 << 20
}
return cfg, nil
}
func (c Config) hasTransportOptions() bool {
return c.Proxy.Mode != "" || c.Proxy.URL != "" || c.MaxConnsPerHost != 0 || c.MaxIdleConns != 0 || c.MaxIdleConnsPerHost != 0 || c.DialTimeout != 0 || c.DialKeepAlive != 0 || c.TLSHandshakeTimeout != 0 || c.ResponseHeaderTimeout != 0 || c.ExpectContinueTimeout != 0 || c.IdleConnTimeout != 0
}
func proxyFunction(c ProxyConfig) (func(*http.Request) (*url.URL, error), error) {
if c.Mode != ProxyURL && c.URL != "" {
return nil, errors.New("httpc: proxy URL requires mode url")
}
switch c.Mode {
case "":
return nil, nil
case ProxyDirect:
return nil, nil
case ProxyEnvironment:
return http.ProxyFromEnvironment, nil
case ProxyURL:
u, err := url.Parse(c.URL)
if err != nil || u == nil || u.Hostname() == "" || u.Opaque != "" || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || (u.Path != "" && u.Path != "/") {
return nil, errors.New("httpc: proxy URL must be an absolute proxy origin without path, query or fragment")
}
switch u.Scheme {
case "http", "https", "socks5", "socks5h":
default:
return nil, errors.New("httpc: proxy scheme must be http, https, socks5 or socks5h")
}
if strings.HasSuffix(u.Host, ":") {
return nil, errors.New("httpc: proxy port must be 1..65535")
}
if port := u.Port(); port != "" {
n, e := strconv.Atoi(port)
if e != nil || n < 1 || n > 65535 {
return nil, errors.New("httpc: proxy port must be 1..65535")
}
}
u.Path = ""
u.RawPath = ""
return http.ProxyURL(u), nil
default:
return nil, errors.New("httpc: proxy mode must be environment, direct or url")
}
}
func ownedTransport(cfg Config) (*http.Transport, error) {
base, ok := http.DefaultTransport.(*http.Transport)
if !ok || base == nil {
return nil, errors.New("httpc: provide Transport when DefaultTransport is replaced")
}
if (base.DialTLSContext != nil || base.DialTLS != nil) && (cfg.DialTimeout > 0 || cfg.DialKeepAlive > 0 || cfg.TLSHandshakeTimeout > 0) {
return nil, errors.New("httpc: provide Transport when DefaultTransport has a custom TLS dialer")
}
t := base.Clone()
t.MaxConnsPerHost = cfg.MaxConnsPerHost
t.MaxIdleConnsPerHost = cfg.MaxIdleConnsPerHost
t.MaxIdleConns = cfg.MaxIdleConns
if cfg.Proxy.Mode != "" {
proxy, err := proxyFunction(cfg.Proxy)
if err != nil {
return nil, err
}
t.Proxy = proxy
}
if cfg.DialTimeout > 0 || cfg.DialKeepAlive > 0 {
dialer := &net.Dialer{Timeout: 30 * time.Second, KeepAlive: 30 * time.Second}
if cfg.DialTimeout > 0 {
dialer.Timeout = cfg.DialTimeout
}
if cfg.DialKeepAlive > 0 {
dialer.KeepAlive = cfg.DialKeepAlive
}
t.DialContext = dialer.DialContext
t.Dial = nil
}
if cfg.TLSHandshakeTimeout > 0 {
t.TLSHandshakeTimeout = cfg.TLSHandshakeTimeout
}
if cfg.ResponseHeaderTimeout > 0 {
t.ResponseHeaderTimeout = cfg.ResponseHeaderTimeout
}
if cfg.ExpectContinueTimeout > 0 {
t.ExpectContinueTimeout = cfg.ExpectContinueTimeout
}
if cfg.IdleConnTimeout > 0 {
t.IdleConnTimeout = cfg.IdleConnTimeout
}
return t, nil
}

277
httpc/config_test.go Normal file
View File

@@ -0,0 +1,277 @@
package httpc
import (
"context"
"errors"
"io"
"net"
"net/http"
"net/http/httptest"
"net/url"
"os"
"os/exec"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
func testClient(t *testing.T, cfg Config) *Client {
t.Helper()
cfg.DisableLogging = true
c, err := New(cfg)
if err != nil {
t.Fatal(err)
}
t.Cleanup(func() { c.Close() })
return c
}
func TestTransportConfigurationAndIsolation(t *testing.T) {
base := http.DefaultTransport.(*http.Transport)
oldConns, oldIdle, oldTimeout := base.MaxConnsPerHost, base.MaxIdleConns, base.ResponseHeaderTimeout
cfg := Config{Proxy: ProxyConfig{Mode: ProxyDirect}, DialTimeout: time.Second, DialKeepAlive: 20 * time.Second, TLSHandshakeTimeout: 2 * time.Second, ResponseHeaderTimeout: 3 * time.Second, ExpectContinueTimeout: 4 * time.Second, IdleConnTimeout: 5 * time.Second, MaxConnsPerHost: 8, MaxIdleConns: 12, MaxIdleConnsPerHost: 3}
c := testClient(t, cfg)
if c.owned == base || c.owned.Proxy != nil || c.owned.DialContext == nil || c.owned.MaxConnsPerHost != 8 || c.owned.MaxIdleConns != 12 || c.owned.MaxIdleConnsPerHost != 3 || c.owned.TLSHandshakeTimeout != 2*time.Second || c.owned.ResponseHeaderTimeout != 3*time.Second || c.owned.ExpectContinueTimeout != 4*time.Second || c.owned.IdleConnTimeout != 5*time.Second {
t.Fatal("transport options not applied")
}
if base.MaxConnsPerHost != oldConns || base.MaxIdleConns != oldIdle || base.ResponseHeaderTimeout != oldTimeout {
t.Fatal("modified shared transport")
}
if c.owned.TLSClientConfig != nil && c.owned.TLSClientConfig.InsecureSkipVerify {
t.Fatal("TLS verification disabled")
}
d := testClient(t, Config{})
if d.owned.MaxConnsPerHost != 64 || d.owned.MaxIdleConnsPerHost != 64 || d.owned.MaxIdleConns != 128 || d.owned.IdleConnTimeout != base.IdleConnTimeout || d.owned.TLSHandshakeTimeout != base.TLSHandshakeTimeout || d.owned.ResponseHeaderTimeout != base.ResponseHeaderTimeout {
t.Fatal("default behavior changed")
}
ctx, cancel := context.WithCancel(context.Background())
cancel()
if conn, err := c.owned.DialContext(ctx, "tcp", "127.0.0.1:1"); err == nil {
conn.Close()
t.Fatal("dial ignored context")
}
}
func TestProxyValidation(t *testing.T) {
for _, scheme := range []string{"http", "https", "socks5", "socks5h"} {
c := testClient(t, Config{Proxy: ProxyConfig{Mode: ProxyURL, URL: scheme + "://user:private-value@proxy.invalid:8080"}})
req, _ := http.NewRequest("GET", "https://target.invalid", nil)
u, err := c.owned.Proxy(req)
if err != nil || u.Host != "proxy.invalid:8080" || u.Scheme != scheme {
t.Fatal("explicit proxy unavailable")
}
}
bad := []Config{
{Proxy: ProxyConfig{Mode: "private-value"}},
{Proxy: ProxyConfig{URL: "http://private-value"}},
{Proxy: ProxyConfig{Mode: ProxyDirect, URL: "http://private-value"}},
{Proxy: ProxyConfig{Mode: ProxyURL, URL: "http://user:private-value@proxy.invalid:bad"}},
{Proxy: ProxyConfig{Mode: ProxyURL, URL: "file:///private-value"}},
{Proxy: ProxyConfig{Mode: ProxyURL, URL: "http://proxy.invalid/private-value"}},
{Proxy: ProxyConfig{Mode: ProxyURL, URL: "http://proxy.invalid?token=private-value"}},
{Proxy: ProxyConfig{Mode: ProxyURL, URL: "http://proxy.invalid:65536"}},
{DialTimeout: -1}, {DialKeepAlive: -1}, {TLSHandshakeTimeout: -1}, {ResponseHeaderTimeout: -1}, {ExpectContinueTimeout: -1}, {IdleConnTimeout: -1}, {MaxIdleConns: -1}, {MaxIdleConnsPerHost: 65537},
{Transport: benchmarkTransport{}, Proxy: ProxyConfig{Mode: ProxyDirect}},
{Transport: benchmarkTransport{}, DialTimeout: time.Second}, {Transport: benchmarkTransport{}, MaxConnsPerHost: 2},
}
for i, cfg := range bad {
if _, err := New(cfg); err == nil || strings.Contains(err.Error(), "private-value") {
t.Fatalf("case %d: unsafe or missing error %v", i, err)
}
}
var nilTransport *http.Transport
if _, err := New(Config{Transport: nilTransport}); err == nil {
t.Fatal("typed nil accepted")
}
}
func TestProxyRouting(t *testing.T) {
var proxied, target atomic.Int32
proxy := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
proxied.Add(1)
if r.URL.Scheme != "http" || r.URL.Host != "target.invalid" {
t.Error("not an absolute proxy request")
}
io.WriteString(w, `{"proxy":true}`)
}))
defer proxy.Close()
origin := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { target.Add(1); io.WriteString(w, `{}`) }))
defer origin.Close()
// Isolate the standard library's process-cached environment proxy lookup.
executable, err := os.Executable()
if err != nil {
t.Fatal(err)
}
ctx, cancel := context.WithTimeout(context.Background(), 15*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, executable, "-test.run=^TestProxyEnvironmentProcess$")
for _, value := range os.Environ() {
key := strings.SplitN(value, "=", 2)[0]
switch strings.ToUpper(key) {
case "HTTP_PROXY", "HTTPS_PROXY", "NO_PROXY", "ALL_PROXY", "GO_OMNI_HTTPC_PROXY_CHILD", "GO_OMNI_HTTPC_TARGET":
continue
}
cmd.Env = append(cmd.Env, value)
}
cmd.Env = append(cmd.Env, "GO_OMNI_HTTPC_PROXY_CHILD=1", "GO_OMNI_HTTPC_TARGET="+origin.URL, "HTTP_PROXY="+proxy.URL, "HTTPS_PROXY="+proxy.URL, "NO_PROXY=bypass.invalid")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatal(err, string(out))
}
if proxied.Load() != 3 || target.Load() != 1 {
t.Fatalf("proxy=%d direct=%d", proxied.Load(), target.Load())
}
}
func TestProxyEnvironmentProcess(t *testing.T) {
if os.Getenv("GO_OMNI_HTTPC_PROXY_CHILD") != "1" {
t.Skip("subprocess only")
}
for _, proxy := range []ProxyConfig{{}, {Mode: ProxyEnvironment}, {Mode: ProxyURL, URL: os.Getenv("HTTP_PROXY")}} {
c := testClient(t, Config{Proxy: proxy})
if err := c.JSON(context.Background(), "GET", "http://target.invalid/path", nil, nil); err != nil {
t.Fatal(err)
}
}
environment := testClient(t, Config{Proxy: ProxyConfig{Mode: ProxyEnvironment}})
req, _ := http.NewRequest("GET", "http://bypass.invalid", nil)
u, err := environment.owned.Proxy(req)
if err != nil || u != nil {
t.Fatal("NO_PROXY ignored")
}
direct := testClient(t, Config{Proxy: ProxyConfig{Mode: ProxyDirect}})
if direct.owned.Proxy != nil {
t.Fatal("direct still has proxy")
}
if err := direct.JSON(context.Background(), "GET", os.Getenv("GO_OMNI_HTTPC_TARGET"), nil, nil); err != nil {
t.Fatal(err)
}
}
func TestResponseHeaderTimeoutRecovery(t *testing.T) {
s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path == "/slow" {
<-r.Context().Done()
return
}
io.WriteString(w, `{}`)
}))
defer s.Close()
c := testClient(t, Config{Timeout: 2 * time.Second, ResponseHeaderTimeout: 30 * time.Millisecond})
start := time.Now()
err := c.JSON(context.Background(), "GET", s.URL+"/slow", nil, nil)
var network net.Error
if !errors.As(err, &network) || !network.Timeout() || time.Since(start) > time.Second {
t.Fatal("header timeout", err)
}
if err = c.JSON(context.Background(), "GET", s.URL+"/ok", nil, nil); err != nil {
t.Fatal("did not recover", err)
}
}
func TestTLSHandshakeTimeout(t *testing.T) {
listener, err := net.Listen("tcp", "127.0.0.1:0")
if err != nil {
t.Fatal(err)
}
defer listener.Close()
done := make(chan struct{})
go func() {
defer close(done)
conn, err := listener.Accept()
if err != nil {
return
}
defer conn.Close()
_, _ = io.Copy(io.Discard, conn)
}()
c := testClient(t, Config{Timeout: 2 * time.Second, Proxy: ProxyConfig{Mode: ProxyDirect}, TLSHandshakeTimeout: 30 * time.Millisecond})
start := time.Now()
err = c.JSON(context.Background(), "GET", "https://"+listener.Addr().String(), nil, nil)
var network net.Error
if !errors.As(err, &network) || !network.Timeout() || time.Since(start) > time.Second {
t.Fatal("handshake timeout", err)
}
select {
case <-done:
case <-time.After(time.Second):
t.Fatal("TLS connection leaked")
}
}
func TestPoolLimitsAndIdleExpiry(t *testing.T) {
var connections atomic.Int32
entered := make(chan struct{}, 12)
release := make(chan struct{})
closed := make(chan struct{}, 12)
s := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
entered <- struct{}{}
select {
case <-release:
case <-r.Context().Done():
return
}
io.WriteString(w, `{}`)
}))
s.Config.ConnState = func(_ net.Conn, state http.ConnState) {
if state == http.StateNew {
connections.Add(1)
}
if state == http.StateClosed {
closed <- struct{}{}
}
}
s.Start()
defer s.Close()
c := testClient(t, Config{Timeout: 2 * time.Second, MaxConnsPerHost: 2, MaxIdleConnsPerHost: 2, MaxIdleConns: 2, IdleConnTimeout: 40 * time.Millisecond})
var wg sync.WaitGroup
for i := 0; i < 12; i++ {
wg.Add(1)
go func() {
defer wg.Done()
if err := c.JSON(context.Background(), "GET", s.URL, nil, nil); err != nil {
t.Error(err)
}
}()
}
for i := 0; i < 2; i++ {
select {
case <-entered:
case <-time.After(time.Second):
close(release)
t.Fatal("requests did not start")
}
}
if connections.Load() != 2 {
t.Error("connection cap", connections.Load())
}
close(release)
wg.Wait()
if connections.Load() != 2 {
t.Fatal("pool was not reused", connections.Load())
}
for i := 0; i < 2; i++ {
select {
case <-closed:
case <-time.After(time.Second):
t.Fatal("idle pool was not reclaimed")
}
}
if err := c.JSON(context.Background(), "GET", s.URL, nil, nil); err != nil {
t.Fatal(err)
}
if connections.Load() != 3 {
t.Fatal("new connection not established after idle timeout")
}
c.Close()
select {
case <-closed:
case <-time.After(time.Second):
t.Fatal("Close left idle connections open")
}
}
func TestHTTPSVerificationRetained(t *testing.T) {
s := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { io.WriteString(w, `{}`) }))
defer s.Close()
c := testClient(t, Config{Proxy: ProxyConfig{Mode: ProxyDirect}})
err := c.JSON(context.Background(), "GET", s.URL, nil, nil)
var urlError *url.Error
if !errors.As(err, &urlError) {
t.Fatal("self-signed server unexpectedly trusted", err)
}
}

119
httpc/json.go Normal file
View File

@@ -0,0 +1,119 @@
package httpc
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net/http"
"time"
)
// JSON bounds response memory, closes the body, and decodes only 2xx responses.
// Optional headers/limits belong to this call and never alter the client.
// output=nil discards a bounded stream; it does not buffer or validate JSON.
// Empty successful bodies leave output unchanged. StatusError contains no body.
func (c *Client) JSON(ctx context.Context, method, url string, input, output any, options ...RequestOption) error {
if ctx == nil {
return errors.New("httpc: nil context")
}
if c.closed.Load() {
return ErrClosed
}
if err := ctx.Err(); err != nil {
return err
}
settings := requestOptions{maxResponseBytes: c.config.MaxResponseBytes}
if len(options) > 0 {
var err error
settings, err = jsonOptions(c.config.MaxResponseBytes, options)
if err != nil {
return err
}
}
var body io.Reader
if input != nil {
data, err := json.Marshal(input)
if err != nil {
return err
}
body = bytes.NewReader(data)
}
request, err := http.NewRequestWithContext(ctx, method, url, body)
if err != nil {
return err
}
if settings.headers != nil {
request.Header = settings.headers
}
if input != nil {
if _, exists := request.Header["Content-Type"]; !exists {
request.Header.Set("Content-Type", "application/json")
}
}
response, err := c.Do(request)
if err != nil {
return err
}
defer response.Body.Close()
if response.StatusCode < 200 || response.StatusCode >= 300 {
drainStatusBody(response, settings.maxResponseBytes)
return &StatusError{StatusCode: response.StatusCode}
}
if response.ContentLength > settings.maxResponseBytes {
return ErrResponseTooLarge
}
limited := io.LimitReader(response.Body, settings.maxResponseBytes+1)
if output == nil {
count, err := io.Copy(io.Discard, limited)
if err != nil {
return err
}
if count > settings.maxResponseBytes {
return ErrResponseTooLarge
}
return nil
}
data, err := io.ReadAll(limited)
if err != nil {
return err
}
if int64(len(data)) > settings.maxResponseBytes {
return ErrResponseTooLarge
}
if len(bytes.TrimSpace(data)) == 0 {
return nil
}
return json.Unmarshal(data, output)
}
// A known small status body may still be slow. Spend at most a short grace
// period trying to retain the connection, then cancel the HTTP read. The status
// remains authoritative and the deferred Close still owns body cleanup.
func drainStatusBody(response *http.Response, limit int64) {
if response.ContentLength < 0 || response.ContentLength > min(limit, 32<<10) {
return
}
body, ok := response.Body.(*responseBody)
if !ok {
return
}
timer := time.AfterFunc(25*time.Millisecond, body.cancel)
defer timer.Stop()
drainSmallBody(response, limit)
}
// Keep the option builder out of the allocation path of existing plain calls.
func jsonOptions(limit int64, options []RequestOption) (requestOptions, error) {
settings := requestOptions{maxResponseBytes: limit}
for _, option := range options {
if option == nil {
return requestOptions{}, errors.New("httpc: nil request option")
}
if err := option(&settings); err != nil {
return requestOptions{}, err
}
}
return settings, nil
}

99
httpc/request.go Normal file
View File

@@ -0,0 +1,99 @@
package httpc
import (
"errors"
"net/http"
"strings"
)
type requestOptions struct {
headers http.Header
maxResponseBytes int64
}
// RequestOption applies only to one JSON call. Returned options can be reused
// concurrently; request headers are independent, with no client-wide credentials.
type RequestOption func(*requestOptions) error
// WithHeader sets a header for this call; later options replace earlier values.
// Framing and Host must be controlled through a standard Request passed to Do.
func WithHeader(name, value string) RequestOption {
name = http.CanonicalHeaderKey(name)
err := checkHeader(name, []string{value})
return func(o *requestOptions) error {
if err != nil {
return err
}
if o.headers == nil {
o.headers = make(http.Header)
}
o.headers[name] = []string{value}
return nil
}
}
// WithHeaders snapshots headers immediately. Editing the original map after
// construction does not change the option; each invocation gets separate slices.
func WithHeaders(headers http.Header) RequestOption {
snapshot := make(http.Header, len(headers))
var err error
for key, values := range headers {
name := http.CanonicalHeaderKey(key)
if err = checkHeader(name, values); err != nil {
break
}
if _, exists := snapshot[name]; exists {
err = errors.New("httpc: duplicate canonical header name")
break
}
snapshot[name] = append([]string(nil), values...)
}
return func(o *requestOptions) error {
if err != nil {
return err
}
if o.headers == nil {
o.headers = make(http.Header, len(snapshot))
}
for name, values := range snapshot {
o.headers[name] = append([]string(nil), values...)
}
return nil
}
}
// WithMaxResponseBytes overrides JSON's bounded response read for one call.
// The size is positive and at most 1 GiB. Do remains a caller-owned stream.
func WithMaxResponseBytes(limit int64) RequestOption {
return func(o *requestOptions) error {
if limit <= 0 || limit > maxResponseBytes {
return errors.New("httpc: response limit must be positive and at most 1 GiB")
}
o.maxResponseBytes = limit
return nil
}
}
func checkHeader(name string, values []string) error {
if name == "" {
return errors.New("httpc: header name must not be empty")
}
for _, c := range []byte(name) {
if c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || strings.ContainsRune("!#$%&'*+-.^_`|~", rune(c)) {
continue
}
return errors.New("httpc: invalid header name")
}
switch name {
case "Host", "Content-Length", "Transfer-Encoding", "Trailer":
return errors.New("httpc: use Do for Host or request framing controls")
}
for _, value := range values {
for i := 0; i < len(value); i++ {
c := value[i]
if c == '\x7f' || c < ' ' && c != '\t' {
return errors.New("httpc: invalid header value")
}
}
}
return nil
}

316
httpc/request_test.go Normal file
View File

@@ -0,0 +1,316 @@
package httpc
import (
"bytes"
"compress/gzip"
"context"
"errors"
"fmt"
"git.yuntuo.hk/k2safe/go-omni/logx"
"io"
"net"
"net/http"
"net/http/httptest"
"strings"
"sync"
"sync/atomic"
"testing"
"time"
)
func TestJSONHeadersConcurrentSnapshot(t *testing.T) {
var leaked atomic.Bool
s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer "+r.URL.Query().Get("caller") {
leaked.Store(true)
}
if r.Header.Get("X-Shared") != "original" || len(r.Header.Values("X-Tag")) != 2 {
leaked.Store(true)
}
if r.Header.Get("Content-Type") != "application/vnd.example+json" {
leaked.Store(true)
}
io.WriteString(w, `{"ok":true}`)
}))
defer s.Close()
c := testClient(t, Config{})
headers := http.Header{"X-Shared": []string{"original"}, "X-Tag": []string{"one", "two"}}
snapshot := WithHeaders(headers)
headers.Set("X-Shared", "changed")
headers["X-Tag"][0] = "changed"
var wg sync.WaitGroup
for i := 0; i < 24; i++ {
wg.Add(1)
go func(i int) {
defer wg.Done()
id := fmt.Sprint(i)
var output struct {
OK bool `json:"ok"`
}
if err := c.JSON(context.Background(), "POST", s.URL+"?caller="+id, map[string]bool{"hello": true}, &output, snapshot, WithHeader("Authorization", "Bearer "+id), WithHeader("content-type", "application/vnd.example+json")); err != nil || !output.OK {
t.Errorf("JSON failed: %v", err)
}
}(i)
}
wg.Wait()
if leaked.Load() {
t.Fatal("headers leaked across requests or changed with source map")
}
}
func TestRequestOptionsValidationBeforeNetwork(t *testing.T) {
var calls atomic.Int32
c := testClient(t, Config{Transport: roundTripperFunc(func(r *http.Request) (*http.Response, error) {
calls.Add(1)
return benchmarkTransport{`{}`, 200}.RoundTrip(r)
})})
invalid := []RequestOption{nil, WithHeader("bad name", "private-token"), WithHeader("Authorization", "private-token\r\nInjected: yes"), WithHeader("Host", "example.invalid"), WithHeader("Content-Length", "1"), WithHeader("Transfer-Encoding", "chunked"), WithHeader("Trailer", "X-Signature"), WithHeaders(http.Header{"X-Name": []string{"one"}, "x-name": []string{"two"}}), WithMaxResponseBytes(0), WithMaxResponseBytes(maxResponseBytes + 1)}
for _, option := range invalid {
err := c.JSON(context.Background(), "GET", "http://example.invalid", nil, nil, option)
if err == nil || strings.Contains(err.Error(), "private-token") {
t.Fatal("missing or unsafe error", err)
}
}
if calls.Load() != 0 {
t.Fatal("invalid options reached network")
}
}
func TestJSONResponseLimitsAndStreaming(t *testing.T) {
var calls atomic.Int32
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls.Add(1)
switch r.URL.Path {
case "/empty":
w.WriteHeader(204)
case "/chunked":
w.(http.Flusher).Flush()
io.WriteString(w, strings.Repeat("x", 17))
case "/status-stream":
w.WriteHeader(401)
w.(http.Flusher).Flush()
<-r.Context().Done()
case "/multiple":
io.WriteString(w, `{} {}`)
default:
io.WriteString(w, `{"ok":true}`)
}
}))
defer server.Close()
c := testClient(t, Config{MaxResponseBytes: 8, Timeout: time.Second})
var out struct {
OK bool `json:"ok"`
}
if err := c.JSON(context.Background(), "GET", server.URL, nil, &out, WithMaxResponseBytes(32)); err != nil || !out.OK {
t.Fatal(err)
}
if err := c.JSON(context.Background(), "GET", server.URL, nil, nil); !errors.Is(err, ErrResponseTooLarge) {
t.Fatal("limit changed permanently", err)
}
for _, target := range []any{nil, &out} {
if err := c.JSON(context.Background(), "GET", server.URL+"/chunked", nil, target, WithMaxResponseBytes(16)); !errors.Is(err, ErrResponseTooLarge) {
t.Fatal("unbounded chunked body", err)
}
}
if err := c.JSON(context.Background(), "GET", server.URL+"/empty", nil, &out); err != nil || !out.OK {
t.Fatal("empty success changed output", err)
}
if err := c.JSON(context.Background(), "GET", server.URL+"/multiple", nil, &out); err == nil {
t.Fatal("multiple JSON documents accepted")
}
start := time.Now()
err := c.JSON(context.Background(), "GET", server.URL+"/status-stream", nil, nil)
var status *StatusError
if !errors.As(err, &status) || status.StatusCode != 401 || time.Since(start) > 500*time.Millisecond {
t.Fatal("waited for unknown status body", err)
}
if calls.Load() != 7 {
t.Fatal(calls.Load())
}
}
func TestErrorBodyPoolReuseAndDrainBound(t *testing.T) {
var connections atomic.Int32
s := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(401)
io.WriteString(w, "private-error-body")
}))
s.Config.ConnState = func(_ net.Conn, state http.ConnState) {
if state == http.StateNew {
connections.Add(1)
}
}
s.Start()
defer s.Close()
c := testClient(t, Config{})
for i := 0; i < 10; i++ {
err := c.JSON(context.Background(), "GET", s.URL, nil, nil)
var status *StatusError
if !errors.As(err, &status) || strings.Contains(err.Error(), "private-error-body") {
t.Fatal(err)
}
}
if connections.Load() != 1 {
t.Fatal("small status response prevented reuse", connections.Load())
}
body := &countedBody{Reader: strings.NewReader(strings.Repeat("x", 1<<20))}
custom := testClient(t, Config{Transport: roundTripperFunc(func(r *http.Request) (*http.Response, error) {
return &http.Response{StatusCode: 401, ContentLength: 1, Body: body, Header: make(http.Header)}, nil
})})
err := custom.JSON(context.Background(), "GET", "http://example.invalid", nil, nil)
var status *StatusError
if !errors.As(err, &status) || !body.closed || body.read > (32<<10)+1 {
t.Fatal("unbounded/missing drain", body.read, body.closed, err)
}
}
type countedBody struct {
io.Reader
read int
closed bool
}
func (b *countedBody) Read(p []byte) (int, error) {
n, err := b.Reader.Read(p)
b.read += n
return n, err
}
func (b *countedBody) Close() error { b.closed = true; return nil }
type roundTripperFunc func(*http.Request) (*http.Response, error)
func (f roundTripperFunc) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
type borrowedTransport struct{ closed atomic.Bool }
func (t *borrowedTransport) RoundTrip(r *http.Request) (*http.Response, error) {
return benchmarkTransport{`{}`, 200}.RoundTrip(r)
}
func (t *borrowedTransport) CloseIdleConnections() { t.closed.Store(true) }
func TestBorrowedTransportAndHeaderRetry(t *testing.T) {
owned := &borrowedTransport{}
c := testClient(t, Config{Transport: owned})
c.Close()
if owned.closed.Load() {
t.Fatal("closed caller's transport")
}
var attempts atomic.Int32
retry := testClient(t, Config{MaxRetries: 1, Backoff: time.Millisecond, Transport: roundTripperFunc(func(r *http.Request) (*http.Response, error) {
if r.Header.Get("Authorization") != "Bearer test" {
t.Error("retry lost header")
}
status := 200
if attempts.Add(1) == 1 {
status = 503
}
return benchmarkTransport{`{}`, status}.RoundTrip(r)
})})
if err := retry.JSON(context.Background(), "GET", "http://example.invalid", nil, nil, WithHeader("Authorization", "Bearer test")); err != nil || attempts.Load() != 2 {
t.Fatal(err, attempts.Load())
}
}
func FuzzRequestHeaderOptions(f *testing.F) {
f.Add("Authorization", "Bearer example")
f.Add("X-Name", "one\r\nX-Test: bad")
f.Fuzz(func(t *testing.T, name, value string) {
if len(name)+len(value) > 8192 {
t.Skip()
}
option := WithHeader(name, value)
var settings requestOptions
if err := option(&settings); err != nil {
return
}
req, err := http.NewRequest("GET", "http://example.invalid", nil)
if err != nil {
t.Fatal(err)
}
req.Header = settings.headers
// The standard Request writer must be able to serialize accepted headers.
if err = req.Write(io.Discard); err != nil {
t.Fatal("accepted malformed header", err)
}
})
}
func TestHTTPLogDoesNotContainCredentials(t *testing.T) {
var buffer bytes.Buffer
logger, err := logx.New(logx.Config{Writer: &buffer})
if err != nil {
t.Fatal(err)
}
defer logger.Close()
c, err := New(Config{Transport: benchmarkTransport{"private-response", 401}})
if err != nil {
t.Fatal(err)
}
defer c.Close()
ctx := logx.WithLogger(context.Background(), logger)
err = c.JSON(ctx, "GET", "http://private-user:private-password@example.invalid/private-path?token=private-query", nil, nil, WithHeader("Authorization", "Bearer private-token"))
if err == nil {
t.Fatal("expected status error")
}
for _, secret := range []string{"private-user", "private-password", "private-path", "private-query", "private-token", "private-response"} {
if strings.Contains(buffer.String(), secret) || strings.Contains(err.Error(), secret) {
t.Fatal("sensitive field leaked")
}
}
if !strings.Contains(buffer.String(), "http.client") {
t.Fatal("missing access log")
}
}
func TestCloseAllowsInflightCallToFinish(t *testing.T) {
entered := make(chan struct{})
release := make(chan struct{})
var once sync.Once
releaseNow := func() { once.Do(func() { close(release) }) }
s := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { close(entered); <-release; io.WriteString(w, `{}`) }))
defer s.Close()
c := testClient(t, Config{Timeout: time.Second})
done := make(chan error, 1)
go func() { done <- c.JSON(context.Background(), "GET", s.URL, nil, nil) }()
defer releaseNow()
select {
case <-entered:
case err := <-done:
t.Fatal("request failed before handler", err)
case <-time.After(2 * time.Second):
t.Fatal("handler not entered")
}
c.Close()
releaseNow()
if err := <-done; err != nil {
t.Fatal("Close interrupted in-flight call", err)
}
if err := c.JSON(context.Background(), "GET", s.URL, nil, nil); !errors.Is(err, ErrClosed) {
t.Fatal("accepted new call after Close", err)
}
}
func TestSlowKnownErrorBodyHasShortDrainBudget(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Length", "100")
w.WriteHeader(503)
w.(http.Flusher).Flush()
<-r.Context().Done()
}))
defer server.Close()
c := testClient(t, Config{Timeout: 2 * time.Second})
start := time.Now()
err := c.JSON(context.Background(), "GET", server.URL, nil, nil)
var status *StatusError
if !errors.As(err, &status) || status.StatusCode != 503 || time.Since(start) > 500*time.Millisecond {
t.Fatal("slow error body delayed status", err, time.Since(start))
}
}
func TestJSONLimitAppliesAfterGzipExpansion(t *testing.T) {
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Encoding", "gzip")
writer := gzip.NewWriter(w)
_, _ = io.WriteString(writer, strings.Repeat("x", 4096))
_ = writer.Close()
}))
defer server.Close()
c := testClient(t, Config{MaxResponseBytes: 64})
if err := c.JSON(context.Background(), "GET", server.URL, nil, nil); !errors.Is(err, ErrResponseTooLarge) {
t.Fatal("gzip bypassed response limit", err)
}
}

View File

@@ -0,0 +1,61 @@
# v0.20 统一配置加载验收(2026-09-26)
本轮修改 go-omni 配置包、生成模板与文档。没有修改接入项目的业务配置/数据库,也没有使用线上服务器做压力测试。
## 回归结果
- 全量 `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...`:47 个有测试的包通过,包含实际生成项目、生命周期失败清理、错误目录/响应继承、上传和拷贝工具回归。
- `go vet ./...`、`go build ./...` 通过。
- `GOTOOLCHAIN=go1.22.12 go test ./config ./examples/configuration` 通过。
- 新生成消费者的 `go test -race ./...`、`go build .`、generate/check/dry-run 通过。
- checked-in managedService 已显式迁移旧 Load,生成检查、race 和构建通过;三种格式示例与环境覆盖命令实际运行通过。
针对性验证:
| 场景 | 结果 |
| --- | --- |
| JSON / YAML / XML 的 defaults→file→env→Validate | 同值,0/false 可覆盖,envDefault 不覆盖文件 |
| 大于 2^53 的整数、动态 json.Number | 精度保留 |
| null、空数组、嵌套指针、map、duration、TextUnmarshaler、JSON tag / 嵌入字段 | 语义回归通过 |
| 文件字段类型错、未知字段、重复字段、业务 Validate 失败 | 文件/字段/可用行列与安全原因 |
| 环境类型或校验失败、集合子字段校验 | 定位环境来源,不错误归因于文件 |
| 内部错误链和默认值隔离 | errors.Is/As 可用,失败不提交半成品/修改默认值 |
| 24 个并发加载、自定义 Decoder、禁止事后注册 | race 通过,实例间不泄漏 |
| 超大文件、超深输入、重复 XML 根、DTD/entity、YAML alias/merge、UTF-8 | 拒绝;解析器 panic 不公开 panic 内容 |
| 业务新增字段与自定义格式 | 不改 main/loadGen,生成保留手写文件 |
| 配置校验失败 | 依赖初始化之前返回;未触发测试中的 Init panic |
| 旧手写 Load、loadGen 漂移 | generate/check/dry-run 拒绝冲突,不改业务文件 |
30 秒、4 worker 的 `FuzzConfigurationDecoders`:383586 次执行,无失败(总运行 31.309 秒,包含停止收尾)。模糊输入上限 8 KiB;另有确定性 1 MiB/深度上限和恶意格式回归,不把 fuzz 次数当作形式证明。
## 性能
Go 1.27.1、darwin/arm64、Apple M2 Max;无 race 的本地基准,每个场景三轮。原始数据:[configurationBenchmark.txt](configurationBenchmark.txt)。
| 场景 | 中位数耗时 | 典型分配 |
| --- | ---: | ---: |
| 小配置 JSON 解析+绑定 | 5.068 µs | 3781 B / 75 allocs |
| 小配置 YAML 解析+绑定 | 14.855 µs | 15602 B / 181 allocs |
| 小配置 XML 解析+绑定 | 7.137 µs | 5605 B / 122 allocs |
| 含完整 Server 默认值:文件读取+环境覆盖+校验 | 107.660 µs | 约 67.4 KB / 1288 allocs |
| 1000 个条目,约 28 KB JSON 解析+绑定 | 2.548 ms | 约 2.40 MB / 31585 allocs |
配置在启动阶段加载,不经过业务请求路径。这些是暖文件缓存、本机测试结果,不能等同冷磁盘启动耗时、请求 QPS 或跨机器性能承诺。没有为此引入全局反射缓存、解析器池或后台线程。
## 边界与迁移
- 旧手写 Load 必须先明确迁移,生成器不会猜测其解密/其他特殊行为。
- 内置 XML 是明确的配置子集;YAML alias/merge/custom tag 拒绝;固定 Go 数组严格长度。
- 自定义 Decoder/反序列化/Validate 是可信 Go 回调,其并发、副作用与公开 Reason 的脱敏由项目负责。
- 第三方非结构化验证错误仅定位子树,并保留内部原因;业务用 Invalid/AtField 提供准确的公开字段与原因。
- 未进行业务数据库/第三方真实依赖验收,未修改或替代各接入项目的业务回归。
## 远程发布验收
`v0.20.0-preview` 标签固定在 `c60841f901a3bcc3e9582eece6cfb2cad008fb9e`。空 module 缓存安装已发布 CLI;生成消费者引用相同远程 module 和 checksum,**没有本地 replace**。
根目录 `go run .` / 构建、nethttp/native 退出信号与端口释放、上传下载/原始回调、两服务共享错误目录,以及 8 组消费者 race 回归全部通过。新消费者验证 JSON/YAML/XML/自定义扩展、环境来源、初始化前验证、业务文件保留与 loadGen 漂移检查。
完整机器记录:[configurationRemote.json](configurationRemote.json),本轮耗时 50.24 秒。标签不因补充验收记录移动;记录随后提交到 main。
完整使用与迁移:[CONFIG.md](../CONFIG.md)。

View File

@@ -0,0 +1,21 @@
goos: darwin
goarch: arm64
pkg: git.yuntuo.hk/k2safe/go-omni/config
cpu: Apple M2 Max
BenchmarkConfiguration/json-12 239197 5068 ns/op 3781 B/op 75 allocs/op
BenchmarkConfiguration/json-12 234421 5058 ns/op 3781 B/op 75 allocs/op
BenchmarkConfiguration/json-12 233330 5564 ns/op 3781 B/op 75 allocs/op
BenchmarkConfiguration/yaml-12 83022 14827 ns/op 15602 B/op 181 allocs/op
BenchmarkConfiguration/yaml-12 83197 14855 ns/op 15604 B/op 181 allocs/op
BenchmarkConfiguration/yaml-12 74828 16070 ns/op 15602 B/op 181 allocs/op
BenchmarkConfiguration/xml-12 167509 7144 ns/op 5606 B/op 122 allocs/op
BenchmarkConfiguration/xml-12 171202 7086 ns/op 5605 B/op 122 allocs/op
BenchmarkConfiguration/xml-12 167727 7137 ns/op 5605 B/op 122 allocs/op
BenchmarkConfigurationStartup-12 10000 107660 ns/op 67429 B/op 1288 allocs/op
BenchmarkConfigurationStartup-12 10000 107420 ns/op 67427 B/op 1288 allocs/op
BenchmarkConfigurationStartup-12 10000 107828 ns/op 67430 B/op 1288 allocs/op
BenchmarkConfigurationLarge-12 481 2548066 ns/op 10.99 MB/s 2403822 B/op 31585 allocs/op
BenchmarkConfigurationLarge-12 471 2532248 ns/op 11.06 MB/s 2403851 B/op 31586 allocs/op
BenchmarkConfigurationLarge-12 477 2641417 ns/op 10.61 MB/s 2403402 B/op 31583 allocs/op
PASS
ok git.yuntuo.hk/k2safe/go-omni/config 20.682s

View File

@@ -0,0 +1,146 @@
{
"passed": true,
"frameworkVersion": "v0.20.0-preview",
"frameworkSum": "h1:UoymLOIS36Uf7GJ2FhUMP5f1RLpGQXpkLjpqQUZzpv4=",
"freshModuleCache": true,
"noLocalReplace": true,
"durationSeconds": 50.24,
"checks": [
"remoteCLIInstall",
"freshModuleCache",
"fixedVersionNoReplace",
"contractChangePreview",
"persistentLogicSignatureFailure",
"businessSourcePreserved",
"fixedConsumerBuild",
"rootGoRun",
"nethttpAndNativeSignals",
"listenerReleased",
"default404And405",
"goOwnedErrorCatalog",
"unwrappedDownloads",
"rawCallback",
"multipart400And413",
"nethttpAndNativeTransfers",
"sharedGoCatalogExport",
"documentOnlyErrorSelection",
"workspaceCheckNoWrites",
"staleCatalogRejected",
"twoIndependentServiceBuilds",
"initializationAndListenFailureCleanup",
"drainBeforeDependencyClose",
"generatedConsumerRaceTests",
"errorParamsAndInheritance",
"manualHooksAndScalars",
"dependencyFatalShutdown",
"fatalProcessExitCode",
"initializationShutdownNotification",
"configuredBuiltinEndpoints",
"projectHealthEnvelope",
"endpointConflictCleanup",
"responseArrayNullableNull",
"primitiveAndDynamicArrays",
"strictRequestPolicies",
"declarativeMultipartBinding",
"backgroundStopDrainResourceOrder",
"backgroundInitAndListenFailureCleanup",
"backgroundTimeoutRetainsResourcesAndExits",
"flatFieldComposition",
"copyxThreeEndpointConsumer",
"explicitNullAndMoneyPolicies",
"fieldAdditionWithoutCopyRuleChanges",
"middlewareFactoryInjectionAndCleanup",
"optionalRepositoryPreservesBusiness",
"generatedUnifiedConfig",
"jsonYamlXmlAndCustomDecoder",
"configValidationBeforeDependencies",
"configEnvProvenance",
"configExtensionPreserved",
"configLoaderDriftCheck"
],
"sourceHashes": {
"lifecycle/runtime.go": "7d890150b931cdc6c5d03f3706ea09b1d220a958d91fbd6ddc6715a9ec13a5d2",
"lifecycle/phases.go": "1436856c708d004ebb28704a867316bf7a44553de56b3e29152d41475a2f793a",
"config/lifecycle_test.go": "8e5fccf0f7432c64d7fa8a52687002307d3ebb1fd039adb6ae09da0c9a2891bf",
"task/task.go": "acb1a2cc0e26029331065545582c30e0c547e3b6869f7d8b16c7d25ab0077545",
"cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "e8575c04b0923d924d96e4334731313076284fd068c4990715b7a07a2724f969",
"cmd/go-omni/apigen/templates.go": "60bdb5d9443ff9f772c0db1ccd637e60ec9906413d3fb4c52ea162e5c4341a22",
"config/project.go": "e3478bc08125895c0c65a61b818a7fb893b276af955865c9a088d878d1b11e51",
"contract/bind.go": "7590c3100a9bf90236aba7fda0a021d065d9dacd8c432c8eed1066f2ed8d37c8",
"contract/policy.go": "73a3a55fb88195b32f22b2bc066d9b13dccd7fd520e98c5a1b2979a3fd2447eb",
"contract/multipart.go": "db0377599b1b8b39246d41129513e8187109e44c1892971244d581c22ccc8ecd",
"contract/file.go": "20dd2de17e4854d452afc3c65f0293d504fb8fc25e91583ef45207fdab30948d",
"cmd/go-omni/apigen/business_test.go": "3b484352282d945f5a58047682079e3d64d4b0bf6ec6bafc23151b1197bb3942",
"cmd/go-omni/apigen/generate.go": "b0234696420ab9f9fb8b54ba1df682ced7d155bae03c65f37fe4498a6b574685",
"lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31",
"config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d",
"health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866",
"httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb",
"app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0",
"app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73",
"scripts/verifyManagedRelease.py": "f9775c8497d258723955e3e28966de16dc641564e9ac4edc21e1292c85f06deb",
"scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18",
"cmd/go-omni/apigen/managed_test.go": "e0b2b5aa6c4d6383590b467d57d7b4e00505ca35d6b6cde2475e4ad574041e94",
"cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247",
"cmd/go-omni/apigen/testdata/managed/background.go": "fb357b5deefd5187f2df406be5189fc4d57da857bd55b474f4726bf065c77167",
"cmd/go-omni/apigen/testdata/managed/client.go": "cdd7d394ddad4bcdc10224890a884aa513ed2ffd8275ecdad8de9963dd10986a",
"cmd/go-omni/apigen/testdata/managed/phases_test.go": "4d826d7133027ac2399f8209f03efa46ed54aa45dcd0a35274a9009314941dca",
"cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319",
"cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f",
"cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1",
"cmd/go-omni/apigen/testdata/business/accessFactory.go": "2f982cf39b5e1a4be98596ce5d0a296af0fbf8c94616543a268e67ec376f31b4",
"cmd/go-omni/apigen/testdata/business/client.go": "d4cbaeec18dfb3a0e2821217e8eb0515000e351a745a93ec811dabca44e58c8a",
"cmd/go-omni/apigen/testdata/business/consumer_test.go": "e4159862a729ca2c5c9699840cc16a56350fc2914c43841f08946e9a6e9c8091",
"cmd/go-omni/apigen/testdata/business/operations.go": "11ded7921a0753f22975c525e334e3d45366143625e41d24fe500504e785c64c",
"cmd/go-omni/apigen/responseShape.go": "5b705130b66a3d4aaed842025b75abe54e6427acac8366366ca88ae9862429e1",
"cmd/go-omni/apigen/documentRefs.go": "7b841a668154d872493104eda0f9049340c67d9323eb99f1621a1ef6ba343891",
"cmd/go-omni/apigen/catalogDocument.go": "94df60328b2f105c35b73d9c6362ea14abadce6cb18e4d9921193d2d46854252",
"cmd/go-omni/apigen/workspaceCatalog.go": "c00784220b379ed80deb07c060532b756cbc47434ecb8b1ca10bc12b12159328",
"config/env.go": "ea84bccdcd39e3f6b4958a0c8ac8e7d6a0be516b24930f93039248165361bf35",
"cmd/go-omni/apigen/arrayResponse_test.go": "9e3e4475f76683d3346d349cdfa94e9e75cf913ac6979bd7a4ed86628466d879",
"cmd/go-omni/apigen/testdata/arrays/values.go": "d5e7f3adf12aabb301f46c296e8c894a7b2f6f70a6b6e7e02c74da7f1fd17bc5",
"cmd/go-omni/apigen/testdata/arrays/consumer_test.go": "435932f2952ad897943e2cda83b04eccb2950579d8d88772661541e1e2a2333d",
"cmd/go-omni/apigen/composition.go": "7ed3761b27d9c932d8ddc43e63c2aca798bdbe38f2b40571af8916c8401ea650",
"cmd/go-omni/apigen/generatorConfig.go": "78db0c7808cf6ec997c5b64a4a5fbea11b74049709bc5900ccfbc5d8dabebbc8",
"cmd/go-omni/apigen/copyConsumer_test.go": "e08ae4daed5dd25ab89d8872f653caf19499984dcaddc9835c13884c410fc0c3",
"cmd/go-omni/apigen/generatorConfig_test.go": "a6d8c8ebea5f1accf0e37dd044e57b48e422821fe75086d8d2eb3abe3acb147c",
"cmd/go-omni/apigen/middlewareFactory.go": "cc0220edb3eff25d7f66b2c37e206455f8730a2b909613713322bee7caf72303",
"cmd/go-omni/apigen/testdata/composition/accessFactory.go": "e1e5057ff79f806b3218e93249e8191d097b5c4844977204a8c32848443cba33",
"cmd/go-omni/apigen/testdata/composition/domain.go": "ab27ebd7ad5e3ed5505a2aa268b9b5ceedbb01477b59011bec63bb270114078b",
"cmd/go-omni/apigen/testdata/composition/amount.go": "f2d1834502d71d9ecf72ebe3b1077e5ef41f44fd816abd4617ac19f9bb962fdd",
"cmd/go-omni/apigen/testdata/composition/copy.go": "32d37d6f1f0e9aa2409aeb90b052801d1cdca70e504c2dd294cdae01422c9cb7",
"cmd/go-omni/apigen/testdata/composition/client.go": "19102e191c0471cae9e7405623906d0d804865c40b80ff1d5b240a41ceab6a4e",
"cmd/go-omni/apigen/testdata/composition/service.api": "01d230d5409756e2f3eb602a7fe065efa7d9599f7a4089da4bf8f07a2474444e",
"cmd/go-omni/apigen/testdata/composition/consumer_test.go": "74aaf60318cd0de67495fc9142b3f67c3b6e232bb48b46b3e134de5aadd225ad",
"cmd/go-omni/apigen/testdata/composition/dependencies.go": "edc153204b06da00c0e45c52db43a4a22f8ffe22c4a6d3f3ebec0137b861f84d",
"cmd/go-omni/apigen/testdata/composition/converters.go": "ecdcfe31c75ca815dab8d1a82533b12cbbd40e12b3a5c444e815097f6c4df08e",
"copyx/options.go": "edea2f3d2043bf813e56dedf0aea319aaa52301d3b589f41fef986e33a77dc32",
"copyx/copy.go": "a752a81acc0e847158043daaf0fdaaecec43abaf27d5561725a47b3e120ffbdc",
"copyx/graph.go": "ae6a2a11a8175033d327b580cd570c97fe9d2ddb814bb50884cc19a90766badd",
"copyx/benchmark_test.go": "b1bf408957cadea47b47215ffb099d140d5d1a2eb061024a6d0c1d769f469a79",
"copyx/copy_test.go": "89b65a6281591d7cd5823cc1c7d2c384c6bd446ef06aa1763ddc3acaa84dfb4a",
"config/decoders.go": "6d31f67c82563c2d63e37ad1ce9b9fbad6232b2b785b9b58dd025be857f0fc28",
"config/config.go": "e51ece1fa824d8b0fc7385e027b4a4408e459ec2c89444fab99e1e6a19e8c1cf",
"config/errorCatalog_test.go": "8dcaa73bfc0c98f30b39103f30048e22fc3190d66f5ad70f5670decedf837479",
"config/http_test.go": "0cae3dcf213369cfaff5487bf630c2bc3ec097739ca1d40794124e69c3880db8",
"config/response_test.go": "b4ccb111f813f80dc180a7ddb11034a409b9409cbb1b57e654c04d678465d201",
"config/projectAdvanced_test.go": "8734a4c24a48f7f28c417e4a5e50783096d4782050b658ecc8199c257f93d1e2",
"config/file_test.go": "1479d625aad26a1f773bc4a08c146347f163bbf7b4f11385a32f687119df5817",
"config/aliases_test.go": "38de1dc2633c5513fc3870cb87ff42217e968487f934e8b7fec1488ba3555eb3",
"config/errorParams_test.go": "a5a07c2f13524dfc136174d502e7cff37b45db97dc8b26a5dc8b89bafcb6b8af",
"config/http.go": "6fbeac9c21f5dedad05471deb03a6b4465a0cb10f252d651da2f6f6cb3841998",
"config/loader.go": "c1cf9ede71ad0dd9a16d0c3e4e0eebb8b0e0e2045b5a66dcab042692a9ca3520",
"config/file.go": "cc24c1f647d925fd91656a12f5817660ed97b4e46ba26f36f4934c4af7c988e9",
"config/diagnostic.go": "1e6324cdc229872c16e77a5d5eb1ce2a1d81b730a8573e518d06c02ed08127d8",
"config/bind.go": "b757082fdc58e7eee3b8229cac6093eb31e3a9cd226806984c476ff6bfbbc724",
"config/clone.go": "6146bf8a7d7549d975142a843db5c6f95c67e98b62cbd98e20c77842abd38040",
"config/endpoints_test.go": "3c0117939955f41cdbffc70da7c2d92e6cb184b14124c357d4f1024cb00389d5",
"config/config_test.go": "f85630d171d54b7a1e9a3eed6b622cabef244409027e7d6c1dce3a3b9eb9aac4",
"config/project_test.go": "040757b9512cf0490f76566510d4038349b0312eac0623ff473307b2c610731d",
"config/loader_test.go": "b09aeebfdfbcae0d4c207715f341fc473c9b3d199532d4e8358658460fc815cf",
"cmd/go-omni/apigen/configLoader_test.go": "4910930bd26b84632446d7993bf98d3a88a34b392440bd6f0a78639d0fed5b47",
"cmd/go-omni/apigen/testdata/configuration/consumer_test.go": "3f0d2230d00684d2da46f10ece33d9acdbac41d1100bc8d310cf6588ad4c5a8f",
"cmd/go-omni/scaffold/templates/api-load.go.tmpl": "86d92dbac769d0f2748ea9c957457800939bf0038643917222e1a3455f2dce6f",
"cmd/go-omni/scaffold/templates/api-loader.go.tmpl": "847ac407827808022dcf493823b58671e1ae50349b173a5e4500580e07c64150"
}
}

View File

@@ -68,3 +68,15 @@ go run ./cmd/go-omni api generate --dir examples/managedService --check
- converter 是受信项目代码,必须只读输入;不能自动回滚其副作用、证明其未共享外部全局对象,或强杀无限循环。 - converter 是受信项目代码,必须只读输入;不能自动回滚其副作用、证明其未共享外部全局对象,或强杀无限循环。
- 当前记录是本机库/生成器验收,不是线上服务吞吐、Linux soak、真实业务数据库回归或 OSS/R2 云账号验收。本轮没有对线上服务器施压。 - 当前记录是本机库/生成器验收,不是线上服务吞吐、Linux soak、真实业务数据库回归或 OSS/R2 云账号验收。本轮没有对线上服务器施压。
- 旧消费者仍需按 [COPY.md](../COPY.md#从旧-mapping-迁移) 迁移,不会被这次框架发布自动修改。 - 旧消费者仍需按 [COPY.md](../COPY.md#从旧-mapping-迁移) 迁移,不会被这次框架发布自动修改。
## 发布后远程验收
标签 `v0.19.0-preview` 固定在 `fd83235ce7fc15ce88a482d385d70e61f25d7b32`。空 module 缓存安装远程 CLI,消费者固定同版依赖、具有校验和、无本地 replace,7 组实际生成消费者 race 回归通过;包含三端 copyx 示例。
根目录 `go run .` / 构建、nethttp/native 的 SIGINT/SIGTERM、404/405、Go 错误目录、原始回调、上传/下载、workspace 共用错误目录及资源收尾均通过。远程单独执行 `go run git.yuntuo.hk/k2safe/go-omni/examples/copying@v0.19.0-preview` 也验证深拷贝隔离与共享 converter 拒绝。
```sh
python3 scripts/verifyManagedRelease.py --version v0.19.0-preview --report reports/copyxRemote.json
```
发布后验收工具同步移除了旧 mapping 用例,改为 copyx 消费者并核对实际远程依赖校验和。详细检查项、源摘要见 [copyxRemote.json](copyxRemote.json)。标签不随验收记录的后续文档提交移动。

117
reports/copyxRemote.json Normal file
View File

@@ -0,0 +1,117 @@
{
"passed": true,
"frameworkVersion": "v0.19.0-preview",
"frameworkSum": "h1:pMMAyBOgKpGvU8WDxKIwvMU4jWf72cx0aKHVO+snFE4=",
"freshModuleCache": true,
"noLocalReplace": true,
"durationSeconds": 62.46,
"checks": [
"remoteCLIInstall",
"freshModuleCache",
"fixedVersionNoReplace",
"contractChangePreview",
"persistentLogicSignatureFailure",
"businessSourcePreserved",
"fixedConsumerBuild",
"rootGoRun",
"nethttpAndNativeSignals",
"listenerReleased",
"default404And405",
"goOwnedErrorCatalog",
"unwrappedDownloads",
"rawCallback",
"multipart400And413",
"nethttpAndNativeTransfers",
"sharedGoCatalogExport",
"documentOnlyErrorSelection",
"workspaceCheckNoWrites",
"staleCatalogRejected",
"twoIndependentServiceBuilds",
"initializationAndListenFailureCleanup",
"drainBeforeDependencyClose",
"generatedConsumerRaceTests",
"errorParamsAndInheritance",
"manualHooksAndScalars",
"dependencyFatalShutdown",
"fatalProcessExitCode",
"initializationShutdownNotification",
"configuredBuiltinEndpoints",
"projectHealthEnvelope",
"endpointConflictCleanup",
"responseArrayNullableNull",
"primitiveAndDynamicArrays",
"strictRequestPolicies",
"declarativeMultipartBinding",
"backgroundStopDrainResourceOrder",
"backgroundInitAndListenFailureCleanup",
"backgroundTimeoutRetainsResourcesAndExits",
"flatFieldComposition",
"copyxThreeEndpointConsumer",
"explicitNullAndMoneyPolicies",
"fieldAdditionWithoutCopyRuleChanges",
"middlewareFactoryInjectionAndCleanup",
"optionalRepositoryPreservesBusiness"
],
"sourceHashes": {
"lifecycle/runtime.go": "7d890150b931cdc6c5d03f3706ea09b1d220a958d91fbd6ddc6715a9ec13a5d2",
"lifecycle/phases.go": "1436856c708d004ebb28704a867316bf7a44553de56b3e29152d41475a2f793a",
"config/lifecycle_test.go": "8e5fccf0f7432c64d7fa8a52687002307d3ebb1fd039adb6ae09da0c9a2891bf",
"task/task.go": "acb1a2cc0e26029331065545582c30e0c547e3b6869f7d8b16c7d25ab0077545",
"cmd/go-omni/scaffold/templates/api-runtime.go.tmpl": "e8575c04b0923d924d96e4334731313076284fd068c4990715b7a07a2724f969",
"cmd/go-omni/apigen/templates.go": "980856557a247784a3d8e63212bf7fcc3965bcbf7689119f8ee999ccd7867f26",
"config/project.go": "0917888f4e471761b897ddd59426eb098bf586edb5c5dc856320b84b3bdb5f7f",
"contract/bind.go": "7590c3100a9bf90236aba7fda0a021d065d9dacd8c432c8eed1066f2ed8d37c8",
"contract/policy.go": "73a3a55fb88195b32f22b2bc066d9b13dccd7fd520e98c5a1b2979a3fd2447eb",
"contract/multipart.go": "db0377599b1b8b39246d41129513e8187109e44c1892971244d581c22ccc8ecd",
"contract/file.go": "20dd2de17e4854d452afc3c65f0293d504fb8fc25e91583ef45207fdab30948d",
"cmd/go-omni/apigen/business_test.go": "3b484352282d945f5a58047682079e3d64d4b0bf6ec6bafc23151b1197bb3942",
"cmd/go-omni/apigen/generate.go": "c4bc6652784714f7cb0fd4815094a1668f230b244827fc33cb8f4aa90bbc76fc",
"lifecycle/controller.go": "1fb9393df1908afed93d87a1bf3ecea8d79d78e437a656b56bb7f1e5f6e51d31",
"config/endpoints.go": "9f5123f962d52b71e5b73d5942316170875e895129a94e8b669ce782969ef27d",
"health/health.go": "41f07f27a7fdcfa7973c2c554a22b73129ebc5d44e945e70e054e1cff24ab866",
"httpx/response.go": "2cf78b8f0eee8049ee612bb409a6eecc1d39eab9ab8f3b533ea21e3cac1611cb",
"app/app.go": "e5255ffdd36cba1c155037afc42f1c921b1f91bc79db75798a7feb0e9d51b1c0",
"app/transport.go": "88212d67d375e4c48c250a2e4c62fbff4e7873a67f91f2deb113a60634149f73",
"scripts/verifyManagedRelease.py": "fdc33d4357d501a9bf37d3612fe4fc51aa2a3a5686a475328ad62f1663eab022",
"scripts/verifyManagedProject.py": "b0e4b1aea4f44d61470c35c41f971e0882ae36b0eb5df2be190400d62fc60a18",
"cmd/go-omni/apigen/managed_test.go": "e0b2b5aa6c4d6383590b467d57d7b4e00505ca35d6b6cde2475e4ad574041e94",
"cmd/go-omni/apigen/testdata/managed/options.go": "30d48e4b5aab476c887ac53cc89b48b2b7e250c8cd25e55fecb49bb2a3589247",
"cmd/go-omni/apigen/testdata/managed/background.go": "fb357b5deefd5187f2df406be5189fc4d57da857bd55b474f4726bf065c77167",
"cmd/go-omni/apigen/testdata/managed/client.go": "cdd7d394ddad4bcdc10224890a884aa513ed2ffd8275ecdad8de9963dd10986a",
"cmd/go-omni/apigen/testdata/managed/phases_test.go": "4d826d7133027ac2399f8209f03efa46ed54aa45dcd0a35274a9009314941dca",
"cmd/go-omni/apigen/testdata/managed/shutdown_test.go": "79fbf8dd4508b8217af888a31aaf2f2a01c1fa3869b2e80420605af697c66319",
"cmd/go-omni/apigen/testdata/managed/consumer_test.go": "9e103c9de7a1a906903bc09787e1b38905dc5bb4d891c88938028da5dee98f6f",
"cmd/go-omni/apigen/testdata/managed/operations.go": "44cea63e6023458c5818c9fdae223631cd1721f0841e671457398bfde8bc48f1",
"cmd/go-omni/apigen/testdata/business/accessFactory.go": "2f982cf39b5e1a4be98596ce5d0a296af0fbf8c94616543a268e67ec376f31b4",
"cmd/go-omni/apigen/testdata/business/client.go": "d4cbaeec18dfb3a0e2821217e8eb0515000e351a745a93ec811dabca44e58c8a",
"cmd/go-omni/apigen/testdata/business/consumer_test.go": "e4159862a729ca2c5c9699840cc16a56350fc2914c43841f08946e9a6e9c8091",
"cmd/go-omni/apigen/testdata/business/operations.go": "11ded7921a0753f22975c525e334e3d45366143625e41d24fe500504e785c64c",
"cmd/go-omni/apigen/responseShape.go": "5b705130b66a3d4aaed842025b75abe54e6427acac8366366ca88ae9862429e1",
"cmd/go-omni/apigen/documentRefs.go": "7b841a668154d872493104eda0f9049340c67d9323eb99f1621a1ef6ba343891",
"cmd/go-omni/apigen/catalogDocument.go": "94df60328b2f105c35b73d9c6362ea14abadce6cb18e4d9921193d2d46854252",
"cmd/go-omni/apigen/workspaceCatalog.go": "c00784220b379ed80deb07c060532b756cbc47434ecb8b1ca10bc12b12159328",
"config/env.go": "ea84bccdcd39e3f6b4958a0c8ac8e7d6a0be516b24930f93039248165361bf35",
"cmd/go-omni/apigen/arrayResponse_test.go": "9e3e4475f76683d3346d349cdfa94e9e75cf913ac6979bd7a4ed86628466d879",
"cmd/go-omni/apigen/testdata/arrays/values.go": "d5e7f3adf12aabb301f46c296e8c894a7b2f6f70a6b6e7e02c74da7f1fd17bc5",
"cmd/go-omni/apigen/testdata/arrays/consumer_test.go": "435932f2952ad897943e2cda83b04eccb2950579d8d88772661541e1e2a2333d",
"cmd/go-omni/apigen/composition.go": "7ed3761b27d9c932d8ddc43e63c2aca798bdbe38f2b40571af8916c8401ea650",
"cmd/go-omni/apigen/generatorConfig.go": "78db0c7808cf6ec997c5b64a4a5fbea11b74049709bc5900ccfbc5d8dabebbc8",
"cmd/go-omni/apigen/copyConsumer_test.go": "e08ae4daed5dd25ab89d8872f653caf19499984dcaddc9835c13884c410fc0c3",
"cmd/go-omni/apigen/generatorConfig_test.go": "a6d8c8ebea5f1accf0e37dd044e57b48e422821fe75086d8d2eb3abe3acb147c",
"cmd/go-omni/apigen/middlewareFactory.go": "cc0220edb3eff25d7f66b2c37e206455f8730a2b909613713322bee7caf72303",
"cmd/go-omni/apigen/testdata/composition/accessFactory.go": "e1e5057ff79f806b3218e93249e8191d097b5c4844977204a8c32848443cba33",
"cmd/go-omni/apigen/testdata/composition/domain.go": "ab27ebd7ad5e3ed5505a2aa268b9b5ceedbb01477b59011bec63bb270114078b",
"cmd/go-omni/apigen/testdata/composition/amount.go": "f2d1834502d71d9ecf72ebe3b1077e5ef41f44fd816abd4617ac19f9bb962fdd",
"cmd/go-omni/apigen/testdata/composition/copy.go": "32d37d6f1f0e9aa2409aeb90b052801d1cdca70e504c2dd294cdae01422c9cb7",
"cmd/go-omni/apigen/testdata/composition/client.go": "19102e191c0471cae9e7405623906d0d804865c40b80ff1d5b240a41ceab6a4e",
"cmd/go-omni/apigen/testdata/composition/service.api": "01d230d5409756e2f3eb602a7fe065efa7d9599f7a4089da4bf8f07a2474444e",
"cmd/go-omni/apigen/testdata/composition/consumer_test.go": "74aaf60318cd0de67495fc9142b3f67c3b6e232bb48b46b3e134de5aadd225ad",
"cmd/go-omni/apigen/testdata/composition/dependencies.go": "edc153204b06da00c0e45c52db43a4a22f8ffe22c4a6d3f3ebec0137b861f84d",
"cmd/go-omni/apigen/testdata/composition/converters.go": "ecdcfe31c75ca815dab8d1a82533b12cbbd40e12b3a5c444e815097f6c4df08e",
"copyx/options.go": "edea2f3d2043bf813e56dedf0aea319aaa52301d3b589f41fef986e33a77dc32",
"copyx/copy.go": "a752a81acc0e847158043daaf0fdaaecec43abaf27d5561725a47b3e120ffbdc",
"copyx/graph.go": "ae6a2a11a8175033d327b580cd570c97fe9d2ddb814bb50884cc19a90766badd",
"copyx/benchmark_test.go": "b1bf408957cadea47b47215ffb099d140d5d1a2eb061024a6d0c1d769f469a79",
"copyx/copy_test.go": "89b65a6281591d7cd5823cc1c7d2c384c6bd446ef06aa1763ddc3acaa84dfb4a"
}
}

View File

@@ -0,0 +1,47 @@
# v0.21 httpc 验收(2026-09-26)
目标:让普通项目直接配置代理、连接阶段与空闲池参数,带请求头完成有界 JSON 调用,同时保留标准 Do/Transport 扩展。没有加入业务服务规则,也没有修改业务项目或在线服务器。
## 功能与安全验证
- 代理:独立子进程验证环境代理缓存、默认/显式 environment、NO_PROXY;本地代理与本地目标验证固定代理/直连。配置错误不回显凭证。
- Transport:克隆池与进程默认池互不修改;可配置建连/keepalive/握手/响应头/100-continue/空闲参数,保留 TLS 证书验证。借用 Transport 不修改、不关闭,混合配置明确失败,typed nil 拒绝。
- 实际 socket:慢 TLS 握手、慢响应头、超时后恢复、12 并发下最多 2 条 HTTP/1.1 连接、空闲回收与关闭后的连接释放。
- 请求选项:24 个并发调用的认证头独立;WithHeaders 快照源 map、多值头与大小写规范化,重试不丢头,控制字符/分帧头拒绝,错误不会带 Token。
- JSON:单次响应限额不影响其他请求,已知/未知长度、chunked、自动 gzip 解压后的限制、空成功体、多 JSON 文档、状态错误、调用后关闭。
- 错误响应:已知小体复用连接;读取大小有上限;已知但缓慢的错误体最多花 25ms 尝试读取(标准 Transport 遵守 context),未知流直接关闭,不等完整 body。
- 生命周期:Close 不中断已开始的调用,拒绝新调用;借用 Transport 生命周期保持项目所有。既有重试、总超时、请求 body 拒绝时关闭、重定向不跟随测试保留。
- 日志不含 URL 路径、查询、URL 密码、Authorization 或响应体;标准返回的 URL 错误不承诺自动脱敏,项目应自行选择安全日志字段。
全量 `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...`:48 个测试包通过,包含实际生成项目回归。另验证 Go 1.22.12 的 httpc/示例、vet、build 与 managedService 生成检查。没有改动生成模板的兼容门槛;CLI 当前版本与最低模板版本分开,v0.20 的生成能力保持兼容。
请求头 Fuzz:30s / 4 worker,1104189 次执行,无失败,总收尾时间 31.293s。模糊输入上限 8 KiB,验证被接受的头可以由标准 Request writer 序列化;不等同形式化安全证明。
## 本机性能对比
Go 1.27.1 / darwin arm64 / Apple M2 Max,关闭请求日志;每项 500ms,三轮取中位数。基线源代码为 `068e85132f74edccb7657629b162505d038154f7`(v0.20 发布后文档提交),使用相同的基准函数。
前七项使用内存 RoundTripper 测量包装/JSON 开销;Pool 项使用本地 TCP 服务;Headers 项是新增选项。原始记录:[修改前](httpcBaselineBenchmark.txt)、[修改后](httpcBenchmark.txt)。列内单位依次为 ns/op、B/op、allocs/op。
| 场景 | 修改前 | 修改后 |
| --- | ---: | ---: |
| Client/128B/decode | 1681 / 2861 / 25 | 1408 / 2348 / 22 |
| Client/128B/discard | 1226 / 2856 / 25 | 1070 / 1836 / 21 |
| Client/4KiB/decode | 7590 / 17599 / 34 | 7650 / 17085 / 31 |
| Client/4KiB/discard | 2775 / 13480 / 33 | 1130 / 1836 / 21 |
| Client/64KiB/decode | 82261 / 214437 / 41 | 83673 / 213925 / 38 |
| Client/64KiB/discard | 17397 / 148648 / 40 | 2285 / 1836 / 21 |
| Client/Do | 843.6 / 1808 / 20 | 712.1 / 1296 / 17 |
| Pool/OK | 39726 / 7338 / 79 | 39598 / 6372 / 75 |
| Pool/Unauthorized | 42912 / 7333 / 83 | 41195 / 6503 / 77 |
| Headers | 新增能力 | 1761 / 3059 / 30 |
64 KiB discard 的分配从约 149 KB 降到 1.8 KB,不再为丢弃响应整体 ReadAll;普通 Do 少一次请求复制,也不在成功路径计算退避。大 JSON 解码耗时基本持平(4 KiB 约 +0.8%,64 KiB 约 +1.7%),分配略减;本轮没有更换 JSON 实现,也不宣称每个场景都更快。Headers 有独立快照/请求复制成本,见新增场景。
这些数字不是公网吞吐、冷 DNS、TLS 建连或高压容量承诺。连接容量与超时用确定性本地场景验证,未对线上资源施压。
## 兼容与使用
原 JSON 直接调用保持有效;自定义 interface/函数值中的 JSON 方法签名须补 variadic RequestOption。过去自定义 Transport 下被忽略的 MaxConnsPerHost,现在与其他代理/阶段选项一起明确拒绝混配;应配置借用 Transport 本身。
无新增第三方依赖。出站实现仍是标准 net/http,Do/Transport 保持通用扩展;HTTP 状态与业务响应码由调用项目解释。完整 [httpc 指南](../httpc/README.md)、[运行示例](../examples/httpClient/README.md)。远程固定版本安装验证在发布后追加。

View File

@@ -0,0 +1,33 @@
goos: darwin
goarch: arm64
pkg: git.yuntuo.hk/k2safe/go-omni/httpc
cpu: Apple M2 Max
BenchmarkHTTPClient/128B/decode-12 386178 2005 ns/op 2861 B/op 25 allocs/op
BenchmarkHTTPClient/128B/decode-12 382789 1681 ns/op 2861 B/op 25 allocs/op
BenchmarkHTTPClient/128B/decode-12 374196 1561 ns/op 2861 B/op 25 allocs/op
BenchmarkHTTPClient/128B/discard-12 453388 1243 ns/op 2856 B/op 25 allocs/op
BenchmarkHTTPClient/128B/discard-12 512002 1226 ns/op 2856 B/op 25 allocs/op
BenchmarkHTTPClient/128B/discard-12 511116 1221 ns/op 2856 B/op 25 allocs/op
BenchmarkHTTPClient/4KiB/decode-12 79958 7905 ns/op 17599 B/op 34 allocs/op
BenchmarkHTTPClient/4KiB/decode-12 82110 7590 ns/op 17599 B/op 34 allocs/op
BenchmarkHTTPClient/4KiB/decode-12 81975 7333 ns/op 17598 B/op 34 allocs/op
BenchmarkHTTPClient/4KiB/discard-12 234456 2796 ns/op 13480 B/op 33 allocs/op
BenchmarkHTTPClient/4KiB/discard-12 216720 2764 ns/op 13480 B/op 33 allocs/op
BenchmarkHTTPClient/4KiB/discard-12 225117 2775 ns/op 13480 B/op 33 allocs/op
BenchmarkHTTPClient/64KiB/decode-12 7014 82261 ns/op 214441 B/op 41 allocs/op
BenchmarkHTTPClient/64KiB/decode-12 7381 82513 ns/op 214437 B/op 41 allocs/op
BenchmarkHTTPClient/64KiB/decode-12 7546 81872 ns/op 214430 B/op 41 allocs/op
BenchmarkHTTPClient/64KiB/discard-12 36520 17397 ns/op 148648 B/op 40 allocs/op
BenchmarkHTTPClient/64KiB/discard-12 35896 17424 ns/op 148649 B/op 40 allocs/op
BenchmarkHTTPClient/64KiB/discard-12 33877 17195 ns/op 148648 B/op 40 allocs/op
BenchmarkHTTPClient/Do-12 640717 846.8 ns/op 1808 B/op 20 allocs/op
BenchmarkHTTPClient/Do-12 738248 843.6 ns/op 1808 B/op 20 allocs/op
BenchmarkHTTPClient/Do-12 695224 830.3 ns/op 1808 B/op 20 allocs/op
BenchmarkHTTPPool/OK-12 15036 40489 ns/op 7338 B/op 79 allocs/op
BenchmarkHTTPPool/OK-12 15148 39087 ns/op 7337 B/op 79 allocs/op
BenchmarkHTTPPool/OK-12 14715 39726 ns/op 7338 B/op 79 allocs/op
BenchmarkHTTPPool/Unauthorized-12 13894 42471 ns/op 7333 B/op 83 allocs/op
BenchmarkHTTPPool/Unauthorized-12 14020 43046 ns/op 7351 B/op 83 allocs/op
BenchmarkHTTPPool/Unauthorized-12 13992 42912 ns/op 7328 B/op 83 allocs/op
PASS
ok git.yuntuo.hk/k2safe/go-omni/httpc 20.475s

View File

@@ -0,0 +1,36 @@
goos: darwin
goarch: arm64
pkg: git.yuntuo.hk/k2safe/go-omni/httpc
cpu: Apple M2 Max
BenchmarkHTTPClient/128B/decode-12 403567 1408 ns/op 2348 B/op 22 allocs/op
BenchmarkHTTPClient/128B/decode-12 462315 1400 ns/op 2348 B/op 22 allocs/op
BenchmarkHTTPClient/128B/decode-12 416222 1427 ns/op 2348 B/op 22 allocs/op
BenchmarkHTTPClient/128B/discard-12 595370 1041 ns/op 1837 B/op 21 allocs/op
BenchmarkHTTPClient/128B/discard-12 583654 1070 ns/op 1836 B/op 21 allocs/op
BenchmarkHTTPClient/128B/discard-12 550502 1086 ns/op 1836 B/op 21 allocs/op
BenchmarkHTTPClient/4KiB/decode-12 77176 7547 ns/op 17084 B/op 31 allocs/op
BenchmarkHTTPClient/4KiB/decode-12 81969 7677 ns/op 17085 B/op 31 allocs/op
BenchmarkHTTPClient/4KiB/decode-12 77571 7650 ns/op 17087 B/op 31 allocs/op
BenchmarkHTTPClient/4KiB/discard-12 544194 1132 ns/op 1836 B/op 21 allocs/op
BenchmarkHTTPClient/4KiB/discard-12 539278 1128 ns/op 1836 B/op 21 allocs/op
BenchmarkHTTPClient/4KiB/discard-12 522939 1130 ns/op 1836 B/op 21 allocs/op
BenchmarkHTTPClient/64KiB/decode-12 6986 84111 ns/op 213930 B/op 38 allocs/op
BenchmarkHTTPClient/64KiB/decode-12 7156 83673 ns/op 213925 B/op 38 allocs/op
BenchmarkHTTPClient/64KiB/decode-12 7516 83041 ns/op 213921 B/op 38 allocs/op
BenchmarkHTTPClient/64KiB/discard-12 270482 2285 ns/op 1836 B/op 21 allocs/op
BenchmarkHTTPClient/64KiB/discard-12 253564 2259 ns/op 1835 B/op 21 allocs/op
BenchmarkHTTPClient/64KiB/discard-12 254343 2290 ns/op 1836 B/op 21 allocs/op
BenchmarkHTTPClient/Do-12 857305 713.4 ns/op 1296 B/op 17 allocs/op
BenchmarkHTTPClient/Do-12 811660 712.1 ns/op 1296 B/op 17 allocs/op
BenchmarkHTTPClient/Do-12 861867 708.3 ns/op 1296 B/op 17 allocs/op
BenchmarkHTTPPool/OK-12 14582 40133 ns/op 6374 B/op 75 allocs/op
BenchmarkHTTPPool/OK-12 15596 38152 ns/op 6372 B/op 75 allocs/op
BenchmarkHTTPPool/OK-12 15513 39598 ns/op 6369 B/op 75 allocs/op
BenchmarkHTTPPool/Unauthorized-12 14952 41483 ns/op 6503 B/op 77 allocs/op
BenchmarkHTTPPool/Unauthorized-12 14442 40927 ns/op 6516 B/op 77 allocs/op
BenchmarkHTTPPool/Unauthorized-12 14697 41195 ns/op 6498 B/op 77 allocs/op
BenchmarkHTTPHeaders-12 319768 1761 ns/op 3059 B/op 30 allocs/op
BenchmarkHTTPHeaders-12 334965 1731 ns/op 3059 B/op 30 allocs/op
BenchmarkHTTPHeaders-12 348751 1774 ns/op 3059 B/op 30 allocs/op
PASS
ok git.yuntuo.hk/k2safe/go-omni/httpc 21.809s

View File

@@ -124,7 +124,7 @@ def main():
checks += ["sharedGoCatalogExport", "documentOnlyErrorSelection", "workspaceCheckNoWrites", "staleCatalogRejected", "twoIndependentServiceBuilds"] checks += ["sharedGoCatalogExport", "documentOnlyErrorSelection", "workspaceCheckNoWrites", "staleCatalogRejected", "twoIndependentServiceBuilds"]
env.update(GO_OMNI_SCAFFOLD_INTEGRATION="1", GO_OMNI_RELEASE_CLI=str(cli), GO_OMNI_RELEASE_VERSION=args.version) env.update(GO_OMNI_SCAFFOLD_INTEGRATION="1", GO_OMNI_RELEASE_CLI=str(cli), GO_OMNI_RELEASE_VERSION=args.version)
tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer", "TestCompositionMappingConsumer", "TestNullableMappingConsumer"] tests = ["TestManagedGeneratedProject", "TestErrorInheritanceConsumer", "TestErrorParamsConsumer", "TestCompatibilityConsumer", "TestBusinessGeneratedProject", "TestPrimitiveArrayConsumer", "TestCopyxThreeEndpointConsumer", "TestGeneratedConfigurationConsumer"]
print("Running generated consumers against the remote framework, including lifecycle failure cleanup", flush=True) print("Running generated consumers against the remote framework, including lifecycle failure cleanup", flush=True)
output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=240s", "./cmd/go-omni/apigen", "-run", "^(" + "|".join(tests) + ")$"], ROOT, env) output = run(["go", "test", "-race", "-count=1", "-json", "-timeout=240s", "./cmd/go-omni/apigen", "-run", "^(" + "|".join(tests) + ")$"], ROOT, env)
events = [json.loads(line) for line in output.splitlines() if line.startswith("{")] events = [json.loads(line) for line in output.splitlines() if line.startswith("{")]
@@ -135,19 +135,33 @@ def main():
assert len(sums) == len(tests) and all(s == dependency["Sum"] for s in sums), "remote consumer checksum mismatch" assert len(sums) == len(tests) and all(s == dependency["Sum"] for s in sums), "remote consumer checksum mismatch"
checks += ["initializationAndListenFailureCleanup", "drainBeforeDependencyClose", "generatedConsumerRaceTests", "errorParamsAndInheritance", "manualHooksAndScalars", "dependencyFatalShutdown", "fatalProcessExitCode", "initializationShutdownNotification", "configuredBuiltinEndpoints", "projectHealthEnvelope", "endpointConflictCleanup", "responseArrayNullableNull", "primitiveAndDynamicArrays", "strictRequestPolicies", "declarativeMultipartBinding", "backgroundStopDrainResourceOrder", "backgroundInitAndListenFailureCleanup", "backgroundTimeoutRetainsResourcesAndExits"] checks += ["initializationAndListenFailureCleanup", "drainBeforeDependencyClose", "generatedConsumerRaceTests", "errorParamsAndInheritance", "manualHooksAndScalars", "dependencyFatalShutdown", "fatalProcessExitCode", "initializationShutdownNotification", "configuredBuiltinEndpoints", "projectHealthEnvelope", "endpointConflictCleanup", "responseArrayNullableNull", "primitiveAndDynamicArrays", "strictRequestPolicies", "declarativeMultipartBinding", "backgroundStopDrainResourceOrder", "backgroundInitAndListenFailureCleanup", "backgroundTimeoutRetainsResourcesAndExits"]
checks += ["flatFieldComposition", "typedStructureMapping", "explicitNullAndMoneyPolicies", "mappingFailureBeforeWrites", "mappingConsumerRaceTests", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"] # Run tests from the downloaded module, not the framework checkout.
http_output = run(["go", "test", "-race", "-count=1", "-json", MODULE + "/httpc"], starter, env)
http_events = [json.loads(line) for line in http_output.splitlines() if line.startswith("{")]
http_passed = {e.get("Test") for e in http_events if e.get("Action") == "pass"}
assert {"TestProxyRouting", "TestJSONHeadersConcurrentSnapshot", "TestPoolLimitsAndIdleExpiry",
"TestErrorBodyPoolReuseAndDrainBound", "TestSlowKnownErrorBodyHasShortDrainBudget"} <= http_passed
example_output = run(["go", "run", MODULE + "/examples/httpClient"], starter, env)
assert "authenticated request succeeded" in example_output
checks += ["remoteHTTPClientRaceTests", "remoteHTTPProxyAndTimeouts", "remoteHTTPHeaderIsolation",
"remoteHTTPPoolAndDrainBudgets", "remoteHTTPExample"]
checks += ["nullableNestedMappings", "splitMappingFiles", "explicitNilPolicies", "pointerResultHelpers", "mappingWorkspaceRegeneration", "nestedErrorChainAndPrivateCause", "mappingPreviewNoWrites"] checks += ["flatFieldComposition", "copyxThreeEndpointConsumer", "explicitNullAndMoneyPolicies", "fieldAdditionWithoutCopyRuleChanges", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"]
checks += ["generatedUnifiedConfig", "jsonYamlXmlAndCustomDecoder", "configValidationBeforeDependencies", "configEnvProvenance", "configExtensionPreserved", "configLoaderDriftCheck"]
sources = ["lifecycle/runtime.go", "lifecycle/phases.go", "config/lifecycle_test.go", "task/task.go", "cmd/go-omni/scaffold/templates/api-runtime.go.tmpl", "cmd/go-omni/apigen/templates.go", "config/project.go", "contract/bind.go", "contract/policy.go", "contract/multipart.go", "contract/file.go", "cmd/go-omni/apigen/business_test.go", "cmd/go-omni/apigen/generate.go", "lifecycle/controller.go", "config/endpoints.go", "health/health.go", "httpx/response.go", "app/app.go", "app/transport.go", "scripts/verifyManagedRelease.py", "scripts/verifyManagedProject.py", "cmd/go-omni/apigen/managed_test.go"] sources = ["lifecycle/runtime.go", "lifecycle/phases.go", "config/lifecycle_test.go", "task/task.go", "cmd/go-omni/scaffold/templates/api-runtime.go.tmpl", "cmd/go-omni/apigen/templates.go", "config/project.go", "contract/bind.go", "contract/policy.go", "contract/multipart.go", "contract/file.go", "cmd/go-omni/apigen/business_test.go", "cmd/go-omni/apigen/generate.go", "lifecycle/controller.go", "config/endpoints.go", "health/health.go", "httpx/response.go", "app/app.go", "app/transport.go", "scripts/verifyManagedRelease.py", "scripts/verifyManagedProject.py", "cmd/go-omni/apigen/managed_test.go"]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/managed").glob("*.go")] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/managed").glob("*.go")]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/business").glob("*.go")] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/business").glob("*.go")]
sources += ["cmd/go-omni/apigen/responseShape.go", "cmd/go-omni/apigen/documentRefs.go", "cmd/go-omni/apigen/catalogDocument.go", "cmd/go-omni/apigen/workspaceCatalog.go", "config/env.go", "cmd/go-omni/apigen/arrayResponse_test.go"] sources += ["cmd/go-omni/apigen/responseShape.go", "cmd/go-omni/apigen/documentRefs.go", "cmd/go-omni/apigen/catalogDocument.go", "cmd/go-omni/apigen/workspaceCatalog.go", "config/env.go", "cmd/go-omni/apigen/arrayResponse_test.go"]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/arrays").glob("*.go")] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/arrays").glob("*.go")]
sources += ["cmd/go-omni/apigen/" + name for name in ("composition.go", "generatorConfig.go", "mapping.go", "mappingLoad.go", "mapping_test.go", "middlewareFactory.go")] sources += ["cmd/go-omni/apigen/" + name for name in ("composition.go", "generatorConfig.go", "copyConsumer_test.go", "generatorConfig_test.go", "middlewareFactory.go")]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/composition").iterdir() if p.is_file()] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/composition").iterdir() if p.is_file()]
sources += ["cmd/go-omni/apigen/" + name for name in ("mappingPointer.go", "mappingNullable_test.go", "generatorConfig_test.go")] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "copyx").glob("*.go")]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "cmd/go-omni/apigen/testdata/nullable").iterdir() if p.is_file()] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "config").glob("*.go")]
sources += ["cmd/go-omni/apigen/configLoader_test.go", "cmd/go-omni/apigen/testdata/configuration/consumer_test.go", "cmd/go-omni/scaffold/templates/api-load.go.tmpl", "cmd/go-omni/scaffold/templates/api-loader.go.tmpl"]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "httpc").glob("*.go")]
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "examples/httpClient").glob("*.go")]
report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"], report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"],
"freshModuleCache": True, "noLocalReplace": True, "freshModuleCache": True, "noLocalReplace": True,
"durationSeconds": round(time.monotonic() - started, 2), "checks": checks, "durationSeconds": round(time.monotonic() - started, 2), "checks": checks,