feat: add precise diagnostics and optional client copy logging controls
This commit is contained in:
@@ -1,5 +1,13 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## v0.23.0-preview(2026-09-26)
|
||||||
|
|
||||||
|
- MySQL/连接池、日志、HTTP 客户端、生命周期和请求策略返回字段级安全诊断,配置加载器保留文件/变量/位置与原始错误链。
|
||||||
|
- httpc.JSON 新增 WithSuccessStatuses,单次调用可限制允许的 2xx,默认兼容,拒绝响应继续关闭和有界回收。
|
||||||
|
- copyx.Check[S,T] / Copier.Check 预检嵌套类型与现有规则;修正空集合隐藏类型不匹配,动态接口和 converter 的具体行为继续运行时验证。
|
||||||
|
- logx.Config.TimeZone 绑定 Logger 实例,JSON/text/子 Logger/标准日志适配一致;不修改 time.Local 或业务时间属性。
|
||||||
|
- 无新映射文件、强制模块或第三方依赖;更新 README、各包指南和 developerTools 可运行示例。错误文本更精确,空集合中的确定类型错误现在会被拒绝。
|
||||||
|
|
||||||
## v0.22.0-preview(2026-09-26)
|
## v0.22.0-preview(2026-09-26)
|
||||||
|
|
||||||
- 文件显式环境引用与备用值,字符串组合/转义,安全有界展开和类型绑定;变量值原样插入、不递归处理。
|
- 文件显式环境引用与备用值,字符串组合/转义,安全有界展开和类型绑定;变量值原样插入、不递归处理。
|
||||||
|
|||||||
14
CONFIG.md
14
CONFIG.md
@@ -141,6 +141,20 @@ config "etc/config.yaml":7:9 env "DB_PORT" field "database.port": expected a val
|
|||||||
|
|
||||||
文件最多 1 MiB;单个环境值/展开字段最多 1 MiB,展开字符串合计/变量缓存分别最多 4 MiB;解析/展开限制 64 层、100000 个节点与引用计数。拒绝无效 UTF-8、多文档、重复字段、未知字段、非普通文件;YAML 不接受 alias/merge/custom tag,XML 不接受 DTD/外部实体。不执行 shell,不自动加载 `.env`。
|
文件最多 1 MiB;单个环境值/展开字段最多 1 MiB,展开字符串合计/变量缓存分别最多 4 MiB;解析/展开限制 64 层、100000 个节点与引用计数。拒绝无效 UTF-8、多文档、重复字段、未知字段、非普通文件;YAML 不接受 alias/merge/custom tag,XML 不接受 DTD/外部实体。不执行 shell,不自动加载 `.env`。
|
||||||
|
|
||||||
|
## 框架模块的字段诊断
|
||||||
|
|
||||||
|
v0.23 起 MySQL(含连接池/TLS)、通用数据库池、日志(含轮转/时区)、HTTP 客户端、退出阶段和请求策略返回字段级安全原因。沿用项目已有的 `config.AtField("database", err)` / `config.AtField("server", err)`,自动组合嵌套字段,不需业务重复校验基础参数。
|
||||||
|
|
||||||
|
例如文件中 `database.port: "${DB_PORT}"` 的引用值为 70000:
|
||||||
|
|
||||||
|
```text
|
||||||
|
config "etc/config.yaml":3:9 env "DB_PORT" field "database.port": must be between 1 and 65535
|
||||||
|
```
|
||||||
|
|
||||||
|
日志级别非法指向 `server.log.level`;池参数指向 `database.pool.maxIdleConns` / `pingTimeout`。框架只公开已定义的安全原因,第三方/项目普通 error 仍显示 validation failed 并保留 errors.Is/As 原因链;业务使用 Invalid/AtField 提供公开原因。不要解析错误字符串;使用 `*config.Error` 字段。未提供字段的位置可能不可用,字段与原因仍保留。
|
||||||
|
|
||||||
|
生成项目现有 Load 和 `-check-config` 自动获得这些诊断。日志可在 `server.log.timeZone` 配置 IANA 名称,见 [实例时区](logx/README.md#实例时区)。
|
||||||
|
|
||||||
## 可选解析器与测试来源
|
## 可选解析器与测试来源
|
||||||
|
|
||||||
内置格式无需项目手写加载器。需要额外格式时在已有 NewLoader 注册 Decoder,后续仍自动经过同一引用/绑定/校验流程:
|
内置格式无需项目手写加载器。需要额外格式时在已有 NewLoader 注册 Decoder,后续仍自动经过同一引用/绑定/校验流程:
|
||||||
|
|||||||
25
COPY.md
25
COPY.md
@@ -28,6 +28,31 @@ if err := copyx.Copy(&input, req); err != nil { return err }
|
|||||||
|
|
||||||
新字段:普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时,两边各声明一次;同名同类型不增加拷贝配置。检查在运行时发生,实际类型对必须有测试,不能把运行时工具当作旧生成器的编译期字段覆盖检查。
|
新字段:普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时,两边各声明一次;同名同类型不增加拷贝配置。检查在运行时发生,实际类型对必须有测试,不能把运行时工具当作旧生成器的编译期字段覆盖检查。
|
||||||
|
|
||||||
|
## 类型预检
|
||||||
|
|
||||||
|
v0.23 新增可选预检,无需填写映射文件或构造测试值:
|
||||||
|
|
||||||
|
```go
|
||||||
|
// 参数方向是源 → 目标;与 Copy 使用同一组选项。
|
||||||
|
if err := copyx.Check[Request, BusinessInput](options...); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// 复用已有 Copier:传结果类型,不添加 Copy 所需的额外输出指针。
|
||||||
|
err := copier.Check(reflect.TypeFor[Request](), reflect.TypeFor[BusinessInput]())
|
||||||
|
```
|
||||||
|
|
||||||
|
适合 CI 测试或初始化检查;这是程序执行的类型检查,不是 Go 编译器自动证明。递归检查切片/数组元素、map 键值、指针、嵌套字段、改名、忽略和严格覆盖;用已访问类型对处理递归结构。不执行 converter,也不调用构造器,不读取实例值。错误路径如 `$.Rows[].Value: copyx: incompatible types: string -> int`。
|
||||||
|
|
||||||
|
Check 复用 Copy 的规则,包括浅拷贝对同类型容器的直接赋值:这种情况下内部 converter 不会自动逐项执行。需要处理内部字段时选择深拷贝或容器 converter。
|
||||||
|
|
||||||
|
- 未知具体值的源 interface 需要运行时验证;无法静态判定时返回 `ErrDynamicType`。项目可给该接口类型对提供精确 converter 并承担内容验证。
|
||||||
|
- 已注册 converter 的签名被接受,但函数不在 Check 中执行;其返回错误、panic、金额规则不能靠预检证明。
|
||||||
|
- nil 到非可空目标、map 键转换碰撞、深拷贝结果是否残留源引用继续在 Copy 时检查。Check 成功不会关闭这些保护。
|
||||||
|
- 实际 Copy 也修正了空集合的检查缺口:确定的嵌套类型不匹配不再因为切片/map 为空或数组长度为 0 而通过。需要这些转换时提供显式 converter;目标仍只在整体成功后更新。
|
||||||
|
- 不引入全局配置、映射语言或业务类型;Copier.Check 可并发调用,不共享每次检查的遍历状态。
|
||||||
|
|
||||||
|
示例:[developerTools](examples/developerTools/README.md)。普通同名同类型字段不需要额外配置。
|
||||||
|
|
||||||
## 深拷贝与构造器
|
## 深拷贝与构造器
|
||||||
|
|
||||||
```go
|
```go
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
# 通用业务开发指南(v0.22.0-preview)
|
# 通用业务开发指南(v0.23.0-preview)
|
||||||
|
|
||||||
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
|
从 [README](README.md) 创建服务后,业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力;字段、错误码、权限、表结构和事务边界都由项目定义。
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
# 生成项目与文件维护边界
|
# 生成项目与文件维护边界
|
||||||
|
|
||||||
v0.22.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
v0.23.0-preview 提供受管理的生成项目。安装同版远程 CLI,生成的项目直接引用该版本,无需本地 replace。根目录入口支持 `go run .` 和 `go build .`;框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
||||||
|
|
||||||
## 创建和日常开发
|
## 创建和日常开发
|
||||||
|
|
||||||
@@ -215,7 +215,7 @@ get /download (DownloadReq)
|
|||||||
|
|
||||||
## 已有服务接入步骤
|
## 已有服务接入步骤
|
||||||
|
|
||||||
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.22.0-preview:
|
每个独立服务生成一个项目,module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.23.0-preview:
|
||||||
|
|
||||||
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
|
1. 先保留旧服务,整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`;不对旧入口强制覆盖。
|
||||||
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。
|
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`,显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。
|
||||||
|
|||||||
38
README.md
38
README.md
@@ -2,7 +2,9 @@
|
|||||||
|
|
||||||
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
|
GoOmni 是面向多个业务项目复用的 Go 后端框架:用 `.api` 定义接口、请求和返回类型,生成项目目录、HTTP 接入代码及 OpenAPI,业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`,或单独引入日志、存储等模块。
|
||||||
|
|
||||||
**从这份 README 开始接入。** 当前版本 **`v0.22.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
|
**从这份 README 开始接入。** 当前版本 **`v0.23.0-preview`**,新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入;文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md),版本变化与验收见 [发布说明](RELEASE.md)。
|
||||||
|
|
||||||
|
**v0.23 改动**:配置校验定位具体字段和安全原因;`httpc.JSON` 可选限制成功状态;`copyx.Check` 无需样本预检类型;`logx` 支持实例独立时区。[四项完整示例](examples/developerTools/README.md)。
|
||||||
|
|
||||||
**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。
|
**v0.19 改动**:新增可选 `copyx` 工具,用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换;深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**,已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整,不能直接删除仍在调用的旧文件。
|
||||||
|
|
||||||
@@ -37,21 +39,21 @@ Go **1.22+**。框架不依赖 Gin,默认使用标准库网络层,提供可
|
|||||||
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
|
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`;Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
|
||||||
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
|
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次;`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
|
||||||
| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
|
| OpenAPI 与错误公共引用 | `.api` 自动生成;手写接口可使用 `openapi` 注册器;Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
|
||||||
| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`;支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
|
| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`;支持独立时区、轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
|
||||||
| 文件配置与环境引用 | Default()→JSON/YAML/XML 字段引用→类型绑定→校验;无隐式覆盖,支持 `-check-config` | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
|
| 文件配置与环境引用 | Default()→JSON/YAML/XML 字段引用→类型绑定→校验;无隐式覆盖,支持 `-check-config` | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
|
||||||
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
|
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain,RequestShutdown 通知故障;健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
|
||||||
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
|
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`;事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
|
||||||
| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
|
| 可选 GORM 与迁移 | gormx 复用已有连接池;`go-omni migrate` 预览,显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
|
||||||
| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
|
| 认证与权限接口 | 注入 Authenticator / Policy;JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
|
||||||
| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
|
| 缓存 | `cache.NewMemory` 或 Redis 实现;业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
|
||||||
| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时;JSON 支持单次请求头/限额;`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
|
| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时;JSON 支持单次成功状态限制/请求头/限额;`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
|
||||||
| 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
|
| 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware;跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
|
||||||
| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
|
| 本地后台任务与调度 | `task.New`、Submit、Every;有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
|
||||||
| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
|
| 持久队列 | 业务使用 Publisher / Consumer;Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
|
||||||
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
|
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete;Local、S3/R2、OSS 按配置装配;扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
|
||||||
| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
|
| 日志关联、指标与链路 | `metrics.New().Middleware()`;telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
|
||||||
| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
|
| 审计与分页 | `audit.New` 注入 Sink/白名单;`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
|
||||||
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
|
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`;`Check[S,T]` 预检类型,Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
|
||||||
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
|
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
|
||||||
|
|
||||||
## 带认证的出站 HTTP
|
## 带认证的出站 HTTP
|
||||||
@@ -126,7 +128,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
|
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
|
||||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.22.0-preview
|
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview
|
||||||
export PATH="$(go env GOPATH)/bin:$PATH"
|
export PATH="$(go env GOPATH)/bin:$PATH"
|
||||||
go-omni --help
|
go-omni --help
|
||||||
```
|
```
|
||||||
@@ -283,7 +285,7 @@ go-omni new --api catalog.api \
|
|||||||
在业务项目的 **go.mod 所在目录**执行:
|
在业务项目的 **go.mod 所在目录**执行:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go get git.yuntuo.hk/k2safe/go-omni@v0.22.0-preview
|
go get git.yuntuo.hk/k2safe/go-omni@v0.23.0-preview
|
||||||
```
|
```
|
||||||
|
|
||||||
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
|
已有项目可直接复用现有目录、Service、Repository 和数据库连接,先选择一个接口接入。也可以只用 `logx`、`storage` 等包,暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目,按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
|
||||||
@@ -509,6 +511,28 @@ if err := copier.Copy(&view, input); err != nil { return err }
|
|||||||
|
|
||||||
自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错;字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md),完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查,类型对需测试;深拷贝不适用于连接、锁等资源。
|
自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错;字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md),完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查,类型对需测试;深拷贝不适用于连接、锁等资源。
|
||||||
|
|
||||||
|
### 减少接入代码:四个常用入口
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
log:
|
||||||
|
timeZone: Asia/Shanghai
|
||||||
|
```
|
||||||
|
|
||||||
|
时区由 Logger 实例管理,不修改全局 `time.Local`。非法日志级别、数据库端口/连接池等配置会指出 `server.log.level`、`database.port` 等字段和原因,不打印输入值。详见 [配置诊断](CONFIG.md#框架模块的字段诊断)、[日志时区](logx/README.md#实例时区)。
|
||||||
|
|
||||||
|
```go
|
||||||
|
// 本次请求只接受 HTTP 200;默认仍接受所有 2xx。
|
||||||
|
err := client.JSON(ctx, http.MethodGet, url, nil, &result,
|
||||||
|
httpc.WithSuccessStatuses(http.StatusOK),
|
||||||
|
)
|
||||||
|
|
||||||
|
// CI/启动时检查类型,无需构造样本;仍使用 Copy 的同一组 Option。
|
||||||
|
err = copyx.Check[Request, BusinessInput](options...)
|
||||||
|
```
|
||||||
|
|
||||||
|
状态不符合要求返回 `*httpc.StatusError`,不会写入 result,响应体仍由框架关闭。Check 不执行 converter;动态 interface、具体 nil/转换失败和深拷贝引用检查有明确边界,见 [类型预检](COPY.md#类型预检)。完整可运行代码见 [developerTools](examples/developerTools/README.md)。
|
||||||
|
|
||||||
### 数据库、存储及其他模块
|
### 数据库、存储及其他模块
|
||||||
|
|
||||||
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
|
生成项目在 `internal/svc` 中创建依赖,提供给 Logic/Repository,并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc,再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`;不要对同一资源重复登记。
|
||||||
@@ -587,7 +611,7 @@ go test ./...
|
|||||||
|
|
||||||
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
|
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理,手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目,重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段,明确的签名错误会给出原签名与期望签名;生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交,CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
|
||||||
|
|
||||||
CLI 与业务 module 固定使用 `v0.22.0-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load,再升级依赖与 CLI 并重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
CLI 与业务 module 固定使用 `v0.23.0-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load,再升级依赖与 CLI 并重新生成,框架维护的 main/runtimeGen 自动更新;v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理,不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移),旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
||||||
|
|
||||||
## 按需深入
|
## 按需深入
|
||||||
|
|
||||||
|
|||||||
25
RELEASE.md
25
RELEASE.md
@@ -1,4 +1,27 @@
|
|||||||
# v0.22.0-preview
|
# v0.23.0-preview
|
||||||
|
|
||||||
|
减少业务接入时重复维护的配置校验、HTTP 包装和类型检查。
|
||||||
|
|
||||||
|
- 配置诊断定位 database.port、database.pool.maxIdleConns、server.log.level/timeZone 等具体字段,保留文件、环境变量、行列和安全原因;凭据不进入公开提示,原始原因可 errors.Is/As。
|
||||||
|
- JSON 可传 `httpc.WithSuccessStatuses(200)`,默认仍接受全部 2xx;不匹配时不改目标,继续负责有界读取、关闭和连接回收,不添加新的重试规则。
|
||||||
|
- `copyx.Check[S,T](options...)` 或已有 Copier.Check 复用同一组 Go 选项,无需样本和映射文件;实际拷贝不再因集合为空而漏过确定的类型不匹配。动态 interface、nil 值、converter 失败及深拷贝共享引用仍有运行时检查。
|
||||||
|
- `server.log.timeZone: Asia/Shanghai` 配置实例时区;JSON/text/子日志一致,业务 time.Time 属性保持原样。极简镜像可用 `go build -tags timetzdata .` 内置时区数据,库不强制增加体积。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go get git.yuntuo.hk/k2safe/go-omni@v0.23.0-preview
|
||||||
|
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview
|
||||||
|
go-omni api generate
|
||||||
|
go-omni api generate --check
|
||||||
|
go run . -config etc/config.yaml -check-config
|
||||||
|
```
|
||||||
|
|
||||||
|
现有调用接口保留;配置错误文本现在更精确,业务应通过类型/字段识别错误。空集合里确定的类型错误会提前暴露,需要显式 converter 或调整实际类型,不应靠空值跳过校验。预检通过不代表 converter 对所有业务值必定成功。当前模板最低版本仍是 v0.22,本轮新增 runtime 能力需使用 v0.23。
|
||||||
|
|
||||||
|
[完整示例](examples/developerTools/README.md)、[配置](CONFIG.md)、[JSON 选项](httpc/README.md)、[类型预检](COPY.md#类型预检)、[日志时区](logx/README.md#实例时区)。验收与性能记录见 [本轮验收](reports/developerToolsAcceptance.md)。仍为 preview。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
# v0.22.0-preview(历史版本)
|
||||||
|
|
||||||
配置文件明确决定每个值的来源,项目只维护 Config、Default()、Validate() 和文件。
|
配置文件明确决定每个值的来源,项目只维护 Config、Default()、Validate() 和文件。
|
||||||
|
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目
|
|||||||
|
|
||||||
```sh
|
```sh
|
||||||
# 仓库读取权限与 PATH 配置见根目录 README
|
# 仓库读取权限与 PATH 配置见根目录 README
|
||||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.22.0-preview
|
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview
|
||||||
go-omni new --module example.com/hello ./hello
|
go-omni new --module example.com/hello ./hello
|
||||||
|
|
||||||
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
|
# 从自己的契约生成;service.api 及其相对 import 文件需要在本地存在
|
||||||
@@ -49,14 +49,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护;业务文件按字节
|
|||||||
|
|
||||||
## 版本与迁移
|
## 版本与迁移
|
||||||
|
|
||||||
CLI 与框架依赖固定使用 v0.22.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
|
CLI 与框架依赖固定使用 v0.23.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod,不添加本地 replace;`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
# 在 go-omni 源码仓库中执行
|
# 在 go-omni 源码仓库中执行
|
||||||
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
|
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
|
||||||
```
|
```
|
||||||
|
|
||||||
当前模板与依赖以 v0.22.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
|
当前模板与依赖以 v0.23.0-preview 为基线,继续使用 manifest v2;GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级):原始模板可自动替换,含业务修改的基础文件按预览合并;通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务,不使用旧 `add` 命令的手动路由装配。
|
||||||
|
|
||||||
## 数据库迁移命令
|
## 数据库迁移命令
|
||||||
|
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
|||||||
|
|
||||||
// Runtime-only releases need not raise the managed template compatibility floor.
|
// Runtime-only releases need not raise the managed template compatibility floor.
|
||||||
const minimumVersion = "v0.22.0-preview"
|
const minimumVersion = "v0.22.0-preview"
|
||||||
const currentVersion = "v0.22.0-preview"
|
const currentVersion = "v0.23.0-preview"
|
||||||
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||||
|
|
||||||
type Options struct {
|
type Options struct {
|
||||||
|
|||||||
@@ -60,7 +60,9 @@ Unimplemented/nil factories fail startup and clean registered dependencies.
|
|||||||
Existing projects retain middleware: handlers until explicitly migrated.
|
Existing projects retain middleware: handlers until explicitly migrated.
|
||||||
Ordinary Logic uses the generated Request/Response types directly. Where an
|
Ordinary Logic uses the generated Request/Response types directly. Where an
|
||||||
independent business type is needed, use ordinary Go or the optional copyx package.
|
independent business type is needed, use ordinary Go or the optional copyx package.
|
||||||
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use explicit ${NAME} / ${NAME:-fallback} references; fixed values are never overridden by inferred environment names. Use -check-config to validate without opening dependencies. See the framework CONFIG.md.
|
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use explicit ${NAME} / ${NAME:-fallback} references; fixed values are never overridden by inferred environment names. Use -check-config to validate without opening dependencies. Framework validation
|
||||||
|
identifies fields such as server.log.level and database.port. server.log.timeZone
|
||||||
|
optionally selects a per-logger IANA zone. See the framework CONFIG.md.
|
||||||
|
|
||||||
There are no mappings/mappingFiles settings or generated mapping files in v0.19.
|
There are no mappings/mappingFiles settings or generated mapping files in v0.19.
|
||||||
See COPY.md in the framework repository for copy policies and migration steps,
|
See COPY.md in the framework repository for copy policies and migration steps,
|
||||||
|
|||||||
@@ -91,3 +91,23 @@ func TestConfigurationCheckProcess(t *testing.T) {
|
|||||||
t.Fatal(err, string(out))
|
t.Fatal(err, string(out))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestGeneratedModuleFieldDiagnostic(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||||
|
t.Setenv("EXAMPLE_LOG_LEVEL", "invalid-private-level")
|
||||||
|
if err := os.WriteFile(path, []byte("server:\n log:\n level: '${EXAMPLE_LOG_LEVEL}'\n"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
err := run(context.Background(), path)
|
||||||
|
var d *omniConfig.Error
|
||||||
|
if !errors.As(err, &d) || d.Field != "server.log.level" || d.Line != 3 || d.Env != "EXAMPLE_LOG_LEVEL" || !strings.Contains(d.Reason, "debug") || strings.Contains(err.Error(), "invalid-private-level") {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(path, []byte("server:\n log:\n timeZone: Asia/Shanghai\n"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg, err := config.Load(path)
|
||||||
|
if err != nil || cfg.Server.Log.TimeZone != "Asia/Shanghai" {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
|||||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||||
// downloaded module version, which also has a module checksum.
|
// downloaded module version, which also has a module checksum.
|
||||||
const defaultFrameworkVersion = "v0.22.0-preview"
|
const defaultFrameworkVersion = "v0.23.0-preview"
|
||||||
|
|
||||||
// This target is not a claim that the release already exists. Source users
|
// This target is not a claim that the release already exists. Source users
|
||||||
// select a local framework checkout until the module templates are published.
|
// select a local framework checkout until the module templates are published.
|
||||||
|
|||||||
@@ -4,6 +4,8 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Error is a value-free diagnostic. Cause may contain private parser details;
|
// Error is a value-free diagnostic. Cause may contain private parser details;
|
||||||
@@ -56,6 +58,10 @@ func validationError(err error) *Error {
|
|||||||
out.cause = err
|
out.cause = err
|
||||||
return &out
|
return &out
|
||||||
}
|
}
|
||||||
|
var nested *configdiag.Error
|
||||||
|
if errors.As(err, &nested) {
|
||||||
|
return &Error{Field: nested.Field, Reason: nested.Reason, cause: err}
|
||||||
|
}
|
||||||
var c *ConfigError
|
var c *ConfigError
|
||||||
if errors.As(err, &c) {
|
if errors.As(err, &c) {
|
||||||
return &Error{Field: c.Field, Reason: c.Message, cause: err}
|
return &Error{Field: c.Field, Reason: c.Message, cause: err}
|
||||||
|
|||||||
85
config/moduleDiagnostics_test.go
Normal file
85
config/moduleDiagnostics_test.go
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
package config_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||||
|
mysql "git.yuntuo.hk/k2safe/go-omni/database/mysql"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpc"
|
||||||
|
)
|
||||||
|
|
||||||
|
type moduleConfig struct {
|
||||||
|
Database mysql.Config `json:"database"`
|
||||||
|
Server config.Config `json:"server"`
|
||||||
|
Client httpc.Config `json:"client"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c moduleConfig) Validate() error {
|
||||||
|
if err := c.Database.Validate(); err != nil {
|
||||||
|
return config.AtField("database", err)
|
||||||
|
}
|
||||||
|
if err := c.Server.Validate(); err != nil {
|
||||||
|
return config.AtField("server", err)
|
||||||
|
}
|
||||||
|
return config.AtField("client", c.Client.Validate())
|
||||||
|
}
|
||||||
|
func TestModuleFieldDiagnostics(t *testing.T) {
|
||||||
|
for _, tc := range []struct{ name, source, field, reason string }{
|
||||||
|
{"port", "database:\n port: 70000\n", "database.port", "1 and 65535"},
|
||||||
|
{"host", "database:\n host: 'invalid host'\n", "database.host", "hostname"},
|
||||||
|
{"pool", "database:\n pool:\n maxIdleConns: 999\n", "database.pool.maxIdleConns", "maxOpenConns"},
|
||||||
|
{"ping", "database:\n pool:\n pingTimeout: 0s\n", "database.pool.pingTimeout", "positive"},
|
||||||
|
{"timeout", "database:\n readTimeout: -1s\n", "database.readTimeout", "nonnegative"},
|
||||||
|
{"mysqlZone", "database:\n location: Secret/Not_A_Zone\n", "database.location", "IANA"},
|
||||||
|
{"tlsPair", "database:\n tls:\n certFile: secret-file\n keyFile: ''\n", "database.tls.keyFile", "certFile"},
|
||||||
|
{"level", "server:\n log:\n level: secret-level\n", "server.log.level", "debug"},
|
||||||
|
{"format", "server:\n log:\n format: secret-format\n", "server.log.format", "json or text"},
|
||||||
|
{"rotation", "server:\n log:\n rotation:\n maxBackups: -1\n", "server.log.rotation.maxBackups", "10000"},
|
||||||
|
{"zone", "server:\n log:\n timeZone: Secret/Not_A_Zone\n", "server.log.timeZone", "IANA"},
|
||||||
|
{"lifecycle", "server:\n lifecycle:\n stopTimeout: -1s\n", "server.lifecycle.stopTimeout", "nonnegative"},
|
||||||
|
{"policy", "server:\n request:\n body: secret-mode\n", "server.request.body", "required"},
|
||||||
|
{"clientPool", "client:\n maxIdleConns: -1\n", "client.maxIdleConns", "131072"},
|
||||||
|
{"proxy", "client:\n proxy:\n mode: url\n url: 'https://private:secret@localhost/path'\n", "client.proxy.url", "origin"},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||||
|
if err := os.WriteFile(path, []byte(tc.source), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
_, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()})
|
||||||
|
var d *config.Error
|
||||||
|
if !errors.As(err, &d) || d.Field != tc.field || d.File != path || d.Line <= 1 || d.Column <= 0 || !strings.Contains(d.Reason, tc.reason) {
|
||||||
|
t.Fatalf("diagnostic: %v", err)
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), "secret") || strings.Contains(err.Error(), "Secret/") || strings.Contains(err.Error(), "private") {
|
||||||
|
t.Fatalf("value leaked: %v", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestModuleDiagnosticsKeepReferenceAndCause(t *testing.T) {
|
||||||
|
t.Setenv("CHECK_PORT", "70000")
|
||||||
|
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||||
|
os.WriteFile(path, []byte("database:\n port: '${CHECK_PORT}'\n"), 0600)
|
||||||
|
_, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()})
|
||||||
|
var d *config.Error
|
||||||
|
if !errors.As(err, &d) || d.Field != "database.port" || d.Env != "CHECK_PORT" || d.Line != 2 {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
original := mysql.DefaultConfig()
|
||||||
|
original.Port = 70000
|
||||||
|
cause := original.Validate()
|
||||||
|
wrapped := config.AtField("database", cause)
|
||||||
|
if !errors.Is(wrapped, cause) {
|
||||||
|
t.Fatal("lost original validation error")
|
||||||
|
}
|
||||||
|
// Unknown project errors continue to be redacted rather than echoed.
|
||||||
|
wrapped = config.AtField("custom", errors.New("private-password"))
|
||||||
|
if strings.Contains(wrapped.Error(), "private-password") {
|
||||||
|
t.Fatal(wrapped)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,9 +3,11 @@ package contract
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
|
||||||
"io"
|
"io"
|
||||||
|
"strings"
|
||||||
"unicode/utf8"
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RequestPolicy describes protocol strictness, independently of business
|
// RequestPolicy describes protocol strictness, independently of business
|
||||||
@@ -35,7 +37,7 @@ func (p RequestPolicy) Validate() error {
|
|||||||
ok = ok || value == item.value
|
ok = ok || value == item.value
|
||||||
}
|
}
|
||||||
if !ok {
|
if !ok {
|
||||||
return fmt.Errorf("request policy: invalid %s", item.name)
|
return configdiag.New(item.name, "must be one of "+strings.Join(item.allowed, ", "))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
|
|||||||
192
copyx/check.go
Normal file
192
copyx/check.go
Normal file
@@ -0,0 +1,192 @@
|
|||||||
|
package copyx
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"reflect"
|
||||||
|
"sort"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ErrDynamicType means a source interface requires a concrete runtime value.
|
||||||
|
// An exact converter can explicitly own that dynamic boundary.
|
||||||
|
var ErrDynamicType = errors.New("copyx: dynamic source type requires runtime validation or an exact converter")
|
||||||
|
|
||||||
|
// Check validates a source/target type pair using the same options as Copy.
|
||||||
|
// It never executes converters and requires no example values. Nil policies,
|
||||||
|
// converter failures, map-key collisions and deep reference isolation still
|
||||||
|
// require runtime checks. Options remain local; no mapping files or registration.
|
||||||
|
func Check[S, T any](options ...Option) error {
|
||||||
|
c, err := New(options...)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return c.Check(reflect.TypeFor[S](), reflect.TypeFor[T]())
|
||||||
|
}
|
||||||
|
|
||||||
|
// Check validates types with an existing Copier's immutable settings. The target
|
||||||
|
// is the result type (not Copy's extra output pointer). Safe for concurrent use.
|
||||||
|
func (c *Copier) Check(source, target reflect.Type) error {
|
||||||
|
if c == nil {
|
||||||
|
return errors.New("copyx: nil Copier")
|
||||||
|
}
|
||||||
|
if source == nil || target == nil {
|
||||||
|
return errors.New("copyx: source and target types are required")
|
||||||
|
}
|
||||||
|
return newTypeChecker(&c.options, false).check(source, target, "$", 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
type typeChecker struct {
|
||||||
|
options *settings
|
||||||
|
allowDynamic bool
|
||||||
|
seen map[pair]bool
|
||||||
|
nodes int
|
||||||
|
}
|
||||||
|
|
||||||
|
func newTypeChecker(options *settings, allowDynamic bool) *typeChecker {
|
||||||
|
return &typeChecker{options: options, allowDynamic: allowDynamic, seen: map[pair]bool{}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Absent elements can reveal type/field mismatches, but contain no actual
|
||||||
|
// resource handles or dynamic interface values to validate. Runtime checks stay
|
||||||
|
// responsible for those value-dependent cases.
|
||||||
|
func (s *state) checkAbsent(from, to reflect.Type, path string) error {
|
||||||
|
if s.checkedTypes == nil {
|
||||||
|
s.checkedTypes = map[pair]bool{}
|
||||||
|
}
|
||||||
|
checker := typeChecker{options: s.options, allowDynamic: true, seen: s.checkedTypes, nodes: s.checkedNodes}
|
||||||
|
err := checker.check(from, to, path, 0)
|
||||||
|
s.checkedNodes = checker.nodes
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
func (c *typeChecker) check(from, to reflect.Type, path string, depth int) error {
|
||||||
|
c.nodes++
|
||||||
|
if depth > maxDepth || c.nodes > maxNodes {
|
||||||
|
return at(path, ErrLimit)
|
||||||
|
}
|
||||||
|
key := pair{from, to}
|
||||||
|
if c.seen[key] {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
c.seen[key] = true
|
||||||
|
if !c.allowDynamic && !shallowSafe(to) {
|
||||||
|
return at(path, ErrUnsupported)
|
||||||
|
}
|
||||||
|
if _, ok := c.options.converters[key]; ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if from == to && len(c.options.converters) == 0 && len(c.options.fields) == 0 && !c.options.skipZero && c.options.nilPolicy != NilSkip {
|
||||||
|
if !c.options.deep || simpleValue(to) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if from.Kind() == reflect.Interface {
|
||||||
|
if c.allowDynamic {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return at(path, ErrDynamicType)
|
||||||
|
}
|
||||||
|
if to.Kind() == reflect.Interface {
|
||||||
|
if !from.Implements(to) {
|
||||||
|
return at(path, ErrType)
|
||||||
|
}
|
||||||
|
return c.check(from, from, path, depth+1)
|
||||||
|
}
|
||||||
|
if !c.allowDynamic && c.options.deep && forbidden(to) {
|
||||||
|
return at(path, ErrUnsupported)
|
||||||
|
}
|
||||||
|
if from.Kind() == reflect.Pointer && to.Kind() != reflect.Pointer {
|
||||||
|
return c.check(from.Elem(), to, path, depth+1)
|
||||||
|
}
|
||||||
|
if to.Kind() == reflect.Pointer {
|
||||||
|
if !c.options.deep && from == to {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if from.Kind() == reflect.Pointer {
|
||||||
|
from = from.Elem()
|
||||||
|
}
|
||||||
|
return c.check(from, to.Elem(), path, depth+1)
|
||||||
|
}
|
||||||
|
if from.Kind() != to.Kind() {
|
||||||
|
return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to))
|
||||||
|
}
|
||||||
|
if to == timeType && from == to {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
switch to.Kind() {
|
||||||
|
case reflect.Struct:
|
||||||
|
for i := 0; i < to.NumField(); i++ {
|
||||||
|
f := to.Field(i)
|
||||||
|
if !c.allowDynamic && f.PkgPath != "" && (from != to || c.options.deep && privateMutable(f.Type)) {
|
||||||
|
return at(path+"."+f.Name, ErrUnsupported)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sources, targets := exportedFields(from), exportedFields(to)
|
||||||
|
rules := c.options.fields[key]
|
||||||
|
if rules == nil {
|
||||||
|
rules = &fields{}
|
||||||
|
}
|
||||||
|
reverse, used := map[string]string{}, map[string]bool{}
|
||||||
|
for a, b := range rules.names {
|
||||||
|
reverse[b] = a
|
||||||
|
}
|
||||||
|
for _, name := range sortedFieldNames(targets) {
|
||||||
|
if rules.ignoreTarget[name] {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
sourceName := name
|
||||||
|
if n, ok := reverse[name]; ok {
|
||||||
|
sourceName = n
|
||||||
|
}
|
||||||
|
source, ok := sources[sourceName]
|
||||||
|
if !ok || rules.ignoreSource[sourceName] {
|
||||||
|
return at(path+"."+name, ErrField)
|
||||||
|
}
|
||||||
|
used[sourceName] = true
|
||||||
|
if err := c.check(source.Type, targets[name].Type, path+"."+name, depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if c.options.strict {
|
||||||
|
for _, name := range sortedFieldNames(sources) {
|
||||||
|
if !used[name] && !rules.ignoreSource[name] {
|
||||||
|
return at(path+"."+name, ErrField)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
case reflect.Slice:
|
||||||
|
if !c.options.deep && from == to {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return c.check(from.Elem(), to.Elem(), path+"[]", depth+1)
|
||||||
|
case reflect.Array:
|
||||||
|
if from.Len() != to.Len() {
|
||||||
|
return at(path, ErrType)
|
||||||
|
}
|
||||||
|
return c.check(from.Elem(), to.Elem(), path+"[]", depth+1)
|
||||||
|
case reflect.Map:
|
||||||
|
if !c.options.deep && from == to {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := c.check(from.Key(), to.Key(), path+"[key]", depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return c.check(from.Elem(), to.Elem(), path+"[value]", depth+1)
|
||||||
|
case reflect.Func, reflect.Chan, reflect.UnsafePointer:
|
||||||
|
if c.options.deep && !c.allowDynamic {
|
||||||
|
return at(path, ErrUnsupported)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if from != to {
|
||||||
|
return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func sortedFieldNames(fields map[string]reflect.StructField) []string {
|
||||||
|
names := make([]string, 0, len(fields))
|
||||||
|
for name := range fields {
|
||||||
|
names = append(names, name)
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
return names
|
||||||
|
}
|
||||||
192
copyx/check_test.go
Normal file
192
copyx/check_test.go
Normal file
@@ -0,0 +1,192 @@
|
|||||||
|
package copyx_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||||
|
)
|
||||||
|
|
||||||
|
type checkSource struct{ Rows []checkSourceRow }
|
||||||
|
type checkSourceRow struct{ Value string }
|
||||||
|
type checkTarget struct{ Rows []checkTargetRow }
|
||||||
|
type checkTargetRow struct{ Value int }
|
||||||
|
type recursiveSource struct {
|
||||||
|
Children []*recursiveSource
|
||||||
|
Value string
|
||||||
|
}
|
||||||
|
type recursiveTarget struct {
|
||||||
|
Children []*recursiveTarget
|
||||||
|
Value int
|
||||||
|
}
|
||||||
|
type recursiveCompatible struct {
|
||||||
|
Children []*recursiveCompatible
|
||||||
|
Value string
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCheckTypesAndEmptyCollections(t *testing.T) {
|
||||||
|
err := copyx.Check[checkSource, checkTarget]()
|
||||||
|
var field *copyx.Error
|
||||||
|
if !errors.Is(err, copyx.ErrType) || !errors.As(err, &field) || field.Path != "$.Rows[].Value" {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, rows := range [][]checkSourceRow{nil, {}, {{Value: "42"}}} {
|
||||||
|
out := checkTarget{Rows: []checkTargetRow{{Value: 99}}}
|
||||||
|
if err := copyx.Copy(&out, checkSource{rows}); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatalf("rows %v: %v", rows, err)
|
||||||
|
}
|
||||||
|
if len(out.Rows) != 1 || out.Rows[0].Value != 99 {
|
||||||
|
t.Fatal("partial target committed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := copyx.Check[map[string]checkSourceRow, map[string]checkTargetRow](); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var dst map[string]checkTargetRow
|
||||||
|
if err := copyx.Copy(&dst, map[string]checkSourceRow{}); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var array [0]checkTargetRow
|
||||||
|
if err := copyx.Copy(&array, [0]checkSourceRow{}); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[map[string]int, map[int]int](); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[[2]int, [3]int](); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[*recursiveSource, *recursiveTarget](); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[*recursiveSource, *recursiveCompatible](copyx.WithDeepCopy()); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestCheckRulesAndConverterBoundaries(t *testing.T) {
|
||||||
|
type from struct {
|
||||||
|
Name string
|
||||||
|
Extra bool
|
||||||
|
}
|
||||||
|
type to struct {
|
||||||
|
Label string
|
||||||
|
Keep int
|
||||||
|
}
|
||||||
|
options := []copyx.Option{copyx.WithStrictFields(), copyx.WithFieldNames[from, to](map[string]string{"Name": "Label"}), copyx.WithIgnoreSource[from, to]("Extra"), copyx.WithIgnoreTarget[from, to]("Keep")}
|
||||||
|
if err := copyx.Check[from, to](options...); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[from, to](); !errors.Is(err, copyx.ErrField) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
type short struct{ Name string }
|
||||||
|
if err := copyx.Check[from, short](copyx.WithStrictFields()); !errors.Is(err, copyx.ErrField) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[checkSource, checkTarget](copyx.WithConverter(func(string) (int, error) { panic("check must not execute converters") })); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[struct{ Value any }, struct{ Value int }](); !errors.Is(err, copyx.ErrDynamicType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[any, int](copyx.WithConverter(func(any) (int, error) { panic("not called") })); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[*int, int](); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
} // Actual nil remains a runtime error.
|
||||||
|
var src *int
|
||||||
|
dst := 1
|
||||||
|
if err := copyx.Copy(&dst, src); !errors.Is(err, copyx.ErrNil) || dst != 1 {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var empty []int
|
||||||
|
if err := copyx.Copy(&empty, []any{}); err != nil || empty == nil {
|
||||||
|
t.Fatal("dynamic emptiness changed", err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[sync.Mutex, sync.Mutex](); !errors.Is(err, copyx.ErrUnsupported) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
option := copyx.WithConverter(func(v []int) ([]int, error) { return v, nil })
|
||||||
|
if err := copyx.Check[[]int, []int](copyx.WithDeepCopy(), option); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := copyx.Clone([]int{1}, option); !errors.Is(err, copyx.ErrSharedReference) {
|
||||||
|
t.Fatal("type check bypassed graph audit", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestCopierCheckConcurrentAndOptionsIsolation(t *testing.T) {
|
||||||
|
good, _ := copyx.New(copyx.WithConverter(func(string) (int, error) { return 42, nil }))
|
||||||
|
bad, _ := copyx.New()
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 24; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
for j := 0; j < 20; j++ {
|
||||||
|
if err := good.Check(reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
if err := bad.Check(reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
var target checkTarget
|
||||||
|
if err := good.Copy(&target, checkSource{Rows: []checkSourceRow{}}); err != nil || target.Rows == nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
if err := good.Check(nil, reflect.TypeFor[int]()); err == nil {
|
||||||
|
t.Fatal("nil type accepted")
|
||||||
|
}
|
||||||
|
var absent *copyx.Copier
|
||||||
|
if err := absent.Check(reflect.TypeFor[int](), reflect.TypeFor[int]()); err == nil {
|
||||||
|
t.Fatal("nil copier accepted")
|
||||||
|
}
|
||||||
|
if err := copyx.Check[int, string](); err == nil || !strings.Contains(err.Error(), "int -> string") {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func BenchmarkTypeCheck(b *testing.B) {
|
||||||
|
c, _ := copyx.New(copyx.WithConverter(func(string) (int, error) { return 1, nil }))
|
||||||
|
from, to := reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()
|
||||||
|
b.ReportAllocs()
|
||||||
|
for i := 0; i < b.N; i++ {
|
||||||
|
if err := c.Check(from, to); err != nil {
|
||||||
|
b.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEmptyTypeCheckPreservesAbsentValueSemantics(t *testing.T) {
|
||||||
|
var callback func()
|
||||||
|
if out, err := copyx.Clone(callback); err != nil || out != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if out, err := copyx.Clone([]func(){}); err != nil || out == nil || len(out) != 0 {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := copyx.Clone([]func(){func() {}}); !errors.Is(err, copyx.ErrUnsupported) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := copyx.Check[[]func(), []func()](copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrUnsupported) {
|
||||||
|
t.Fatal("preflight must report potentially non-nil handles", err)
|
||||||
|
}
|
||||||
|
type from struct {
|
||||||
|
Ignored *int
|
||||||
|
Value string
|
||||||
|
}
|
||||||
|
type to struct {
|
||||||
|
Ignored *int
|
||||||
|
Value int
|
||||||
|
}
|
||||||
|
var dst []to
|
||||||
|
if err := copyx.Copy(&dst, []from{}, copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrType) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -95,6 +95,8 @@ type state struct {
|
|||||||
nodes int
|
nodes int
|
||||||
allocated uint64
|
allocated uint64
|
||||||
valueRoot *reference
|
valueRoot *reference
|
||||||
|
checkedTypes map[pair]bool
|
||||||
|
checkedNodes int
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *state) reserve(count int, size uintptr, path string) error {
|
func (s *state) reserve(count int, size uintptr, path string) error {
|
||||||
@@ -239,6 +241,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth
|
|||||||
case reflect.Struct:
|
case reflect.Struct:
|
||||||
return s.structure(src, base, t, path, depth)
|
return s.structure(src, base, t, path, depth)
|
||||||
case reflect.Slice:
|
case reflect.Slice:
|
||||||
|
if src.Len() == 0 {
|
||||||
|
if err := s.checkAbsent(src.Type(), t, path); err != nil {
|
||||||
|
return reflect.Value{}, err
|
||||||
|
}
|
||||||
|
}
|
||||||
if !s.options.deep && src.Type() == t {
|
if !s.options.deep && src.Type() == t {
|
||||||
return src, nil
|
return src, nil
|
||||||
}
|
}
|
||||||
@@ -264,6 +271,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth
|
|||||||
}
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
case reflect.Array:
|
case reflect.Array:
|
||||||
|
if src.Len() == 0 {
|
||||||
|
if err := s.checkAbsent(src.Type(), t, path); err != nil {
|
||||||
|
return reflect.Value{}, err
|
||||||
|
}
|
||||||
|
}
|
||||||
if src.Len() != t.Len() {
|
if src.Len() != t.Len() {
|
||||||
return reflect.Value{}, at(path, ErrType)
|
return reflect.Value{}, at(path, ErrType)
|
||||||
}
|
}
|
||||||
@@ -280,6 +292,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth
|
|||||||
}
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
case reflect.Map:
|
case reflect.Map:
|
||||||
|
if src.Len() == 0 {
|
||||||
|
if err := s.checkAbsent(src.Type(), t, path); err != nil {
|
||||||
|
return reflect.Value{}, err
|
||||||
|
}
|
||||||
|
}
|
||||||
if !s.options.deep && src.Type() == t {
|
if !s.options.deep && src.Type() == t {
|
||||||
return src, nil
|
return src, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -7,6 +7,8 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"reflect"
|
"reflect"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
// PoolConfig contains only driver-independent settings. Connection credentials,
|
// PoolConfig contains only driver-independent settings. Connection credentials,
|
||||||
@@ -25,15 +27,29 @@ func DefaultPoolConfig() PoolConfig {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (c PoolConfig) Validate() error {
|
func (c PoolConfig) Validate() error {
|
||||||
if c.MaxOpenConns < 0 || c.MaxIdleConns < 0 {
|
for _, f := range []struct {
|
||||||
return errors.New("database pool limits cannot be negative")
|
name string
|
||||||
|
value int
|
||||||
|
}{{"maxOpenConns", c.MaxOpenConns}, {"maxIdleConns", c.MaxIdleConns}} {
|
||||||
|
if f.value < 0 {
|
||||||
|
return configdiag.New(f.name, "must be nonnegative")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if c.MaxOpenConns > 0 && c.MaxIdleConns > c.MaxOpenConns {
|
if c.MaxOpenConns > 0 && c.MaxIdleConns > c.MaxOpenConns {
|
||||||
return errors.New("max idle connections cannot exceed max open connections")
|
return configdiag.New("maxIdleConns", "must not exceed maxOpenConns")
|
||||||
}
|
}
|
||||||
if c.ConnMaxLifetime < 0 || c.ConnMaxIdleTime < 0 || c.PingTimeout <= 0 {
|
for _, f := range []struct {
|
||||||
return errors.New("database timeouts are invalid")
|
name string
|
||||||
|
value time.Duration
|
||||||
|
}{{"connMaxLifetime", c.ConnMaxLifetime}, {"connMaxIdleTime", c.ConnMaxIdleTime}} {
|
||||||
|
if f.value < 0 {
|
||||||
|
return configdiag.New(f.name, "must be nonnegative")
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
if c.PingTimeout <= 0 {
|
||||||
|
return configdiag.New("pingTimeout", "must be positive")
|
||||||
|
}
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -10,6 +10,8 @@ import (
|
|||||||
"fmt"
|
"fmt"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
type Config struct {
|
type Config struct {
|
||||||
@@ -43,10 +45,10 @@ func (c Config) PoolConfig() PoolConfig {
|
|||||||
|
|
||||||
func (c Config) Validate() error {
|
func (c Config) Validate() error {
|
||||||
if strings.TrimSpace(c.Driver) == "" {
|
if strings.TrimSpace(c.Driver) == "" {
|
||||||
return errors.New("database driver is required")
|
return configdiag.New("driver", "is required")
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(c.DSN) == "" {
|
if strings.TrimSpace(c.DSN) == "" {
|
||||||
return errors.New("database DSN is required")
|
return configdiag.New("dsn", "is required")
|
||||||
}
|
}
|
||||||
return c.PoolConfig().Validate()
|
return c.PoolConfig().Validate()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -70,3 +70,7 @@ func (c *ServiceContext) DatabaseOptions() []mysqladapter.Option {
|
|||||||
`WithDriverConfig(func(*mysql.Config) error)` 支持高级拨号、驱动参数等,配置在回调后克隆;这是项目明确选择的底层扩展。`NewConnector` 返回标准 driver.Connector,供已有基础设施使用;它不接受启动检查选项。任意驱动仍可用核心 `database.OpenConnector`。
|
`WithDriverConfig(func(*mysql.Config) error)` 支持高级拨号、驱动参数等,配置在回调后克隆;这是项目明确选择的底层扩展。`NewConnector` 返回标准 driver.Connector,供已有基础设施使用;它不接受启动检查选项。任意驱动仍可用核心 `database.OpenConnector`。
|
||||||
|
|
||||||
错误有安全摘要,errors.Is/As 保留 TLS/驱动/项目检查的原因。配置验证不读证书、不连接数据库;Open 才读取证书、创建池并握手。不要自行打印原始配置或展开可能包含内部信息的 driver 字段给用户。
|
错误有安全摘要,errors.Is/As 保留 TLS/驱动/项目检查的原因。配置验证不读证书、不连接数据库;Open 才读取证书、创建池并握手。不要自行打印原始配置或展开可能包含内部信息的 driver 字段给用户。
|
||||||
|
|
||||||
|
## 字段级配置错误
|
||||||
|
|
||||||
|
Validate 返回独立字段与安全原因,例如 port(1–65535)、readTimeout、pool.maxIdleConns、tls.keyFile。项目调用 `config.AtField("database", err)` 后,统一加载器补齐 database 前缀、文件、环境名和行列,无需重复写端口/连接池校验。TLS 成对配置会指出缺失的那个字段。连接失败、驱动和证书文件的内部原因仍通过 errors.Is/As 保留,不把凭据放进公开提示。见 [配置诊断](../../CONFIG.md#框架模块的字段诊断)。
|
||||||
|
|||||||
@@ -19,6 +19,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
mysqldriver "github.com/go-sql-driver/mysql"
|
mysqldriver "github.com/go-sql-driver/mysql"
|
||||||
)
|
)
|
||||||
@@ -61,45 +62,60 @@ var identifier = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`)
|
|||||||
// Validate does not read certificate files or open connections. All defaults
|
// Validate does not read certificate files or open connections. All defaults
|
||||||
// come from DefaultConfig; explicit zero I/O timeouts mean no driver deadline.
|
// come from DefaultConfig; explicit zero I/O timeouts mean no driver deadline.
|
||||||
func (c Config) Validate() error {
|
func (c Config) Validate() error {
|
||||||
if c.Socket == "" && (c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") || c.Port < 1 || c.Port > 65535) {
|
if c.Socket == "" {
|
||||||
return errors.New("mysql: invalid host or port")
|
if c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") {
|
||||||
|
return configdiag.New("host", "must be a nonempty hostname or IP address")
|
||||||
}
|
}
|
||||||
if strings.ContainsRune(c.Socket, 0) || strings.ContainsRune(c.Database, 0) || strings.ContainsRune(c.User, 0) {
|
if c.Port < 1 || c.Port > 65535 {
|
||||||
return errors.New("mysql: invalid connection field")
|
return configdiag.New("port", "must be between 1 and 65535")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, f := range []struct{ name, value string }{{"socket", c.Socket}, {"database", c.Database}, {"user", c.User}} {
|
||||||
|
if strings.ContainsRune(f.value, 0) {
|
||||||
|
return configdiag.New(f.name, "must not contain a NUL character")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, f := range []struct {
|
||||||
|
name string
|
||||||
|
value time.Duration
|
||||||
|
}{{"connectTimeout", c.ConnectTimeout}, {"readTimeout", c.ReadTimeout}, {"writeTimeout", c.WriteTimeout}} {
|
||||||
|
if f.value < 0 {
|
||||||
|
return configdiag.New(f.name, "must be nonnegative")
|
||||||
}
|
}
|
||||||
if c.ConnectTimeout < 0 || c.ReadTimeout < 0 || c.WriteTimeout < 0 {
|
|
||||||
return errors.New("mysql: timeouts cannot be negative")
|
|
||||||
}
|
}
|
||||||
if err := c.Pool.Validate(); err != nil {
|
if err := c.Pool.Validate(); err != nil {
|
||||||
return err
|
return configdiag.At("pool", err)
|
||||||
}
|
}
|
||||||
if c.Location != "" {
|
if c.Location != "" {
|
||||||
if _, err := time.LoadLocation(c.Location); err != nil {
|
if _, err := time.LoadLocation(c.Location); err != nil {
|
||||||
return errors.New("mysql: invalid time location")
|
return configdiag.Wrap("location", "must be an available IANA time zone or UTC", err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if c.Charset != "" && !identifier.MatchString(c.Charset) {
|
if c.Charset != "" && !identifier.MatchString(c.Charset) {
|
||||||
return errors.New("mysql: invalid charset")
|
return configdiag.New("charset", "must be a valid MySQL identifier")
|
||||||
}
|
}
|
||||||
if c.Collation != "" && !identifier.MatchString(c.Collation) {
|
if c.Collation != "" && !identifier.MatchString(c.Collation) {
|
||||||
return errors.New("mysql: invalid collation")
|
return configdiag.New("collation", "must be a valid MySQL identifier")
|
||||||
}
|
}
|
||||||
for key := range c.Session {
|
for key := range c.Session {
|
||||||
if !identifier.MatchString(key) {
|
if !identifier.MatchString(key) {
|
||||||
return errors.New("mysql: invalid session variable name")
|
return configdiag.New("session", "keys must be valid MySQL identifiers")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if c.TLS.Mode != "" && c.TLS.Mode != "required" && c.TLS.Mode != "disabled" {
|
if c.TLS.Mode != "" && c.TLS.Mode != "required" && c.TLS.Mode != "disabled" {
|
||||||
return errors.New("mysql: TLS mode must be required or disabled")
|
return configdiag.New("tls.mode", "must be required or disabled")
|
||||||
}
|
}
|
||||||
if (c.TLS.CertFile == "") != (c.TLS.KeyFile == "") {
|
if (c.TLS.CertFile == "") != (c.TLS.KeyFile == "") {
|
||||||
return errors.New("mysql: TLS client certificate and key must be configured together")
|
if c.TLS.CertFile == "" {
|
||||||
|
return configdiag.New("tls.certFile", "is required when keyFile is configured")
|
||||||
|
}
|
||||||
|
return configdiag.New("tls.keyFile", "is required when certFile is configured")
|
||||||
}
|
}
|
||||||
if c.TLS.Mode == "disabled" && (c.TLS.CAFile != "" || c.TLS.CertFile != "" || c.TLS.ServerName != "") {
|
if c.TLS.Mode == "disabled" && (c.TLS.CAFile != "" || c.TLS.CertFile != "" || c.TLS.ServerName != "") {
|
||||||
return errors.New("mysql: TLS fields cannot be used when disabled")
|
return configdiag.New("tls.mode", "must be required when TLS fields are configured")
|
||||||
}
|
}
|
||||||
if c.Socket != "" && c.TLS.Mode != "disabled" && c.TLS.ServerName == "" {
|
if c.Socket != "" && c.TLS.Mode != "disabled" && c.TLS.ServerName == "" {
|
||||||
return errors.New("mysql: TLS over a unix socket requires serverName")
|
return configdiag.New("tls.serverName", "is required for TLS over a unix socket")
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
18
examples/developerTools/README.md
Normal file
18
examples/developerTools/README.md
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
# 配置诊断、严格 HTTP 状态、类型预检和日志时区
|
||||||
|
|
||||||
|
在框架根目录运行,不需要数据库、账号或外部服务。HTTP 示例只访问临时 localhost 服务:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./examples/developerTools
|
||||||
|
DEMO_LOG_TIME_ZONE=UTC go run ./examples/developerTools
|
||||||
|
# 以下两条应非零退出,输出具体字段和环境变量名,不输出原值:
|
||||||
|
DEMO_DB_PORT=70000 go run ./examples/developerTools
|
||||||
|
DEMO_LOG_LEVEL=invalid go run ./examples/developerTools
|
||||||
|
```
|
||||||
|
|
||||||
|
- 配置只做校验,没有连接数据库。失败指向 `database.port` / `server.log.level`,并保留文件行列与变量名。
|
||||||
|
- Logger 使用选定时区;只改变记录的时间戳,不修改全局 time.Local 和业务时间字段。
|
||||||
|
- `copyx.Check[Input, Result]` 发现切片元素内的不兼容字段;加上显式 `strconv.Atoi` converter 后通过。预检不执行 converter,真实非法数字仍在 Copy 时返回错误。
|
||||||
|
- JSON 调用显式 `WithSuccessStatuses(200)`,拒绝 201,框架仍负责关闭响应体。
|
||||||
|
|
||||||
|
详见 [配置](../../CONFIG.md)、[日志](../../logx/README.md)、[HTTP 客户端](../../httpc/README.md)、[copyx](../../COPY.md)。
|
||||||
7
examples/developerTools/config.yaml
Normal file
7
examples/developerTools/config.yaml
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
# Validates database settings without connecting in this demonstration.
|
||||||
|
database:
|
||||||
|
port: "${DEMO_DB_PORT:-3306}"
|
||||||
|
server:
|
||||||
|
log:
|
||||||
|
level: "${DEMO_LOG_LEVEL:-info}"
|
||||||
|
timeZone: "${DEMO_LOG_TIME_ZONE:-Asia/Shanghai}"
|
||||||
104
examples/developerTools/main.go
Normal file
104
examples/developerTools/main.go
Normal file
@@ -0,0 +1,104 @@
|
|||||||
|
// Run from the framework root with: go run ./examples/developerTools
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
_ "time/tzdata" // This example embeds zones; production builds can use -tags timetzdata.
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||||
|
mysql "git.yuntuo.hk/k2safe/go-omni/database/mysql"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpc"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Config struct {
|
||||||
|
Database mysql.Config `json:"database"`
|
||||||
|
Server config.Config `json:"server"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func Default() Config {
|
||||||
|
c := config.Default()
|
||||||
|
c.Log.Output = "stdout"
|
||||||
|
return Config{Database: mysql.DefaultConfig(), Server: c}
|
||||||
|
}
|
||||||
|
func (c Config) Validate() error {
|
||||||
|
if err := c.Database.Validate(); err != nil {
|
||||||
|
return config.AtField("database", err)
|
||||||
|
}
|
||||||
|
return config.AtField("server", c.Server.Validate())
|
||||||
|
}
|
||||||
|
|
||||||
|
type Input struct{ Values []TextValue }
|
||||||
|
type TextValue struct{ Number string }
|
||||||
|
type Result struct{ Values []NumberValue }
|
||||||
|
type NumberValue struct{ Number int }
|
||||||
|
|
||||||
|
func run(path string) error {
|
||||||
|
cfg, err := config.LoadProject(path, Default())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// Validation only: this example never opens a database.
|
||||||
|
logger, err := logx.New(cfg.Server.Log)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer logger.Close()
|
||||||
|
logger.InfoContext(context.Background(), "instance time zone", "configuredZone", cfg.Server.Log.TimeZone)
|
||||||
|
if err := copyx.Check[Input, Result](); !errors.Is(err, copyx.ErrType) {
|
||||||
|
return fmt.Errorf("expected a type mismatch, got %v", err)
|
||||||
|
}
|
||||||
|
// Explicit Go converter owns the number conversion; no JSON mapping file.
|
||||||
|
options := []copyx.Option{copyx.WithConverter(strconv.Atoi)}
|
||||||
|
if err := copyx.Check[Input, Result](options...); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
var output Result
|
||||||
|
if err := copyx.Copy(&output, Input{Values: []TextValue{{Number: "42"}}}, options...); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Printf("type check and copy succeeded: %d\n", output.Values[0].Number)
|
||||||
|
|
||||||
|
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
if r.URL.Path == "/created" {
|
||||||
|
w.WriteHeader(http.StatusCreated)
|
||||||
|
}
|
||||||
|
io.WriteString(w, `{"value":"example"}`)
|
||||||
|
}))
|
||||||
|
defer server.Close()
|
||||||
|
client, err := httpc.New(httpc.Config{DisableLogging: true})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer client.Close()
|
||||||
|
var reply struct {
|
||||||
|
Value string `json:"value"`
|
||||||
|
}
|
||||||
|
if err := client.JSON(context.Background(), http.MethodGet, server.URL, nil, &reply, httpc.WithSuccessStatuses(http.StatusOK)); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
err = client.JSON(context.Background(), http.MethodGet, server.URL+"/created", nil, &reply, httpc.WithSuccessStatuses(http.StatusOK))
|
||||||
|
var status *httpc.StatusError
|
||||||
|
if !errors.As(err, &status) || status.StatusCode != http.StatusCreated {
|
||||||
|
return fmt.Errorf("expected status rejection, got %v", err)
|
||||||
|
}
|
||||||
|
fmt.Println("strict JSON: 200 accepted, 201 rejected")
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func main() {
|
||||||
|
path := flag.String("config", "examples/developerTools/config.yaml", "selected configuration file")
|
||||||
|
flag.Parse()
|
||||||
|
if err := run(*path); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -2,7 +2,7 @@ module example.com/managed
|
|||||||
|
|
||||||
go 1.22.0
|
go 1.22.0
|
||||||
|
|
||||||
require git.yuntuo.hk/k2safe/go-omni v0.22.0-preview
|
require git.yuntuo.hk/k2safe/go-omni v0.23.0-preview
|
||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ module example.com/userService
|
|||||||
go 1.22.0
|
go 1.22.0
|
||||||
|
|
||||||
require (
|
require (
|
||||||
git.yuntuo.hk/k2safe/go-omni v0.22.0-preview
|
git.yuntuo.hk/k2safe/go-omni v0.23.0-preview
|
||||||
github.com/go-sql-driver/mysql v1.9.3
|
github.com/go-sql-driver/mysql v1.9.3
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -85,7 +85,20 @@ err := client.JSON(ctx, "POST", endpoint, input, &output,
|
|||||||
- Host、Content-Length、Transfer-Encoding、Trailer 属于请求目标/分帧控制,请用标准 Request 配合 Do;JSON 选项中不允许伪装成普通头。
|
- Host、Content-Length、Transfer-Encoding、Trailer 属于请求目标/分帧控制,请用标准 Request 配合 Do;JSON 选项中不允许伪装成普通头。
|
||||||
- `WithMaxResponseBytes` 是本次 JSON 的正数限额,不修改 Client 默认值,不改变 Do 的流式语义。
|
- `WithMaxResponseBytes` 是本次 JSON 的正数限额,不修改 Client 默认值,不改变 Do 的流式语义。
|
||||||
|
|
||||||
JSON 自动关闭响应体。仅 2xx 解码;空成功响应(例如 204)保留 output 原值。非空响应使用标准 json.Unmarshal,拒绝多个 JSON 文档,类型错误保留 Go 的错误信息与行为。
|
`WithSuccessStatuses(200)` 限制本次调用只接受 200;可传多个,例如 `(200, 201)`。不传时保持全部 2xx。列表必须非空且每项位于 200–299,重复项允许;后一个状态选项替换前一个列表。构造选项时快照,之后修改原 slice 不影响请求,并发复用不串配置。
|
||||||
|
|
||||||
|
```go
|
||||||
|
err := client.JSON(ctx, http.MethodGet, endpoint, nil, &result,
|
||||||
|
httpc.WithHeader("Authorization", credential),
|
||||||
|
httpc.WithSuccessStatuses(http.StatusOK),
|
||||||
|
)
|
||||||
|
var status *httpc.StatusError
|
||||||
|
if errors.As(err, &status) { /* status.StatusCode 是实际收到的状态 */ }
|
||||||
|
```
|
||||||
|
|
||||||
|
未允许的状态(包括 201/204)返回 StatusError,不解码、不改变 output;仍关闭响应体、按原有时间/大小预算回收连接,不读取无限错误正文。状态选项不会增加重试,原有 Do 重试规则保留。它只限制 HTTP 状态,不保证响应非空或业务字段完整,业务仍校验自身协议。严格状态、关闭与连接复用见 [示例](../examples/developerTools/README.md)。
|
||||||
|
|
||||||
|
JSON 自动关闭响应体。默认仅 2xx 解码;指定成功状态列表时只解码列表内状态;空成功响应(例如 204)保留 output 原值。非空响应使用标准 json.Unmarshal,拒绝多个 JSON 文档,类型错误保留 Go 的错误信息与行为。
|
||||||
|
|
||||||
`output == nil` 按限额流式丢弃成功响应,不把整个 body 读进内存,也不检查 JSON 语法。响应限额按实际读取的 body 计算;标准库自动 gzip 解压后仍受上限约束。input 会先 json.Marshal 到内存,大文件上传不适合用 JSON helper。
|
`output == nil` 按限额流式丢弃成功响应,不把整个 body 读进内存,也不检查 JSON 语法。响应限额按实际读取的 body 计算;标准库自动 gzip 解压后仍受上限约束。input 会先 json.Marshal 到内存,大文件上传不适合用 JSON helper。
|
||||||
|
|
||||||
|
|||||||
@@ -2,7 +2,6 @@ package httpc
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
|
||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/url"
|
"net/url"
|
||||||
@@ -10,6 +9,8 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ProxyMode controls the owned transport's proxy selection. Empty preserves
|
// ProxyMode controls the owned transport's proxy selection. Empty preserves
|
||||||
@@ -66,17 +67,23 @@ func normalizeConfig(cfg Config) (Config, error) {
|
|||||||
{"responseHeaderTimeout", cfg.ResponseHeaderTimeout}, {"expectContinueTimeout", cfg.ExpectContinueTimeout}, {"idleConnTimeout", cfg.IdleConnTimeout},
|
{"responseHeaderTimeout", cfg.ResponseHeaderTimeout}, {"expectContinueTimeout", cfg.ExpectContinueTimeout}, {"idleConnTimeout", cfg.IdleConnTimeout},
|
||||||
} {
|
} {
|
||||||
if field.value < 0 {
|
if field.value < 0 {
|
||||||
return Config{}, fmt.Errorf("httpc: %s must not be negative", field.name)
|
return Config{}, configdiag.New(field.name, "must be nonnegative")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if cfg.MaxRetries < 0 || cfg.MaxRetries > 5 {
|
if cfg.MaxRetries < 0 || cfg.MaxRetries > 5 {
|
||||||
return Config{}, errors.New("httpc: maxRetries must be 0..5")
|
return Config{}, configdiag.New("maxRetries", "must be between 0 and 5")
|
||||||
}
|
}
|
||||||
if cfg.MaxConnsPerHost < 0 || cfg.MaxConnsPerHost > 65536 || cfg.MaxIdleConnsPerHost < 0 || cfg.MaxIdleConnsPerHost > 65536 || cfg.MaxIdleConns < 0 || cfg.MaxIdleConns > 131072 {
|
for _, f := range []struct {
|
||||||
return Config{}, errors.New("httpc: invalid connection pool limits")
|
name string
|
||||||
|
value, max int
|
||||||
|
}{{"maxConnsPerHost", cfg.MaxConnsPerHost, 65536}, {"maxIdleConnsPerHost", cfg.MaxIdleConnsPerHost, 65536}, {"maxIdleConns", cfg.MaxIdleConns, 131072}} {
|
||||||
|
if f.value < 0 || f.value > f.max {
|
||||||
|
return Config{}, configdiag.New(f.name, "must be between 0 and "+strconv.Itoa(f.max))
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
if cfg.MaxResponseBytes < 0 || cfg.MaxResponseBytes > maxResponseBytes {
|
if cfg.MaxResponseBytes < 0 || cfg.MaxResponseBytes > maxResponseBytes {
|
||||||
return Config{}, errors.New("httpc: maxResponseBytes must be 0..1 GiB")
|
return Config{}, configdiag.New("maxResponseBytes", "must be between 0 and 1 GiB")
|
||||||
}
|
}
|
||||||
if _, err := proxyFunction(cfg.Proxy); err != nil {
|
if _, err := proxyFunction(cfg.Proxy); err != nil {
|
||||||
return Config{}, err
|
return Config{}, err
|
||||||
@@ -86,11 +93,11 @@ func normalizeConfig(cfg Config) (Config, error) {
|
|||||||
switch value.Kind() {
|
switch value.Kind() {
|
||||||
case reflect.Pointer, reflect.Map, reflect.Func, reflect.Slice, reflect.Interface, reflect.Chan:
|
case reflect.Pointer, reflect.Map, reflect.Func, reflect.Slice, reflect.Interface, reflect.Chan:
|
||||||
if value.IsNil() {
|
if value.IsNil() {
|
||||||
return Config{}, errors.New("httpc: nil Transport")
|
return Config{}, configdiag.New("transport", "must not be a typed nil")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if cfg.hasTransportOptions() {
|
if cfg.hasTransportOptions() {
|
||||||
return Config{}, errors.New("httpc: Transport cannot be combined with proxy, dial, TLS or connection pool options; configure the borrowed transport directly")
|
return Config{}, configdiag.New("transport", "cannot be combined with proxy, dial, TLS or connection pool options; configure the borrowed transport directly")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if cfg.Timeout == 0 {
|
if cfg.Timeout == 0 {
|
||||||
@@ -103,7 +110,7 @@ func normalizeConfig(cfg Config) (Config, error) {
|
|||||||
cfg.MaxBackoff = 2 * time.Second
|
cfg.MaxBackoff = 2 * time.Second
|
||||||
}
|
}
|
||||||
if cfg.Backoff > cfg.MaxBackoff {
|
if cfg.Backoff > cfg.MaxBackoff {
|
||||||
return Config{}, errors.New("httpc: backoff exceeds maxBackoff")
|
return Config{}, configdiag.New("backoff", "must not exceed maxBackoff")
|
||||||
}
|
}
|
||||||
if cfg.MaxConnsPerHost == 0 {
|
if cfg.MaxConnsPerHost == 0 {
|
||||||
cfg.MaxConnsPerHost = 64
|
cfg.MaxConnsPerHost = 64
|
||||||
@@ -124,7 +131,7 @@ func (c Config) hasTransportOptions() bool {
|
|||||||
}
|
}
|
||||||
func proxyFunction(c ProxyConfig) (func(*http.Request) (*url.URL, error), error) {
|
func proxyFunction(c ProxyConfig) (func(*http.Request) (*url.URL, error), error) {
|
||||||
if c.Mode != ProxyURL && c.URL != "" {
|
if c.Mode != ProxyURL && c.URL != "" {
|
||||||
return nil, errors.New("httpc: proxy URL requires mode url")
|
return nil, configdiag.New("proxy.mode", "must be url when a URL is configured")
|
||||||
}
|
}
|
||||||
switch c.Mode {
|
switch c.Mode {
|
||||||
case "":
|
case "":
|
||||||
@@ -136,27 +143,27 @@ func proxyFunction(c ProxyConfig) (func(*http.Request) (*url.URL, error), error)
|
|||||||
case ProxyURL:
|
case ProxyURL:
|
||||||
u, err := url.Parse(c.URL)
|
u, err := url.Parse(c.URL)
|
||||||
if err != nil || u == nil || u.Hostname() == "" || u.Opaque != "" || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || (u.Path != "" && u.Path != "/") {
|
if err != nil || u == nil || u.Hostname() == "" || u.Opaque != "" || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || (u.Path != "" && u.Path != "/") {
|
||||||
return nil, errors.New("httpc: proxy URL must be an absolute proxy origin without path, query or fragment")
|
return nil, configdiag.New("proxy.url", "must be an absolute proxy origin without path, query or fragment")
|
||||||
}
|
}
|
||||||
switch u.Scheme {
|
switch u.Scheme {
|
||||||
case "http", "https", "socks5", "socks5h":
|
case "http", "https", "socks5", "socks5h":
|
||||||
default:
|
default:
|
||||||
return nil, errors.New("httpc: proxy scheme must be http, https, socks5 or socks5h")
|
return nil, configdiag.New("proxy.url", "scheme must be http, https, socks5 or socks5h")
|
||||||
}
|
}
|
||||||
if strings.HasSuffix(u.Host, ":") {
|
if strings.HasSuffix(u.Host, ":") {
|
||||||
return nil, errors.New("httpc: proxy port must be 1..65535")
|
return nil, configdiag.New("proxy.url", "port must be between 1 and 65535")
|
||||||
}
|
}
|
||||||
if port := u.Port(); port != "" {
|
if port := u.Port(); port != "" {
|
||||||
n, e := strconv.Atoi(port)
|
n, e := strconv.Atoi(port)
|
||||||
if e != nil || n < 1 || n > 65535 {
|
if e != nil || n < 1 || n > 65535 {
|
||||||
return nil, errors.New("httpc: proxy port must be 1..65535")
|
return nil, configdiag.New("proxy.url", "port must be between 1 and 65535")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
u.Path = ""
|
u.Path = ""
|
||||||
u.RawPath = ""
|
u.RawPath = ""
|
||||||
return http.ProxyURL(u), nil
|
return http.ProxyURL(u), nil
|
||||||
default:
|
default:
|
||||||
return nil, errors.New("httpc: proxy mode must be environment, direct or url")
|
return nil, configdiag.New("proxy.mode", "must be environment, direct or url")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
func ownedTransport(cfg Config) (*http.Transport, error) {
|
func ownedTransport(cfg Config) (*http.Transport, error) {
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// JSON bounds response memory, closes the body, and decodes only 2xx responses.
|
// JSON bounds response memory, closes the body, and decodes only 2xx responses.
|
||||||
// Optional headers/limits belong to this call and never alter the client.
|
// Optional headers/limits/success statuses belong to this call and never alter the client.
|
||||||
// output=nil discards a bounded stream; it does not buffer or validate JSON.
|
// output=nil discards a bounded stream; it does not buffer or validate JSON.
|
||||||
// Empty successful bodies leave output unchanged. StatusError contains no body.
|
// Empty successful bodies leave output unchanged. StatusError contains no body.
|
||||||
func (c *Client) JSON(ctx context.Context, method, url string, input, output any, options ...RequestOption) error {
|
func (c *Client) JSON(ctx context.Context, method, url string, input, output any, options ...RequestOption) error {
|
||||||
@@ -57,7 +57,7 @@ func (c *Client) JSON(ctx context.Context, method, url string, input, output any
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
defer response.Body.Close()
|
defer response.Body.Close()
|
||||||
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
if !settings.accepts(response.StatusCode) {
|
||||||
drainStatusBody(response, settings.maxResponseBytes)
|
drainStatusBody(response, settings.maxResponseBytes)
|
||||||
return &StatusError{StatusCode: response.StatusCode}
|
return &StatusError{StatusCode: response.StatusCode}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -9,6 +9,8 @@ import (
|
|||||||
type requestOptions struct {
|
type requestOptions struct {
|
||||||
headers http.Header
|
headers http.Header
|
||||||
maxResponseBytes int64
|
maxResponseBytes int64
|
||||||
|
successStatuses [2]uint64
|
||||||
|
restrictStatus bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// RequestOption applies only to one JSON call. Returned options can be reused
|
// RequestOption applies only to one JSON call. Returned options can be reused
|
||||||
@@ -97,3 +99,41 @@ func checkHeader(name string, values []string) error {
|
|||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// WithSuccessStatuses restricts this JSON call to the listed 2xx statuses.
|
||||||
|
// Omission keeps all 2xx successful. Empty/out-of-range lists are rejected before
|
||||||
|
// sending a request. Options snapshot their input and can be reused concurrently.
|
||||||
|
// The policy does not add retries or validate application-specific JSON fields.
|
||||||
|
func WithSuccessStatuses(statuses ...int) RequestOption {
|
||||||
|
var accepted [2]uint64
|
||||||
|
var err error
|
||||||
|
if len(statuses) == 0 {
|
||||||
|
err = errors.New("httpc: at least one success status is required")
|
||||||
|
}
|
||||||
|
for _, status := range statuses {
|
||||||
|
if status < 200 || status >= 300 {
|
||||||
|
err = errors.New("httpc: success statuses must be between 200 and 299")
|
||||||
|
break
|
||||||
|
}
|
||||||
|
bit := status - 200
|
||||||
|
accepted[bit/64] |= uint64(1) << uint(bit%64)
|
||||||
|
}
|
||||||
|
return func(o *requestOptions) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
o.successStatuses = accepted
|
||||||
|
o.restrictStatus = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (o requestOptions) accepts(status int) bool {
|
||||||
|
if status < 200 || status >= 300 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if !o.restrictStatus {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
bit := status - 200
|
||||||
|
return o.successStatuses[bit/64]&(uint64(1)<<uint(bit%64)) != 0
|
||||||
|
}
|
||||||
|
|||||||
174
httpc/status_test.go
Normal file
174
httpc/status_test.go
Normal file
@@ -0,0 +1,174 @@
|
|||||||
|
package httpc
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestJSONSuccessStatuses(t *testing.T) {
|
||||||
|
for _, status := range []int{200, 201, 204, 264, 299, 400} {
|
||||||
|
t.Run(strconv.Itoa(status), func(t *testing.T) {
|
||||||
|
var calls atomic.Int32
|
||||||
|
body := &trackedBody{Reader: strings.NewReader(`{"value":2}`)}
|
||||||
|
c := testClient(t, Config{MaxRetries: 5, Transport: roundTripperFunc(func(r *http.Request) (*http.Response, error) {
|
||||||
|
calls.Add(1)
|
||||||
|
return &http.Response{StatusCode: status, Body: body, ContentLength: 11, Header: make(http.Header), Request: r}, nil
|
||||||
|
})})
|
||||||
|
out := struct {
|
||||||
|
Value int `json:"value"`
|
||||||
|
}{Value: 1}
|
||||||
|
err := c.JSON(context.Background(), "GET", "http://example.invalid", nil, &out, WithSuccessStatuses(200))
|
||||||
|
if status == 200 {
|
||||||
|
if err != nil || out.Value != 2 {
|
||||||
|
t.Fatal(err, out)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
var rejected *StatusError
|
||||||
|
if !errors.As(err, &rejected) || rejected.StatusCode != status || out.Value != 1 {
|
||||||
|
t.Fatal(err, out)
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), "value") {
|
||||||
|
t.Fatal("status error exposed body")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !body.closed || calls.Load() != 1 {
|
||||||
|
t.Fatal("not closed or unexpected retry", calls.Load())
|
||||||
|
}
|
||||||
|
// Unconfigured calls continue to accept every 2xx.
|
||||||
|
c2 := testClient(t, Config{Transport: benchmarkTransport{`{}`, status}})
|
||||||
|
err = c2.JSON(context.Background(), "GET", "http://example.invalid", nil, nil)
|
||||||
|
if (err == nil) != (status >= 200 && status < 300) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if status >= 200 && status < 300 {
|
||||||
|
if err := c2.JSON(context.Background(), "GET", "http://example.invalid", nil, nil, WithSuccessStatuses(status, status)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
var calls atomic.Int32
|
||||||
|
c := testClient(t, Config{Transport: roundTripperFunc(func(r *http.Request) (*http.Response, error) {
|
||||||
|
calls.Add(1)
|
||||||
|
return benchmarkTransport{`{}`, 200}.RoundTrip(r)
|
||||||
|
})})
|
||||||
|
for _, option := range []RequestOption{WithSuccessStatuses(), WithSuccessStatuses(199), WithSuccessStatuses(300), WithSuccessStatuses(-1), WithSuccessStatuses(200, 500)} {
|
||||||
|
if err := c.JSON(context.Background(), "GET", "http://example.invalid", nil, nil, option); err == nil {
|
||||||
|
t.Fatal("invalid policy accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if calls.Load() != 0 {
|
||||||
|
t.Fatal("invalid policy reached transport")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestJSONSuccessStatusConcurrentOptions(t *testing.T) {
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
n, _ := strconv.Atoi(r.URL.Query().Get("status"))
|
||||||
|
w.WriteHeader(n)
|
||||||
|
if n != 204 {
|
||||||
|
io.WriteString(w, `{}`)
|
||||||
|
}
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
c := testClient(t, Config{})
|
||||||
|
statuses := []int{200, 201}
|
||||||
|
option := WithSuccessStatuses(statuses...)
|
||||||
|
statuses[0] = 204
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 30; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(i int) {
|
||||||
|
defer wg.Done()
|
||||||
|
status := 200 + i%2
|
||||||
|
err := c.JSON(context.Background(), "GET", fmt.Sprintf("%s?status=%d", srv.URL, status), nil, nil, option)
|
||||||
|
if err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
err = c.JSON(context.Background(), "GET", srv.URL+"?status=204", nil, nil, option)
|
||||||
|
var rejected *StatusError
|
||||||
|
if !errors.As(err, &rejected) || rejected.StatusCode != 204 {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
}(i)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
// Last explicit policy wins; another call's policy must not leak.
|
||||||
|
if err := c.JSON(context.Background(), "GET", srv.URL+"?status=204", nil, nil, option, WithSuccessStatuses(204)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := c.JSON(context.Background(), "GET", srv.URL+"?status=204", nil, nil); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestJSONRejectedSuccessBodyRecovery(t *testing.T) {
|
||||||
|
var connections atomic.Int32
|
||||||
|
srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(201)
|
||||||
|
if r.URL.Path == "/stream" {
|
||||||
|
w.(http.Flusher).Flush()
|
||||||
|
<-r.Context().Done()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
io.WriteString(w, "private response")
|
||||||
|
}))
|
||||||
|
srv.Config.ConnState = func(_ net.Conn, s http.ConnState) {
|
||||||
|
if s == http.StateNew {
|
||||||
|
connections.Add(1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
srv.Start()
|
||||||
|
defer srv.Close()
|
||||||
|
c := testClient(t, Config{Timeout: time.Second})
|
||||||
|
for i := 0; i < 5; i++ {
|
||||||
|
err := c.JSON(context.Background(), "GET", srv.URL, nil, nil, WithSuccessStatuses(200))
|
||||||
|
var status *StatusError
|
||||||
|
if !errors.As(err, &status) || status.StatusCode != 201 {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if connections.Load() != 1 {
|
||||||
|
t.Fatal("small rejected bodies lost keep-alive", connections.Load())
|
||||||
|
}
|
||||||
|
start := time.Now()
|
||||||
|
err := c.JSON(context.Background(), "GET", srv.URL+"/stream", nil, nil, WithSuccessStatuses(200))
|
||||||
|
var status *StatusError
|
||||||
|
if !errors.As(err, &status) || status.StatusCode != 201 || time.Since(start) > 500*time.Millisecond {
|
||||||
|
t.Fatal("waited for rejected unbounded body", err)
|
||||||
|
}
|
||||||
|
if err := c.JSON(context.Background(), "GET", srv.URL, nil, nil, WithSuccessStatuses(201), WithMaxResponseBytes(4)); !errors.Is(err, ErrResponseTooLarge) {
|
||||||
|
t.Fatal("accepted body bypassed size limit", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func BenchmarkJSONSuccessStatus(b *testing.B) {
|
||||||
|
for _, restrict := range []bool{false, true} {
|
||||||
|
b.Run(strconv.FormatBool(restrict), func(b *testing.B) {
|
||||||
|
c, _ := New(Config{Transport: benchmarkTransport{`{"id":1}`, 200}, DisableLogging: true})
|
||||||
|
defer c.Close()
|
||||||
|
var options []RequestOption
|
||||||
|
if restrict {
|
||||||
|
options = []RequestOption{WithSuccessStatuses(200)}
|
||||||
|
}
|
||||||
|
var out struct {
|
||||||
|
ID int `json:"id"`
|
||||||
|
}
|
||||||
|
b.ReportAllocs()
|
||||||
|
b.ResetTimer()
|
||||||
|
for i := 0; i < b.N; i++ {
|
||||||
|
if err := c.JSON(context.Background(), "GET", "http://example.invalid", nil, &out, options...); err != nil {
|
||||||
|
b.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
34
internal/configdiag/error.go
Normal file
34
internal/configdiag/error.go
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
// Package configdiag carries value-free field diagnostics between framework
|
||||||
|
// modules without making them depend on the application's config package.
|
||||||
|
package configdiag
|
||||||
|
|
||||||
|
import "errors"
|
||||||
|
|
||||||
|
type Error struct {
|
||||||
|
Field, Reason string
|
||||||
|
cause error
|
||||||
|
}
|
||||||
|
|
||||||
|
func (e *Error) Error() string { return e.Field + ": " + e.Reason }
|
||||||
|
func (e *Error) Unwrap() error { return e.cause }
|
||||||
|
func New(field, reason string) error { return &Error{Field: field, Reason: reason} }
|
||||||
|
func Wrap(field, reason string, cause error) error {
|
||||||
|
return &Error{Field: field, Reason: reason, cause: cause}
|
||||||
|
}
|
||||||
|
func At(prefix string, err error) error {
|
||||||
|
if err == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var d *Error
|
||||||
|
if errors.As(err, &d) {
|
||||||
|
field := prefix
|
||||||
|
if d.Field != "" {
|
||||||
|
if field != "" {
|
||||||
|
field += "."
|
||||||
|
}
|
||||||
|
field += d.Field
|
||||||
|
}
|
||||||
|
return Wrap(field, d.Reason, err)
|
||||||
|
}
|
||||||
|
return Wrap(prefix, "validation failed", err)
|
||||||
|
}
|
||||||
@@ -7,6 +7,8 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
"unicode"
|
"unicode"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ErrResourcesRetained means background quiescence could not be established.
|
// ErrResourcesRetained means background quiescence could not be established.
|
||||||
@@ -24,8 +26,13 @@ func DefaultShutdownConfig() ShutdownConfig {
|
|||||||
return ShutdownConfig{StopTimeout: 5 * time.Second, DrainTimeout: 10 * time.Second, CancelTimeout: 5 * time.Second}
|
return ShutdownConfig{StopTimeout: 5 * time.Second, DrainTimeout: 10 * time.Second, CancelTimeout: 5 * time.Second}
|
||||||
}
|
}
|
||||||
func (c ShutdownConfig) Validate() error {
|
func (c ShutdownConfig) Validate() error {
|
||||||
if c.StopTimeout < 0 || c.DrainTimeout < 0 || c.CancelTimeout < 0 {
|
for _, f := range []struct {
|
||||||
return errors.New("lifecycle: shutdown timeouts must be nonnegative")
|
name string
|
||||||
|
value time.Duration
|
||||||
|
}{{"stopTimeout", c.StopTimeout}, {"drainTimeout", c.DrainTimeout}, {"cancelTimeout", c.CancelTimeout}} {
|
||||||
|
if f.value < 0 {
|
||||||
|
return configdiag.New(f.name, "must be nonnegative")
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,6 +14,20 @@ logger.InfoContext(ctx, "invoice.saved", "invoiceId", invoiceID)
|
|||||||
|
|
||||||
`Debug/Info/Warn/Error` 有包级和实例方法,也有带 Context 的方法。`requestId` 自动跟随 HTTP Context;接入 telemetry 后可带 `traceId`。不要在 Service 中改用 `context.Background()`,否则会丢失取消信号和关联字段。
|
`Debug/Info/Warn/Error` 有包级和实例方法,也有带 Context 的方法。`requestId` 自动跟随 HTTP Context;接入 telemetry 后可带 `traceId`。不要在 Service 中改用 `context.Background()`,否则会丢失取消信号和关联字段。
|
||||||
|
|
||||||
|
## 实例时区
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
server:
|
||||||
|
log:
|
||||||
|
timeZone: Asia/Shanghai
|
||||||
|
```
|
||||||
|
|
||||||
|
手写项目使用 `logx.New(logx.Config{TimeZone: "Asia/Shanghai"})`。配置名称采用 IANA 名称或 UTC;空字符串保持已有时间戳行为。无效名称在打开文件之前报 `timeZone` 的安全错误;统一加载后路径为 `server.log.timeZone`。
|
||||||
|
|
||||||
|
时区在创建 Logger 时解析,记录输出不重复加载;JSON/text、With/WithGroup 子日志和 Standard 适配器均继承该实例时区。不同 Logger 可同时使用不同地区,不修改全局 time.Local。仅转换记录时间戳,业务 `time.Time` 属性保留原时区;夏令时由时区数据处理。时区变更需要重建 Logger,现有热加载仍只支持级别。
|
||||||
|
|
||||||
|
使用 Go 标准库查找时区。极简镜像需提供时区数据,或执行 `go build -tags timetzdata .` 将标准库时区数据内置(增加约 450 KB);也可由手写 main 导入 time/tzdata。logx 自身不强制内置数据,不引入第三方依赖。脱敏、日志级别和文件所有权语义保持不变。可运行例子:[developerTools](../examples/developerTools/README.md)。
|
||||||
|
|
||||||
## 文件轮转
|
## 文件轮转
|
||||||
|
|
||||||
```go
|
```go
|
||||||
@@ -53,7 +67,7 @@ application := omni.New(
|
|||||||
)
|
)
|
||||||
```
|
```
|
||||||
|
|
||||||
`WithSlowRequestThreshold` 同时开启访问日志,0 禁用慢请求识别。也可以在 JSON/YAML 中配 `enableLogger: true`、`slowRequestThreshold: "500ms"`,或使用 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 配合 `GO_OMNI_ENABLE_LOGGER=true`。
|
`WithSlowRequestThreshold` 同时开启访问日志,0 禁用慢请求识别。也可以在配置文件的 server 下配 `enableLogger: true`、`slowRequestThreshold: "500ms"`。需要环境变量控制时,显式写 `slowRequestThreshold: "${SLOW_REQUEST_THRESHOLD:-500ms}"`;没有文件引用时不会读取同名环境变量。
|
||||||
|
|
||||||
每个请求记录一条 `http.request`:方法、路径、匹配路由、实际状态码、耗时、客户端 IP 和关联字段。正常请求 INFO,超过阈值 WARN(`slow: true`),5xx ERROR;返回的 error 会记录到结构化字段。计时覆盖 Handler 调用,不包含框架之外的代理排队,也不自动计入 Handler 返回后继续写入的自管流。
|
每个请求记录一条 `http.request`:方法、路径、匹配路由、实际状态码、耗时、客户端 IP 和关联字段。正常请求 INFO,超过阈值 WARN(`slow: true`),5xx ERROR;返回的 error 会记录到结构化字段。计时覆盖 Handler 调用,不包含框架之外的代理排队,也不自动计入 Handler 返回后继续写入的自管流。
|
||||||
|
|
||||||
|
|||||||
@@ -2,27 +2,33 @@ package logx
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
|
||||||
"log"
|
"log"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"strings"
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Validate checks configuration without opening files or creating resources.
|
// Validate checks configuration without opening files or creating resources.
|
||||||
func (c Config) Validate() error {
|
func (c Config) Validate() error { _, err := c.validate(); return err }
|
||||||
|
func (c Config) validate() (*time.Location, error) {
|
||||||
c = normalizeConfig(c)
|
c = normalizeConfig(c)
|
||||||
if _, err := parseLevel(c.Level); err != nil {
|
if _, err := parseLevel(c.Level); err != nil {
|
||||||
return errors.New("logx: invalid level")
|
return nil, configdiag.New("level", "must be debug, info, warn, warning or error")
|
||||||
}
|
}
|
||||||
if c.Format != "json" && c.Format != "text" {
|
if c.Format != "json" && c.Format != "text" {
|
||||||
return errors.New("logx: format must be json or text")
|
return nil, configdiag.New("format", "must be json or text")
|
||||||
}
|
}
|
||||||
switch c.Output {
|
switch c.Output {
|
||||||
case "stdout", "stderr", "discard", "none":
|
case "stdout", "stderr", "discard", "none":
|
||||||
default:
|
default:
|
||||||
return errors.New("logx: invalid output")
|
return nil, configdiag.New("output", "must be stdout, stderr, discard or none")
|
||||||
}
|
}
|
||||||
return c.Rotation.validate(c)
|
if err := c.Rotation.validate(c); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return logLocation(c.TimeZone)
|
||||||
}
|
}
|
||||||
|
|
||||||
type loggerContextKey struct{}
|
type loggerContextKey struct{}
|
||||||
|
|||||||
@@ -22,6 +22,7 @@ import (
|
|||||||
// standard stream. File takes precedence over Output and appends to the named
|
// standard stream. File takes precedence over Output and appends to the named
|
||||||
// file.
|
// file.
|
||||||
type Config struct {
|
type Config struct {
|
||||||
|
TimeZone string `json:"timeZone"` // Empty retains existing timestamp behavior.
|
||||||
Level string `json:"level"`
|
Level string `json:"level"`
|
||||||
Format string `json:"format"`
|
Format string `json:"format"`
|
||||||
Output string `json:"output"`
|
Output string `json:"output"`
|
||||||
@@ -73,7 +74,8 @@ func init() {
|
|||||||
// call Close when the process is shutting down.
|
// call Close when the process is shutting down.
|
||||||
func New(cfg Config) (*Logger, error) {
|
func New(cfg Config) (*Logger, error) {
|
||||||
cfg = normalizeConfig(cfg)
|
cfg = normalizeConfig(cfg)
|
||||||
if err := cfg.Validate(); err != nil {
|
location, err := cfg.validate()
|
||||||
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
level, err := parseLevel(cfg.Level)
|
level, err := parseLevel(cfg.Level)
|
||||||
@@ -95,6 +97,9 @@ func New(cfg Config) (*Logger, error) {
|
|||||||
} else {
|
} else {
|
||||||
handler = slog.NewJSONHandler(writer, opts)
|
handler = slog.NewJSONHandler(writer, opts)
|
||||||
}
|
}
|
||||||
|
if location != nil {
|
||||||
|
handler = zoneHandler{Handler: handler, location: location}
|
||||||
|
}
|
||||||
logger := slog.New(handler)
|
logger := slog.New(handler)
|
||||||
if cfg.Service != "" {
|
if cfg.Service != "" {
|
||||||
logger = logger.With("service", cfg.Service)
|
logger = logger.With("service", cfg.Service)
|
||||||
|
|||||||
@@ -1,13 +1,15 @@
|
|||||||
package logx
|
package logx
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"errors"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"sync"
|
"sync"
|
||||||
|
|
||||||
"gopkg.in/natefinch/lumberjack.v2"
|
"gopkg.in/natefinch/lumberjack.v2"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
)
|
)
|
||||||
|
|
||||||
// RotationConfig opts a file logger into size-based rotation. Zero limits use
|
// RotationConfig opts a file logger into size-based rotation. Zero limits use
|
||||||
@@ -21,12 +23,21 @@ type RotationConfig struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (r RotationConfig) validate(cfg Config) error {
|
func (r RotationConfig) validate(cfg Config) error {
|
||||||
if r.MaxSizeMB < 0 || r.MaxSizeMB > 1024*1024 || r.MaxBackups < 0 || r.MaxBackups > 10000 || r.MaxAgeDays < 0 || r.MaxAgeDays > 36500 {
|
for _, f := range []struct {
|
||||||
return errors.New("logx: invalid rotation limits")
|
name string
|
||||||
|
value, max int
|
||||||
|
}{{"maxSizeMb", r.MaxSizeMB, 1024 * 1024}, {"maxBackups", r.MaxBackups, 10000}, {"maxAgeDays", r.MaxAgeDays, 36500}} {
|
||||||
|
if f.value < 0 || f.value > f.max {
|
||||||
|
return configdiag.New("rotation."+f.name, fmt.Sprintf("must be between 0 and %d", f.max))
|
||||||
}
|
}
|
||||||
if r.Enabled && (cfg.File == "" || cfg.Writer != nil) {
|
|
||||||
return errors.New("logx: rotation requires a file and no custom writer")
|
|
||||||
}
|
}
|
||||||
|
if r.Enabled && cfg.File == "" {
|
||||||
|
return configdiag.New("file", "is required when rotation is enabled")
|
||||||
|
}
|
||||||
|
if r.Enabled && cfg.Writer != nil {
|
||||||
|
return configdiag.New("rotation.enabled", "cannot be enabled with a custom writer")
|
||||||
|
}
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
40
logx/timeZone.go
Normal file
40
logx/timeZone.go
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
package logx
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||||
|
)
|
||||||
|
|
||||||
|
func logLocation(name string) (*time.Location, error) {
|
||||||
|
if name == "" {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
location, err := time.LoadLocation(name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, configdiag.Wrap("timeZone", "must be an available IANA time zone or UTC", err)
|
||||||
|
}
|
||||||
|
return location, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Convert only the record timestamp. User time.Time attributes retain their
|
||||||
|
// original locations; child loggers inherit this instance's location.
|
||||||
|
type zoneHandler struct {
|
||||||
|
slog.Handler
|
||||||
|
location *time.Location
|
||||||
|
}
|
||||||
|
|
||||||
|
func (h zoneHandler) Handle(ctx context.Context, r slog.Record) error {
|
||||||
|
if !r.Time.IsZero() {
|
||||||
|
r.Time = r.Time.In(h.location)
|
||||||
|
}
|
||||||
|
return h.Handler.Handle(ctx, r)
|
||||||
|
}
|
||||||
|
func (h zoneHandler) WithAttrs(attrs []slog.Attr) slog.Handler {
|
||||||
|
return zoneHandler{h.Handler.WithAttrs(attrs), h.location}
|
||||||
|
}
|
||||||
|
func (h zoneHandler) WithGroup(name string) slog.Handler {
|
||||||
|
return zoneHandler{h.Handler.WithGroup(name), h.location}
|
||||||
|
}
|
||||||
133
logx/timeZone_test.go
Normal file
133
logx/timeZone_test.go
Normal file
@@ -0,0 +1,133 @@
|
|||||||
|
package logx
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestLoggerTimeZoneIsolation(t *testing.T) {
|
||||||
|
originalLocal := time.Local
|
||||||
|
stamp := time.Date(2026, 7, 1, 0, 0, 0, 123456000, time.UTC)
|
||||||
|
for _, format := range []string{"json", "text"} {
|
||||||
|
t.Run(format, func(t *testing.T) {
|
||||||
|
var a, b bytes.Buffer
|
||||||
|
sh, err := New(Config{TimeZone: "Asia/Shanghai", Format: format, Writer: &a})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer sh.Close()
|
||||||
|
utc, err := New(Config{TimeZone: "UTC", Format: format, Writer: &b})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer utc.Close()
|
||||||
|
r := slog.NewRecord(stamp, slog.LevelInfo, "timestamp", 0)
|
||||||
|
r.AddAttrs(slog.Time("eventTime", stamp), slog.String("password", "secret"))
|
||||||
|
if err := sh.With("service", "first").WithGroup("data").logger.Handler().Handle(context.Background(), r); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := utc.logger.Handler().Handle(context.Background(), r); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(a.String(), "08:00:00") || !strings.Contains(a.String(), "+08:00") || !strings.Contains(b.String(), "00:00:00") || strings.Contains(b.String(), "+08:00") {
|
||||||
|
t.Fatalf("a=%s b=%s", a.String(), b.String())
|
||||||
|
}
|
||||||
|
if strings.Contains(a.String(), "secret") || strings.Contains(b.String(), "secret") {
|
||||||
|
t.Fatal("redaction lost")
|
||||||
|
}
|
||||||
|
if format == "json" {
|
||||||
|
var out map[string]any
|
||||||
|
if err := json.Unmarshal(a.Bytes(), &out); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if out["time"] != "2026-07-01T08:00:00.123456+08:00" || out["data"].(map[string]any)["eventTime"] != "2026-07-01T00:00:00.123456Z" {
|
||||||
|
t.Fatal(out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
if time.Local != originalLocal {
|
||||||
|
t.Fatal("global time.Local changed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestLoggerTimeZoneValidationAndDST(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "not-created", "app.log")
|
||||||
|
if _, err := New(Config{TimeZone: "Secret/Not_A_Zone", File: path}); err == nil || strings.Contains(err.Error(), "Secret") {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Dir(path)); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("invalid zone opened writer")
|
||||||
|
}
|
||||||
|
var out bytes.Buffer
|
||||||
|
l, err := New(Config{TimeZone: "America/New_York", Writer: &out})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer l.Close()
|
||||||
|
for _, month := range []time.Month{time.January, time.July} {
|
||||||
|
r := slog.NewRecord(time.Date(2026, month, 1, 12, 0, 0, 0, time.UTC), slog.LevelInfo, "dst", 0)
|
||||||
|
if err := l.logger.Handler().Handle(context.Background(), r); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !strings.Contains(out.String(), "-05:00") || !strings.Contains(out.String(), "-04:00") {
|
||||||
|
t.Fatal(out.String())
|
||||||
|
}
|
||||||
|
// Explicit empty location preserves an incoming record's original zone.
|
||||||
|
out.Reset()
|
||||||
|
plain, _ := New(Config{Writer: &out})
|
||||||
|
defer plain.Close()
|
||||||
|
r := slog.NewRecord(time.Date(2026, 1, 1, 0, 0, 0, 0, time.FixedZone("original", 7200)), slog.LevelInfo, "unchanged", 0)
|
||||||
|
plain.logger.Handler().Handle(context.Background(), r)
|
||||||
|
if !strings.Contains(out.String(), "+02:00") {
|
||||||
|
t.Fatal(out.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestLoggerTimeZoneConcurrentChildren(t *testing.T) {
|
||||||
|
var out bytes.Buffer
|
||||||
|
l, err := New(Config{TimeZone: "Asia/Shanghai", Writer: &out})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer l.Close()
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 20; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(i int) {
|
||||||
|
defer wg.Done()
|
||||||
|
child := l.With("worker", i).WithGroup("event")
|
||||||
|
for n := 0; n < 10; n++ {
|
||||||
|
child.Standard(slog.LevelInfo).Print("message")
|
||||||
|
}
|
||||||
|
}(i)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
if strings.Count(out.String(), "+08:00") != 200 {
|
||||||
|
t.Fatal("timestamps missing")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func BenchmarkLoggerTimeZone(b *testing.B) {
|
||||||
|
for _, zone := range []string{"", "UTC", "Asia/Shanghai"} {
|
||||||
|
b.Run(zone, func(b *testing.B) {
|
||||||
|
l, err := New(Config{TimeZone: zone, Writer: io.Discard})
|
||||||
|
if err != nil {
|
||||||
|
b.Fatal(err)
|
||||||
|
}
|
||||||
|
defer l.Close()
|
||||||
|
b.ReportAllocs()
|
||||||
|
b.ResetTimer()
|
||||||
|
for i := 0; i < b.N; i++ {
|
||||||
|
l.InfoContext(context.Background(), "benchmark", "count", 1)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
60
reports/developerToolsAcceptance.md
Normal file
60
reports/developerToolsAcceptance.md
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
# v0.23 接入工具验收(2026-09-26)
|
||||||
|
|
||||||
|
本轮补齐通用配置诊断、JSON 成功状态限制、copyx 类型预检和实例日志时区;没有引入业务类型、映射配置或全局时区修改。
|
||||||
|
|
||||||
|
## 回归
|
||||||
|
|
||||||
|
- `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...`:48 个有测试的包通过,包含实际生成消费者、失败清理、后台收尾、错误目录和上传下载。
|
||||||
|
- 最后调整 copyx 空集合检查和分配后,copyx/config/logx/httpc 的 race 复测通过。
|
||||||
|
- Go 1.22.12 的 config/copyx/logx/httpc/database 测试通过;最后调整后前四个包复测通过。
|
||||||
|
- `go vet ./...`、`go build ./...` 通过。
|
||||||
|
- managedService 生成检查、race 和根目录构建通过;userService 的 race 通过。外部数据库测试没有提供连接配置时按原规则跳过。
|
||||||
|
- developerTools 示例的默认配置、UTC、非法数据库端口、非法日志等级验证通过。错误非零退出,具体字段/原因可见,非法输入值未输出。
|
||||||
|
- `FuzzClone` 最终运行 30 秒、4 worker,执行 3,925,833 次,通过(总时间 30.391 秒)。另有确定性的类型、nil、资源句柄和深拷贝隔离测试;fuzz 执行次数不代表形式化证明。
|
||||||
|
|
||||||
|
| 能力 | 验收场景 |
|
||||||
|
| --- | --- |
|
||||||
|
| 字段诊断 | database.port/pool/tls、server.log.level/rotation/timeZone、HTTP 连接配置、后台关闭超时、请求策略定位具体字段;文件/环境名/行列保留;errors.Is/As 保留原因链;未知业务校验仍安全脱敏 |
|
||||||
|
| JSON 成功状态 | 默认全部 2xx;指定 200;200..299 范围与空列表校验;选项快照/并发隔离;拒绝状态不改输出、不额外重试;响应体关闭;小响应复用连接;无限慢响应有界回收;成功响应大小限制保留 |
|
||||||
|
| copyx 预检 | 无样本检查结构、数组/map、指针、递归类型、重命名/忽略/严格字段覆盖;converter 不执行;并发 Copier 选项隔离;空集合也拒绝确定类型错误;失败不修改目标 |
|
||||||
|
| copyx 值语义 | nil 与空集合保持区别;允许原有合法的 nil 函数/空函数集合;实际非 nil 句柄仍拒绝;动态 interface 提示需要实际值或显式 converter;深拷贝 converter 返回原引用仍报错 |
|
||||||
|
| 日志时区 | JSON/text/子日志/标准库日志;上海和 UTC 实例隔离;纽约夏令时;默认保留原时间;业务 time.Time 属性不改;脱敏保留;非法时区不创建文件;200 条并发日志 race 通过 |
|
||||||
|
| 生成消费者 | 生成 Load 直接定位 server.log.level 和环境变量,校验先于依赖初始化;timeZone 直接绑定;重复生成/check 和业务文件保留通过 |
|
||||||
|
|
||||||
|
## 性能
|
||||||
|
|
||||||
|
Go 1.27.1、darwin/arm64、Apple M2 Max。无 race,每场景三轮、benchtime=200ms,表中取中位数。[基线原始结果](developerToolsBaseline.txt)、[当前原始结果](developerToolsBenchmark.txt)。
|
||||||
|
|
||||||
|
| 场景 | 修改前 | 修改后 | 修改后分配 |
|
||||||
|
| --- | ---: | ---: | ---: |
|
||||||
|
| 结构转换,每次创建 Copier | 1221 ns | 1261 ns | 1120 B / 17 allocs(不变) |
|
||||||
|
| 结构转换,复用 Copier | 733.4 ns | 719.1 ns | 168 B / 7 allocs(不变) |
|
||||||
|
| 64 KiB 字节深拷贝 | 6067 ns | 5613 ns | 65768 B / 9 allocs(不变) |
|
||||||
|
| HTTP 128 B JSON 解码 | 1374 ns | 1369 ns | 约 2348 B / 22 allocs |
|
||||||
|
| HTTP 4 KiB JSON 解码 | 7020 ns | 7130 ns | 约 17085 B / 31 allocs |
|
||||||
|
| HTTP 64 KiB JSON 解码 | 80622 ns | 80571 ns | 约 213924 B / 38 allocs |
|
||||||
|
| HTTP Do | 664.2 ns | 664.5 ns | 1296 B / 17 allocs(不变) |
|
||||||
|
|
||||||
|
新增功能的独立基准:
|
||||||
|
|
||||||
|
- 复用 Copier,预检含嵌套切片和显式 converter 的类型:597.8 ns、32 B、3 次分配。每次显式 Check 检查类型,不在每个正常业务 Copy 前重复完整预检。
|
||||||
|
- 同一简单 JSON 调用默认策略 1224 ns / 2345 B / 22 次分配,传成功状态选项 1249 ns / 2393 B / 23 次分配;此用例每次创建选项,应用也可复用选项。
|
||||||
|
- 同一日志输出到 io.Discard:默认 851.5 ns、UTC 885.0 ns、上海 888.7 ns;均为 112 B / 3 次分配。时区解析只发生在初始化/校验阶段。
|
||||||
|
|
||||||
|
这些是本机微基准,HTTP 使用模拟 Transport,日志使用 io.Discard,不代表实际网络或磁盘吞吐量;三轮测量有波动(64 KiB JSON 有一次较慢样本),不承诺固定性能比例。中途发现的普通拷贝额外分配已在最终测量前消除。
|
||||||
|
|
||||||
|
复现命令:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test ./copyx ./httpc ./logx -run '^$' -bench 'BenchmarkScenarios/(struct|bytes64KiB)|BenchmarkHTTPClient|BenchmarkTypeCheck|BenchmarkJSONSuccessStatus|BenchmarkLoggerTimeZone' -benchmem -benchtime=200ms -count=3
|
||||||
|
go test ./copyx -run '^$' -fuzz '^FuzzClone$' -fuzztime=30s -parallel=4
|
||||||
|
go run ./examples/developerTools
|
||||||
|
```
|
||||||
|
|
||||||
|
## 使用边界
|
||||||
|
|
||||||
|
- 错误文本变得更具体,识别诊断应使用错误类型/字段。自定义业务校验继续通过 config.Invalid/AtField 声明安全原因。
|
||||||
|
- 空集合中的确定类型错误现在会提前返回;需要调整实际类型或显式 converter。Check 不验证具体值、converter 成功性、nil 输入或深拷贝引用隔离,这些仍由实际 Copy 执行。
|
||||||
|
- 成功状态选项只接受 2xx 子集,不改变 HTTP 重试策略;业务响应内容是否完整仍由项目协议校验。
|
||||||
|
- 时区数据由运行环境提供;极简镜像可用 `go build -tags timetzdata .` 内置。logx 不强制引入约 450 KB 的时区数据库。
|
||||||
|
- 本轮未修改接入项目业务代码,未连接业务数据库或使用远程服务器压测。仍需项目升级后的业务验收。
|
||||||
42
reports/developerToolsBaseline.txt
Normal file
42
reports/developerToolsBaseline.txt
Normal file
@@ -0,0 +1,42 @@
|
|||||||
|
goos: darwin
|
||||||
|
goarch: arm64
|
||||||
|
pkg: git.yuntuo.hk/k2safe/go-omni/copyx
|
||||||
|
cpu: Apple M2 Max
|
||||||
|
BenchmarkScenarios/struct/reusefalse-12 195284 1218 ns/op 1120 B/op 17 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusefalse-12 194980 1227 ns/op 1120 B/op 17 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusefalse-12 186867 1221 ns/op 1120 B/op 17 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusetrue-12 340254 780.4 ns/op 168 B/op 7 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusetrue-12 339192 733.4 ns/op 168 B/op 7 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusetrue-12 330416 730.4 ns/op 168 B/op 7 allocs/op
|
||||||
|
BenchmarkScenarios/bytes64KiB-12 37689 6035 ns/op 10858.43 MB/s 65768 B/op 9 allocs/op
|
||||||
|
BenchmarkScenarios/bytes64KiB-12 41599 6222 ns/op 10533.71 MB/s 65768 B/op 9 allocs/op
|
||||||
|
BenchmarkScenarios/bytes64KiB-12 38743 6067 ns/op 10801.83 MB/s 65768 B/op 9 allocs/op
|
||||||
|
PASS
|
||||||
|
ok git.yuntuo.hk/k2safe/go-omni/copyx 2.867s
|
||||||
|
goos: darwin
|
||||||
|
goarch: arm64
|
||||||
|
pkg: git.yuntuo.hk/k2safe/go-omni/httpc
|
||||||
|
cpu: Apple M2 Max
|
||||||
|
BenchmarkHTTPClient/128B/decode-12 152140 1398 ns/op 2348 B/op 22 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/decode-12 181575 1372 ns/op 2348 B/op 22 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/decode-12 177343 1374 ns/op 2348 B/op 22 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/discard-12 235099 997.7 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/discard-12 234962 1012 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/discard-12 231510 990.5 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/decode-12 34695 7010 ns/op 17086 B/op 31 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/decode-12 34256 7020 ns/op 17084 B/op 31 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/decode-12 34084 7043 ns/op 17087 B/op 31 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/discard-12 223120 1058 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/discard-12 205618 1067 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/discard-12 224156 1079 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/decode-12 3010 80622 ns/op 213909 B/op 38 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/decode-12 3122 81358 ns/op 213949 B/op 38 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/decode-12 2982 80473 ns/op 213901 B/op 38 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/discard-12 110815 2154 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/discard-12 112252 2107 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/discard-12 114548 2094 ns/op 1835 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/Do-12 366976 662.1 ns/op 1296 B/op 17 allocs/op
|
||||||
|
BenchmarkHTTPClient/Do-12 351373 664.2 ns/op 1296 B/op 17 allocs/op
|
||||||
|
BenchmarkHTTPClient/Do-12 335104 679.0 ns/op 1296 B/op 17 allocs/op
|
||||||
|
PASS
|
||||||
|
ok git.yuntuo.hk/k2safe/go-omni/httpc 5.889s
|
||||||
66
reports/developerToolsBenchmark.txt
Normal file
66
reports/developerToolsBenchmark.txt
Normal file
@@ -0,0 +1,66 @@
|
|||||||
|
goos: darwin
|
||||||
|
goarch: arm64
|
||||||
|
pkg: git.yuntuo.hk/k2safe/go-omni/copyx
|
||||||
|
cpu: Apple M2 Max
|
||||||
|
BenchmarkScenarios/struct/reusefalse-12 185918 1261 ns/op 1120 B/op 17 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusefalse-12 191277 1254 ns/op 1120 B/op 17 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusefalse-12 192738 1267 ns/op 1120 B/op 17 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusetrue-12 336856 716.9 ns/op 168 B/op 7 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusetrue-12 333480 719.1 ns/op 168 B/op 7 allocs/op
|
||||||
|
BenchmarkScenarios/struct/reusetrue-12 338338 753.9 ns/op 168 B/op 7 allocs/op
|
||||||
|
BenchmarkScenarios/bytes64KiB-12 43203 5588 ns/op 11727.22 MB/s 65768 B/op 9 allocs/op
|
||||||
|
BenchmarkScenarios/bytes64KiB-12 43075 5613 ns/op 11675.18 MB/s 65768 B/op 9 allocs/op
|
||||||
|
BenchmarkScenarios/bytes64KiB-12 42343 5646 ns/op 11608.06 MB/s 65768 B/op 9 allocs/op
|
||||||
|
BenchmarkTypeCheck-12 377830 599.8 ns/op 32 B/op 3 allocs/op
|
||||||
|
BenchmarkTypeCheck-12 408428 597.8 ns/op 32 B/op 3 allocs/op
|
||||||
|
BenchmarkTypeCheck-12 409514 597.7 ns/op 32 B/op 3 allocs/op
|
||||||
|
PASS
|
||||||
|
ok git.yuntuo.hk/k2safe/go-omni/copyx 3.326s
|
||||||
|
goos: darwin
|
||||||
|
goarch: arm64
|
||||||
|
pkg: git.yuntuo.hk/k2safe/go-omni/httpc
|
||||||
|
cpu: Apple M2 Max
|
||||||
|
BenchmarkHTTPClient/128B/decode-12 170685 1369 ns/op 2347 B/op 22 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/decode-12 174030 1384 ns/op 2348 B/op 22 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/decode-12 179733 1368 ns/op 2348 B/op 22 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/discard-12 242395 1009 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/discard-12 242695 1010 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/128B/discard-12 230649 1008 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/decode-12 34294 7130 ns/op 17085 B/op 31 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/decode-12 34273 6992 ns/op 17085 B/op 31 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/decode-12 33758 7184 ns/op 17086 B/op 31 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/discard-12 215698 1059 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/discard-12 226962 1108 ns/op 1837 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/4KiB/discard-12 227298 1045 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/decode-12 3099 79497 ns/op 213929 B/op 38 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/decode-12 2742 80571 ns/op 213924 B/op 38 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/decode-12 2696 121378 ns/op 213907 B/op 38 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/discard-12 111092 2144 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/discard-12 103948 2132 ns/op 1836 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/64KiB/discard-12 115408 2111 ns/op 1835 B/op 21 allocs/op
|
||||||
|
BenchmarkHTTPClient/Do-12 374404 656.7 ns/op 1296 B/op 17 allocs/op
|
||||||
|
BenchmarkHTTPClient/Do-12 353349 664.5 ns/op 1296 B/op 17 allocs/op
|
||||||
|
BenchmarkHTTPClient/Do-12 348849 665.8 ns/op 1296 B/op 17 allocs/op
|
||||||
|
BenchmarkJSONSuccessStatus/false-12 203222 1233 ns/op 2345 B/op 22 allocs/op
|
||||||
|
BenchmarkJSONSuccessStatus/false-12 201913 1213 ns/op 2345 B/op 22 allocs/op
|
||||||
|
BenchmarkJSONSuccessStatus/false-12 201854 1224 ns/op 2345 B/op 22 allocs/op
|
||||||
|
BenchmarkJSONSuccessStatus/true-12 199285 1249 ns/op 2393 B/op 23 allocs/op
|
||||||
|
BenchmarkJSONSuccessStatus/true-12 184888 1254 ns/op 2393 B/op 23 allocs/op
|
||||||
|
BenchmarkJSONSuccessStatus/true-12 194844 1222 ns/op 2393 B/op 23 allocs/op
|
||||||
|
PASS
|
||||||
|
ok git.yuntuo.hk/k2safe/go-omni/httpc 7.606s
|
||||||
|
goos: darwin
|
||||||
|
goarch: arm64
|
||||||
|
pkg: git.yuntuo.hk/k2safe/go-omni/logx
|
||||||
|
cpu: Apple M2 Max
|
||||||
|
BenchmarkLoggerTimeZone/#00-12 256226 850.2 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/#00-12 277039 852.6 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/#00-12 285422 851.5 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/UTC-12 270704 885.0 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/UTC-12 277340 879.1 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/UTC-12 271767 890.3 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/Asia/Shanghai-12 273482 883.3 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/Asia/Shanghai-12 274610 888.7 ns/op 112 B/op 3 allocs/op
|
||||||
|
BenchmarkLoggerTimeZone/Asia/Shanghai-12 270154 892.8 ns/op 112 B/op 3 allocs/op
|
||||||
|
PASS
|
||||||
|
ok git.yuntuo.hk/k2safe/go-omni/logx 2.665s
|
||||||
@@ -140,7 +140,8 @@ def main():
|
|||||||
http_events = [json.loads(line) for line in http_output.splitlines() if line.startswith("{")]
|
http_events = [json.loads(line) for line in http_output.splitlines() if line.startswith("{")]
|
||||||
http_passed = {e.get("Test") for e in http_events if e.get("Action") == "pass"}
|
http_passed = {e.get("Test") for e in http_events if e.get("Action") == "pass"}
|
||||||
assert {"TestProxyRouting", "TestJSONHeadersConcurrentSnapshot", "TestPoolLimitsAndIdleExpiry",
|
assert {"TestProxyRouting", "TestJSONHeadersConcurrentSnapshot", "TestPoolLimitsAndIdleExpiry",
|
||||||
"TestErrorBodyPoolReuseAndDrainBound", "TestSlowKnownErrorBodyHasShortDrainBudget"} <= http_passed
|
"TestErrorBodyPoolReuseAndDrainBound", "TestSlowKnownErrorBodyHasShortDrainBudget",
|
||||||
|
"TestJSONSuccessStatuses", "TestJSONSuccessStatusConcurrentOptions", "TestJSONRejectedSuccessBodyRecovery"} <= http_passed
|
||||||
example_output = run(["go", "run", MODULE + "/examples/httpClient"], starter, env)
|
example_output = run(["go", "run", MODULE + "/examples/httpClient"], starter, env)
|
||||||
assert "authenticated request succeeded" in example_output
|
assert "authenticated request succeeded" in example_output
|
||||||
checks += ["remoteHTTPClientRaceTests", "remoteHTTPProxyAndTimeouts", "remoteHTTPHeaderIsolation",
|
checks += ["remoteHTTPClientRaceTests", "remoteHTTPProxyAndTimeouts", "remoteHTTPHeaderIsolation",
|
||||||
@@ -150,12 +151,24 @@ def main():
|
|||||||
config_events = [json.loads(line) for line in config_output.splitlines() if line.startswith("{")]
|
config_events = [json.loads(line) for line in config_output.splitlines() if line.startswith("{")]
|
||||||
config_passed = {e.get("Test") for e in config_events if e.get("Action") == "pass"}
|
config_passed = {e.get("Test") for e in config_events if e.get("Action") == "pass"}
|
||||||
assert {"TestExplicitReferencesAcrossFormats", "TestReferenceFallbackAndFixedValues",
|
assert {"TestExplicitReferencesAcrossFormats", "TestReferenceFallbackAndFixedValues",
|
||||||
"TestReferenceDiagnosticsAndScalarLimits", "TestRemovedEnvTagsAndFieldSuggestion"} <= config_passed
|
"TestReferenceDiagnosticsAndScalarLimits", "TestRemovedEnvTagsAndFieldSuggestion",
|
||||||
|
"TestModuleFieldDiagnostics", "TestModuleDiagnosticsKeepReferenceAndCause"} <= config_passed
|
||||||
run([cli, "api", "generate", "--check"], starter, env)
|
run([cli, "api", "generate", "--check"], starter, env)
|
||||||
run(["go", "run", ".", "-config", "etc/config.yaml", "-check-config"], starter, env)
|
run(["go", "run", ".", "-config", "etc/config.yaml", "-check-config"], starter, env)
|
||||||
checks += ["remoteExplicitConfigurationRaceTests", "literalAndRequiredDefaultReferences",
|
checks += ["remoteExplicitConfigurationRaceTests", "literalAndRequiredDefaultReferences",
|
||||||
"remoteCheckConfigBeforeDependencies", "removedImplicitEnvironmentRules"]
|
"remoteCheckConfigBeforeDependencies", "removedImplicitEnvironmentRules"]
|
||||||
|
|
||||||
|
for package, expected in {
|
||||||
|
"copyx": {"TestCheckTypesAndEmptyCollections", "TestCheckRulesAndConverterBoundaries", "TestCopierCheckConcurrentAndOptionsIsolation"},
|
||||||
|
"logx": {"TestLoggerTimeZoneIsolation", "TestLoggerTimeZoneValidationAndDST", "TestLoggerTimeZoneConcurrentChildren"},
|
||||||
|
}.items():
|
||||||
|
output = run(["go", "test", "-race", "-count=1", "-json", MODULE + "/" + package], starter, env)
|
||||||
|
events = [json.loads(line) for line in output.splitlines() if line.startswith("{")]
|
||||||
|
passed = {event.get("Test") for event in events if event.get("Action") == "pass"}
|
||||||
|
assert expected <= passed
|
||||||
|
checks += ["remotePreciseModuleDiagnostics", "remoteJSONSuccessStatuses", "remoteRejectedStatusBodyCleanup",
|
||||||
|
"remoteCopyTypePreflight", "remoteEmptyCollectionTypeRejection", "remoteLoggerTimeZoneIsolation"]
|
||||||
|
|
||||||
checks += ["flatFieldComposition", "copyxThreeEndpointConsumer", "explicitNullAndMoneyPolicies", "fieldAdditionWithoutCopyRuleChanges", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"]
|
checks += ["flatFieldComposition", "copyxThreeEndpointConsumer", "explicitNullAndMoneyPolicies", "fieldAdditionWithoutCopyRuleChanges", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"]
|
||||||
|
|
||||||
checks += ["generatedUnifiedConfig", "jsonYamlXmlAndCustomDecoder", "configValidationBeforeDependencies", "configEnvProvenance", "configExtensionPreserved", "configLoaderDriftCheck"]
|
checks += ["generatedUnifiedConfig", "jsonYamlXmlAndCustomDecoder", "configValidationBeforeDependencies", "configEnvProvenance", "configExtensionPreserved", "configLoaderDriftCheck"]
|
||||||
@@ -172,6 +185,9 @@ def main():
|
|||||||
sources += ["cmd/go-omni/apigen/configLoader_test.go", "cmd/go-omni/apigen/testdata/configuration/consumer_test.go", "cmd/go-omni/scaffold/templates/api-load.go.tmpl", "cmd/go-omni/scaffold/templates/api-loader.go.tmpl"]
|
sources += ["cmd/go-omni/apigen/configLoader_test.go", "cmd/go-omni/apigen/testdata/configuration/consumer_test.go", "cmd/go-omni/scaffold/templates/api-load.go.tmpl", "cmd/go-omni/scaffold/templates/api-loader.go.tmpl"]
|
||||||
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "httpc").glob("*.go")]
|
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "httpc").glob("*.go")]
|
||||||
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "examples/httpClient").glob("*.go")]
|
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "examples/httpClient").glob("*.go")]
|
||||||
|
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "logx").glob("*.go")]
|
||||||
|
sources += ["internal/configdiag/error.go", "database/mysql/mysql.go", "database/connector.go", "database/database.go", "contract/policy.go"]
|
||||||
|
sources += [str(p.relative_to(ROOT)) for p in (ROOT / "examples/developerTools").iterdir() if p.is_file()]
|
||||||
report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"],
|
report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"],
|
||||||
"freshModuleCache": True, "noLocalReplace": True,
|
"freshModuleCache": True, "noLocalReplace": True,
|
||||||
"durationSeconds": round(time.monotonic() - started, 2), "checks": checks,
|
"durationSeconds": round(time.monotonic() - started, 2), "checks": checks,
|
||||||
|
|||||||
Reference in New Issue
Block a user