diff --git a/CHANGELOG.md b/CHANGELOG.md index 9ce9a2f..50030c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,13 @@ # Changelog +## v0.23.0-preview（2026-09-26） + +- MySQL/连接池、日志、HTTP 客户端、生命周期和请求策略返回字段级安全诊断，配置加载器保留文件/变量/位置与原始错误链。 +- httpc.JSON 新增 WithSuccessStatuses，单次调用可限制允许的 2xx，默认兼容，拒绝响应继续关闭和有界回收。 +- copyx.Check[S,T] / Copier.Check 预检嵌套类型与现有规则；修正空集合隐藏类型不匹配，动态接口和 converter 的具体行为继续运行时验证。 +- logx.Config.TimeZone 绑定 Logger 实例，JSON/text/子 Logger/标准日志适配一致；不修改 time.Local 或业务时间属性。 +- 无新映射文件、强制模块或第三方依赖；更新 README、各包指南和 developerTools 可运行示例。错误文本更精确，空集合中的确定类型错误现在会被拒绝。 + ## v0.22.0-preview（2026-09-26） - 文件显式环境引用与备用值，字符串组合/转义，安全有界展开和类型绑定；变量值原样插入、不递归处理。 diff --git a/CONFIG.md b/CONFIG.md index e07ca71..a8e5456 100644 --- a/CONFIG.md +++ b/CONFIG.md @@ -141,6 +141,20 @@ config "etc/config.yaml":7:9 env "DB_PORT" field "database.port": expected a val 文件最多 1 MiB；单个环境值/展开字段最多 1 MiB，展开字符串合计/变量缓存分别最多 4 MiB；解析/展开限制 64 层、100000 个节点与引用计数。拒绝无效 UTF-8、多文档、重复字段、未知字段、非普通文件；YAML 不接受 alias/merge/custom tag，XML 不接受 DTD/外部实体。不执行 shell，不自动加载 `.env`。 +## 框架模块的字段诊断 + +v0.23 起 MySQL（含连接池/TLS）、通用数据库池、日志（含轮转/时区）、HTTP 客户端、退出阶段和请求策略返回字段级安全原因。沿用项目已有的 `config.AtField("database", err)` / `config.AtField("server", err)`，自动组合嵌套字段，不需业务重复校验基础参数。 + +例如文件中 `database.port: "${DB_PORT}"` 的引用值为 70000： + +```text +config "etc/config.yaml":3:9 env "DB_PORT" field "database.port": must be between 1 and 65535 +``` + +日志级别非法指向 `server.log.level`；池参数指向 `database.pool.maxIdleConns` / `pingTimeout`。框架只公开已定义的安全原因，第三方/项目普通 error 仍显示 validation failed 并保留 errors.Is/As 原因链；业务使用 Invalid/AtField 提供公开原因。不要解析错误字符串；使用 `*config.Error` 字段。未提供字段的位置可能不可用，字段与原因仍保留。 + +生成项目现有 Load 和 `-check-config` 自动获得这些诊断。日志可在 `server.log.timeZone` 配置 IANA 名称，见 [实例时区](logx/README.md#实例时区)。 + ## 可选解析器与测试来源 内置格式无需项目手写加载器。需要额外格式时在已有 NewLoader 注册 Decoder，后续仍自动经过同一引用/绑定/校验流程： diff --git a/COPY.md b/COPY.md index 43e0c75..52d162e 100644 --- a/COPY.md +++ b/COPY.md @@ -28,6 +28,31 @@ if err := copyx.Copy(&input, req); err != nil { return err } 新字段：普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时，两边各声明一次；同名同类型不增加拷贝配置。检查在运行时发生，实际类型对必须有测试，不能把运行时工具当作旧生成器的编译期字段覆盖检查。 +## 类型预检 + +v0.23 新增可选预检，无需填写映射文件或构造测试值： + +```go +// 参数方向是源 → 目标；与 Copy 使用同一组选项。 +if err := copyx.Check[Request, BusinessInput](options...); err != nil { + return err +} +// 复用已有 Copier：传结果类型，不添加 Copy 所需的额外输出指针。 +err := copier.Check(reflect.TypeFor[Request](), reflect.TypeFor[BusinessInput]()) +``` + +适合 CI 测试或初始化检查；这是程序执行的类型检查，不是 Go 编译器自动证明。递归检查切片/数组元素、map 键值、指针、嵌套字段、改名、忽略和严格覆盖；用已访问类型对处理递归结构。不执行 converter，也不调用构造器，不读取实例值。错误路径如 `$.Rows[].Value: copyx: incompatible types: string -> int`。 + +Check 复用 Copy 的规则，包括浅拷贝对同类型容器的直接赋值：这种情况下内部 converter 不会自动逐项执行。需要处理内部字段时选择深拷贝或容器 converter。 + +- 未知具体值的源 interface 需要运行时验证；无法静态判定时返回 `ErrDynamicType`。项目可给该接口类型对提供精确 converter 并承担内容验证。 +- 已注册 converter 的签名被接受，但函数不在 Check 中执行；其返回错误、panic、金额规则不能靠预检证明。 +- nil 到非可空目标、map 键转换碰撞、深拷贝结果是否残留源引用继续在 Copy 时检查。Check 成功不会关闭这些保护。 +- 实际 Copy 也修正了空集合的检查缺口：确定的嵌套类型不匹配不再因为切片/map 为空或数组长度为 0 而通过。需要这些转换时提供显式 converter；目标仍只在整体成功后更新。 +- 不引入全局配置、映射语言或业务类型；Copier.Check 可并发调用，不共享每次检查的遍历状态。 + +示例：[developerTools](examples/developerTools/README.md)。普通同名同类型字段不需要额外配置。 + ## 深拷贝与构造器 ```go diff --git a/DEVELOPMENT.md b/DEVELOPMENT.md index 04fbbb6..0431069 100644 --- a/DEVELOPMENT.md +++ b/DEVELOPMENT.md @@ -1,4 +1,4 @@ -# 通用业务开发指南（v0.22.0-preview） +# 通用业务开发指南（v0.23.0-preview） 从 [README](README.md) 创建服务后，业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力；字段、错误码、权限、表结构和事务边界都由项目定义。 diff --git a/PROJECT.md b/PROJECT.md index 8e2fd32..754a3c2 100644 --- a/PROJECT.md +++ b/PROJECT.md @@ -1,6 +1,6 @@ # 生成项目与文件维护边界 -v0.22.0-preview 提供受管理的生成项目。安装同版远程 CLI，生成的项目直接引用该版本，无需本地 replace。根目录入口支持 `go run .` 和 `go build .`；框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 +v0.23.0-preview 提供受管理的生成项目。安装同版远程 CLI，生成的项目直接引用该版本，无需本地 replace。根目录入口支持 `go run .` 和 `go build .`；框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。 ## 创建和日常开发 @@ -215,7 +215,7 @@ get /download (DownloadReq) ## 已有服务接入步骤 -每个独立服务生成一个项目，module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.22.0-preview： +每个独立服务生成一个项目，module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.23.0-preview： 1. 先保留旧服务，整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`；不对旧入口强制覆盖。 2. 在 `.api` 中选择 `@app(defaultResponse: ...)`，显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。 diff --git a/README.md b/README.md index 1d23267..3cd2a90 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,9 @@ GoOmni 是面向多个业务项目复用的 Go 后端框架：用 `.api` 定义接口、请求和返回类型，生成项目目录、HTTP 接入代码及 OpenAPI，业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`，或单独引入日志、存储等模块。 -**从这份 README 开始接入。** 当前版本 **`v0.22.0-preview`**，新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。 +**从这份 README 开始接入。** 当前版本 **`v0.23.0-preview`**，新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。 + +**v0.23 改动**：配置校验定位具体字段和安全原因；`httpc.JSON` 可选限制成功状态；`copyx.Check` 无需样本预检类型；`logx` 支持实例独立时区。[四项完整示例](examples/developerTools/README.md)。 **v0.19 改动**：新增可选 `copyx` 工具，用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换；深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**，已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整，不能直接删除仍在调用的旧文件。 @@ -37,21 +39,21 @@ Go **1.22+**。框架不依赖 Gin，默认使用标准库网络层，提供可 | 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`；Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) | | 错误目录与动态提示 | `WithErrors(catalog)` 注册一次；`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) | | OpenAPI 与错误公共引用 | `.api` 自动生成；手写接口可使用 `openapi` 注册器；Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) | -| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`；支持轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) | +| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`；支持独立时区、轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) | | 文件配置与环境引用 | Default()→JSON/YAML/XML 字段引用→类型绑定→校验；无隐式覆盖，支持 `-check-config` | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) | | 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain，RequestShutdown 通知故障；健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) | | 数据库、连接池与事务 | `database.Open` / `OpenConnector`；事务与检查由项目显式调用 | [通用数据库](database/README.md)、[MySQL TLS/Logger](database/mysql/README.md) | | 可选 GORM 与迁移 | gormx 复用已有连接池；`go-omni migrate` 预览，显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) | | 认证与权限接口 | 注入 Authenticator / Policy；JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) | | 缓存 | `cache.NewMemory` 或 Redis 实现；业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) | -| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时；JSON 支持单次请求头/限额；`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) | +| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时；JSON 支持单次成功状态限制/请求头/限额；`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) | | 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware；跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) | | 本地后台任务与调度 | `task.New`、Submit、Every；有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) | | 持久队列 | 业务使用 Publisher / Consumer；Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) | | 文件存储、直传、云端分片 | Store 的 Put/Get/Delete；Local、S3/R2、OSS 按配置装配；扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) | | 日志关联、指标与链路 | `metrics.New().Middleware()`；telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) | | 审计与分页 | `audit.New` 注入 Sink/白名单；`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) | -| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`；Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) | +| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`；`Check[S,T]` 预检类型，Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) | | 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) | ## 带认证的出站 HTTP @@ -126,7 +128,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可 ```sh export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk" -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.22.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview export PATH="$(go env GOPATH)/bin:$PATH" go-omni --help ``` @@ -283,7 +285,7 @@ go-omni new --api catalog.api \ 在业务项目的 **go.mod 所在目录**执行： ```sh -go get git.yuntuo.hk/k2safe/go-omni@v0.22.0-preview +go get git.yuntuo.hk/k2safe/go-omni@v0.23.0-preview ``` 已有项目可直接复用现有目录、Service、Repository 和数据库连接，先选择一个接口接入。也可以只用 `logx`、`storage` 等包，暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目，按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。 @@ -509,6 +511,28 @@ if err := copier.Copy(&view, input); err != nil { return err } 自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错；字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md)，完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查，类型对需测试；深拷贝不适用于连接、锁等资源。 +### 减少接入代码：四个常用入口 + +```yaml +server: + log: + timeZone: Asia/Shanghai +``` + +时区由 Logger 实例管理，不修改全局 `time.Local`。非法日志级别、数据库端口/连接池等配置会指出 `server.log.level`、`database.port` 等字段和原因，不打印输入值。详见 [配置诊断](CONFIG.md#框架模块的字段诊断)、[日志时区](logx/README.md#实例时区)。 + +```go +// 本次请求只接受 HTTP 200；默认仍接受所有 2xx。 +err := client.JSON(ctx, http.MethodGet, url, nil, &result, + httpc.WithSuccessStatuses(http.StatusOK), +) + +// CI/启动时检查类型，无需构造样本；仍使用 Copy 的同一组 Option。 +err = copyx.Check[Request, BusinessInput](options...) +``` + +状态不符合要求返回 `*httpc.StatusError`，不会写入 result，响应体仍由框架关闭。Check 不执行 converter；动态 interface、具体 nil/转换失败和深拷贝引用检查有明确边界，见 [类型预检](COPY.md#类型预检)。完整可运行代码见 [developerTools](examples/developerTools/README.md)。 + ### 数据库、存储及其他模块 生成项目在 `internal/svc` 中创建依赖，提供给 Logic/Repository，并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc，再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`；不要对同一资源重复登记。 @@ -587,7 +611,7 @@ go test ./... `main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理，手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目，重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段，明确的签名错误会给出原签名与期望签名；生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交，CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。 -CLI 与业务 module 固定使用 `v0.22.0-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load，再升级依赖与 CLI 并重新生成，框架维护的 main/runtimeGen 自动更新；v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理，不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移)，旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 +CLI 与业务 module 固定使用 `v0.23.0-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load，再升级依赖与 CLI 并重新生成，框架维护的 main/runtimeGen 自动更新；v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理，不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移)，旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。 ## 按需深入 diff --git a/RELEASE.md b/RELEASE.md index 5affa2b..ed3096d 100644 --- a/RELEASE.md +++ b/RELEASE.md @@ -1,4 +1,27 @@ -# v0.22.0-preview +# v0.23.0-preview + +减少业务接入时重复维护的配置校验、HTTP 包装和类型检查。 + +- 配置诊断定位 database.port、database.pool.maxIdleConns、server.log.level/timeZone 等具体字段，保留文件、环境变量、行列和安全原因；凭据不进入公开提示，原始原因可 errors.Is/As。 +- JSON 可传 `httpc.WithSuccessStatuses(200)`，默认仍接受全部 2xx；不匹配时不改目标，继续负责有界读取、关闭和连接回收，不添加新的重试规则。 +- `copyx.Check[S,T](options...)` 或已有 Copier.Check 复用同一组 Go 选项，无需样本和映射文件；实际拷贝不再因集合为空而漏过确定的类型不匹配。动态 interface、nil 值、converter 失败及深拷贝共享引用仍有运行时检查。 +- `server.log.timeZone: Asia/Shanghai` 配置实例时区；JSON/text/子日志一致，业务 time.Time 属性保持原样。极简镜像可用 `go build -tags timetzdata .` 内置时区数据，库不强制增加体积。 + +```sh +go get git.yuntuo.hk/k2safe/go-omni@v0.23.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview +go-omni api generate +go-omni api generate --check +go run . -config etc/config.yaml -check-config +``` + +现有调用接口保留；配置错误文本现在更精确，业务应通过类型/字段识别错误。空集合里确定的类型错误会提前暴露，需要显式 converter 或调整实际类型，不应靠空值跳过校验。预检通过不代表 converter 对所有业务值必定成功。当前模板最低版本仍是 v0.22，本轮新增 runtime 能力需使用 v0.23。 + +[完整示例](examples/developerTools/README.md)、[配置](CONFIG.md)、[JSON 选项](httpc/README.md)、[类型预检](COPY.md#类型预检)、[日志时区](logx/README.md#实例时区)。验收与性能记录见 [本轮验收](reports/developerToolsAcceptance.md)。仍为 preview。 + +--- + +# v0.22.0-preview（历史版本） 配置文件明确决定每个值的来源，项目只维护 Config、Default()、Validate() 和文件。 diff --git a/cmd/go-omni/README.md b/cmd/go-omni/README.md index 6635063..96a2324 100644 --- a/cmd/go-omni/README.md +++ b/cmd/go-omni/README.md @@ -14,7 +14,7 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目 ```sh # 仓库读取权限与 PATH 配置见根目录 README -go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.22.0-preview +go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview go-omni new --module example.com/hello ./hello # 从自己的契约生成；service.api 及其相对 import 文件需要在本地存在 @@ -49,14 +49,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护；业务文件按字节 ## 版本与迁移 -CLI 与框架依赖固定使用 v0.22.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换： +CLI 与框架依赖固定使用 v0.23.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换： ```sh # 在 go-omni 源码仓库中执行 go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello ``` -当前模板与依赖以 v0.22.0-preview 为基线，继续使用 manifest v2；GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级)：原始模板可自动替换，含业务修改的基础文件按预览合并；通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。 +当前模板与依赖以 v0.23.0-preview 为基线，继续使用 manifest v2；GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级)：原始模板可自动替换，含业务修改的基础文件按预览合并；通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。 ## 数据库迁移命令 diff --git a/cmd/go-omni/apigen/generate.go b/cmd/go-omni/apigen/generate.go index b4bf05e..a5f6336 100644 --- a/cmd/go-omni/apigen/generate.go +++ b/cmd/go-omni/apigen/generate.go @@ -22,7 +22,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" // Runtime-only releases need not raise the managed template compatibility floor. const minimumVersion = "v0.22.0-preview" -const currentVersion = "v0.22.0-preview" +const currentVersion = "v0.23.0-preview" const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n" type Options struct { diff --git a/cmd/go-omni/apigen/templates.go b/cmd/go-omni/apigen/templates.go index 2d868eb..615c9b5 100644 --- a/cmd/go-omni/apigen/templates.go +++ b/cmd/go-omni/apigen/templates.go @@ -60,7 +60,9 @@ Unimplemented/nil factories fail startup and clean registered dependencies. Existing projects retain middleware: handlers until explicitly migrated. Ordinary Logic uses the generated Request/Response types directly. Where an independent business type is needed, use ordinary Go or the optional copyx package. -Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use explicit ${NAME} / ${NAME:-fallback} references; fixed values are never overridden by inferred environment names. Use -check-config to validate without opening dependencies. See the framework CONFIG.md. +Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use explicit ${NAME} / ${NAME:-fallback} references; fixed values are never overridden by inferred environment names. Use -check-config to validate without opening dependencies. Framework validation +identifies fields such as server.log.level and database.port. server.log.timeZone +optionally selects a per-logger IANA zone. See the framework CONFIG.md. There are no mappings/mappingFiles settings or generated mapping files in v0.19. See COPY.md in the framework repository for copy policies and migration steps, diff --git a/cmd/go-omni/apigen/testdata/configuration/consumer_test.go b/cmd/go-omni/apigen/testdata/configuration/consumer_test.go index 12eab95..1b397be 100644 --- a/cmd/go-omni/apigen/testdata/configuration/consumer_test.go +++ b/cmd/go-omni/apigen/testdata/configuration/consumer_test.go @@ -91,3 +91,23 @@ func TestConfigurationCheckProcess(t *testing.T) { t.Fatal(err, string(out)) } } + +func TestGeneratedModuleFieldDiagnostic(t *testing.T) { + path := filepath.Join(t.TempDir(), "config.yaml") + t.Setenv("EXAMPLE_LOG_LEVEL", "invalid-private-level") + if err := os.WriteFile(path, []byte("server:\n log:\n level: '${EXAMPLE_LOG_LEVEL}'\n"), 0600); err != nil { + t.Fatal(err) + } + err := run(context.Background(), path) + var d *omniConfig.Error + if !errors.As(err, &d) || d.Field != "server.log.level" || d.Line != 3 || d.Env != "EXAMPLE_LOG_LEVEL" || !strings.Contains(d.Reason, "debug") || strings.Contains(err.Error(), "invalid-private-level") { + t.Fatal(err) + } + if err := os.WriteFile(path, []byte("server:\n log:\n timeZone: Asia/Shanghai\n"), 0600); err != nil { + t.Fatal(err) + } + cfg, err := config.Load(path) + if err != nil || cfg.Server.Log.TimeZone != "Asia/Shanghai" { + t.Fatal(err) + } +} diff --git a/cmd/go-omni/scaffold/version.go b/cmd/go-omni/scaffold/version.go index 643f3a2..d6c56c0 100644 --- a/cmd/go-omni/scaffold/version.go +++ b/cmd/go-omni/scaffold/version.go @@ -10,7 +10,7 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni" // Source builds can carry an unpublished VCS version (including +dirty). // Keep this fallback in sync with the next release; installed CLIs use their // downloaded module version, which also has a module checksum. -const defaultFrameworkVersion = "v0.22.0-preview" +const defaultFrameworkVersion = "v0.23.0-preview" // This target is not a claim that the release already exists. Source users // select a local framework checkout until the module templates are published. diff --git a/config/diagnostic.go b/config/diagnostic.go index f4484d5..46b975d 100644 --- a/config/diagnostic.go +++ b/config/diagnostic.go @@ -4,6 +4,8 @@ import ( "errors" "fmt" "strings" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) // Error is a value-free diagnostic. Cause may contain private parser details; @@ -56,6 +58,10 @@ func validationError(err error) *Error { out.cause = err return &out } + var nested *configdiag.Error + if errors.As(err, &nested) { + return &Error{Field: nested.Field, Reason: nested.Reason, cause: err} + } var c *ConfigError if errors.As(err, &c) { return &Error{Field: c.Field, Reason: c.Message, cause: err} diff --git a/config/moduleDiagnostics_test.go b/config/moduleDiagnostics_test.go new file mode 100644 index 0000000..e9f5edc --- /dev/null +++ b/config/moduleDiagnostics_test.go @@ -0,0 +1,85 @@ +package config_test + +import ( + "errors" + "os" + "path/filepath" + "strings" + "testing" + + "git.yuntuo.hk/k2safe/go-omni/config" + mysql "git.yuntuo.hk/k2safe/go-omni/database/mysql" + "git.yuntuo.hk/k2safe/go-omni/httpc" +) + +type moduleConfig struct { + Database mysql.Config `json:"database"` + Server config.Config `json:"server"` + Client httpc.Config `json:"client"` +} + +func (c moduleConfig) Validate() error { + if err := c.Database.Validate(); err != nil { + return config.AtField("database", err) + } + if err := c.Server.Validate(); err != nil { + return config.AtField("server", err) + } + return config.AtField("client", c.Client.Validate()) +} +func TestModuleFieldDiagnostics(t *testing.T) { + for _, tc := range []struct{ name, source, field, reason string }{ + {"port", "database:\n port: 70000\n", "database.port", "1 and 65535"}, + {"host", "database:\n host: 'invalid host'\n", "database.host", "hostname"}, + {"pool", "database:\n pool:\n maxIdleConns: 999\n", "database.pool.maxIdleConns", "maxOpenConns"}, + {"ping", "database:\n pool:\n pingTimeout: 0s\n", "database.pool.pingTimeout", "positive"}, + {"timeout", "database:\n readTimeout: -1s\n", "database.readTimeout", "nonnegative"}, + {"mysqlZone", "database:\n location: Secret/Not_A_Zone\n", "database.location", "IANA"}, + {"tlsPair", "database:\n tls:\n certFile: secret-file\n keyFile: ''\n", "database.tls.keyFile", "certFile"}, + {"level", "server:\n log:\n level: secret-level\n", "server.log.level", "debug"}, + {"format", "server:\n log:\n format: secret-format\n", "server.log.format", "json or text"}, + {"rotation", "server:\n log:\n rotation:\n maxBackups: -1\n", "server.log.rotation.maxBackups", "10000"}, + {"zone", "server:\n log:\n timeZone: Secret/Not_A_Zone\n", "server.log.timeZone", "IANA"}, + {"lifecycle", "server:\n lifecycle:\n stopTimeout: -1s\n", "server.lifecycle.stopTimeout", "nonnegative"}, + {"policy", "server:\n request:\n body: secret-mode\n", "server.request.body", "required"}, + {"clientPool", "client:\n maxIdleConns: -1\n", "client.maxIdleConns", "131072"}, + {"proxy", "client:\n proxy:\n mode: url\n url: 'https://private:secret@localhost/path'\n", "client.proxy.url", "origin"}, + } { + t.Run(tc.name, func(t *testing.T) { + path := filepath.Join(t.TempDir(), "config.yaml") + if err := os.WriteFile(path, []byte(tc.source), 0600); err != nil { + t.Fatal(err) + } + _, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()}) + var d *config.Error + if !errors.As(err, &d) || d.Field != tc.field || d.File != path || d.Line <= 1 || d.Column <= 0 || !strings.Contains(d.Reason, tc.reason) { + t.Fatalf("diagnostic: %v", err) + } + if strings.Contains(err.Error(), "secret") || strings.Contains(err.Error(), "Secret/") || strings.Contains(err.Error(), "private") { + t.Fatalf("value leaked: %v", err) + } + }) + } +} +func TestModuleDiagnosticsKeepReferenceAndCause(t *testing.T) { + t.Setenv("CHECK_PORT", "70000") + path := filepath.Join(t.TempDir(), "config.yaml") + os.WriteFile(path, []byte("database:\n port: '${CHECK_PORT}'\n"), 0600) + _, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()}) + var d *config.Error + if !errors.As(err, &d) || d.Field != "database.port" || d.Env != "CHECK_PORT" || d.Line != 2 { + t.Fatal(err) + } + original := mysql.DefaultConfig() + original.Port = 70000 + cause := original.Validate() + wrapped := config.AtField("database", cause) + if !errors.Is(wrapped, cause) { + t.Fatal("lost original validation error") + } + // Unknown project errors continue to be redacted rather than echoed. + wrapped = config.AtField("custom", errors.New("private-password")) + if strings.Contains(wrapped.Error(), "private-password") { + t.Fatal(wrapped) + } +} diff --git a/contract/policy.go b/contract/policy.go index a9ee8e0..aacac65 100644 --- a/contract/policy.go +++ b/contract/policy.go @@ -3,9 +3,11 @@ package contract import ( "context" "errors" - "fmt" "io" + "strings" "unicode/utf8" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) // RequestPolicy describes protocol strictness, independently of business @@ -35,7 +37,7 @@ func (p RequestPolicy) Validate() error { ok = ok || value == item.value } if !ok { - return fmt.Errorf("request policy: invalid %s", item.name) + return configdiag.New(item.name, "must be one of "+strings.Join(item.allowed, ", ")) } } return nil diff --git a/copyx/check.go b/copyx/check.go new file mode 100644 index 0000000..a9c5119 --- /dev/null +++ b/copyx/check.go @@ -0,0 +1,192 @@ +package copyx + +import ( + "errors" + "fmt" + "reflect" + "sort" +) + +// ErrDynamicType means a source interface requires a concrete runtime value. +// An exact converter can explicitly own that dynamic boundary. +var ErrDynamicType = errors.New("copyx: dynamic source type requires runtime validation or an exact converter") + +// Check validates a source/target type pair using the same options as Copy. +// It never executes converters and requires no example values. Nil policies, +// converter failures, map-key collisions and deep reference isolation still +// require runtime checks. Options remain local; no mapping files or registration. +func Check[S, T any](options ...Option) error { + c, err := New(options...) + if err != nil { + return err + } + return c.Check(reflect.TypeFor[S](), reflect.TypeFor[T]()) +} + +// Check validates types with an existing Copier's immutable settings. The target +// is the result type (not Copy's extra output pointer). Safe for concurrent use. +func (c *Copier) Check(source, target reflect.Type) error { + if c == nil { + return errors.New("copyx: nil Copier") + } + if source == nil || target == nil { + return errors.New("copyx: source and target types are required") + } + return newTypeChecker(&c.options, false).check(source, target, "$", 0) +} + +type typeChecker struct { + options *settings + allowDynamic bool + seen map[pair]bool + nodes int +} + +func newTypeChecker(options *settings, allowDynamic bool) *typeChecker { + return &typeChecker{options: options, allowDynamic: allowDynamic, seen: map[pair]bool{}} +} + +// Absent elements can reveal type/field mismatches, but contain no actual +// resource handles or dynamic interface values to validate. Runtime checks stay +// responsible for those value-dependent cases. +func (s *state) checkAbsent(from, to reflect.Type, path string) error { + if s.checkedTypes == nil { + s.checkedTypes = map[pair]bool{} + } + checker := typeChecker{options: s.options, allowDynamic: true, seen: s.checkedTypes, nodes: s.checkedNodes} + err := checker.check(from, to, path, 0) + s.checkedNodes = checker.nodes + return err +} +func (c *typeChecker) check(from, to reflect.Type, path string, depth int) error { + c.nodes++ + if depth > maxDepth || c.nodes > maxNodes { + return at(path, ErrLimit) + } + key := pair{from, to} + if c.seen[key] { + return nil + } + c.seen[key] = true + if !c.allowDynamic && !shallowSafe(to) { + return at(path, ErrUnsupported) + } + if _, ok := c.options.converters[key]; ok { + return nil + } + if from == to && len(c.options.converters) == 0 && len(c.options.fields) == 0 && !c.options.skipZero && c.options.nilPolicy != NilSkip { + if !c.options.deep || simpleValue(to) { + return nil + } + } + if from.Kind() == reflect.Interface { + if c.allowDynamic { + return nil + } + return at(path, ErrDynamicType) + } + if to.Kind() == reflect.Interface { + if !from.Implements(to) { + return at(path, ErrType) + } + return c.check(from, from, path, depth+1) + } + if !c.allowDynamic && c.options.deep && forbidden(to) { + return at(path, ErrUnsupported) + } + if from.Kind() == reflect.Pointer && to.Kind() != reflect.Pointer { + return c.check(from.Elem(), to, path, depth+1) + } + if to.Kind() == reflect.Pointer { + if !c.options.deep && from == to { + return nil + } + if from.Kind() == reflect.Pointer { + from = from.Elem() + } + return c.check(from, to.Elem(), path, depth+1) + } + if from.Kind() != to.Kind() { + return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to)) + } + if to == timeType && from == to { + return nil + } + switch to.Kind() { + case reflect.Struct: + for i := 0; i < to.NumField(); i++ { + f := to.Field(i) + if !c.allowDynamic && f.PkgPath != "" && (from != to || c.options.deep && privateMutable(f.Type)) { + return at(path+"."+f.Name, ErrUnsupported) + } + } + sources, targets := exportedFields(from), exportedFields(to) + rules := c.options.fields[key] + if rules == nil { + rules = &fields{} + } + reverse, used := map[string]string{}, map[string]bool{} + for a, b := range rules.names { + reverse[b] = a + } + for _, name := range sortedFieldNames(targets) { + if rules.ignoreTarget[name] { + continue + } + sourceName := name + if n, ok := reverse[name]; ok { + sourceName = n + } + source, ok := sources[sourceName] + if !ok || rules.ignoreSource[sourceName] { + return at(path+"."+name, ErrField) + } + used[sourceName] = true + if err := c.check(source.Type, targets[name].Type, path+"."+name, depth+1); err != nil { + return err + } + } + if c.options.strict { + for _, name := range sortedFieldNames(sources) { + if !used[name] && !rules.ignoreSource[name] { + return at(path+"."+name, ErrField) + } + } + } + return nil + case reflect.Slice: + if !c.options.deep && from == to { + return nil + } + return c.check(from.Elem(), to.Elem(), path+"[]", depth+1) + case reflect.Array: + if from.Len() != to.Len() { + return at(path, ErrType) + } + return c.check(from.Elem(), to.Elem(), path+"[]", depth+1) + case reflect.Map: + if !c.options.deep && from == to { + return nil + } + if err := c.check(from.Key(), to.Key(), path+"[key]", depth+1); err != nil { + return err + } + return c.check(from.Elem(), to.Elem(), path+"[value]", depth+1) + case reflect.Func, reflect.Chan, reflect.UnsafePointer: + if c.options.deep && !c.allowDynamic { + return at(path, ErrUnsupported) + } + } + if from != to { + return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to)) + } + return nil +} +func sortedFieldNames(fields map[string]reflect.StructField) []string { + names := make([]string, 0, len(fields)) + for name := range fields { + names = append(names, name) + } + sort.Strings(names) + return names +} diff --git a/copyx/check_test.go b/copyx/check_test.go new file mode 100644 index 0000000..0f824f9 --- /dev/null +++ b/copyx/check_test.go @@ -0,0 +1,192 @@ +package copyx_test + +import ( + "errors" + "reflect" + "strings" + "sync" + "testing" + + "git.yuntuo.hk/k2safe/go-omni/copyx" +) + +type checkSource struct{ Rows []checkSourceRow } +type checkSourceRow struct{ Value string } +type checkTarget struct{ Rows []checkTargetRow } +type checkTargetRow struct{ Value int } +type recursiveSource struct { + Children []*recursiveSource + Value string +} +type recursiveTarget struct { + Children []*recursiveTarget + Value int +} +type recursiveCompatible struct { + Children []*recursiveCompatible + Value string +} + +func TestCheckTypesAndEmptyCollections(t *testing.T) { + err := copyx.Check[checkSource, checkTarget]() + var field *copyx.Error + if !errors.Is(err, copyx.ErrType) || !errors.As(err, &field) || field.Path != "$.Rows[].Value" { + t.Fatal(err) + } + for _, rows := range [][]checkSourceRow{nil, {}, {{Value: "42"}}} { + out := checkTarget{Rows: []checkTargetRow{{Value: 99}}} + if err := copyx.Copy(&out, checkSource{rows}); !errors.Is(err, copyx.ErrType) { + t.Fatalf("rows %v: %v", rows, err) + } + if len(out.Rows) != 1 || out.Rows[0].Value != 99 { + t.Fatal("partial target committed") + } + } + if err := copyx.Check[map[string]checkSourceRow, map[string]checkTargetRow](); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + var dst map[string]checkTargetRow + if err := copyx.Copy(&dst, map[string]checkSourceRow{}); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + var array [0]checkTargetRow + if err := copyx.Copy(&array, [0]checkSourceRow{}); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + if err := copyx.Check[map[string]int, map[int]int](); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + if err := copyx.Check[[2]int, [3]int](); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + if err := copyx.Check[*recursiveSource, *recursiveTarget](); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } + if err := copyx.Check[*recursiveSource, *recursiveCompatible](copyx.WithDeepCopy()); err != nil { + t.Fatal(err) + } +} +func TestCheckRulesAndConverterBoundaries(t *testing.T) { + type from struct { + Name string + Extra bool + } + type to struct { + Label string + Keep int + } + options := []copyx.Option{copyx.WithStrictFields(), copyx.WithFieldNames[from, to](map[string]string{"Name": "Label"}), copyx.WithIgnoreSource[from, to]("Extra"), copyx.WithIgnoreTarget[from, to]("Keep")} + if err := copyx.Check[from, to](options...); err != nil { + t.Fatal(err) + } + if err := copyx.Check[from, to](); !errors.Is(err, copyx.ErrField) { + t.Fatal(err) + } + type short struct{ Name string } + if err := copyx.Check[from, short](copyx.WithStrictFields()); !errors.Is(err, copyx.ErrField) { + t.Fatal(err) + } + if err := copyx.Check[checkSource, checkTarget](copyx.WithConverter(func(string) (int, error) { panic("check must not execute converters") })); err != nil { + t.Fatal(err) + } + if err := copyx.Check[struct{ Value any }, struct{ Value int }](); !errors.Is(err, copyx.ErrDynamicType) { + t.Fatal(err) + } + if err := copyx.Check[any, int](copyx.WithConverter(func(any) (int, error) { panic("not called") })); err != nil { + t.Fatal(err) + } + if err := copyx.Check[*int, int](); err != nil { + t.Fatal(err) + } // Actual nil remains a runtime error. + var src *int + dst := 1 + if err := copyx.Copy(&dst, src); !errors.Is(err, copyx.ErrNil) || dst != 1 { + t.Fatal(err) + } + var empty []int + if err := copyx.Copy(&empty, []any{}); err != nil || empty == nil { + t.Fatal("dynamic emptiness changed", err) + } + if err := copyx.Check[sync.Mutex, sync.Mutex](); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatal(err) + } + option := copyx.WithConverter(func(v []int) ([]int, error) { return v, nil }) + if err := copyx.Check[[]int, []int](copyx.WithDeepCopy(), option); err != nil { + t.Fatal(err) + } + if _, err := copyx.Clone([]int{1}, option); !errors.Is(err, copyx.ErrSharedReference) { + t.Fatal("type check bypassed graph audit", err) + } +} +func TestCopierCheckConcurrentAndOptionsIsolation(t *testing.T) { + good, _ := copyx.New(copyx.WithConverter(func(string) (int, error) { return 42, nil })) + bad, _ := copyx.New() + var wg sync.WaitGroup + for i := 0; i < 24; i++ { + wg.Add(1) + go func() { + defer wg.Done() + for j := 0; j < 20; j++ { + if err := good.Check(reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()); err != nil { + t.Error(err) + } + if err := bad.Check(reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]()); !errors.Is(err, copyx.ErrType) { + t.Error(err) + } + var target checkTarget + if err := good.Copy(&target, checkSource{Rows: []checkSourceRow{}}); err != nil || target.Rows == nil { + t.Error(err) + } + } + }() + } + wg.Wait() + if err := good.Check(nil, reflect.TypeFor[int]()); err == nil { + t.Fatal("nil type accepted") + } + var absent *copyx.Copier + if err := absent.Check(reflect.TypeFor[int](), reflect.TypeFor[int]()); err == nil { + t.Fatal("nil copier accepted") + } + if err := copyx.Check[int, string](); err == nil || !strings.Contains(err.Error(), "int -> string") { + t.Fatal(err) + } +} +func BenchmarkTypeCheck(b *testing.B) { + c, _ := copyx.New(copyx.WithConverter(func(string) (int, error) { return 1, nil })) + from, to := reflect.TypeFor[checkSource](), reflect.TypeFor[checkTarget]() + b.ReportAllocs() + for i := 0; i < b.N; i++ { + if err := c.Check(from, to); err != nil { + b.Fatal(err) + } + } +} + +func TestEmptyTypeCheckPreservesAbsentValueSemantics(t *testing.T) { + var callback func() + if out, err := copyx.Clone(callback); err != nil || out != nil { + t.Fatal(err) + } + if out, err := copyx.Clone([]func(){}); err != nil || out == nil || len(out) != 0 { + t.Fatal(err) + } + if _, err := copyx.Clone([]func(){func() {}}); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatal(err) + } + if err := copyx.Check[[]func(), []func()](copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrUnsupported) { + t.Fatal("preflight must report potentially non-nil handles", err) + } + type from struct { + Ignored *int + Value string + } + type to struct { + Ignored *int + Value int + } + var dst []to + if err := copyx.Copy(&dst, []from{}, copyx.WithDeepCopy()); !errors.Is(err, copyx.ErrType) { + t.Fatal(err) + } +} diff --git a/copyx/copy.go b/copyx/copy.go index 97417d3..8b2cd16 100644 --- a/copyx/copy.go +++ b/copyx/copy.go @@ -89,12 +89,14 @@ type visit struct { target reflect.Type } type state struct { - options *settings - seen map[visit]reflect.Value - graph *graph - nodes int - allocated uint64 - valueRoot *reference + options *settings + seen map[visit]reflect.Value + graph *graph + nodes int + allocated uint64 + valueRoot *reference + checkedTypes map[pair]bool + checkedNodes int } func (s *state) reserve(count int, size uintptr, path string) error { @@ -239,6 +241,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth case reflect.Struct: return s.structure(src, base, t, path, depth) case reflect.Slice: + if src.Len() == 0 { + if err := s.checkAbsent(src.Type(), t, path); err != nil { + return reflect.Value{}, err + } + } if !s.options.deep && src.Type() == t { return src, nil } @@ -264,6 +271,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth } return out, nil case reflect.Array: + if src.Len() == 0 { + if err := s.checkAbsent(src.Type(), t, path); err != nil { + return reflect.Value{}, err + } + } if src.Len() != t.Len() { return reflect.Value{}, at(path, ErrType) } @@ -280,6 +292,11 @@ func (s *state) walk(src, base reflect.Value, t reflect.Type, path string, depth } return out, nil case reflect.Map: + if src.Len() == 0 { + if err := s.checkAbsent(src.Type(), t, path); err != nil { + return reflect.Value{}, err + } + } if !s.options.deep && src.Type() == t { return src, nil } diff --git a/database/connector.go b/database/connector.go index 89704be..3d06c94 100644 --- a/database/connector.go +++ b/database/connector.go @@ -7,6 +7,8 @@ import ( "errors" "reflect" "time" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) // PoolConfig contains only driver-independent settings. Connection credentials, @@ -25,15 +27,29 @@ func DefaultPoolConfig() PoolConfig { } func (c PoolConfig) Validate() error { - if c.MaxOpenConns < 0 || c.MaxIdleConns < 0 { - return errors.New("database pool limits cannot be negative") + for _, f := range []struct { + name string + value int + }{{"maxOpenConns", c.MaxOpenConns}, {"maxIdleConns", c.MaxIdleConns}} { + if f.value < 0 { + return configdiag.New(f.name, "must be nonnegative") + } } if c.MaxOpenConns > 0 && c.MaxIdleConns > c.MaxOpenConns { - return errors.New("max idle connections cannot exceed max open connections") + return configdiag.New("maxIdleConns", "must not exceed maxOpenConns") } - if c.ConnMaxLifetime < 0 || c.ConnMaxIdleTime < 0 || c.PingTimeout <= 0 { - return errors.New("database timeouts are invalid") + for _, f := range []struct { + name string + value time.Duration + }{{"connMaxLifetime", c.ConnMaxLifetime}, {"connMaxIdleTime", c.ConnMaxIdleTime}} { + if f.value < 0 { + return configdiag.New(f.name, "must be nonnegative") + } } + if c.PingTimeout <= 0 { + return configdiag.New("pingTimeout", "must be positive") + } + return nil } diff --git a/database/database.go b/database/database.go index cb7d770..6fde91d 100644 --- a/database/database.go +++ b/database/database.go @@ -10,6 +10,8 @@ import ( "fmt" "strings" "time" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) type Config struct { @@ -43,10 +45,10 @@ func (c Config) PoolConfig() PoolConfig { func (c Config) Validate() error { if strings.TrimSpace(c.Driver) == "" { - return errors.New("database driver is required") + return configdiag.New("driver", "is required") } if strings.TrimSpace(c.DSN) == "" { - return errors.New("database DSN is required") + return configdiag.New("dsn", "is required") } return c.PoolConfig().Validate() } diff --git a/database/mysql/README.md b/database/mysql/README.md index 4e104ed..0899925 100644 --- a/database/mysql/README.md +++ b/database/mysql/README.md @@ -70,3 +70,7 @@ func (c *ServiceContext) DatabaseOptions() []mysqladapter.Option { `WithDriverConfig(func(*mysql.Config) error)` 支持高级拨号、驱动参数等，配置在回调后克隆；这是项目明确选择的底层扩展。`NewConnector` 返回标准 driver.Connector，供已有基础设施使用；它不接受启动检查选项。任意驱动仍可用核心 `database.OpenConnector`。 错误有安全摘要，errors.Is/As 保留 TLS/驱动/项目检查的原因。配置验证不读证书、不连接数据库；Open 才读取证书、创建池并握手。不要自行打印原始配置或展开可能包含内部信息的 driver 字段给用户。 + +## 字段级配置错误 + +Validate 返回独立字段与安全原因，例如 port（1–65535）、readTimeout、pool.maxIdleConns、tls.keyFile。项目调用 `config.AtField("database", err)` 后，统一加载器补齐 database 前缀、文件、环境名和行列，无需重复写端口/连接池校验。TLS 成对配置会指出缺失的那个字段。连接失败、驱动和证书文件的内部原因仍通过 errors.Is/As 保留，不把凭据放进公开提示。见 [配置诊断](../../CONFIG.md#框架模块的字段诊断)。 diff --git a/database/mysql/mysql.go b/database/mysql/mysql.go index 8cbcbeb..9d9b257 100644 --- a/database/mysql/mysql.go +++ b/database/mysql/mysql.go @@ -19,6 +19,7 @@ import ( "time" "git.yuntuo.hk/k2safe/go-omni/database" + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" "git.yuntuo.hk/k2safe/go-omni/logx" mysqldriver "github.com/go-sql-driver/mysql" ) @@ -61,45 +62,60 @@ var identifier = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`) // Validate does not read certificate files or open connections. All defaults // come from DefaultConfig; explicit zero I/O timeouts mean no driver deadline. func (c Config) Validate() error { - if c.Socket == "" && (c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") || c.Port < 1 || c.Port > 65535) { - return errors.New("mysql: invalid host or port") + if c.Socket == "" { + if c.Host == "" || strings.ContainsAny(c.Host, "/\r\n\t ") { + return configdiag.New("host", "must be a nonempty hostname or IP address") + } + if c.Port < 1 || c.Port > 65535 { + return configdiag.New("port", "must be between 1 and 65535") + } } - if strings.ContainsRune(c.Socket, 0) || strings.ContainsRune(c.Database, 0) || strings.ContainsRune(c.User, 0) { - return errors.New("mysql: invalid connection field") + for _, f := range []struct{ name, value string }{{"socket", c.Socket}, {"database", c.Database}, {"user", c.User}} { + if strings.ContainsRune(f.value, 0) { + return configdiag.New(f.name, "must not contain a NUL character") + } } - if c.ConnectTimeout < 0 || c.ReadTimeout < 0 || c.WriteTimeout < 0 { - return errors.New("mysql: timeouts cannot be negative") + for _, f := range []struct { + name string + value time.Duration + }{{"connectTimeout", c.ConnectTimeout}, {"readTimeout", c.ReadTimeout}, {"writeTimeout", c.WriteTimeout}} { + if f.value < 0 { + return configdiag.New(f.name, "must be nonnegative") + } } if err := c.Pool.Validate(); err != nil { - return err + return configdiag.At("pool", err) } if c.Location != "" { if _, err := time.LoadLocation(c.Location); err != nil { - return errors.New("mysql: invalid time location") + return configdiag.Wrap("location", "must be an available IANA time zone or UTC", err) } } if c.Charset != "" && !identifier.MatchString(c.Charset) { - return errors.New("mysql: invalid charset") + return configdiag.New("charset", "must be a valid MySQL identifier") } if c.Collation != "" && !identifier.MatchString(c.Collation) { - return errors.New("mysql: invalid collation") + return configdiag.New("collation", "must be a valid MySQL identifier") } for key := range c.Session { if !identifier.MatchString(key) { - return errors.New("mysql: invalid session variable name") + return configdiag.New("session", "keys must be valid MySQL identifiers") } } if c.TLS.Mode != "" && c.TLS.Mode != "required" && c.TLS.Mode != "disabled" { - return errors.New("mysql: TLS mode must be required or disabled") + return configdiag.New("tls.mode", "must be required or disabled") } if (c.TLS.CertFile == "") != (c.TLS.KeyFile == "") { - return errors.New("mysql: TLS client certificate and key must be configured together") + if c.TLS.CertFile == "" { + return configdiag.New("tls.certFile", "is required when keyFile is configured") + } + return configdiag.New("tls.keyFile", "is required when certFile is configured") } if c.TLS.Mode == "disabled" && (c.TLS.CAFile != "" || c.TLS.CertFile != "" || c.TLS.ServerName != "") { - return errors.New("mysql: TLS fields cannot be used when disabled") + return configdiag.New("tls.mode", "must be required when TLS fields are configured") } if c.Socket != "" && c.TLS.Mode != "disabled" && c.TLS.ServerName == "" { - return errors.New("mysql: TLS over a unix socket requires serverName") + return configdiag.New("tls.serverName", "is required for TLS over a unix socket") } return nil } diff --git a/examples/developerTools/README.md b/examples/developerTools/README.md new file mode 100644 index 0000000..7bec865 --- /dev/null +++ b/examples/developerTools/README.md @@ -0,0 +1,18 @@ +# 配置诊断、严格 HTTP 状态、类型预检和日志时区 + +在框架根目录运行，不需要数据库、账号或外部服务。HTTP 示例只访问临时 localhost 服务： + +```sh +go run ./examples/developerTools +DEMO_LOG_TIME_ZONE=UTC go run ./examples/developerTools +# 以下两条应非零退出，输出具体字段和环境变量名，不输出原值： +DEMO_DB_PORT=70000 go run ./examples/developerTools +DEMO_LOG_LEVEL=invalid go run ./examples/developerTools +``` + +- 配置只做校验，没有连接数据库。失败指向 `database.port` / `server.log.level`，并保留文件行列与变量名。 +- Logger 使用选定时区；只改变记录的时间戳，不修改全局 time.Local 和业务时间字段。 +- `copyx.Check[Input, Result]` 发现切片元素内的不兼容字段；加上显式 `strconv.Atoi` converter 后通过。预检不执行 converter，真实非法数字仍在 Copy 时返回错误。 +- JSON 调用显式 `WithSuccessStatuses(200)`，拒绝 201，框架仍负责关闭响应体。 + +详见 [配置](../../CONFIG.md)、[日志](../../logx/README.md)、[HTTP 客户端](../../httpc/README.md)、[copyx](../../COPY.md)。 diff --git a/examples/developerTools/config.yaml b/examples/developerTools/config.yaml new file mode 100644 index 0000000..07dd18f --- /dev/null +++ b/examples/developerTools/config.yaml @@ -0,0 +1,7 @@ +# Validates database settings without connecting in this demonstration. +database: + port: "${DEMO_DB_PORT:-3306}" +server: + log: + level: "${DEMO_LOG_LEVEL:-info}" + timeZone: "${DEMO_LOG_TIME_ZONE:-Asia/Shanghai}" diff --git a/examples/developerTools/main.go b/examples/developerTools/main.go new file mode 100644 index 0000000..f082353 --- /dev/null +++ b/examples/developerTools/main.go @@ -0,0 +1,104 @@ +// Run from the framework root with: go run ./examples/developerTools +package main + +import ( + "context" + "errors" + "flag" + "fmt" + "io" + "net/http" + "net/http/httptest" + "os" + "strconv" + _ "time/tzdata" // This example embeds zones; production builds can use -tags timetzdata. + + "git.yuntuo.hk/k2safe/go-omni/config" + "git.yuntuo.hk/k2safe/go-omni/copyx" + mysql "git.yuntuo.hk/k2safe/go-omni/database/mysql" + "git.yuntuo.hk/k2safe/go-omni/httpc" + "git.yuntuo.hk/k2safe/go-omni/logx" +) + +type Config struct { + Database mysql.Config `json:"database"` + Server config.Config `json:"server"` +} + +func Default() Config { + c := config.Default() + c.Log.Output = "stdout" + return Config{Database: mysql.DefaultConfig(), Server: c} +} +func (c Config) Validate() error { + if err := c.Database.Validate(); err != nil { + return config.AtField("database", err) + } + return config.AtField("server", c.Server.Validate()) +} + +type Input struct{ Values []TextValue } +type TextValue struct{ Number string } +type Result struct{ Values []NumberValue } +type NumberValue struct{ Number int } + +func run(path string) error { + cfg, err := config.LoadProject(path, Default()) + if err != nil { + return err + } + // Validation only: this example never opens a database. + logger, err := logx.New(cfg.Server.Log) + if err != nil { + return err + } + defer logger.Close() + logger.InfoContext(context.Background(), "instance time zone", "configuredZone", cfg.Server.Log.TimeZone) + if err := copyx.Check[Input, Result](); !errors.Is(err, copyx.ErrType) { + return fmt.Errorf("expected a type mismatch, got %v", err) + } + // Explicit Go converter owns the number conversion; no JSON mapping file. + options := []copyx.Option{copyx.WithConverter(strconv.Atoi)} + if err := copyx.Check[Input, Result](options...); err != nil { + return err + } + var output Result + if err := copyx.Copy(&output, Input{Values: []TextValue{{Number: "42"}}}, options...); err != nil { + return err + } + fmt.Printf("type check and copy succeeded: %d\n", output.Values[0].Number) + + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path == "/created" { + w.WriteHeader(http.StatusCreated) + } + io.WriteString(w, `{"value":"example"}`) + })) + defer server.Close() + client, err := httpc.New(httpc.Config{DisableLogging: true}) + if err != nil { + return err + } + defer client.Close() + var reply struct { + Value string `json:"value"` + } + if err := client.JSON(context.Background(), http.MethodGet, server.URL, nil, &reply, httpc.WithSuccessStatuses(http.StatusOK)); err != nil { + return err + } + err = client.JSON(context.Background(), http.MethodGet, server.URL+"/created", nil, &reply, httpc.WithSuccessStatuses(http.StatusOK)) + var status *httpc.StatusError + if !errors.As(err, &status) || status.StatusCode != http.StatusCreated { + return fmt.Errorf("expected status rejection, got %v", err) + } + fmt.Println("strict JSON: 200 accepted, 201 rejected") + return nil +} +func main() { + path := flag.String("config", "examples/developerTools/config.yaml", "selected configuration file") + flag.Parse() + if err := run(*path); err != nil { + fmt.Fprintln(os.Stderr, err) + os.Exit(1) + } +} diff --git a/examples/managedService/go.mod b/examples/managedService/go.mod index 269ea61..12abeb4 100644 --- a/examples/managedService/go.mod +++ b/examples/managedService/go.mod @@ -2,7 +2,7 @@ module example.com/managed go 1.22.0 -require git.yuntuo.hk/k2safe/go-omni v0.22.0-preview +require git.yuntuo.hk/k2safe/go-omni v0.23.0-preview require ( github.com/gabriel-vasile/mimetype v1.4.3 // indirect diff --git a/examples/userService/go.mod b/examples/userService/go.mod index 40e9a71..3d813d0 100644 --- a/examples/userService/go.mod +++ b/examples/userService/go.mod @@ -3,7 +3,7 @@ module example.com/userService go 1.22.0 require ( - git.yuntuo.hk/k2safe/go-omni v0.22.0-preview + git.yuntuo.hk/k2safe/go-omni v0.23.0-preview github.com/go-sql-driver/mysql v1.9.3 ) diff --git a/httpc/README.md b/httpc/README.md index a6df538..36524aa 100644 --- a/httpc/README.md +++ b/httpc/README.md @@ -85,7 +85,20 @@ err := client.JSON(ctx, "POST", endpoint, input, &output, - Host、Content-Length、Transfer-Encoding、Trailer 属于请求目标/分帧控制，请用标准 Request 配合 Do；JSON 选项中不允许伪装成普通头。 - `WithMaxResponseBytes` 是本次 JSON 的正数限额，不修改 Client 默认值，不改变 Do 的流式语义。 -JSON 自动关闭响应体。仅 2xx 解码；空成功响应（例如 204）保留 output 原值。非空响应使用标准 json.Unmarshal，拒绝多个 JSON 文档，类型错误保留 Go 的错误信息与行为。 +`WithSuccessStatuses(200)` 限制本次调用只接受 200；可传多个，例如 `(200, 201)`。不传时保持全部 2xx。列表必须非空且每项位于 200–299，重复项允许；后一个状态选项替换前一个列表。构造选项时快照，之后修改原 slice 不影响请求，并发复用不串配置。 + +```go +err := client.JSON(ctx, http.MethodGet, endpoint, nil, &result, + httpc.WithHeader("Authorization", credential), + httpc.WithSuccessStatuses(http.StatusOK), +) +var status *httpc.StatusError +if errors.As(err, &status) { /* status.StatusCode 是实际收到的状态 */ } +``` + +未允许的状态（包括 201/204）返回 StatusError，不解码、不改变 output；仍关闭响应体、按原有时间/大小预算回收连接，不读取无限错误正文。状态选项不会增加重试，原有 Do 重试规则保留。它只限制 HTTP 状态，不保证响应非空或业务字段完整，业务仍校验自身协议。严格状态、关闭与连接复用见 [示例](../examples/developerTools/README.md)。 + +JSON 自动关闭响应体。默认仅 2xx 解码；指定成功状态列表时只解码列表内状态；空成功响应（例如 204）保留 output 原值。非空响应使用标准 json.Unmarshal，拒绝多个 JSON 文档，类型错误保留 Go 的错误信息与行为。 `output == nil` 按限额流式丢弃成功响应，不把整个 body 读进内存，也不检查 JSON 语法。响应限额按实际读取的 body 计算；标准库自动 gzip 解压后仍受上限约束。input 会先 json.Marshal 到内存，大文件上传不适合用 JSON helper。 diff --git a/httpc/config.go b/httpc/config.go index 383f3c5..9f18267 100644 --- a/httpc/config.go +++ b/httpc/config.go @@ -2,7 +2,6 @@ package httpc import ( "errors" - "fmt" "net" "net/http" "net/url" @@ -10,6 +9,8 @@ import ( "strconv" "strings" "time" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) // ProxyMode controls the owned transport's proxy selection. Empty preserves @@ -66,17 +67,23 @@ func normalizeConfig(cfg Config) (Config, error) { {"responseHeaderTimeout", cfg.ResponseHeaderTimeout}, {"expectContinueTimeout", cfg.ExpectContinueTimeout}, {"idleConnTimeout", cfg.IdleConnTimeout}, } { if field.value < 0 { - return Config{}, fmt.Errorf("httpc: %s must not be negative", field.name) + return Config{}, configdiag.New(field.name, "must be nonnegative") } } if cfg.MaxRetries < 0 || cfg.MaxRetries > 5 { - return Config{}, errors.New("httpc: maxRetries must be 0..5") + return Config{}, configdiag.New("maxRetries", "must be between 0 and 5") } - if cfg.MaxConnsPerHost < 0 || cfg.MaxConnsPerHost > 65536 || cfg.MaxIdleConnsPerHost < 0 || cfg.MaxIdleConnsPerHost > 65536 || cfg.MaxIdleConns < 0 || cfg.MaxIdleConns > 131072 { - return Config{}, errors.New("httpc: invalid connection pool limits") + for _, f := range []struct { + name string + value, max int + }{{"maxConnsPerHost", cfg.MaxConnsPerHost, 65536}, {"maxIdleConnsPerHost", cfg.MaxIdleConnsPerHost, 65536}, {"maxIdleConns", cfg.MaxIdleConns, 131072}} { + if f.value < 0 || f.value > f.max { + return Config{}, configdiag.New(f.name, "must be between 0 and "+strconv.Itoa(f.max)) + } } + if cfg.MaxResponseBytes < 0 || cfg.MaxResponseBytes > maxResponseBytes { - return Config{}, errors.New("httpc: maxResponseBytes must be 0..1 GiB") + return Config{}, configdiag.New("maxResponseBytes", "must be between 0 and 1 GiB") } if _, err := proxyFunction(cfg.Proxy); err != nil { return Config{}, err @@ -86,11 +93,11 @@ func normalizeConfig(cfg Config) (Config, error) { switch value.Kind() { case reflect.Pointer, reflect.Map, reflect.Func, reflect.Slice, reflect.Interface, reflect.Chan: if value.IsNil() { - return Config{}, errors.New("httpc: nil Transport") + return Config{}, configdiag.New("transport", "must not be a typed nil") } } if cfg.hasTransportOptions() { - return Config{}, errors.New("httpc: Transport cannot be combined with proxy, dial, TLS or connection pool options; configure the borrowed transport directly") + return Config{}, configdiag.New("transport", "cannot be combined with proxy, dial, TLS or connection pool options; configure the borrowed transport directly") } } if cfg.Timeout == 0 { @@ -103,7 +110,7 @@ func normalizeConfig(cfg Config) (Config, error) { cfg.MaxBackoff = 2 * time.Second } if cfg.Backoff > cfg.MaxBackoff { - return Config{}, errors.New("httpc: backoff exceeds maxBackoff") + return Config{}, configdiag.New("backoff", "must not exceed maxBackoff") } if cfg.MaxConnsPerHost == 0 { cfg.MaxConnsPerHost = 64 @@ -124,7 +131,7 @@ func (c Config) hasTransportOptions() bool { } func proxyFunction(c ProxyConfig) (func(*http.Request) (*url.URL, error), error) { if c.Mode != ProxyURL && c.URL != "" { - return nil, errors.New("httpc: proxy URL requires mode url") + return nil, configdiag.New("proxy.mode", "must be url when a URL is configured") } switch c.Mode { case "": @@ -136,27 +143,27 @@ func proxyFunction(c ProxyConfig) (func(*http.Request) (*url.URL, error), error) case ProxyURL: u, err := url.Parse(c.URL) if err != nil || u == nil || u.Hostname() == "" || u.Opaque != "" || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || (u.Path != "" && u.Path != "/") { - return nil, errors.New("httpc: proxy URL must be an absolute proxy origin without path, query or fragment") + return nil, configdiag.New("proxy.url", "must be an absolute proxy origin without path, query or fragment") } switch u.Scheme { case "http", "https", "socks5", "socks5h": default: - return nil, errors.New("httpc: proxy scheme must be http, https, socks5 or socks5h") + return nil, configdiag.New("proxy.url", "scheme must be http, https, socks5 or socks5h") } if strings.HasSuffix(u.Host, ":") { - return nil, errors.New("httpc: proxy port must be 1..65535") + return nil, configdiag.New("proxy.url", "port must be between 1 and 65535") } if port := u.Port(); port != "" { n, e := strconv.Atoi(port) if e != nil || n < 1 || n > 65535 { - return nil, errors.New("httpc: proxy port must be 1..65535") + return nil, configdiag.New("proxy.url", "port must be between 1 and 65535") } } u.Path = "" u.RawPath = "" return http.ProxyURL(u), nil default: - return nil, errors.New("httpc: proxy mode must be environment, direct or url") + return nil, configdiag.New("proxy.mode", "must be environment, direct or url") } } func ownedTransport(cfg Config) (*http.Transport, error) { diff --git a/httpc/json.go b/httpc/json.go index 7b9175a..1c00fac 100644 --- a/httpc/json.go +++ b/httpc/json.go @@ -11,7 +11,7 @@ import ( ) // JSON bounds response memory, closes the body, and decodes only 2xx responses. -// Optional headers/limits belong to this call and never alter the client. +// Optional headers/limits/success statuses belong to this call and never alter the client. // output=nil discards a bounded stream; it does not buffer or validate JSON. // Empty successful bodies leave output unchanged. StatusError contains no body. func (c *Client) JSON(ctx context.Context, method, url string, input, output any, options ...RequestOption) error { @@ -57,7 +57,7 @@ func (c *Client) JSON(ctx context.Context, method, url string, input, output any return err } defer response.Body.Close() - if response.StatusCode < 200 || response.StatusCode >= 300 { + if !settings.accepts(response.StatusCode) { drainStatusBody(response, settings.maxResponseBytes) return &StatusError{StatusCode: response.StatusCode} } diff --git a/httpc/request.go b/httpc/request.go index 7c04ada..186a916 100644 --- a/httpc/request.go +++ b/httpc/request.go @@ -9,6 +9,8 @@ import ( type requestOptions struct { headers http.Header maxResponseBytes int64 + successStatuses [2]uint64 + restrictStatus bool } // RequestOption applies only to one JSON call. Returned options can be reused @@ -97,3 +99,41 @@ func checkHeader(name string, values []string) error { } return nil } + +// WithSuccessStatuses restricts this JSON call to the listed 2xx statuses. +// Omission keeps all 2xx successful. Empty/out-of-range lists are rejected before +// sending a request. Options snapshot their input and can be reused concurrently. +// The policy does not add retries or validate application-specific JSON fields. +func WithSuccessStatuses(statuses ...int) RequestOption { + var accepted [2]uint64 + var err error + if len(statuses) == 0 { + err = errors.New("httpc: at least one success status is required") + } + for _, status := range statuses { + if status < 200 || status >= 300 { + err = errors.New("httpc: success statuses must be between 200 and 299") + break + } + bit := status - 200 + accepted[bit/64] |= uint64(1) << uint(bit%64) + } + return func(o *requestOptions) error { + if err != nil { + return err + } + o.successStatuses = accepted + o.restrictStatus = true + return nil + } +} +func (o requestOptions) accepts(status int) bool { + if status < 200 || status >= 300 { + return false + } + if !o.restrictStatus { + return true + } + bit := status - 200 + return o.successStatuses[bit/64]&(uint64(1)<= 200 && status < 300) { + t.Fatal(err) + } + if status >= 200 && status < 300 { + if err := c2.JSON(context.Background(), "GET", "http://example.invalid", nil, nil, WithSuccessStatuses(status, status)); err != nil { + t.Fatal(err) + } + } + }) + } + var calls atomic.Int32 + c := testClient(t, Config{Transport: roundTripperFunc(func(r *http.Request) (*http.Response, error) { + calls.Add(1) + return benchmarkTransport{`{}`, 200}.RoundTrip(r) + })}) + for _, option := range []RequestOption{WithSuccessStatuses(), WithSuccessStatuses(199), WithSuccessStatuses(300), WithSuccessStatuses(-1), WithSuccessStatuses(200, 500)} { + if err := c.JSON(context.Background(), "GET", "http://example.invalid", nil, nil, option); err == nil { + t.Fatal("invalid policy accepted") + } + } + if calls.Load() != 0 { + t.Fatal("invalid policy reached transport") + } +} +func TestJSONSuccessStatusConcurrentOptions(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + n, _ := strconv.Atoi(r.URL.Query().Get("status")) + w.WriteHeader(n) + if n != 204 { + io.WriteString(w, `{}`) + } + })) + defer srv.Close() + c := testClient(t, Config{}) + statuses := []int{200, 201} + option := WithSuccessStatuses(statuses...) + statuses[0] = 204 + var wg sync.WaitGroup + for i := 0; i < 30; i++ { + wg.Add(1) + go func(i int) { + defer wg.Done() + status := 200 + i%2 + err := c.JSON(context.Background(), "GET", fmt.Sprintf("%s?status=%d", srv.URL, status), nil, nil, option) + if err != nil { + t.Error(err) + } + err = c.JSON(context.Background(), "GET", srv.URL+"?status=204", nil, nil, option) + var rejected *StatusError + if !errors.As(err, &rejected) || rejected.StatusCode != 204 { + t.Error(err) + } + }(i) + } + wg.Wait() + // Last explicit policy wins; another call's policy must not leak. + if err := c.JSON(context.Background(), "GET", srv.URL+"?status=204", nil, nil, option, WithSuccessStatuses(204)); err != nil { + t.Fatal(err) + } + if err := c.JSON(context.Background(), "GET", srv.URL+"?status=204", nil, nil); err != nil { + t.Fatal(err) + } +} +func TestJSONRejectedSuccessBodyRecovery(t *testing.T) { + var connections atomic.Int32 + srv := httptest.NewUnstartedServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(201) + if r.URL.Path == "/stream" { + w.(http.Flusher).Flush() + <-r.Context().Done() + return + } + io.WriteString(w, "private response") + })) + srv.Config.ConnState = func(_ net.Conn, s http.ConnState) { + if s == http.StateNew { + connections.Add(1) + } + } + srv.Start() + defer srv.Close() + c := testClient(t, Config{Timeout: time.Second}) + for i := 0; i < 5; i++ { + err := c.JSON(context.Background(), "GET", srv.URL, nil, nil, WithSuccessStatuses(200)) + var status *StatusError + if !errors.As(err, &status) || status.StatusCode != 201 { + t.Fatal(err) + } + } + if connections.Load() != 1 { + t.Fatal("small rejected bodies lost keep-alive", connections.Load()) + } + start := time.Now() + err := c.JSON(context.Background(), "GET", srv.URL+"/stream", nil, nil, WithSuccessStatuses(200)) + var status *StatusError + if !errors.As(err, &status) || status.StatusCode != 201 || time.Since(start) > 500*time.Millisecond { + t.Fatal("waited for rejected unbounded body", err) + } + if err := c.JSON(context.Background(), "GET", srv.URL, nil, nil, WithSuccessStatuses(201), WithMaxResponseBytes(4)); !errors.Is(err, ErrResponseTooLarge) { + t.Fatal("accepted body bypassed size limit", err) + } +} +func BenchmarkJSONSuccessStatus(b *testing.B) { + for _, restrict := range []bool{false, true} { + b.Run(strconv.FormatBool(restrict), func(b *testing.B) { + c, _ := New(Config{Transport: benchmarkTransport{`{"id":1}`, 200}, DisableLogging: true}) + defer c.Close() + var options []RequestOption + if restrict { + options = []RequestOption{WithSuccessStatuses(200)} + } + var out struct { + ID int `json:"id"` + } + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + if err := c.JSON(context.Background(), "GET", "http://example.invalid", nil, &out, options...); err != nil { + b.Fatal(err) + } + } + }) + } +} diff --git a/internal/configdiag/error.go b/internal/configdiag/error.go new file mode 100644 index 0000000..7041bff --- /dev/null +++ b/internal/configdiag/error.go @@ -0,0 +1,34 @@ +// Package configdiag carries value-free field diagnostics between framework +// modules without making them depend on the application's config package. +package configdiag + +import "errors" + +type Error struct { + Field, Reason string + cause error +} + +func (e *Error) Error() string { return e.Field + ": " + e.Reason } +func (e *Error) Unwrap() error { return e.cause } +func New(field, reason string) error { return &Error{Field: field, Reason: reason} } +func Wrap(field, reason string, cause error) error { + return &Error{Field: field, Reason: reason, cause: cause} +} +func At(prefix string, err error) error { + if err == nil { + return nil + } + var d *Error + if errors.As(err, &d) { + field := prefix + if d.Field != "" { + if field != "" { + field += "." + } + field += d.Field + } + return Wrap(field, d.Reason, err) + } + return Wrap(prefix, "validation failed", err) +} diff --git a/lifecycle/phases.go b/lifecycle/phases.go index 6b3fe7a..d2c5cd3 100644 --- a/lifecycle/phases.go +++ b/lifecycle/phases.go @@ -7,6 +7,8 @@ import ( "strings" "time" "unicode" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) // ErrResourcesRetained means background quiescence could not be established. @@ -24,8 +26,13 @@ func DefaultShutdownConfig() ShutdownConfig { return ShutdownConfig{StopTimeout: 5 * time.Second, DrainTimeout: 10 * time.Second, CancelTimeout: 5 * time.Second} } func (c ShutdownConfig) Validate() error { - if c.StopTimeout < 0 || c.DrainTimeout < 0 || c.CancelTimeout < 0 { - return errors.New("lifecycle: shutdown timeouts must be nonnegative") + for _, f := range []struct { + name string + value time.Duration + }{{"stopTimeout", c.StopTimeout}, {"drainTimeout", c.DrainTimeout}, {"cancelTimeout", c.CancelTimeout}} { + if f.value < 0 { + return configdiag.New(f.name, "must be nonnegative") + } } return nil } diff --git a/logx/README.md b/logx/README.md index d890f7e..373bbfb 100644 --- a/logx/README.md +++ b/logx/README.md @@ -14,6 +14,20 @@ logger.InfoContext(ctx, "invoice.saved", "invoiceId", invoiceID) `Debug/Info/Warn/Error` 有包级和实例方法，也有带 Context 的方法。`requestId` 自动跟随 HTTP Context；接入 telemetry 后可带 `traceId`。不要在 Service 中改用 `context.Background()`，否则会丢失取消信号和关联字段。 +## 实例时区 + +```yaml +server: + log: + timeZone: Asia/Shanghai +``` + +手写项目使用 `logx.New(logx.Config{TimeZone: "Asia/Shanghai"})`。配置名称采用 IANA 名称或 UTC；空字符串保持已有时间戳行为。无效名称在打开文件之前报 `timeZone` 的安全错误；统一加载后路径为 `server.log.timeZone`。 + +时区在创建 Logger 时解析，记录输出不重复加载；JSON/text、With/WithGroup 子日志和 Standard 适配器均继承该实例时区。不同 Logger 可同时使用不同地区，不修改全局 time.Local。仅转换记录时间戳，业务 `time.Time` 属性保留原时区；夏令时由时区数据处理。时区变更需要重建 Logger，现有热加载仍只支持级别。 + +使用 Go 标准库查找时区。极简镜像需提供时区数据，或执行 `go build -tags timetzdata .` 将标准库时区数据内置（增加约 450 KB）；也可由手写 main 导入 time/tzdata。logx 自身不强制内置数据，不引入第三方依赖。脱敏、日志级别和文件所有权语义保持不变。可运行例子：[developerTools](../examples/developerTools/README.md)。 + ## 文件轮转 ```go @@ -53,7 +67,7 @@ application := omni.New( ) ``` -`WithSlowRequestThreshold` 同时开启访问日志，0 禁用慢请求识别。也可以在 JSON/YAML 中配 `enableLogger: true`、`slowRequestThreshold: "500ms"`，或使用 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 配合 `GO_OMNI_ENABLE_LOGGER=true`。 +`WithSlowRequestThreshold` 同时开启访问日志，0 禁用慢请求识别。也可以在配置文件的 server 下配 `enableLogger: true`、`slowRequestThreshold: "500ms"`。需要环境变量控制时，显式写 `slowRequestThreshold: "${SLOW_REQUEST_THRESHOLD:-500ms}"`；没有文件引用时不会读取同名环境变量。 每个请求记录一条 `http.request`：方法、路径、匹配路由、实际状态码、耗时、客户端 IP 和关联字段。正常请求 INFO，超过阈值 WARN（`slow: true`），5xx ERROR；返回的 error 会记录到结构化字段。计时覆盖 Handler 调用，不包含框架之外的代理排队，也不自动计入 Handler 返回后继续写入的自管流。 diff --git a/logx/adapters.go b/logx/adapters.go index 5e0195f..a9d36ac 100644 --- a/logx/adapters.go +++ b/logx/adapters.go @@ -2,27 +2,33 @@ package logx import ( "context" - "errors" "log" "log/slog" "strings" + "time" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) // Validate checks configuration without opening files or creating resources. -func (c Config) Validate() error { +func (c Config) Validate() error { _, err := c.validate(); return err } +func (c Config) validate() (*time.Location, error) { c = normalizeConfig(c) if _, err := parseLevel(c.Level); err != nil { - return errors.New("logx: invalid level") + return nil, configdiag.New("level", "must be debug, info, warn, warning or error") } if c.Format != "json" && c.Format != "text" { - return errors.New("logx: format must be json or text") + return nil, configdiag.New("format", "must be json or text") } switch c.Output { case "stdout", "stderr", "discard", "none": default: - return errors.New("logx: invalid output") + return nil, configdiag.New("output", "must be stdout, stderr, discard or none") } - return c.Rotation.validate(c) + if err := c.Rotation.validate(c); err != nil { + return nil, err + } + return logLocation(c.TimeZone) } type loggerContextKey struct{} diff --git a/logx/logx.go b/logx/logx.go index 16d5b29..36e6c35 100644 --- a/logx/logx.go +++ b/logx/logx.go @@ -22,6 +22,7 @@ import ( // standard stream. File takes precedence over Output and appends to the named // file. type Config struct { + TimeZone string `json:"timeZone"` // Empty retains existing timestamp behavior. Level string `json:"level"` Format string `json:"format"` Output string `json:"output"` @@ -73,7 +74,8 @@ func init() { // call Close when the process is shutting down. func New(cfg Config) (*Logger, error) { cfg = normalizeConfig(cfg) - if err := cfg.Validate(); err != nil { + location, err := cfg.validate() + if err != nil { return nil, err } level, err := parseLevel(cfg.Level) @@ -95,6 +97,9 @@ func New(cfg Config) (*Logger, error) { } else { handler = slog.NewJSONHandler(writer, opts) } + if location != nil { + handler = zoneHandler{Handler: handler, location: location} + } logger := slog.New(handler) if cfg.Service != "" { logger = logger.With("service", cfg.Service) diff --git a/logx/rotation.go b/logx/rotation.go index b5f17e4..67ade46 100644 --- a/logx/rotation.go +++ b/logx/rotation.go @@ -1,13 +1,15 @@ package logx import ( - "errors" + "fmt" "io" "os" "path/filepath" "sync" "gopkg.in/natefinch/lumberjack.v2" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" ) // RotationConfig opts a file logger into size-based rotation. Zero limits use @@ -21,12 +23,21 @@ type RotationConfig struct { } func (r RotationConfig) validate(cfg Config) error { - if r.MaxSizeMB < 0 || r.MaxSizeMB > 1024*1024 || r.MaxBackups < 0 || r.MaxBackups > 10000 || r.MaxAgeDays < 0 || r.MaxAgeDays > 36500 { - return errors.New("logx: invalid rotation limits") + for _, f := range []struct { + name string + value, max int + }{{"maxSizeMb", r.MaxSizeMB, 1024 * 1024}, {"maxBackups", r.MaxBackups, 10000}, {"maxAgeDays", r.MaxAgeDays, 36500}} { + if f.value < 0 || f.value > f.max { + return configdiag.New("rotation."+f.name, fmt.Sprintf("must be between 0 and %d", f.max)) + } } - if r.Enabled && (cfg.File == "" || cfg.Writer != nil) { - return errors.New("logx: rotation requires a file and no custom writer") + if r.Enabled && cfg.File == "" { + return configdiag.New("file", "is required when rotation is enabled") } + if r.Enabled && cfg.Writer != nil { + return configdiag.New("rotation.enabled", "cannot be enabled with a custom writer") + } + return nil } diff --git a/logx/timeZone.go b/logx/timeZone.go new file mode 100644 index 0000000..efb92f4 --- /dev/null +++ b/logx/timeZone.go @@ -0,0 +1,40 @@ +package logx + +import ( + "context" + "log/slog" + "time" + + "git.yuntuo.hk/k2safe/go-omni/internal/configdiag" +) + +func logLocation(name string) (*time.Location, error) { + if name == "" { + return nil, nil + } + location, err := time.LoadLocation(name) + if err != nil { + return nil, configdiag.Wrap("timeZone", "must be an available IANA time zone or UTC", err) + } + return location, nil +} + +// Convert only the record timestamp. User time.Time attributes retain their +// original locations; child loggers inherit this instance's location. +type zoneHandler struct { + slog.Handler + location *time.Location +} + +func (h zoneHandler) Handle(ctx context.Context, r slog.Record) error { + if !r.Time.IsZero() { + r.Time = r.Time.In(h.location) + } + return h.Handler.Handle(ctx, r) +} +func (h zoneHandler) WithAttrs(attrs []slog.Attr) slog.Handler { + return zoneHandler{h.Handler.WithAttrs(attrs), h.location} +} +func (h zoneHandler) WithGroup(name string) slog.Handler { + return zoneHandler{h.Handler.WithGroup(name), h.location} +} diff --git a/logx/timeZone_test.go b/logx/timeZone_test.go new file mode 100644 index 0000000..e234574 --- /dev/null +++ b/logx/timeZone_test.go @@ -0,0 +1,133 @@ +package logx + +import ( + "bytes" + "context" + "encoding/json" + "io" + "log/slog" + "os" + "path/filepath" + "strings" + "sync" + "testing" + "time" +) + +func TestLoggerTimeZoneIsolation(t *testing.T) { + originalLocal := time.Local + stamp := time.Date(2026, 7, 1, 0, 0, 0, 123456000, time.UTC) + for _, format := range []string{"json", "text"} { + t.Run(format, func(t *testing.T) { + var a, b bytes.Buffer + sh, err := New(Config{TimeZone: "Asia/Shanghai", Format: format, Writer: &a}) + if err != nil { + t.Fatal(err) + } + defer sh.Close() + utc, err := New(Config{TimeZone: "UTC", Format: format, Writer: &b}) + if err != nil { + t.Fatal(err) + } + defer utc.Close() + r := slog.NewRecord(stamp, slog.LevelInfo, "timestamp", 0) + r.AddAttrs(slog.Time("eventTime", stamp), slog.String("password", "secret")) + if err := sh.With("service", "first").WithGroup("data").logger.Handler().Handle(context.Background(), r); err != nil { + t.Fatal(err) + } + if err := utc.logger.Handler().Handle(context.Background(), r); err != nil { + t.Fatal(err) + } + if !strings.Contains(a.String(), "08:00:00") || !strings.Contains(a.String(), "+08:00") || !strings.Contains(b.String(), "00:00:00") || strings.Contains(b.String(), "+08:00") { + t.Fatalf("a=%s b=%s", a.String(), b.String()) + } + if strings.Contains(a.String(), "secret") || strings.Contains(b.String(), "secret") { + t.Fatal("redaction lost") + } + if format == "json" { + var out map[string]any + if err := json.Unmarshal(a.Bytes(), &out); err != nil { + t.Fatal(err) + } + if out["time"] != "2026-07-01T08:00:00.123456+08:00" || out["data"].(map[string]any)["eventTime"] != "2026-07-01T00:00:00.123456Z" { + t.Fatal(out) + } + } + }) + } + if time.Local != originalLocal { + t.Fatal("global time.Local changed") + } +} +func TestLoggerTimeZoneValidationAndDST(t *testing.T) { + path := filepath.Join(t.TempDir(), "not-created", "app.log") + if _, err := New(Config{TimeZone: "Secret/Not_A_Zone", File: path}); err == nil || strings.Contains(err.Error(), "Secret") { + t.Fatal(err) + } + if _, err := os.Stat(filepath.Dir(path)); !os.IsNotExist(err) { + t.Fatal("invalid zone opened writer") + } + var out bytes.Buffer + l, err := New(Config{TimeZone: "America/New_York", Writer: &out}) + if err != nil { + t.Fatal(err) + } + defer l.Close() + for _, month := range []time.Month{time.January, time.July} { + r := slog.NewRecord(time.Date(2026, month, 1, 12, 0, 0, 0, time.UTC), slog.LevelInfo, "dst", 0) + if err := l.logger.Handler().Handle(context.Background(), r); err != nil { + t.Fatal(err) + } + } + if !strings.Contains(out.String(), "-05:00") || !strings.Contains(out.String(), "-04:00") { + t.Fatal(out.String()) + } + // Explicit empty location preserves an incoming record's original zone. + out.Reset() + plain, _ := New(Config{Writer: &out}) + defer plain.Close() + r := slog.NewRecord(time.Date(2026, 1, 1, 0, 0, 0, 0, time.FixedZone("original", 7200)), slog.LevelInfo, "unchanged", 0) + plain.logger.Handler().Handle(context.Background(), r) + if !strings.Contains(out.String(), "+02:00") { + t.Fatal(out.String()) + } +} +func TestLoggerTimeZoneConcurrentChildren(t *testing.T) { + var out bytes.Buffer + l, err := New(Config{TimeZone: "Asia/Shanghai", Writer: &out}) + if err != nil { + t.Fatal(err) + } + defer l.Close() + var wg sync.WaitGroup + for i := 0; i < 20; i++ { + wg.Add(1) + go func(i int) { + defer wg.Done() + child := l.With("worker", i).WithGroup("event") + for n := 0; n < 10; n++ { + child.Standard(slog.LevelInfo).Print("message") + } + }(i) + } + wg.Wait() + if strings.Count(out.String(), "+08:00") != 200 { + t.Fatal("timestamps missing") + } +} +func BenchmarkLoggerTimeZone(b *testing.B) { + for _, zone := range []string{"", "UTC", "Asia/Shanghai"} { + b.Run(zone, func(b *testing.B) { + l, err := New(Config{TimeZone: zone, Writer: io.Discard}) + if err != nil { + b.Fatal(err) + } + defer l.Close() + b.ReportAllocs() + b.ResetTimer() + for i := 0; i < b.N; i++ { + l.InfoContext(context.Background(), "benchmark", "count", 1) + } + }) + } +} diff --git a/reports/developerToolsAcceptance.md b/reports/developerToolsAcceptance.md new file mode 100644 index 0000000..0c8caf8 --- /dev/null +++ b/reports/developerToolsAcceptance.md @@ -0,0 +1,60 @@ +# v0.23 接入工具验收（2026-09-26） + +本轮补齐通用配置诊断、JSON 成功状态限制、copyx 类型预检和实例日志时区；没有引入业务类型、映射配置或全局时区修改。 + +## 回归 + +- `GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...`：48 个有测试的包通过，包含实际生成消费者、失败清理、后台收尾、错误目录和上传下载。 +- 最后调整 copyx 空集合检查和分配后，copyx/config/logx/httpc 的 race 复测通过。 +- Go 1.22.12 的 config/copyx/logx/httpc/database 测试通过；最后调整后前四个包复测通过。 +- `go vet ./...`、`go build ./...` 通过。 +- managedService 生成检查、race 和根目录构建通过；userService 的 race 通过。外部数据库测试没有提供连接配置时按原规则跳过。 +- developerTools 示例的默认配置、UTC、非法数据库端口、非法日志等级验证通过。错误非零退出，具体字段/原因可见，非法输入值未输出。 +- `FuzzClone` 最终运行 30 秒、4 worker，执行 3,925,833 次，通过（总时间 30.391 秒）。另有确定性的类型、nil、资源句柄和深拷贝隔离测试；fuzz 执行次数不代表形式化证明。 + +| 能力 | 验收场景 | +| --- | --- | +| 字段诊断 | database.port/pool/tls、server.log.level/rotation/timeZone、HTTP 连接配置、后台关闭超时、请求策略定位具体字段；文件/环境名/行列保留；errors.Is/As 保留原因链；未知业务校验仍安全脱敏 | +| JSON 成功状态 | 默认全部 2xx；指定 200；200..299 范围与空列表校验；选项快照/并发隔离；拒绝状态不改输出、不额外重试；响应体关闭；小响应复用连接；无限慢响应有界回收；成功响应大小限制保留 | +| copyx 预检 | 无样本检查结构、数组/map、指针、递归类型、重命名/忽略/严格字段覆盖；converter 不执行；并发 Copier 选项隔离；空集合也拒绝确定类型错误；失败不修改目标 | +| copyx 值语义 | nil 与空集合保持区别；允许原有合法的 nil 函数/空函数集合；实际非 nil 句柄仍拒绝；动态 interface 提示需要实际值或显式 converter；深拷贝 converter 返回原引用仍报错 | +| 日志时区 | JSON/text/子日志/标准库日志；上海和 UTC 实例隔离；纽约夏令时；默认保留原时间；业务 time.Time 属性不改；脱敏保留；非法时区不创建文件；200 条并发日志 race 通过 | +| 生成消费者 | 生成 Load 直接定位 server.log.level 和环境变量，校验先于依赖初始化；timeZone 直接绑定；重复生成/check 和业务文件保留通过 | + +## 性能 + +Go 1.27.1、darwin/arm64、Apple M2 Max。无 race，每场景三轮、benchtime=200ms，表中取中位数。[基线原始结果](developerToolsBaseline.txt)、[当前原始结果](developerToolsBenchmark.txt)。 + +| 场景 | 修改前 | 修改后 | 修改后分配 | +| --- | ---: | ---: | ---: | +| 结构转换，每次创建 Copier | 1221 ns | 1261 ns | 1120 B / 17 allocs（不变） | +| 结构转换，复用 Copier | 733.4 ns | 719.1 ns | 168 B / 7 allocs（不变） | +| 64 KiB 字节深拷贝 | 6067 ns | 5613 ns | 65768 B / 9 allocs（不变） | +| HTTP 128 B JSON 解码 | 1374 ns | 1369 ns | 约 2348 B / 22 allocs | +| HTTP 4 KiB JSON 解码 | 7020 ns | 7130 ns | 约 17085 B / 31 allocs | +| HTTP 64 KiB JSON 解码 | 80622 ns | 80571 ns | 约 213924 B / 38 allocs | +| HTTP Do | 664.2 ns | 664.5 ns | 1296 B / 17 allocs（不变） | + +新增功能的独立基准： + +- 复用 Copier，预检含嵌套切片和显式 converter 的类型：597.8 ns、32 B、3 次分配。每次显式 Check 检查类型，不在每个正常业务 Copy 前重复完整预检。 +- 同一简单 JSON 调用默认策略 1224 ns / 2345 B / 22 次分配，传成功状态选项 1249 ns / 2393 B / 23 次分配；此用例每次创建选项，应用也可复用选项。 +- 同一日志输出到 io.Discard：默认 851.5 ns、UTC 885.0 ns、上海 888.7 ns；均为 112 B / 3 次分配。时区解析只发生在初始化/校验阶段。 + +这些是本机微基准，HTTP 使用模拟 Transport，日志使用 io.Discard，不代表实际网络或磁盘吞吐量；三轮测量有波动（64 KiB JSON 有一次较慢样本），不承诺固定性能比例。中途发现的普通拷贝额外分配已在最终测量前消除。 + +复现命令： + +```sh +go test ./copyx ./httpc ./logx -run '^$' -bench 'BenchmarkScenarios/(struct|bytes64KiB)|BenchmarkHTTPClient|BenchmarkTypeCheck|BenchmarkJSONSuccessStatus|BenchmarkLoggerTimeZone' -benchmem -benchtime=200ms -count=3 +go test ./copyx -run '^$' -fuzz '^FuzzClone$' -fuzztime=30s -parallel=4 +go run ./examples/developerTools +``` + +## 使用边界 + +- 错误文本变得更具体，识别诊断应使用错误类型/字段。自定义业务校验继续通过 config.Invalid/AtField 声明安全原因。 +- 空集合中的确定类型错误现在会提前返回；需要调整实际类型或显式 converter。Check 不验证具体值、converter 成功性、nil 输入或深拷贝引用隔离，这些仍由实际 Copy 执行。 +- 成功状态选项只接受 2xx 子集，不改变 HTTP 重试策略；业务响应内容是否完整仍由项目协议校验。 +- 时区数据由运行环境提供；极简镜像可用 `go build -tags timetzdata .` 内置。logx 不强制引入约 450 KB 的时区数据库。 +- 本轮未修改接入项目业务代码，未连接业务数据库或使用远程服务器压测。仍需项目升级后的业务验收。 diff --git a/reports/developerToolsBaseline.txt b/reports/developerToolsBaseline.txt new file mode 100644 index 0000000..54969c9 --- /dev/null +++ b/reports/developerToolsBaseline.txt @@ -0,0 +1,42 @@ +goos: darwin +goarch: arm64 +pkg: git.yuntuo.hk/k2safe/go-omni/copyx +cpu: Apple M2 Max +BenchmarkScenarios/struct/reusefalse-12 195284 1218 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusefalse-12 194980 1227 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusefalse-12 186867 1221 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusetrue-12 340254 780.4 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/struct/reusetrue-12 339192 733.4 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/struct/reusetrue-12 330416 730.4 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/bytes64KiB-12 37689 6035 ns/op 10858.43 MB/s 65768 B/op 9 allocs/op +BenchmarkScenarios/bytes64KiB-12 41599 6222 ns/op 10533.71 MB/s 65768 B/op 9 allocs/op +BenchmarkScenarios/bytes64KiB-12 38743 6067 ns/op 10801.83 MB/s 65768 B/op 9 allocs/op +PASS +ok git.yuntuo.hk/k2safe/go-omni/copyx 2.867s +goos: darwin +goarch: arm64 +pkg: git.yuntuo.hk/k2safe/go-omni/httpc +cpu: Apple M2 Max +BenchmarkHTTPClient/128B/decode-12 152140 1398 ns/op 2348 B/op 22 allocs/op +BenchmarkHTTPClient/128B/decode-12 181575 1372 ns/op 2348 B/op 22 allocs/op +BenchmarkHTTPClient/128B/decode-12 177343 1374 ns/op 2348 B/op 22 allocs/op +BenchmarkHTTPClient/128B/discard-12 235099 997.7 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/128B/discard-12 234962 1012 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/128B/discard-12 231510 990.5 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/4KiB/decode-12 34695 7010 ns/op 17086 B/op 31 allocs/op +BenchmarkHTTPClient/4KiB/decode-12 34256 7020 ns/op 17084 B/op 31 allocs/op +BenchmarkHTTPClient/4KiB/decode-12 34084 7043 ns/op 17087 B/op 31 allocs/op +BenchmarkHTTPClient/4KiB/discard-12 223120 1058 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/4KiB/discard-12 205618 1067 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/4KiB/discard-12 224156 1079 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/64KiB/decode-12 3010 80622 ns/op 213909 B/op 38 allocs/op +BenchmarkHTTPClient/64KiB/decode-12 3122 81358 ns/op 213949 B/op 38 allocs/op +BenchmarkHTTPClient/64KiB/decode-12 2982 80473 ns/op 213901 B/op 38 allocs/op +BenchmarkHTTPClient/64KiB/discard-12 110815 2154 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/64KiB/discard-12 112252 2107 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/64KiB/discard-12 114548 2094 ns/op 1835 B/op 21 allocs/op +BenchmarkHTTPClient/Do-12 366976 662.1 ns/op 1296 B/op 17 allocs/op +BenchmarkHTTPClient/Do-12 351373 664.2 ns/op 1296 B/op 17 allocs/op +BenchmarkHTTPClient/Do-12 335104 679.0 ns/op 1296 B/op 17 allocs/op +PASS +ok git.yuntuo.hk/k2safe/go-omni/httpc 5.889s diff --git a/reports/developerToolsBenchmark.txt b/reports/developerToolsBenchmark.txt new file mode 100644 index 0000000..9804359 --- /dev/null +++ b/reports/developerToolsBenchmark.txt @@ -0,0 +1,66 @@ +goos: darwin +goarch: arm64 +pkg: git.yuntuo.hk/k2safe/go-omni/copyx +cpu: Apple M2 Max +BenchmarkScenarios/struct/reusefalse-12 185918 1261 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusefalse-12 191277 1254 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusefalse-12 192738 1267 ns/op 1120 B/op 17 allocs/op +BenchmarkScenarios/struct/reusetrue-12 336856 716.9 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/struct/reusetrue-12 333480 719.1 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/struct/reusetrue-12 338338 753.9 ns/op 168 B/op 7 allocs/op +BenchmarkScenarios/bytes64KiB-12 43203 5588 ns/op 11727.22 MB/s 65768 B/op 9 allocs/op +BenchmarkScenarios/bytes64KiB-12 43075 5613 ns/op 11675.18 MB/s 65768 B/op 9 allocs/op +BenchmarkScenarios/bytes64KiB-12 42343 5646 ns/op 11608.06 MB/s 65768 B/op 9 allocs/op +BenchmarkTypeCheck-12 377830 599.8 ns/op 32 B/op 3 allocs/op +BenchmarkTypeCheck-12 408428 597.8 ns/op 32 B/op 3 allocs/op +BenchmarkTypeCheck-12 409514 597.7 ns/op 32 B/op 3 allocs/op +PASS +ok git.yuntuo.hk/k2safe/go-omni/copyx 3.326s +goos: darwin +goarch: arm64 +pkg: git.yuntuo.hk/k2safe/go-omni/httpc +cpu: Apple M2 Max +BenchmarkHTTPClient/128B/decode-12 170685 1369 ns/op 2347 B/op 22 allocs/op +BenchmarkHTTPClient/128B/decode-12 174030 1384 ns/op 2348 B/op 22 allocs/op +BenchmarkHTTPClient/128B/decode-12 179733 1368 ns/op 2348 B/op 22 allocs/op +BenchmarkHTTPClient/128B/discard-12 242395 1009 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/128B/discard-12 242695 1010 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/128B/discard-12 230649 1008 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/4KiB/decode-12 34294 7130 ns/op 17085 B/op 31 allocs/op +BenchmarkHTTPClient/4KiB/decode-12 34273 6992 ns/op 17085 B/op 31 allocs/op +BenchmarkHTTPClient/4KiB/decode-12 33758 7184 ns/op 17086 B/op 31 allocs/op +BenchmarkHTTPClient/4KiB/discard-12 215698 1059 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/4KiB/discard-12 226962 1108 ns/op 1837 B/op 21 allocs/op +BenchmarkHTTPClient/4KiB/discard-12 227298 1045 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/64KiB/decode-12 3099 79497 ns/op 213929 B/op 38 allocs/op +BenchmarkHTTPClient/64KiB/decode-12 2742 80571 ns/op 213924 B/op 38 allocs/op +BenchmarkHTTPClient/64KiB/decode-12 2696 121378 ns/op 213907 B/op 38 allocs/op +BenchmarkHTTPClient/64KiB/discard-12 111092 2144 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/64KiB/discard-12 103948 2132 ns/op 1836 B/op 21 allocs/op +BenchmarkHTTPClient/64KiB/discard-12 115408 2111 ns/op 1835 B/op 21 allocs/op +BenchmarkHTTPClient/Do-12 374404 656.7 ns/op 1296 B/op 17 allocs/op +BenchmarkHTTPClient/Do-12 353349 664.5 ns/op 1296 B/op 17 allocs/op +BenchmarkHTTPClient/Do-12 348849 665.8 ns/op 1296 B/op 17 allocs/op +BenchmarkJSONSuccessStatus/false-12 203222 1233 ns/op 2345 B/op 22 allocs/op +BenchmarkJSONSuccessStatus/false-12 201913 1213 ns/op 2345 B/op 22 allocs/op +BenchmarkJSONSuccessStatus/false-12 201854 1224 ns/op 2345 B/op 22 allocs/op +BenchmarkJSONSuccessStatus/true-12 199285 1249 ns/op 2393 B/op 23 allocs/op +BenchmarkJSONSuccessStatus/true-12 184888 1254 ns/op 2393 B/op 23 allocs/op +BenchmarkJSONSuccessStatus/true-12 194844 1222 ns/op 2393 B/op 23 allocs/op +PASS +ok git.yuntuo.hk/k2safe/go-omni/httpc 7.606s +goos: darwin +goarch: arm64 +pkg: git.yuntuo.hk/k2safe/go-omni/logx +cpu: Apple M2 Max +BenchmarkLoggerTimeZone/#00-12 256226 850.2 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/#00-12 277039 852.6 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/#00-12 285422 851.5 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/UTC-12 270704 885.0 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/UTC-12 277340 879.1 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/UTC-12 271767 890.3 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/Asia/Shanghai-12 273482 883.3 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/Asia/Shanghai-12 274610 888.7 ns/op 112 B/op 3 allocs/op +BenchmarkLoggerTimeZone/Asia/Shanghai-12 270154 892.8 ns/op 112 B/op 3 allocs/op +PASS +ok git.yuntuo.hk/k2safe/go-omni/logx 2.665s diff --git a/scripts/verifyManagedRelease.py b/scripts/verifyManagedRelease.py index d597605..eb4c18d 100644 --- a/scripts/verifyManagedRelease.py +++ b/scripts/verifyManagedRelease.py @@ -140,7 +140,8 @@ def main(): http_events = [json.loads(line) for line in http_output.splitlines() if line.startswith("{")] http_passed = {e.get("Test") for e in http_events if e.get("Action") == "pass"} assert {"TestProxyRouting", "TestJSONHeadersConcurrentSnapshot", "TestPoolLimitsAndIdleExpiry", - "TestErrorBodyPoolReuseAndDrainBound", "TestSlowKnownErrorBodyHasShortDrainBudget"} <= http_passed + "TestErrorBodyPoolReuseAndDrainBound", "TestSlowKnownErrorBodyHasShortDrainBudget", + "TestJSONSuccessStatuses", "TestJSONSuccessStatusConcurrentOptions", "TestJSONRejectedSuccessBodyRecovery"} <= http_passed example_output = run(["go", "run", MODULE + "/examples/httpClient"], starter, env) assert "authenticated request succeeded" in example_output checks += ["remoteHTTPClientRaceTests", "remoteHTTPProxyAndTimeouts", "remoteHTTPHeaderIsolation", @@ -150,12 +151,24 @@ def main(): config_events = [json.loads(line) for line in config_output.splitlines() if line.startswith("{")] config_passed = {e.get("Test") for e in config_events if e.get("Action") == "pass"} assert {"TestExplicitReferencesAcrossFormats", "TestReferenceFallbackAndFixedValues", - "TestReferenceDiagnosticsAndScalarLimits", "TestRemovedEnvTagsAndFieldSuggestion"} <= config_passed + "TestReferenceDiagnosticsAndScalarLimits", "TestRemovedEnvTagsAndFieldSuggestion", + "TestModuleFieldDiagnostics", "TestModuleDiagnosticsKeepReferenceAndCause"} <= config_passed run([cli, "api", "generate", "--check"], starter, env) run(["go", "run", ".", "-config", "etc/config.yaml", "-check-config"], starter, env) checks += ["remoteExplicitConfigurationRaceTests", "literalAndRequiredDefaultReferences", "remoteCheckConfigBeforeDependencies", "removedImplicitEnvironmentRules"] + for package, expected in { + "copyx": {"TestCheckTypesAndEmptyCollections", "TestCheckRulesAndConverterBoundaries", "TestCopierCheckConcurrentAndOptionsIsolation"}, + "logx": {"TestLoggerTimeZoneIsolation", "TestLoggerTimeZoneValidationAndDST", "TestLoggerTimeZoneConcurrentChildren"}, + }.items(): + output = run(["go", "test", "-race", "-count=1", "-json", MODULE + "/" + package], starter, env) + events = [json.loads(line) for line in output.splitlines() if line.startswith("{")] + passed = {event.get("Test") for event in events if event.get("Action") == "pass"} + assert expected <= passed + checks += ["remotePreciseModuleDiagnostics", "remoteJSONSuccessStatuses", "remoteRejectedStatusBodyCleanup", + "remoteCopyTypePreflight", "remoteEmptyCollectionTypeRejection", "remoteLoggerTimeZoneIsolation"] + checks += ["flatFieldComposition", "copyxThreeEndpointConsumer", "explicitNullAndMoneyPolicies", "fieldAdditionWithoutCopyRuleChanges", "middlewareFactoryInjectionAndCleanup", "optionalRepositoryPreservesBusiness"] checks += ["generatedUnifiedConfig", "jsonYamlXmlAndCustomDecoder", "configValidationBeforeDependencies", "configEnvProvenance", "configExtensionPreserved", "configLoaderDriftCheck"] @@ -172,6 +185,9 @@ def main(): sources += ["cmd/go-omni/apigen/configLoader_test.go", "cmd/go-omni/apigen/testdata/configuration/consumer_test.go", "cmd/go-omni/scaffold/templates/api-load.go.tmpl", "cmd/go-omni/scaffold/templates/api-loader.go.tmpl"] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "httpc").glob("*.go")] sources += [str(p.relative_to(ROOT)) for p in (ROOT / "examples/httpClient").glob("*.go")] + sources += [str(p.relative_to(ROOT)) for p in (ROOT / "logx").glob("*.go")] + sources += ["internal/configdiag/error.go", "database/mysql/mysql.go", "database/connector.go", "database/database.go", "contract/policy.go"] + sources += [str(p.relative_to(ROOT)) for p in (ROOT / "examples/developerTools").iterdir() if p.is_file()] report = {"passed": True, "frameworkVersion": args.version, "frameworkSum": dependency["Sum"], "freshModuleCache": True, "noLocalReplace": True, "durationSeconds": round(time.monotonic() - started, 2), "checks": checks,