Compare commits
18 Commits
v0.17.0-pr
...
v0.26.0-pr
| Author | SHA1 | Date | |
|---|---|---|---|
| d1a747c36f | |||
| 6a95cad184 | |||
| 7a177744ab | |||
| 8eddef9cc8 | |||
| 800974cfff | |||
| 9fade3bedb | |||
| 3bbc6f60d7 | |||
| 69585dab61 | |||
| d598b2ebee | |||
| 4bf6e88cfe | |||
| 96e7d9629b | |||
| 068e85132f | |||
| c60841f901 | |||
| 91a6beb2f3 | |||
| fd83235ce7 | |||
| ad9257d202 | |||
| 4563a6354f | |||
| 5fc82509a1 |
@@ -11,7 +11,7 @@
|
||||
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应；不自动推断任意 Handler 行为，不附带 Swagger UI |
|
||||
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供；不自动获取 token 指定的密钥 |
|
||||
| `telemetry` | HTTP 入站/出站、Redis、SQL spans，HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发；SQL Query span 不覆盖后续遍历；不接管全局 provider |
|
||||
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别、轮转/脱敏/堆栈/慢请求 | 热更新只开放日志级别；连接池、监听、认证等静态变更要求重启 |
|
||||
| `config` / `logx` | JSON/YAML/XML 文件、显式环境引用、动态日志级别、轮转/脱敏/堆栈/慢请求 | 热更新只开放日志级别；连接池、监听、认证等静态变更要求重启 |
|
||||
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖；默认内存状态，不做跨进程全局熔断 |
|
||||
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace；TTL 窗口内保护不等于永久业务去重 |
|
||||
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付，处理器必须幂等；持久性取决于 Redis 部署 |
|
||||
@@ -102,7 +102,7 @@ err = consumer.Run(serviceContext, func(ctx context.Context, item queue.Delivery
|
||||
|
||||
## 配置与日志热更新
|
||||
|
||||
`config.LoadFile(path, prefix)` 按默认值→文件→非空环境变量加载并验证。两种文件格式都使用 JSON tag 的 lowerCamelCase，duration 可以写 `"5s"`；文件上限 1 MiB，拒绝未知字段、多文档和非法值。自定义 prefix 要包含期望的尾部下划线，例如 `BILLING_`。空环境变量沿用文件值，不清空已配置字段。
|
||||
`config.LoadFile(path)` 使用默认值→文件字段显式引用→类型绑定→校验；没有自动环境覆盖。用 `${NAME}` 或 `${NAME:-fallback}` 指定来源，完整规则与迁移见 [CONFIG.md](CONFIG.md)。
|
||||
|
||||
```yaml
|
||||
address: '127.0.0.1:8080'
|
||||
@@ -113,7 +113,7 @@ log:
|
||||
format: json
|
||||
```
|
||||
|
||||
先按配置创建 `logx.New(cfg.Log)`，再 `config.NewReloader(path, prefix, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`；后者阻塞，生命周期由应用负责。只有 `log.level` 允许变化，校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
|
||||
先按配置创建 `logx.New(cfg.Log)`，再 `config.NewReloader(path, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`；后者阻塞，生命周期由应用负责。只有 `log.level` 允许变化，校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
|
||||
|
||||
## 熔断与观测
|
||||
|
||||
|
||||
10
AGENTS.md
Normal file
10
AGENTS.md
Normal file
@@ -0,0 +1,10 @@
|
||||
# 开发约定
|
||||
|
||||
- 框架保持通用，不内置接入项目的业务类型、字段、错误码或业务规则。
|
||||
- 2026-09-25 用户要求：每次功能开发完成，必须同步维护根 README.md。README 需要覆盖当前全部公开功能，解释用途、基本用法，并提供可运行示例或链接到 examples 内的完整示例。
|
||||
- 新增、调整和删除功能都必须同步文档、示例、测试与发布说明。删除功能要明确说明已移除内容、影响范围和迁移步骤，不能继续把历史用法当作当前能力推荐。
|
||||
- examples 使用通用场景，避免依赖业务项目、私有数据库或线上服务。必要的外部基础设施配置应明确说明。
|
||||
- 2026-09-25 用户决定：移除 mappings/mappingFiles 及结构映射代码生成机制，改用可选 copyx 工具包。不得恢复另一套 JSON 或 Go 映射声明语言。
|
||||
- 深拷贝的自定义转换结果不得与输入共享可变引用；发现共享应报错，不能降级成浅拷贝或静默接受。
|
||||
|
||||
- 2026-09-26 用户决定：配置文件显式声明固定值或 `${NAME}` / `${NAME:-fallback}`，旧自动环境覆盖、前缀/别名及 env/envDefault 标签模式移除，不得重新引入隐式二次覆盖。项目维护 Config/Default/Validate/文件，生成器维护 Load/main。
|
||||
4
API.md
4
API.md
@@ -48,7 +48,7 @@ go run ./cmd/go-omni new --api examples/contracts/user.api \
|
||||
--log-output file --log-format json ../userService
|
||||
```
|
||||
|
||||
`api generate` 在新目录中也会生成完整项目。默认只监听 localhost；`GO_OMNI_HTTP_ADDR` 修改地址，`GO_OMNI_TRANSPORT=native` 切换网络层。配置顺序为默认值 → `etc/config.yaml` → 环境变量。文件日志支持现有轮转配置。远程试用使用与 CLI 一致的固定版本，见 [发布说明](RELEASE.md)。
|
||||
`api generate` 在新目录中也会生成完整项目。默认只监听 localhost；`GO_OMNI_HTTP_ADDR` 修改地址，`GO_OMNI_TRANSPORT=native` 切换网络层。配置顺序为 Default() → `etc/config.yaml` 中的显式环境引用 → 类型绑定 → Validate()；固定值不再被隐式覆盖。文件日志支持现有轮转配置。远程试用使用与 CLI 一致的固定版本，见 [发布说明](RELEASE.md)。
|
||||
|
||||
API 项目现在支持组合数据库、存储和日志：
|
||||
|
||||
@@ -63,7 +63,7 @@ go-omni new --api user.api --module example.com/userService \
|
||||
|
||||
初次选项记录在 `.goOmni/apiManifest.json`，以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错，因为初始化代码属于业务方；可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。new 与 api new 统一生成契约项目，不再使用旧式 CRUD 目录。
|
||||
|
||||
MySQL 使用官方 `database/mysql` 结构化适配器，密码从 `GO_OMNI_DB_PASSWORD` 覆盖，非秘密池配置位于 YAML，环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`；API 项目不会额外生成公开的签名发放接口，鉴权和对象权限由业务实现。初始化失败会清理已打开资源，退出清理可重复调用。
|
||||
MySQL 使用官方 `database/mysql` 结构化适配器，生成文件以 `password: "${GO_OMNI_DB_PASSWORD}"` 显式读取密码；其他字段可写固定值或引用，缺失项保留 Default()。存储配置也只读取文件明确引用的变量；API 项目不会额外生成公开的签名发放接口，鉴权和对象权限由业务实现。初始化失败会清理已打开资源，退出清理可重复调用。
|
||||
|
||||
HTTP 请求类型不能推导数据库表结构；迁移和 Repository 由业务自己维护。当前目录的完整示例见 [managedService](examples/managedService/README.md)。MySQL、事务审计和鉴权实现可参考 [旧目录用户管理示例](examples/userService/README.md)，接入时按 [项目指南](PROJECT.md) 装配到 config/svc/logic。
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@
|
||||
## 实施与验收清单
|
||||
|
||||
- [x] 运行期：资源登记、逆序释放、幂等关闭、失败回滚、故障原因保留归框架；生成入口与项目依赖初始化边界明确。多 App 日志隔离、标准 log.Logger 适配、可信请求元数据与依赖中间件示例。
|
||||
- [x] 配置：项目完整结构体环境覆盖；默认值→文件→环境；嵌套、duration、slice、显式零值、空值策略、必填与脱敏；日志无副作用完整校验。
|
||||
- [x] 配置：Default()→文件显式环境引用→类型绑定→Validate；嵌套、duration、slice、显式零值、必填/备用值、转义与安全诊断；日志无副作用完整校验。
|
||||
- [x] 数据库：官方 MySQL 结构化配置/TLS/驱动日志适配，继续复用 OpenConnector 和唯一连接池；独立启动检查、失败清理、生成器接入；不内置版本/表结构/业务错误策略。
|
||||
- [x] 响应契约：数组、可空对象、成功 null、独立 HTTP 204；Logic/Handler/OpenAPI/漂移检查一致。
|
||||
- [x] 请求契约：未知 query、JSON UTF-8/charset、无 Body/必填空对象/可选 Body，App 默认与路由覆盖；scalar/any/数组/大小限制一致，普通 JSON 只绑定一次。
|
||||
|
||||
69
CHANGELOG.md
69
CHANGELOG.md
@@ -1,5 +1,74 @@
|
||||
# Changelog
|
||||
|
||||
## v0.26.0-preview（2026-09-29）
|
||||
|
||||
- 新增独立 i18n：不可变命名空间词典、源/默认/启用语言、BCP 47 匹配、显式回退、参数/条件/基数复数和有界资源加载；不依赖 HTTP、业务或数据库。
|
||||
- Go/浏览器共用固定 CLDR 46.1 复数规则；JSON 资源版本、类型导出和按命名空间选择的语言包独立于业务错误格式。
|
||||
- 新增可选 httpi18n/errori18n 适配；错误 translation 元数据、PublicMessage 展示入口，保留原业务码/状态/失败语义/原始原因和首次幂等响应。
|
||||
- 新增 i18n check/export/types/extract；静态与动态键诊断、未知字段/重复定义/参数校验、显式覆盖；输入只读，不执行任意项目 Go。
|
||||
- 新增无前端框架依赖的浏览器模块、按需加载/切换竞态/失败保留、CLI/HTTP/表单示例及通用验收。旧项目默认行为不变。
|
||||
- 新 translation 生成代码要求 v0.26.0-preview 或具备能力的本地源码，未使用该元数据的模板最低版本保持 v0.22。
|
||||
|
||||
- 修复 JSON 对象/数组结束定界符读取失败时丢失原始错误的问题，保留请求体限额的 HTTP 413；共享 Go 目录测试工程改用声明支持的 Go 1.22。
|
||||
|
||||
## v0.25.0-preview（2026-09-26）
|
||||
|
||||
- database.InTx 新增 TxError、TxStage、TxOutcome，区分开启/回调/提交失败以及未执行回调、已确认回滚、结果不确定；原签名保留。
|
||||
- 回调失败后只有显式回滚返回成功才标记 rolledBack；提交报错一律保守返回 unknown，不根据断网/超时或 ErrTxDone 推断回滚。不自动重试事务，不处理文件清理等业务决策。
|
||||
- Err/RollbackErr 分开保留，errors.As/Is 可遍历两条原因链；业务错误目录和公开响应行为保持。nil context 接受，nil 回调在开启前报错，panic/Goexit 保持传播并尝试回滚。
|
||||
- 新增阶段/取消竞争/确认丢失/并发/公开响应回归、真实 MySQL 驱动边界故障验收、性能比较和 examples/transactions；同步 README 与 database 指南。
|
||||
- 错误现在有类型包装；直接错误相等、直接类型断言和错误文本依赖需改为 errors.Is/As。标准库异步回滚不公开的底层原因无法还原，结果保持 unknown；不改变 GORM 自己的 Transaction。
|
||||
|
||||
## v0.24.0-preview（2026-09-26）
|
||||
|
||||
- 新增可选 cors 包和 server.cors 配置；生成项目直接加载，手写项目使用 omni.WithCORS，独立 HTTP 服务可使用 Policy.Wrap。默认关闭，无新增第三方依赖。
|
||||
- 精确来源白名单、方法/请求头、暴露头、凭据和预检缓存可配置；拒绝凭据与通配来源混用、非法来源/头及歧义的重复请求头，校验错误定位具体字段。
|
||||
- 在路由和鉴权之前处理预检；实际请求仍执行原有中间件，统一响应、404/405、上传下载和流式能力保持。nethttp/native/HTTP2 共用策略。
|
||||
- 补充真实生成项目、TLS 三种 transport、20 项真实浏览器检查、race/fuzz/性能基准；同步 README、配置指南、cors 指南及 examples/cors。
|
||||
- 没有移除现有功能或修改默认跨域行为。代理已负责 CORS 的项目保持关闭，避免重复头；高级调用 Router().ServeHTTP 不经过应用外层 CORS。
|
||||
|
||||
## v0.23.0-preview（2026-09-26）
|
||||
|
||||
- MySQL/连接池、日志、HTTP 客户端、生命周期和请求策略返回字段级安全诊断，配置加载器保留文件/变量/位置与原始错误链。
|
||||
- httpc.JSON 新增 WithSuccessStatuses，单次调用可限制允许的 2xx，默认兼容，拒绝响应继续关闭和有界回收。
|
||||
- copyx.Check[S,T] / Copier.Check 预检嵌套类型与现有规则；修正空集合隐藏类型不匹配，动态接口和 converter 的具体行为继续运行时验证。
|
||||
- logx.Config.TimeZone 绑定 Logger 实例，JSON/text/子 Logger/标准日志适配一致；不修改 time.Local 或业务时间属性。
|
||||
- 无新映射文件、强制模块或第三方依赖；更新 README、各包指南和 developerTools 可运行示例。错误文本更精确，空集合中的确定类型错误现在会被拒绝。
|
||||
|
||||
## v0.22.0-preview（2026-09-26）
|
||||
|
||||
- 文件显式环境引用与备用值，字符串组合/转义，安全有界展开和类型绑定；变量值原样插入、不递归处理。
|
||||
- 移除隐式环境覆盖、EnvOptions/前缀/别名及 env/envDefault；数据库统一使用文件加载，不再单独扫描环境。
|
||||
- 生成 main/Load 新增 -check-config，精简配置文件，保留开发者扩展；错误字段建议和迁移检查。
|
||||
- README、CONFIG、数据库/HTTP 客户端指南及示例同步新规则；配置中的响应模板需要 $$ 转义，.api 不变。
|
||||
|
||||
## v0.21.0-preview（2026-09-26）
|
||||
|
||||
- httpc 可配置环境/直连/固定代理、常用连接阶段超时和独立空闲池参数。
|
||||
- JSON 新增逐请求 Header 快照与响应大小选项，保持有界读取、关闭和认证隔离。
|
||||
- 成功请求少复制和退避计算，丢弃响应流式处理，错误体有界回收以改善连接复用。
|
||||
- 增加真实本地代理、超时、连接限制/回收、并发、日志脱敏、gzip、fuzz 和修改前后基准。
|
||||
- JSON 自定义方法接口签名需同步 variadic 参数；借用 Transport 的参数混配现在报错。详见 httpc/README.md。
|
||||
|
||||
|
||||
## v0.20.0-preview（2026-09-26）
|
||||
|
||||
- 生成项目配置加载入口持续由生成器维护；保留业务字段、默认值、校验与 NewLoader 扩展文件。
|
||||
- config.LoadProject 统一 JSON/YAML/XML、环境覆盖和校验；提供实例级 Decoder 注册。
|
||||
- 文件/字段/环境/行列的安全诊断，Invalid/AtField 与原始错误链。
|
||||
- 增加格式等价、精度、null/空数组、并发、资源限制、默认值隔离、真实生成项目与性能验收。
|
||||
- 旧手写 Load 必须显式迁移；低层 API 保留。固定数组严格长度、YAML alias/merge/custom tag 拒绝。用法及限制见 CONFIG.md。
|
||||
|
||||
|
||||
## v0.19.0-preview（2026-09-25）
|
||||
|
||||
- 新增通用可选 copyx：构造器、浅/深拷贝、精确类型转换、字段规则、nil/零值策略、并发复用。
|
||||
- 深拷贝拒绝自定义结果共享源可变引用；原子提交目标、保留错误链、限制遍历及估算分配。
|
||||
- 移除 JSON 结构映射配置与生成；提供明确迁移诊断，保留公共字段组合等其他生成能力。
|
||||
- 同步 README 全部能力索引、COPY.md、可运行示例和开发文档约定。
|
||||
- 验收与不兼容迁移见 [发布说明](RELEASE.md)、[copyx 验收](reports/copyxAcceptance.md)。v0.14–v0.18 的完整变更保留在发布说明历史段落。
|
||||
|
||||
|
||||
## v0.13.0-preview（2026-09-24）
|
||||
|
||||
- 新增通用 lifecycle.Controller，svc.RequestShutdown 支持依赖主动通知正常停止或致命故障，通知不阻塞，首个故障原因保留；生成入口排空 HTTP 后关闭依赖与日志，故障退出码为 1。
|
||||
|
||||
199
CONFIG.md
Normal file
199
CONFIG.md
Normal file
@@ -0,0 +1,199 @@
|
||||
# 配置：文件明确声明值的来源
|
||||
|
||||
v0.22 起只使用这一条流程：
|
||||
|
||||
```text
|
||||
项目 Default() → 解析选中的一个文件 → 解析字段值中的环境引用
|
||||
→ 按 Go 类型绑定已提供的字段 → 项目 Validate() → 初始化日志和依赖
|
||||
```
|
||||
|
||||
没有环境前缀、别名表或隐式二次覆盖。业务只维护 `Config`、`Default()`、`Validate()` 和配置文件。项目 Validate 中继续调用嵌套框架配置的 Validate；生成模板已包含这些调用。验证函数只检查值，不应打开资源。
|
||||
|
||||
## 直接使用
|
||||
|
||||
```go
|
||||
type Config struct {
|
||||
Server omniConfig.Config `json:"server"`
|
||||
BatchSize int `json:"batchSize"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Password string `json:"password"`
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
}
|
||||
func Default() Config {
|
||||
return Config{Server: omniConfig.Default(), BatchSize: 100, Enabled: true,
|
||||
Timeout: 3*time.Second}
|
||||
}
|
||||
func (c Config) Validate() error {
|
||||
if err := c.Server.Validate(); err != nil {
|
||||
return omniConfig.AtField("server", err)
|
||||
}
|
||||
if c.BatchSize < 0 { return omniConfig.Invalid("batchSize", "must not be negative") }
|
||||
return nil
|
||||
}
|
||||
// 手写项目；生成项目的 Load 已自动调用此入口。
|
||||
cfg, err := omniConfig.LoadProject("etc/config.yaml", Default())
|
||||
```
|
||||
|
||||
```yaml
|
||||
server:
|
||||
address: "${HTTP_ADDR:-127.0.0.1:8080}"
|
||||
password: "${DB_PASSWORD}"
|
||||
timeout: "${REQUEST_TIMEOUT:-3s}"
|
||||
enabled: false
|
||||
```
|
||||
|
||||
文件未写 batchSize，保留 100；enabled 明确写 false，就使用 false。变量名字完全由文件决定，不需要对应 Go 字段名。配置文件可只列出部署需要变动的字段；不会自动导出/打印包含凭据的完整配置。
|
||||
|
||||
## 引用语法
|
||||
|
||||
| 文件里的值 | 含义 |
|
||||
| --- | --- |
|
||||
| `host: 127.0.0.1` | 固定值，不读取任何环境变量 |
|
||||
| `password: "${DB_PASSWORD}"` | 未设置或空字符串报错，不回退到 Default() |
|
||||
| `port: "${DB_PORT:-3306}"` | 未设置或为空时使用 3306 |
|
||||
| `sessionToken: "${SESSION_TOKEN:-}"` | 未设置或为空时使用空字符串，是否合法由目标类型和 Validate 决定 |
|
||||
| `url: "https://${HOST}:${PORT:-443}/v1"` | 普通字符串中可出现多个引用 |
|
||||
| `literal: "$${DB_PASSWORD}"` | 得到字面量 `${DB_PASSWORD}`，不读取变量 |
|
||||
| `literal: "$$cost"` | `$$` 转义一个 `$`，得到 `$cost` |
|
||||
|
||||
变量名遵循 `[A-Za-z_][A-Za-z0-9_]*`。只支持 `${NAME}`、`${NAME:-fallback}` 和 `$$` 转义，不支持 `${NAME-default}`、嵌套引用或 shell 表达式。普通单独的 `$` 保留。fallback 到第一个 `}` 结束，可包含冒号、`:-`、引号等文件格式允许的字符；需要包含 `}` 的值可使用固定值或环境变量。
|
||||
|
||||
解析只扫描文件的**已解码字段值**，不替换 YAML/JSON/XML 源文本，不扩展字段名，也不扫描 Default() 的字符串。插入的环境值和 fallback 不再展开。密码 `DDSFLKJlsf:saf:-123`、引号、换行、`${OTHER}` 均原样保留，不破坏文档结构。字符串不统一 TrimSpace；`"0"`、`"false"` 和全空格字符串均属于已提供。
|
||||
|
||||
同一变量每次加载最多读取一次；下一次加载会重新读取。没有全局变量值缓存。并发加载各自维护结果，不能把调用者正在修改的默认配置当作共享可变输入。
|
||||
|
||||
## 类型与覆盖
|
||||
|
||||
目标 Go 类型决定引用结果的转换：整数、浮点数、布尔值、`time.Duration`、字符串、指针及已有自定义 JSON/Text 反序列化。数值范围按目标位宽检查，整数不经过 float64。自定义反序列化仍由自定义类型解释文本，不猜测其业务含义。
|
||||
|
||||
- 文件缺失字段保留初始值；显式 0、false、空字符串正常覆盖，不用零值判断字段是否存在。
|
||||
- `${MISSING}` 失败就报错；需要备用值必须写 `${MISSING:-value}`。
|
||||
- 字符串编号 `"001"` 保持 `001`；绑定到 int 才转整数。环境里的 `null` 不自动变成空指针，`[1,2]` 不自动变成数组。
|
||||
- 集合直接在文件里声明，可在每个标量元素中使用引用，例如 `ports: ["${PORT_A:-8080}", "${PORT_B:-8081}"]`。
|
||||
- 数组整体替换；空 slice 保持非 nil；固定长度 Go 数组要求长度一致。map 合并键，已提供键的值整体替换。
|
||||
- 固定文件 null 沿用 JSON 语义：清空指针/map/slice/interface，普通值字段保持初始值。需要区分 null 时使用指针。
|
||||
- `any` 的引用结果保持字符串，不猜测动态类型。固定 JSON/YAML 数字使用 json.Number 保留精度。
|
||||
- 引用成功后的值不会再覆盖一次。旧 `GO_OMNI_*` 名称只有出现在文件引用中才有作用；不隐式重置日志 file/rotation。
|
||||
|
||||
普通固定字符串仍按原有严格绑定规则处理；不会为了支持变量，把所有 `"123"` 都自动转换成数字。
|
||||
|
||||
## YAML、JSON、XML 一致
|
||||
|
||||
使用 Go 的 `json` 字段名，无需维护三份 tag。以下三份文件等价：
|
||||
|
||||
```yaml
|
||||
port: "${PORT:-3306}"
|
||||
password: "${PASSWORD}"
|
||||
```
|
||||
|
||||
```json
|
||||
{"port":"${PORT:-3306}","password":"${PASSWORD}"}
|
||||
```
|
||||
|
||||
```xml
|
||||
<config><port>${PORT:-3306}</port><password>${PASSWORD}</password></config>
|
||||
```
|
||||
|
||||
XML 使用 `<config>` 根，标量占位符写在普通叶元素或 `type="string"` 元素中，由 Go 字段决定类型；显式 `type="number"` / `type="bool"` 表示文件里已经有合法数字/布尔常量。XML 数组用 `type="array"` 加 `<item>`；空对象 `type="object"`、null 用 `type="null"`。不从重复元素猜测数组。
|
||||
|
||||
`.api` 不经过配置加载器，响应模板 `${code}` 不变。**如果把响应模板写在 YAML/JSON/XML 配置文件里，须转义：**
|
||||
|
||||
```yaml
|
||||
server:
|
||||
response:
|
||||
body:
|
||||
code: "$${code}"
|
||||
msg: "$${message}"
|
||||
data: "$${data}"
|
||||
```
|
||||
|
||||
加载后恢复为 `${code}` 等响应占位符，仍由响应编码器处理。
|
||||
|
||||
## 生成项目与检查
|
||||
|
||||
| 文件 | 维护方 |
|
||||
| --- | --- |
|
||||
| `main.go`、`internal/config/loadGen.go` | 生成器维护，纳入 `generate --check` |
|
||||
| `internal/config/config.go` | 项目维护 Config、Default、Validate |
|
||||
| `etc/config.yaml` 等选定文件 | 项目维护，重新生成不覆盖 |
|
||||
| `internal/config/loader.go` | 初次生成即可使用，仅添加自定义格式时才修改 |
|
||||
|
||||
```sh
|
||||
go run . -config etc/config.test.yaml -check-config
|
||||
go run . -config etc/config.local.yaml
|
||||
go build .
|
||||
go-omni api generate --check
|
||||
```
|
||||
|
||||
`-check-config` 使用与启动完全相同的 Load，通过时退出 0、失败时非零；不打印配置，不创建日志文件、数据库、后台组件或 HTTP 监听。它只校验配置，不检测真实数据库连接或证书握手。
|
||||
|
||||
`-config` 只选一个文件，不自动叠加 local/test/default。指定文件不存在时报错；`-config ''` 只使用 Default() 后校验，不读取任何环境变量，也不寻找其他文件。示例：[examples/configuration](examples/configuration/README.md)。
|
||||
|
||||
## 安全诊断与限制
|
||||
|
||||
```text
|
||||
config "etc/config.yaml":7:9 env "DB_PORT" field "database.port": expected a valid int
|
||||
```
|
||||
|
||||
`*config.Error` 包含 File/Field/Env/Line/Column/Reason。引用错误同时保留文件与环境变量名；组合引用失败可列出多个名字。行列定位原文件字段，不把环境值伪装成源文件文本。无法取得位置时仍提供可用的文件/字段信息。
|
||||
|
||||
用 `config.Invalid(field, publicReason)` 或 `config.AtField(prefix, err)` 报业务校验错误。未知字段给出唯一且接近的已声明字段建议，不自动修改。错误默认不打印原值；自定义 Reason 不应拼入秘密值。原始 cause 仍支持 errors.Is/As，不要把 Unwrap 后的错误或整个配置写到日志。
|
||||
|
||||
文件最多 1 MiB；单个环境值/展开字段最多 1 MiB，展开字符串合计/变量缓存分别最多 4 MiB；解析/展开限制 64 层、100000 个节点与引用计数。拒绝无效 UTF-8、多文档、重复字段、未知字段、非普通文件；YAML 不接受 alias/merge/custom tag，XML 不接受 DTD/外部实体。不执行 shell，不自动加载 `.env`。
|
||||
|
||||
## 框架模块的字段诊断
|
||||
|
||||
v0.23 起 MySQL（含连接池/TLS）、通用数据库池、日志（含轮转/时区）、HTTP 客户端、退出阶段和请求策略返回字段级安全原因。沿用项目已有的 `config.AtField("database", err)` / `config.AtField("server", err)`，自动组合嵌套字段，不需业务重复校验基础参数。
|
||||
|
||||
例如文件中 `database.port: "${DB_PORT}"` 的引用值为 70000：
|
||||
|
||||
```text
|
||||
config "etc/config.yaml":3:9 env "DB_PORT" field "database.port": must be between 1 and 65535
|
||||
```
|
||||
|
||||
日志级别非法指向 `server.log.level`；池参数指向 `database.pool.maxIdleConns` / `pingTimeout`。框架只公开已定义的安全原因，第三方/项目普通 error 仍显示 validation failed 并保留 errors.Is/As 原因链；业务使用 Invalid/AtField 提供公开原因。不要解析错误字符串；使用 `*config.Error` 字段。未提供字段的位置可能不可用，字段与原因仍保留。
|
||||
|
||||
生成项目现有 Load 和 `-check-config` 自动获得这些诊断。日志可在 `server.log.timeZone` 配置 IANA 名称，见 [实例时区](logx/README.md#实例时区)。
|
||||
|
||||
## 可选解析器与测试来源
|
||||
|
||||
内置格式无需项目手写加载器。需要额外格式时在已有 NewLoader 注册 Decoder，后续仍自动经过同一引用/绑定/校验流程：
|
||||
|
||||
```go
|
||||
func NewLoader() (*omniConfig.Loader, error) {
|
||||
return omniConfig.NewLoader(
|
||||
omniConfig.WithDecoder("custom", omniConfig.JSONDecoder{}),
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
Decoder 返回 Document（map/string/bool/json.Number/nil/[]any，XML 类标量可用 Text），可提供 Positions。实例注册无全局 init；插件必须可并发调用、不修改共享输入，并且是可信 Go 代码。
|
||||
|
||||
测试或嵌入宿主可以用 `WithLookupEnv(func(string)(string,bool))` 替换引用查找函数。它仍然只读取文件明确引用的名称，不启用任何自动覆盖。普通业务不需要这个选项。
|
||||
|
||||
## v0.22 迁移：旧模式已移除
|
||||
|
||||
这是不兼容调整，不保留两套加载逻辑：
|
||||
|
||||
- 删除 `EnvOptions`、`LoadFromEnv`、`ApplyEnv`、`OverlayEnv`、`ApplyDefaults`、`ServerEnvAliases` 和 `env`/`envDefault` 标签处理；检出遗留标签会提示迁移。
|
||||
- 删除 `database.ConfigFromEnv` / `database.ApplyEnv`；数据库配置作为项目 Config 的字段绑定。DSN 字段可在文件里使用 `dsn: "${DB_DSN}"`，序列化 database.Config 不输出 DSN 原值。
|
||||
- `LoadProject(path, defaults[, loader])` 不再接收 EnvOptions；`LoadFile(path)`、`NewReloader(path, logger)` 不再接收前缀。
|
||||
- `omni.LoadConfig(path)` 的参数现在是文件路径；空路径只用框架默认值。
|
||||
|
||||
迁移顺序：
|
||||
|
||||
1. 将 envDefault 默认值搬入 Default()，移除 env/envDefault 标签。需要文件绑定的旧 `json:"-"` 密钥字段改成明确 json 名称；不打印配置对象。
|
||||
2. 按实际部署需要，在选定文件写环境引用。固定值保持固定，不按原前缀猜测。原先依赖环境切换日志输出的项目，应在文件明确配置 file/output/rotation。
|
||||
3. 配置文件中的其他 `${...}` 模板改成 `$${...}` 字面量；`.api` 无需修改。
|
||||
4. 手写项目改为 LoadProject。生成项目保留 Config/Default/Validate/NewLoader，删除旧手写 Load 后重新生成；main/loadGen 由生成器更新，业务配置文件不会被静默改写。
|
||||
5. 升级匹配的 CLI 与框架，执行 generate、generate --check、go test、go build 和 `-check-config`，再使用真实部署配置启动。
|
||||
|
||||
低层 `DecodeFile(path, defaults[, loader])` 也解析显式引用，但不调用 Validate；默认值和原始 Document 都不会因失败而被部分修改。日志 Reloader 仍只允许 log.level 变化，其他静态字段变更要求重启。
|
||||
|
||||
## 国际化配置
|
||||
|
||||
通用国际化的 `i18n.Config` 可作为项目配置的字段显式加载并调用 Validate。正文词典使用 `i18n.LoadFS` / `DecodeJSON`，不经过环境引用展开。构建 Bundle 后通过 `WithLanguagePolicy` / `WithErrorLocalizer` 装配借用的只读实例；没有隐式环境变量或全局当前语言。见 [国际化指南](i18n/README.md) 和 [完整示例](examples/i18n/README.md)。
|
||||
|
||||
## 跨域配置
|
||||
|
||||
生成项目的 `server.cors` 由本加载流程直接绑定、校验并交给 App，不需要修改生成入口。支持固定来源数组或元素内显式环境引用；非法来源定位具体数组元素。默认关闭，完整配置与行为见 [CORS 指南](cors/README.md)。
|
||||
166
COPY.md
Normal file
166
COPY.md
Normal file
@@ -0,0 +1,166 @@
|
||||
# 可选拷贝工具 copyx
|
||||
|
||||
v0.19 用普通 Go 调用取代 JSON 结构映射生成。`copyx` 只依赖标准库，不需要注册到 App，不执行 JSON 往返，也不会自动暴露数据库模型。普通 Logic 直接使用 `.api` 生成的 DTO；只有存在独立业务类型或需要隔离可变数据时才用它。
|
||||
|
||||
完整可运行代码：[examples/copying](examples/copying)。在框架根目录运行：
|
||||
|
||||
```sh
|
||||
go run ./examples/copying
|
||||
go test -race ./copyx ./examples/copying
|
||||
```
|
||||
|
||||
## 最简单的调用
|
||||
|
||||
```go
|
||||
import "git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
|
||||
type Request struct { Name string; Enabled bool }
|
||||
type Input struct { Name string; Enabled bool }
|
||||
|
||||
req := Request{Name: "sample", Enabled: false}
|
||||
var input Input
|
||||
if err := copyx.Copy(&input, req); err != nil { return err }
|
||||
```
|
||||
|
||||
同名、同类型字段直接赋值。默认不会跳过 `false`、`0`、空字符串。目标字段没有来源时报错；多余源字段默认忽略，使用 `WithStrictFields()` 可要求源字段也全部使用或显式忽略。
|
||||
|
||||
默认是浅拷贝：同类型指针、map、slice 共享引用，符合普通 Go 赋值。JSON tag 不参与字段匹配。不会自动调用方法、SQL Scanner/Valuer，或把 `int32`、`int64`、string、命名标量互相强转。
|
||||
|
||||
新字段：普通接口只改 `.api` 和使用该字段的 Logic。独立 DTO/业务类型都需要此字段时，两边各声明一次；同名同类型不增加拷贝配置。检查在运行时发生，实际类型对必须有测试，不能把运行时工具当作旧生成器的编译期字段覆盖检查。
|
||||
|
||||
## 类型预检
|
||||
|
||||
v0.23 新增可选预检，无需填写映射文件或构造测试值：
|
||||
|
||||
```go
|
||||
// 参数方向是源 → 目标；与 Copy 使用同一组选项。
|
||||
if err := copyx.Check[Request, BusinessInput](options...); err != nil {
|
||||
return err
|
||||
}
|
||||
// 复用已有 Copier：传结果类型，不添加 Copy 所需的额外输出指针。
|
||||
err := copier.Check(reflect.TypeFor[Request](), reflect.TypeFor[BusinessInput]())
|
||||
```
|
||||
|
||||
适合 CI 测试或初始化检查；这是程序执行的类型检查，不是 Go 编译器自动证明。递归检查切片/数组元素、map 键值、指针、嵌套字段、改名、忽略和严格覆盖；用已访问类型对处理递归结构。不执行 converter，也不调用构造器，不读取实例值。错误路径如 `$.Rows[].Value: copyx: incompatible types: string -> int`。
|
||||
|
||||
Check 复用 Copy 的规则，包括浅拷贝对同类型容器的直接赋值：这种情况下内部 converter 不会自动逐项执行。需要处理内部字段时选择深拷贝或容器 converter。
|
||||
|
||||
- 未知具体值的源 interface 需要运行时验证；无法静态判定时返回 `ErrDynamicType`。项目可给该接口类型对提供精确 converter 并承担内容验证。
|
||||
- 已注册 converter 的签名被接受，但函数不在 Check 中执行；其返回错误、panic、金额规则不能靠预检证明。
|
||||
- nil 到非可空目标、map 键转换碰撞、深拷贝结果是否残留源引用继续在 Copy 时检查。Check 成功不会关闭这些保护。
|
||||
- 实际 Copy 也修正了空集合的检查缺口：确定的嵌套类型不匹配不再因为切片/map 为空或数组长度为 0 而通过。需要这些转换时提供显式 converter；目标仍只在整体成功后更新。
|
||||
- 不引入全局配置、映射语言或业务类型；Copier.Check 可并发调用，不共享每次检查的遍历状态。
|
||||
|
||||
示例：[developerTools](examples/developerTools/README.md)。普通同名同类型字段不需要额外配置。
|
||||
|
||||
## 深拷贝与构造器
|
||||
|
||||
```go
|
||||
snapshot, err := copyx.Clone(source) // 保持源的静态类型，始终深拷贝
|
||||
if err != nil { return err }
|
||||
|
||||
copier, err := copyx.New(copyx.WithDeepCopy(), copyx.WithStrictFields())
|
||||
if err != nil { return err }
|
||||
var result Output
|
||||
if err := copier.Copy(&result, source); err != nil { return err }
|
||||
```
|
||||
|
||||
`Copy` 的目标必须是非 nil 指针；`Clone[T]` 返回 `(T, error)`。重复执行时在初始化阶段创建 Copier 并复用；配置之后不可变，每次调用独立持有遍历状态，可以并发使用。源、目标的并发读写仍由调用方同步，自定义函数也须支持并发。
|
||||
|
||||
深拷贝会分配指针、map、slice，递归处理结构、数组、接口中的具体值。nil 集合与非 nil 空集合保持区别；slice 结果容量为长度，不复制长度之外的隐藏元素。普通循环和重复指针可处理。循环图应保留根指针，使用 `Clone(sourcePointer)` 或 `Copy(&targetPointer, sourcePointer)`；将循环指针根压成值结构会明确报错。
|
||||
|
||||
它保证结果不引用输入中的可变对象，不保证保留所有底层内存布局关系：不同重叠 slice 视图会分别复制，内部指针指向某个切片元素的关系也不重建。字符串和零尺寸值可安全共享；`time.Time` 按 Go 不可变值使用（Location 不做独立复制）。不要通过 unsafe 或 uintptr 隐藏可变引用后要求工具识别。
|
||||
|
||||
## 字段差异与自定义转换
|
||||
|
||||
```go
|
||||
type Input struct { Name string; UpdatedAt time.Time; InternalID string }
|
||||
type View struct { DisplayName string; UpdatedAt string; LocalNote string }
|
||||
|
||||
copier, err := copyx.New(
|
||||
copyx.WithDeepCopy(),
|
||||
copyx.WithStrictFields(),
|
||||
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
|
||||
copyx.WithIgnoreSource[Input, View]("InternalID"),
|
||||
copyx.WithIgnoreTarget[Input, View]("LocalNote"),
|
||||
copyx.WithConverter(func(v time.Time) (string, error) {
|
||||
return v.UTC().Format(time.RFC3339), nil
|
||||
}),
|
||||
)
|
||||
if err != nil { return err }
|
||||
out := View{LocalNote: "preserve"}
|
||||
if err := copier.Copy(&out, input); err != nil { return err }
|
||||
```
|
||||
|
||||
字段规则的类型参数必须是结构类型，字段名使用导出的 Go 字段名。构造时检查不存在的字段、重名及冲突。不支持字符串路径表达式；复杂嵌套差异为该嵌套类型注册 converter，或直接写 Go。
|
||||
|
||||
`WithIgnoreSource` 只是声明该源字段不参与使用；目标同名字段若仍需保留，还需 `WithIgnoreTarget`。忽略目标会保留原目标值，Clone 中则为零值。深拷贝时如果保留值仍与源共享可变引用，也会拒绝。
|
||||
|
||||
转换器的签名是 `func(S) (T, error)`，按**精确类型对**匹配，不能重复注册。金额、编号、日期格式由项目明确转换。需要逐字段不同规则时，给整个源/目标结构注册函数；函数返回独立的新目标。无需另一份映射配置或全局注册。
|
||||
|
||||
精确 converter 优先于 nil/零值策略，能收到 typed nil；例如 `func(*Money) (Money, error)` 可自行决定 nil 是否允许。绑定到 `any` 字段的 converter 能接收 nil interface，但顶层未带类型的 `nil` 没有精确类型可匹配。浅拷贝中已经直接共享的同类型容器不会继续逐项转换其内部；需要遍历内部时选择深拷贝，或为该容器类型注册 converter。
|
||||
|
||||
## 深拷贝不能原样返回可变引用
|
||||
|
||||
```go
|
||||
copier, err := copyx.New(
|
||||
copyx.WithDeepCopy(),
|
||||
copyx.WithConverter(func(v []string) ([]string, error) {
|
||||
return v, nil // 会返回 ErrSharedReference
|
||||
}),
|
||||
)
|
||||
```
|
||||
|
||||
检查覆盖完整源图与转换结果，包括新父对象引用旧子对象、原 map、重叠子切片、`source[:0]` 的可写容量、指向源数组元素的指针，以及未导出字段中的引用。会检查每个 converter 的结果和最终目标，不允许静默降级为浅拷贝。
|
||||
|
||||
返回相同字符串、整数并不会报错，它们不是可变引用。正确切片 converter 可以写：
|
||||
|
||||
```go
|
||||
copyx.WithConverter(func(v []string) ([]string, error) {
|
||||
if v == nil { return nil, nil }
|
||||
out := make([]string, len(v))
|
||||
copy(out, v)
|
||||
return out, nil
|
||||
})
|
||||
```
|
||||
|
||||
第三方 Decimal/自定义 Money 如果含未导出的指针，默认深拷贝拒绝；注册该类型的 converter，使用它自己的构造/Clone API 创建独立值，避免转成 float64 损失精度。框架不内置某一金额库的规则。无法独立复制的类型直接返回错误。
|
||||
|
||||
## nil、零值和错误
|
||||
|
||||
| 选项 | 行为 |
|
||||
| --- | --- |
|
||||
| 默认 `NilError` | nil → 可空目标保留 nil；nil → 非可空目标报 `ErrNil` |
|
||||
| `WithNilPolicy(NilZero)` | 显式允许 nil → 零值；可空目标仍为 nil |
|
||||
| `WithNilPolicy(NilSkip)` | nil 保留已有目标；新集合元素没有旧目标，保留元素零值 |
|
||||
| `WithSkipZero()` | 显式跳过 Go 零值，包括 `false`、`0`、`""`；非 nil 空集合不是 Go 零值 |
|
||||
|
||||
所有 nil 策略都不把不兼容类型自动变成兼容类型。非 nil 指针与值之间可以按对应元素转换；nil 到非可空目标仍需选择策略。局部更新不能默认使用 NilZero/SkipZero，它们可能改变清空字段或布尔值的业务语义。
|
||||
|
||||
工具只能复制绑定后的 Go 值：如果输入类型已经将“缺失”和“JSON null”合并为 nil，无法恢复两者区别。需要三态时，应先设计包含 presence 信息的类型和绑定逻辑。
|
||||
|
||||
赋值使用临时结果，成功后才替换目标；转换失败不会交付部分结果。`*copyx.Error` 包含 Go 字段/索引路径，`errors.Is/As` 保留内部错误链，工具不主动在错误中加入源值。converter 自己返回的错误文本仍由项目负责。
|
||||
|
||||
converter 必须只读输入；其外部副作用、输入修改无法回滚，panic 原样传播。框架只检查与本次输入的共享引用，无法证明 converter 未返回外部全局缓存的引用。它是受信 Go 代码扩展，不是可运行不可信代码的沙箱。
|
||||
|
||||
## 支持范围与资源限制
|
||||
|
||||
- 支持结构体、指针、interface、slice、同长度 array、map。map key 转换后冲突或不可比较则报错。
|
||||
- 导出匿名结构字段可按提升后的无歧义名称访问；源嵌入指针为 nil 且需要其字段时明确报错。跨类型目标包含私有字段时拒绝；同类型私有纯值可作为整体保留。
|
||||
- 不克隆文件、数据库连接、网络连接、锁、原子状态。已知标准库资源/同步结构按值复制也拒绝；深拷贝中的非 nil func、chan、unsafe.Pointer 拒绝。其他封装资源类型应由项目排除，工具不能识别所有自定义句柄。
|
||||
- 单次遍历深度上限 256、节点上限 200,000；内置分配的估算预算为 64 MiB，超限返回 `ErrLimit`。预算不是进程堆内存硬限制，reflect 元数据和 converter 自行分配不计入。大对象流式读写应使用 IO/存储接口。
|
||||
- 只缓存不可变类型元数据，不缓存请求值、用户 converter 或实例策略。没有 `unsafe` 写入、JSON 往返、隐式数值截断或方法调用。
|
||||
- 这是运行时反射工具，会比直接 Go 赋值多检查与分配。热点应复用 Copier，并用真实数据测量；性能记录见 [本版验收](reports/copyxAcceptance.md)。
|
||||
|
||||
## 从旧 mapping 迁移
|
||||
|
||||
v0.19 **彻底移除**映射配置加载、Go 类型加载、映射代码生成及 `pointerResult` 等附属选项。公共 `.api` 字段组合、scalar、中间件工厂、Repository 开关继续存在。
|
||||
|
||||
1. 先检查每个转换是否必要。普通 Logic 可直接使用生成 DTO；独立核心保留自己的类型。
|
||||
2. 将 `mapping.Xxx(...)` 调用改为普通 Go 或小型 `copyx` helper。字段改名、nil、金额规则写在该 helper 同一处。指针返回可直接 `var out *Response; err := copier.Copy(&out, input)`。
|
||||
3. 用真实类型对测试缺失/null/零值/空数组、字段覆盖、金额精度和错误链。需要源字段新增时提醒遗漏，启用 `WithStrictFields()` 并测试。
|
||||
4. 从 `omni.gen.json` 删除 `mappings` / `mappingFiles`，删除无其他用途的映射 JSON 文件。
|
||||
5. 确认已无调用，再删除 `internal/mapping/mappingsGen.go`。如之前手改过文件，先迁出手写内容。生成器遇到该旧文件会阻止生成并提示迁移，**不会静默删除**；其余业务文件照常保留。
|
||||
6. 升级框架与 CLI 到同版，执行 generate、generate --check、go test ./...；多个服务再执行 workspace generate/check。新 manifest 自动去掉已移除文件的所有权记录。
|
||||
|
||||
历史发布记录保留旧能力说明，仅用于理解旧版本，不代表 v0.19 继续支持。
|
||||
@@ -1,8 +1,8 @@
|
||||
# 通用业务开发指南（v0.17.0-preview）
|
||||
# 通用业务开发指南（v0.26.0-preview）
|
||||
|
||||
从 [README](README.md) 创建服务后，业务主要维护 `.api`、配置、`svc.Init`、Logic、Repository 和项目中间件。本页说明通用接入能力；字段、错误码、权限、表结构和事务边界都由项目定义。
|
||||
|
||||
本版新增的公共字段组合、可选结构映射、自动中间件工厂和 Repository 开关，见 [生成能力指南](GENERATION.md)。
|
||||
公共字段组合、自动中间件工厂和 Repository 开关，见 [生成能力指南](GENERATION.md)。必要的结构转换使用可选 [copyx](COPY.md)，v0.19 已移除旧 JSON 映射生成。
|
||||
|
||||
## 返回数组、可空对象和 null
|
||||
|
||||
@@ -100,42 +100,13 @@ defer reader.Close()
|
||||
|
||||
不声明 multipart 字段时，保留原来的 `body *httpx.Body` 流式入口，不创建临时文件。该方式适合特殊协议；项目应在不可撤销副作用之前完成校验并调用 `body.Complete()`。下载和原始回调继续使用 `@produces` / `@body(mode: raw)`。
|
||||
|
||||
## 完整业务配置的环境覆盖
|
||||
## 文件声明配置来源
|
||||
|
||||
生成项目默认执行：代码与 `envDefault` 默认值 → JSON/YAML 文件 → 全结构体环境覆盖 → 既有框架环境别名 → `Validate`。增加业务字段不需要另外写 `os.Getenv`。
|
||||
项目维护 Config、Default()、Validate() 和配置文件；生成器维护 Load/main。流程为默认值→选定文件→显式 `${NAME}` / `${NAME:-fallback}` 引用→类型绑定→校验。没有前缀、别名或 env/envDefault 隐式覆盖。
|
||||
|
||||
```go
|
||||
type ClientConfig struct {
|
||||
Endpoint string `json:"endpoint" env:"CLIENT_ENDPOINT,required"`
|
||||
Token string `json:"-" env:"CLIENT_TOKEN,required,secret,empty=reject"`
|
||||
Timeout time.Duration `json:"timeout" envDefault:"3s"`
|
||||
Retries []int `json:"retries"`
|
||||
Enabled bool `json:"enabled"`
|
||||
}
|
||||
```
|
||||
固定值保持固定，缺失字段保留 Default()，0/false/空字符串正常覆盖；引用失败不回退初始化默认值。`$$` 转义 `$`，字符串凭据保留空格与特殊字符。
|
||||
|
||||
把字段加入项目 `Config` 后，默认环境名取 `GO_OMNI_` 加 JSON 路径的 UPPER_SNAKE_CASE，如 `GO_OMNI_CLIENT_TIMEOUT`。显式 env 名是完整名称，可以使用自己的业务前缀。`env:"-"` 排除整个子树；`json:"-"` 字段只有显式 env 标签才参与。
|
||||
|
||||
duration 使用 `250ms`、`3s`；slice/map 使用 JSON，如 `[1,3,5]`、`{"retry":"2s"}`。支持嵌套、零/false、空数组、自定义 TextUnmarshaler。空环境值默认允许；可选择 `empty=ignore` 或 `empty=reject`。required 要求字符串非空、指针/集合非 nil；0 和 false 可以是合法业务值，范围由项目 Validate 决定。
|
||||
|
||||
手写项目可使用 `config.LoadProject(path, defaults, config.EnvOptions{Prefix:"MY_APP_"})`，或分别调用 `ApplyDefaults/DecodeFile/OverlayEnv`。读取失败不修改输入默认值；诊断不会打印输入值。不要主动打印含密钥的整个配置，或把底层解析错误展开到公开响应。Validate 不打开日志文件、数据库或存储连接；证书的可读性和握手信任在依赖初始化阶段验证。
|
||||
|
||||
### 一次覆盖完整配置与 HTTP/日志短名称
|
||||
|
||||
```go
|
||||
cfg, err = config.OverlayEnv(cfg, config.EnvOptions{
|
||||
Prefix: "MY_APP_",
|
||||
Empty: "reject", // allow / ignore / reject，项目自行选择
|
||||
Aliases: config.ServerEnvAliases("MY_APP_", "server"),
|
||||
// Lookup 可注入测试或项目配置源；默认 os.LookupEnv。
|
||||
})
|
||||
```
|
||||
|
||||
不要再紧接着调用 `ApplyEnv(cfg.Server, ...)`。上面的调用同时处理完整配置和 HTTP_ADDR、SERVICE、LOG_LEVEL 等框架短名称。例如 `MY_APP_SERVER_ADDRESS` 与 `MY_APP_HTTP_ADDR` 指向同一字段，**完整名称优先**；未设置时才读取短名称。完整名称显式设空仍优先：allow 交给该字段解析（string 可清空；数字等仍要求合法值），ignore 保留文件/默认值，reject 报错。不会因 ignore 再退回别名。字段 `env` 标签的 empty 规则优先于全局规则。
|
||||
|
||||
自定义别名用 `EnvOptions.Aliases`，键是完整环境名、值是按优先级排列的别名数组。两种名称都使用同一 Lookup、空值策略和类型转换；布尔 false、数值 0 与未设置不同。集合始终使用 JSON，包括 `MY_APP_TRUSTED_PROXIES='["127.0.0.1"]'`。
|
||||
|
||||
新项目模板一次调用 OverlayEnv，默认 `Empty: "ignore"`。旧 `ApplyEnv` 保留原有忽略空值、修剪空白及 TRUSTED_PROXIES 逗号列表行为。已有项目的 `internal/config` 属于开发者文件，升级不覆盖它；按上例合并原来的两次覆盖调用后，保留项目自己的 Validate。日志输出的显式完整名称也优先于短名称。
|
||||
`go run . -config etc/config.test.yaml -check-config` 只校验，不打开日志、数据库或端口。生成/check/dry-run 保留业务文件；新的模板要求 v0.22。完整格式、错误定位、解析器与移除清单见 [CONFIG.md](CONFIG.md)。
|
||||
|
||||
## 依赖、日志与中间件
|
||||
|
||||
@@ -278,7 +249,7 @@ go-omni workspace check --file omni.workspace.json
|
||||
|
||||
## v0.13 项目升级
|
||||
|
||||
先使用与目标框架匹配的 CLI；CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.17.0-preview（或显式引用当前源码）。
|
||||
先使用与目标框架匹配的 CLI；CLI 会提示与 go.mod 不同的版本。当前生成模板至少需要 v0.22.0-preview（或显式引用当前源码）。
|
||||
|
||||
```sh
|
||||
go-omni api upgrade --dir . --dry-run
|
||||
|
||||
136
GENERATION.md
136
GENERATION.md
@@ -1,18 +1,10 @@
|
||||
# 公共字段、结构映射与生成选项
|
||||
# 公共字段、中间件工厂与生成选项
|
||||
|
||||
本页对应 `v0.17.0-preview`。CLI 与框架依赖固定到同一版本：
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
|
||||
# 已有业务项目内执行：
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
|
||||
```
|
||||
|
||||
私有仓库访问设置见 [安装说明](README.md#准备环境与安装)。升级不会覆盖业务文件；已有中间件保持 handlers 模式，切换工厂模式按本页迁移步骤完成。
|
||||
本页对应 `v0.19.0-preview`。CLI 与框架依赖固定到同一版本，安装见 [README](README.md#准备环境与安装)。
|
||||
|
||||
## 一份项目生成配置
|
||||
|
||||
新项目根目录生成开发者维护的 `omni.gen.json`：
|
||||
新项目根目录的 `omni.gen.json` 只描述生成选项：
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -21,12 +13,11 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
|
||||
}
|
||||
```
|
||||
|
||||
该文件与 `.api` 一起提交。每次 `api generate`、`--check`、`--dry-run` 和 `workspace generate/check` 都会读取；`api validate/export/format` 只处理契约，不加载业务 Go 类型。
|
||||
|
||||
- `repository: none` 是默认值，不创建空 Repository 文件。`group` 按 `.api` 的 group 补齐 `internal/repository/<group>/repository.go`；文件始终属于业务。切回 none 后不再补骨架，已有文件不删除、不覆盖。实际 SQL、ORM 和仓储划分由项目决定。
|
||||
- `middleware: factories` 是新项目默认值，按下文自动初始化。`handlers` 保留原有 `Defaults()` 与 `func(next Handler) Handler` 接口，供既有项目逐步迁移。
|
||||
- `mappings` 可省略；只有明确列出的类型对才生成映射。未启用时不会运行 Go 类型加载命令，也不增加运行时依赖。
|
||||
- 升级首次遇到没有配置的旧 manifest 时，补一份 `middleware: handlers` 的配置，保留原有手写注册；Repository 默认 none。生成配置建立后不能直接删除，否则生成失败，避免误关闭映射或改变中间件模式。
|
||||
- `repository: none` 默认不创建空 Repository 文件；`group` 按 group 补齐可编辑骨架。已有文件始终保留；切回 none 不会删除业务文件。SQL、ORM 和仓储划分由项目决定。
|
||||
- `middleware: factories` 是新项目默认值，自动调用带依赖的工厂。`handlers` 保留原有 `Defaults()` 及手动注册方式，供既有项目迁移。
|
||||
- 该文件与 `.api` 一起提交。generate/check/dry-run 和 workspace 采用相同规则；validate/export/format 只处理接口契约。
|
||||
- 旧 manifest 没有配置时补 `middleware: handlers`、`repository: none`。已有配置不能直接删除，避免意外切换中间件模式。
|
||||
- v0.19 已删除 `mappings` 和 `mappingFiles`；遇到旧配置会明确报错，不读取子映射文件或执行映射生成。处理步骤见 [copyx 迁移](COPY.md#从旧-mapping-迁移)。
|
||||
|
||||
## 公共字段组合
|
||||
|
||||
@@ -53,104 +44,13 @@ type AdminRequest {
|
||||
|
||||
允许多层组合，上限 32 层、单类型 4096 字段、全契约展开 65536 字段。Go 匿名指针嵌入并非本功能；生成 DTO 中不会使用匿名嵌入。
|
||||
|
||||
## 可选结构映射
|
||||
## DTO 与业务类型
|
||||
|
||||
`.api` 生成 DTO，业务独立维护输入/结果类型。项目明确选择需要自动赋值的类型对；不从数据库反向生成公开接口，不自动推断反向映射。
|
||||
普通 Logic 直接接收生成的 Request、返回生成的 Response，不额外创建相同结构。只有独立核心或明确不同的输入/输出职责才保留业务类型。
|
||||
|
||||
假设 `types.AdminRequest` 有 `Name string`、`Salary amount.Money`、`CompanyID string`，业务的 `domain.EmployeeInput` 有 `EmployeeName string` 和同类型 `Salary`：
|
||||
必要转换直接写 Go，或用可选 `copyx.Copy(&target, source)`。同名同类型字段自动赋值，字段改名和类型变化通过 Go 参数、转换函数明确设置，见 [完整示例](examples/copying/main.go)。无需 JSON 映射配置、包路径字符串或额外生成文件。
|
||||
|
||||
```json
|
||||
{
|
||||
"repository": "none",
|
||||
"middleware": "factories",
|
||||
"mappings": [
|
||||
{
|
||||
"name": "ToEmployeeInput",
|
||||
"source": {"package": "example.com/payroll/internal/types", "type": "AdminRequest"},
|
||||
"target": {"package": "example.com/payroll/internal/domain", "type": "EmployeeInput"},
|
||||
"fields": {
|
||||
"EmployeeName": {"source": "Name"}
|
||||
},
|
||||
"ignoreSource": {
|
||||
"CompanyID": "由 Logic 单独做企业范围授权，不属于请求体输入"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
执行 `go-omni api generate` 后，产生独立的生成文件 `internal/mapping/mappingsGen.go`：
|
||||
|
||||
```go
|
||||
func ToEmployeeInput(source types.AdminRequest) (domain.EmployeeInput, error) {
|
||||
var result domain.EmployeeInput
|
||||
result.EmployeeName = source.Name
|
||||
result.Salary = source.Salary
|
||||
return result, nil
|
||||
}
|
||||
```
|
||||
|
||||
包别名由生成器决定。业务调用 `mapping.ToEmployeeInput(*req)`；反向响应映射按另一条 source/target 规则声明。两个类型完全相同时通常无需映射。
|
||||
|
||||
### 类型与覆盖检查
|
||||
|
||||
自动赋值要求同名且 **Go 类型一致**（类型别名按 Go 类型系统处理），字段名使用 Go 名称，不按 JSON 标签猜对应关系。int32 → int64、`*Money` → Money、数值 ID → string 都必须显式转换。不会暗中补零、解引用或执行数字强转。
|
||||
|
||||
每个源字段必须被使用或出现在 `ignoreSource`；每个目标字段必须被赋值或出现在 `ignoreTarget`。忽略规则必须给出非空原因，目标忽略明确意味着保留零值；规则引用不存在的字段也会失败。新字段若无法自动匹配，会在生成阶段报错。两边同时添加的同名同类型字段会自动加入生成代码，`--check` 检出未重新生成的差异。
|
||||
|
||||
根类型是项目可导入的具名结构；支持值类型匿名嵌入，规则可用 `Employee.EmployeeName` 这样的实际 Go 字段路径。同名提升歧义、未导出字段和匿名指针嵌入拒绝处理。普通具名嵌套字段使用独立映射或转换函数，不猜测内部结构。泛型声明不能直接作为根；具体字段类型按 Go 类型系统检查。
|
||||
|
||||
配置变化、遗漏映射、类型错误会在写入前失败。映射生成与其他生成产物使用同一套文件所有权机制；手改映射生成文件会被拒绝覆盖。
|
||||
|
||||
### 显式转换函数
|
||||
|
||||
```json
|
||||
"fields": {
|
||||
"Salary": {
|
||||
"source": "Salary",
|
||||
"convert": {"package": "example.com/payroll/internal/convert", "name": "SalaryValue"}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
支持这两种精确签名：
|
||||
|
||||
```go
|
||||
func SalaryValue(source *amount.Money) amount.Money
|
||||
func SalaryValue(source *amount.Money) (amount.Money, error)
|
||||
```
|
||||
|
||||
项目选择一种实现。nil、默认值、金额精度、日期格式和错误码由函数负责；若两边已经是同一个 Money 类型，无需再次解析或序列化。转换函数应仅负责值转换，不执行 SQL 或改变授权范围。
|
||||
|
||||
错误通过 `%w` 加上映射名/目标字段路径，保留 `errors.Is/As` 和框架错误链；返回零目标值，不交付部分完成的结果。包装不主动添加源值或转换后的值。错误是否公开仍由项目错误目录和响应策略决定。
|
||||
|
||||
### 嵌套对象、数组与引用
|
||||
|
||||
`mapping` 指向同配置中的另一条映射，`elements` 明确要求逐项映射：
|
||||
|
||||
```json
|
||||
"fields": {
|
||||
"Summary": {"source": "Summary", "mapping": "ToSummary"},
|
||||
"Items": {"source": "Items", "mapping": "ToItem", "elements": true}
|
||||
}
|
||||
```
|
||||
|
||||
`mapping` 与 `convert` 互斥。`elements` 可以配合其中任意一种，也可以对类型相同的元素直接赋值；支持 slice→slice、等长 array→array，禁止隐式互转。nil slice 保持 nil，非 nil 空 slice 保持非 nil，逐项失败的错误包含索引，循环映射声明被拒绝。
|
||||
|
||||
未选择 elements 的同类型指针、map、slice 使用普通 Go 赋值，**共享引用**，不做隐式深拷贝。elements 会分配新的目标 slice，但元素内的引用仍遵循其映射规则。需要深拷贝时显式提供转换函数。
|
||||
|
||||
映射不保存原 JSON 的字段存在信息。普通 `*T` 在绑定后可能已把缺失和 null 合并为 nil；无法在转换阶段恢复区分。有三态需要的项目必须先使用能携带该信息的输入类型及绑定扩展；没有声明三态时，保持当前输入类型的语义。完整表单更新和局部更新由业务决定，框架不把 nil 自动解释成“保留旧值”。
|
||||
|
||||
### 生成顺序与 CI
|
||||
|
||||
1. 先从 `.api` 生成项目，添加业务结构、scalar 和转换函数，执行 `go mod tidy`。
|
||||
2. 在 `omni.gen.json` 列出映射，执行 `go-omni api generate`。
|
||||
3. Logic 调用生成函数。后续改字段后重复生成，不修改 `*Gen.go`。
|
||||
4. CI 同时运行 `go-omni api generate --check` 与 `go test ./...`。
|
||||
|
||||
启用映射后，CLI 通过 `go list -deps -export` 加载明确声明的包与依赖，按当前构建环境做类型检查，不执行业务代码、init 或转换函数。采用待生成 DTO 的临时 overlay，避免“新增字段还看旧 DTO”。go.mod/go.sum 的副本及 overlay 都在临时目录，check 不修改项目文件；Go 自己的构建/模块缓存可能更新。使用当前 GOOS/GOARCH/GOFLAGS、`GOWORK=off`；依赖用 go.mod 与 replace 表达，生成超时为 90 秒。
|
||||
|
||||
类型包/转换函数包不能依赖输出包 `internal/mapping`；把转换函数放在独立包中，避免导入环。若包本身无法编译，先运行 `go list -deps -export <相关包>` 排查；CLI 不回显可能含私有仓库凭据的 Go 下载输出。纯值字段转换只生成标准 Go 代码，不引入运行时反射搬运、JSON 往返、ORM 或队列依赖。
|
||||
`copyx` 在运行时检查类型和字段，**不提供旧映射生成阶段的静态字段覆盖保证**；CI 应给实际类型对添加测试。普通业务字段新增通常只改 `.api` 和 Logic；独立 DTO/领域类型都需要此字段时，两边各加一次，同名同类型无需追加拷贝配置。
|
||||
|
||||
## 中间件工厂自动注册
|
||||
|
||||
@@ -204,9 +104,13 @@ factory 模式不再在 Init 写 `Middlewares["..."]`；残留非空手写注册
|
||||
| 项目维护 | 生成器持续维护 |
|
||||
| --- | --- |
|
||||
| `.api` 公共字段与接口 | 展开后的 DTO、Handler、OpenAPI |
|
||||
| `omni.gen.json` 的映射与选项 | `internal/mapping/mappingsGen.go` |
|
||||
| Go 业务类型、scalar、转换函数 | 编译期可检查的赋值与调用代码 |
|
||||
| `omni.gen.json` 的 Repository/中间件选项 | 对应骨架和注册代码；已有手写骨架不覆盖 |
|
||||
| 独立 Go 业务类型、scalar、可选 copyx 调用和测试 | 不生成结构转换文件 |
|
||||
| middleware Dependencies、具名工厂、svc 访问器 | 中间件 registry、启动调用 |
|
||||
| Logic、按需 Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 |
|
||||
| Logic、Repository、配置、svc 初始化/清理登记 | main、runtime、路由接入 |
|
||||
|
||||
回归样例位于 `cmd/go-omni/apigen/testdata/composition`，为独立三端服务，不导入任何 eTax 代码。
|
||||
独立三端工资契约测试位于 `cmd/go-omni/apigen/testdata/composition`，不导入任何接入项目。它验证公共字段、Money scalar、三端 Handler、copyx、重新生成保留业务内容，以及新增字段不需要更新转换规则。
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test ./cmd/go-omni/apigen -run TestCopyxThreeEndpointConsumer -count=1
|
||||
```
|
||||
|
||||
46
MODULES.md
46
MODULES.md
@@ -6,6 +6,24 @@
|
||||
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`，中间件依赖
|
||||
`httpx`，不依赖具体服务端网络实现。各模块按需创建、注入 Service，再注册到应用关闭流程。
|
||||
|
||||
## 请求追踪与基础指标
|
||||
|
||||
`omni.New()` 的默认链包含 RequestID、访问日志（可配置开关）和 Recovery；无需重复挂载。业务通过 `c.Context()` 传递 requestId，`logx.FromContext(ctx)` 取得该 App 的 Logger。手动组装 httpx 链时，也可使用 `middleware.RequestID("X-Request-ID")`、`middleware.AccessLog()` 和 `middleware.Recovery()`。
|
||||
|
||||
轻量进程指标直接使用 `metrics`，无需 OTel collector：
|
||||
|
||||
```go
|
||||
// import omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
// import "git.yuntuo.hk/k2safe/go-omni/metrics"
|
||||
app := omni.New()
|
||||
measurements := metrics.New()
|
||||
app.Use(measurements.Middleware())
|
||||
app.GET("/metrics", measurements.Handler())
|
||||
// 也可以直接读取 measurements.Snapshot()。
|
||||
```
|
||||
|
||||
记录请求数、失败数、处理中数量、累计耗时与 HTTP 状态分布；返回的是 JSON 快照，并非 Prometheus 文本格式。指标接口是否暴露以及访问权限由项目配置。需要 HTTP/SQL/Redis spans 和标准 OTel exporter 时，使用 [telemetry 示例](examples/platform/main.go)。需要跨域时配置 `server.cors`，由框架在路由与鉴权前处理预检；默认关闭。已有代理 CORS 可继续使用，避免两处重复设置。见 [CORS 配置与示例](cors/README.md)。
|
||||
|
||||
## 缓存：cache 与 cache/redis
|
||||
|
||||
```go
|
||||
@@ -52,30 +70,26 @@ application.OnShutdown(client.Close)
|
||||
|
||||
```go
|
||||
client, err := httpc.New(httpc.Config{
|
||||
Timeout: 3*time.Second, MaxRetries: 2, MaxConnsPerHost: 32,
|
||||
MaxResponseBytes: 1 << 20,
|
||||
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
|
||||
Timeout: 3*time.Second, DialTimeout: time.Second,
|
||||
TLSHandshakeTimeout: time.Second, ResponseHeaderTimeout: time.Second,
|
||||
MaxConnsPerHost: 32, MaxIdleConnsPerHost: 16, MaxIdleConns: 64,
|
||||
IdleConnTimeout: 60*time.Second, MaxResponseBytes: 1<<20,
|
||||
})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(client.Close)
|
||||
err = client.JSON(ctx, "GET", serviceURL, nil, &result)
|
||||
err = client.JSON(ctx, "GET", serviceURL, nil, &result,
|
||||
httpc.WithHeader("Authorization", "Bearer "+token),
|
||||
)
|
||||
```
|
||||
|
||||
客户端应在服务启动时创建并复用。默认使用独立的 `net/http.Transport` 连接池，每个目标主机
|
||||
有连接上限；该上限不是整个进程的全局连接数上限。也可以注入自有 `http.RoundTripper`，
|
||||
其连接限制和生命周期由调用方负责。这样出站实现同样可以替换。
|
||||
启动时创建并复用 Client。代理可选环境、直连、固定地址；不配置时保留标准库原代理策略。JSON 支持单次请求头和响应限额，不保存 Client 全局 Token。新阶段超时与空闲连接参数可选，0 沿用默认。
|
||||
|
||||
总超时覆盖建连、重试、等待和读取响应体。`Do` 返回后调用方必须读取/关闭 `response.Body`；
|
||||
`JSON` 自动关闭，并限制解码前读取的字节数。`Do` 允许流式读取，本身不限制响应体大小。
|
||||
`Close` 拒绝新请求、关闭自有池的空闲连接；在途请求应先排空。
|
||||
成功响应读取有界；output=nil 流式丢弃，不整体缓存。小错误体只在大小/时间预算内尝试读完以复用连接；未知或大错误体及时关闭。Do 保留标准流式 Request/Response，由调用者限量读取并关闭 body。Close 拒绝新调用并关闭自有池空闲连接，项目先排空在途工作。
|
||||
|
||||
`MaxRetries` 默认 0，最多 5。封装只对可重放的 GET/HEAD/OPTIONS 重试网络错误及
|
||||
429/502/503/504；POST/PUT/PATCH/DELETE 不做应用层自动重试。标准库 Transport 和借用的
|
||||
RoundTripper 自身的重放规则仍生效，不能据此承诺“只执行一次”。退避带抖动，尊重
|
||||
`Retry-After`；服务器要求等待超过 `MaxBackoff` 时直接返回该响应，不提前重试。
|
||||
重定向交给业务处理，3xx 不会自动跟随。
|
||||
自定义 Transport 借用、不修改、不关闭；不能同时指定池/代理/阶段选项。总超时持续覆盖 HTTP 等待、重试和读取 body，但不抢占 JSON 编解码或不遵守 context 的自定义代码。MaxRetries 默认 0，只对可重放 GET/HEAD/OPTIONS 重试；3xx 不跟随。标准库和借用 Transport 自身仍可能重放，不能据此承诺业务只执行一次。
|
||||
|
||||
默认从 context 传递 `X-Request-ID`，日志只包含方法、主机、状态、尝试次数和收到响应头的耗时，
|
||||
不记录完整 URL、查询参数、凭证和响应体。非 2xx 的 JSON 调用返回只带状态码的 `StatusError`。
|
||||
完整参数、NO_PROXY、TLS、安全日志、重试、签名兼容说明见 [httpc 指南](httpc/README.md)，可运行 [HTTP Client 示例](examples/httpClient/README.md)。
|
||||
|
||||
## 请求保护：middleware
|
||||
|
||||
|
||||
18
PROJECT.md
18
PROJECT.md
@@ -1,6 +1,6 @@
|
||||
# 生成项目与文件维护边界
|
||||
|
||||
v0.17.0-preview 提供受管理的生成项目。安装同版远程 CLI，生成的项目直接引用该版本，无需本地 replace。根目录入口支持 `go run .` 和 `go build .`；框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
||||
v0.26.0-preview 提供受管理的生成项目。安装同版远程 CLI，生成的项目直接引用该版本，无需本地 replace。根目录入口支持 `go run .` 和 `go build .`；框架本身仍然是可直接引用的库。安装和私有仓库访问见 [README](README.md#准备环境与安装)。
|
||||
|
||||
## 创建和日常开发
|
||||
|
||||
@@ -38,7 +38,7 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
|
||||
|
||||
这是轻量源码分析，不运行项目 init、不执行 Go 命令、不下载依赖。项目自定义类型别名、嵌入方法或 build tags 等无法可靠判断时，会给 REVIEW 并交给 Go 编译器验证；不把这些情况强行判为签名错误。它不检查方法体是否引用已删除字段，也不证明业务逻辑正确。CI 中应同时执行 `go-omni api generate --check` 和 `go test ./...`，按项目实际构建参数测试。当前版本首次接手缺少快照的 manifest v2 时，会明确提示建立基线，不猜测旧契约；Logic 签名检查仍会执行。
|
||||
|
||||
本版新增生成选项、公共字段组合和映射，详见 [生成能力指南](GENERATION.md)。
|
||||
生成选项、公共字段组合和可选 Go 拷贝工具，详见 [生成能力指南](GENERATION.md)。
|
||||
|
||||
## 哪些文件由用户维护
|
||||
|
||||
@@ -46,7 +46,8 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
|
||||
| --- | --- |
|
||||
| `api/*.api` | 路由、DTO、具名中间件、响应模板、请求/响应模式 |
|
||||
| `etc/config.yaml` | 运行参数；密钥通过环境变量提供 |
|
||||
| `internal/config/config.go` | 自定义配置字段、加载、默认值与校验 |
|
||||
| `internal/config/config.go` | 自定义配置字段、默认值与校验 |
|
||||
| `internal/config/loader.go` | 需要额外文件格式时注册 Decoder |
|
||||
| `internal/svc/serviceContext.go` | 依赖字段和初始化，持有加载后的 Config |
|
||||
| `internal/svc/options.go` | AppOptions、GlobalMiddleware；注册 Go 错误目录和全局扩展 |
|
||||
| `internal/svc/databaseOptions.go` | 可选 MySQL 驱动与项目启动检查扩展 |
|
||||
@@ -54,14 +55,15 @@ ERROR [logicSignature] internal/logic/greet/greet.go:12 (GET /api/greet/:name):
|
||||
| `internal/logic/<group>/*.go` | 业务方法，使用标准 context.Context |
|
||||
| `internal/repository/<group>/*.go` | 数据访问；默认不补空骨架，可选 group 模式 |
|
||||
| `internal/middleware/*Factory.go`、`dependencies.go` | 中间件工厂与依赖小接口，svc 实现访问器 |
|
||||
| `omni.gen.json`、映射转换函数 | 生成选项、显式类型对与差异规则 |
|
||||
| `omni.gen.json` | Repository 和中间件生成选项；不再声明结构映射 |
|
||||
| 可选的 Go 转换函数 / copyx 调用 | 只有独立类型确实需要转换时维护，见 [copyx](COPY.md) |
|
||||
| 项目 Go 错误包 | 自行维护 Catalog 和常量，可跨服务共用 |
|
||||
| 自定义标量文件、`*Bind.go`、`*Manual.go` | 只有显式选择相应扩展时才生成/使用 |
|
||||
| 项目测试、迁移、部署脚本、go.mod/go.sum | 项目所有，生成器不重置其内容 |
|
||||
|
||||
以上骨架只补缺失文件，既有内容按字节保留。删除已废弃接口后，对应业务文件也不会被自动删除；由开发者整理编译器指出的失效类型或签名。
|
||||
|
||||
生成器持续管理：`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、可选映射 `internal/mapping/mappingsGen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误，还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT；手改会被拒绝覆盖，需先移走自定义内容。
|
||||
生成器持续管理：`internal/config/loadGen.go`、`internal/svc/runtimeGen.go`、`main.go`、`internal/handler/*Gen.go`、分组的 `handlersGen.go`、`internal/types/*Gen.go`、中间件 `registryGen.go`、`docs/openapi.json`、`.goOmni/apiManifest.json`。如果在 `.api` 里选择声明错误，还会管理 `internal/apierrors/errorsGen.go`。文件标注 DO NOT EDIT；手改会被拒绝覆盖，需先移走自定义内容。
|
||||
|
||||
Logic 实例在注册时创建并由请求并发复用；请求数据用方法参数和局部变量传递，不存入共享 Logic/svc 字段。
|
||||
|
||||
@@ -113,6 +115,8 @@ c.RequestShutdown(err)
|
||||
|
||||
`Lifecycle *lifecycle.Controller` 是 svc 持有的通用控制器，初始化时创建，生成入口使用它的 Context。该 Context 表示服务正在停止，不代表数据库已经关闭。不能把它当作在途请求的 Context，也不要在它取消时立刻关闭请求仍依赖的资源。发号器失去租约后拒绝发号等业务安全规则仍由项目实现。
|
||||
|
||||
配置加载入口 `internal/config/loadGen.go` 也由生成器维护。业务扩展 Config/Default/Validate；只有新格式才需要修改 NewLoader。JSON/YAML/XML、错误定位和旧 Load 迁移见 [配置指南](CONFIG.md)。
|
||||
|
||||
## 配置内置接口
|
||||
|
||||
生成项目在 `etc/config.yaml` 明确列出以下默认配置；可以独立关闭、修改路径，或让健康接口返回原始 JSON：
|
||||
@@ -139,7 +143,7 @@ server:
|
||||
|
||||
`responseMode: raw` 返回 `{"status":"ready"}` 等健康数据，有依赖时包含检查名称、结果与耗时；不包装公共体。存活检查不执行依赖探针；服务开始退出后，就绪检查返回 503，存活仍可返回 200，直到监听关闭。OpenAPI 始终输出规范文档本身，不使用业务响应包装；关闭 HTTP 文档端点不影响生成 `docs/openapi.json`。
|
||||
|
||||
环境变量覆盖配置文件，例如 `GO_OMNI_ENDPOINTS_READINESS_PATH=/readyz`、`GO_OMNI_ENDPOINTS_READINESS_RESPONSE_MODE=raw`、`GO_OMNI_ENDPOINTS_OPENAPI_ENABLED=false`；liveness/readiness/openapi 都支持对应的 ENABLED 和 PATH。Config/AppOptions 的最终有效配置决定是否注册。
|
||||
内置路由配置由项目文件明确设置，也可以使用 `${NAME:-fallback}` 引用；不再按字段名自动读取环境变量。Config/AppOptions 的最终有效配置决定是否注册。
|
||||
|
||||
关闭或改名后，旧路径不再由框架保留，项目可以在 `.api` 自己声明。启用的内置 GET 路由与业务路由相同或被业务参数/通配路径覆盖时，在创建 App、开始监听之前报错；不会覆盖业务路由。内置接口仍经过全局中间件；项目在鉴权中间件中明确决定这些路径的访问规则，框架不自动绕过鉴权。
|
||||
|
||||
@@ -211,7 +215,7 @@ get /download (DownloadReq)
|
||||
|
||||
## 已有服务接入步骤
|
||||
|
||||
每个独立服务生成一个项目，module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.17.0-preview：
|
||||
每个独立服务生成一个项目，module 名称、上层目录和共享业务包由项目选择。接入时固定 CLI 与框架版本为 v0.26.0-preview：
|
||||
|
||||
1. 先保留旧服务，整理实际路由及现有响应、鉴权和超时要求。在新目录执行 `go-omni new --api /path/to/service.api --module example.com/service ./service`；不对旧入口强制覆盖。
|
||||
2. 在 `.api` 中选择 `@app(defaultResponse: ...)`，显式保留原项目的字段、成功码、错误码和 `dataMode`。需要独立回执的路由单独声明 raw/produces。普通路由、404/405 和提前返回的中间件一起验收。
|
||||
|
||||
216
README.md
216
README.md
@@ -2,22 +2,140 @@
|
||||
|
||||
GoOmni 是面向多个业务项目复用的 Go 后端框架：用 `.api` 定义接口、请求和返回类型，生成项目目录、HTTP 接入代码及 OpenAPI，业务代码负责 Logic 和 Repository。也可以在已有项目中直接使用 `omni.New()`，或单独引入日志、存储等模块。
|
||||
|
||||
**从这份 README 开始接入。** 当前版本 **`v0.17.0-preview`**，新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。
|
||||
**v0.26 改动**：通用 [i18n](i18n/README.md)，提供独立词典、语言匹配、回退、命名参数/条件/复数、可选 HTTP 与错误适配、检查/导出 CLI 及前端运行时。文案由项目维护；[完整示例](examples/i18n/README.md)可直接运行。CLI 与框架使用同版 v0.26.0-preview。同时修复 Go 1.22 下请求体限额错误的识别。
|
||||
|
||||
**本版新增**：公共字段组合、可选结构映射、带依赖的中间件工厂、按需 Repository 骨架，使用与迁移见 [生成能力指南](GENERATION.md)。
|
||||
**从这份 README 开始接入。** 当前版本 **`v0.26.0-preview`**，新生成项目采用根目录入口和 `config / svc / logic / handler`。可以固定版本开展业务接入；文件维护边界和已有服务迁移步骤见 [项目指南](PROJECT.md)，版本变化与验收见 [发布说明](RELEASE.md)。
|
||||
|
||||
**v0.25 改动**：`database.InTx` 返回可识别的事务阶段与结果，保留业务/驱动及回滚错误链；提交报错保守标记为结果不确定，不等同于已回滚。[使用规则](database/README.md#事务阶段与结果)、[可运行故障示例](examples/transactions/README.md)。
|
||||
|
||||
**v0.24 改动**：新增可选 CORS，生成项目通过 `server.cors` 自动接入，支持预检、错误响应和明确的来源白名单；默认关闭。[配置与规则](cors/README.md)、[可运行示例](examples/cors/README.md)。
|
||||
|
||||
**v0.23 改动**：配置校验定位具体字段和安全原因；`httpc.JSON` 可选限制成功状态；`copyx.Check` 无需样本预检类型；`logx` 支持实例独立时区。[四项完整示例](examples/developerTools/README.md)。
|
||||
|
||||
**v0.19 改动**：新增可选 `copyx` 工具，用普通 Go 调用完成浅拷贝、深拷贝和显式类型转换；深拷贝会拒绝自定义转换中残留的源对象可变引用。**移除了 `mappings` / `mappingFiles` 和映射代码生成**，已有使用者须按 [迁移步骤](COPY.md#从旧-mapping-迁移) 调整，不能直接删除仍在调用的旧文件。
|
||||
|
||||
普通接口的流程是：**修改 `.api` → 生成 Request/Response → 写 Logic → 测试**。有独立职责的业务类型才需要转换；可直接写 Go，也可以按需使用 [copyx](COPY.md)。公共字段组合、中间件工厂、Repository 开关继续支持，见 [生成能力指南](GENERATION.md)。
|
||||
|
||||
Go **1.22+**。框架不依赖 Gin，默认使用标准库网络层，提供可切换的 native HTTP/1.1 和实验性 HTTP/2 适配器。
|
||||
|
||||
响应字段、业务码、登录方式、权限规则和数据库表结构由接入项目决定。文中的 `code/msg/data`、`200/9999` 都是示例配置，可以替换。
|
||||
|
||||
本版补齐 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录，以及统一环境变量别名覆盖。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。
|
||||
契约支持 `returns([]string/[]int64/[]bool/[]any)`、OpenAPI 公共引用、workspace 一次导出共享 Go 错误目录，以及文件显式环境引用与配置检查。详细配置与升级步骤见 [业务开发指南](DEVELOPMENT.md)。
|
||||
|
||||
已有通用开发能力见 [业务开发指南](DEVELOPMENT.md)：数组/null、严格请求策略、声明式上传、完整配置环境覆盖、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
|
||||
已有通用开发能力见 [业务开发指南](DEVELOPMENT.md)：数组/null、严格请求策略、声明式上传、文件显式配置引用、资源生命周期、共享契约和升级工具。MySQL 直接使用 [官方适配器](database/mysql/README.md)。
|
||||
|
||||
后台组件可在 svc 登记 [分阶段关闭](lifecycle/README.md#后台组件分阶段关闭)，先停止接收工作并等待收尾，再关闭共享资源；无需绑定队列或 ORM。
|
||||
|
||||
使用 GORM 的项目可以选择 [gormx 适配器](database/gormx/README.md)，复用数据库初始化和连接池，并将 SQL 日志接入 logx。
|
||||
|
||||
## 当前功能与使用入口
|
||||
|
||||
下表是当前公开能力的索引。每项给出调用入口和完整说明；模块按需使用，`omni.New()` 不会自动初始化数据库、Redis、ORM、队列或云存储。示例中的 import 统一以 `git.yuntuo.hk/k2safe/go-omni/` 开头。
|
||||
|
||||
| 功能与用途 | 怎么用 | 解释与示例 |
|
||||
| --- | --- | --- |
|
||||
| 项目、接口与文档生成 | `go-omni new`；改 `.api` 后 `api generate`；`--check` 检查漂移 | [下方快速开始](#新项目生成并运行)、[CLI](cmd/go-omni/README.md)、[完整生成项目](examples/managedService/README.md) |
|
||||
| 公共字段与跨服务共享契约 | `.api` 组合对象字段；`omni.imports.json` 声明共享根；workspace 批量生成 | [公共字段示例](GENERATION.md#公共字段组合)、[共享契约与 workspace](DEVELOPMENT.md) |
|
||||
| 中间件依赖工厂、可选 Repository | `omni.gen.json` 选择 factories / none；在 `New<Name>(deps)` 写一次实现 | [配置与完整工厂示例](GENERATION.md) |
|
||||
| 路由、分组、请求上下文 | `omni.New()`、`app.GET` / `Group` / `Use`；业务使用 `context.Context` | [基本服务](examples/basic/main.go)、[已有项目接入](#已有项目按模块接入) |
|
||||
| HTTP 网络与扩展适配 | 配置 transport 为 nethttp / native / http2；自定义 Router/Runner | [网络层配置与契约](transport/README.md)、[性能与限制](PERFORMANCE.md) |
|
||||
| 浏览器跨域 CORS | `server.cors` / `omni.WithCORS`；独立 HTTP 用 `cors.New(...).Wrap(handler)` | [配置、默认值与协议边界](cors/README.md)、[鉴权与预检示例](examples/cors/README.md) |
|
||||
| HTTP 老接口桥接 | 通过官方 Handler/Middleware 适配器接入标准 `net/http`，跟踪实际响应状态 | [接入说明](MIGRATION.md)、[桥接/上传/自定义 ID 示例](examples/compatibility/main.go) |
|
||||
| 请求校验、动态 JSON、自定义标量 | `.api` 声明来源/校验/scalar；手写契约用 `contract.New[T]`；简单绑定可用 `validate.JSON/Query/URI` | [语法与例子](API.md)、[请求策略](DEVELOPMENT.md) |
|
||||
| 数组、null、原始请求、上传下载 | `.api` 声明数组/可空返回、body/multipart 与大小限制；必要时使用 Bind/Manual 扩展 | [契约示例](examples/contracts)、[含上传/回调/下载的项目](examples/managedService/README.md) |
|
||||
| 公共响应与特殊协议 | `.api` 定义 response 并 `@app(defaultResponse: ...)`；Go 用 `WithResponse` / Encoder | [响应配置](RESPONSE.md)、[响应模板契约](examples/contracts/responses.api) |
|
||||
| 错误目录与动态提示 | `WithErrors(catalog)` 注册一次；`ByCode(...).WithParams(...).WithCause(err)` | [错误目录](errors/README.md)、[模板参数示例](examples/contracts/errorParams.api) |
|
||||
| 通用国际化 | `i18n.New/Render`；可选 `WithLanguagePolicy/WithErrorLocalizer`；`go-omni i18n check/export/types/extract` | [资源格式与边界](i18n/README.md)、[CLI/HTTP/前端示例](examples/i18n/README.md)、[生成契约](examples/contracts/i18n.api) |
|
||||
| OpenAPI 与错误公共引用 | `.api` 自动生成；手写接口可使用 `openapi` 注册器；Go 错误目录显式导出 | [契约工具](DEVELOPMENT.md)、[注册示例](examples/platform/main.go) |
|
||||
| 结构化业务日志 | `logx.Info/Error`、`logx.FromContext(ctx)`、`logx.New`；支持独立时区、轮转、脱敏、慢请求和级别更新 | [下方日志示例](#日志)、[完整配置](logx/README.md) |
|
||||
| 文件配置与环境引用 | Default()→JSON/YAML/XML 字段引用→类型绑定→校验；无隐式覆盖，支持 `-check-config` | [统一配置指南](CONFIG.md)、[完整可运行示例](examples/configuration/README.md)、[动态级别](ADVANCED.md#配置与日志热更新) |
|
||||
| 生命周期与健康检查 | svc 登记 OnClose/OnStop/OnDrain，RequestShutdown 通知故障；健康路径/开关可配 | [生命周期用法](lifecycle/README.md)、[健康检查](health/README.md)、[生成项目扩展](PROJECT.md) |
|
||||
| 数据库、连接池与事务 | `database.Open` / `OpenConnector`；`InTx` 的 `TxError` 提供阶段、已知结果与原因链 | [通用数据库](database/README.md)、[事务失败示例](examples/transactions/README.md)、[MySQL TLS/Logger](database/mysql/README.md) |
|
||||
| 可选 GORM 与迁移 | gormx 复用已有连接池；`go-omni migrate` 预览，显式 apply | [GORM 示例](database/gormx/README.md)、[迁移用法](ADVANCED.md#迁移审计与分页) |
|
||||
| 认证与权限接口 | 注入 Authenticator / Policy；JWT 或 Session 只是可选实现 | [认证说明](auth/README.md)、[鉴权完整例子](examples/platform/README.md) |
|
||||
| 缓存 | `cache.NewMemory` 或 Redis 实现；业务面向 Store 的 Get/Set/Delete | [缓存配置/代码](MODULES.md)、[可运行模块服务](examples/modules/README.md) |
|
||||
| 出站 HTTP、熔断 | `httpc.New` 配置代理、连接池和阶段超时；JSON 支持单次成功状态限制/请求头/限额；`resilience.New` 隔离故障 | [HTTP 完整用法](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[熔断/观测](ADVANCED.md#熔断与观测) |
|
||||
| 请求追踪、访问日志、恢复、限流、幂等、超时 | `app.Use(...)` 按需组合 middleware；跨副本选择 Redis 实现 | [模块说明与限制](MODULES.md#请求追踪与基础指标)、[分布式实现](ADVANCED.md#分布式限流与幂等) |
|
||||
| 本地后台任务与调度 | `task.New`、Submit、Every；有界队列、排空关闭 | [task 示例与生命周期](MODULES.md#后台执行task) |
|
||||
| 持久队列 | 业务使用 Publisher / Consumer；Redis Streams 提供至少一次交付 | [队列初始化/消费示例](ADVANCED.md#可靠队列) |
|
||||
| 文件存储、直传、云端分片 | Store 的 Put/Get/Delete；Local、S3/R2、OSS 按配置装配；扩展接口提供分片续传 | [存储接口与完整例子](storage/README.md)、[本地存储服务](examples/platform/README.md) |
|
||||
| 日志关联、指标与链路 | `metrics.New().Middleware()`；telemetry 注入 OTel exporter/reader | [基础指标](MODULES.md#请求追踪与基础指标)、[OTel 用法](ADVANCED.md#熔断与观测)、[完整接入](examples/platform/main.go) |
|
||||
| 审计与分页 | `audit.New` 注入 Sink/白名单；`pagination.Parse` / `NewResult` | [解释与调用例子](ADVANCED.md#迁移审计与分页) |
|
||||
| 可选结构转换与深拷贝 | `copyx.Copy` / `Clone` / `New`；`Check[S,T]` 预检类型，Go converter 处理差异 | [下方示例](#可选结构转换与深拷贝)、[完整 copyx 指南](COPY.md)、[可运行代码](examples/copying) |
|
||||
| 自定义适配器验收 | `transporttest.Run`、`storagetest.Run` / `RunUploads` 验证扩展契约 | [网络契约测试入口](transport/transporttest/router.go)、[存储契约测试入口](storage/storagetest/contract.go) |
|
||||
|
||||
## 浏览器跨域 CORS
|
||||
|
||||
同源部署保持默认关闭；需要跨域时，在生成项目的配置文件中加入：
|
||||
|
||||
```yaml
|
||||
server:
|
||||
cors:
|
||||
enabled: true
|
||||
allowedOrigins: [https://admin.example.com, https://merchant.example.com]
|
||||
allowedMethods: [GET, HEAD, POST, PUT, PATCH, DELETE]
|
||||
allowedHeaders: [Authorization, Content-Type]
|
||||
exposedHeaders: [Content-Disposition, X-Request-ID]
|
||||
allowCredentials: false
|
||||
maxAge: 10m
|
||||
```
|
||||
|
||||
不需要修改 main、Logic 或 `.api`。框架在鉴权前处理合法预检；实际请求仍鉴权，401/404/405/500 保持项目响应格式并附带允许来源的跨域头。没有 Origin 的服务端回调正常处理。配置只决定浏览器读取权限，不替代登录、验签或 CSRF 防护。
|
||||
|
||||
手写项目用 `omni.WithCORS(omni.CORSConfig{...})`，独立 HTTP 服务用 `cors.New(...).Wrap(handler)`；代理已经处理 CORS 时保持关闭。完整 [默认值与边界](cors/README.md)、[可运行示例](examples/cors/README.md)。
|
||||
|
||||
## 带认证的出站 HTTP
|
||||
|
||||
HTTP Client 在 svc 初始化一次，业务重复使用。普通内部请求可直接选择直连和连接参数，无需手写 Transport：
|
||||
|
||||
```go
|
||||
client, err := httpc.New(httpc.Config{
|
||||
Proxy: httpc.ProxyConfig{Mode: httpc.ProxyDirect},
|
||||
Timeout: 3*time.Second, DialTimeout: time.Second,
|
||||
TLSHandshakeTimeout: time.Second, ResponseHeaderTimeout: time.Second,
|
||||
MaxConnsPerHost: 32, MaxIdleConnsPerHost: 16, MaxIdleConns: 64,
|
||||
})
|
||||
if err != nil { return err }
|
||||
// 在 svc 生命周期登记 client.Close；关闭前先排空使用它的工作。
|
||||
err = client.JSON(ctx, "GET", endpoint, nil, &result,
|
||||
httpc.WithHeader("Authorization", "Bearer "+token),
|
||||
httpc.WithMaxResponseBytes(64<<10),
|
||||
)
|
||||
```
|
||||
|
||||
请求头只作用于本次调用，不把用户 Token 留在共享 Client。默认代理/参数保持原行为；也可选择环境代理或固定代理。JSON 自动关闭并限制响应，Do 保留下载/流式读取扩展。完整配置、重试边界和已有自定义 Transport 的迁移见 [httpc](httpc/README.md)。
|
||||
|
||||
```sh
|
||||
go run ./examples/httpClient
|
||||
```
|
||||
|
||||
## 统一配置加载
|
||||
|
||||
业务只维护 Config、Default()、Validate() 和配置文件。生成器维护 Load/main；文件缺失字段保留默认值，显式 0/false/空字符串正常覆盖，固定值不会被同名环境变量再次覆盖。
|
||||
|
||||
```go
|
||||
BatchSize int `json:"batchSize"` // 加入 Config；在 Default() 中设置 BatchSize: 100。
|
||||
```
|
||||
|
||||
```yaml
|
||||
server:
|
||||
address: "${HTTP_ADDR:-127.0.0.1:8080}"
|
||||
batchSize: "${BATCH_SIZE:-100}"
|
||||
password: "${DB_PASSWORD}"
|
||||
sessionToken: "${SESSION_TOKEN:-}"
|
||||
```
|
||||
|
||||
`${NAME}` 未设置或为空就报错；`${NAME:-value}` 明确指定备用值。`$$` 转义 `$`；密码中的冒号、换行、引号和 `${...}` 原样插入，不二次解释。JSON/YAML/XML 共用解析后的字段处理和 Go 类型绑定，错误同时定位文件、字段、变量及可用行列，不打印原值。
|
||||
|
||||
```sh
|
||||
go run ./examples/configuration -format xml
|
||||
DEMO_CLIENT_TIMEOUT=500ms go run ./examples/configuration
|
||||
# 在生成项目中，仅检查选中的文件，不初始化依赖：
|
||||
go run . -config etc/config.test.yaml -check-config
|
||||
```
|
||||
|
||||
完整 [语义与迁移](CONFIG.md)、[可运行示例](examples/configuration/README.md)。**v0.22 移除旧自动环境覆盖、前缀/别名及 env/envDefault 标签**，生成项目先迁移开发者配置再重新生成；文件中的响应模板 `${code}` 写成 `$${code}`，`.api` 保持原样。
|
||||
|
||||
## 选择接入方式
|
||||
|
||||
| 你的项目 | 从哪里开始 |
|
||||
@@ -38,7 +156,7 @@ module 地址是 `git.yuntuo.hk/k2safe/go-omni`。先确保本机 Go 和 Git 可
|
||||
|
||||
```sh
|
||||
export GOPRIVATE="$(go env GOPRIVATE),git.yuntuo.hk"
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.26.0-preview
|
||||
export PATH="$(go env GOPATH)/bin:$PATH"
|
||||
go-omni --help
|
||||
```
|
||||
@@ -195,7 +313,7 @@ go-omni new --api catalog.api \
|
||||
在业务项目的 **go.mod 所在目录**执行：
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.17.0-preview
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.26.0-preview
|
||||
```
|
||||
|
||||
已有项目可直接复用现有目录、Service、Repository 和数据库连接，先选择一个接口接入。也可以只用 `logx`、`storage` 等包，暂时保留原来的 HTTP 框架。希望入口和 HTTP 接入都交给生成器的项目，按 [已有服务接入步骤](PROJECT.md#已有服务接入步骤) 在新目录生成后迁入业务代码。
|
||||
@@ -389,6 +507,60 @@ return apierrors.TextTooLong("企业名称", 100)
|
||||
|
||||
更详细的配置、作用域和升级步骤见 [统一错误目录](errors/README.md)。
|
||||
|
||||
### 可选结构转换与深拷贝
|
||||
|
||||
普通接口不需要任何拷贝配置，直接使用生成的 Request/Response。独立业务结构之间有转换需求时：
|
||||
|
||||
```go
|
||||
// import "git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
var input DomainInput
|
||||
if err := copyx.Copy(&input, req); err != nil { return err }
|
||||
|
||||
// 源与目标之间的指针、map、slice 需要隔离时：
|
||||
snapshot, err := copyx.Clone(source)
|
||||
if err != nil { return err }
|
||||
```
|
||||
|
||||
默认浅拷贝，同名同类型字段自动赋值，保留零值；默认不会把 nil 填成非可空目标的零值。不要为使用工具新增一套重复模型。
|
||||
|
||||
需要重复使用或自定义规则时创建 Copier：
|
||||
|
||||
```go
|
||||
copier, err := copyx.New(
|
||||
copyx.WithDeepCopy(),
|
||||
copyx.WithFieldNames[Input, View](map[string]string{"Name": "DisplayName"}),
|
||||
copyx.WithConverter(func(v time.Time) (string, error) {
|
||||
return v.UTC().Format(time.RFC3339), nil
|
||||
}),
|
||||
)
|
||||
if err != nil { return err }
|
||||
if err := copier.Copy(&view, input); err != nil { return err }
|
||||
```
|
||||
|
||||
自定义函数的源/目标类型由 Go 泛型确定。深拷贝时返回原指针、原 map、子切片或嵌套旧引用都会报错；字符串/整数原值可以返回。差异规则、nil 策略、字段覆盖、资源限制和原 mapping 移除步骤见 [copyx 指南](COPY.md)，完整类型及可运行代码见 [examples/copying](examples/copying/main.go)。工具使用运行时检查，类型对需测试；深拷贝不适用于连接、锁等资源。
|
||||
|
||||
### 减少接入代码：四个常用入口
|
||||
|
||||
```yaml
|
||||
server:
|
||||
log:
|
||||
timeZone: Asia/Shanghai
|
||||
```
|
||||
|
||||
时区由 Logger 实例管理，不修改全局 `time.Local`。非法日志级别、数据库端口/连接池等配置会指出 `server.log.level`、`database.port` 等字段和原因，不打印输入值。详见 [配置诊断](CONFIG.md#框架模块的字段诊断)、[日志时区](logx/README.md#实例时区)。
|
||||
|
||||
```go
|
||||
// 本次请求只接受 HTTP 200；默认仍接受所有 2xx。
|
||||
err := client.JSON(ctx, http.MethodGet, url, nil, &result,
|
||||
httpc.WithSuccessStatuses(http.StatusOK),
|
||||
)
|
||||
|
||||
// CI/启动时检查类型，无需构造样本；仍使用 Copy 的同一组 Option。
|
||||
err = copyx.Check[Request, BusinessInput](options...)
|
||||
```
|
||||
|
||||
状态不符合要求返回 `*httpc.StatusError`，不会写入 result，响应体仍由框架关闭。Check 不执行 converter；动态 interface、具体 nil/转换失败和深拷贝引用检查有明确边界，见 [类型预检](COPY.md#类型预检)。完整可运行代码见 [developerTools](examples/developerTools/README.md)。
|
||||
|
||||
### 数据库、存储及其他模块
|
||||
|
||||
生成项目在 `internal/svc` 中创建依赖，提供给 Logic/Repository，并用 `c.OnClose(db.Close)` 登记自有资源。生成的 main 在 HTTP 排空后关闭 svc，再关闭日志。直接使用框架的手写项目则可以用 `app.OnShutdown(db.Close)`；不要对同一资源重复登记。
|
||||
@@ -407,20 +579,22 @@ API 项目的数据库迁移需要单独维护并显式执行；启动不会自
|
||||
|
||||
## 配置与运行
|
||||
|
||||
生成项目从 `etc/config.yaml` 读取配置，优先级为 **默认值 → 配置文件 → 环境变量**。密钥通过环境变量提供。手写入口需显式调用配置加载函数并传给 `omni.New(omni.WithConfig(settings))`；单独调用 `omni.New()` 不会自动读取环境变量。
|
||||
生成项目使用 **Default() → 选定文件的显式引用 → 类型绑定 → Validate()**。固定值不会被环境变量覆盖；配置规则、转义及迁移见 [配置指南](CONFIG.md)。手写入口显式加载配置，再传入 `omni.New(omni.WithConfig(settings))`。
|
||||
|
||||
| 环境变量 | 用途 |
|
||||
| 文件字段 | 用途 |
|
||||
| --- | --- |
|
||||
| `GO_OMNI_HTTP_ADDR` | 监听地址；生成项目默认 `127.0.0.1:8080` |
|
||||
| `GO_OMNI_TRANSPORT` | `nethttp`（默认）、`native`、`http2` |
|
||||
| `GO_OMNI_TLS_CERT_FILE` / `GO_OMNI_TLS_KEY_FILE` | 成对设置 TLS 证书和私钥路径 |
|
||||
| `GO_OMNI_HTTP_READ_HEADER_TIMEOUT` / `GO_OMNI_HTTP_READ_TIMEOUT` | 请求头 / 请求读取超时，如 `5s` / `30s` |
|
||||
| `GO_OMNI_HTTP_WRITE_TIMEOUT` / `GO_OMNI_HTTP_IDLE_TIMEOUT` | 响应写入 / Keep-Alive 空闲超时 |
|
||||
| `GO_OMNI_HTTP_MAX_HEADER_BYTES` | 请求头大小限制，字节数 |
|
||||
| `GO_OMNI_SHUTDOWN_TIMEOUT` | 优雅退出等待时间，默认 `10s` |
|
||||
| `GO_OMNI_ENABLE_LOGGER` / `GO_OMNI_LOG_LEVEL` | 访问日志开关、日志级别 |
|
||||
| `GO_OMNI_TRUSTED_PROXIES` | 可解析转发地址头的可信代理 IP/CIDR，逗号分隔 |
|
||||
| `GO_OMNI_DB_HOST/USER/PASSWORD/DATABASE` | 官方 MySQL 适配器的结构化连接配置 |
|
||||
| `server.address` | 监听地址，生成项目默认 127.0.0.1:8080 |
|
||||
| `server.transport` | nethttp（默认）、native、http2 |
|
||||
| `server.tls.certFile/keyFile` | 成对配置 TLS 证书和私钥路径 |
|
||||
| `server.http.readHeaderTimeout/readTimeout` | 请求头 / 请求读取超时 |
|
||||
| `server.http.writeTimeout/idleTimeout` | 响应写入 / Keep-Alive 空闲超时 |
|
||||
| `server.http.maxHeaderBytes` | 请求头大小限制，字节数 |
|
||||
| `server.shutdownTimeout` | 优雅退出等待时间，默认 10s |
|
||||
| `server.enableLogger` / `server.log.level` | 访问日志开关、日志级别 |
|
||||
| `server.trustedProxies` | 可信代理 IP/CIDR，文件数组 |
|
||||
| `database.host/user/password/database` | 官方 MySQL 结构化连接配置 |
|
||||
|
||||
初始生成文件已为地址、网络层和选中模块的必要配置声明环境引用；其他字段按需要添加固定值或引用。
|
||||
|
||||
例如在生成项目中切换 native：
|
||||
|
||||
@@ -465,7 +639,7 @@ go test ./...
|
||||
|
||||
`main.go`、`Gen.go`、`docs/openapi.json` 和 manifest 由生成器管理，手动修改会报告冲突。Logic、Repository、中间件实现、配置和依赖装配属于项目，重复生成保留手写内容。契约变化会提示受影响的 Logic 文件和 DTO 字段，明确的签名错误会给出原签名与期望签名；生成后 `--check` 仍能检出未修复的问题。将 `.api`、`omni.gen.json`、生成文件和 `.goOmni/apiManifest.json` 一起提交，CI 同时运行 `generate --check` 与 `go test ./...`。详见 [契约变更提示与检查边界](PROJECT.md#契约变更提示)。
|
||||
|
||||
CLI 与业务 module 固定使用 `v0.17.0-preview`。v0.14.x 项目升级依赖与 CLI 后重新生成，框架维护的 main/runtimeGen 自动更新；v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理，不直接覆盖手写入口或业务文件。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
||||
CLI 与业务 module 固定使用 `v0.26.0-preview`。v0.21 及更早项目先按 [配置迁移](CONFIG.md#v022-迁移旧模式已移除) 整理旧标签、文件与 Load，再升级依赖与 CLI 并重新生成，框架维护的 main/runtimeGen 自动更新；v0.13 及更早项目按 [升级预览](DEVELOPMENT.md#v013-项目升级) 处理，不直接覆盖手写入口或业务文件。使用过旧 mapping 的项目必须先完成 [移除迁移](COPY.md#从旧-mapping-迁移)，旧配置或旧生成文件会阻止生成。升级后执行生成检查与业务回归。版本变化、迁移边界和验收记录见 [发布说明](RELEASE.md)。
|
||||
|
||||
## 按需深入
|
||||
|
||||
@@ -481,3 +655,7 @@ CLI 与业务 module 固定使用 `v0.17.0-preview`。v0.14.x 项目升级依赖
|
||||
| 网络性能、测试条件与适用范围 | [性能说明](PERFORMANCE.md)、[HTTP/2 验收](transport/native/HTTP2.md) |
|
||||
|
||||
当前版本可用于固定版本接入与业务验证，仍标记为 preview。`.api` 借鉴 go-zero 的接口开发方式，支持范围以本仓库文档为准，不是完整 goctl 语法兼容层；HTTP/2 保持实验状态，OSS/R2 云账号实网验收不属于本次发布范围。
|
||||
|
||||
## 后续开发约定
|
||||
|
||||
每次功能开发完成，同步更新这份功能索引、用法说明、`examples/` 和发布记录；删除或改变功能必须注明影响与迁移。代码、示例和文档一起回归后再发布。约定已记录在 [AGENTS.md](AGENTS.md)，后续开发继续遵循。
|
||||
|
||||
225
RELEASE.md
225
RELEASE.md
@@ -1,4 +1,223 @@
|
||||
# v0.17.0-preview
|
||||
# v0.26.0-preview
|
||||
|
||||
新增可独立使用的 i18n 底层模块及可选 HTTP/错误适配。项目维护文案和语言设置，框架提供不可变词典、语言匹配、回退、参数与分支、检查和导出；没有绑定业务身份、页面框架、数据库或响应字段名。
|
||||
|
||||
前后端共享版本化消息资源和固定 Unicode CLDR 46.1 基数复数规则。CLI 只读输入，支持源目录提取、缺译/参数/动态引用检查、按命名空间导出语言包和 TypeScript 类型。可选浏览器模块先加载并验证再切换，失败保留当前语言，旧请求不能覆盖新选择。
|
||||
|
||||
错误目录新增可选 `translation` 引用。内置响应使用展示译文，原目录文案继续参与契约校验；自定义 Encoder 通过 `PublicMessage()` 显式采用译文。业务码、HTTP 状态、成功/失败语义、cause、直接协议输出和幂等首次快照保持。旧配置与生成项目不启用时无需迁移。
|
||||
|
||||
完整 [使用指南](i18n/README.md)、[通用示例](examples/i18n/README.md)、[验收记录](reports/i18nAcceptance.md)。包含 translation 的新契约远程依赖最低版本为 v0.26.0-preview；未使用该元数据的模板最低版本仍为 v0.22。CLI 与框架使用同版。
|
||||
|
||||
同时修复 JSON 结束定界符读取失败时丢失原始错误的问题，使请求体超过外层限制时在 Go 1.22 中也保持 HTTP 413；共享目录测试工程使用 Go 1.22。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.26.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.26.0-preview
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# v0.25.0-preview
|
||||
|
||||
事务失败可以按阶段和已知结果处理，避免把“提交错误”误判为“数据库没有保存”。
|
||||
|
||||
- `database.InTx` 签名不变，普通错误返回 `*database.TxError`；Stage 为 begin/work/commit，Outcome 为 notStarted/rolledBack/unknown。成功提交仍返回 nil。
|
||||
- Commit 报错一律 unknown；回调失败且显式 Rollback 返回 nil 才标记 rolledBack。BeginTx 失败不执行回调。ErrTxDone 不作为回滚成功证据，框架不重试事务。
|
||||
- `Err` 与 `RollbackErr` 保留两条原因链，可通过 errors.Is/As 识别；APIError、目录参数提示和公开响应继续工作。内部原因不自动进入 HTTP 提示。
|
||||
- 提供无需外部数据库的 [故障演示](examples/transactions/README.md)。真实 MySQL 与性能结果见 [验收记录](reports/transactionAcceptance.md)。不引入新依赖，不包含文件删除、补偿任务或业务规则。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.25.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.25.0-preview
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
**兼容边界：** 既有 errors.Is/As 保持；直接 `err == sentinel`、直接类型断言或字符串匹配需调整。nil context 使用 Background，nil 回调在开启前返回 typed error。panic/Goexit 继续传播并尝试回滚，不转换为 error；标准库自动回滚不公开的原因无法还原，不能推断已回滚。结果仅覆盖按约定使用的当前事务，不覆盖非事务表、隐式提交 SQL、其他连接或外部副作用。GORM 原生 Transaction 没有被替换。
|
||||
|
||||
[database 使用规则](database/README.md#事务阶段与结果) 已同步。本轮 runtime 能力需要 v0.25，模板最低版本仍为 v0.22；没有新增必填配置，当前仍为 preview。
|
||||
|
||||
49 个测试包全量 race、Go 1.22、vet/build、取消竞争复测通过；成功事务没有新增分配。固定远程版本在空缓存下通过 77 项发布检查和 9 组生成消费者，耗时 71.93 秒。独立 MySQL TLS 容器的 53 条测试记录（含子用例）通过，包含真实提交/回滚后的确认丢失、可选 GORM、生成项目和后台关闭，耗时 28.49 秒。均无本地 replace，标签固定在 `7a177744ab2bb85f23efc0ec267fd905d349bd4b`。见 [远程发布记录](reports/transactionRemote.json)、[远程 MySQL 记录](reports/transactionMySQLRemote.json)。
|
||||
|
||||
---
|
||||
|
||||
# v0.24.0-preview（历史版本）
|
||||
|
||||
业务项目通过配置开启跨域，不需要手写 OPTIONS Handler 或逐条修改接口。
|
||||
|
||||
- 默认关闭，在 `server.cors` 显式配置来源、方法和请求头；统一文件加载、环境引用和字段级诊断自动生效。生成项目无需修改 main、svc 或 `.api`。
|
||||
- 预检在鉴权前返回 204/403 空响应；实际请求继续鉴权、业务错误编码和路由处理。允许来源可读取 401/403/404/405/500，上传下载和流式能力保留。
|
||||
- 只支持精确 HTTP(S) 来源，公开无凭据服务可配置单一 `*`；禁止 `*` 与凭据混用。配置初始化快照、请求并发隔离、Vary 缓存分流和有界请求头处理。
|
||||
- 同一个 App 策略覆盖标准库、native 和 HTTP2；独立服务可使用 `cors.New(...).Wrap(handler)`。没有新增第三方依赖，没有移除现有功能。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.24.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.24.0-preview
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go run . -config etc/config.yaml -check-config
|
||||
```
|
||||
|
||||
现有项目不配置 CORS 时行为不变。代理已处理 CORS 的项目继续关闭；跨域不替代登录、权限或 CSRF 防护。通过 App/Engine/Run 接入，直接对外暴露底层 Router 会绕过应用外层策略。本轮新增 runtime 配置需 v0.24，未使用该配置的模板最低版本仍为 v0.22。
|
||||
|
||||
[配置与规则](cors/README.md)、[可运行示例](examples/cors/README.md)、[验收与性能](reports/corsAcceptance.md) 已同步。49 个测试包全量 race、Go 1.22、vet/build、实际生成项目、三种 transport 的 TLS 场景和 20 项真实浏览器检查通过；30 秒 fuzz 约 288 万次。当前仍为 preview。
|
||||
|
||||
空缓存安装远程版本后，9 组生成消费者和 70 项发布检查通过，耗时 66.78 秒，无本地 replace；包括下载模块的 cors/app race、三种 transport 的 TLS 检查和只改配置的生成项目。标签固定在 `800974cfff6228c2697373d9961d3c7c75611d08`，见 [远程记录](reports/corsRemote.json)。
|
||||
|
||||
---
|
||||
|
||||
# v0.23.0-preview（历史版本）
|
||||
|
||||
减少业务接入时重复维护的配置校验、HTTP 包装和类型检查。
|
||||
|
||||
- 配置诊断定位 database.port、database.pool.maxIdleConns、server.log.level/timeZone 等具体字段，保留文件、环境变量、行列和安全原因；凭据不进入公开提示，原始原因可 errors.Is/As。
|
||||
- JSON 可传 `httpc.WithSuccessStatuses(200)`，默认仍接受全部 2xx；不匹配时不改目标，继续负责有界读取、关闭和连接回收，不添加新的重试规则。
|
||||
- `copyx.Check[S,T](options...)` 或已有 Copier.Check 复用同一组 Go 选项，无需样本和映射文件；实际拷贝不再因集合为空而漏过确定的类型不匹配。动态 interface、nil 值、converter 失败及深拷贝共享引用仍有运行时检查。
|
||||
- `server.log.timeZone: Asia/Shanghai` 配置实例时区；JSON/text/子日志一致，业务 time.Time 属性保持原样。极简镜像可用 `go build -tags timetzdata .` 内置时区数据，库不强制增加体积。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.23.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.23.0-preview
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go run . -config etc/config.yaml -check-config
|
||||
```
|
||||
|
||||
现有调用接口保留；配置错误文本现在更精确，业务应通过类型/字段识别错误。空集合里确定的类型错误会提前暴露，需要显式 converter 或调整实际类型，不应靠空值跳过校验。预检通过不代表 converter 对所有业务值必定成功。当前模板最低版本仍是 v0.22，本轮新增 runtime 能力需使用 v0.23。
|
||||
|
||||
[完整示例](examples/developerTools/README.md)、[配置](CONFIG.md)、[JSON 选项](httpc/README.md)、[类型预检](COPY.md#类型预检)、[日志时区](logx/README.md#实例时区)。验收与性能记录见 [本轮验收](reports/developerToolsAcceptance.md)。仍为 preview。
|
||||
|
||||
48 个测试包全量 race、Go 1.22、vet/build、实际生成消费者及约 392 万次 copyx fuzz 通过。空缓存安装远程版本后，66 项发布检查通过，包括下载模块的 config/httpc/copyx/logx race；耗时 57.06 秒，无本地 replace。标签固定在 `3bbc6f60d76cb9eae6f01fa6ca122eb1fbf481af`，见 [远程验收](reports/developerToolsRemote.json)。
|
||||
|
||||
---
|
||||
|
||||
# v0.22.0-preview（历史版本）
|
||||
|
||||
配置文件明确决定每个值的来源，项目只维护 Config、Default()、Validate() 和文件。
|
||||
|
||||
- `${NAME}` 必填，`${NAME:-fallback}` 在未设置/空字符串时使用备用值，`$$` 转义 `$`。支持字符串组合；环境值不二次展开，凭据空格和特殊字符原样保留。
|
||||
- 在 JSON/YAML/XML 已解析的字段值上展开，再按目标 Go 类型绑定；缺失字段保留默认值，0/false/空字符串可显式覆盖。字段错误定位文件、环境名和可用行列，不回显值。
|
||||
- 生成 Load/main 自动接入；新增 `-check-config`，选定一个文件检查，成功退出，不启动日志/依赖/监听。精简生成配置，重新生成保留业务 Config、Default、Validate 和文件。
|
||||
- **不兼容变更：旧自动环境覆盖、前缀/别名、env/envDefault、database.ConfigFromEnv/ApplyEnv 已移除。** LoadFile/NewReloader 不再接收 prefix，omni.LoadConfig 参数改为文件路径。发现旧标签提示迁移。
|
||||
- 配置文件中的响应模板用 `$${code}` 等转义；`.api` 中的 `${code}` 不变。日志 file/output/rotation 均由显式文件配置决定，不再隐式清除。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.22.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.22.0-preview
|
||||
# 按 CONFIG.md 迁移开发者配置后：
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go run . -config etc/config.yaml -check-config
|
||||
```
|
||||
|
||||
[完整规则与迁移](CONFIG.md)、[可运行示例](examples/configuration/README.md)、[场景与性能验收](reports/explicitConfigurationAcceptance.md) 已同步。48 个测试包全量 race、实际生成消费者、Go 1.22、vet/build 和约 28 万次引用 fuzz 通过；小配置固定值/带引用加载约 25 µs，属于本机启动阶段基准。当前仍为 preview。
|
||||
|
||||
空缓存远程 CLI 安装、下载模块 config/httpc race、8 组实际生成消费者与 `-check-config` 验收通过，无本地 replace。标签固定在 `d598b2ebee8cde36859021e882a6bc1e0bb4fd4c`；见 [远程验收记录](reports/explicitConfigurationRemote.json)。
|
||||
|
||||
---
|
||||
|
||||
# v0.21.0-preview（历史版本）
|
||||
|
||||
让带认证的 HTTP 调用和内部直连可以直接使用 httpc，减少手写 Transport 与 JSON 响应处理。
|
||||
|
||||
- Proxy 支持环境、直连、固定代理；默认保留既有策略。
|
||||
- 建连、TCP keepalive、TLS 握手、响应头、100-continue、空闲超时和独立空闲连接数量可选配置；不修改全局 Transport，不关闭借用实例。
|
||||
- JSON 支持 WithHeader / WithHeaders / WithMaxResponseBytes；认证信息逐请求隔离，自动关闭和有界响应处理保留。
|
||||
- 成功路径减少请求复制和无用退避；output=nil 按限额流式丢弃；小错误体在大小和时间预算内尝试复用连接。
|
||||
- 不新增依赖、不增加业务规则；Do 保留标准流式扩展。CLI 最新版本与模板最低版本分开，本轮模板兼容门槛仍是 v0.20。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.21.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.21.0-preview
|
||||
```
|
||||
|
||||
**兼容注意**：JSON 旧的直接调用仍有效，自定义 interface/函数值需要同步新增的 variadic 参数。自定义 Transport 与显式连接/代理/阶段参数混用现在报错，避免参数被忽略；迁移时在自己的 Transport 配置，或改用内置字段。没有删除原调用入口。
|
||||
|
||||
[参数与使用说明](httpc/README.md)、[可运行示例](examples/httpClient/README.md)、[场景与性能验收](reports/httpcAcceptance.md) 已同步到 README。48 个测试包全量 race、Go 1.22 兼容、vet/build、约 110 万次请求头 fuzz 通过。性能来自本机同条件基准，不能代替项目实际流量验证；仍为 preview。
|
||||
|
||||
空缓存远程安装、下载模块的 httpc race 与示例、8 组生成消费者验收通过，无本地 replace。标签固定在 `96e7d9629b5c8b78e9cfefbcb05da0ab702c242b`，见 [远程记录](reports/httpcRemote.json)。
|
||||
|
||||
---
|
||||
|
||||
# v0.20.0-preview（历史版本）
|
||||
|
||||
统一配置加载，减少生成项目重复维护的启动代码。
|
||||
|
||||
- 生成器维护 `internal/config/loadGen.go`，调用 `config.LoadProject`；项目只扩展 Config/Default/Validate，可选在 loader.go/NewLoader 注册新格式。入口、环境覆盖、日志输出规范化无需业务重复实现。
|
||||
- JSON/YAML/XML 使用同一个加载与绑定流程；Decoder 通过实例注册，无全局 init 或额外强制依赖。内置格式、字段、文件大小与层数均有明确限制。
|
||||
- `config.Error` 携带文件、字段、环境变量、可用行列及安全原因；`Invalid` / `AtField` 支持业务验证。保留内部错误链，不自动输出原值；大整数、空集合、null、默认值隔离和自定义反序列化有回归覆盖。
|
||||
- 配置解析在日志、依赖和监听初始化之前；生成/check/dry-run 使用一致入口和文件所有权规则。
|
||||
- **迁移要求：旧项目先整理手写 Load，再重新生成。** 不静默覆盖业务文件。低层配置 API 保留；新项目不再复制手写加载流程。固定 Go 数组严格检查长度，YAML alias/merge/custom tag 不再接受，避免模糊覆盖和扩展行为。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.20.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.20.0-preview
|
||||
# 先按 CONFIG.md 迁移旧 Load，再执行：
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
go build .
|
||||
```
|
||||
|
||||
完整用法、格式、脱敏边界和迁移见 [统一配置指南](CONFIG.md)，可运行例子见 [examples/configuration](examples/configuration/README.md)。回归、模糊测试与启动加载性能见 [验收记录](reports/configurationAcceptance.md)。47 个测试包的全量 race、Go 1.22 兼容、vet/build、30 秒 fuzz 与 5 个基准场景三轮测量通过。空缓存远程安装和 8 组生成消费者回归通过，无本地 replace；标签固定在 `c60841f901a3bcc3e9582eece6cfb2cad008fb9e`，见 [远程记录](reports/configurationRemote.json)。仍为 preview；项目的实际配置和业务依赖应自行验收。
|
||||
|
||||
---
|
||||
|
||||
# v0.19.0-preview（历史版本）
|
||||
|
||||
将可选结构转换改为直接 Go 调用，减少业务项目需要维护的配置。普通接口继续使用 `.api` 生成 DTO，不要求增加独立模型或任何拷贝规则。
|
||||
|
||||
- 新增只依赖标准库的 `copyx`：Copy、Clone、可复用 Copier；浅/深拷贝、严格字段覆盖、nil/零值策略、字段改名、忽略字段、精确类型 converter。没有全局注册和隐式数字强转。
|
||||
- 深拷贝检查整个输入和输出图。自定义 converter 原样返回指针、map、slice，或新对象仍嵌套旧引用时返回错误；保留普通循环图，拒绝资源对象及超出遍历/估算分配预算的输入。
|
||||
- 转换失败不提交部分目标；保留错误路径和原始错误链。并发复用只共享不可变类型元数据；源/目标读写和 converter 副作用由调用方负责。
|
||||
- **不兼容变更：删除 mappings / mappingFiles、Go 类型加载与映射代码生成。** 检出旧配置/旧生成文件时给出迁移提示；不会静默删除业务正在使用的代码。公共字段组合、scalar、中间件工厂、Repository 选项保留。
|
||||
- README 补齐当前能力入口、解释和用法，新增 [完整 copyx 指南](COPY.md)、[可运行示例](examples/copying)；[AGENTS.md](AGENTS.md) 记录每次修改同步文档/示例/移除说明的约定。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.19.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.19.0-preview
|
||||
# 如用过旧 mapping，先按 COPY.md 完成迁移：
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
迁移步骤和语义边界见 [copyx](COPY.md#从旧-mapping-迁移)。新工具做运行时检查，不能声称具备旧生成器的编译期字段覆盖保证；实际类型对必须有测试。独立三端工资示例验证金额精度、缺失/null/零值、空集合、错误链和字段新增；不修改或代替接入项目的完整业务回归。
|
||||
|
||||
46 个测试包的全量 race（含实际生成项目）、vet/build、Go 1.22 兼容测试通过；30 秒 fuzz 无失败，14 种基准场景各测量三轮。验收条件、性能数据及本轮未覆盖的外部环境见 [v0.19 验收记录](reports/copyxAcceptance.md)。本版仍为 preview。 空缓存远程 CLI 与 7 组真实生成消费者验收通过，无本地 replace；标签固定在 `fd83235ce7fc15ce88a482d385d70e61f25d7b32`，详见 [远程记录](reports/copyxRemote.json)。
|
||||
|
||||
---
|
||||
|
||||
以下是历史版本记录，其中 mapping 功能已在 v0.19 移除，不应继续作为当前接入方式。
|
||||
|
||||
# v0.18.0-preview（历史版本）
|
||||
|
||||
补齐可空嵌套映射和按模块拆分配置，减少重复转换代码，业务空值规则仍由项目显式选择：
|
||||
|
||||
- 嵌套 mapping 支持 `*S → *T` 保留 nil、`S → *T`，以及指定 nil zero/error 后的 `*S → T`；同样适用于 slice/等长数组元素。错误保留字段路径、索引和内部错误链，失败不返回部分结果。
|
||||
- `mappingFiles` 显式引用项目内的映射文件，支持跨文件向前引用；统一检查重名、循环、类型与字段覆盖。禁止多层导入、路径越界和隐式覆盖，生成期间核对全部配置输入。
|
||||
- 同类型 `*T → T` 可显式配置 nil zero/error；现有 converter 保持完整控制，不默认放宽类型或混入领域规则。
|
||||
- `pointerResult: true` 额外生成 `<Name>Ptr(S) (*T,error)`，原值函数保留；失败返回 nil 与原错误，辅助函数名冲突在生成前拒绝。
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.18.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.18.0-preview
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
v0.17 的单文件 mappings、现有 converter 和业务文件继续保留。配置拆分时移动完整映射条目，主配置显式列出子文件；不保留重复定义。新能力均为可选生成配置，不增加运行时依赖。nil zero 表示 Go 零值并跳过子映射；业务必填、金额精度、编号格式和快照复制仍由项目负责。
|
||||
|
||||
使用与迁移见 [生成能力指南](GENERATION.md)，回归和发布后验收见 [验收记录](reports/nullableMappingAcceptance.md)。本版仍为 preview。
|
||||
|
||||
全量 44 个测试包的 race、vet、build 和实际进程验收通过；空缓存远程 CLI 与 8 组实际生成消费者回归也通过，同版依赖且无本地 replace。标签固定在 `4563a6354fac19f22cd8d407fd76d98b8cc9c406`。
|
||||
|
||||
---
|
||||
|
||||
# v0.17.0-preview（历史版本）
|
||||
|
||||
减少项目里的重复声明、字段赋值和中间件装配代码，业务规则继续由项目控制：
|
||||
|
||||
@@ -19,6 +238,8 @@ go test ./...
|
||||
|
||||
使用方法、文件归属和升级步骤见 [生成能力指南](GENERATION.md)，回归与发布验收见 [验收记录](reports/generationExperienceAcceptance.md)。本版仍为 preview；框架验收不等于业务项目完成迁移。
|
||||
|
||||
全量 44 个测试包的 race、vet、build 和实际进程验收通过；发布后的空缓存远程 CLI 与 7 组生成消费者回归也通过，同版依赖且无本地 replace。标签固定在 `8b588a24cd653dac77dd68f6777e0db367981f19`。
|
||||
|
||||
---
|
||||
|
||||
# v0.16.0-preview（历史版本）
|
||||
@@ -38,7 +259,7 @@ go-omni api generate --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
旧项目 config/Logic/svc 继续由开发者维护，不会被重新生成覆盖。已有环境配置需按 [一次覆盖完整配置](DEVELOPMENT.md#一次覆盖完整配置与-http日志短名称) 合并 OverlayEnv 与 ApplyEnv；共享错误目录接入见 [workspace 配置](DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。已有手写权限 Handler 切换自动生成前，先迁出实现并显式移除 Manual 文件，生成器不会静默删除。
|
||||
旧项目 config/Logic/svc 继续由开发者维护，不会被重新生成覆盖。已有环境配置需按 本版当时的统一覆盖方式合并 OverlayEnv 与 ApplyEnv（该模式已在 v0.22 移除，当前升级见 [配置迁移](CONFIG.md#v022-迁移旧模式已移除)）；共享错误目录接入见 [workspace 配置](DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。已有手写权限 Handler 切换自动生成前，先迁出实现并显式移除 Manual 文件，生成器不会静默删除。
|
||||
|
||||
全量 race（含实际生成消费者）、vet、build、根入口/退出/再生成验证通过。eTax 只读契约验收：API 文档 20,751,149 → 847,171 字节，ID 文档 1,008,210 → 392,481 字节；70 个操作展开引用后与原文档完全一致。空缓存远程 CLI / 固定版本消费者验收也已通过，没有本地 replace；标签固定在 `64e3bd6cfced225476d58e95037249cd0a4fe695`。详见 [验收记录](reports/contractFollowupAcceptance.md)。这不代表已完成业务项目迁移。
|
||||
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
# 公共响应体与独立接口协议
|
||||
|
||||
v0.26 新增的 [国际化适配](i18n/README.md#接入-go-omni-错误与-http)保留现有响应契约。`Result.Message` 是校验使用的目录原文；`MessageOverride` 是可选展示译文，`PublicMessage()` 返回最终文案。内置 Formatter 自动使用译文；自定义 Encoder 显式选择，固定文本/XML 回执和直接输出不自动改写。
|
||||
|
||||
这些功能从 `v0.8.0-preview` 提供。CLI 与项目框架依赖请固定到同一版本；源码开发可显式使用 `--framework-dir`。`v0.7.0-preview.1` 不包含响应模板声明。
|
||||
|
||||
go-omni 的响应编码由应用或接口配置。没有配置时保留旧版默认值；业务可以定义自己的字段、嵌套结构、错误码类型、提示和空值行为。`response.Response` 仅保留为兼容 DTO，运行时与生成器不再通过它固定响应结构。
|
||||
|
||||
@@ -1,5 +1,8 @@
|
||||
# GoOmni 成熟度计划
|
||||
|
||||
当前发布基线为 v0.19.0-preview：普通接口通过 `.api` → DTO → Logic 开发；独立类型需要转换时可选 copyx。旧 JSON mapping 生成已移除，见 [迁移说明](COPY.md)。当前能力、使用入口和限制以 [README](README.md) 与 [发布记录](RELEASE.md) 为准。下文保留阶段性建设记录。
|
||||
|
||||
|
||||
当前进展：`v0.14.0-preview` 已完成契约驱动的业务开发能力、通用 Runtime/配置和官方 MySQL 适配器，并通过全量 race、实际生成服务及空缓存远程验收。新项目/升级以 [README](README.md) 和 [开发指南](DEVELOPMENT.md) 为准；下方为历史阶段记录，完整本轮验收见 [报告](reports/businessDevelopmentAcceptance.md)。具体 eTax/ePay 业务迁移与业务回归仍由接入项目完成。
|
||||
|
||||
## 1. 新项目启动与发布流程（已完成，v0.1.1）
|
||||
|
||||
18
app/app.go
18
app/app.go
@@ -12,6 +12,7 @@ import (
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
@@ -24,6 +25,7 @@ import (
|
||||
// httpx.Router, leaving the network implementation replaceable.
|
||||
type App struct {
|
||||
router httpx.Router
|
||||
handler http.Handler
|
||||
config config.Config
|
||||
middlewares []httpx.Middleware
|
||||
closers []func() error
|
||||
@@ -67,6 +69,11 @@ func configure(options ...config.Option) config.Config {
|
||||
}
|
||||
|
||||
func newApp(router httpx.Router, settings config.Config) *App {
|
||||
settings.CORS = settings.CORS.Clone()
|
||||
corsPolicy, corsErr := cors.New(settings.CORS)
|
||||
if corsErr != nil {
|
||||
panic(config.AtField("cors", corsErr))
|
||||
}
|
||||
settings.Response = settings.Response.Clone()
|
||||
settings.Errors = settings.Errors.Clone()
|
||||
encoder, responseErr := response.New(settings.Response)
|
||||
@@ -93,6 +100,7 @@ func newApp(router httpx.Router, settings config.Config) *App {
|
||||
}
|
||||
middlewares = append(middlewares, middleware.Recovery())
|
||||
application := &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver, responseEncoder: responseEncoder}
|
||||
application.handler = corsPolicy.Wrap(router)
|
||||
if fallback, ok := router.(httpx.FallbackRouter); ok {
|
||||
adaptError := func(status int, code, message string) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -104,21 +112,23 @@ func newApp(router httpx.Router, settings config.Config) *App {
|
||||
return application
|
||||
}
|
||||
|
||||
// Router returns the transport contract used by this application.
|
||||
// Router exposes the underlying routing contract for advanced registration.
|
||||
// Serve requests through App or Engine so application CORS policy is applied.
|
||||
func (a *App) Router() httpx.Router { return a.router }
|
||||
|
||||
// Engine is kept as a compatibility name for the old API. It returns a
|
||||
// standard http.Handler, so callers do not depend on a router type.
|
||||
func (a *App) Engine() http.Handler { return a.router }
|
||||
func (a *App) Engine() http.Handler { return a.handler }
|
||||
|
||||
// ServeHTTP implements http.Handler for standard servers and HTTP tests.
|
||||
func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
|
||||
a.router.ServeHTTP(writer, request)
|
||||
a.handler.ServeHTTP(writer, request)
|
||||
}
|
||||
|
||||
// Config returns the effective application configuration.
|
||||
func (a *App) Config() config.Config {
|
||||
settings := a.config
|
||||
settings.CORS = settings.CORS.Clone()
|
||||
settings.Response = settings.Response.Clone()
|
||||
settings.Errors = settings.Errors.Clone()
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
@@ -177,7 +187,7 @@ func (a *App) NewServer(address string) *http.Server {
|
||||
}
|
||||
}
|
||||
cfg := a.config.HTTP
|
||||
return &http.Server{Addr: address, Handler: a.router,
|
||||
return &http.Server{Addr: address, Handler: a.handler,
|
||||
ReadHeaderTimeout: cfg.ReadHeaderTimeout, ReadTimeout: cfg.ReadTimeout,
|
||||
WriteTimeout: cfg.WriteTimeout, IdleTimeout: cfg.IdleTimeout,
|
||||
MaxHeaderBytes: cfg.MaxHeaderBytes, ErrorLog: cfg.ErrorLog}
|
||||
|
||||
187
app/cors_test.go
Normal file
187
app/cors_test.go
Normal file
@@ -0,0 +1,187 @@
|
||||
package app_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
func corsApp() *omni.App {
|
||||
return omni.New(omni.WithCORS(omni.CORSConfig{Enabled: true, AllowedOrigins: []string{"https://web.example"}, AllowedMethods: []string{"GET", "POST", "PUT"}, AllowedHeaders: []string{"Authorization", "Content-Type"}, ExposedHeaders: []string{"Content-Disposition"}}), omni.WithResponse(response.Config{CodeType: "int32", SuccessCode: 0, ErrorCode: 999, MessageField: "msg", DataMode: "always"}))
|
||||
}
|
||||
func TestAppCORSRoutingAndErrors(t *testing.T) {
|
||||
a := corsApp()
|
||||
defer a.Close()
|
||||
var middlewareCalls int
|
||||
a.Use(func(next omni.Handler) omni.Handler {
|
||||
return func(c *omni.Context) error { middlewareCalls++; return next(c) }
|
||||
})
|
||||
a.GET("/secure", func(c *omni.Context) error { return c.OK("private") }, func(next omni.Handler) omni.Handler {
|
||||
return func(c *omni.Context) error { return omni.Unauthorized("LOGIN_REQUIRED", "login required") }
|
||||
})
|
||||
a.GET("/failure", func(c *omni.Context) error { return omni.Internal(errors.New("private detail")) })
|
||||
a.GET("/panic", func(c *omni.Context) error { panic("private panic") })
|
||||
a.Handle("OPTIONS", "/options", func(c *omni.Context) error { return c.Text(200, "ordinary options") })
|
||||
for name, entry := range map[string]http.Handler{"App": a, "Engine": a.Engine(), "NewServer": a.NewServer("").Handler} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
method, path string
|
||||
status int
|
||||
}{{"GET", "/secure", 401}, {"GET", "/failure", 500}, {"GET", "/panic", 500}, {"GET", "/absent", 404}, {"PUT", "/secure", 405}} {
|
||||
r := httptest.NewRequest(tc.method, tc.path, nil)
|
||||
r.Header.Set("Origin", "https://web.example")
|
||||
w := httptest.NewRecorder()
|
||||
entry.ServeHTTP(w, r)
|
||||
var body map[string]any
|
||||
err := json.Unmarshal(w.Body.Bytes(), &body)
|
||||
if err != nil || w.Code != tc.status || w.Header().Get("Access-Control-Allow-Origin") != "https://web.example" || body["code"] != float64(999) || body["data"] != nil || body["msg"] == nil || strings.Contains(w.Body.String(), "private") {
|
||||
t.Fatal(w.Code, w.Header(), w.Body.String(), err)
|
||||
}
|
||||
}
|
||||
for _, path := range []string{"/secure", "/absent"} {
|
||||
before := middlewareCalls
|
||||
r := httptest.NewRequest("OPTIONS", path, nil)
|
||||
r.Header.Set("Origin", "https://web.example")
|
||||
r.Header.Set("Access-Control-Request-Method", "POST")
|
||||
r.Header.Set("Access-Control-Request-Headers", "authorization,content-type")
|
||||
w := httptest.NewRecorder()
|
||||
entry.ServeHTTP(w, r)
|
||||
if w.Code != 204 || w.Body.Len() != 0 || middlewareCalls != before {
|
||||
t.Fatal(w.Code, w.Body.String(), middlewareCalls, before)
|
||||
}
|
||||
}
|
||||
r := httptest.NewRequest("OPTIONS", "/options", nil)
|
||||
w := httptest.NewRecorder()
|
||||
entry.ServeHTTP(w, r)
|
||||
if w.Code != 200 || w.Body.String() != "ordinary options" {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppCORSConfigIsolation(t *testing.T) {
|
||||
cfg := config.Default()
|
||||
cfg.CORS = omni.CORSConfig{Enabled: true, AllowedOrigins: []string{"https://one.example"}, AllowedHeaders: []string{"Authorization"}}
|
||||
option := omni.WithConfig(cfg)
|
||||
cfg.CORS.AllowedOrigins[0] = "https://evil.example"
|
||||
a := omni.New(option)
|
||||
defer a.Close()
|
||||
b := omni.New(option)
|
||||
defer b.Close()
|
||||
a.Config().CORS.AllowedOrigins[0] = "https://evil.example"
|
||||
for _, app := range []*omni.App{a, b} {
|
||||
w := httptest.NewRecorder()
|
||||
r := httptest.NewRequest("GET", "/absent", nil)
|
||||
r.Header.Set("Origin", "https://one.example")
|
||||
app.ServeHTTP(w, r)
|
||||
if w.Header().Get("Access-Control-Allow-Origin") != "https://one.example" {
|
||||
t.Fatal(w.Header())
|
||||
}
|
||||
}
|
||||
defer func() {
|
||||
if recover() == nil {
|
||||
t.Error("invalid CORS passed direct New")
|
||||
}
|
||||
}()
|
||||
omni.New(omni.WithCORS(omni.CORSConfig{Enabled: true}))
|
||||
}
|
||||
|
||||
func TestCORSTransports(t *testing.T) {
|
||||
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
cfg := omni.CORSConfig{Enabled: true, AllowedOrigins: []string{"https://web.example"}, AllowedMethods: []string{"GET", "POST"}, AllowedHeaders: []string{"Content-Type"}, ExposedHeaders: []string{"Content-Disposition"}}
|
||||
a := omni.New(omni.WithCORS(cfg), omni.WithTransport(transport), omni.WithTLS(cert, key))
|
||||
var called atomic.Int32
|
||||
a.Use(func(next omni.Handler) omni.Handler {
|
||||
return func(c *omni.Context) error { called.Add(1); return next(c) }
|
||||
})
|
||||
a.POST("/upload", func(c *omni.Context) error {
|
||||
b, err := io.ReadAll(c.Request().Body)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Text(200, string(b))
|
||||
})
|
||||
a.GET("/download", func(c *omni.Context) error {
|
||||
return c.WriteStream(200, "application/octet-stream", &httpx.Stream{Content: io.NopCloser(strings.NewReader("file bytes")), Name: "example.txt"})
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- a.ServeContext(ctx, listener) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("server did not close")
|
||||
listener.Close()
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
url := "https://" + listener.Addr().String()
|
||||
for _, tc := range []struct {
|
||||
method, path, body string
|
||||
status int
|
||||
}{{"OPTIONS", "/upload", "", 204}, {"POST", "/upload", "raw upload", 200}, {"GET", "/download", "file bytes", 200}, {"GET", "/missing", "", 404}} {
|
||||
var input io.Reader
|
||||
if tc.method == "POST" {
|
||||
input = strings.NewReader(tc.body)
|
||||
}
|
||||
r, _ := http.NewRequest(tc.method, url+tc.path, input)
|
||||
r.Header.Set("Origin", "https://web.example")
|
||||
if tc.method == "OPTIONS" {
|
||||
r.Header.Set("Access-Control-Request-Method", "POST")
|
||||
r.Header.Set("Access-Control-Request-Headers", "content-type")
|
||||
}
|
||||
res, err := client.Do(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
body, err := io.ReadAll(res.Body)
|
||||
res.Body.Close()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if res.StatusCode != tc.status || res.Header.Get("Access-Control-Allow-Origin") != "https://web.example" {
|
||||
t.Fatal(res.StatusCode, res.Header)
|
||||
}
|
||||
if tc.status == 200 && string(body) != tc.body {
|
||||
t.Fatal(string(body))
|
||||
}
|
||||
if tc.status == 204 && (len(body) != 0 || called.Load() != 0) {
|
||||
t.Fatal("preflight reached middleware")
|
||||
}
|
||||
if tc.path == "/download" && (!strings.Contains(res.Header.Get("Content-Disposition"), "example.txt") || res.Header.Get("Access-Control-Expose-Headers") != "Content-Disposition") {
|
||||
t.Fatal(res.Header)
|
||||
}
|
||||
if transport == omni.TransportHTTP2 && res.ProtoMajor != 2 {
|
||||
t.Fatal(res.Proto)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
241
app/i18n_test.go
Normal file
241
app/i18n_test.go
Normal file
@@ -0,0 +1,241 @@
|
||||
package app_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n/errori18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n/httpi18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
func localizedCatalog(namespace string, status int) apierrors.Catalog {
|
||||
return errori18n.Bind(apierrors.Catalog{DefaultStatus: status, Definitions: map[string]apierrors.Definition{
|
||||
"SAME": {Code: int64(9223372036854775807), Message: namespace + ":{value}", Params: []apierrors.Parameter{{Name: "value", Type: "string"}}},
|
||||
"INTERNAL_ERROR": {Status: 500, Code: int64(500), Message: "internal"},
|
||||
"FIXED": {Status: 401, Code: int64(401), Message: "fixed {literal}"},
|
||||
}}, namespace)
|
||||
}
|
||||
func localizedOptions(t *testing.T, catalogs ...apierrors.Catalog) (*httpi18n.Policy, *errori18n.Localizer) {
|
||||
t.Helper()
|
||||
docs := []i18n.Document{}
|
||||
for _, catalog := range catalogs {
|
||||
source, err := errori18n.Source(catalog, "en")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
docs = append(docs, source...)
|
||||
for _, d := range source {
|
||||
docs = append(docs, i18n.Document{Version: 1, Namespace: d.Namespace, Locale: "zh-Hans", Messages: map[string]i18n.Message{
|
||||
"SAME": {Value: i18n.Pattern{Text: d.Namespace + "译文:{value}"}}, "FIXED": {Value: i18n.Pattern{Text: "需要认证"}}, "INTERNAL_ERROR": {Value: i18n.Pattern{Text: "系统错误"}},
|
||||
}})
|
||||
}
|
||||
}
|
||||
b, err := i18n.New(i18n.Config{SourceLocale: "en", DefaultLocale: "en", Locales: []string{"en", "zh-Hans"}}, docs...)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
l := errori18n.New(b, nil)
|
||||
for _, c := range catalogs {
|
||||
if err := l.Validate(c); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
p, err := httpi18n.New(b)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return p, l
|
||||
}
|
||||
|
||||
func TestI18nRouteScopesEarlyFailuresAndReplay(t *testing.T) {
|
||||
global, local := localizedCatalog("app", 200), localizedCatalog("route", 409)
|
||||
policy, localizer := localizedOptions(t, global, local)
|
||||
format := response.Config{CodeType: "int64", SuccessCode: int64(0), ErrorCode: int64(999), Body: map[string]any{"error": map[string]any{"reason": "${message}", "id": "${code}"}, "ok": "${ok}", "data": "${data}"}, DataMode: "always", Messages: map[string]string{"FIXED": "fixed {literal}"}}
|
||||
app := omni.New(omni.WithErrors(global), omni.WithResponse(format), omni.WithLanguagePolicy(policy), omni.WithErrorLocalizer(localizer))
|
||||
defer app.Close()
|
||||
private := errors.New("private-value")
|
||||
app.Use(func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
switch c.Header("X-Early") {
|
||||
case "auth":
|
||||
return apierrors.ByCode("FIXED")
|
||||
case "panic":
|
||||
panic(private)
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
})
|
||||
handler := func(c *httpx.Context) error {
|
||||
if c.Header("X-Params") == "missing" {
|
||||
return apierrors.ByCode("SAME")
|
||||
}
|
||||
return apierrors.ByCode("SAME").WithParams(map[string]any{"value": "{literal}"}).WithCause(private)
|
||||
}
|
||||
app.GET("/global", handler)
|
||||
bridge := httpx.WrapMiddleware(func(next http.Handler) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { next.ServeHTTP(w, r) })
|
||||
})
|
||||
app.HandleWithResponse("GET", "/route", response.RawJSON(response.MustWithErrors(local, response.MustNew(format))), handler, bridge)
|
||||
for _, path := range []string{"/global", "/route"} {
|
||||
for _, locale := range []string{"en", "zh-CN"} {
|
||||
for _, mode := range []string{"", "auth", "panic", "missing"} {
|
||||
r := httptest.NewRequest("GET", path, nil)
|
||||
r.Header.Set("Accept-Language", locale)
|
||||
r.Header.Set("X-Early", mode)
|
||||
r.Header.Set("X-Params", mode)
|
||||
w := httptest.NewRecorder()
|
||||
app.ServeHTTP(w, r)
|
||||
var body struct {
|
||||
Error struct {
|
||||
Reason string `json:"reason"`
|
||||
ID json.Number `json:"id"`
|
||||
} `json:"error"`
|
||||
OK bool `json:"ok"`
|
||||
Data any `json:"data"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
|
||||
t.Fatal(err, w.Body.String())
|
||||
}
|
||||
ns, status := "app", 200
|
||||
if path == "/route" {
|
||||
ns, status = "route", 409
|
||||
}
|
||||
want := ns + ":{literal}"
|
||||
if locale != "en" {
|
||||
want = ns + "译文:{literal}"
|
||||
}
|
||||
if mode == "auth" {
|
||||
status = 401
|
||||
want = "fixed {literal}"
|
||||
if locale != "en" {
|
||||
want = "需要认证"
|
||||
}
|
||||
} else if mode == "panic" || mode == "missing" {
|
||||
status = 500
|
||||
want = "internal"
|
||||
if locale != "en" {
|
||||
want = "系统错误"
|
||||
}
|
||||
}
|
||||
if w.Code != status || body.OK || body.Error.Reason != want || strings.Contains(w.Body.String(), "private-value") {
|
||||
t.Fatal(path, locale, mode, w.Code, w.Body.String())
|
||||
}
|
||||
if mode == "" && body.Error.ID.String() != "9223372036854775807" {
|
||||
t.Fatal("public code changed", body.Error)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, shared := range []bool{false, true} {
|
||||
cfg := middleware.IdempotencyConfig{Scope: func(*httpx.Context) (string, error) { return "test", nil }}
|
||||
if shared {
|
||||
cfg.Store = &catalogStore{records: map[string]middleware.IdempotencyRecord{}}
|
||||
}
|
||||
idem, err := middleware.NewIdempotency(cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
path := "/localReplay"
|
||||
if shared {
|
||||
path = "/sharedReplay"
|
||||
}
|
||||
calls := 0
|
||||
app.POST(path, func(c *httpx.Context) error { calls++; return handler(c) }, idem.Middleware())
|
||||
var first string
|
||||
for _, locale := range []string{"en", "zh-CN"} {
|
||||
r := httptest.NewRequest("POST", path, strings.NewReader(`{}`))
|
||||
r.Header.Set("Idempotency-Key", "same")
|
||||
r.Header.Set("Accept-Language", locale)
|
||||
w := httptest.NewRecorder()
|
||||
app.ServeHTTP(w, r)
|
||||
if locale == "en" {
|
||||
first = w.Body.String()
|
||||
} else if w.Body.String() != first || w.Header().Get("Idempotency-Replayed") != "true" {
|
||||
t.Fatal("replay changed", w.Body.String(), first)
|
||||
}
|
||||
}
|
||||
if calls != 1 {
|
||||
t.Fatal("language retriggered business", calls)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestI18nExplicitProtocolsAndConcurrentApps(t *testing.T) {
|
||||
apps := []*omni.App{}
|
||||
for _, ns := range []string{"one", "two"} {
|
||||
c := localizedCatalog(ns, 200)
|
||||
p, l := localizedOptions(t, c)
|
||||
a := omni.New(omni.WithErrors(c), omni.WithResponse(response.Config{CodeType: "int64", SuccessCode: 0, ErrorCode: 999}), omni.WithLanguagePolicy(p), omni.WithErrorLocalizer(l))
|
||||
defer a.Close()
|
||||
a.GET("/", func(*httpx.Context) error { return apierrors.ByCode("FIXED") })
|
||||
a.HandleWithResponse("GET", "/ack", response.EncoderFunc(func(r response.Result) (response.Payload, error) {
|
||||
if !r.IsError() || r.Message != "fixed {literal}" || r.PublicMessage() != "需要认证" {
|
||||
t.Error(r)
|
||||
}
|
||||
return response.Text(200, "ACK"), nil
|
||||
}), func(*httpx.Context) error { return apierrors.ByCode("FIXED") })
|
||||
a.GET("/direct", func(c *httpx.Context) error { return c.Text(200, "fixed-protocol") })
|
||||
apps = append(apps, a)
|
||||
}
|
||||
var wg sync.WaitGroup
|
||||
for n := 0; n < 60; n++ {
|
||||
wg.Add(1)
|
||||
go func(n int) {
|
||||
defer wg.Done()
|
||||
locale, want := "en", "fixed {literal}"
|
||||
if n%2 == 0 {
|
||||
locale, want = "zh-CN", "需要认证"
|
||||
}
|
||||
r := httptest.NewRequest("GET", "/", nil)
|
||||
r.Header.Set("Accept-Language", locale)
|
||||
w := httptest.NewRecorder()
|
||||
apps[n%len(apps)].ServeHTTP(w, r)
|
||||
if w.Code != 401 || !strings.Contains(w.Body.String(), want) {
|
||||
t.Error(w.Code, w.Body.String())
|
||||
}
|
||||
}(n)
|
||||
}
|
||||
wg.Wait()
|
||||
for _, path := range []string{"/ack", "/direct"} {
|
||||
r := httptest.NewRequest("GET", path, nil)
|
||||
r.Header.Set("Accept-Language", "zh-CN")
|
||||
w := httptest.NewRecorder()
|
||||
apps[0].ServeHTTP(w, r)
|
||||
want := "ACK"
|
||||
if path == "/direct" {
|
||||
want = "fixed-protocol"
|
||||
}
|
||||
if w.Body.String() != want {
|
||||
t.Fatal(w.Body.String())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
type panickingLocalizer struct{}
|
||||
|
||||
func (panickingLocalizer) Localize(context.Context, *apierrors.Registry, *apierrors.APIError) (string, bool) {
|
||||
panic("presentation failed")
|
||||
}
|
||||
func TestI18nAdapterFailurePreservesBusiness(t *testing.T) {
|
||||
c := localizedCatalog("test", 200)
|
||||
a := omni.New(omni.WithErrors(c), omni.WithResponse(response.Config{CodeType: "int64", SuccessCode: 0, ErrorCode: 999}), omni.WithErrorLocalizer(panickingLocalizer{}))
|
||||
defer a.Close()
|
||||
a.GET("/", func(*httpx.Context) error { return apierrors.ByCode("FIXED") })
|
||||
w := httptest.NewRecorder()
|
||||
a.ServeHTTP(w, httptest.NewRequest("GET", "/", nil))
|
||||
if w.Code != 401 || !strings.Contains(w.Body.String(), "fixed {literal}") {
|
||||
t.Fatal(w.Code, w.Body.String())
|
||||
}
|
||||
}
|
||||
@@ -28,6 +28,9 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid
|
||||
}
|
||||
}
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
if a.config.LanguagePolicy != nil {
|
||||
request = a.config.LanguagePolicy.Apply(writer, request)
|
||||
}
|
||||
request = request.WithContext(logx.WithLogger(contract.WithRequestPolicy(request.Context(), a.config.Request), a.config.LogInstance))
|
||||
if len(a.config.TrustedProxies) > 0 {
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
|
||||
@@ -40,6 +43,7 @@ func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, mid
|
||||
requestContext.SetErrorRegistry(provider.ErrorRegistry())
|
||||
}
|
||||
requestContext.SetResponseEncoder(encoder)
|
||||
requestContext.SetErrorLocalizer(a.config.ErrorLocalizer)
|
||||
defer func() { httpx.CleanupMultipart(requestContext.Request()) }()
|
||||
if err := wrapped(requestContext); err != nil {
|
||||
a.writeError(requestContext, err)
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
# go-omni CLI
|
||||
|
||||
v0.17 增加公共字段组合、可选结构映射、自动中间件工厂和按需 Repository。配置和迁移见 [生成能力指南](../../GENERATION.md)。
|
||||
v0.26 新增 `i18n check|export|types|extract`：检查独立语言资源、提取 Go 静态消息引用或已导出的错误目录，导出指定命名空间的前端包和 TypeScript 类型。输入只读，结果写 stdout；动态引用明确报告。完整命令见 [国际化工具链](../../i18n/README.md#cli-提取检查与导出)。`.api errorCatalog` 可声明 translation 引用，旧 runtime 在生成前拒绝，接入时使用同版 v0.26 CLI 与框架。
|
||||
|
||||
v0.19 移除结构映射生成：`omni.gen.json` 只保留 Repository 和中间件选项。普通接口直接使用生成 DTO；需要类型转换时用普通 Go 或可选 [copyx](../../COPY.md)。旧 `mappings` / `mappingFiles` 必须按迁移步骤清理，CLI 不会静默删除仍在调用的旧生成文件。
|
||||
|
||||
公共字段组合、中间件工厂和按需 Repository 继续支持，见 [生成能力指南](../../GENERATION.md)。
|
||||
|
||||
v0.16 增加基础类型/动态 JSON 数组返回、OpenAPI 公共引用和 workspace Go Catalog 统一导出。见 [接入与筛选规则](../../DEVELOPMENT.md#同一份-go-错误目录供多个服务生成)。
|
||||
|
||||
@@ -12,7 +16,7 @@ v0.14 新增 `api format [--check]`、`api upgrade [--dry-run|--preview <空目
|
||||
|
||||
```sh
|
||||
# 仓库读取权限与 PATH 配置见根目录 README
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.17.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.26.0-preview
|
||||
go-omni new --module example.com/hello ./hello
|
||||
|
||||
# 从自己的契约生成；service.api 及其相对 import 文件需要在本地存在
|
||||
@@ -47,14 +51,14 @@ main、Gen.go、OpenAPI 和 manifest 由生成器维护；业务文件按字节
|
||||
|
||||
## 版本与迁移
|
||||
|
||||
CLI 与框架依赖固定使用 v0.17.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换：
|
||||
CLI 与框架依赖固定使用 v0.26.0-preview。远程安装的 CLI 默认把自身版本写入 go.mod，不添加本地 replace；`--framework-version` 可显式指定已发布的兼容版本。仅在开发框架源码时使用本地替换：
|
||||
|
||||
```sh
|
||||
# 在 go-omni 源码仓库中执行
|
||||
go run ./cmd/go-omni new --module example.com/hello --framework-dir . ../hello
|
||||
```
|
||||
|
||||
当前模板与依赖以 v0.17.0-preview 为基线，继续使用 manifest v2；GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级)：原始模板可自动替换，含业务修改的基础文件按预览合并；通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。
|
||||
当前模板与依赖以 v0.26.0-preview 为基线，继续使用 manifest v2；GORM 可选适配器从 v0.14.1-preview 提供。已有项目使用 [定点升级与预览](../../DEVELOPMENT.md#v013-项目升级)：原始模板可自动替换，含业务修改的基础文件按预览合并；通用清理由 Runtime 负责。独立工具命令不会被删除。受管理项目通过 `.api` 模块导入增加业务，不使用旧 `add` 命令的手动路由装配。
|
||||
|
||||
## 数据库迁移命令
|
||||
|
||||
@@ -67,3 +71,9 @@ go-omni migrate --dir migrations --apply
|
||||
```
|
||||
|
||||
SQL 命名示例 `0001_createUsers.up.sql`；预览只显示版本和 SHA-256，不连接数据库。失败保留 dirty 状态，不自动 down/force。细节见 [扩展模块](../../ADVANCED.md)。
|
||||
|
||||
### v0.22 文件显式配置
|
||||
|
||||
新生成 `internal/config/loadGen.go` 调用统一加载，纳入生成检查；项目 config.go 只维护字段、默认值和校验，loader.go 用于注册额外格式。JSON/YAML/XML 内置。旧手写 Load 会阻止生成，请先按 [迁移步骤](../../CONFIG.md#v022-迁移旧模式已移除) 整理，不会自动覆盖业务代码。
|
||||
|
||||
`go run . -config etc/config.test.yaml -check-config` 只做配置解析和校验，不初始化日志、数据库或监听。文件中的固定值不受环境自动覆盖；`${NAME}` 为必填引用，`${NAME:-fallback}` 提供备用值。
|
||||
|
||||
136
cmd/go-omni/apigen/configLoader_test.go
Normal file
136
cmd/go-omni/apigen/configLoader_test.go
Normal file
@@ -0,0 +1,136 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestGeneratedConfigurationConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
api := filepath.Join(t.TempDir(), "service.api")
|
||||
writeFile(t, api, "service configuration {\n@handler ping\nget /ping\n}\n")
|
||||
o := Options{API: api, Dir: dir, Module: "example.com/configuration", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
main := readFile(t, filepath.Join(dir, "main.go"))
|
||||
// The business only changes fields/defaults/validation, not the loading flow.
|
||||
writeFile(t, filepath.Join(dir, "internal/config/config.go"), `package config
|
||||
import("git.yuntuo.hk/k2safe/go-omni/config";"git.yuntuo.hk/k2safe/go-omni/logx")
|
||||
type Config struct { Server config.Config `+"`json:\"server\"`"+`;Limit int `+"`json:\"limit\"`"+` }
|
||||
func Default()Config{ c:=config.Default();c.Log.File="./var/log/app.log";c.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:10};return Config{Server:c,Limit:10} }
|
||||
func(c Config)Validate()error{if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)};if c.Limit<0{return config.Invalid("limit","must not be negative")};return nil}
|
||||
`)
|
||||
writeFile(t, filepath.Join(dir, "internal/config/loader.go"), `package config
|
||||
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
|
||||
func NewLoader()(*omniConfig.Loader,error){return omniConfig.NewLoader(omniConfig.WithDecoder("custom",omniConfig.JSONDecoder{}))}
|
||||
`)
|
||||
svc := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
writeFile(t, svc, strings.Replace(readFile(t, svc), "return nil", "panic(\"dependencies must not initialize on invalid config\")", 1))
|
||||
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/configuration/consumer_test.go"))
|
||||
runConsumerGo(t, dir, "mod", "tidy")
|
||||
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
|
||||
runConsumerGo(t, dir, "build", ".")
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
before := snapshot(t, dir)
|
||||
for _, mode := range []string{"dry", "generate", "check"} {
|
||||
o.DryRun = mode == "dry"
|
||||
o.Check = mode == "check"
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("business files overwritten")
|
||||
}
|
||||
if main != readFile(t, filepath.Join(dir, "main.go")) {
|
||||
t.Fatal("main changed")
|
||||
}
|
||||
load := filepath.Join(dir, "internal/config/loadGen.go")
|
||||
original := readFile(t, load)
|
||||
writeFile(t, load, original+"\n// drift\n")
|
||||
if err := generateManagedTestProject(t, o); err == nil {
|
||||
t.Fatal("loadGen drift accepted")
|
||||
}
|
||||
writeFile(t, load, original)
|
||||
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
data, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatal(err, string(data))
|
||||
}
|
||||
var dependency struct {
|
||||
Version, Sum string
|
||||
Replace any
|
||||
}
|
||||
if err = json.Unmarshal(data, &dependency); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
|
||||
t.Fatal("not a remote consumer")
|
||||
}
|
||||
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
|
||||
}
|
||||
}
|
||||
func TestConfigLoadMigrationGuard(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
o := Options{Dir: dir, Module: "example.com/upgrade", FrameworkDir: frameworkRoot(t)}
|
||||
if _, err := New(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
matches, err := filepath.Glob(filepath.Join(dir, "api/*.api"))
|
||||
if err != nil || len(matches) != 1 {
|
||||
t.Fatal(matches, err)
|
||||
}
|
||||
o.API = matches[0]
|
||||
writeFile(t, filepath.Join(dir, "internal/config/legacy.go"), "package config\nfunc Load(path string)(Config,error){return Default(),nil}\n")
|
||||
before := snapshot(t, dir)
|
||||
for _, mode := range []string{"dry", "generate", "check"} {
|
||||
o.DryRun = mode == "dry"
|
||||
o.Check = mode == "check"
|
||||
_, err := Generate(o)
|
||||
if err == nil || !strings.Contains(err.Error(), "CONFIG.md") {
|
||||
t.Fatal("missing migration diagnostic", err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("migration guard changed files")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemovedEnvironmentTagsRejectBeforeGeneration(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
o := Options{Dir: dir, Module: "example.com/migrate", FrameworkDir: frameworkRoot(t)}
|
||||
if _, err := New(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
path := filepath.Join(dir, "internal/config/config.go")
|
||||
source := readFile(t, path)
|
||||
writeFile(t, path, strings.Replace(source, "json:\"server\"", "json:\"server\" env:\"OLD_SERVER\"", 1))
|
||||
before := snapshot(t, dir)
|
||||
for _, mode := range []string{"dry", "generate", "check"} {
|
||||
o.DryRun = mode == "dry"
|
||||
o.Check = mode == "check"
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "env/envDefault") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("migration rejection wrote files")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -55,6 +55,7 @@ func prepareMetadata(original *Spec) (*Spec, error) {
|
||||
for i := range spec.Errors {
|
||||
spec.Errors[i].BusinessCode = normalized.Definitions[spec.Errors[i].Code].Code
|
||||
spec.Errors[i].Params = normalized.Definitions[spec.Errors[i].Code].Params
|
||||
spec.Errors[i].Translation = normalized.Definitions[spec.Errors[i].Code].Translation
|
||||
}
|
||||
if err := validateErrorPolicies(&spec); err != nil {
|
||||
return nil, err
|
||||
|
||||
118
cmd/go-omni/apigen/copyConsumer_test.go
Normal file
118
cmd/go-omni/apigen/copyConsumer_test.go
Normal file
@@ -0,0 +1,118 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func runConsumerGo(t *testing.T, dir string, args ...string) {
|
||||
t.Helper()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", args...)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("go %v: %v\n%s", args, err, out)
|
||||
}
|
||||
}
|
||||
func TestCopyxThreeEndpointConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for path, fixture := range map[string]string{
|
||||
"internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go", "internal/convert/copy.go": "copy.go",
|
||||
"internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go",
|
||||
} {
|
||||
writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture))
|
||||
}
|
||||
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1)
|
||||
source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1)
|
||||
writeFile(t, svcFile, source)
|
||||
configFile := filepath.Join(dir, "internal/config/config.go")
|
||||
writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1))
|
||||
for _, group := range []string{"Enterprise", "Tenant", "Admin"} {
|
||||
text := fmt.Sprintf(`package %s
|
||||
import("context";"example.com/composition/internal/types";"example.com/composition/internal/convert";"example.com/composition/internal/domain")
|
||||
func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){
|
||||
input,err:=convert.Employee(*req);if err!=nil{return nil,err}
|
||||
result,err:=convert.Response(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil
|
||||
}
|
||||
`, strings.ToLower(group), group)
|
||||
writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text)
|
||||
}
|
||||
runConsumerGo(t, dir, "mod", "tidy")
|
||||
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
before := snapshot(t, dir)
|
||||
o.DryRun = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.DryRun = false
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("generation changed business files")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/mapping")); !os.IsNotExist(err) {
|
||||
t.Fatal("mapping directory generated")
|
||||
}
|
||||
// Adding a common field requires one contract edit and one independent
|
||||
// business field. Three endpoint converters need no new rules.
|
||||
contract := readFile(t, o.API)
|
||||
contract = strings.Replace(contract, "type SalaryFields {", "type SalaryFields {\n Note string `json:\"note,optional\"`", 1)
|
||||
writeFile(t, o.API, contract)
|
||||
domainPath := filepath.Join(dir, "internal/domain/employee.go")
|
||||
writeFile(t, domainPath, strings.Replace(readFile(t, domainPath), "type Employee struct {", "type Employee struct {\n Note string", 1))
|
||||
o.Check = false
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
|
||||
runConsumerGo(t, dir, "build", ".")
|
||||
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
data, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatal(err, string(data))
|
||||
}
|
||||
var dependency struct {
|
||||
Version, Sum string
|
||||
Replace any
|
||||
}
|
||||
if err = json.Unmarshal(data, &dependency); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
|
||||
t.Fatal("not a fixed remote consumer")
|
||||
}
|
||||
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
|
||||
}
|
||||
}
|
||||
59
cmd/go-omni/apigen/corsConsumer_test.go
Normal file
59
cmd/go-omni/apigen/corsConsumer_test.go
Normal file
@@ -0,0 +1,59 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestGeneratedCORSConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
api := filepath.Join(t.TempDir(), "service.api")
|
||||
writeFile(t, api, "service corsdemo {\n@handler ping\nget /ping\n}\n")
|
||||
o := Options{API: api, Dir: dir, Module: "example.com/corsdemo", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "consumer_test.go"), readFile(t, "testdata/cors/consumer_test.go"))
|
||||
writeFile(t, filepath.Join(dir, "etc/config.yaml"), "server:\n log:\n output: discard\n cors:\n enabled: true\n allowedOrigins: [https://web.example]\n allowedMethods: [GET, POST, PUT]\n allowedHeaders: [Authorization, Content-Type]\n maxAge: 10m\n")
|
||||
runConsumerGo(t, dir, "mod", "tidy")
|
||||
runConsumerGo(t, dir, "test", "-race", "-count=1", "./...")
|
||||
runConsumerGo(t, dir, "build", ".")
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
before := snapshot(t, dir)
|
||||
for _, mode := range []string{"dry", "generate", "check"} {
|
||||
o.DryRun = mode == "dry"
|
||||
o.Check = mode == "check"
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("regeneration changed project files")
|
||||
}
|
||||
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
|
||||
cmd := exec.Command("go", "list", "-m", "-json", frameworkModule)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatal(err, string(out))
|
||||
}
|
||||
var dependency struct {
|
||||
Version, Sum string
|
||||
Replace any
|
||||
}
|
||||
if err := json.Unmarshal(out, &dependency); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
|
||||
t.Fatal("not a fixed remote consumer")
|
||||
}
|
||||
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
|
||||
}
|
||||
}
|
||||
@@ -11,14 +11,15 @@ import (
|
||||
// ErrorDefinition declares a public, stable error. Generated constructors return
|
||||
// a fresh error each time so WithCause cannot mutate a shared singleton.
|
||||
type ErrorDefinition struct {
|
||||
Name string `json:"name"`
|
||||
BusinessCode any `json:"businessCode,omitempty"`
|
||||
Params []apierrors.Parameter `json:"params,omitempty"`
|
||||
Catalog string `json:"catalog,omitempty"`
|
||||
Status int `json:"status"`
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Position string `json:"-"`
|
||||
Name string `json:"name"`
|
||||
BusinessCode any `json:"businessCode,omitempty"`
|
||||
Params []apierrors.Parameter `json:"params,omitempty"`
|
||||
Translation *apierrors.Translation `json:"translation,omitempty"`
|
||||
Catalog string `json:"catalog,omitempty"`
|
||||
Status int `json:"status"`
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Position string `json:"-"`
|
||||
}
|
||||
type AuthDefinition struct {
|
||||
Name, Type, In, KeyName, Middleware, Description, BearerFormat, Position string
|
||||
|
||||
@@ -49,11 +49,12 @@ func (p *parser) parseErrorCatalog() error {
|
||||
}
|
||||
}
|
||||
type entry struct {
|
||||
Key string `json:"key"`
|
||||
Code any `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Status int `json:"status"`
|
||||
Params []apierrors.Parameter `json:"params"`
|
||||
Key string `json:"key"`
|
||||
Code any `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Status int `json:"status"`
|
||||
Params []apierrors.Parameter `json:"params"`
|
||||
Translation *apierrors.Translation `json:"translation"`
|
||||
}
|
||||
var entries map[string]entry
|
||||
if err := apierrors.DecodeCatalogJSON([]byte(props["errors"]), &entries); err != nil {
|
||||
@@ -75,7 +76,7 @@ func (p *parser) parseErrorCatalog() error {
|
||||
if e.Status == 0 {
|
||||
e.Status = defaultStatus
|
||||
}
|
||||
p.spec.Errors = append(p.spec.Errors, ErrorDefinition{Name: fn, Status: e.Status, Code: e.Key, Message: e.Message, BusinessCode: e.Code, Params: e.Params, Catalog: name, Position: pos})
|
||||
p.spec.Errors = append(p.spec.Errors, ErrorDefinition{Name: fn, Status: e.Status, Code: e.Key, Message: e.Message, BusinessCode: e.Code, Params: e.Params, Translation: e.Translation, Catalog: name, Position: pos})
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -90,7 +91,7 @@ func errorCatalog(spec *Spec, names []string) apierrors.Catalog {
|
||||
if names != nil && !selected[e.Name] {
|
||||
continue
|
||||
}
|
||||
c.Definitions[e.Code] = apierrors.Definition{Status: e.Status, Code: e.BusinessCode, Message: e.Message, Params: e.Params}
|
||||
c.Definitions[e.Code] = apierrors.Definition{Status: e.Status, Code: e.BusinessCode, Message: e.Message, Params: e.Params, Translation: e.Translation}
|
||||
}
|
||||
return c
|
||||
}
|
||||
|
||||
@@ -19,7 +19,10 @@ import (
|
||||
)
|
||||
|
||||
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
const minimumVersion = "v0.17.0-preview"
|
||||
|
||||
// Runtime-only releases need not raise the managed template compatibility floor.
|
||||
const minimumVersion = "v0.22.0-preview"
|
||||
const currentVersion = "v0.26.0-preview"
|
||||
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||
|
||||
type Options struct {
|
||||
@@ -106,6 +109,9 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
if err = checkRuntimeExtension(directory); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
if err = checkConfigLoadExtension(directory); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
for _, r := range c.Spec.Routes {
|
||||
if !r.Manual {
|
||||
path := filepath.Join(directory, "internal/handler", r.Group, lower(r.Handler)+"Manual.go")
|
||||
@@ -152,7 +158,7 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
if !filepath.IsAbs(path) {
|
||||
path = filepath.Join(directory, path)
|
||||
}
|
||||
if source, err := os.ReadFile(filepath.Join(path, "config", "env.go")); err == nil && strings.Contains(string(source), "func ServerEnvAliases(") {
|
||||
if source, err := os.ReadFile(filepath.Join(path, "config", "references.go")); err == nil && strings.Contains(string(source), "func WithLookupEnv(") {
|
||||
localCurrent = true
|
||||
}
|
||||
}
|
||||
@@ -188,6 +194,13 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
}
|
||||
seeds["go.mod"] = data
|
||||
}
|
||||
capabilityModule := moduleData
|
||||
if seeded, ok := seeds["go.mod"]; ok {
|
||||
capabilityModule = seeded
|
||||
}
|
||||
if err := checkI18nRuntime(c.Spec, capabilityModule, directory); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
if err = module.CheckImportPath(o.Module); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
@@ -252,20 +265,17 @@ func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := renderMappings(o, generated, seeds); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
currentConfig, stillExists, err := readGeneratorConfig(directory)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
beforeConfig, _ := json.Marshal(genConfig)
|
||||
afterConfig, _ := json.Marshal(currentConfig)
|
||||
if hasGenConfig != stillExists || string(beforeConfig) != string(afterConfig) {
|
||||
return Output{}, errors.New("omni.gen.json changed during generation; retry")
|
||||
if hasGenConfig != stillExists || genConfig.inputDigest != currentConfig.inputDigest || string(beforeConfig) != string(afterConfig) {
|
||||
return Output{}, errors.New("generation configuration changed during generation; retry")
|
||||
}
|
||||
out, err := apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile(), snapshotContract(c.Spec), o.lockHeld)
|
||||
expectedVersion := minimumVersion
|
||||
expectedVersion := currentVersion
|
||||
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" && semver.IsValid(info.Main.Version) {
|
||||
expectedVersion = info.Main.Version
|
||||
}
|
||||
@@ -304,7 +314,7 @@ func moduleFile(o Options) ([]byte, error) {
|
||||
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
|
||||
version = info.Main.Version
|
||||
} else {
|
||||
version = minimumVersion
|
||||
version = currentVersion
|
||||
}
|
||||
}
|
||||
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {
|
||||
|
||||
@@ -1,23 +1,25 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
)
|
||||
|
||||
const generatorConfigFile = "omni.gen.json"
|
||||
|
||||
// GeneratorConfig is developer-owned and checked on every generation. Mapping
|
||||
// rules never change the public API or infer a persistence model.
|
||||
// GeneratorConfig contains developer-owned scaffolding options only.
|
||||
type GeneratorConfig struct {
|
||||
Repository string `json:"repository,omitempty"`
|
||||
Middleware string `json:"middleware,omitempty"`
|
||||
Mappings []Mapping `json:"mappings,omitempty"`
|
||||
Repository string `json:"repository,omitempty"`
|
||||
Middleware string `json:"middleware,omitempty"`
|
||||
inputDigest string
|
||||
}
|
||||
|
||||
func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
|
||||
@@ -35,7 +37,7 @@ func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
|
||||
return cfg, false, err
|
||||
}
|
||||
if previous.GeneratorConfig {
|
||||
return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options and mappings before regenerating")
|
||||
return cfg, false, errors.New("omni.gen.json is missing; restore the project-owned generation options before regenerating")
|
||||
}
|
||||
cfg.Middleware = "handlers"
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
@@ -49,21 +51,60 @@ func readGeneratorConfig(root string) (GeneratorConfig, bool, error) {
|
||||
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
|
||||
return cfg, false, errors.New("omni.gen.json must be a regular file of at most 1 MiB")
|
||||
}
|
||||
raw, err := os.ReadFile(filepath.Join(root, generatorConfigFile))
|
||||
raw, err := readGeneratorConfigFile(root, generatorConfigFile)
|
||||
if err != nil {
|
||||
return cfg, false, err
|
||||
}
|
||||
var keys map[string]json.RawMessage
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &keys); err != nil {
|
||||
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
|
||||
}
|
||||
for _, key := range []string{"mappings", "mappingFiles"} {
|
||||
if _, exists := keys[key]; exists {
|
||||
return cfg, false, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go, remove mappings/mappingFiles from omni.gen.json, then remove internal/mapping/mappingsGen.go; see COPY.md")
|
||||
}
|
||||
}
|
||||
if err = apierrors.DecodeCatalogJSON(raw, &cfg); err != nil {
|
||||
return cfg, false, fmt.Errorf("omni.gen.json: %w", err)
|
||||
}
|
||||
digest := sha256.Sum256(raw)
|
||||
cfg.inputDigest = fmt.Sprintf("%x", digest)
|
||||
if cfg.Repository != "none" && cfg.Repository != "group" {
|
||||
return cfg, false, errors.New("omni.gen.json: repository must be none or group")
|
||||
}
|
||||
if cfg.Middleware != "factories" && cfg.Middleware != "handlers" {
|
||||
return cfg, false, errors.New("omni.gen.json: middleware must be factories or handlers")
|
||||
}
|
||||
if len(cfg.Mappings) > 128 {
|
||||
return cfg, false, errors.New("omni.gen.json: at most 128 mappings supported")
|
||||
}
|
||||
return cfg, true, nil
|
||||
}
|
||||
|
||||
// Read through a bounded reader as well as checking stat: a growing file cannot
|
||||
// make configuration loading unbounded. safePath also rejects symlink escapes.
|
||||
func readGeneratorConfigFile(root, name string) ([]byte, error) {
|
||||
if err := safePath(root, name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
f, err := os.Open(filepath.Join(root, filepath.FromSlash(name)))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", name, err)
|
||||
}
|
||||
defer f.Close()
|
||||
stat, err := f.Stat()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !stat.Mode().IsRegular() || stat.Size() > 1<<20 {
|
||||
return nil, fmt.Errorf("%s must be a regular file of at most 1 MiB", name)
|
||||
}
|
||||
raw, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(raw) > 1<<20 {
|
||||
return nil, fmt.Errorf("%s exceeds 1 MiB", name)
|
||||
}
|
||||
if trimmed := strings.TrimSpace(string(raw)); !strings.HasPrefix(trimmed, "{") {
|
||||
return nil, fmt.Errorf("%s must contain a JSON object", name)
|
||||
}
|
||||
return raw, nil
|
||||
}
|
||||
|
||||
96
cmd/go-omni/apigen/generatorConfig_test.go
Normal file
96
cmd/go-omni/apigen/generatorConfig_test.go
Normal file
@@ -0,0 +1,96 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestWorkspaceRejectsRemovedMappingWithoutWrites(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
for _, name := range []string{"one", "two"} {
|
||||
if _, err := New(Options{Dir: filepath.Join(root, name), Module: "example.com/" + name, FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
firstAPI := filepath.Join(root, "one/api/service.api")
|
||||
writeFile(t, firstAPI, strings.Replace(readFile(t, firstAPI), "Greet a visitor", "Changed description", 1))
|
||||
writeFile(t, filepath.Join(root, "two", generatorConfigFile), `{"mappingFiles":["../../private.json"]}`)
|
||||
workspace := filepath.Join(root, "omni.workspace.json")
|
||||
writeFile(t, workspace, `{"services":[{"dir":"one"},{"dir":"two"}]}`)
|
||||
before := snapshot(t, root)
|
||||
for _, operation := range []string{"generate", "check"} {
|
||||
if err := Workspace(workspace, operation, io.Discard); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
|
||||
t.Fatal(operation, err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("workspace migration failure wrote files")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemovedMappingConfiguration(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, raw := range []string{`{"mappings":[]}`, `{"mappingFiles":[]}`, `{"mappings":null}`, `{"mappingFiles":["../../private.json"]}`} {
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), raw)
|
||||
before := snapshot(t, dir)
|
||||
for _, o := range []Options{{Dir: dir}, {Dir: dir, Check: true}, {Dir: dir, DryRun: true}} {
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("rejected config wrote project")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestOldMappingFileRequiresExplicitMigration(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/removed", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
name := "internal/mapping/mappingsGen.go"
|
||||
body := generatedHeader + "package mapping\n"
|
||||
writeFile(t, filepath.Join(dir, name), body)
|
||||
var m manifest
|
||||
if err := json.Unmarshal([]byte(readFile(t, filepath.Join(dir, manifestPath))), &m); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Generated[name] = digest([]byte(body))
|
||||
raw, _ := json.Marshal(m)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
before := snapshot(t, dir)
|
||||
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||
for _, mode := range []string{"generate", "check", "dry-run"} {
|
||||
o.Check = mode == "check"
|
||||
o.DryRun = mode == "dry-run"
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "mapping generation was removed") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("legacy file deleted or project changed")
|
||||
}
|
||||
}
|
||||
if err := os.Remove(filepath.Join(dir, name)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = false
|
||||
o.DryRun = false
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o.Check = true
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(readFile(t, filepath.Join(dir, manifestPath)), name) {
|
||||
t.Fatal("old ownership retained")
|
||||
}
|
||||
}
|
||||
57
cmd/go-omni/apigen/i18nVersion.go
Normal file
57
cmd/go-omni/apigen/i18nVersion.go
Normal file
@@ -0,0 +1,57 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"golang.org/x/mod/modfile"
|
||||
"golang.org/x/mod/semver"
|
||||
)
|
||||
|
||||
// Translation metadata was introduced in this release. Consumers of older
|
||||
// runtimes must upgrade or use an explicitly supported local replacement.
|
||||
const i18nMinimumVersion = "v0.26.0-preview"
|
||||
|
||||
func checkI18nRuntime(spec *Spec, moduleData []byte, directory string) error {
|
||||
needed := false
|
||||
for _, e := range spec.Errors {
|
||||
needed = needed || e.Translation != nil
|
||||
}
|
||||
if !needed {
|
||||
return nil
|
||||
}
|
||||
mf, err := modfile.Parse("go.mod", moduleData, nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
version := ""
|
||||
for _, r := range mf.Require {
|
||||
if r.Mod.Path == frameworkModule {
|
||||
version = r.Mod.Version
|
||||
}
|
||||
}
|
||||
for _, r := range mf.Replace {
|
||||
if r.Old.Path != frameworkModule || r.Old.Version != "" && r.Old.Version != version {
|
||||
continue
|
||||
}
|
||||
if r.New.Version != "" {
|
||||
version = r.New.Version
|
||||
continue
|
||||
}
|
||||
root := r.New.Path
|
||||
if !filepath.IsAbs(root) {
|
||||
root = filepath.Join(directory, root)
|
||||
}
|
||||
data, e := os.ReadFile(filepath.Join(root, "errors", "localization.go"))
|
||||
if e == nil && strings.Contains(string(data), "type Translation struct") {
|
||||
return nil
|
||||
}
|
||||
return fmt.Errorf("translation metadata requires a local framework with errors.Translation support")
|
||||
}
|
||||
if semver.Compare(version, i18nMinimumVersion) < 0 {
|
||||
return fmt.Errorf("translation metadata requires %s or newer; use --framework-dir for unpublished source", i18nMinimumVersion)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
42
cmd/go-omni/apigen/i18n_test.go
Normal file
42
cmd/go-omni/apigen/i18n_test.go
Normal file
@@ -0,0 +1,42 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestI18nTranslationMetadataAndVersionBoundary(t *testing.T) {
|
||||
spec, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/i18n.api"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
compiled, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ref := compiled.Spec.Errors[0].Translation
|
||||
if ref == nil || ref.Namespace != "limits" || ref.Key != "exceeded" {
|
||||
t.Fatal(ref)
|
||||
}
|
||||
if err := checkI18nRuntime(spec, []byte("module example.com/test\ngo 1.22\nrequire "+frameworkModule+" v0.25.0-preview\n"), t.TempDir()); err == nil {
|
||||
t.Fatal("old runtime accepted")
|
||||
}
|
||||
generated, _, err := renderProject(Options{Module: "example.com/test"}, compiled, false)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(generated["internal/apierrors/errorsGen.go"]), "translation") || !strings.Contains(string(generated["docs/openapi.json"]), "translation") {
|
||||
t.Fatal("translation metadata lost")
|
||||
}
|
||||
bad := readFile(t, filepath.Join(frameworkRoot(t), "examples/contracts/i18n.api"))
|
||||
bad = strings.Replace(bad, `"namespace":"limits"`, `"namespace":""`, 1)
|
||||
parsed, err := parseText(t, bad)
|
||||
if err == nil {
|
||||
_, err = Compile(parsed)
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatal("invalid translation reference accepted")
|
||||
}
|
||||
}
|
||||
func TestI18nGeneratedConsumer(t *testing.T) { testCatalogConsumer(t, "i18n.api", "i18n") }
|
||||
@@ -176,6 +176,9 @@ func generateManagedTestProject(t *testing.T, o Options) error {
|
||||
if o.Check {
|
||||
args = append(args, "--check")
|
||||
}
|
||||
if o.DryRun {
|
||||
args = append(args, "--dry-run")
|
||||
}
|
||||
cmd := exec.CommandContext(ctx, cli, args...)
|
||||
cmd.Dir = filepath.Dir(o.Dir)
|
||||
if data, err := cmd.CombinedOutput(); err != nil {
|
||||
|
||||
@@ -1,433 +0,0 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"go/ast"
|
||||
"go/format"
|
||||
goparser "go/parser"
|
||||
gotoken "go/token"
|
||||
"go/types"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"golang.org/x/mod/module"
|
||||
)
|
||||
|
||||
// GoTypeRef names an existing, exported, non-generic Go struct.
|
||||
type GoTypeRef struct {
|
||||
Package string `json:"package"`
|
||||
Type string `json:"type"`
|
||||
}
|
||||
type GoFunctionRef struct {
|
||||
Package string `json:"package"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
type MappingField struct {
|
||||
Source string `json:"source"`
|
||||
Convert *GoFunctionRef `json:"convert,omitempty"`
|
||||
Mapping string `json:"mapping,omitempty"`
|
||||
Elements bool `json:"elements,omitempty"`
|
||||
}
|
||||
type Mapping struct {
|
||||
Name string `json:"name"`
|
||||
Source GoTypeRef `json:"source"`
|
||||
Target GoTypeRef `json:"target"`
|
||||
Fields map[string]MappingField `json:"fields,omitempty"`
|
||||
IgnoreSource map[string]string `json:"ignoreSource,omitempty"`
|
||||
IgnoreTarget map[string]string `json:"ignoreTarget,omitempty"`
|
||||
}
|
||||
type mappingField struct {
|
||||
path, name string
|
||||
typ types.Type
|
||||
}
|
||||
type mappingTypes struct{ source, target types.Type }
|
||||
type mappingRenderer struct {
|
||||
importer types.Importer
|
||||
imports map[string]string
|
||||
definitions map[string]Mapping
|
||||
types map[string]mappingTypes
|
||||
}
|
||||
|
||||
func renderMappings(o Options, generated, seeds map[string][]byte) error {
|
||||
if len(o.generation.Mappings) == 0 {
|
||||
return nil
|
||||
}
|
||||
refs, err := validateMappings(o.generation.Mappings, o.Module)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
importer, close, err := loadMappingTypes(o, refs, generated, seeds)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer close()
|
||||
r := mappingRenderer{importer: importer, imports: map[string]string{}, definitions: map[string]Mapping{}, types: map[string]mappingTypes{}}
|
||||
for _, m := range o.generation.Mappings {
|
||||
source, err := r.lookup(m.Source)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
target, err := r.lookup(m.Target)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
r.definitions[m.Name] = m
|
||||
r.types[m.Name] = mappingTypes{source, target}
|
||||
}
|
||||
names := make([]string, 0, len(r.definitions))
|
||||
for name := range r.definitions {
|
||||
names = append(names, name)
|
||||
}
|
||||
sort.Strings(names)
|
||||
var body strings.Builder
|
||||
for _, name := range names {
|
||||
source, err := r.render(r.definitions[name])
|
||||
if err != nil {
|
||||
return fmt.Errorf("mapping %s: %w", name, err)
|
||||
}
|
||||
body.WriteString(source)
|
||||
}
|
||||
imports := make([]string, 0, len(r.imports))
|
||||
for path := range r.imports {
|
||||
imports = append(imports, path)
|
||||
}
|
||||
sort.Strings(imports)
|
||||
text := generatedHeader + "package mapping\nimport(\n"
|
||||
for _, path := range imports {
|
||||
text += r.imports[path] + " " + strconv.Quote(path) + "\n"
|
||||
}
|
||||
text += ")\n" + body.String()
|
||||
code, err := format.Source([]byte(text))
|
||||
if err != nil {
|
||||
return fmt.Errorf("format mappings: %w", err)
|
||||
}
|
||||
// The generated package is type-checked before it is written. No application
|
||||
// code (including init or converters) is executed during generation.
|
||||
fset := gotoken.NewFileSet()
|
||||
file, err := goparser.ParseFile(fset, "mappingsGen.go", code, 0)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
check := types.Config{Importer: importer}
|
||||
if _, err = check.Check(o.Module+"/internal/mapping", fset, []*ast.File{file}, nil); err != nil {
|
||||
return fmt.Errorf("generated mapping type check: %w", err)
|
||||
}
|
||||
generated["internal/mapping/mappingsGen.go"] = code
|
||||
return nil
|
||||
}
|
||||
|
||||
func validateMappings(defs []Mapping, project string) ([]string, error) {
|
||||
refs := map[string]bool{}
|
||||
names := map[string]bool{}
|
||||
validPackage := func(path string) bool {
|
||||
if module.CheckImportPath(path) != nil || path == project+"/internal/mapping" {
|
||||
return false
|
||||
}
|
||||
// go/types does not enforce the compiler's internal import visibility.
|
||||
if strings.HasPrefix(path, "internal/") {
|
||||
return false
|
||||
}
|
||||
if i := strings.LastIndex(path, "/internal/"); i >= 0 {
|
||||
return strings.HasPrefix(project+"/internal/mapping", path[:i]+"/")
|
||||
}
|
||||
return true
|
||||
}
|
||||
for _, m := range defs {
|
||||
if !gotoken.IsIdentifier(m.Name) || !gotoken.IsExported(m.Name) || names[m.Name] {
|
||||
return nil, fmt.Errorf("mapping name %q must be unique and exported", m.Name)
|
||||
}
|
||||
names[m.Name] = true
|
||||
for _, ref := range []GoTypeRef{m.Source, m.Target} {
|
||||
if !validPackage(ref.Package) || !gotoken.IsIdentifier(ref.Type) || !gotoken.IsExported(ref.Type) {
|
||||
return nil, fmt.Errorf("mapping %s: invalid Go type reference", m.Name)
|
||||
}
|
||||
refs[ref.Package] = true
|
||||
}
|
||||
if len(m.Fields) > 4096 || len(m.IgnoreSource) > 4096 || len(m.IgnoreTarget) > 4096 {
|
||||
return nil, fmt.Errorf("mapping %s: field rule limit exceeded", m.Name)
|
||||
}
|
||||
for target, f := range m.Fields {
|
||||
if target == "" || f.Source == "" {
|
||||
return nil, fmt.Errorf("mapping %s: explicit field rule requires target and source paths", m.Name)
|
||||
}
|
||||
if f.Convert != nil {
|
||||
if f.Mapping != "" || !validPackage(f.Convert.Package) || !gotoken.IsIdentifier(f.Convert.Name) || !gotoken.IsExported(f.Convert.Name) {
|
||||
return nil, fmt.Errorf("mapping %s.%s: invalid or conflicting converter", m.Name, target)
|
||||
}
|
||||
refs[f.Convert.Package] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
visiting, done := map[string]bool{}, map[string]bool{}
|
||||
byName := map[string]Mapping{}
|
||||
for _, m := range defs {
|
||||
byName[m.Name] = m
|
||||
}
|
||||
var visit func(string) error
|
||||
visit = func(name string) error {
|
||||
if done[name] {
|
||||
return nil
|
||||
}
|
||||
if visiting[name] {
|
||||
return fmt.Errorf("mapping cycle at %s", name)
|
||||
}
|
||||
m, ok := byName[name]
|
||||
if !ok {
|
||||
return fmt.Errorf("unknown mapping %s", name)
|
||||
}
|
||||
visiting[name] = true
|
||||
for _, f := range m.Fields {
|
||||
if f.Mapping != "" {
|
||||
if err := visit(f.Mapping); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
delete(visiting, name)
|
||||
done[name] = true
|
||||
return nil
|
||||
}
|
||||
for _, m := range defs {
|
||||
if err := visit(m.Name); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
out := make([]string, 0, len(refs))
|
||||
for path := range refs {
|
||||
out = append(out, path)
|
||||
}
|
||||
sort.Strings(out)
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (r *mappingRenderer) qualifier(p *types.Package) string {
|
||||
if alias, ok := r.imports[p.Path()]; ok {
|
||||
return alias
|
||||
}
|
||||
alias := fmt.Sprintf("pkg%d", len(r.imports))
|
||||
r.imports[p.Path()] = alias
|
||||
return alias
|
||||
}
|
||||
func (r *mappingRenderer) typeName(t types.Type) string { return types.TypeString(t, r.qualifier) }
|
||||
func (r *mappingRenderer) lookup(ref GoTypeRef) (types.Type, error) {
|
||||
pkg, err := r.importer.Import(ref.Package)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("mapping type %s.%s: %w", ref.Package, ref.Type, err)
|
||||
}
|
||||
obj, ok := pkg.Scope().Lookup(ref.Type).(*types.TypeName)
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("mapping type %s.%s not found", ref.Package, ref.Type)
|
||||
}
|
||||
typ := obj.Type()
|
||||
if n, ok := typ.(*types.Named); ok && n.TypeParams() != nil && n.TypeParams().Len() > 0 {
|
||||
return nil, fmt.Errorf("mapping root %s must be a non-generic struct", ref.Type)
|
||||
}
|
||||
if _, ok := typ.Underlying().(*types.Struct); !ok {
|
||||
return nil, fmt.Errorf("mapping root %s must be a struct", ref.Type)
|
||||
}
|
||||
return typ, nil
|
||||
}
|
||||
|
||||
func mappingFields(typ types.Type) (map[string]mappingField, error) {
|
||||
out := map[string]mappingField{}
|
||||
leaves := map[string]string{}
|
||||
var walk func(types.Type, string, int) error
|
||||
walk = func(typ types.Type, prefix string, depth int) error {
|
||||
if depth > 32 {
|
||||
return fmt.Errorf("embedded field nesting exceeds 32")
|
||||
}
|
||||
st, ok := typ.Underlying().(*types.Struct)
|
||||
if !ok {
|
||||
return fmt.Errorf("embedded field %s must be a value struct", prefix)
|
||||
}
|
||||
for i := 0; i < st.NumFields(); i++ {
|
||||
f := st.Field(i)
|
||||
path := prefix + f.Name()
|
||||
if !f.Exported() {
|
||||
return fmt.Errorf("unexported field %s cannot be mapped; use an explicit converter on the containing value", path)
|
||||
}
|
||||
if f.Embedded() {
|
||||
if err := walk(f.Type(), path+".", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if previous, exists := leaves[f.Name()]; exists {
|
||||
return fmt.Errorf("ambiguous embedded fields %s and %s", previous, path)
|
||||
}
|
||||
leaves[f.Name()] = path
|
||||
out[path] = mappingField{path, f.Name(), f.Type()}
|
||||
if len(out) > 4096 {
|
||||
return fmt.Errorf("mapping field limit exceeded")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := walk(typ, "", 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func (r *mappingRenderer) render(m Mapping) (string, error) {
|
||||
mt := r.types[m.Name]
|
||||
source, err := mappingFields(mt.source)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
target, err := mappingFields(mt.target)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
for key, reason := range m.IgnoreSource {
|
||||
if _, ok := source[key]; !ok || strings.TrimSpace(reason) == "" {
|
||||
return "", fmt.Errorf("ignoreSource %s requires an existing field and reason", key)
|
||||
}
|
||||
}
|
||||
for key, reason := range m.IgnoreTarget {
|
||||
if _, ok := target[key]; !ok || strings.TrimSpace(reason) == "" {
|
||||
return "", fmt.Errorf("ignoreTarget %s requires an existing field and reason", key)
|
||||
}
|
||||
}
|
||||
for key := range m.Fields {
|
||||
if _, ok := target[key]; !ok {
|
||||
return "", fmt.Errorf("unknown target field %s", key)
|
||||
}
|
||||
if _, ignored := m.IgnoreTarget[key]; ignored {
|
||||
return "", fmt.Errorf("target %s is both mapped and ignored", key)
|
||||
}
|
||||
}
|
||||
names := make([]string, 0, len(target))
|
||||
for name := range target {
|
||||
names = append(names, name)
|
||||
}
|
||||
sort.Strings(names)
|
||||
used := map[string]bool{}
|
||||
text := fmt.Sprintf("// %s maps one value. Pointer/map/slice assignments retain Go reference semantics.\nfunc %s(source %s)(%s,error){\nvar result %s\n", m.Name, m.Name, r.typeName(mt.source), r.typeName(mt.target), r.typeName(mt.target))
|
||||
for _, path := range names {
|
||||
dst := target[path]
|
||||
if reason, ignored := m.IgnoreTarget[path]; ignored {
|
||||
text += "// Target " + path + " is intentionally left at zero; see omni.gen.json.\n"
|
||||
_ = reason
|
||||
continue
|
||||
}
|
||||
rule, explicit := m.Fields[path]
|
||||
if !explicit {
|
||||
for key, f := range source {
|
||||
if f.name == dst.name {
|
||||
rule.Source = key
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
src, ok := source[rule.Source]
|
||||
if !ok {
|
||||
return "", fmt.Errorf("target %s has no source; add a field rule or ignoreTarget reason", path)
|
||||
}
|
||||
if _, ignored := m.IgnoreSource[src.path]; ignored {
|
||||
return "", fmt.Errorf("source %s is both mapped and ignored", src.path)
|
||||
}
|
||||
used[src.path] = true
|
||||
from, to := src.typ, dst.typ
|
||||
srcExpr, dstExpr := "source."+src.path, "result."+path
|
||||
if rule.Elements {
|
||||
switch d := to.Underlying().(type) {
|
||||
case *types.Slice:
|
||||
s, ok := from.Underlying().(*types.Slice)
|
||||
if !ok {
|
||||
return "", fmt.Errorf("%s: elements requires slice to slice or equal-length array to array", path)
|
||||
}
|
||||
text += fmt.Sprintf("if %s!=nil {\n%s=make(%s,len(%s))\nfor index:=range %s {\n", srcExpr, dstExpr, r.typeName(to), srcExpr, srcExpr)
|
||||
from, to = s.Elem(), d.Elem()
|
||||
case *types.Array:
|
||||
s, ok := from.Underlying().(*types.Array)
|
||||
if !ok || s.Len() != d.Len() {
|
||||
return "", fmt.Errorf("%s: array lengths must match", path)
|
||||
}
|
||||
text += fmt.Sprintf("for index:=range %s {\n", srcExpr)
|
||||
from, to = s.Elem(), d.Elem()
|
||||
default:
|
||||
return "", fmt.Errorf("%s: elements requires a slice or array", path)
|
||||
}
|
||||
srcExpr += "[index]"
|
||||
dstExpr += "[index]"
|
||||
}
|
||||
expr, fallible, err := r.expression(rule, from, to, srcExpr)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("%s -> %s: %w", src.path, path, err)
|
||||
}
|
||||
if fallible {
|
||||
fmtPkg, err := r.importer.Import("fmt")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
alias := r.qualifier(fmtPkg)
|
||||
message := m.Name + "." + path + ": %w"
|
||||
args := "err"
|
||||
if rule.Elements {
|
||||
message = m.Name + "." + path + "[%d]: %w"
|
||||
args = "index,err"
|
||||
}
|
||||
text += fmt.Sprintf("{ value,err:=%s\nif err!=nil{var zero %s;return zero,%s.Errorf(%q,%s)}\n%s=value\n}\n", expr, r.typeName(mt.target), alias, message, args, dstExpr)
|
||||
} else {
|
||||
text += dstExpr + "=" + expr + "\n"
|
||||
}
|
||||
if rule.Elements {
|
||||
text += "}\n"
|
||||
if _, ok := dst.typ.Underlying().(*types.Slice); ok {
|
||||
text += "}\n"
|
||||
}
|
||||
}
|
||||
}
|
||||
keys := make([]string, 0, len(source))
|
||||
for key := range source {
|
||||
keys = append(keys, key)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
for _, key := range keys {
|
||||
if !used[key] {
|
||||
if _, ignored := m.IgnoreSource[key]; !ignored {
|
||||
return "", fmt.Errorf("unmapped source field %s; map it or provide an ignoreSource reason", key)
|
||||
}
|
||||
}
|
||||
}
|
||||
text += "return result,nil\n}\n"
|
||||
return text, nil
|
||||
}
|
||||
|
||||
func (r *mappingRenderer) expression(rule MappingField, from, to types.Type, source string) (string, bool, error) {
|
||||
if rule.Mapping != "" {
|
||||
m := r.types[rule.Mapping]
|
||||
if !types.Identical(from, m.source) || !types.Identical(to, m.target) {
|
||||
return "", false, fmt.Errorf("nested mapping %s has incompatible types", rule.Mapping)
|
||||
}
|
||||
return rule.Mapping + "(" + source + ")", true, nil
|
||||
}
|
||||
if rule.Convert != nil {
|
||||
pkg, err := r.importer.Import(rule.Convert.Package)
|
||||
if err != nil {
|
||||
return "", false, err
|
||||
}
|
||||
fn, ok := pkg.Scope().Lookup(rule.Convert.Name).(*types.Func)
|
||||
if !ok {
|
||||
return "", false, fmt.Errorf("converter function %s not found", rule.Convert.Name)
|
||||
}
|
||||
sig := fn.Type().(*types.Signature)
|
||||
if sig.Variadic() || (sig.TypeParams() != nil && sig.TypeParams().Len() > 0) || sig.Params().Len() != 1 || sig.Results().Len() < 1 || sig.Results().Len() > 2 {
|
||||
return "", false, fmt.Errorf("converter must have signature func(Source) Target or func(Source) (Target,error)")
|
||||
}
|
||||
if !types.Identical(sig.Params().At(0).Type(), from) || !types.Identical(sig.Results().At(0).Type(), to) {
|
||||
return "", false, fmt.Errorf("converter signature does not match field types")
|
||||
}
|
||||
fallible := sig.Results().Len() == 2
|
||||
if fallible && !types.Identical(sig.Results().At(1).Type(), types.Universe.Lookup("error").Type()) {
|
||||
return "", false, fmt.Errorf("converter second result must be error")
|
||||
}
|
||||
return r.qualifier(pkg) + "." + rule.Convert.Name + "(" + source + ")", fallible, nil
|
||||
}
|
||||
if !types.Identical(from, to) {
|
||||
return "", false, fmt.Errorf("types %s and %s differ; provide an explicit converter", types.TypeString(from, nil), types.TypeString(to, nil))
|
||||
}
|
||||
return source, false, nil
|
||||
}
|
||||
@@ -1,175 +0,0 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/importer"
|
||||
gotoken "go/token"
|
||||
"go/types"
|
||||
"io"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"golang.org/x/mod/modfile"
|
||||
)
|
||||
|
||||
// loadMappingTypes uses the Go compiler's export data, including the pending
|
||||
// DTOs via an overlay. Module files and sum updates remain in a temporary dir.
|
||||
// It compiles only explicitly selected packages and their dependencies; never
|
||||
// runs project code. Build tags/GOOS follow the caller; GOWORK is disabled.
|
||||
func loadMappingTypes(o Options, refs []string, generated, seeds map[string][]byte) (types.Importer, func(), error) {
|
||||
// go list resolves its working directory (notably /var -> /private/var on
|
||||
// macOS). Overlay keys must name that same physical path or stale DTOs win.
|
||||
o.Dir = canonical(o.Dir)
|
||||
temp, err := os.MkdirTemp("", "go-omni-mapping-*")
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
cleanup := func() { _ = os.RemoveAll(temp) }
|
||||
fail := func(err error) (types.Importer, func(), error) { cleanup(); return nil, nil, err }
|
||||
raw, err := os.ReadFile(filepath.Join(o.Dir, "go.mod"))
|
||||
if err != nil {
|
||||
return fail(errors.New("mapping generation requires an existing Go module; scaffold first and resolve dependencies"))
|
||||
}
|
||||
mf, err := modfile.Parse("go.mod", raw, nil)
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
replacements := append([]*modfile.Replace(nil), mf.Replace...)
|
||||
for _, r := range replacements {
|
||||
if r.New.Version == "" && !filepath.IsAbs(r.New.Path) {
|
||||
if err = mf.AddReplace(r.Old.Path, r.Old.Version, filepath.Join(o.Dir, r.New.Path), ""); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
raw, err = mf.Format()
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
modPath := filepath.Join(temp, "go.mod")
|
||||
if err = os.WriteFile(modPath, raw, 0600); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
if sums, e := os.ReadFile(filepath.Join(o.Dir, "go.sum")); e == nil {
|
||||
if err = os.WriteFile(filepath.Join(temp, "go.sum"), sums, 0600); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
return fail(e)
|
||||
}
|
||||
overlay := map[string]string{}
|
||||
// Hide stale owned Go files, which may otherwise redeclare a renamed DTO.
|
||||
var previous manifest
|
||||
if data, e := os.ReadFile(filepath.Join(o.Dir, manifestPath)); e == nil {
|
||||
if err = json.Unmarshal(data, &previous); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
for name := range previous.Generated {
|
||||
if strings.HasSuffix(name, ".go") {
|
||||
if !ownedPath(name) {
|
||||
return fail(fmt.Errorf("invalid owned path %s", name))
|
||||
}
|
||||
if err = safePath(o.Dir, name); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
if _, ok := generated[name]; !ok {
|
||||
overlay[filepath.Join(o.Dir, filepath.FromSlash(name))] = ""
|
||||
}
|
||||
}
|
||||
}
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
return fail(e)
|
||||
}
|
||||
for i, files := range []map[string][]byte{generated, seeds} {
|
||||
names := make([]string, 0, len(files))
|
||||
for name := range files {
|
||||
if strings.HasSuffix(name, ".go") {
|
||||
names = append(names, name)
|
||||
}
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, name := range names {
|
||||
if err = safePath(o.Dir, name); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
path := filepath.Join(o.Dir, filepath.FromSlash(name))
|
||||
if i == 1 {
|
||||
if _, e := os.Stat(path); e == nil {
|
||||
continue
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
return fail(e)
|
||||
}
|
||||
}
|
||||
backing := filepath.Join(temp, fmt.Sprintf("overlay%d.go", len(overlay)))
|
||||
if err = os.WriteFile(backing, files[name], 0600); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
overlay[path] = backing
|
||||
}
|
||||
}
|
||||
raw, err = json.Marshal(map[string]any{"Replace": overlay})
|
||||
if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
overlayPath := filepath.Join(temp, "overlay.json")
|
||||
if err = os.WriteFile(overlayPath, raw, 0600); err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
args := []string{"list", "-deps", "-export", "-json", "-mod=readonly", "-modfile", modPath, "-overlay", overlayPath}
|
||||
// fmt is used for fallible conversions even if neither struct imports it.
|
||||
args = append(args, "fmt")
|
||||
args = append(args, refs...)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", args...)
|
||||
cmd.Dir = o.Dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
cmd.WaitDelay = 2 * time.Second
|
||||
stdout := &limitedCatalogBuffer{limit: 32 << 20}
|
||||
stderr := &limitedCatalogBuffer{limit: 64 << 10}
|
||||
cmd.Stdout = stdout
|
||||
cmd.Stderr = stderr
|
||||
if err = cmd.Run(); err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return fail(fmt.Errorf("mapping type loading: %w", ctx.Err()))
|
||||
}
|
||||
// Go download errors may contain authenticated remote URLs. Keep arbitrary
|
||||
// command output out of diagnostics; exact mapping errors follow type loading.
|
||||
return fail(fmt.Errorf("mapping type loading failed (%v); run go list -deps -export for the configured source/target/converter packages; check dependencies, build tags and import cycles", err))
|
||||
}
|
||||
exports := map[string]string{}
|
||||
dec := json.NewDecoder(bytes.NewReader(stdout.Bytes()))
|
||||
for {
|
||||
var p struct {
|
||||
ImportPath, Export string
|
||||
Error *struct{ Err string }
|
||||
}
|
||||
if err = dec.Decode(&p); err == io.EOF {
|
||||
break
|
||||
} else if err != nil {
|
||||
return fail(err)
|
||||
}
|
||||
if p.Error != nil {
|
||||
return fail(fmt.Errorf("mapping package %s could not be compiled", p.ImportPath))
|
||||
}
|
||||
if p.Export != "" {
|
||||
exports[p.ImportPath] = p.Export
|
||||
}
|
||||
}
|
||||
imp := importer.ForCompiler(gotoken.NewFileSet(), "gc", func(path string) (io.ReadCloser, error) {
|
||||
file, ok := exports[path]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("no export data for mapping dependency %s", path)
|
||||
}
|
||||
return os.Open(file)
|
||||
})
|
||||
return imp, cleanup, nil
|
||||
}
|
||||
@@ -1,266 +0,0 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func compositionConfig() GeneratorConfig {
|
||||
pkg := "example.com/composition/internal/"
|
||||
cfg := GeneratorConfig{Repository: "none", Middleware: "factories"}
|
||||
for _, name := range []string{"Enterprise", "Tenant", "Admin"} {
|
||||
fields := map[string]MappingField{
|
||||
"EmployeeName": {Source: "Name", Convert: &GoFunctionRef{pkg + "convert", "Name"}},
|
||||
"Salary": {Source: "Salary", Convert: &GoFunctionRef{pkg + "convert", "Money"}},
|
||||
"Enabled": {Source: "Enabled", Convert: &GoFunctionRef{pkg + "convert", "Bool"}},
|
||||
}
|
||||
m := Mapping{Name: "From" + name, Source: GoTypeRef{pkg + "types", name + "Request"}, Target: GoTypeRef{pkg + "domain", "Employee"}, Fields: fields}
|
||||
if name != "Enterprise" {
|
||||
m.IgnoreSource = map[string]string{"CompanyID": "Authorized separately by Logic; never copy path scope into body"}
|
||||
}
|
||||
cfg.Mappings = append(cfg.Mappings, m)
|
||||
}
|
||||
cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToResponse", Source: GoTypeRef{pkg + "domain", "Result"}, Target: GoTypeRef{pkg + "types", "EmployeeResponse"}, Fields: map[string]MappingField{"ID": {Source: "ID", Convert: &GoFunctionRef{pkg + "convert", "ID"}}}})
|
||||
cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToRow", Source: GoTypeRef{pkg + "domain", "Row"}, Target: GoTypeRef{pkg + "domain", "RowDTO"}})
|
||||
cfg.Mappings = append(cfg.Mappings, Mapping{Name: "ToBatch", Source: GoTypeRef{pkg + "domain", "Batch"}, Target: GoTypeRef{pkg + "domain", "BatchDTO"}, Fields: map[string]MappingField{
|
||||
"Rows": {Source: "Rows", Mapping: "ToRow", Elements: true}, "Fixed": {Source: "Fixed", Mapping: "ToRow", Elements: true},
|
||||
}})
|
||||
return cfg
|
||||
}
|
||||
func writeGeneratorConfig(t *testing.T, dir string, cfg GeneratorConfig) {
|
||||
t.Helper()
|
||||
raw, err := json.MarshalIndent(cfg, "", " ")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, generatorConfigFile), string(raw)+"\n")
|
||||
}
|
||||
func runCompositionGo(t *testing.T, dir string, args ...string) {
|
||||
t.Helper()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", args...)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("go %v: %v\n%s", args, err, out)
|
||||
}
|
||||
}
|
||||
|
||||
func TestCompositionMappingConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
o := Options{API: filepath.Join(frameworkRoot(t), "cmd/go-omni/apigen/testdata/composition/service.api"), Dir: dir, Module: "example.com/composition", FrameworkDir: frameworkRoot(t)}
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
fixtures := map[string]string{
|
||||
"internal/amount/money.go": "amount.go", "internal/domain/employee.go": "domain.go", "internal/convert/fields.go": "converters.go",
|
||||
"internal/middleware/dependencies.go": "dependencies.go", "internal/middleware/accessFactory.go": "accessFactory.go", "internal/svc/client.go": "client.go", "consumer_test.go": "consumer_test.go",
|
||||
}
|
||||
for path, fixture := range fixtures {
|
||||
writeFile(t, filepath.Join(dir, path), readFile(t, "testdata/composition/"+fixture))
|
||||
}
|
||||
svcFile := filepath.Join(dir, "internal/svc/serviceContext.go")
|
||||
source := strings.Replace(readFile(t, svcFile), "type ServiceContext struct {", "type ServiceContext struct {\n client *Client", 1)
|
||||
source = strings.Replace(source, "return nil", "return c.initClient(ctx)", 1)
|
||||
writeFile(t, svcFile, source)
|
||||
configFile := filepath.Join(dir, "internal/config/config.go")
|
||||
writeFile(t, configFile, strings.Replace(readFile(t, configFile), "type Config struct {", "type Config struct {\n FactoryMode string `json:\"factoryMode\"`", 1))
|
||||
// Resolve dependencies before opting into compile-time mapping. The first
|
||||
// generation creates DTOs; the second sees current DTOs through an overlay.
|
||||
// Logic does not yet import the generated mapping package.
|
||||
testSource := readFile(t, filepath.Join(dir, "consumer_test.go"))
|
||||
if err := os.Remove(filepath.Join(dir, "consumer_test.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
runCompositionGo(t, dir, "mod", "tidy")
|
||||
cfg := compositionConfig()
|
||||
writeGeneratorConfig(t, dir, cfg)
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "consumer_test.go"), testSource)
|
||||
for _, group := range []string{"Enterprise", "Tenant", "Admin"} {
|
||||
text := fmt.Sprintf(`package %s
|
||||
import("context";"example.com/composition/internal/types";"example.com/composition/internal/mapping";"example.com/composition/internal/domain")
|
||||
func(s *Logic)Save(ctx context.Context,req *types.%sRequest)(*types.EmployeeResponse,error){
|
||||
input,err:=mapping.From%s(*req);if err!=nil{return nil,err}
|
||||
result,err:=mapping.ToResponse(domain.Result{Employee:input,ID:9007199254740993});if err!=nil{return nil,err};return &result,nil
|
||||
}
|
||||
`, strings.ToLower(group), group, group)
|
||||
writeFile(t, filepath.Join(dir, "internal/logic", strings.ToLower(group), "save.go"), text)
|
||||
}
|
||||
before := snapshot(t, dir)
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("regeneration changed project")
|
||||
}
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("check changed project")
|
||||
}
|
||||
runCompositionGo(t, dir, "test", "-race", "-count=1", "./...")
|
||||
runCompositionGo(t, dir, "build", ".")
|
||||
generated := readFile(t, filepath.Join(dir, "internal/mapping/mappingsGen.go"))
|
||||
if strings.Contains(generated, "encoding/json") || strings.Contains(generated, "reflect") || !strings.Contains(generated, "result.EmployeeName = value") {
|
||||
t.Fatal("mapping does not emit typed assignments")
|
||||
}
|
||||
// Adding a source or target field is detected before any files are written.
|
||||
for _, tc := range []struct{ name, path, before, after, want string }{
|
||||
{"source", o.API, "Name *string", "NewField *string `json:\"newField,optional\"`\n Name *string", "unmapped source field NewField"},
|
||||
{"target", filepath.Join(dir, "internal/domain/employee.go"), "EmployeeName string", "Unmapped string\n EmployeeName string", "target Unmapped has no source"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
original := readFile(t, tc.path)
|
||||
writeFile(t, tc.path, strings.Replace(original, tc.before, tc.after, 1))
|
||||
defer writeFile(t, tc.path, original)
|
||||
snap := snapshot(t, dir)
|
||||
if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(snap, snapshot(t, dir)) {
|
||||
t.Fatal("failed check wrote files")
|
||||
}
|
||||
o.Check = false
|
||||
defer func() { o.Check = true }()
|
||||
if err := generateManagedTestProject(t, o); err == nil {
|
||||
t.Fatal("generate accepted missing mapping")
|
||||
}
|
||||
if !sameFiles(snap, snapshot(t, dir)) {
|
||||
t.Fatal("failed generate wrote files")
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
edit func(*GeneratorConfig)
|
||||
want string
|
||||
}{
|
||||
{"pointerPolicy", func(c *GeneratorConfig) { delete(c.Mappings[0].Fields, "Salary") }, "provide an explicit converter"},
|
||||
{"unknownSource", func(c *GeneratorConfig) { c.Mappings[0].Fields["Salary"] = MappingField{Source: "Missing"} }, "has no source"},
|
||||
{"wrongConverter", func(c *GeneratorConfig) {
|
||||
f := c.Mappings[0].Fields["Salary"]
|
||||
f.Convert.Name = "ID"
|
||||
c.Mappings[0].Fields["Salary"] = f
|
||||
}, "signature does not match"},
|
||||
{"unhandledScope", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource = nil }, "unmapped source field CompanyID"},
|
||||
{"badIgnore", func(c *GeneratorConfig) { c.Mappings[1].IgnoreSource["Missing"] = "stale rule" }, "ignoreSource Missing"},
|
||||
{"arrayLength", func(c *GeneratorConfig) {
|
||||
f := c.Mappings[5].Fields["Fixed"]
|
||||
f.Source = "Rows"
|
||||
c.Mappings[5].Fields["Fixed"] = f
|
||||
}, "array lengths must match"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
changed := compositionConfig()
|
||||
tc.edit(&changed)
|
||||
writeGeneratorConfig(t, dir, changed)
|
||||
defer writeGeneratorConfig(t, dir, cfg)
|
||||
snap := snapshot(t, dir)
|
||||
if err := generateManagedTestProject(t, o); err == nil || !strings.Contains(err.Error(), tc.want) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(snap, snapshot(t, dir)) {
|
||||
t.Fatal("invalid rule wrote files")
|
||||
}
|
||||
})
|
||||
}
|
||||
// All diagnostics exercise current DTOs, not stale exported generated types.
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Exercise the repository switch with the same actual CLI as the mapper.
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/repository")); !os.IsNotExist(err) {
|
||||
t.Fatalf("default repository skeleton: %v", err)
|
||||
}
|
||||
o.Check = false
|
||||
cfg.Repository = "group"
|
||||
writeGeneratorConfig(t, dir, cfg)
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
repo := filepath.Join(dir, "internal/repository/enterprise/repository.go")
|
||||
writeFile(t, repo, readFile(t, repo)+"\n// Project-owned persistence.\n")
|
||||
cfg.Repository = "none"
|
||||
writeGeneratorConfig(t, dir, cfg)
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(readFile(t, repo), "Project-owned persistence.") {
|
||||
t.Fatal("disabling skeleton overwrote business code")
|
||||
}
|
||||
o.Check = true
|
||||
if err := generateManagedTestProject(t, o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if os.Getenv("GO_OMNI_RELEASE_CLI") != "" {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
data, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatal(err, string(data))
|
||||
}
|
||||
var dependency struct {
|
||||
Version, Sum string
|
||||
Replace any
|
||||
}
|
||||
if err = json.Unmarshal(data, &dependency); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if dependency.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || dependency.Replace != nil || dependency.Sum == "" {
|
||||
t.Fatal("not a fixed remote consumer")
|
||||
}
|
||||
t.Logf("remote framework %s checksum %s; no local replace", dependency.Version, dependency.Sum)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMappingConfigurationRejection(t *testing.T) {
|
||||
cases := []struct {
|
||||
name string
|
||||
edit func(*GeneratorConfig)
|
||||
want string
|
||||
}{
|
||||
{"cycle", func(c *GeneratorConfig) {
|
||||
c.Mappings[4].Fields = map[string]MappingField{"Value": {Source: "Value", Mapping: "ToRow"}}
|
||||
}, "cycle"},
|
||||
{"unknown", func(c *GeneratorConfig) {
|
||||
c.Mappings[5].Fields["Rows"] = MappingField{Source: "Rows", Mapping: "Missing", Elements: true}
|
||||
}, "unknown mapping"},
|
||||
{"conflict", func(c *GeneratorConfig) {
|
||||
f := c.Mappings[0].Fields["Salary"]
|
||||
f.Mapping = "ToRow"
|
||||
c.Mappings[0].Fields["Salary"] = f
|
||||
}, "conflicting converter"},
|
||||
{"internal", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.net/private/internal/types" }, "invalid Go type"},
|
||||
{"outputCycle", func(c *GeneratorConfig) { c.Mappings[0].Source.Package = "example.com/composition/internal/mapping" }, "invalid Go type"},
|
||||
{"duplicates", func(c *GeneratorConfig) { c.Mappings[1].Name = c.Mappings[0].Name }, "unique and exported"},
|
||||
}
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
cfg := compositionConfig()
|
||||
tc.edit(&cfg)
|
||||
if _, err := validateMappings(cfg.Mappings, "example.com/composition"); err == nil || !strings.Contains(err.Error(), tc.want) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -63,18 +63,23 @@ func TestAPIInfrastructureConsumers(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
test := `package config
|
||||
import("testing";"os")
|
||||
import("testing";"os";"strings")
|
||||
func TestConfiguration(t *testing.T){
|
||||
t.Setenv("GO_OMNI_DB_DSN","test:test@tcp(127.0.0.1:1)/test")
|
||||
t.Setenv("GO_OMNI_DB_DATABASE","test");t.Setenv("GO_OMNI_DB_USER","test");t.Setenv("GO_OMNI_DB_PASSWORD","test")
|
||||
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY","01234567890123456789012345678901")
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","test");t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","test")
|
||||
t.Setenv("GO_OMNI_STORAGE_BUCKET","test");t.Setenv("GO_OMNI_STORAGE_ENDPOINT","ENDPOINT")
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID","01234567890123456789012345678901")
|
||||
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
|
||||
t.Setenv("GO_OMNI_SERVER_LOG_FILE","./explicit.log");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!="./explicit.log"{t.Fatalf("canonical log file: %v",err)}
|
||||
t.Setenv("GO_OMNI_SERVER_LOG_OUTPUT","stderr");cfg,err=Load("");if err!=nil||cfg.Server.Log.Output!="stderr"{t.Fatalf("canonical output: %v",err)}
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");t.Setenv("GO_OMNI_SERVER_LOG_FILE","./implicit.log")
|
||||
cfg,err=Load("../../etc/config.yaml");if err!=nil||cfg.Server.Log.File!=Default().Server.Log.File{t.Fatalf("implicit environment override: %v",err)}
|
||||
raw,err:=os.ReadFile("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||
// Add explicit fields while keeping storage and DB requirements.
|
||||
raw=[]byte(strings.Replace(string(raw),"server:\n","server:\n log:\n output: '${OUTPUT:-stdout}'\n file: ''\n rotation:\n enabled: false\n",1))
|
||||
path:="explicit.yaml";if err=os.WriteFile(path,raw,0600);err!=nil{t.Fatal(err)};defer os.Remove(path)
|
||||
cfg,err=Load(path);if err!=nil||cfg.Server.Log.Output!="stdout"||cfg.Server.Log.File!=""||cfg.Server.Log.Rotation.Enabled{t.Fatalf("explicit file override: %v",err)}
|
||||
|
||||
_=os.Getenv("unused")
|
||||
}`
|
||||
endpoint := ""
|
||||
|
||||
@@ -19,8 +19,10 @@ import (
|
||||
"MODULE/internal/svc"
|
||||
)
|
||||
func main(){
|
||||
path:=flag.String("config","etc/config.yaml","configuration file; empty uses defaults and environment")
|
||||
path:=flag.String("config","etc/config.yaml","configuration file; empty uses only Default()")
|
||||
check:=flag.Bool("check-config",false,"validate configuration and exit without starting dependencies")
|
||||
flag.Parse()
|
||||
if *check {if _,err:=config.Load(*path);err!=nil{log.Fatal(err)};return}
|
||||
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err:=run(ctx,*path);err!=nil{log.Fatal(err)}
|
||||
@@ -56,10 +58,18 @@ factories for new projects. Named middleware factories run after svc.Init;
|
||||
implement Dependencies accessors on ServiceContext and New<Name>(deps) once.
|
||||
Unimplemented/nil factories fail startup and clean registered dependencies.
|
||||
Existing projects retain middleware: handlers until explicitly migrated.
|
||||
Optional mappings generate typed assignments in internal/mapping/mappingsGen.go;
|
||||
configure source/target types and explicit conversion rules after resolving module
|
||||
dependencies. Missing fields fail generation and --check. See GENERATION.md in
|
||||
the framework repository for composition, null/reference semantics and migration.
|
||||
Ordinary Logic uses the generated Request/Response types directly. Where an
|
||||
independent business type is needed, use ordinary Go or the optional copyx package.
|
||||
Configuration loading is generator-managed in internal/config/loadGen.go. Extend fields/defaults/Validate in config.go; additional decoders belong in loader.go/NewLoader. JSON/YAML/XML use explicit ${NAME} / ${NAME:-fallback} references; fixed values are never overridden by inferred environment names. Use -check-config to validate without opening dependencies. Framework validation
|
||||
identifies fields such as server.log.level and database.port. server.log.timeZone
|
||||
optionally selects a per-logger IANA zone. Optional server.cors is loaded and installed
|
||||
automatically before routing/authentication; disabled by default. Configure allowedOrigins,
|
||||
allowedMethods and allowedHeaders in the selected file; no main/handler edits.
|
||||
See the framework CONFIG.md and cors/README.md.
|
||||
|
||||
There are no mappings/mappingFiles settings or generated mapping files in v0.19.
|
||||
See COPY.md in the framework repository for copy policies and migration steps,
|
||||
and GENERATION.md for field composition and middleware factories.
|
||||
|
||||
## Run
|
||||
|
||||
@@ -74,7 +84,7 @@ Configure server.endpoints to disable or move each built-in route. Health
|
||||
responseMode is app by default (project format), or raw for plain health data.
|
||||
Probe HTTP statuses remain 200/503 even when business errors use HTTP 200.
|
||||
OpenAPI remains an unwrapped document; disabled endpoints reserve no paths.
|
||||
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
|
||||
The default file explicitly references GO_OMNI_TRANSPORT; set it to native or edit server.transport.
|
||||
The greeting starter works immediately; other new operation stubs return 501 until
|
||||
their business logic is implemented. Authentication hooks also fail closed with 501.
|
||||
|
||||
@@ -136,7 +146,7 @@ Full reference: https://git.yuntuo.hk/k2safe/go-omni/src/branch/main/DEVELOPMENT
|
||||
api/ # editable contract and imported types
|
||||
main.go # GENERATED startup, lifecycle and route registration
|
||||
etc/config.yaml # lowerCamelCase configuration
|
||||
internal/config/ # defaults -> file -> environment
|
||||
internal/config/ # defaults -> file references -> typed binding -> validation
|
||||
internal/svc/ # dependency initialization and cleanup
|
||||
internal/types/*Gen.go # DTOs split by source API, one types package
|
||||
internal/svc/runtimeGen.go # GENERATED lifecycle and infrastructure startup
|
||||
@@ -145,8 +155,7 @@ internal/handler/<group>/ # generated binding and Logic adapters
|
||||
internal/logic/<group>/ # developer-owned business operations
|
||||
internal/repository/<group>/ # optional; repository: group creates editable seeds
|
||||
internal/middleware/ # typed dependency interface, factories and registry
|
||||
omni.gen.json # developer-owned generator options and mappings
|
||||
internal/mapping/*Gen.go # optional compile-checked struct mappings
|
||||
omni.gen.json # developer-owned repository/middleware options
|
||||
docs/openapi.json # generated OpenAPI 3.0.3
|
||||
.goOmni/apiManifest.json # ownership and content hashes
|
||||
`+"```"+`
|
||||
@@ -238,13 +247,16 @@ func infrastructureREADME(o Options) string {
|
||||
|
||||
Selected database: %s; storage: %s; logging: %s/%s.
|
||||
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
|
||||
Use structured database/mysql configuration and GO_OMNI_DB_PASSWORD. Other connection/pool/TLS settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||
Use structured database/mysql configuration. The generated etc/config.yaml explicitly
|
||||
references GO_OMNI_DB_PASSWORD and other required connection values; add fixed
|
||||
values or explicit references for pool/TLS fields. No implicit environment overlay.
|
||||
No tables are invented or migrated automatically from request/response DTOs.
|
||||
The none/memory selection opens no database; implement repositories per business.
|
||||
|
||||
With storage enabled, services receive deps.Files, Uploads, Downloads and
|
||||
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
|
||||
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
|
||||
the generated file references GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and
|
||||
GO_OMNI_STORAGE_PUBLIC_URL. File values may instead be fixed explicitly.
|
||||
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
|
||||
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
|
||||
and authorize before issuing them. No public upload-link route is added.
|
||||
|
||||
@@ -18,7 +18,6 @@ import (
|
||||
"example.com/composition/internal/convert"
|
||||
"example.com/composition/internal/domain"
|
||||
"example.com/composition/internal/handler"
|
||||
"example.com/composition/internal/mapping"
|
||||
"example.com/composition/internal/middleware"
|
||||
"example.com/composition/internal/svc"
|
||||
"example.com/composition/internal/types"
|
||||
@@ -86,16 +85,16 @@ func TestHTTPComposition(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestFullFormAndMappingSemantics(t *testing.T) {
|
||||
func TestFullFormAndCopySemantics(t *testing.T) {
|
||||
name := "employee"
|
||||
money := amount.Money("2.00")
|
||||
yes := true
|
||||
req := types.EnterpriseRequest{Name: &name, Salary: &money, Enabled: &yes, Labels: []string{"one"}}
|
||||
first, e := mapping.FromEnterprise(req)
|
||||
first, e := convert.Employee(req)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
second, e := mapping.FromEnterprise(types.EnterpriseRequest{})
|
||||
second, e := convert.Employee(types.EnterpriseRequest{})
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
@@ -108,26 +107,26 @@ func TestFullFormAndMappingSemantics(t *testing.T) {
|
||||
t.Fatal("unexpected implicit deep copy")
|
||||
}
|
||||
req.Employer = &money
|
||||
first, e = mapping.FromEnterprise(req)
|
||||
first, e = convert.Employee(req)
|
||||
if e != nil || first.Employer != req.Employer {
|
||||
t.Fatal("nullable pointer changed", e)
|
||||
}
|
||||
bad := "reject"
|
||||
failed, e := mapping.FromEnterprise(types.EnterpriseRequest{Name: &bad})
|
||||
failed, e := convert.Employee(types.EnterpriseRequest{Name: &bad})
|
||||
if !errors.Is(e, convert.ErrName) || failed.EmployeeName != "" || !strings.Contains(e.Error(), "EmployeeName") {
|
||||
t.Fatal("converter cause/zero result lost", e)
|
||||
}
|
||||
empty, e := mapping.ToBatch(domain.Batch{})
|
||||
empty, e := convert.ToBatch(domain.Batch{})
|
||||
if e != nil || empty.Rows != nil {
|
||||
t.Fatal("nil slice changed", e)
|
||||
}
|
||||
src := domain.Batch{Rows: []domain.Row{}, Fixed: [2]domain.Row{{Value: "a"}, {Value: "b"}}, Optional: &name}
|
||||
out, e := mapping.ToBatch(src)
|
||||
out, e := convert.ToBatch(src)
|
||||
if e != nil || out.Rows == nil || len(out.Rows) != 0 || out.Fixed[1].Value != "b" || out.Optional != src.Optional {
|
||||
t.Fatal(out, e)
|
||||
}
|
||||
src.Rows = []domain.Row{{Value: "one"}}
|
||||
out, e = mapping.ToBatch(src)
|
||||
out, e = convert.ToBatch(src)
|
||||
if e != nil || out.Rows[0].Value != "one" {
|
||||
t.Fatal(out, e)
|
||||
}
|
||||
@@ -140,7 +139,7 @@ func TestFullFormAndMappingSemantics(t *testing.T) {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
result, err := mapping.FromEnterprise(types.EnterpriseRequest{Name: &name, Salary: &money})
|
||||
result, err := convert.Employee(types.EnterpriseRequest{Name: &name, Salary: &money})
|
||||
if err != nil || result.EmployeeName != name || result.Salary != money {
|
||||
t.Error(result, err)
|
||||
}
|
||||
|
||||
@@ -2,7 +2,6 @@ package convert
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"example.com/composition/internal/amount"
|
||||
"strconv"
|
||||
)
|
||||
|
||||
@@ -19,11 +18,4 @@ func Name(value *string) (string, error) {
|
||||
}
|
||||
return *value, nil
|
||||
}
|
||||
func Money(value *amount.Money) amount.Money {
|
||||
if value == nil {
|
||||
return ""
|
||||
}
|
||||
return *value
|
||||
}
|
||||
func Bool(value *bool) bool { return value != nil && *value }
|
||||
func ID(value int64) string { return strconv.FormatInt(value, 10) }
|
||||
|
||||
29
cmd/go-omni/apigen/testdata/composition/copy.go
vendored
Normal file
29
cmd/go-omni/apigen/testdata/composition/copy.go
vendored
Normal file
@@ -0,0 +1,29 @@
|
||||
package convert
|
||||
|
||||
import (
|
||||
"example.com/composition/internal/domain"
|
||||
"example.com/composition/internal/types"
|
||||
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
)
|
||||
|
||||
// The three endpoints reuse one ordinary Go helper and their existing scalar.
|
||||
// There is no field list or package/type name JSON to keep in sync.
|
||||
func Employee[S any](source S) (domain.Employee, error) {
|
||||
var target domain.Employee
|
||||
err := copyx.Copy(&target, source,
|
||||
copyx.WithFieldNames[S, domain.Employee](map[string]string{"Name": "EmployeeName"}),
|
||||
copyx.WithNilPolicy(copyx.NilZero),
|
||||
copyx.WithConverter(Name),
|
||||
)
|
||||
return target, err
|
||||
}
|
||||
func Response(source domain.Result) (types.EmployeeResponse, error) {
|
||||
var target types.EmployeeResponse
|
||||
err := copyx.Copy(&target, source, copyx.WithConverter(func(v int64) (string, error) { return ID(v), nil }))
|
||||
return target, err
|
||||
}
|
||||
func ToBatch(source domain.Batch) (domain.BatchDTO, error) {
|
||||
var target domain.BatchDTO
|
||||
err := copyx.Copy(&target, source)
|
||||
return target, err
|
||||
}
|
||||
@@ -14,7 +14,7 @@ type Result struct {
|
||||
ID int64
|
||||
}
|
||||
|
||||
// Distinct container item types exercise explicitly selected element mappings.
|
||||
// Distinct container item types exercise explicit Go collection copying.
|
||||
type Row struct{ Value string }
|
||||
type RowDTO struct{ Value string }
|
||||
type Batch struct {
|
||||
|
||||
113
cmd/go-omni/apigen/testdata/configuration/consumer_test.go
vendored
Normal file
113
cmd/go-omni/apigen/testdata/configuration/consumer_test.go
vendored
Normal file
@@ -0,0 +1,113 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"example.com/configuration/internal/config"
|
||||
omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestGeneratedConfiguration(t *testing.T) {
|
||||
for ext, source := range map[string]string{"json": `{"limit":"${LIMIT:-7}"}`, "yaml": "limit: '${LIMIT:-7}'\n", "xml": "<config><limit>${LIMIT:-7}</limit></config>", "custom": `{"limit":"${LIMIT:-7}"}`} {
|
||||
t.Run(ext, func(t *testing.T) {
|
||||
t.Setenv("LIMIT", "")
|
||||
t.Setenv("GO_OMNI_LIMIT", "999")
|
||||
path := filepath.Join(t.TempDir(), "app."+ext)
|
||||
if err := os.WriteFile(path, []byte(source), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.Load(path)
|
||||
if err != nil || cfg.Limit != 7 {
|
||||
t.Fatal(cfg.Limit, err)
|
||||
}
|
||||
t.Setenv("LIMIT", "0")
|
||||
cfg, err = config.Load(path)
|
||||
if err != nil || cfg.Limit != 0 {
|
||||
t.Fatal(cfg.Limit, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT", "stderr")
|
||||
cfg, err := config.Load("")
|
||||
if err != nil || cfg.Limit != 10 || cfg.Server.Log.File != "./var/log/app.log" || !cfg.Server.Log.Rotation.Enabled {
|
||||
t.Fatal("implicit overlay", err)
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "bad.yaml")
|
||||
os.WriteFile(path, []byte("limit: '${LIMIT}'\n"), 0600)
|
||||
t.Setenv("LIMIT", "-1")
|
||||
// svc.Init panics if reached. Both invalid startup and successful check must
|
||||
// stop before opening the configured file logger or project dependencies.
|
||||
err = run(context.Background(), path)
|
||||
var d *omniConfig.Error
|
||||
if !errors.As(err, &d) || d.Field != "limit" || d.File != path || d.Line != 1 || d.Env != "LIMIT" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("LIMIT", "private-invalid")
|
||||
_, err = config.Load(path)
|
||||
if !errors.As(err, &d) || d.Env != "LIMIT" || strings.Contains(err.Error(), "private-invalid") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestConfigurationCheckProcess(t *testing.T) {
|
||||
exe := filepath.Join(t.TempDir(), "server")
|
||||
cmd := exec.Command("go", "build", "-o", exe, ".")
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatal(err, string(out))
|
||||
}
|
||||
dir := t.TempDir()
|
||||
path := filepath.Join(dir, "selected.yaml")
|
||||
if err := os.WriteFile(path, []byte("limit: '${LIMIT:-0}'\nserver:\n address: 127.0.0.1:1\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// A broken neighboring default/local/test file must never be layered in.
|
||||
for _, name := range []string{"config.yaml", "config.local.yaml", "config.test.yaml"} {
|
||||
os.WriteFile(filepath.Join(dir, name), []byte("broken: ["), 0600)
|
||||
}
|
||||
cmd = exec.Command(exe, "-config", path, "-check-config")
|
||||
cmd.Dir = dir
|
||||
if out, err := cmd.CombinedOutput(); err != nil || len(out) != 0 {
|
||||
t.Fatal(err, string(out))
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "var")); !os.IsNotExist(err) {
|
||||
t.Fatal("check opened logger/dependencies")
|
||||
}
|
||||
os.WriteFile(path, []byte("limit: '${MISSING_REQUIRED}'"), 0600)
|
||||
cmd = exec.Command(exe, "-config", path, "-check-config")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "MISSING_REQUIRED=")
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err == nil || !strings.Contains(string(out), "MISSING_REQUIRED") || strings.Contains(string(out), "panic") {
|
||||
t.Fatal(err, string(out))
|
||||
}
|
||||
cmd = exec.Command(exe, "-config", path)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "MISSING_REQUIRED=")
|
||||
if out, err := cmd.CombinedOutput(); err == nil || strings.Contains(string(out), "panic") {
|
||||
t.Fatal(err, string(out))
|
||||
}
|
||||
}
|
||||
|
||||
func TestGeneratedModuleFieldDiagnostic(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||
t.Setenv("EXAMPLE_LOG_LEVEL", "invalid-private-level")
|
||||
if err := os.WriteFile(path, []byte("server:\n log:\n level: '${EXAMPLE_LOG_LEVEL}'\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err := run(context.Background(), path)
|
||||
var d *omniConfig.Error
|
||||
if !errors.As(err, &d) || d.Field != "server.log.level" || d.Line != 3 || d.Env != "EXAMPLE_LOG_LEVEL" || !strings.Contains(d.Reason, "debug") || strings.Contains(err.Error(), "invalid-private-level") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(path, []byte("server:\n log:\n timeZone: Asia/Shanghai\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.Load(path)
|
||||
if err != nil || cfg.Server.Log.TimeZone != "Asia/Shanghai" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
69
cmd/go-omni/apigen/testdata/cors/consumer_test.go
vendored
Normal file
69
cmd/go-omni/apigen/testdata/cors/consumer_test.go
vendored
Normal file
@@ -0,0 +1,69 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"example.com/corsdemo/internal/config"
|
||||
"example.com/corsdemo/internal/handler"
|
||||
"example.com/corsdemo/internal/svc"
|
||||
omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
func TestGeneratedCORSConfiguration(t *testing.T) {
|
||||
cfg, err := config.Load("etc/config.yaml")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
logger, err := logx.New(cfg.Server.Log)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer logger.Close()
|
||||
deps, err := svc.New(context.Background(), logger, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer deps.Close()
|
||||
a, err := handler.NewApp(deps)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer a.Close()
|
||||
for _, tc := range []struct {
|
||||
method, path string
|
||||
status int
|
||||
}{{"GET", "/ping", 501}, {"GET", "/missing", 404}, {"PUT", "/ping", 405}, {"GET", "/health/live", 200}, {"GET", "/health/ready", 200}, {"GET", "/openapi.json", 200}, {"OPTIONS", "/ping", 204}} {
|
||||
r := httptest.NewRequest(tc.method, tc.path, nil)
|
||||
r.Header.Set("Origin", "https://web.example")
|
||||
if tc.method == "OPTIONS" {
|
||||
r.Header.Set("Access-Control-Request-Method", "GET")
|
||||
r.Header.Set("Access-Control-Request-Headers", "authorization")
|
||||
}
|
||||
w := httptest.NewRecorder()
|
||||
a.ServeHTTP(w, r)
|
||||
if w.Code != tc.status || w.Header().Get("Access-Control-Allow-Origin") != "https://web.example" {
|
||||
t.Fatal(tc, w.Code, w.Header(), w.Body.String())
|
||||
}
|
||||
if tc.status == 204 && (w.Body.Len() != 0 || w.Header().Get("Access-Control-Max-Age") != "600") {
|
||||
t.Fatal(w)
|
||||
}
|
||||
}
|
||||
// Startup config validation uses the normal generated run/Load, before log
|
||||
// files or dependencies. No changes to generated main/HTTP composition.
|
||||
path := filepath.Join(t.TempDir(), "invalid.yaml")
|
||||
if err := os.WriteFile(path, []byte("server:\n cors:\n enabled: true\n allowedOrigins: ['*']\n allowCredentials: true\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
err = run(context.Background(), path)
|
||||
var d *omniConfig.Error
|
||||
if !errors.As(err, &d) || d.Field != "server.cors.allowedOrigins[0]" || !strings.Contains(d.Reason, "allowCredentials") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
70
cmd/go-omni/apigen/testdata/i18n/consumer_test.go
vendored
Normal file
70
cmd/go-omni/apigen/testdata/i18n/consumer_test.go
vendored
Normal file
@@ -0,0 +1,70 @@
|
||||
package main_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
|
||||
"example.com/errorCatalog/internal/apierrors"
|
||||
"example.com/errorCatalog/internal/handler"
|
||||
"example.com/errorCatalog/internal/svc"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n/errori18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n/httpi18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
func TestGeneratedTranslationAndParameterCompatibility(t *testing.T) {
|
||||
catalog := apierrors.Catalog()
|
||||
source, err := errori18n.Source(catalog, "en")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
source = append(source, i18n.Document{Version: 1, Locale: "zh-Hans", Namespace: "limits", Messages: map[string]i18n.Message{"exceeded": {Value: i18n.Pattern{Text: "上限：{max}"}}}})
|
||||
b, err := i18n.New(i18n.Config{SourceLocale: "en", DefaultLocale: "en", Locales: []string{"en", "zh-Hans"}}, source...)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
p, err := httpi18n.New(b)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
lc := logx.Default()
|
||||
lc.Writer = io.Discard
|
||||
logger, err := logx.New(lc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer logger.Close()
|
||||
deps, err := svc.New(context.Background(), logger)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer deps.Close()
|
||||
a := omni.New(omni.WithLanguagePolicy(p), omni.WithErrorLocalizer(errori18n.New(b, nil)))
|
||||
defer a.Close()
|
||||
a.Use(func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error { return apierrors.LimitExceeded(9223372036854775807) }
|
||||
})
|
||||
if err := handler.RegisterRoutes(a, deps); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r := httptest.NewRequest("GET", "/value", nil)
|
||||
r.Header.Set("Accept-Language", "zh-CN")
|
||||
w := httptest.NewRecorder()
|
||||
a.ServeHTTP(w, r)
|
||||
var body map[string]any
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &body); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if w.Code != 200 || body["detail"] != "上限：9223372036854775807" || body["code"] != float64(400001) || body["ok"] != false {
|
||||
t.Fatal(w.Code, body)
|
||||
}
|
||||
if apierrors.LimitExceeded(5).Message != "Limit: 5" {
|
||||
t.Fatal("non-HTTP constructor changed")
|
||||
}
|
||||
}
|
||||
@@ -123,7 +123,7 @@ func TestGeneratedTransfers(t *testing.T) {
|
||||
func configFile(t *testing.T, address string, fail bool) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(t.TempDir(), "config.json")
|
||||
data, _ := json.Marshal(map[string]any{"server": map[string]any{"address": address}, "failInit": fail})
|
||||
data, _ := json.Marshal(map[string]any{"server": map[string]any{"address": address, "transport": "${GO_OMNI_TRANSPORT:-nethttp}"}, "failInit": fail})
|
||||
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -24,7 +24,7 @@ func phasedConfig(t *testing.T, address, mode string, fail bool) string {
|
||||
if mode == "stuck" {
|
||||
drainTimeout = "50ms"
|
||||
}
|
||||
data, _ := json.Marshal(map[string]any{"backgroundMode": mode, "failInit": fail, "server": map[string]any{"address": address, "lifecycle": map[string]string{"stopTimeout": "1s", "drainTimeout": drainTimeout, "cancelTimeout": "50ms"}}})
|
||||
data, _ := json.Marshal(map[string]any{"backgroundMode": mode, "failInit": fail, "server": map[string]any{"address": address, "transport": "${GO_OMNI_TRANSPORT:-nethttp}", "lifecycle": map[string]string{"stopTimeout": "1s", "drainTimeout": drainTimeout, "cancelTimeout": "50ms"}}})
|
||||
if err := os.WriteFile(path, data, 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -14,7 +14,9 @@ import (
|
||||
"golang.org/x/mod/semver"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
@@ -30,6 +32,55 @@ type UpgradePlan struct {
|
||||
Module, Entry string
|
||||
}
|
||||
|
||||
// Handwritten old loaders must be migrated explicitly, never silently replaced
|
||||
// or left competing with the generated entry point.
|
||||
func checkConfigLoadExtension(root string) error {
|
||||
dir := filepath.Join(root, "internal/config")
|
||||
entries, err := os.ReadDir(dir)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, entry := range entries {
|
||||
name := entry.Name()
|
||||
if entry.IsDir() || !strings.HasSuffix(name, ".go") || strings.HasSuffix(name, "_test.go") || name == "loadGen.go" {
|
||||
continue
|
||||
}
|
||||
rel := "internal/config/" + name
|
||||
if err := safePath(root, rel); err != nil {
|
||||
return err
|
||||
}
|
||||
f, err := goparser.ParseFile(gotoken.NewFileSet(), filepath.Join(root, rel), nil, 0)
|
||||
if err != nil {
|
||||
return fmt.Errorf("parse %s: %w", rel, err)
|
||||
}
|
||||
var removed bool
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
field, ok := n.(*ast.Field)
|
||||
if ok && field.Tag != nil {
|
||||
if raw, e := strconv.Unquote(field.Tag.Value); e == nil {
|
||||
tag := reflect.StructTag(raw)
|
||||
_, a := tag.Lookup("env")
|
||||
_, b := tag.Lookup("envDefault")
|
||||
removed = removed || a || b
|
||||
}
|
||||
}
|
||||
return true
|
||||
})
|
||||
if removed {
|
||||
return fmt.Errorf("%s: env/envDefault tags were removed; migrate defaults and explicit file references before generating; see CONFIG.md", rel)
|
||||
}
|
||||
for _, decl := range f.Decls {
|
||||
if fn, ok := decl.(*ast.FuncDecl); ok && fn.Recv == nil && fn.Name.Name == "Load" {
|
||||
return fmt.Errorf("%s: migrate the project-owned Load function before generating internal/config/loadGen.go; retain Config/Default/Validate and move decoder options to NewLoader; see CONFIG.md", rel)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// PlanUpgrade compares infrastructure against the exact previous release seeds.
|
||||
// A custom file is never classified as safe to replace. Logic, middleware,
|
||||
// options and repositories outside the changed templates are left untouched.
|
||||
|
||||
@@ -39,7 +39,7 @@ func oldInfrastructureProject(t *testing.T) string {
|
||||
raw, _ := json.Marshal(m)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
path := filepath.Join(dir, "go.mod")
|
||||
writeFile(t, path, strings.ReplaceAll(readFile(t, path), minimumVersion, "v0.13.0-preview"))
|
||||
writeFile(t, path, strings.ReplaceAll(readFile(t, path), currentVersion, "v0.13.0-preview"))
|
||||
return dir
|
||||
}
|
||||
|
||||
@@ -74,7 +74,7 @@ func TestUpgradeV014PreservesExtensions(t *testing.T) {
|
||||
raw, _ := json.Marshal(m)
|
||||
writeFile(t, filepath.Join(dir, manifestPath), string(raw))
|
||||
mod := filepath.Join(dir, "go.mod")
|
||||
writeFile(t, mod, strings.ReplaceAll(readFile(t, mod), minimumVersion, "v0.14.1-preview"))
|
||||
writeFile(t, mod, strings.ReplaceAll(readFile(t, mod), currentVersion, "v0.14.1-preview"))
|
||||
owned := map[string]string{}
|
||||
for _, name := range []string{"internal/svc/serviceContext.go", "internal/config/config.go", "etc/config.yaml"} {
|
||||
path := filepath.Join(dir, name)
|
||||
|
||||
@@ -13,7 +13,7 @@ import (
|
||||
|
||||
func TestWorkspaceSharedGoCatalog(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeFile(t, filepath.Join(root, "go.mod"), fmt.Sprintf("module example.com/catalogs\n\ngo 1.24.0\nrequire %s v0.15.0-preview\nreplace %s => %s\n", frameworkModule, frameworkModule, strconvQuote(frameworkRoot(t))))
|
||||
writeFile(t, filepath.Join(root, "go.mod"), fmt.Sprintf("module example.com/catalogs\n\ngo 1.22.0\nrequire %s v0.15.0-preview\nreplace %s => %s\n", frameworkModule, frameworkModule, strconvQuote(frameworkRoot(t))))
|
||||
writeFile(t, filepath.Join(root, "internal/apierrors/catalog.go"), `package apierrors
|
||||
import errors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
func Catalog() errors.Catalog {return errors.Catalog{DefaultStatus:400,Definitions:map[string]errors.Definition{
|
||||
|
||||
@@ -117,6 +117,17 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
|
||||
var changes []change
|
||||
// Check ALL previous outputs, even stale ones, before writing any new file.
|
||||
for name, hash := range old.Generated {
|
||||
// Migration guard only; old mappings are never regenerated or silently
|
||||
// deleted. Once the project removes them, forget the stale ownership.
|
||||
if name == "internal/mapping/mappingsGen.go" {
|
||||
if err = safePath(root, name); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if _, e := os.Lstat(filepath.Join(root, name)); !errors.Is(e, os.ErrNotExist) {
|
||||
return result, errors.New("mapping generation was removed: migrate callers to copyx or ordinary Go and remove internal/mapping/mappingsGen.go before generating; see COPY.md")
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !ownedPath(name) {
|
||||
return result, fmt.Errorf("api: invalid owned path %q", name)
|
||||
}
|
||||
@@ -273,7 +284,7 @@ func apply(root, module, entry string, generated, seeds map[string][]byte, check
|
||||
}
|
||||
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
|
||||
func ownedPath(name string) bool {
|
||||
if name == "internal/mapping/mappingsGen.go" {
|
||||
if name == "internal/config/loadGen.go" {
|
||||
return true
|
||||
}
|
||||
if strings.HasPrefix(name, "internal/types/dto") && strings.HasSuffix(name, "Gen.go") && goName.MatchString(strings.TrimSuffix(filepath.Base(name), ".go")) {
|
||||
|
||||
195
cmd/go-omni/i18n.go
Normal file
195
cmd/go-omni/i18n.go
Normal file
@@ -0,0 +1,195 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n/errori18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n/i18ntool"
|
||||
)
|
||||
|
||||
type i18nFlags []string
|
||||
|
||||
func (v *i18nFlags) String() string { return strings.Join(*v, ",") }
|
||||
func (v *i18nFlags) Set(s string) error {
|
||||
if s == "" {
|
||||
return errors.New("empty value")
|
||||
}
|
||||
*v = append(*v, s)
|
||||
return nil
|
||||
}
|
||||
|
||||
func runI18n(args []string) error { return runI18nTo(args, os.Stdout) }
|
||||
func runI18nTo(args []string, w io.Writer) error {
|
||||
if len(args) == 0 {
|
||||
return errors.New("expected check, export, types or extract")
|
||||
}
|
||||
fs := flag.NewFlagSet("i18n "+args[0], flag.ContinueOnError)
|
||||
fs.SetOutput(io.Discard)
|
||||
var resources, namespaces, goFiles i18nFlags
|
||||
configPath := fs.String("config", "", "i18n Config JSON file")
|
||||
locale := fs.String("locale", "", "selected/source locale")
|
||||
catalog := fs.String("catalog", "", "exported errors.Catalog JSON (extract only)")
|
||||
fs.Var(&resources, "resource", "resource JSON file (repeatable)")
|
||||
fs.Var(&namespaces, "namespace", "export namespace (repeatable)")
|
||||
fs.Var(&goFiles, "go", "Go source file (repeatable)")
|
||||
if err := fs.Parse(args[1:]); err != nil {
|
||||
return err
|
||||
}
|
||||
if fs.NArg() != 0 {
|
||||
return errors.New("unexpected positional arguments")
|
||||
}
|
||||
encode := func(v any) error { e := json.NewEncoder(w); e.SetIndent("", " "); return e.Encode(v) }
|
||||
if args[0] == "extract" && *catalog != "" {
|
||||
if len(goFiles) > 0 || len(resources) > 0 || *configPath != "" || len(namespaces) != 1 || *locale == "" {
|
||||
return errors.New("catalog extraction requires --catalog, --locale and exactly one --namespace")
|
||||
}
|
||||
raw, err := readI18nFile(*catalog)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var c apierrors.Catalog
|
||||
if err := apierrors.DecodeCatalogJSON(raw, &c); err != nil {
|
||||
return err
|
||||
}
|
||||
docs, err := errori18n.Source(errori18n.Bind(c, namespaces[0]), *locale)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, doc := range docs {
|
||||
if doc.Namespace == namespaces[0] {
|
||||
return encode(doc)
|
||||
}
|
||||
}
|
||||
return errors.New("catalog has no messages in the selected namespace")
|
||||
}
|
||||
if *catalog != "" {
|
||||
return errors.New("--catalog is only supported by extract")
|
||||
}
|
||||
if args[0] == "extract" {
|
||||
if len(goFiles) == 0 || len(resources) > 0 || len(namespaces) > 0 || *configPath != "" || *locale != "" {
|
||||
return errors.New("Go extraction requires one or more --go files")
|
||||
}
|
||||
out := i18ntool.Extraction{References: []i18ntool.Reference{}, Findings: []i18ntool.Finding{}}
|
||||
for _, path := range goFiles {
|
||||
raw, err := readI18nFile(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
result, err := i18ntool.ExtractGo(path, raw)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out.References = append(out.References, result.References...)
|
||||
out.Findings = append(out.Findings, result.Findings...)
|
||||
}
|
||||
return encode(out)
|
||||
}
|
||||
if args[0] != "check" && args[0] != "export" && args[0] != "types" {
|
||||
return errors.New("expected check, export, types or extract")
|
||||
}
|
||||
if *configPath == "" || len(resources) == 0 {
|
||||
return errors.New("--config and at least one --resource are required")
|
||||
}
|
||||
data, err := readI18nFile(*configPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var cfg i18n.Config
|
||||
if err := i18n.DecodeJSON(data, &cfg); err != nil {
|
||||
return err
|
||||
}
|
||||
docs := []i18n.Document{}
|
||||
total := 0
|
||||
for _, path := range resources {
|
||||
raw, err := readI18nFile(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
total += len(raw)
|
||||
if total > 4<<20 {
|
||||
return errors.New("resources exceed 4 MiB")
|
||||
}
|
||||
var doc i18n.Document
|
||||
if err := i18n.DecodeJSON(raw, &doc); err != nil {
|
||||
return fmt.Errorf("%s: %w", path, err)
|
||||
}
|
||||
docs = append(docs, doc)
|
||||
}
|
||||
bundle, err := i18n.New(cfg, docs...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch args[0] {
|
||||
case "export":
|
||||
if len(goFiles) > 0 {
|
||||
return errors.New("--go is only supported by check/extract")
|
||||
}
|
||||
return bundle.Export(w, *locale, namespaces...)
|
||||
case "types":
|
||||
if *locale != "" || len(goFiles) > 0 {
|
||||
return errors.New("types does not accept --locale or --go")
|
||||
}
|
||||
return bundle.TypeScript(w, namespaces...)
|
||||
case "check":
|
||||
if *locale != "" || len(namespaces) > 0 {
|
||||
return errors.New("check validates all configured locales and namespaces")
|
||||
}
|
||||
findings := bundle.Check()
|
||||
for _, path := range goFiles {
|
||||
raw, e := readI18nFile(path)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
ex, e := i18ntool.ExtractGo(path, raw)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
for _, r := range ex.References {
|
||||
if _, ok := bundle.Schema(i18n.Ref{Namespace: r.Namespace, Key: r.Key}); !ok {
|
||||
findings = append(findings, i18n.Diagnostic{Kind: "missing", Namespace: r.Namespace, Key: r.Key, Problem: fmt.Sprintf("referenced by %s:%d", r.File, r.Line)})
|
||||
}
|
||||
}
|
||||
for _, f := range ex.Findings {
|
||||
findings = append(findings, i18n.Diagnostic{Kind: "dynamic", Problem: fmt.Sprintf("%s:%d: %s", f.File, f.Line, f.Problem)})
|
||||
}
|
||||
}
|
||||
if err := encode(findings); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(findings) > 0 {
|
||||
return errors.New("internationalization checks failed")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func readI18nFile(path string) ([]byte, error) {
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer f.Close()
|
||||
s, err := f.Stat()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !s.Mode().IsRegular() {
|
||||
return nil, errors.New("expected a regular file")
|
||||
}
|
||||
b, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(b) > 1<<20 {
|
||||
return nil, errors.New("file exceeds 1 MiB")
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
80
cmd/go-omni/i18n_test.go
Normal file
80
cmd/go-omni/i18n_test.go
Normal file
@@ -0,0 +1,80 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n"
|
||||
)
|
||||
|
||||
func TestI18nCommandsAreReadOnlyAndCheckMissing(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
files := map[string]string{
|
||||
"config.json": `{"sourceLocale":"en","defaultLocale":"en","locales":["en","fr"]}`,
|
||||
"en.json": `{"version":1,"namespace":"ui","locale":"en","messages":{"hello":{"params":{"type":"string"},"value":{"text":"Hello {type}"}}}}`,
|
||||
"fr.json": `{"version":1,"namespace":"ui","locale":"fr","messages":{"hello":{"value":{"text":"Bonjour {type}"}}}}`,
|
||||
"use.go": "package demo\nimport words \"git.yuntuo.hk/k2safe/go-omni/i18n\"\nvar key=words.Ref{Namespace:\"ui\",Key:\"hello\"}\n",
|
||||
}
|
||||
for path, data := range files {
|
||||
if err := os.WriteFile(filepath.Join(dir, path), []byte(data), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
base := []string{"--config", filepath.Join(dir, "config.json"), "--resource", filepath.Join(dir, "en.json"), "--resource", filepath.Join(dir, "fr.json")}
|
||||
var out bytes.Buffer
|
||||
args := append([]string{"check"}, base...)
|
||||
args = append(args, "--go", filepath.Join(dir, "use.go"))
|
||||
if err := runI18nTo(args, &out); err != nil || strings.TrimSpace(out.String()) != "[]" {
|
||||
t.Fatal(err, out.String())
|
||||
}
|
||||
out.Reset()
|
||||
args = append([]string{"export"}, base...)
|
||||
args = append(args, "--locale", "fr", "--namespace", "ui")
|
||||
if err := runI18nTo(args, &out); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var pack i18n.Pack
|
||||
if err := json.Unmarshal(out.Bytes(), &pack); err != nil || pack.Locale != "fr" || len(pack.Documents) != 2 {
|
||||
t.Fatal(err, pack)
|
||||
}
|
||||
out.Reset()
|
||||
args = append([]string{"types"}, base...)
|
||||
args = append(args, "--namespace", "ui")
|
||||
if err := runI18nTo(args, &out); err != nil || !strings.Contains(out.String(), `"type": string`) {
|
||||
t.Fatal(err, out.String())
|
||||
}
|
||||
out.Reset()
|
||||
if err := runI18nTo([]string{"extract", "--go", filepath.Join(dir, "use.go")}, &out); err != nil || !strings.Contains(out.String(), `"key": "hello"`) {
|
||||
t.Fatal(err, out.String())
|
||||
}
|
||||
for path, want := range files {
|
||||
got, _ := os.ReadFile(filepath.Join(dir, path))
|
||||
if string(got) != want {
|
||||
t.Fatal("command mutated", path)
|
||||
}
|
||||
}
|
||||
out.Reset()
|
||||
args = append([]string{"check"}, base[:4]...)
|
||||
if err := runI18nTo(args, &out); err == nil || !strings.Contains(out.String(), "direct translation missing") {
|
||||
t.Fatal(err, out.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestI18nCatalogExtractionPreservesLiteralBraces(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "errors.json")
|
||||
if err := os.WriteFile(path, []byte(`{"definitions":{"FIXED":{"code":1,"message":"${NAME}"}}}`), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var out bytes.Buffer
|
||||
if err := runI18nTo([]string{"extract", "--catalog", path, "--namespace", "errors", "--locale", "en"}, &out); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var d i18n.Document
|
||||
if err := i18n.DecodeJSON(out.Bytes(), &d); err != nil || !d.Messages["FIXED"].Value.Literal || d.Messages["FIXED"].Value.Text != "${NAME}" {
|
||||
t.Fatal(err, d)
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,11 @@ func main() {
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "i18n":
|
||||
if err := runI18n(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni i18n:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "workspace":
|
||||
if err := runWorkspace(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni workspace:", err)
|
||||
@@ -170,6 +175,7 @@ func usage() {
|
||||
fmt.Fprintln(os.Stderr, "GoOmni project tool")
|
||||
fmt.Fprintln(os.Stderr, "")
|
||||
fmt.Fprintln(os.Stderr, "Usage:")
|
||||
fmt.Fprintln(os.Stderr, " go-omni i18n <check|export|types|extract> Validate and export independent language resources")
|
||||
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
|
||||
|
||||
@@ -8,7 +8,7 @@ import (
|
||||
)
|
||||
|
||||
func TestNewWithRemoteFrameworkVersion(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--framework-version", "v0.17.0-preview"}, {"--framework-version=v0.17.0-preview"}} {
|
||||
for _, flag := range [][]string{{"--framework-version", "v0.22.0-preview"}, {"--framework-version=v0.22.0-preview"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
args := append([]string{"--module", "example.com/billing"}, flag...)
|
||||
if err := runNew(append(args, project)); err != nil {
|
||||
@@ -18,7 +18,7 @@ func TestNewWithRemoteFrameworkVersion(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(content), "go-omni v0.17.0-preview") || strings.Contains(string(content), "replace ") {
|
||||
if !strings.Contains(string(content), "go-omni v0.22.0-preview") || strings.Contains(string(content), "replace ") {
|
||||
t.Fatalf("CLI did not pin remote dependency: %s", content)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -21,6 +21,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
|
||||
"internal/svc/runtimeGen.go": "templates/api-runtime.go.tmpl",
|
||||
"internal/svc/options.go": "templates/api-options.go.tmpl",
|
||||
"internal/config/config.go": "templates/api-config.go.tmpl",
|
||||
"internal/config/loadGen.go": "templates/api-load.go.tmpl",
|
||||
"internal/config/loader.go": "templates/api-loader.go.tmpl",
|
||||
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
|
||||
"internal/svc/serviceContext.go": "templates/api-context.go.tmpl",
|
||||
}
|
||||
@@ -31,6 +33,8 @@ func apiInfrastructure(module, database, storage, logOutput, logFormat string, l
|
||||
files["internal/svc/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||
}
|
||||
if legacy {
|
||||
delete(files, "internal/config/loadGen.go")
|
||||
delete(files, "internal/config/loader.go")
|
||||
delete(files, "internal/svc/runtimeGen.go")
|
||||
delete(files, "internal/svc/databaseOptions.go")
|
||||
files["internal/svc/close.go"] = "templates/api-close.go.tmpl"
|
||||
|
||||
@@ -66,9 +66,13 @@ func TestGenerateModuleProfiles(t *testing.T) {
|
||||
t.Fatal("replacement does not resolve", err)
|
||||
}
|
||||
cfg, _ := os.ReadFile(filepath.Join(project, "etc/config.yaml"))
|
||||
if strings.Contains(string(cfg), "secretKey:") || strings.Contains(string(cfg), "filesToken:") {
|
||||
t.Fatal("secret defaults emitted")
|
||||
for _, line := range strings.Split(string(cfg), "\n") {
|
||||
key, value, ok := strings.Cut(strings.TrimSpace(line), ":")
|
||||
if ok && (key == "secretKey" || key == "filesToken" || key == "signingKey") && !strings.Contains(value, "${GO_OMNI_") {
|
||||
t.Fatal("secret must be an explicit reference")
|
||||
}
|
||||
}
|
||||
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -76,6 +80,7 @@ func TestModuleOptionsRejectBeforeWriting(t *testing.T) {
|
||||
for _, options := range []Options{
|
||||
{LogOutput: "invalid"}, {LogFormat: "xml"}, {Storage: "ftp"},
|
||||
{Storage: "local", FrameworkVersion: "v0.6.0-preview"},
|
||||
{FrameworkVersion: "v0.21.0-preview"},
|
||||
{Module: "example.com/x\nrequire malicious v1.0.0"},
|
||||
} {
|
||||
options.Dir = filepath.Join(t.TempDir(), "project")
|
||||
|
||||
@@ -34,9 +34,13 @@ type Options struct {
|
||||
}
|
||||
|
||||
var files = map[string]string{
|
||||
"internal/settings/config.go": "templates/modules-settings.go.tmpl",
|
||||
"internal/settings/config_test.go": "templates/modules-settings_test.go.tmpl",
|
||||
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
|
||||
|
||||
"go.mod": "templates/go.mod.tmpl",
|
||||
"README.md": "templates/README.md.tmpl",
|
||||
"cmd/server/main.go": "templates/cmd-server-main.go.tmpl",
|
||||
"cmd/server/main.go": "templates/modules-main.go.tmpl",
|
||||
"internal/model/user.go": "templates/internal-model-user.go.tmpl",
|
||||
"internal/model/errors.go": "templates/internal-model-errors.go.tmpl",
|
||||
"internal/repository/users.go": "templates/internal-repository-users.go.tmpl",
|
||||
@@ -109,7 +113,7 @@ func Generate(options Options) error {
|
||||
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
|
||||
replace = frameworkReplace(directory, options.FrameworkDir)
|
||||
}
|
||||
if modules && replace == "" && semver.Compare(version, modulesFrameworkVersion) < 0 && version != installedFrameworkVersion() {
|
||||
if replace == "" && semver.Compare(version, modulesFrameworkVersion) < 0 && version != installedFrameworkVersion() {
|
||||
return fmt.Errorf("module templates require %s or newer; use --framework-dir for unpublished source", modulesFrameworkVersion)
|
||||
}
|
||||
data := templateData{
|
||||
|
||||
@@ -39,7 +39,7 @@ func TestGenerateCreatesServiceLayout(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("read generated server: %v", err)
|
||||
}
|
||||
for _, expected := range []string{"config.LoadFromEnv(\"\")", "logx.SetDefault(logger)", "omni.WithConfig(settings)", "servicecontext.New(ctx, logger)", "services.Health.ReadinessHandler()", "application.RunContext(ctx, settings.Address)"} {
|
||||
for _, expected := range []string{"settings.Load(path)", "logx.SetDefault(logger)", "omni.WithConfig(cfg.Server)", "servicecontext.New(ctx, logger, cfg)", "services.Health.ReadinessHandler()", "application.RunContext(ctx, cfg.Server.Address)"} {
|
||||
if !strings.Contains(string(mainContent), expected) {
|
||||
t.Fatalf("generated server missing %q: %s", expected, mainContent)
|
||||
}
|
||||
@@ -110,7 +110,7 @@ func TestFrameworkReplaceUsesCanonicalPaths(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestGenerateRemoteProjectPinsVersion(t *testing.T) {
|
||||
for _, version := range []string{"v0.1.0", "v1.2.3-rc.1", "v0.0.0-20260920140624-8b19aa79eea0"} {
|
||||
for _, version := range []string{"v0.22.0-preview", "v1.2.3-rc.1", "v0.22.1-0.20260926000000-8b19aa79eea0"} {
|
||||
t.Run(version, func(t *testing.T) {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: project, Module: "example.com/billing", FrameworkVersion: version}); err != nil {
|
||||
|
||||
@@ -32,46 +32,44 @@ go mod tidy
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
运行时配置从 `GO_OMNI_*` 环境变量加载，例如：
|
||||
配置入口是 `internal/settings.Config`、`Default()`、`Validate()` 和 `etc/config.yaml`。
|
||||
只有文件里的 `${NAME}` / `${NAME:-fallback}` 会读取环境变量，固定值不受进程环境覆盖。生成的文件已声明地址、网络层，以及所选数据库/存储的必要引用；其他参数直接写进文件。
|
||||
|
||||
```sh
|
||||
GO_OMNI_ENABLE_LOGGER=true GO_OMNI_LOG_LEVEL=debug go run ./cmd/server
|
||||
```yaml
|
||||
server:
|
||||
address: "${GO_OMNI_HTTP_ADDR:-127.0.0.1:8080}"
|
||||
transport: "${GO_OMNI_TRANSPORT:-nethttp}"
|
||||
log:
|
||||
level: debug
|
||||
shutdownTimeout: 10s
|
||||
```
|
||||
|
||||
`GO_OMNI_HTTP_ADDR` 可以修改监听地址。生成项目默认提供 `/health/live` 和
|
||||
`/health/ready`，日志、请求 ID 和优雅退出等基础能力由 GoOmni 统一初始化；业务依赖集中放在项目自己的 `internal/servicecontext`。
|
||||
|
||||
服务使用 `signal.NotifyContext` 处理 `SIGINT` / `SIGTERM`。部署时构建并运行服务二进制，
|
||||
使信号直接到达服务进程：
|
||||
用 `-config` 选择一个 YAML/JSON/XML 文件；`-config ''` 只使用 Default()；`-check-config` 仅加载和校验，不创建依赖或监听：
|
||||
|
||||
```sh
|
||||
go run ./cmd/server -config etc/config.yaml -check-config
|
||||
go build -o server ./cmd/server
|
||||
./server
|
||||
./server -config etc/config.yaml
|
||||
```
|
||||
|
||||
`GO_OMNI_SHUTDOWN_TIMEOUT` 控制等待请求完成的时间，默认 `10s`。
|
||||
服务通过 `signal.NotifyContext` 处理 `SIGINT` / `SIGTERM`，先排空 HTTP 请求，再关闭依赖和日志。生成项目默认提供 `/health/live` 和 `/health/ready`，业务依赖集中在 `internal/servicecontext`。
|
||||
|
||||
使用支持统一网络配置的框架版本时，设置 `GO_OMNI_TRANSPORT=native` 即可切换为自研 HTTP/1.1；
|
||||
默认 `nethttp`。`GO_OMNI_TLS_CERT_FILE` 和 `GO_OMNI_TLS_KEY_FILE` 配对启用 HTTPS。
|
||||
实验版 HTTP/2 使用 `GO_OMNI_TRANSPORT=http2`，必须同时配置 TLS 证书和私钥。
|
||||
业务 Handler / Service / Repository 不需要修改；三种适配器共用上述退出等待时间。
|
||||
通过文件中的引用设置 `GO_OMNI_TRANSPORT=native` 可切换自研 HTTP/1.1，默认 nethttp。HTTPS 在文件的 `server.tls.certFile` / `server.tls.keyFile` 成对配置；实验 HTTP/2 选择 `http2` 且必须配置 TLS。不会自动读取未在文件声明的 TLS 环境变量。
|
||||
{{if eq .Database "mysql"}}
|
||||
`/health/ready` 已注册数据库探针：连接失败返回 `503`，数据库恢复后重新返回 `200`。
|
||||
`/health/live` 始终只检查服务存活。`GO_OMNI_DB_PING_TIMEOUT` 限制初始化检查和数据库探针的等待时间。
|
||||
退出时先排空 HTTP 请求，再关闭数据库连接池，最后关闭日志。
|
||||
`/health/ready` 已注册数据库探针，连接失败返回 503，恢复后返回 200；`/health/live` 只检查进程存活。连接池参数直接写入 `database`，缺失字段保留 Default()：
|
||||
|
||||
数据库连接池参数：
|
||||
|
||||
```text
|
||||
GO_OMNI_DB_MAX_OPEN_CONNS=10
|
||||
GO_OMNI_DB_MAX_IDLE_CONNS=5
|
||||
GO_OMNI_DB_CONN_MAX_LIFETIME=30m
|
||||
GO_OMNI_DB_CONN_MAX_IDLE_TIME=5m
|
||||
GO_OMNI_DB_PING_TIMEOUT=5s
|
||||
```yaml
|
||||
database:
|
||||
driver: mysql
|
||||
dsn: "${GO_OMNI_DB_DSN}"
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
```
|
||||
|
||||
`MAX_IDLE_CONNS=0` 禁用空闲连接，`MAX_OPEN_CONNS=0` 表示不限制连接数。
|
||||
项目会启用 MySQL `parseTime` 并按 UTC 读取 DATETIME 字段，JSON 字段保持 lowerCamelCase。
|
||||
`maxIdleConns: 0` 禁用空闲连接；此项目要求 maxOpenConns 为正数。项目启用 MySQL parseTime，并按 UTC 读取 DATETIME；JSON 字段保持 lowerCamelCase。
|
||||
|
||||
事务内复用 Repository（在业务代码中导入 `database/sql` 和项目的 `internal/repository`）：
|
||||
|
||||
|
||||
@@ -1,10 +1,9 @@
|
||||
package settings
|
||||
import (
|
||||
"os"
|
||||
{{if and (not .API) (ne .Storage "none")}}"strings"{{end}}
|
||||
"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
{{if eq .LogOutput "file"}}"git.yuntuo.hk/k2safe/go-omni/logx"{{end}}
|
||||
{{if eq .Database "mysql"}}mysqladapter "git.yuntuo.hk/k2safe/go-omni/database/mysql"{{end}}
|
||||
{{if ne .Storage "none"}}"errors"
|
||||
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||
@@ -12,8 +11,8 @@ import (
|
||||
)
|
||||
type Config struct {
|
||||
Server config.Config `json:"server"`
|
||||
{{if eq .Database "mysql"}}Database mysqladapter.Config `json:"database" env:"GO_OMNI_DB"`{{end}}
|
||||
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage" env:"GO_OMNI_STORAGE"`{{end}}
|
||||
{{if eq .Database "mysql"}}Database mysqladapter.Config `json:"database"`{{end}}
|
||||
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
|
||||
}
|
||||
func Default() Config {
|
||||
server:=config.Default()
|
||||
@@ -35,22 +34,10 @@ func Default() Config {
|
||||
}{{end}}}
|
||||
}
|
||||
|
||||
// Load applies defaults -> file -> typed environment -> validation.
|
||||
// New fields automatically use GO_OMNI_<JSON_PATH>, or an explicit env tag.
|
||||
func Load(path string)(Config,error){
|
||||
cfg:=Default();var err error
|
||||
cfg,err=config.ApplyDefaults(cfg);if err!=nil{return Config{},err}
|
||||
if path!="" {cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err}}
|
||||
cfg,err=config.OverlayEnv(cfg,config.EnvOptions{Prefix:"GO_OMNI_",Empty:"ignore",Aliases:config.ServerEnvAliases("GO_OMNI_","server")});if err!=nil{return Config{},err}
|
||||
output,outputSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_OUTPUT");if !outputSet {output=os.Getenv("GO_OMNI_LOG_OUTPUT")}
|
||||
file,fileSet:=os.LookupEnv("GO_OMNI_SERVER_LOG_FILE");if !fileSet {file=os.Getenv("GO_OMNI_LOG_FILE")}
|
||||
if output!="" && file=="" {cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}}
|
||||
return cfg,cfg.Validate()
|
||||
}
|
||||
func(c Config)Validate()error{
|
||||
if err:=c.Server.Validate();err!=nil{return err}
|
||||
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return err}{{end}}
|
||||
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return err}{{end}}
|
||||
if err:=c.Server.Validate();err!=nil{return config.AtField("server",err)}
|
||||
{{if eq .Database "mysql"}}if err:=c.Database.Validate();err!=nil{return config.AtField("database",err)}{{end}}
|
||||
{{if ne .Storage "none"}}if err:=c.Storage.Validate();err!=nil{return config.AtField("storage",err)}{{end}}
|
||||
return nil
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
@@ -67,12 +54,12 @@ type StorageConfig struct {
|
||||
ContentTypes []string `json:"contentTypes"`
|
||||
LinkTTL time.Duration `json:"linkTtl"`
|
||||
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||
// Secrets are environment-only and cannot be populated by config files.
|
||||
FilesToken string `json:"-" env:"GO_OMNI_FILES_TOKEN,secret"`
|
||||
SigningKey string `json:"-" env:"GO_OMNI_STORAGE_SIGNING_KEY,secret"`
|
||||
AccessKey string `json:"-" env:"GO_OMNI_STORAGE_ACCESS_KEY,secret"`
|
||||
SecretKey string `json:"-" env:"GO_OMNI_STORAGE_SECRET_KEY,secret"`
|
||||
SessionToken string `json:"-" env:"GO_OMNI_STORAGE_SESSION_TOKEN,secret"`
|
||||
// File values may explicitly reference environment variables. Never log this struct.
|
||||
FilesToken string `json:"filesToken"`
|
||||
SigningKey string `json:"signingKey"`
|
||||
AccessKey string `json:"accessKey"`
|
||||
SecretKey string `json:"secretKey"`
|
||||
SessionToken string `json:"sessionToken"`
|
||||
}
|
||||
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||
func (s StorageConfig) Validate()error {
|
||||
@@ -80,14 +67,14 @@ func (s StorageConfig) Validate()error {
|
||||
if err:=s.Policy().Validate();err!=nil{return err}
|
||||
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("storage.filesToken requires at least 32 bytes without whitespace")}{{end}}
|
||||
{{if eq .Storage "local"}}
|
||||
if s.Root=="" {return errors.New("storage.root is required")}
|
||||
u,err:=url.Parse(s.PublicURL)
|
||||
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||
if len(s.SigningKey)<32 {return errors.New("storage.signingKey requires a stable secret of at least 32 bytes")}
|
||||
{{else}}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region, accessKey and secretKey are required")}
|
||||
{{if eq .Storage "r2"}}
|
||||
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||
|
||||
8
cmd/go-omni/scaffold/templates/api-load.go.tmpl
Normal file
8
cmd/go-omni/scaffold/templates/api-load.go.tmpl
Normal file
@@ -0,0 +1,8 @@
|
||||
package config
|
||||
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
|
||||
// Load resolves explicit file references, binds onto Default and validates.
|
||||
// Ordinary projects only edit Config, Default, Validate and the selected file.
|
||||
func Load(path string)(Config,error){
|
||||
loader,err:=NewLoader();if err!=nil{return Config{},err}
|
||||
return omniConfig.LoadProject(path,Default(),loader)
|
||||
}
|
||||
7
cmd/go-omni/scaffold/templates/api-loader.go.tmpl
Normal file
7
cmd/go-omni/scaffold/templates/api-loader.go.tmpl
Normal file
@@ -0,0 +1,7 @@
|
||||
package config
|
||||
|
||||
import omniConfig "git.yuntuo.hk/k2safe/go-omni/config"
|
||||
|
||||
// NewLoader is project-owned. JSON/YAML/XML work without changes. Register
|
||||
// additional format decoders here only when the project needs them.
|
||||
func NewLoader() (*omniConfig.Loader,error) { return omniConfig.NewLoader() }
|
||||
@@ -1,57 +0,0 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/handler"
|
||||
"{{.Module}}/internal/service"
|
||||
"{{.Module}}/internal/servicecontext"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
settings, err := config.LoadFromEnv("")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if strings.TrimSpace(settings.Log.Service) == "" {
|
||||
settings.Log.Service = "{{.Module}}"
|
||||
}
|
||||
logger, err := logx.New(settings.Log)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer logger.Close()
|
||||
// Access logs and business logs share the same configured sink.
|
||||
logx.SetDefault(logger)
|
||||
|
||||
services, err := servicecontext.New(ctx, logger)
|
||||
if err != nil { return err }
|
||||
defer services.Close()
|
||||
userService := service.NewUsers(services)
|
||||
userHandler := handler.NewUsers(userService)
|
||||
|
||||
application := omni.New(omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
application.OnShutdown(services.Close)
|
||||
application.GET("/health/live", services.Health.LivenessHandler())
|
||||
application.GET("/health/ready", services.Health.ReadinessHandler())
|
||||
userHandler.RegisterRoutes(application)
|
||||
return application.RunContext(ctx, settings.Address)
|
||||
}
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/model"
|
||||
"{{.Module}}/internal/settings"
|
||||
"{{.Module}}/internal/repository"
|
||||
"{{.Module}}/internal/servicecontext"
|
||||
)
|
||||
@@ -23,15 +24,12 @@ func databaseContext(t *testing.T) *servicecontext.Context {
|
||||
t.Helper()
|
||||
dsn := os.Getenv("GO_OMNI_TEST_MYSQL_DSN")
|
||||
if dsn == "" { t.Skip("set GO_OMNI_TEST_MYSQL_DSN to a dedicated test database") }
|
||||
for _, key := range []string{"MAX_OPEN_CONNS", "MAX_IDLE_CONNS", "CONN_MAX_LIFETIME", "CONN_MAX_IDLE_TIME", "PING_TIMEOUT"} {
|
||||
t.Setenv("GO_OMNI_DB_"+key, "")
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", dsn)
|
||||
t.Setenv("GO_OMNI_DB_DRIVER", "mysql")
|
||||
logger, err := logx.New(logx.Config{Writer: io.Discard})
|
||||
if err != nil { t.Fatal(err) }
|
||||
t.Cleanup(func() { _ = logger.Close() })
|
||||
services, err := servicecontext.New(context.Background(), logger)
|
||||
cfg,err:=settings.Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||
services, err := servicecontext.New(context.Background(), logger,cfg)
|
||||
if err != nil { t.Fatal(err) }
|
||||
t.Cleanup(func() { _ = services.Close() })
|
||||
return services
|
||||
|
||||
@@ -2,6 +2,7 @@ package servicecontext
|
||||
|
||||
import (
|
||||
"context"
|
||||
"{{.Module}}/internal/settings"
|
||||
{{if and .Modules (ne .Storage "none")}}"errors"
|
||||
"net/http"
|
||||
"sync"
|
||||
@@ -38,16 +39,16 @@ type Context struct {
|
||||
|
||||
// New initializes the project's dependencies before the HTTP listener opens.
|
||||
// The caller owns Logger; Close releases only resources created here.
|
||||
func New(ctx context.Context, logger *logx.Logger) (*Context, error) {
|
||||
func New(ctx context.Context, logger *logx.Logger, project settings.Config) (*Context, error) {
|
||||
if ctx == nil { ctx = context.Background() }
|
||||
if err := ctx.Err(); err != nil { return nil, err }
|
||||
checker, err := health.New()
|
||||
if err != nil { return nil, err }
|
||||
services := &Context{Logger: logger, Health: checker}
|
||||
{{if eq .Database "mysql"}}
|
||||
cfg, err := database.ConfigFromEnv("")
|
||||
if err != nil { return nil, err }
|
||||
if cfg.Driver != "mysql" { return nil, fmt.Errorf("this project requires GO_OMNI_DB_DRIVER=mysql") }
|
||||
cfg:=project.Database
|
||||
if err:=cfg.Validate();err!=nil{return nil,err}
|
||||
if cfg.Driver != "mysql" { return nil, fmt.Errorf("this project requires database.driver=mysql") }
|
||||
dsn, err := mysql.ParseDSN(cfg.DSN)
|
||||
if err != nil { return nil, fmt.Errorf("parse mysql connection settings: %w", err) }
|
||||
if dsn.DBName == "" { return nil, fmt.Errorf("mysql connection settings require a database name") }
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
本项目选用日志输出 `{{.LogOutput}}`、格式 `{{.LogFormat}}`、对象存储 `{{.Storage}}`。数据库选择 `{{.Database}}` 独立于对象存储。
|
||||
|
||||
配置入口为 `etc/config.yaml`，通过 `go run ./cmd/server -config /path/to/config.yaml` 指定其他 JSON/YAML 文件；`-config ''` 只用代码默认值和环境变量。优先级：默认值 → 文件 → 非空环境变量。未知字段、无效参数会在监听前拒绝。
|
||||
配置入口为 `etc/config.yaml`，通过 `go run ./cmd/server -config /path/to/config.yaml` 指定其他 JSON/YAML/XML 文件；`-config ''` 只用代码默认值。文件用 `${NAME}` 或 `${NAME:-fallback}` 显式引用变量，固定值不受隐式环境覆盖；`-check-config` 只检查、不初始化依赖。未知字段、无效参数会在监听前拒绝。
|
||||
|
||||
```sh
|
||||
go mod tidy
|
||||
@@ -24,7 +24,7 @@ go run ./cmd/server
|
||||
|
||||
{{if eq .Database "mysql"}}还需按 README 配置独立 MySQL 数据库并应用 schema.sql；应用不自动迁移。{{end}}
|
||||
|
||||
默认仅监听 `127.0.0.1:8080`。`GO_OMNI_HTTP_ADDR` 覆盖地址；本地签名 URL 的外部地址通过 `GO_OMNI_STORAGE_PUBLIC_URL` 单独配置，始终以 `/storage/object` 结尾，不使用请求 Host 自动拼接可信链接。
|
||||
默认仅监听 `127.0.0.1:8080`。文件中的 `${GO_OMNI_HTTP_ADDR:-127.0.0.1:8080}` 设置地址；本地签名 URL 的外部地址通过 `GO_OMNI_STORAGE_PUBLIC_URL` 单独配置，始终以 `/storage/object` 结尾，不使用请求 Host 自动拼接可信链接。
|
||||
|
||||
## 日志
|
||||
|
||||
@@ -35,7 +35,7 @@ curl -H 'Content-Type: application/json' -H 'X-Request-ID: example-request' \
|
||||
-d '{"name":"Example","email":"example@example.com"}' http://127.0.0.1:8080/users
|
||||
```
|
||||
|
||||
`GO_OMNI_LOG_LEVEL`、`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_FILE` 覆盖对应项；显式设置 `GO_OMNI_LOG_OUTPUT=stdout` 且未设置 LOG_FILE 时，切换到控制台并关闭文件轮转。慢请求阈值为 500ms，可通过 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 调整。字段脱敏不扫描 msg/error 中的自由文本；不要把凭据或签名 URL 拼到日志中。
|
||||
日志参数直接写在 `server.log`，例如 `level: debug`；也可显式写 `level: "${LOG_LEVEL:-info}"`。切换文件日志为控制台时，明确设置 `file: ""`、`output: stdout` 和 `rotation.enabled: false`，不会隐式重置其他字段。慢请求阈值使用 `server.slowRequestThreshold`，默认 500ms。字段脱敏不扫描 msg/error 自由文本；不要把凭据或签名 URL 拼到日志中。
|
||||
|
||||
{{if ne .Storage "none"}}## 文件示例
|
||||
|
||||
@@ -66,18 +66,18 @@ curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: applicati
|
||||
|
||||
响应包括随机 `data.key` 和 `data.request`（method/url/headers/expiresAt）。按返回的 method 和 headers 用原始文件字节上传到 URL；不能额外包 FormData。浏览器自动填写 Content-Length，其余签名 Header 原样发送；云桶需配置允许业务域名的 CORS。云端直传只绑定声明类型，实际内容需上传后验证；本地和 API 中转上传会检查允许类型及前缀嗅探结果。未完成内容验证前不要把暂存文件当作可信文件公开。
|
||||
|
||||
| 参数 | 默认值 / 环境变量 |
|
||||
| 文件字段 | 默认值 / 含义 |
|
||||
| --- | --- |
|
||||
| 大小 | 32 MiB / GO_OMNI_STORAGE_MAX_BYTES |
|
||||
| 类型 | text/plain、image/png、image/jpeg、application/pdf / GO_OMNI_STORAGE_CONTENT_TYPES（逗号分隔） |
|
||||
| 链接有效期 | 5m / GO_OMNI_STORAGE_LINK_TTL（1s..1h） |
|
||||
| 请求超时 | 30s / GO_OMNI_STORAGE_REQUEST_TIMEOUT（>0 且 <=5m） |
|
||||
| 本地目录 | var/files / GO_OMNI_STORAGE_ROOT |
|
||||
| 本地外部 URL | http://127.0.0.1:8080/storage/object / GO_OMNI_STORAGE_PUBLIC_URL |
|
||||
| 云配置 | GO_OMNI_STORAGE_ENDPOINT、REGION、BUCKET、ACCOUNT_ID、SECURE |
|
||||
| 环境密钥 | GO_OMNI_FILES_TOKEN、GO_OMNI_STORAGE_SIGNING_KEY、GO_OMNI_STORAGE_ACCESS_KEY、SECRET_KEY、SESSION_TOKEN |
|
||||
| `storage.maxBytes` | 32 MiB |
|
||||
| `storage.contentTypes` | text/plain、image/png、image/jpeg、application/pdf；文件数组 |
|
||||
| `storage.linkTtl` | 5m，允许 1s..1h |
|
||||
| `storage.requestTimeout` | 30s，大于 0 且不超过 5m |
|
||||
| `storage.root` | var/files |
|
||||
| `storage.publicUrl` | http://127.0.0.1:8080/storage/object |
|
||||
| `storage.endpoint/region/bucket/accountId/secure` | 对应平台连接配置 |
|
||||
| `storage.filesToken/signingKey/accessKey/secretKey/sessionToken` | 文件显式引用环境变量提供凭据 |
|
||||
|
||||
全部 JSON/YAML 字段使用 lowerCamelCase。密钥字段仅允许环境变量，配置文件中写入会报未知字段；不要把运行环境文件提交到仓库。S3 的 endpoint 为 host[:port]，OSS 的可选 endpoint 为 HTTPS origin；R2 从 accountId 派生官方 endpoint，region 为 auto。仅 S3/MinIO 的本地测试可设置 SECURE=false。
|
||||
全部 JSON/YAML/XML 字段使用 lowerCamelCase。文件可使用固定值或显式环境引用，不按字段名自动推导环境变量；凭据建议使用 `${NAME}`，不把含秘密的配置提交到仓库。S3 endpoint 为 host[:port]，OSS 可选 endpoint 为 HTTPS origin；R2 从 accountId 派生官方 endpoint，region 为 auto。仅 S3/MinIO 本地测试可设置 `storage.secure: false`。
|
||||
|
||||
错误响应：未授权 401、无 Content-Length 411、超限 413、类型不允许 415、长度不符/非法 key 400、对象不存在 404、存储超时 504，其余内部错误统一隐藏细节。底层签名入口遵循存储协议返回 201/403 等状态，流式传输且访问日志保留实际状态。
|
||||
|
||||
|
||||
@@ -1,68 +1,30 @@
|
||||
# Non-secret settings only. Environment variables override these values.
|
||||
# Only fields listed here override Default(). Environment names are explicit.
|
||||
server:
|
||||
address: '127.0.0.1:8080'
|
||||
enableLogger: true
|
||||
slowRequestThreshold: 500ms
|
||||
shutdownTimeout: 10s
|
||||
{{if .API}} lifecycle:
|
||||
stopTimeout: 5s
|
||||
drainTimeout: 10s
|
||||
cancelTimeout: 5s
|
||||
endpoints:
|
||||
liveness:
|
||||
enabled: true
|
||||
path: /health/live
|
||||
responseMode: app
|
||||
readiness:
|
||||
enabled: true
|
||||
path: /health/ready
|
||||
responseMode: app
|
||||
openapi:
|
||||
enabled: true
|
||||
path: /openapi.json
|
||||
{{end}}
|
||||
log:
|
||||
service: {{printf "%q" .Module}}
|
||||
level: info
|
||||
format: {{.LogFormat}}
|
||||
{{if eq .LogOutput "file"}} file: './var/log/server.log'
|
||||
rotation:
|
||||
enabled: true
|
||||
maxSizeMb: 100
|
||||
maxBackups: 10
|
||||
maxAgeDays: 7
|
||||
compress: true
|
||||
{{else}} output: {{.LogOutput}}
|
||||
{{end}} redactKeys: []
|
||||
# Optional cross-origin access: server.cors (disabled by default). See framework cors/README.md.
|
||||
address: "${GO_OMNI_HTTP_ADDR:-127.0.0.1:8080}"
|
||||
transport: "${GO_OMNI_TRANSPORT:-nethttp}"
|
||||
{{if ne .Storage "none"}}storage:
|
||||
provider: {{.Storage}}
|
||||
maxBytes: 33554432
|
||||
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||
linkTtl: 5m
|
||||
requestTimeout: 30s
|
||||
{{if eq .Storage "local"}} root: './var/files'
|
||||
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
|
||||
secure: true
|
||||
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
|
||||
region: auto
|
||||
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
|
||||
region: us-east-1
|
||||
{{else}} region: cn-hangzhou
|
||||
endpoint: '' # empty uses the official regional endpoint
|
||||
{{if not .API}} filesToken: "${GO_OMNI_FILES_TOKEN}"
|
||||
{{end}}{{if eq .Storage "local"}} root: "${GO_OMNI_STORAGE_ROOT:-var/files}"
|
||||
publicUrl: "${GO_OMNI_STORAGE_PUBLIC_URL:-http://127.0.0.1:8080/storage/object}"
|
||||
signingKey: "${GO_OMNI_STORAGE_SIGNING_KEY}"
|
||||
{{else}} bucket: "${GO_OMNI_STORAGE_BUCKET}"
|
||||
accessKey: "${GO_OMNI_STORAGE_ACCESS_KEY}"
|
||||
secretKey: "${GO_OMNI_STORAGE_SECRET_KEY}"
|
||||
sessionToken: "${GO_OMNI_STORAGE_SESSION_TOKEN:-}"
|
||||
{{if eq .Storage "r2"}} accountId: "${GO_OMNI_STORAGE_ACCOUNT_ID}"
|
||||
{{else if eq .Storage "s3"}} endpoint: "${GO_OMNI_STORAGE_ENDPOINT}"
|
||||
region: "${GO_OMNI_STORAGE_REGION:-us-east-1}"
|
||||
{{else}} region: "${GO_OMNI_STORAGE_REGION:-cn-hangzhou}"
|
||||
{{end}}{{end}}{{end}}
|
||||
|
||||
{{if and .API (eq .Database "mysql")}}database:
|
||||
host: 127.0.0.1
|
||||
port: 3306
|
||||
database: '' # GO_OMNI_DB_DATABASE
|
||||
user: '' # GO_OMNI_DB_USER; password via GO_OMNI_DB_PASSWORD
|
||||
tls:
|
||||
mode: required
|
||||
pool:
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
host: "${GO_OMNI_DB_HOST:-127.0.0.1}"
|
||||
port: "${GO_OMNI_DB_PORT:-3306}"
|
||||
database: "${GO_OMNI_DB_DATABASE}"
|
||||
user: "${GO_OMNI_DB_USER}"
|
||||
password: "${GO_OMNI_DB_PASSWORD}"
|
||||
{{end}}
|
||||
{{if and (not .API) (eq .Database "mysql")}}database:
|
||||
driver: mysql
|
||||
dsn: "${GO_OMNI_DB_DSN}"
|
||||
{{end}}
|
||||
|
||||
@@ -18,8 +18,10 @@ import (
|
||||
)
|
||||
|
||||
func main(){
|
||||
path:=flag.String("config","etc/config.yaml","JSON/YAML settings; empty uses defaults and environment")
|
||||
path:=flag.String("config","etc/config.yaml","configuration file; empty uses only Default()")
|
||||
check:=flag.Bool("check-config",false,"validate without starting dependencies")
|
||||
flag.Parse()
|
||||
if *check {if _,err:=settings.Load(*path);err!=nil{log.Fatal(err)};return}
|
||||
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||
}
|
||||
func run(path string)(runErr error){
|
||||
@@ -31,7 +33,7 @@ func run(path string)(runErr error){
|
||||
if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||
logx.SetDefault(logger)
|
||||
services,err:=servicecontext.New(ctx,logger)
|
||||
services,err:=servicecontext.New(ctx,logger,cfg)
|
||||
if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,services.Close())}()
|
||||
{{if ne .Storage "none"}}if err=services.InitStorage(ctx,cfg.Storage);err!=nil{return err}{{end}}
|
||||
|
||||
@@ -3,21 +3,19 @@ package settings
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
{{if ne .Storage "none"}}"fmt"
|
||||
{{if ne .Storage "none"}}
|
||||
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||
"strconv"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||
{{if eq .Database "mysql"}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
{{if eq .LogOutput "file"}}"git.yuntuo.hk/k2safe/go-omni/logx"{{end}}
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Server config.Config `json:"server"`
|
||||
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
|
||||
{{if eq .Database "mysql"}}Database database.Config `json:"database"`{{end}}
|
||||
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
|
||||
}
|
||||
|
||||
@@ -31,7 +29,8 @@ func Default() Config {
|
||||
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
|
||||
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
|
||||
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
|
||||
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||
{{if eq .Database "mysql"}}db:=database.DefaultConfig();db.Driver="mysql"{{end}}
|
||||
return Config{Server:server{{if eq .Database "mysql"}},Database:db{{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
|
||||
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
|
||||
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
|
||||
@@ -41,27 +40,10 @@ func Default() Config {
|
||||
}{{end}}}
|
||||
}
|
||||
|
||||
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||
// Pass an empty path to use only defaults and environment.
|
||||
func Load(path string)(Config,error) {
|
||||
cfg:=Default()
|
||||
var err error
|
||||
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
|
||||
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
|
||||
if err!=nil{return Config{},err}
|
||||
// A stream explicitly selected by environment overrides a file profile.
|
||||
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
|
||||
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
|
||||
{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
|
||||
if err=cfg.Validate();err!=nil{return Config{},err}
|
||||
return cfg,nil
|
||||
}
|
||||
// Load uses Default(), the selected file's explicit references, and Validate.
|
||||
func Load(path string)(Config,error){return config.LoadProject(path,Default())}
|
||||
func (c Config) Validate()error {
|
||||
{{if and .API (eq .Database "mysql")}}
|
||||
{{if eq .Database "mysql"}}
|
||||
if err:=c.Database.Validate();err!=nil{return err}
|
||||
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
|
||||
{{end}}
|
||||
@@ -85,45 +67,27 @@ type StorageConfig struct {
|
||||
ContentTypes []string `json:"contentTypes"`
|
||||
LinkTTL time.Duration `json:"linkTtl"`
|
||||
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||
// Secrets are environment-only and cannot be populated by config files.
|
||||
FilesToken string `json:"-"`
|
||||
SigningKey string `json:"-"`
|
||||
AccessKey string `json:"-"`
|
||||
SecretKey string `json:"-"`
|
||||
SessionToken string `json:"-"`
|
||||
// Use explicit environment references in the configuration file for secrets.
|
||||
FilesToken string `json:"filesToken"`
|
||||
SigningKey string `json:"signingKey"`
|
||||
AccessKey string `json:"accessKey"`
|
||||
SecretKey string `json:"secretKey"`
|
||||
SessionToken string `json:"sessionToken"`
|
||||
}
|
||||
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||
func (s *StorageConfig) applyEnv()error {
|
||||
values:=map[string]*string{
|
||||
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
|
||||
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
|
||||
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
|
||||
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
|
||||
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
|
||||
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
|
||||
}
|
||||
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
|
||||
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
|
||||
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
|
||||
}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
|
||||
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
|
||||
return nil
|
||||
}
|
||||
func (s StorageConfig) Validate()error {
|
||||
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
|
||||
if err:=s.Policy().Validate();err!=nil{return err}
|
||||
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("storage.filesToken requires at least 32 bytes without whitespace")}{{end}}
|
||||
{{if eq .Storage "local"}}
|
||||
if s.Root=="" {return errors.New("storage.root is required")}
|
||||
u,err:=url.Parse(s.PublicURL)
|
||||
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||
if len(s.SigningKey)<32 {return errors.New("storage.signingKey requires a stable secret of at least 32 bytes")}
|
||||
{{else}}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region, accessKey and secretKey are required")}
|
||||
{{if eq .Storage "r2"}}
|
||||
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||
|
||||
@@ -21,18 +21,18 @@ func environment(t *testing.T){
|
||||
}
|
||||
func TestSettingsPrecedenceAndStrictness(t *testing.T){
|
||||
environment(t)
|
||||
{{if eq .Database "mysql"}}t.Setenv("GO_OMNI_DB_DSN","test:secret@tcp(localhost)/demo"){{end}}
|
||||
source,err:=os.ReadFile("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||
path:=filepath.Join(t.TempDir(),"config.yaml")
|
||||
if err:=os.WriteFile(path,[]byte("server:\n address: '127.0.0.1:8123'\n log:\n level: debug\n"),0600);err!=nil{t.Fatal(err)}
|
||||
source=[]byte(strings.Replace(string(source),"${GO_OMNI_HTTP_ADDR:-127.0.0.1:8080}","127.0.0.1:8123",1))
|
||||
if err=os.WriteFile(path,source,0600);err!=nil{t.Fatal(err)}
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR","127.0.0.1:9123")
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout")
|
||||
cfg,err:=Load(path);if err!=nil{t.Fatal(err)}
|
||||
if cfg.Server.Address!="127.0.0.1:9123" || cfg.Server.Log.Level!="debug" || cfg.Server.Log.Output!="stdout" || cfg.Server.Log.File!="" || cfg.Server.Log.Rotation.Enabled {t.Fatal("configuration precedence failed")}
|
||||
for _,body:=range []string{"unexpected: true","server:\n slowRequestThreshold: -1s","server:\n log:\n format: invalid"{{if ne .Storage "none"}},"storage:\n filesToken: forbidden","storage:\n maxBytes: -1","storage:\n provider: unsupported"{{end}}}{
|
||||
if err:=os.WriteFile(path,[]byte(body),0600);err!=nil{t.Fatal(err)}
|
||||
if _,err:=Load(path);err==nil{t.Fatal("invalid configuration accepted")}
|
||||
initial:=Default()
|
||||
if cfg.Server.Address!="127.0.0.1:8123"||cfg.Server.Log.Output!=initial.Server.Log.Output||cfg.Server.Log.File!=initial.Server.Log.File {t.Fatal("implicit overwrite")}
|
||||
for _,body:=range []string{"unexpected: true","server:\n slowRequestThreshold: -1s","server:\n log:\n format: invalid"}{
|
||||
if err=os.WriteFile(path,[]byte(body),0600);err!=nil{t.Fatal(err)}
|
||||
if _,err=Load(path);err==nil{t.Fatal("invalid configuration accepted")}
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
t.Setenv("GO_OMNI_FILES_TOKEN","")
|
||||
if _,err:=Load("");err==nil{t.Fatal("missing file token accepted")}
|
||||
{{end}}
|
||||
}
|
||||
|
||||
@@ -10,11 +10,11 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||
// downloaded module version, which also has a module checksum.
|
||||
const defaultFrameworkVersion = "v0.17.0-preview"
|
||||
const defaultFrameworkVersion = "v0.26.0-preview"
|
||||
|
||||
// This target is not a claim that the release already exists. Source users
|
||||
// select a local framework checkout until the module templates are published.
|
||||
const modulesFrameworkVersion = "v0.7.0-preview"
|
||||
const modulesFrameworkVersion = "v0.22.0-preview"
|
||||
|
||||
func installedFrameworkVersion() string {
|
||||
info, _ := debug.ReadBuildInfo()
|
||||
|
||||
@@ -1,66 +0,0 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestUnifiedEnvAliases(t *testing.T) {
|
||||
type project struct {
|
||||
Server Config `json:"server"`
|
||||
Label string `json:"label"`
|
||||
}
|
||||
base := project{Server: Default(), Label: "base"}
|
||||
for _, policy := range []string{"allow", "ignore", "reject"} {
|
||||
t.Run(policy, func(t *testing.T) {
|
||||
env := map[string]string{"TEST_HTTP_ADDR": "127.0.0.1:8099", "TEST_LOG_SERVICE": "", "TEST_SERVICE": "", "TEST_LABEL": ""}
|
||||
opts := EnvOptions{Prefix: "TEST_", Empty: policy, Aliases: ServerEnvAliases("TEST_", "server"), Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
|
||||
got, err := OverlayEnv(base, opts)
|
||||
if policy == "reject" {
|
||||
if err == nil || !strings.Contains(err.Error(), "empty") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Server.Address != "127.0.0.1:8099" {
|
||||
t.Fatal("alias ignored")
|
||||
}
|
||||
if policy == "allow" && got.Label != "" || policy == "ignore" && got.Label != "base" {
|
||||
t.Fatal("empty policy differs")
|
||||
}
|
||||
env["TEST_SERVER_ADDRESS"] = "127.0.0.1:9001"
|
||||
env["TEST_HTTP_READ_TIMEOUT"] = "3s"
|
||||
env["TEST_ENABLE_LOGGER"] = "false"
|
||||
env["TEST_TRUSTED_PROXIES"] = `["127.0.0.1"]`
|
||||
got, err = OverlayEnv(base, opts)
|
||||
if err != nil || got.Server.Address != "127.0.0.1:9001" || got.Server.HTTP.ReadTimeout != 3*time.Second || got.Server.EnableLogger || len(got.Server.TrustedProxies) != 1 {
|
||||
t.Fatal(got, err)
|
||||
}
|
||||
env["TEST_SERVER_ADDRESS"] = ""
|
||||
got, err = OverlayEnv(base, opts)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
expected := ""
|
||||
if policy == "ignore" {
|
||||
expected = base.Server.Address
|
||||
}
|
||||
if got.Server.Address != expected {
|
||||
t.Fatal("empty canonical unexpectedly fell through to alias")
|
||||
}
|
||||
})
|
||||
}
|
||||
if base.Server.Address != ":8080" || base.Label != "base" {
|
||||
t.Fatal("defaults mutated")
|
||||
}
|
||||
}
|
||||
func TestAliasTypeErrorsDoNotExposeValue(t *testing.T) {
|
||||
opts := EnvOptions{Prefix: "X_", Empty: "reject", Aliases: ServerEnvAliases("X_", ""), Lookup: func(k string) (string, bool) { return "private-invalid", k == "X_HTTP_READ_TIMEOUT" }}
|
||||
if _, err := OverlayEnv(Default(), opts); err == nil || strings.Contains(err.Error(), "private-invalid") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
336
config/bind.go
Normal file
336
config/bind.go
Normal file
@@ -0,0 +1,336 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
var jsonUnmarshalerType = reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
|
||||
|
||||
type documentBinder struct {
|
||||
doc Document
|
||||
nodes int
|
||||
}
|
||||
|
||||
func (b *documentBinder) step(path string, depth int) error {
|
||||
b.nodes++
|
||||
if depth > maxConfigDepth || b.nodes > maxConfigNodes {
|
||||
return b.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func bindDocument[T any](doc Document, defaults T) (T, error) {
|
||||
var zero T
|
||||
base := reflect.ValueOf(&defaults).Elem()
|
||||
if base.Kind() != reflect.Struct && (base.Kind() != reflect.Pointer || base.Type().Elem().Kind() != reflect.Struct) {
|
||||
return zero, &Error{Reason: "project configuration must be a struct"}
|
||||
}
|
||||
result, err := cloneConfiguration(base, 0)
|
||||
if err != nil {
|
||||
return zero, &Error{Reason: "cannot clone configuration defaults", cause: err}
|
||||
}
|
||||
b := documentBinder{doc: doc}
|
||||
if err = b.bind(doc.Value, result, "", 0, false); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result.Interface().(T), nil
|
||||
}
|
||||
|
||||
func (b *documentBinder) decode(v any, dst reflect.Value, path string) (err error) {
|
||||
defer func() {
|
||||
if recover() != nil {
|
||||
err = b.doc.diagnostic(path, "custom value decoder panicked", nil)
|
||||
}
|
||||
}()
|
||||
raw, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
return b.doc.diagnostic(path, "unsupported configuration value", err)
|
||||
}
|
||||
if err = json.Unmarshal(raw, dst.Addr().Interface()); err != nil {
|
||||
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (b *documentBinder) bind(v any, dst reflect.Value, path string, depth int, quoted bool) error {
|
||||
if err := b.step(path, depth); err != nil {
|
||||
return err
|
||||
}
|
||||
if text, ok := v.(resolvedText); ok {
|
||||
if dst.Kind() != reflect.Pointer && dst.Type() != durationType && dst.CanAddr() && !dst.Addr().Type().Implements(jsonUnmarshalerType) && !dst.Addr().Type().Implements(textType) {
|
||||
switch dst.Kind() {
|
||||
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64:
|
||||
if err := parseTextValue(dst, string(text)); err != nil {
|
||||
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if dst.Kind() != reflect.Pointer {
|
||||
v = string(text)
|
||||
}
|
||||
}
|
||||
if v == nil {
|
||||
return b.decode(nil, dst, path)
|
||||
} // Preserve encoding/json null semantics.
|
||||
if dst.Kind() == reflect.Pointer {
|
||||
if dst.IsNil() {
|
||||
dst.Set(reflect.New(dst.Type().Elem()))
|
||||
}
|
||||
return b.bind(v, dst.Elem(), path, depth+1, quoted)
|
||||
}
|
||||
if dst.Type() == durationType {
|
||||
var raw string
|
||||
switch x := v.(type) {
|
||||
case string:
|
||||
raw = x
|
||||
case Text:
|
||||
raw = string(x)
|
||||
default:
|
||||
return b.decode(v, dst, path)
|
||||
}
|
||||
if err := parseTextValue(dst, strings.TrimSpace(raw)); err != nil {
|
||||
return b.doc.diagnostic(path, "expected a duration such as 5s", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if dst.CanAddr() && dst.Addr().Type().Implements(jsonUnmarshalerType) {
|
||||
normalized, err := b.dynamic(v, path, depth+1)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return b.decode(normalized, dst, path)
|
||||
}
|
||||
if dst.CanAddr() && dst.Addr().Type().Implements(textType) {
|
||||
if text, ok := v.(Text); ok {
|
||||
v = string(text)
|
||||
}
|
||||
return b.decode(v, dst, path)
|
||||
}
|
||||
if text, ok := v.(Text); ok {
|
||||
if dst.Kind() == reflect.String || dst.Kind() == reflect.Interface || (dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8) {
|
||||
return b.decode(string(text), dst, path)
|
||||
}
|
||||
switch dst.Kind() {
|
||||
case reflect.Bool, reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64, reflect.Float32, reflect.Float64:
|
||||
if err := parseTextValue(dst, strings.TrimSpace(string(text))); err != nil {
|
||||
return b.doc.diagnostic(path, "expected a valid "+dst.Type().String(), err)
|
||||
}
|
||||
return nil
|
||||
default:
|
||||
return b.doc.diagnostic(path, "expected an object or array, not scalar text", nil)
|
||||
}
|
||||
}
|
||||
if quoted {
|
||||
s, ok := v.(string)
|
||||
if !ok {
|
||||
return b.doc.diagnostic(path, "expected a JSON-encoded string", nil)
|
||||
}
|
||||
if err := json.Unmarshal([]byte(s), dst.Addr().Interface()); err != nil {
|
||||
return b.doc.diagnostic(path, "invalid string-encoded value", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
switch dst.Kind() {
|
||||
case reflect.Struct:
|
||||
m, ok := v.(map[string]any)
|
||||
if !ok {
|
||||
return b.doc.diagnostic(path, "expected an object", nil)
|
||||
}
|
||||
fields := configurationFields(dst.Type())
|
||||
used := map[string]bool{}
|
||||
for _, name := range sortedKeys(m) {
|
||||
child := joinField(path, name)
|
||||
f, ok := fields[name]
|
||||
canonical := name
|
||||
if !ok {
|
||||
for n, field := range fields {
|
||||
if strings.EqualFold(n, name) {
|
||||
if ok {
|
||||
return b.doc.diagnostic(child, "ambiguous field name", nil)
|
||||
}
|
||||
f, ok, canonical = field, true, n
|
||||
}
|
||||
}
|
||||
}
|
||||
if !ok {
|
||||
return b.doc.diagnostic(child, unknownFieldReason(name, fields), nil)
|
||||
}
|
||||
if used[canonical] {
|
||||
return b.doc.diagnostic(child, "duplicate field binding", nil)
|
||||
}
|
||||
used[canonical] = true
|
||||
target := dst
|
||||
for _, index := range f.Index {
|
||||
if target.Kind() == reflect.Pointer {
|
||||
if target.IsNil() {
|
||||
if !target.CanSet() {
|
||||
return b.doc.diagnostic(child, "field cannot be assigned", nil)
|
||||
}
|
||||
target.Set(reflect.New(target.Type().Elem()))
|
||||
}
|
||||
target = target.Elem()
|
||||
}
|
||||
target = target.Field(index)
|
||||
}
|
||||
if !target.CanSet() {
|
||||
return b.doc.diagnostic(child, "field cannot be assigned", nil)
|
||||
}
|
||||
if err := b.bind(m[name], target, child, depth+1, strings.Contains(","+f.Tag.Get("json")+",", ",string,")); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
case reflect.Map:
|
||||
m, ok := v.(map[string]any)
|
||||
if !ok || dst.Type().Key().Kind() != reflect.String {
|
||||
return b.doc.diagnostic(path, "expected an object with string keys", nil)
|
||||
}
|
||||
if dst.IsNil() {
|
||||
dst.Set(reflect.MakeMap(dst.Type()))
|
||||
}
|
||||
for _, key := range sortedKeys(m) {
|
||||
value := reflect.New(dst.Type().Elem()).Elem()
|
||||
if err := b.bind(m[key], value, joinField(path, key), depth+1, false); err != nil {
|
||||
return err
|
||||
}
|
||||
k := reflect.New(dst.Type().Key()).Elem()
|
||||
k.SetString(key)
|
||||
dst.SetMapIndex(k, value)
|
||||
}
|
||||
return nil
|
||||
case reflect.Slice, reflect.Array:
|
||||
if _, ok := v.(string); ok && dst.Kind() == reflect.Slice && dst.Type().Elem().Kind() == reflect.Uint8 {
|
||||
return b.decode(v, dst, path)
|
||||
}
|
||||
items, ok := v.([]any)
|
||||
if !ok {
|
||||
return b.doc.diagnostic(path, "expected an array", nil)
|
||||
}
|
||||
if dst.Kind() == reflect.Array && len(items) != dst.Len() {
|
||||
return b.doc.diagnostic(path, "array length does not match its declared type", nil)
|
||||
}
|
||||
if dst.Kind() == reflect.Slice {
|
||||
dst.Set(reflect.MakeSlice(dst.Type(), len(items), len(items)))
|
||||
}
|
||||
for i, value := range items {
|
||||
if err := b.bind(value, dst.Index(i), fmt.Sprintf("%s[%d]", path, i), depth+1, false); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
case reflect.Interface:
|
||||
v, err := b.dynamic(v, path, depth+1)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if dst.Type().NumMethod() == 0 {
|
||||
dst.Set(reflect.ValueOf(v))
|
||||
return nil
|
||||
}
|
||||
return b.decode(v, dst, path)
|
||||
default:
|
||||
return b.decode(v, dst, path)
|
||||
}
|
||||
}
|
||||
func sortedKeys(m map[string]any) []string {
|
||||
keys := make([]string, 0, len(m))
|
||||
for k := range m {
|
||||
keys = append(keys, k)
|
||||
}
|
||||
sort.Strings(keys)
|
||||
return keys
|
||||
}
|
||||
func (b *documentBinder) dynamic(v any, path string, depth int) (any, error) {
|
||||
if err := b.step(path, depth); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch x := v.(type) {
|
||||
case Text:
|
||||
return string(x), nil
|
||||
case resolvedText:
|
||||
return string(x), nil
|
||||
case map[string]any:
|
||||
out := make(map[string]any, len(x))
|
||||
for _, k := range sortedKeys(x) {
|
||||
c, err := b.dynamic(x[k], joinField(path, k), depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[k] = c
|
||||
}
|
||||
return out, nil
|
||||
case []any:
|
||||
out := make([]any, len(x))
|
||||
for i, v := range x {
|
||||
c, err := b.dynamic(v, fmt.Sprintf("%s[%d]", path, i), depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[i] = c
|
||||
}
|
||||
return out, nil
|
||||
default:
|
||||
return v, nil
|
||||
}
|
||||
}
|
||||
|
||||
// Match encoding/json's field promotion: the shallowest declaration wins;
|
||||
// explicit tags win ties, and ambiguous fields remain unavailable.
|
||||
func configurationFields(t reflect.Type) map[string]reflect.StructField {
|
||||
type candidate struct {
|
||||
field reflect.StructField
|
||||
tagged bool
|
||||
}
|
||||
candidates := map[string][]candidate{}
|
||||
var visit func(reflect.Type, []int, map[reflect.Type]bool)
|
||||
visit = func(current reflect.Type, prefix []int, visiting map[reflect.Type]bool) {
|
||||
if visiting[current] {
|
||||
return
|
||||
}
|
||||
visiting[current] = true
|
||||
defer delete(visiting, current)
|
||||
for i := 0; i < current.NumField(); i++ {
|
||||
f := current.Field(i)
|
||||
name := strings.Split(f.Tag.Get("json"), ",")[0]
|
||||
if name == "-" {
|
||||
continue
|
||||
}
|
||||
child := f.Type
|
||||
if child.Kind() == reflect.Pointer {
|
||||
child = child.Elem()
|
||||
}
|
||||
if f.PkgPath != "" && (!f.Anonymous || child.Kind() != reflect.Struct) {
|
||||
continue
|
||||
}
|
||||
f.Index = append(append([]int(nil), prefix...), i)
|
||||
if name == "" && f.Anonymous && child.Kind() == reflect.Struct {
|
||||
visit(child, f.Index, visiting)
|
||||
continue
|
||||
}
|
||||
tagged := name != ""
|
||||
if name == "" {
|
||||
name = f.Name
|
||||
}
|
||||
candidates[name] = append(candidates[name], candidate{f, tagged})
|
||||
}
|
||||
}
|
||||
visit(t, nil, map[reflect.Type]bool{})
|
||||
fields := map[string]reflect.StructField{}
|
||||
for name, list := range candidates {
|
||||
sort.Slice(list, func(i, j int) bool {
|
||||
if len(list[i].field.Index) != len(list[j].field.Index) {
|
||||
return len(list[i].field.Index) < len(list[j].field.Index)
|
||||
}
|
||||
return list[i].tagged && !list[j].tagged
|
||||
})
|
||||
if len(list) > 1 && len(list[0].field.Index) == len(list[1].field.Index) && list[0].tagged == list[1].tagged {
|
||||
continue
|
||||
}
|
||||
fields[name] = list[0].field
|
||||
}
|
||||
return fields
|
||||
}
|
||||
@@ -5,8 +5,10 @@ import (
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/i18n/httpi18n"
|
||||
"git.yuntuo.hk/k2safe/go-omni/lifecycle"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
@@ -28,6 +30,7 @@ type TLSConfig struct {
|
||||
|
||||
// Config controls the default runtime behavior of an App.
|
||||
type Config struct {
|
||||
CORS cors.Config `json:"cors"`
|
||||
Request contract.RequestPolicy `json:"request"`
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
@@ -45,6 +48,8 @@ type Config struct {
|
||||
Response response.Config `json:"response"`
|
||||
ResponseEncoder response.Encoder `json:"-"`
|
||||
LogInstance *logx.Logger `json:"-"`
|
||||
LanguagePolicy *httpi18n.Policy `json:"-"`
|
||||
ErrorLocalizer apierrors.Localizer `json:"-"`
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
@@ -61,20 +66,23 @@ func Default() Config {
|
||||
|
||||
// Validate checks configuration values before an application starts.
|
||||
func (c Config) Validate() error {
|
||||
if err := c.CORS.Validate(); err != nil {
|
||||
return AtField("cors", err)
|
||||
}
|
||||
if err := c.Lifecycle.Validate(); err != nil {
|
||||
return err
|
||||
return AtField("lifecycle", err)
|
||||
}
|
||||
if err := c.Request.Validate(); err != nil {
|
||||
return err
|
||||
return AtField("request", err)
|
||||
}
|
||||
if err := c.Log.Validate(); err != nil {
|
||||
return err
|
||||
return AtField("log", err)
|
||||
}
|
||||
if err := c.Endpoints.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := c.Errors.Validate(); err != nil {
|
||||
return err
|
||||
return AtField("errors", err)
|
||||
}
|
||||
encoder := c.ResponseEncoder
|
||||
if encoder == nil {
|
||||
@@ -90,7 +98,7 @@ func (c Config) Validate() error {
|
||||
return err
|
||||
}
|
||||
if err := c.Response.Validate(); err != nil {
|
||||
return err
|
||||
return AtField("response", err)
|
||||
}
|
||||
if c.SlowRequestThreshold < 0 {
|
||||
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
|
||||
@@ -99,13 +107,13 @@ func (c Config) Validate() error {
|
||||
return &ConfigError{Field: "address", Message: "is required"}
|
||||
}
|
||||
if strings.TrimSpace(c.RequestIDHeader) == "" {
|
||||
return &ConfigError{Field: "request_id_header", Message: "is required"}
|
||||
return &ConfigError{Field: "requestIdHeader", Message: "is required"}
|
||||
}
|
||||
if c.ShutdownTimeout <= 0 {
|
||||
return &ConfigError{Field: "shutdown_timeout", Message: "must be positive"}
|
||||
return &ConfigError{Field: "shutdownTimeout", Message: "must be positive"}
|
||||
}
|
||||
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
|
||||
return &ConfigError{Field: "trusted_proxies", Message: err.Error()}
|
||||
return &Error{Field: "trustedProxies", Reason: "expected IP addresses or CIDRs", cause: err}
|
||||
}
|
||||
return c.ValidateTransport()
|
||||
}
|
||||
@@ -188,12 +196,14 @@ func WithTrustedProxies(proxies ...string) Option {
|
||||
// WithConfig applies a complete configuration loaded from a file or
|
||||
// environment. Later options can still override individual fields.
|
||||
func WithConfig(settings Config) Option {
|
||||
settings.CORS = settings.CORS.Clone()
|
||||
settings.Response = settings.Response.Clone()
|
||||
settings.Errors = settings.Errors.Clone()
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
return func(cfg *Config) {
|
||||
*cfg = settings
|
||||
cfg.CORS = settings.CORS.Clone()
|
||||
cfg.Response = settings.Response.Clone()
|
||||
cfg.Errors = settings.Errors.Clone()
|
||||
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
@@ -201,6 +211,12 @@ func WithConfig(settings Config) Option {
|
||||
}
|
||||
}
|
||||
|
||||
// WithCORS installs an app-local CORS policy, before routing and authentication.
|
||||
func WithCORS(settings cors.Config) Option {
|
||||
settings = settings.Clone()
|
||||
return func(cfg *Config) { cfg.CORS = settings.Clone() }
|
||||
}
|
||||
|
||||
// WithResponse configures this application's default JSON envelope.
|
||||
func WithResponse(settings response.Config) Option {
|
||||
settings = settings.Clone()
|
||||
@@ -240,6 +256,15 @@ func WithErrors(catalog apierrors.Catalog) Option {
|
||||
// WithLogInstance borrows a logger for this App without changing process logging.
|
||||
func WithLogInstance(logger *logx.Logger) Option { return func(c *Config) { c.LogInstance = logger } }
|
||||
|
||||
// WithLanguagePolicy borrows an immutable HTTP language policy. Resource loading
|
||||
// remains explicit in the project; configuration strings are not dictionaries.
|
||||
func WithLanguagePolicy(policy *httpi18n.Policy) Option {
|
||||
return func(c *Config) { c.LanguagePolicy = policy }
|
||||
}
|
||||
func WithErrorLocalizer(localizer apierrors.Localizer) Option {
|
||||
return func(c *Config) { c.ErrorLocalizer = localizer }
|
||||
}
|
||||
|
||||
func WithRequestPolicy(policy contract.RequestPolicy) Option {
|
||||
return func(c *Config) { c.Request = policy }
|
||||
}
|
||||
|
||||
@@ -1,132 +1,6 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
for _, key := range []string{
|
||||
"GO_OMNI_HTTP_ADDR", "GO_OMNI_REQUEST_ID_HEADER", "GO_OMNI_ENABLE_LOGGER",
|
||||
"GO_OMNI_TRUSTED_PROXIES",
|
||||
"GO_OMNI_TRANSPORT", "GO_OMNI_TLS_CERT_FILE", "GO_OMNI_TLS_KEY_FILE",
|
||||
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
||||
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
||||
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
||||
"GO_OMNI_SLOW_REQUEST_THRESHOLD",
|
||||
} {
|
||||
t.Setenv(key, "")
|
||||
}
|
||||
|
||||
cfg, err := LoadFromEnv("")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Address != ":8080" || cfg.Transport != TransportNetHTTP || cfg.TLS != (TLSConfig{}) || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
t.Fatalf("unexpected defaults: %+v", cfg)
|
||||
}
|
||||
if cfg.EnableLogger || cfg.Log.Level != "info" || cfg.Log.Format != "json" || cfg.Log.Output != "stderr" {
|
||||
t.Fatalf("unexpected logger defaults: %+v", cfg.Log)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTransportEnvironment(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport, cert, key, errorField string
|
||||
}{
|
||||
{"default", "", "", "", ""},
|
||||
{"native", "native", "", "", ""},
|
||||
{"nativeTLS", "native", "server.crt", "server.key", ""},
|
||||
{"http2", "http2", "server.crt", "server.key", ""},
|
||||
{"unknown", "typo", "", "", "transport"},
|
||||
{"certOnly", "nethttp", "server.crt", "", "tls"},
|
||||
{"keyOnly", "native", "", "server.key", "tls"},
|
||||
{"http2WithoutTLS", "http2", "", "", "tls"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
t.Setenv("TRANSPORT_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_CERT_FILE", test.cert)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_KEY_FILE", test.key)
|
||||
cfg, err := LoadFromEnv("TRANSPORT_TEST_")
|
||||
if test.errorField != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorField) {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := test.transport
|
||||
if want == "" {
|
||||
want = TransportNetHTTP
|
||||
}
|
||||
if cfg.Transport != want || cfg.TLS.CertFile != test.cert || cfg.TLS.KeyFile != test.key {
|
||||
t.Fatalf("settings: %+v", cfg)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:9090")
|
||||
t.Setenv("GO_OMNI_REQUEST_ID_HEADER", "X-Correlation-ID")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "127.0.0.1, 10.0.0.0/8")
|
||||
t.Setenv("GO_OMNI_ENABLE_LOGGER", "true")
|
||||
t.Setenv("GO_OMNI_LOG_LEVEL", "debug")
|
||||
t.Setenv("GO_OMNI_LOG_FORMAT", "text")
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT", "stdout")
|
||||
t.Setenv("GO_OMNI_SERVICE", "billing")
|
||||
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
|
||||
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
|
||||
t.Setenv("GO_OMNI_SLOW_REQUEST_THRESHOLD", "500ms")
|
||||
|
||||
cfg, err := LoadFromEnv("")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Address != "127.0.0.1:9090" || cfg.RequestIDHeader != "X-Correlation-ID" || !cfg.EnableLogger || len(cfg.TrustedProxies) != 2 || cfg.TrustedProxies[1] != "10.0.0.0/8" {
|
||||
t.Fatalf("unexpected application settings: %+v", cfg)
|
||||
}
|
||||
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
||||
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
|
||||
}
|
||||
if cfg.SlowRequestThreshold != 500*time.Millisecond {
|
||||
t.Fatal(cfg.SlowRequestThreshold)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_ENABLE_LOGGER", "sometimes")
|
||||
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "ENABLE_LOGGER") {
|
||||
t.Fatalf("expected invalid logger flag error, got %v", err)
|
||||
}
|
||||
|
||||
t.Setenv("GO_OMNI_ENABLE_LOGGER", "")
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "later")
|
||||
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "SHUTDOWN_TIMEOUT") {
|
||||
t.Fatalf("expected invalid timeout error, got %v", err)
|
||||
}
|
||||
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com")
|
||||
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trusted_proxies") {
|
||||
t.Fatalf("expected invalid trusted proxy error, got %v", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "")
|
||||
cfg := Default()
|
||||
cfg.Address = ""
|
||||
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "address") {
|
||||
t.Fatalf("expected invalid address error, got %v", err)
|
||||
}
|
||||
cfg = Default()
|
||||
cfg.RequestIDHeader = " "
|
||||
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "request_id_header") {
|
||||
t.Fatalf("expected invalid request header error, got %v", err)
|
||||
}
|
||||
}
|
||||
import "testing"
|
||||
|
||||
func TestWithConfigAndWithLogConfig(t *testing.T) {
|
||||
settings := Default()
|
||||
@@ -147,23 +21,3 @@ func TestWithConfigAndWithLogConfig(t *testing.T) {
|
||||
t.Fatalf("log config was not replaced: %+v", cfg.Log)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyEnvPreservesProjectDefaults(t *testing.T) {
|
||||
cfg := Default()
|
||||
cfg.Address = "127.0.0.1:8123"
|
||||
cfg.Log.RedactKeys = []string{"bankAccount"}
|
||||
cfg.TrustedProxies = []string{"127.0.0.1"}
|
||||
t.Setenv("OVERLAY_TEST_LOG_LEVEL", "debug")
|
||||
got, err := ApplyEnv(cfg, "OVERLAY_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Address != cfg.Address || got.Log.Level != "debug" {
|
||||
t.Fatal(got)
|
||||
}
|
||||
got.Log.RedactKeys[0] = "changed"
|
||||
got.TrustedProxies[0] = "192.0.2.1"
|
||||
if cfg.Log.RedactKeys[0] != "bankAccount" || cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("caller-owned defaults mutated")
|
||||
}
|
||||
}
|
||||
|
||||
51
config/cors_test.go
Normal file
51
config/cors_test.go
Normal file
@@ -0,0 +1,51 @@
|
||||
package config_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
)
|
||||
|
||||
type corsProject struct {
|
||||
Server config.Config `json:"server"`
|
||||
}
|
||||
|
||||
func (c corsProject) Validate() error { return config.AtField("server", c.Server.Validate()) }
|
||||
|
||||
func TestCORSConfiguration(t *testing.T) {
|
||||
t.Setenv("WEB_ORIGIN", "https://web.example")
|
||||
for ext, source := range map[string]string{
|
||||
"yaml": "server:\n cors:\n enabled: true\n allowedOrigins:\n - '${WEB_ORIGIN}'\n allowedMethods: [GET, POST]\n allowedHeaders: [Authorization, Content-Type]\n maxAge: 10m\n",
|
||||
"json": `{"server":{"cors":{"enabled":true,"allowedOrigins":["${WEB_ORIGIN}"],"allowedMethods":["GET","POST"],"allowedHeaders":["Authorization","Content-Type"],"maxAge":"10m"}}}`,
|
||||
"xml": `<config><server><cors><enabled type="bool">true</enabled><allowedOrigins type="array"><item>${WEB_ORIGIN}</item></allowedOrigins><allowedMethods type="array"><item>GET</item><item>POST</item></allowedMethods><allowedHeaders type="array"><item>Authorization</item><item>Content-Type</item></allowedHeaders><maxAge>10m</maxAge></cors></server></config>`,
|
||||
} {
|
||||
t.Run(ext, func(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "config."+ext)
|
||||
if err := os.WriteFile(path, []byte(source), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.LoadProject(path, corsProject{Server: config.Default()})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !cfg.Server.CORS.Enabled || cfg.Server.CORS.AllowedOrigins[0] != "https://web.example" || cfg.Server.CORS.MaxAge != 10*time.Minute {
|
||||
t.Fatal(cfg.Server.CORS)
|
||||
}
|
||||
})
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "bad.yaml")
|
||||
if err := os.WriteFile(path, []byte("server:\n cors:\n enabled: true\n allowedOrigins:\n - '${WEB_ORIGIN}'\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("WEB_ORIGIN", "https://user:secret@web.example")
|
||||
_, err := config.LoadProject(path, corsProject{Server: config.Default()})
|
||||
var d *config.Error
|
||||
if !errors.As(err, &d) || d.Field != "server.cors.allowedOrigins[0]" || d.Env != "WEB_ORIGIN" || d.Line != 5 || strings.Contains(err.Error(), "secret") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
398
config/decoders.go
Normal file
398
config/decoders.go
Normal file
@@ -0,0 +1,398 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"encoding/xml"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
type JSONDecoder struct{}
|
||||
type YAMLDecoder struct{}
|
||||
type XMLDecoder struct{}
|
||||
|
||||
type parseState struct {
|
||||
doc Document
|
||||
nodes int
|
||||
}
|
||||
|
||||
func newParseState() *parseState { return &parseState{doc: Document{Positions: map[string]Position{}}} }
|
||||
func (p *parseState) enter(path string, pos Position, depth int) error {
|
||||
p.nodes++
|
||||
p.doc.Positions[path] = pos
|
||||
if depth > maxConfigDepth || p.nodes > maxConfigNodes {
|
||||
return p.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type sourcePositions struct{ starts []int }
|
||||
|
||||
func newSourcePositions(data []byte) sourcePositions {
|
||||
p := sourcePositions{starts: []int{0}}
|
||||
for i, c := range data {
|
||||
if c == '\n' {
|
||||
p.starts = append(p.starts, i+1)
|
||||
}
|
||||
}
|
||||
return p
|
||||
}
|
||||
func (p sourcePositions) at(offset int64) Position {
|
||||
if offset < 0 {
|
||||
offset = 0
|
||||
}
|
||||
i := sort.Search(len(p.starts), func(i int) bool { return int64(p.starts[i]) > offset }) - 1
|
||||
return Position{Line: i + 1, Column: int(offset) - p.starts[i] + 1}
|
||||
}
|
||||
|
||||
var yamlLine = regexp.MustCompile(`line ([0-9]+)`)
|
||||
|
||||
func yamlSyntax(err error) *Error {
|
||||
d := &Error{Reason: "invalid YAML syntax", cause: err}
|
||||
if m := yamlLine.FindStringSubmatch(err.Error()); len(m) == 2 {
|
||||
d.Line, _ = strconv.Atoi(m[1])
|
||||
d.Column = 1
|
||||
}
|
||||
return d
|
||||
}
|
||||
func (JSONDecoder) Decode(data []byte) (Document, error) {
|
||||
p := newParseState()
|
||||
if len(data) > maxConfigBytes {
|
||||
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
|
||||
}
|
||||
if !utf8.Valid(data) {
|
||||
return p.doc, &Error{Reason: "configuration must be UTF-8"}
|
||||
}
|
||||
positions := newSourcePositions(data)
|
||||
d := json.NewDecoder(bytes.NewReader(data))
|
||||
d.UseNumber()
|
||||
var parse func(string, int) (any, error)
|
||||
parse = func(path string, depth int) (any, error) {
|
||||
offset := d.InputOffset()
|
||||
for offset < int64(len(data)) && strings.ContainsRune(" \t\r\n:,", rune(data[offset])) {
|
||||
offset++
|
||||
}
|
||||
if err := p.enter(path, positions.at(offset), depth); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
token, err := d.Token()
|
||||
if err != nil {
|
||||
pos := positions.at(d.InputOffset())
|
||||
var syntax *json.SyntaxError
|
||||
if errors.As(err, &syntax) {
|
||||
pos = positions.at(syntax.Offset - 1)
|
||||
}
|
||||
return nil, &Error{Field: path, Line: pos.Line, Column: pos.Column, Reason: "invalid JSON syntax", cause: err}
|
||||
}
|
||||
if delim, ok := token.(json.Delim); ok {
|
||||
switch delim {
|
||||
case '{':
|
||||
out := map[string]any{}
|
||||
for d.More() {
|
||||
k, e := d.Token()
|
||||
if e != nil {
|
||||
return nil, p.doc.diagnostic(path, "invalid object key", e)
|
||||
}
|
||||
key, ok := k.(string)
|
||||
if !ok {
|
||||
return nil, p.doc.diagnostic(path, "object key must be a string", nil)
|
||||
}
|
||||
child := joinField(path, key)
|
||||
if _, exists := out[key]; exists {
|
||||
return nil, p.doc.diagnostic(child, "duplicate field", nil)
|
||||
}
|
||||
out[key], e = parse(child, depth+1)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
}
|
||||
if _, err = d.Token(); err != nil {
|
||||
return nil, p.doc.diagnostic(path, "unterminated object", err)
|
||||
}
|
||||
return out, nil
|
||||
case '[':
|
||||
out := []any{}
|
||||
for d.More() {
|
||||
v, e := parse(fmt.Sprintf("%s[%d]", path, len(out)), depth+1)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
if _, err = d.Token(); err != nil {
|
||||
return nil, p.doc.diagnostic(path, "unterminated array", err)
|
||||
}
|
||||
return out, nil
|
||||
default:
|
||||
return nil, p.doc.diagnostic(path, "unexpected JSON delimiter", nil)
|
||||
}
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
var err error
|
||||
p.doc.Value, err = parse("", 0)
|
||||
if err == nil {
|
||||
if _, e := d.Token(); e != io.EOF {
|
||||
err = p.doc.diagnostic("", "exactly one JSON document required", e)
|
||||
}
|
||||
}
|
||||
return p.doc, err
|
||||
}
|
||||
|
||||
func (YAMLDecoder) Decode(data []byte) (Document, error) {
|
||||
p := newParseState()
|
||||
if len(data) > maxConfigBytes {
|
||||
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
|
||||
}
|
||||
if !utf8.Valid(data) {
|
||||
return p.doc, &Error{Reason: "configuration must be UTF-8"}
|
||||
}
|
||||
d := yaml.NewDecoder(bytes.NewReader(data))
|
||||
var root yaml.Node
|
||||
if err := d.Decode(&root); err != nil {
|
||||
return p.doc, yamlSyntax(err)
|
||||
}
|
||||
var extra yaml.Node
|
||||
if err := d.Decode(&extra); err != io.EOF {
|
||||
return p.doc, &Error{Reason: "exactly one YAML document required", cause: err}
|
||||
}
|
||||
var walk func(*yaml.Node, string, int) (any, error)
|
||||
walk = func(n *yaml.Node, path string, depth int) (any, error) {
|
||||
if err := p.enter(path, Position{n.Line, n.Column}, depth); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch n.Kind {
|
||||
case yaml.MappingNode:
|
||||
out := map[string]any{}
|
||||
for i := 0; i < len(n.Content); i += 2 {
|
||||
k, v := n.Content[i], n.Content[i+1]
|
||||
if k.Tag != "!!str" {
|
||||
return nil, p.doc.diagnostic(path, "mapping keys must be strings; merge keys are not supported", nil)
|
||||
}
|
||||
child := joinField(path, k.Value)
|
||||
if _, ok := out[k.Value]; ok {
|
||||
return nil, &Error{Field: child, Line: k.Line, Column: k.Column, Reason: "duplicate field"}
|
||||
}
|
||||
x, err := walk(v, child, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[k.Value] = x
|
||||
}
|
||||
return out, nil
|
||||
case yaml.SequenceNode:
|
||||
out := make([]any, len(n.Content))
|
||||
for i, c := range n.Content {
|
||||
var err error
|
||||
out[i], err = walk(c, fmt.Sprintf("%s[%d]", path, i), depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
case yaml.AliasNode:
|
||||
return nil, p.doc.diagnostic(path, "YAML aliases are not supported", nil)
|
||||
case yaml.ScalarNode:
|
||||
switch n.Tag {
|
||||
case "!!str", "!!timestamp":
|
||||
return n.Value, nil
|
||||
case "!!null":
|
||||
return nil, nil
|
||||
case "!!bool", "!!int", "!!float":
|
||||
var v any
|
||||
if err := n.Decode(&v); err != nil {
|
||||
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
|
||||
}
|
||||
raw, err := json.Marshal(v)
|
||||
if err != nil {
|
||||
return nil, p.doc.diagnostic(path, "number must be finite", err)
|
||||
}
|
||||
var out any
|
||||
dec := json.NewDecoder(bytes.NewReader(raw))
|
||||
dec.UseNumber()
|
||||
if err = dec.Decode(&out); err != nil {
|
||||
return nil, p.doc.diagnostic(path, "invalid YAML scalar", err)
|
||||
}
|
||||
return out, nil
|
||||
default:
|
||||
return nil, p.doc.diagnostic(path, "unsupported YAML tag", nil)
|
||||
}
|
||||
}
|
||||
return nil, p.doc.diagnostic(path, "unsupported YAML node", nil)
|
||||
}
|
||||
if len(root.Content) != 1 {
|
||||
return p.doc, &Error{Reason: "one nonempty YAML document required"}
|
||||
}
|
||||
var err error
|
||||
p.doc.Value, err = walk(root.Content[0], "", 0)
|
||||
return p.doc, err
|
||||
}
|
||||
|
||||
// XML uses a config root, element field names, type="array" with item children,
|
||||
// type="object" for empty objects, and type="null" for explicit null. Scalar
|
||||
// text is bound using the destination type; optional number/bool/string types
|
||||
// let dynamic any values preserve their intended JSON type.
|
||||
func (XMLDecoder) Decode(data []byte) (Document, error) {
|
||||
p := newParseState()
|
||||
if len(data) > maxConfigBytes {
|
||||
return p.doc, &Error{Reason: "file exceeds 1 MiB"}
|
||||
}
|
||||
if !utf8.Valid(data) {
|
||||
return p.doc, &Error{Reason: "configuration must be UTF-8"}
|
||||
}
|
||||
d := xml.NewDecoder(bytes.NewReader(data))
|
||||
var parse func(xml.StartElement, string, int, Position) (any, error)
|
||||
parse = func(start xml.StartElement, path string, depth int, pos Position) (any, error) {
|
||||
if err := p.enter(path, pos, depth); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if start.Name.Space != "" {
|
||||
return nil, p.doc.diagnostic(path, "XML namespaces are not supported", nil)
|
||||
}
|
||||
kind := ""
|
||||
attrSeen := false
|
||||
for _, a := range start.Attr {
|
||||
if a.Name.Space != "" || a.Name.Local != "type" || attrSeen {
|
||||
return nil, p.doc.diagnostic(path, "only one XML type attribute is supported", nil)
|
||||
}
|
||||
kind = a.Value
|
||||
attrSeen = true
|
||||
}
|
||||
if depth == 0 && kind == "" {
|
||||
kind = "object"
|
||||
}
|
||||
switch kind {
|
||||
case "", "object", "array", "null", "string", "number", "bool":
|
||||
default:
|
||||
return nil, p.doc.diagnostic(path, "unsupported XML type", nil)
|
||||
}
|
||||
children := map[string]any{}
|
||||
array := []any{}
|
||||
var text strings.Builder
|
||||
for {
|
||||
line, column := d.InputPos()
|
||||
token, err := d.Token()
|
||||
if err != nil {
|
||||
return nil, &Error{Field: path, Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
|
||||
}
|
||||
switch x := token.(type) {
|
||||
case xml.StartElement:
|
||||
if kind != "" && kind != "object" && kind != "array" {
|
||||
return nil, p.doc.diagnostic(path, "scalar XML element cannot contain children", nil)
|
||||
}
|
||||
child := joinField(path, x.Name.Local)
|
||||
if kind == "array" {
|
||||
if x.Name.Local != "item" {
|
||||
return nil, p.doc.diagnostic(path, "XML array children must be item elements", nil)
|
||||
}
|
||||
child = fmt.Sprintf("%s[%d]", path, len(array))
|
||||
} else if _, ok := children[x.Name.Local]; ok {
|
||||
return nil, p.doc.diagnostic(child, "duplicate field; use type=\"array\" for lists", nil)
|
||||
}
|
||||
v, e := parse(x, child, depth+1, Position{line, column})
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
if kind == "array" {
|
||||
array = append(array, v)
|
||||
} else {
|
||||
children[x.Name.Local] = v
|
||||
}
|
||||
case xml.CharData:
|
||||
text.Write([]byte(x))
|
||||
case xml.Comment:
|
||||
case xml.EndElement:
|
||||
raw := text.String()
|
||||
if len(children) > 0 || kind == "array" || kind == "object" {
|
||||
if strings.TrimSpace(raw) != "" {
|
||||
return nil, p.doc.diagnostic(path, "mixed XML text and elements are not supported", nil)
|
||||
}
|
||||
if kind == "array" {
|
||||
return array, nil
|
||||
}
|
||||
return children, nil
|
||||
}
|
||||
switch kind {
|
||||
case "null":
|
||||
if strings.TrimSpace(raw) != "" {
|
||||
return nil, p.doc.diagnostic(path, "null element must be empty", nil)
|
||||
}
|
||||
return nil, nil
|
||||
case "string":
|
||||
return raw, nil
|
||||
case "number":
|
||||
var n any
|
||||
dec := json.NewDecoder(strings.NewReader(strings.TrimSpace(raw)))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&n); err != nil {
|
||||
return nil, p.doc.diagnostic(path, "invalid number", err)
|
||||
}
|
||||
if _, ok := n.(json.Number); !ok {
|
||||
return nil, p.doc.diagnostic(path, "expected number", nil)
|
||||
}
|
||||
if _, e := dec.Token(); e != io.EOF {
|
||||
return nil, p.doc.diagnostic(path, "invalid number", e)
|
||||
}
|
||||
return n, nil
|
||||
case "bool":
|
||||
b, e := strconv.ParseBool(strings.TrimSpace(raw))
|
||||
if e != nil {
|
||||
return nil, p.doc.diagnostic(path, "expected boolean", e)
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
return Text(raw), nil
|
||||
default:
|
||||
return nil, p.doc.diagnostic(path, "XML directives and processing instructions are not supported", nil)
|
||||
}
|
||||
}
|
||||
}
|
||||
found := false
|
||||
prolog := false
|
||||
for {
|
||||
line, column := d.InputPos()
|
||||
token, err := d.Token()
|
||||
if err == io.EOF {
|
||||
break
|
||||
}
|
||||
if err != nil {
|
||||
return p.doc, &Error{Line: line, Column: column, Reason: "invalid XML syntax", cause: err}
|
||||
}
|
||||
switch x := token.(type) {
|
||||
case xml.StartElement:
|
||||
if found || x.Name.Local != "config" {
|
||||
return p.doc, &Error{Line: line, Column: column, Reason: "exactly one config root required"}
|
||||
}
|
||||
found = true
|
||||
p.doc.Value, err = parse(x, "", 0, Position{line, column})
|
||||
if err != nil {
|
||||
return p.doc, err
|
||||
}
|
||||
case xml.CharData:
|
||||
if strings.TrimSpace(string(x)) != "" {
|
||||
return p.doc, &Error{Reason: "unexpected text outside XML root"}
|
||||
}
|
||||
case xml.Comment:
|
||||
case xml.ProcInst:
|
||||
if found || prolog || x.Target != "xml" {
|
||||
return p.doc, &Error{Reason: "XML processing instructions are not supported"}
|
||||
}
|
||||
prolog = true
|
||||
default:
|
||||
return p.doc, &Error{Reason: "XML directives are not supported"}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return p.doc, &Error{Reason: "config root required"}
|
||||
}
|
||||
return p.doc, nil
|
||||
}
|
||||
101
config/diagnostic.go
Normal file
101
config/diagnostic.go
Normal file
@@ -0,0 +1,101 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||
)
|
||||
|
||||
// Error is a value-free diagnostic. Cause may contain private parser details;
|
||||
// errors.Is/As remain available, but Error never formats Cause automatically.
|
||||
type Error struct {
|
||||
File, Field, Env, Reason string
|
||||
Line, Column int
|
||||
cause error
|
||||
}
|
||||
|
||||
func (e *Error) Error() string {
|
||||
where := "config"
|
||||
if e.File != "" {
|
||||
where = fmt.Sprintf("config %q", e.File)
|
||||
}
|
||||
if e.Line > 0 {
|
||||
where += fmt.Sprintf(":%d:%d", e.Line, e.Column)
|
||||
}
|
||||
if e.Env != "" {
|
||||
where += fmt.Sprintf(" env %q", e.Env)
|
||||
}
|
||||
if e.Field != "" {
|
||||
where += fmt.Sprintf(" field %q", e.Field)
|
||||
}
|
||||
return where + ": " + e.Reason
|
||||
}
|
||||
func (e *Error) Unwrap() error { return e.cause }
|
||||
|
||||
// DecodeError is retained for callers of errors.As from earlier releases.
|
||||
type DecodeError = Error
|
||||
|
||||
// Invalid reports a public, value-free validation reason for a configuration field.
|
||||
func Invalid(field, reason string) error { return &ConfigError{Field: field, Message: reason} }
|
||||
|
||||
// AtField prefixes a nested validation error without losing its original cause.
|
||||
// Unstructured validation errors are deliberately not echoed; use Invalid when
|
||||
// a custom validator needs a public reason rather than a private error message.
|
||||
func AtField(field string, err error) error {
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
d := validationError(err)
|
||||
d.Field = joinField(field, d.Field)
|
||||
return d
|
||||
}
|
||||
func validationError(err error) *Error {
|
||||
var d *Error
|
||||
if errors.As(err, &d) {
|
||||
out := *d
|
||||
out.cause = err
|
||||
return &out
|
||||
}
|
||||
var nested *configdiag.Error
|
||||
if errors.As(err, &nested) {
|
||||
return &Error{Field: nested.Field, Reason: nested.Reason, cause: err}
|
||||
}
|
||||
var c *ConfigError
|
||||
if errors.As(err, &c) {
|
||||
return &Error{Field: c.Field, Reason: c.Message, cause: err}
|
||||
}
|
||||
return &Error{Reason: "validation failed", cause: err}
|
||||
}
|
||||
func joinField(parent, child string) string {
|
||||
if parent == "" {
|
||||
return child
|
||||
}
|
||||
if child == "" {
|
||||
return parent
|
||||
}
|
||||
if strings.HasPrefix(child, "[") {
|
||||
return parent + child
|
||||
}
|
||||
return parent + "." + child
|
||||
}
|
||||
|
||||
// Position uses one-based source line/column numbers. Zero means unavailable.
|
||||
type Position struct{ Line, Column int }
|
||||
|
||||
// Document is the common decoder result. Values use JSON-compatible data:
|
||||
// string, bool, json.Number, nil, []any, map[string]any. Text is available for
|
||||
// formats such as XML whose scalar tokens have no inherent JSON type.
|
||||
type Document struct {
|
||||
Value any
|
||||
Positions map[string]Position
|
||||
environments map[string][]string
|
||||
}
|
||||
|
||||
type Text string
|
||||
|
||||
func (d Document) diagnostic(field, reason string, cause error) *Error {
|
||||
p := d.Positions[field]
|
||||
return &Error{Field: field, Reason: reason, Line: p.Line, Column: p.Column, cause: cause}
|
||||
}
|
||||
@@ -18,14 +18,15 @@ func TestEndpointConfiguration(t *testing.T) {
|
||||
if cfg.Endpoints.Liveness.Enabled || cfg.Endpoints.OpenAPI.Enabled || !cfg.Endpoints.Readiness.Enabled || cfg.Endpoints.Readiness.ResponseMode != "raw" {
|
||||
t.Fatal(cfg.Endpoints)
|
||||
}
|
||||
refs := configFile(t, "yaml", "endpoints:\n readiness:\n path: ${TEST_ENDPOINTS_READINESS_PATH}\n openapi:\n enabled: ${TEST_ENDPOINTS_OPENAPI_ENABLED}\n")
|
||||
t.Setenv("TEST_ENDPOINTS_READINESS_PATH", "/internal/ready")
|
||||
t.Setenv("TEST_ENDPOINTS_OPENAPI_ENABLED", "true")
|
||||
cfg, err = ApplyEnv(cfg, "TEST_")
|
||||
cfg, err = LoadProject(refs, cfg)
|
||||
if err != nil || cfg.Endpoints.Readiness.Path != "/internal/ready" || !cfg.Endpoints.OpenAPI.Enabled {
|
||||
t.Fatal(cfg.Endpoints, err)
|
||||
}
|
||||
t.Setenv("TEST_ENDPOINTS_OPENAPI_ENABLED", "invalid")
|
||||
if _, err = ApplyEnv(cfg, "TEST_"); err == nil {
|
||||
if _, err = LoadProject(refs, cfg); err == nil {
|
||||
t.Fatal("invalid bool accepted")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,83 +0,0 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"os"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const defaultEnvPrefix = "GO_OMNI_"
|
||||
|
||||
// ServerEnvAliases returns the documented short names for Config fields.
|
||||
// field is the JSON path of Config in a project (usually "server"), or ""
|
||||
// for Config itself. Canonical names win over aliases. Both use the same
|
||||
// EnvOptions.Empty/Lookup and typed parser; collections use JSON syntax.
|
||||
func ServerEnvAliases(prefix, field string) map[string][]string {
|
||||
root := prefix
|
||||
if field != "" {
|
||||
for _, part := range strings.Split(field, ".") {
|
||||
root += envName(part) + "_"
|
||||
}
|
||||
}
|
||||
names := map[string]string{
|
||||
"ADDRESS": "HTTP_ADDR", "TRANSPORT": "TRANSPORT", "TLS_CERT_FILE": "TLS_CERT_FILE", "TLS_KEY_FILE": "TLS_KEY_FILE",
|
||||
"REQUEST_ID_HEADER": "REQUEST_ID_HEADER", "ENABLE_LOGGER": "ENABLE_LOGGER", "SLOW_REQUEST_THRESHOLD": "SLOW_REQUEST_THRESHOLD",
|
||||
"TRUSTED_PROXIES": "TRUSTED_PROXIES", "SHUTDOWN_TIMEOUT": "SHUTDOWN_TIMEOUT",
|
||||
"LOG_LEVEL": "LOG_LEVEL", "LOG_FORMAT": "LOG_FORMAT", "LOG_OUTPUT": "LOG_OUTPUT", "LOG_FILE": "LOG_FILE", "LOG_SERVICE": "SERVICE", "LOG_ENVIRONMENT": "ENVIRONMENT", "LOG_ADD_SOURCE": "LOG_ADD_SOURCE",
|
||||
"HTTP_READ_HEADER_TIMEOUT": "HTTP_READ_HEADER_TIMEOUT", "HTTP_READ_TIMEOUT": "HTTP_READ_TIMEOUT", "HTTP_WRITE_TIMEOUT": "HTTP_WRITE_TIMEOUT", "HTTP_IDLE_TIMEOUT": "HTTP_IDLE_TIMEOUT", "HTTP_MAX_HEADER_BYTES": "HTTP_MAX_HEADER_BYTES",
|
||||
"LIFECYCLE_STOP_TIMEOUT": "LIFECYCLE_STOP_TIMEOUT", "LIFECYCLE_DRAIN_TIMEOUT": "LIFECYCLE_DRAIN_TIMEOUT", "LIFECYCLE_CANCEL_TIMEOUT": "LIFECYCLE_CANCEL_TIMEOUT",
|
||||
}
|
||||
for _, endpoint := range []string{"LIVENESS", "READINESS", "OPENAPI"} {
|
||||
for _, part := range []string{"ENABLED", "PATH", "RESPONSE_MODE"} {
|
||||
if endpoint == "OPENAPI" && part == "RESPONSE_MODE" {
|
||||
continue
|
||||
}
|
||||
name := "ENDPOINTS_" + endpoint + "_" + part
|
||||
names[name] = name
|
||||
}
|
||||
}
|
||||
aliases := map[string][]string{}
|
||||
for key, alias := range names {
|
||||
if root+key != prefix+alias {
|
||||
aliases[root+key] = []string{prefix + alias}
|
||||
}
|
||||
}
|
||||
return aliases
|
||||
}
|
||||
|
||||
// LoadFromEnv loads defaults using the legacy short-name conventions.
|
||||
func LoadFromEnv(prefix string) (Config, error) { return ApplyEnv(Default(), prefix) }
|
||||
|
||||
// ApplyEnv retains legacy whitespace trimming, ignored empty variables and
|
||||
// comma-separated TRUSTED_PROXIES. New projects should use OverlayEnv with
|
||||
// ServerEnvAliases for one explicit empty-value policy across the whole config.
|
||||
func ApplyEnv(defaults Config, prefix string) (Config, error) {
|
||||
prefix = strings.TrimSpace(prefix)
|
||||
if prefix == "" {
|
||||
prefix = defaultEnvPrefix
|
||||
}
|
||||
lookup := func(key string) (string, bool) {
|
||||
raw, ok := os.LookupEnv(key)
|
||||
raw = strings.TrimSpace(raw)
|
||||
if ok && raw != "" && key == prefix+"TRUSTED_PROXIES" && !strings.HasPrefix(raw, "[") {
|
||||
values := strings.Split(raw, ",")
|
||||
for i := range values {
|
||||
values[i] = strings.TrimSpace(values[i])
|
||||
}
|
||||
data, _ := json.Marshal(values)
|
||||
raw = string(data)
|
||||
}
|
||||
return raw, ok
|
||||
}
|
||||
defaults.Errors = defaults.Errors.Clone()
|
||||
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
|
||||
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
|
||||
cfg, err := OverlayEnv(defaults, EnvOptions{Prefix: prefix, Empty: "ignore", Lookup: lookup, Aliases: ServerEnvAliases(prefix, "")})
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
if err = cfg.Validate(); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
@@ -21,7 +21,7 @@ func TestCatalogConfigFiles(t *testing.T) {
|
||||
if err := os.WriteFile(path, []byte(file.body), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.LoadFile(path, "CATALOG_FIXTURE_")
|
||||
cfg, err := config.LoadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -40,7 +40,7 @@ func TestCatalogConfigFiles(t *testing.T) {
|
||||
if err := os.WriteFile(path, []byte(body), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := config.LoadFile(path, "CATALOG_FIXTURE_"); err == nil {
|
||||
if _, err := config.LoadFile(path); err == nil {
|
||||
t.Fatal("accepted invalid catalog", body)
|
||||
}
|
||||
}
|
||||
@@ -57,7 +57,7 @@ func TestCatalogReloadSnapshotIsolation(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer logger.Close()
|
||||
reloader, err := config.NewReloader(path, "CATALOG_RELOAD_", logger)
|
||||
reloader, err := config.NewReloader(path, logger)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ func TestTemplateConfigFiles(t *testing.T) {
|
||||
if err := os.WriteFile(path, []byte(file.body), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.LoadFile(path, "TEMPLATE_TEST_")
|
||||
cfg, err := config.LoadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
169
config/file.go
169
config/file.go
@@ -1,190 +1,41 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gopkg.in/yaml.v3"
|
||||
)
|
||||
|
||||
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
|
||||
// field names for both formats, rejects unknown fields and accepts durations
|
||||
// such as "3s". Application-specific validation remains with the caller.
|
||||
func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||
result, err := decodeFile(path, defaults)
|
||||
if err != nil {
|
||||
var zero T
|
||||
return zero, &DecodeError{cause: err}
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
// DecodeError keeps input values out of public diagnostics while preserving
|
||||
// errors.Is/As for explicitly handled underlying I/O and syntax errors.
|
||||
type DecodeError struct{ cause error }
|
||||
|
||||
func (e *DecodeError) Error() string {
|
||||
return "config: cannot decode file; check its path, format, field names and value types"
|
||||
}
|
||||
func (e *DecodeError) Unwrap() error { return e.cause }
|
||||
func decodeFile[T any](path string, defaults T) (T, error) {
|
||||
var zero T
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if len(data) > 1<<20 {
|
||||
return zero, errors.New("config: file exceeds 1 MiB")
|
||||
}
|
||||
var value any
|
||||
switch strings.ToLower(filepath.Ext(path)) {
|
||||
case ".yaml", ".yml":
|
||||
dec := yaml.NewDecoder(bytes.NewReader(data))
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one YAML document required")
|
||||
}
|
||||
case ".json":
|
||||
if err := apierrors.DecodeCatalogJSON(data, &value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
default:
|
||||
return zero, errors.New("config: expected .json, .yaml or .yml")
|
||||
}
|
||||
if _, ok := value.(map[string]any); !ok {
|
||||
return zero, errors.New("config: root must be an object")
|
||||
}
|
||||
value, err = durations(value, reflect.TypeOf(defaults))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
// Clone configuration without serializing borrowed json:"-" capabilities.
|
||||
var result T
|
||||
original, err := cloneConfiguration(reflect.ValueOf(&defaults).Elem(), 0)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
reflect.ValueOf(&result).Elem().Set(original)
|
||||
dec := json.NewDecoder(bytes.NewReader(encoded))
|
||||
dec.DisallowUnknownFields()
|
||||
if err = dec.Decode(&result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func durations(v any, t reflect.Type) (any, error) {
|
||||
if t == nil {
|
||||
return v, nil
|
||||
}
|
||||
for t.Kind() == reflect.Pointer {
|
||||
t = t.Elem()
|
||||
}
|
||||
if t == reflect.TypeOf(time.Duration(0)) {
|
||||
if s, ok := v.(string); ok {
|
||||
d, err := time.ParseDuration(s)
|
||||
return int64(d), err
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
if t.Kind() == reflect.Struct {
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
for n := 0; n < t.NumField(); n++ {
|
||||
field := t.Field(n)
|
||||
name := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||
if name == "-" {
|
||||
continue
|
||||
}
|
||||
if name == "" {
|
||||
name = field.Name
|
||||
}
|
||||
if item, exists := m[name]; exists {
|
||||
converted, err := durations(item, field.Type)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m[name] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Map && t.Key().Kind() == reflect.String {
|
||||
if values, ok := v.(map[string]any); ok {
|
||||
for key, value := range values {
|
||||
converted, err := durations(value, t.Elem())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
values[key] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Slice {
|
||||
if items, ok := v.([]any); ok {
|
||||
for n, item := range items {
|
||||
converted, err := durations(item, t.Elem())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items[n] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
func LoadFile(path, prefix string) (Config, error) {
|
||||
cfg, err := DecodeFile(path, Default())
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return ApplyEnv(cfg, prefix)
|
||||
}
|
||||
// LoadFile loads and validates framework configuration using explicit file references.
|
||||
func LoadFile(path string) (Config, error) { return LoadProject(path, Default()) }
|
||||
|
||||
// Reloader only accepts log.level changes. Static changes and invalid files
|
||||
// leave the current snapshot and logger untouched. Watch does not own a goroutine.
|
||||
type Reloader struct {
|
||||
mu sync.Mutex
|
||||
path, prefix string
|
||||
current Config
|
||||
logger *logx.Logger
|
||||
mu sync.Mutex
|
||||
path string
|
||||
current Config
|
||||
logger *logx.Logger
|
||||
}
|
||||
|
||||
func NewReloader(path, prefix string, logger *logx.Logger) (*Reloader, error) {
|
||||
cfg, err := LoadFile(path, prefix)
|
||||
func NewReloader(path string, logger *logx.Logger) (*Reloader, error) {
|
||||
cfg, err := LoadFile(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = logger.SetLevel(cfg.Log.Level); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Reloader{path: path, prefix: prefix, current: cfg, logger: logger}, nil
|
||||
return &Reloader{path: path, current: cfg, logger: logger}, nil
|
||||
}
|
||||
func (r *Reloader) Reload() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
next, err := LoadFile(r.path, r.prefix)
|
||||
next, err := LoadFile(r.path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -20,8 +20,8 @@ func TestFileMergeAndSafeReload(t *testing.T) {
|
||||
}
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: info\n")
|
||||
t.Setenv("TEST_MODULE_HTTP_ADDR", ":9123")
|
||||
cfg, err := LoadFile(path, "TEST_MODULE_")
|
||||
if err != nil || cfg.Address != ":9123" || cfg.ShutdownTimeout != 7*time.Second {
|
||||
cfg, err := LoadFile(path)
|
||||
if err != nil || cfg.Address != ":8123" || cfg.ShutdownTimeout != 7*time.Second {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
var output bytes.Buffer
|
||||
@@ -33,7 +33,7 @@ func TestFileMergeAndSafeReload(t *testing.T) {
|
||||
}
|
||||
defer l.Close()
|
||||
child := l.With("scope", "test")
|
||||
r, err := NewReloader(path, "TEST_MODULE_", l)
|
||||
r, err := NewReloader(path, l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
@@ -22,7 +22,7 @@ func TestHTTPFileAndEnvironment(t *testing.T) {
|
||||
if err := os.WriteFile(path, []byte(file.body), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.LoadFile(path, "HTTP_TEST_")
|
||||
cfg, err := config.LoadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -35,7 +35,10 @@ func TestHTTPFileAndEnvironment(t *testing.T) {
|
||||
t.Setenv("HTTP_TEST_HTTP_WRITE_TIMEOUT", "8s")
|
||||
t.Setenv("HTTP_TEST_HTTP_IDLE_TIMEOUT", "9s")
|
||||
t.Setenv("HTTP_TEST_HTTP_MAX_HEADER_BYTES", "16384")
|
||||
got, err := config.ApplyEnv(cfg, "HTTP_TEST_")
|
||||
if err := os.WriteFile(path, []byte(`{"http":{"readHeaderTimeout":"${HTTP_TEST_HTTP_READ_HEADER_TIMEOUT}","readTimeout":"${HTTP_TEST_HTTP_READ_TIMEOUT}","writeTimeout":"${HTTP_TEST_HTTP_WRITE_TIMEOUT}","idleTimeout":"${HTTP_TEST_HTTP_IDLE_TIMEOUT}","maxHeaderBytes":"${HTTP_TEST_HTTP_MAX_HEADER_BYTES}"}}`), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
got, err := config.LoadProject(path, cfg)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -55,7 +58,12 @@ func TestHTTPInvalidAndUnsupportedSettings(t *testing.T) {
|
||||
value = "-1s"
|
||||
}
|
||||
t.Setenv("HTTP_INVALID_HTTP_"+key, value)
|
||||
if _, err := config.LoadFromEnv("HTTP_INVALID_"); err == nil {
|
||||
names := map[string]string{"READ_HEADER_TIMEOUT": "readHeaderTimeout", "READ_TIMEOUT": "readTimeout", "WRITE_TIMEOUT": "writeTimeout", "IDLE_TIMEOUT": "idleTimeout", "MAX_HEADER_BYTES": "maxHeaderBytes"}
|
||||
path := filepath.Join(t.TempDir(), "config.json")
|
||||
if err := os.WriteFile(path, []byte(`{"http":{"`+names[key]+`":"${HTTP_INVALID_HTTP_`+key+`}"}}`), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := config.LoadFile(path); err == nil {
|
||||
t.Fatal("invalid HTTP value accepted")
|
||||
}
|
||||
})
|
||||
|
||||
@@ -14,7 +14,7 @@ func TestLifecycleConfiguration(t *testing.T) {
|
||||
if err := os.WriteFile(path, []byte("lifecycle:\n stopTimeout: 2s\n drainTimeout: 3s\n cancelTimeout: 4s\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err := config.LoadFile(path, "PHASE_TEST_")
|
||||
cfg, err := config.LoadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
@@ -24,13 +24,16 @@ func TestLifecycleConfiguration(t *testing.T) {
|
||||
for _, key := range []string{"STOP_TIMEOUT", "DRAIN_TIMEOUT", "CANCEL_TIMEOUT"} {
|
||||
t.Setenv("PHASE_TEST_LIFECYCLE_"+key, "6s")
|
||||
}
|
||||
cfg, err = config.ApplyEnv(cfg, "PHASE_TEST_")
|
||||
if err := os.WriteFile(path, []byte("lifecycle:\n stopTimeout: ${PHASE_TEST_LIFECYCLE_STOP_TIMEOUT}\n drainTimeout: ${PHASE_TEST_LIFECYCLE_DRAIN_TIMEOUT}\n cancelTimeout: ${PHASE_TEST_LIFECYCLE_CANCEL_TIMEOUT}\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cfg, err = config.LoadProject(path, cfg)
|
||||
if err != nil || cfg.Lifecycle.StopTimeout != 6*time.Second || cfg.Lifecycle.DrainTimeout != 6*time.Second || cfg.Lifecycle.CancelTimeout != 6*time.Second {
|
||||
t.Fatal(cfg.Lifecycle, err)
|
||||
}
|
||||
for _, v := range []string{"bad", "-1s"} {
|
||||
t.Setenv("PHASE_TEST_LIFECYCLE_DRAIN_TIMEOUT", v)
|
||||
if _, err := config.ApplyEnv(cfg, "PHASE_TEST_"); err == nil {
|
||||
if _, err := config.LoadProject(path, cfg); err == nil {
|
||||
t.Fatal("invalid duration accepted")
|
||||
}
|
||||
}
|
||||
|
||||
177
config/loader.go
Normal file
177
config/loader.go
Normal file
@@ -0,0 +1,177 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const maxConfigBytes = 1 << 20
|
||||
const maxConfigDepth = 64
|
||||
const maxConfigNodes = 100000
|
||||
|
||||
// Decoder must not retain or mutate input and must support concurrent calls
|
||||
// when its Loader is shared. It is trusted application code, not a sandbox.
|
||||
type Decoder interface {
|
||||
Decode([]byte) (Document, error)
|
||||
}
|
||||
type DecoderFunc func([]byte) (Document, error)
|
||||
|
||||
func (f DecoderFunc) Decode(data []byte) (Document, error) { return f(data) }
|
||||
|
||||
// Loader owns an immutable per-instance decoder registry. No global registration.
|
||||
type Loader struct {
|
||||
decoders map[string]Decoder
|
||||
sealed bool
|
||||
lookup func(string) (string, bool)
|
||||
}
|
||||
type LoaderOption func(*Loader) error
|
||||
|
||||
// WithDecoder adds a new extension; builtin/duplicate extensions cannot be replaced.
|
||||
func WithDecoder(extension string, decoder Decoder) LoaderOption {
|
||||
return func(l *Loader) error {
|
||||
if l == nil || l.decoders == nil || l.sealed {
|
||||
return errors.New("config: decoder registry cannot be changed after construction")
|
||||
}
|
||||
ext := strings.ToLower(strings.TrimPrefix(extension, "."))
|
||||
if ext == "" || strings.ContainsAny(ext, "./\\ \t\r\n") {
|
||||
return errors.New("config: invalid decoder extension")
|
||||
}
|
||||
if decoder == nil {
|
||||
return errors.New("config: nil decoder")
|
||||
}
|
||||
v := reflect.ValueOf(decoder)
|
||||
if (v.Kind() == reflect.Pointer || v.Kind() == reflect.Func || v.Kind() == reflect.Map || v.Kind() == reflect.Slice || v.Kind() == reflect.Interface) && v.IsNil() {
|
||||
return errors.New("config: nil decoder")
|
||||
}
|
||||
if _, exists := l.decoders[ext]; exists {
|
||||
return errors.New("config: decoder extension already registered")
|
||||
}
|
||||
l.decoders[ext] = decoder
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func NewLoader(options ...LoaderOption) (*Loader, error) {
|
||||
l := &Loader{lookup: os.LookupEnv, decoders: map[string]Decoder{"json": JSONDecoder{}, "yaml": YAMLDecoder{}, "yml": YAMLDecoder{}, "xml": XMLDecoder{}}}
|
||||
for _, o := range options {
|
||||
if o == nil {
|
||||
return nil, errors.New("config: nil loader option")
|
||||
}
|
||||
if err := o(l); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
l.sealed = true
|
||||
return l, nil
|
||||
}
|
||||
|
||||
var defaultLoader, _ = NewLoader()
|
||||
|
||||
func selectLoader(loaders []*Loader) (*Loader, error) {
|
||||
if len(loaders) == 0 {
|
||||
return defaultLoader, nil
|
||||
}
|
||||
if len(loaders) != 1 || loaders[0] == nil {
|
||||
return nil, errors.New("config: expected one non-nil loader")
|
||||
}
|
||||
return loaders[0], nil
|
||||
}
|
||||
func (l *Loader) read(path string) (doc Document, err error) {
|
||||
defer func() {
|
||||
if err != nil {
|
||||
var d *Error
|
||||
if errors.As(err, &d) {
|
||||
c := *d
|
||||
c.File = path
|
||||
err = &c
|
||||
} else {
|
||||
reason := "cannot read configuration file"
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
reason = "configuration file does not exist"
|
||||
} else if errors.Is(err, os.ErrPermission) {
|
||||
reason = "permission denied reading configuration file"
|
||||
}
|
||||
err = &Error{File: path, Reason: reason, cause: err}
|
||||
}
|
||||
}
|
||||
}()
|
||||
decoder := l.decoders[strings.TrimPrefix(strings.ToLower(filepath.Ext(path)), ".")]
|
||||
if decoder == nil {
|
||||
return doc, &Error{Reason: "unsupported configuration format"}
|
||||
}
|
||||
info, e := os.Stat(path)
|
||||
if e != nil {
|
||||
return doc, e
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return doc, &Error{Reason: "configuration must be a regular file"}
|
||||
}
|
||||
f, e := os.Open(path)
|
||||
if e != nil {
|
||||
return doc, e
|
||||
}
|
||||
defer f.Close()
|
||||
info, e = f.Stat()
|
||||
if e != nil {
|
||||
return doc, e
|
||||
}
|
||||
if !info.Mode().IsRegular() {
|
||||
return doc, &Error{Reason: "configuration must be a regular file"}
|
||||
}
|
||||
data, e := io.ReadAll(io.LimitReader(f, maxConfigBytes+1))
|
||||
if e != nil {
|
||||
return doc, e
|
||||
}
|
||||
if len(data) > maxConfigBytes {
|
||||
return doc, &Error{Reason: "file exceeds 1 MiB"}
|
||||
}
|
||||
doc, err = decodeSafely(decoder, data)
|
||||
if err != nil {
|
||||
return doc, err
|
||||
}
|
||||
if _, ok := doc.Value.(map[string]any); !ok {
|
||||
return doc, doc.diagnostic("", "root must be an object", nil)
|
||||
}
|
||||
return resolveDocument(doc, l.lookup)
|
||||
}
|
||||
func decodeSafely(decoder Decoder, data []byte) (doc Document, err error) {
|
||||
defer func() {
|
||||
if recover() != nil {
|
||||
err = &Error{Reason: "decoder panicked"}
|
||||
}
|
||||
}()
|
||||
doc, err = decoder.Decode(data)
|
||||
if err != nil {
|
||||
var d *Error
|
||||
if !errors.As(err, &d) {
|
||||
err = &Error{Reason: "invalid configuration syntax", cause: err}
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// DecodeFile overlays a bounded document onto independent defaults. With no
|
||||
// loader argument it accepts JSON/YAML/XML. It never opens dependencies.
|
||||
func DecodeFile[T any](path string, defaults T, loaders ...*Loader) (T, error) {
|
||||
var zero T
|
||||
l, err := selectLoader(loaders)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
doc, err := l.read(path)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if err := rejectEnvironmentTags(reflect.TypeOf(defaults), map[reflect.Type]bool{}); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
result, err := bindDocument(doc, defaults)
|
||||
if err != nil {
|
||||
return zero, locateError(err, path, doc)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
377
config/loader_test.go
Normal file
377
config/loader_test.go
Normal file
@@ -0,0 +1,377 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type loaderExample struct {
|
||||
Server Config `json:"server"`
|
||||
Upload struct {
|
||||
MaxBytes int64 `json:"maxBytes"`
|
||||
} `json:"upload"`
|
||||
Values []int64 `json:"values"`
|
||||
Optional *string `json:"optional"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Dynamic any `json:"dynamic"`
|
||||
}
|
||||
|
||||
func (c loaderExample) Validate() error {
|
||||
if err := c.Server.Validate(); err != nil {
|
||||
return AtField("server", err)
|
||||
}
|
||||
if c.Upload.MaxBytes < 0 {
|
||||
return Invalid("upload.maxBytes", "must be nonnegative")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func noEnvironment(string) (string, bool) { return "", false }
|
||||
func configFile(t *testing.T, ext, raw string) string {
|
||||
t.Helper()
|
||||
path := filepath.Join(t.TempDir(), "service."+ext)
|
||||
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return path
|
||||
}
|
||||
|
||||
func TestUnifiedFormatAndPrecedence(t *testing.T) {
|
||||
formats := map[string]string{
|
||||
"json": `{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":"${APP_UPLOAD_MAX_BYTES}"},"values":[9007199254740993],"optional":null,"enabled":"${APP_ENABLED}","dynamic":{"id":9007199254740993}}`,
|
||||
"yaml": "server:\n http:\n readTimeout: 2s\nupload:\n maxBytes: \"${APP_UPLOAD_MAX_BYTES}\"\nvalues: [9007199254740993]\noptional: null\nenabled: \"${APP_ENABLED}\"\ndynamic:\n id: 9007199254740993\n",
|
||||
"xml": `<config><server><http><readTimeout>2s</readTimeout></http></server><upload><maxBytes>${APP_UPLOAD_MAX_BYTES}</maxBytes></upload><values type="array"><item>9007199254740993</item></values><optional type="null"/><enabled>${APP_ENABLED}</enabled><dynamic><id type="number">9007199254740993</id></dynamic></config>`,
|
||||
}
|
||||
var reference loaderExample
|
||||
for _, ext := range []string{"json", "yaml", "xml"} {
|
||||
t.Run(ext, func(t *testing.T) {
|
||||
old := "old"
|
||||
base := loaderExample{Server: Default(), Optional: &old}
|
||||
cfg, err := LoadProject(configFile(t, ext, formats[ext]), base, testLoader(func(key string) (string, bool) {
|
||||
v, ok := map[string]string{"APP_UPLOAD_MAX_BYTES": "0", "APP_ENABLED": "false"}[key]
|
||||
return v, ok
|
||||
}))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Upload.MaxBytes != 0 || cfg.Enabled || cfg.Optional != nil || cfg.Values[0] != 9007199254740993 || cfg.Server.HTTP.ReadTimeout != 2*time.Second {
|
||||
t.Fatal("values changed", cfg)
|
||||
}
|
||||
if cfg.Dynamic.(map[string]any)["id"] != json.Number("9007199254740993") {
|
||||
t.Fatal("dynamic integer precision lost")
|
||||
}
|
||||
if ext == "json" {
|
||||
reference = cfg
|
||||
} else if !reflect.DeepEqual(reference, cfg) {
|
||||
t.Fatal("format-dependent semantics")
|
||||
}
|
||||
if base.Optional == nil || *base.Optional != "old" {
|
||||
t.Fatal("defaults mutated")
|
||||
}
|
||||
})
|
||||
}
|
||||
cfg, err := LoadProject("", loaderExample{Server: Default()}, testLoader(noEnvironment))
|
||||
if err != nil || cfg.Upload.MaxBytes != 0 {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfigDiagnosticsAndCauses(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
ext, body, field string
|
||||
line bool
|
||||
}{
|
||||
{"json", "{\n\"upload\": {\"maxBytes\":\"private-value\"}}", "upload.maxBytes", true},
|
||||
{"yaml", "server:\n http:\n readTimeout: private-value", "server.http.readTimeout", true},
|
||||
{"xml", "<config>\n<upload><maxBytes>private-value</maxBytes></upload></config>", "upload.maxBytes", true},
|
||||
{"json", `{"upload":{"maxBytes":-1}}`, "upload.maxBytes", true},
|
||||
{"yaml", "server:\n http:\n readTimeout: -1s", "server.http.readTimeout", true},
|
||||
{"json", `{"upload":{"typo":3}}`, "upload.typo", true},
|
||||
} {
|
||||
t.Run(tc.ext+tc.field, func(t *testing.T) {
|
||||
path := configFile(t, tc.ext, tc.body)
|
||||
_, err := LoadProject(path, loaderExample{Server: Default()}, testLoader(noEnvironment))
|
||||
var d *Error
|
||||
if !errors.As(err, &d) || d.File != path || d.Field != tc.field || d.Reason == "" || tc.line && d.Line < 1 {
|
||||
t.Fatalf("missing location: %#v %v", d, err)
|
||||
}
|
||||
if strings.Contains(err.Error(), "private-value") {
|
||||
t.Fatal("secret leaked", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
path := filepath.Join(t.TempDir(), "missing.json")
|
||||
_, err := LoadProject(path, loaderExample{})
|
||||
if !errors.Is(err, os.ErrNotExist) {
|
||||
t.Fatal("I/O cause discarded", err)
|
||||
}
|
||||
_, err = DecodeFile(configFile(t, "json", `{"count":"private-value"}`), struct {
|
||||
Count int `json:"count"`
|
||||
}{})
|
||||
var typed *json.UnmarshalTypeError
|
||||
if !errors.As(err, &typed) {
|
||||
t.Fatal("type cause discarded", err)
|
||||
}
|
||||
_, err = LoadProject(configFile(t, "yaml", "upload:\n maxBytes: \"${APP_UPLOAD_MAX_BYTES}\""), loaderExample{Server: Default()}, testLoader(func(k string) (string, bool) { return "-1", k == "APP_UPLOAD_MAX_BYTES" }))
|
||||
var d *Error
|
||||
if !errors.As(err, &d) || d.Env != "APP_UPLOAD_MAX_BYTES" || d.Field != "upload.maxBytes" || d.File == "" || d.Line != 2 {
|
||||
t.Fatal("missing file and variable provenance", err)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func TestMalformedFormatsAndLimits(t *testing.T) {
|
||||
for ext, bodies := range map[string][]string{
|
||||
"json": {`{"enabled":true,"enabled":false}`, `{} {}`, `{"upload":`, `{"enabled":true,"Enabled":false}`, strings.Repeat(`{"a":`, 70) + `0` + strings.Repeat(`}`, 70)},
|
||||
"yaml": {"enabled: true\nenabled: false", "---\n{}\n---\n{}", "a: &x [1]\nb: *x", "a:\n <<: {}", "upload: [\n"},
|
||||
"xml": {`<!DOCTYPE config><config/>`, `<config>&private-value;</config>`, `<config/><config/>`, `<config xmlns="x"/>`, `<config type="" type=""/>`, `<config><enabled>true</enabled><enabled>false</enabled></config>`, `<config>text<enabled>true</enabled></config>`, `<config><values type="array"><bad>1</bad></values></config>`},
|
||||
} {
|
||||
for i, raw := range bodies {
|
||||
t.Run(fmt.Sprintf("%s%d", ext, i), func(t *testing.T) {
|
||||
_, err := LoadProject(configFile(t, ext, raw), loaderExample{Server: Default()}, testLoader(noEnvironment))
|
||||
if err == nil || strings.Contains(err.Error(), "private-value") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
if _, err := DecodeFile(configFile(t, "json", strings.Repeat(" ", maxConfigBytes+1)), loaderExample{}); err == nil {
|
||||
t.Fatal("unbounded file")
|
||||
}
|
||||
if _, err := DecodeFile(configFile(t, "json", string([]byte{'{', '"', 0xff, '"', ':', '0', '}'})), loaderExample{}); err == nil {
|
||||
t.Fatal("invalid UTF-8")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDecoderRegistryAndConcurrentUse(t *testing.T) {
|
||||
decoder := DecoderFunc(func([]byte) (Document, error) {
|
||||
return Document{Value: map[string]any{"upload": map[string]any{"maxBytes": json.Number("42")}}}, nil
|
||||
})
|
||||
l, err := NewLoader(WithDecoder("custom", decoder))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
path := configFile(t, "custom", "ignored")
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 24; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
cfg, e := LoadProject(path, loaderExample{Server: Default()}, l)
|
||||
if e != nil || cfg.Upload.MaxBytes != 42 {
|
||||
t.Error(e)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if _, err := DecodeFile(path, loaderExample{}); err == nil {
|
||||
t.Fatal("decoder registration leaked globally")
|
||||
}
|
||||
for _, options := range [][]LoaderOption{{WithDecoder("json", decoder)}, {WithDecoder("x", decoder), WithDecoder("x", decoder)}, {WithDecoder("x", DecoderFunc(nil))}} {
|
||||
if _, err = NewLoader(options...); err == nil {
|
||||
t.Fatal("invalid registration")
|
||||
}
|
||||
}
|
||||
l, _ = NewLoader(WithDecoder("panic", DecoderFunc(func([]byte) (Document, error) { panic("private-value") })))
|
||||
_, err = DecodeFile(configFile(t, "panic", ""), loaderExample{}, l)
|
||||
if err == nil || strings.Contains(err.Error(), "private-value") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cycle := map[string]any{}
|
||||
cycle["self"] = cycle
|
||||
l, _ = NewLoader(WithDecoder("cycle", DecoderFunc(func([]byte) (Document, error) { return Document{Value: map[string]any{"dynamic": cycle}}, nil })))
|
||||
if _, err = DecodeFile(configFile(t, "cycle", ""), loaderExample{}, l); err == nil {
|
||||
t.Fatal("decoder bypassed traversal limit")
|
||||
}
|
||||
}
|
||||
|
||||
func FuzzConfigurationDecoders(f *testing.F) {
|
||||
f.Add(uint8(0), []byte(`{"count":1}`))
|
||||
f.Add(uint8(1), []byte("count: 1"))
|
||||
f.Add(uint8(2), []byte("<config><count>1</count></config>"))
|
||||
f.Fuzz(func(t *testing.T, format uint8, data []byte) {
|
||||
if len(data) > 8192 {
|
||||
data = data[:8192]
|
||||
}
|
||||
ds := []Decoder{JSONDecoder{}, YAMLDecoder{}, XMLDecoder{}}
|
||||
doc, err := ds[int(format)%len(ds)].Decode(data)
|
||||
if err == nil {
|
||||
_, _ = bindDocument(doc, struct {
|
||||
Count int `json:"count"`
|
||||
}{})
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
func BenchmarkConfiguration(b *testing.B) {
|
||||
for name, tc := range map[string]struct {
|
||||
d Decoder
|
||||
raw string
|
||||
}{"json": {JSONDecoder{}, `{"count":9007199254740993,"timeout":"5s","values":[1,2,3]}`}, "yaml": {YAMLDecoder{}, "count: 9007199254740993\ntimeout: 5s\nvalues: [1,2,3]"}, "xml": {XMLDecoder{}, `<config><count>9007199254740993</count><timeout>5s</timeout><values type="array"><item>1</item><item>2</item><item>3</item></values></config>`}} {
|
||||
b.Run(name, func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
raw := []byte(tc.raw)
|
||||
for i := 0; i < b.N; i++ {
|
||||
doc, err := tc.d.Decode(raw)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
_, err = bindDocument(doc, struct {
|
||||
Count int64 `json:"count"`
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
Values []int `json:"values"`
|
||||
}{})
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Text-only scalars and embedded config fields use the same behavior in each format.
|
||||
type testText struct{ Value string }
|
||||
|
||||
func (v *testText) UnmarshalText(data []byte) error { v.Value = string(data); return nil }
|
||||
func TestConfigScalarAndEmbedding(t *testing.T) {
|
||||
type Fields struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
type C struct {
|
||||
Fields
|
||||
Nested Fields `json:"nested"`
|
||||
Scalar testText `json:"scalar"`
|
||||
Bytes []byte `json:"bytes"`
|
||||
}
|
||||
cases := map[string]string{
|
||||
"json": `{"name":"outer","nested":{"name":"inner"},"scalar":"text","bytes":"aGk="}`,
|
||||
"yaml": "name: outer\nnested:\n name: inner\nscalar: text\nbytes: aGk=\n",
|
||||
"xml": `<config><name>outer</name><nested><name>inner</name></nested><scalar>text</scalar><bytes>aGk=</bytes></config>`,
|
||||
}
|
||||
for ext, data := range cases {
|
||||
t.Run(ext, func(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "config."+ext)
|
||||
os.WriteFile(path, []byte(data), 0600)
|
||||
got, err := DecodeFile(path, &C{})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Name != "outer" || got.Nested.Name != "inner" || got.Scalar.Value != "text" || string(got.Bytes) != "hi" {
|
||||
t.Fatal(got)
|
||||
}
|
||||
})
|
||||
}
|
||||
type Tagged struct {
|
||||
Fields `json:"nested"`
|
||||
}
|
||||
doc, _ := (JSONDecoder{}).Decode([]byte(`{"name":"wrong"}`))
|
||||
if _, err := bindDocument(doc, Tagged{}); err == nil {
|
||||
t.Fatal("tagged embed incorrectly flattened")
|
||||
}
|
||||
}
|
||||
func TestConfigDefaultsIsolationAndEmptyValues(t *testing.T) {
|
||||
type Child struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
type C struct {
|
||||
Child *Child `json:"child"`
|
||||
Items []int `json:"items"`
|
||||
Labels map[string]string `json:"labels"`
|
||||
Enabled bool `json:"enabled"`
|
||||
}
|
||||
base := C{Child: &Child{Name: "old"}, Items: []int{1}, Labels: map[string]string{"old": "yes"}, Enabled: true}
|
||||
doc, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"new"},"labels":{"new":"yes"},"items":[],"enabled":false}`))
|
||||
got, err := bindDocument(doc, base)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Child.Name != "new" || got.Items == nil || len(got.Items) != 0 || got.Enabled {
|
||||
t.Fatal(got)
|
||||
}
|
||||
if base.Child.Name != "old" || len(base.Items) != 1 || len(base.Labels) != 1 {
|
||||
t.Fatal("mutated defaults")
|
||||
}
|
||||
bad, _ := (JSONDecoder{}).Decode([]byte(`{"child":{"name":"changed"},"items":"bad"}`))
|
||||
if _, err := bindDocument(bad, base); err == nil {
|
||||
t.Fatal("invalid input accepted")
|
||||
}
|
||||
if base.Child.Name != "old" {
|
||||
t.Fatal("failed load changed default pointer")
|
||||
}
|
||||
null, _ := (JSONDecoder{}).Decode([]byte(`{"child":null,"items":null,"labels":null,"enabled":null}`))
|
||||
got, err = bindDocument(null, base)
|
||||
if err != nil || got.Child != nil || got.Items != nil || got.Labels != nil || !got.Enabled {
|
||||
t.Fatal(got, err)
|
||||
}
|
||||
}
|
||||
func TestDefaultStageDoesNotReadEnvironment(t *testing.T) {
|
||||
reads := 0
|
||||
type C struct {
|
||||
Value int `json:"value"`
|
||||
}
|
||||
got, err := LoadProject("", C{Value: 3}, testLoader(func(string) (string, bool) { reads++; return "9", true }))
|
||||
if err != nil || got.Value != 3 || reads != 0 {
|
||||
t.Fatal(got, err, reads)
|
||||
}
|
||||
}
|
||||
|
||||
func testLoader(lookup func(string) (string, bool)) *Loader {
|
||||
l, err := NewLoader(WithLookupEnv(lookup))
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return l
|
||||
}
|
||||
func TestDecoderRegistryFrozen(t *testing.T) {
|
||||
l, err := NewLoader()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = WithDecoder("extra", JSONDecoder{})(l); err == nil {
|
||||
t.Fatal("mutable registry")
|
||||
}
|
||||
}
|
||||
func BenchmarkConfigurationStartup(b *testing.B) {
|
||||
path := filepath.Join(b.TempDir(), "app.json")
|
||||
if err := os.WriteFile(path, []byte(`{"server":{"http":{"readTimeout":"2s"}},"upload":{"maxBytes":4096}}`), 0600); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
|
||||
defaults := loaderExample{Server: Default()}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
if _, err := LoadProject(path, defaults); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
func BenchmarkConfigurationLarge(b *testing.B) {
|
||||
type Entry struct {
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
Count int `json:"count"`
|
||||
}
|
||||
type Large struct {
|
||||
Entries []Entry `json:"entries"`
|
||||
}
|
||||
raw := []byte(`{"entries":[` + strings.TrimSuffix(strings.Repeat(`{"timeout":"3s","count":10},`, 1000), ",") + `]}`)
|
||||
b.SetBytes(int64(len(raw)))
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
doc, err := (JSONDecoder{}).Decode(raw)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
if _, err = bindDocument(doc, Large{}); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
85
config/moduleDiagnostics_test.go
Normal file
85
config/moduleDiagnostics_test.go
Normal file
@@ -0,0 +1,85 @@
|
||||
package config_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
mysql "git.yuntuo.hk/k2safe/go-omni/database/mysql"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpc"
|
||||
)
|
||||
|
||||
type moduleConfig struct {
|
||||
Database mysql.Config `json:"database"`
|
||||
Server config.Config `json:"server"`
|
||||
Client httpc.Config `json:"client"`
|
||||
}
|
||||
|
||||
func (c moduleConfig) Validate() error {
|
||||
if err := c.Database.Validate(); err != nil {
|
||||
return config.AtField("database", err)
|
||||
}
|
||||
if err := c.Server.Validate(); err != nil {
|
||||
return config.AtField("server", err)
|
||||
}
|
||||
return config.AtField("client", c.Client.Validate())
|
||||
}
|
||||
func TestModuleFieldDiagnostics(t *testing.T) {
|
||||
for _, tc := range []struct{ name, source, field, reason string }{
|
||||
{"port", "database:\n port: 70000\n", "database.port", "1 and 65535"},
|
||||
{"host", "database:\n host: 'invalid host'\n", "database.host", "hostname"},
|
||||
{"pool", "database:\n pool:\n maxIdleConns: 999\n", "database.pool.maxIdleConns", "maxOpenConns"},
|
||||
{"ping", "database:\n pool:\n pingTimeout: 0s\n", "database.pool.pingTimeout", "positive"},
|
||||
{"timeout", "database:\n readTimeout: -1s\n", "database.readTimeout", "nonnegative"},
|
||||
{"mysqlZone", "database:\n location: Secret/Not_A_Zone\n", "database.location", "IANA"},
|
||||
{"tlsPair", "database:\n tls:\n certFile: secret-file\n keyFile: ''\n", "database.tls.keyFile", "certFile"},
|
||||
{"level", "server:\n log:\n level: secret-level\n", "server.log.level", "debug"},
|
||||
{"format", "server:\n log:\n format: secret-format\n", "server.log.format", "json or text"},
|
||||
{"rotation", "server:\n log:\n rotation:\n maxBackups: -1\n", "server.log.rotation.maxBackups", "10000"},
|
||||
{"zone", "server:\n log:\n timeZone: Secret/Not_A_Zone\n", "server.log.timeZone", "IANA"},
|
||||
{"lifecycle", "server:\n lifecycle:\n stopTimeout: -1s\n", "server.lifecycle.stopTimeout", "nonnegative"},
|
||||
{"policy", "server:\n request:\n body: secret-mode\n", "server.request.body", "required"},
|
||||
{"clientPool", "client:\n maxIdleConns: -1\n", "client.maxIdleConns", "131072"},
|
||||
{"proxy", "client:\n proxy:\n mode: url\n url: 'https://private:secret@localhost/path'\n", "client.proxy.url", "origin"},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||
if err := os.WriteFile(path, []byte(tc.source), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()})
|
||||
var d *config.Error
|
||||
if !errors.As(err, &d) || d.Field != tc.field || d.File != path || d.Line <= 1 || d.Column <= 0 || !strings.Contains(d.Reason, tc.reason) {
|
||||
t.Fatalf("diagnostic: %v", err)
|
||||
}
|
||||
if strings.Contains(err.Error(), "secret") || strings.Contains(err.Error(), "Secret/") || strings.Contains(err.Error(), "private") {
|
||||
t.Fatalf("value leaked: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestModuleDiagnosticsKeepReferenceAndCause(t *testing.T) {
|
||||
t.Setenv("CHECK_PORT", "70000")
|
||||
path := filepath.Join(t.TempDir(), "config.yaml")
|
||||
os.WriteFile(path, []byte("database:\n port: '${CHECK_PORT}'\n"), 0600)
|
||||
_, err := config.LoadProject(path, moduleConfig{Database: mysql.DefaultConfig(), Server: config.Default()})
|
||||
var d *config.Error
|
||||
if !errors.As(err, &d) || d.Field != "database.port" || d.Env != "CHECK_PORT" || d.Line != 2 {
|
||||
t.Fatal(err)
|
||||
}
|
||||
original := mysql.DefaultConfig()
|
||||
original.Port = 70000
|
||||
cause := original.Validate()
|
||||
wrapped := config.AtField("database", cause)
|
||||
if !errors.Is(wrapped, cause) {
|
||||
t.Fatal("lost original validation error")
|
||||
}
|
||||
// Unknown project errors continue to be redacted rather than echoed.
|
||||
wrapped = config.AtField("custom", errors.New("private-password"))
|
||||
if strings.Contains(wrapped.Error(), "private-password") {
|
||||
t.Fatal(wrapped)
|
||||
}
|
||||
}
|
||||
@@ -1,221 +1,105 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"math"
|
||||
"os"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
// EnvOptions controls typed project configuration overlays. Explicit env tags
|
||||
// are absolute names; otherwise JSON names become PREFIX_NESTED_FIELD names.
|
||||
// Empty is allow (default), ignore or reject. No input values appear in errors.
|
||||
type EnvOptions struct {
|
||||
Prefix string
|
||||
Empty string
|
||||
Lookup func(string) (string, bool)
|
||||
// Aliases maps canonical absolute environment names to ordered fallback names.
|
||||
// A present canonical value wins, including an explicit empty value.
|
||||
Aliases map[string][]string
|
||||
}
|
||||
|
||||
// ApplyDefaults applies envDefault tags to zero fields before file overlays.
|
||||
func ApplyDefaults[T any](base T) (T, error) { return projectOverlay(base, EnvOptions{}, true) }
|
||||
|
||||
// LoadProject applies code/tag defaults, a strict optional file, environment,
|
||||
// then the project's Validate method, before any logger or dependency is opened.
|
||||
// envDefault is only applied to zero-valued defaults before reading the file.
|
||||
func LoadProject[T any](path string, defaults T, options EnvOptions) (T, error) {
|
||||
// LoadProject clones defaults, resolves only explicit file references, binds the
|
||||
// provided fields and validates. Empty path uses only defaults. No env tags,
|
||||
// prefixes, aliases or implicit environment overlays are applied.
|
||||
func LoadProject[T any](path string, defaults T, loaders ...*Loader) (T, error) {
|
||||
var zero T
|
||||
cfg, err := projectOverlay(defaults, options, true)
|
||||
loader, err := selectLoader(loaders)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if err = rejectEnvironmentTags(reflect.TypeOf(defaults), map[reflect.Type]bool{}); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
doc := Document{Value: map[string]any{}}
|
||||
if path != "" {
|
||||
cfg, err = DecodeFile(path, cfg)
|
||||
doc, err = loader.read(path)
|
||||
if err != nil {
|
||||
return zero, errors.New("config: cannot decode file; check its path, format, field names and value types")
|
||||
return zero, err
|
||||
}
|
||||
}
|
||||
cfg, err = OverlayEnv(cfg, options)
|
||||
cfg, err := bindDocument(doc, defaults)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
return zero, locateError(err, path, doc)
|
||||
}
|
||||
if valid, ok := any(&cfg).(interface{ Validate() error }); ok {
|
||||
if err = valid.Validate(); err != nil {
|
||||
return zero, err
|
||||
return zero, locateError(err, path, doc)
|
||||
}
|
||||
} else if valid, ok := any(cfg).(interface{ Validate() error }); ok {
|
||||
if err = valid.Validate(); err != nil {
|
||||
return zero, locateError(err, path, doc)
|
||||
}
|
||||
}
|
||||
return cfg, nil
|
||||
}
|
||||
func locateError(err error, path string, doc Document) *Error {
|
||||
d := validationError(err)
|
||||
d.File = path
|
||||
if d.Line == 0 {
|
||||
p := doc.Positions[d.Field]
|
||||
d.Line = p.Line
|
||||
d.Column = p.Column
|
||||
}
|
||||
for field := d.Field; field != ""; {
|
||||
if names := doc.environments[field]; len(names) > 0 {
|
||||
d.Env = strings.Join(names, ", ")
|
||||
break
|
||||
}
|
||||
i := strings.LastIndexAny(field, ".[")
|
||||
if i < 0 {
|
||||
break
|
||||
}
|
||||
field = field[:i]
|
||||
}
|
||||
return d
|
||||
}
|
||||
|
||||
// OverlayEnv overlays the entire project struct without mutating the input.
|
||||
// Tags: env:"CUSTOM_NAME,required,secret,empty=reject"; env:"-" excludes a
|
||||
// subtree. required means nonempty string/non-nil pointer or collection; false
|
||||
// and zero are valid scalars. Use project Validate for semantic constraints.
|
||||
// Slices/maps use JSON syntax, avoiding ambiguous comma and empty semantics.
|
||||
func OverlayEnv[T any](base T, options EnvOptions) (T, error) {
|
||||
return projectOverlay(base, options, false)
|
||||
// Reject removed declarations rather than silently ignoring deployment rules.
|
||||
func rejectEnvironmentTags(t reflect.Type, seen map[reflect.Type]bool) error {
|
||||
if t == nil || seen[t] {
|
||||
return nil
|
||||
}
|
||||
seen[t] = true
|
||||
switch t.Kind() {
|
||||
case reflect.Pointer, reflect.Slice, reflect.Array, reflect.Map:
|
||||
return rejectEnvironmentTags(t.Elem(), seen)
|
||||
case reflect.Struct:
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
f := t.Field(i)
|
||||
if f.PkgPath != "" {
|
||||
continue
|
||||
}
|
||||
_, env := f.Tag.Lookup("env")
|
||||
_, def := f.Tag.Lookup("envDefault")
|
||||
if env || def {
|
||||
return &Error{Field: f.Name, Reason: "env/envDefault tags were removed; use Default() and explicit file references; see CONFIG.md"}
|
||||
}
|
||||
if f.Tag.Get("json") != "-" {
|
||||
if err := rejectEnvironmentTags(f.Type, seen); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
var textType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
|
||||
var durationType = reflect.TypeOf(time.Duration(0))
|
||||
|
||||
func projectOverlay[T any](base T, options EnvOptions, defaults bool) (T, error) {
|
||||
var zero T
|
||||
if options.Lookup == nil {
|
||||
options.Lookup = os.LookupEnv
|
||||
}
|
||||
if options.Empty == "" {
|
||||
options.Empty = "allow"
|
||||
}
|
||||
if options.Empty != "allow" && options.Empty != "ignore" && options.Empty != "reject" {
|
||||
return zero, errors.New("config: empty policy must be allow, ignore or reject")
|
||||
}
|
||||
v := reflect.ValueOf(&base).Elem()
|
||||
if v.Kind() != reflect.Struct {
|
||||
return zero, errors.New("config: project configuration must be a struct")
|
||||
}
|
||||
result := reflect.New(v.Type()).Elem()
|
||||
result.Set(v)
|
||||
seen := map[string]bool{}
|
||||
var walk func(reflect.Value, string, int) error
|
||||
walk = func(value reflect.Value, prefix string, depth int) error {
|
||||
if depth > 32 {
|
||||
return errors.New("config: nesting exceeds 32")
|
||||
}
|
||||
for i := 0; i < value.NumField(); i++ {
|
||||
field := value.Type().Field(i)
|
||||
dst := value.Field(i)
|
||||
if !dst.CanSet() || field.PkgPath != "" {
|
||||
continue
|
||||
}
|
||||
tag := strings.Split(field.Tag.Get("env"), ",")
|
||||
if tag[0] == "-" {
|
||||
continue
|
||||
}
|
||||
jsonName := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||
if jsonName == "-" && tag[0] == "" {
|
||||
continue
|
||||
}
|
||||
if jsonName == "" {
|
||||
jsonName = field.Name
|
||||
}
|
||||
key := prefix + envName(jsonName)
|
||||
if tag[0] != "" {
|
||||
key = tag[0]
|
||||
}
|
||||
empty, required := options.Empty, false
|
||||
for _, opt := range tag[1:] {
|
||||
switch opt {
|
||||
case "required":
|
||||
required = true
|
||||
case "secret", "":
|
||||
case "empty=allow":
|
||||
empty = "allow"
|
||||
case "empty=ignore":
|
||||
empty = "ignore"
|
||||
case "empty=reject":
|
||||
empty = "reject"
|
||||
default:
|
||||
return fmt.Errorf("config: invalid env tag for %s", key)
|
||||
}
|
||||
}
|
||||
if dst.Kind() == reflect.Struct && !dst.Addr().Type().Implements(textType) {
|
||||
if err := walk(dst, key+"_", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if dst.Kind() == reflect.Pointer && dst.Type().Elem().Kind() == reflect.Struct && !dst.Type().Implements(textType) {
|
||||
// Clone before traversing so a failed overlay never edits defaults.
|
||||
child := reflect.New(dst.Type().Elem())
|
||||
if !dst.IsNil() {
|
||||
child.Elem().Set(dst.Elem())
|
||||
}
|
||||
if err := walk(child.Elem(), key+"_", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
if !dst.IsNil() || !child.Elem().IsZero() {
|
||||
dst.Set(child)
|
||||
}
|
||||
if required && !defaults && dst.IsNil() {
|
||||
return fmt.Errorf("config: %s is required", key)
|
||||
}
|
||||
continue
|
||||
}
|
||||
// Unexported runtime capabilities are skipped unless explicitly tagged.
|
||||
if dst.Kind() == reflect.Func || dst.Kind() == reflect.Chan || dst.Kind() == reflect.Interface {
|
||||
if tag[0] != "" {
|
||||
return fmt.Errorf("config: %s has unsupported environment type", key)
|
||||
}
|
||||
continue
|
||||
}
|
||||
if seen[key] {
|
||||
return fmt.Errorf("config: duplicate environment key %s", key)
|
||||
}
|
||||
seen[key] = true
|
||||
raw, present := options.Lookup(key)
|
||||
selectedKey := key
|
||||
if !present && !defaults {
|
||||
for _, alias := range options.Aliases[key] {
|
||||
if raw, present = options.Lookup(alias); present {
|
||||
selectedKey = alias
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if defaults {
|
||||
raw, present = field.Tag.Lookup("envDefault")
|
||||
present = present && dst.IsZero()
|
||||
}
|
||||
if present && raw == "" && !defaults {
|
||||
if empty == "reject" {
|
||||
return fmt.Errorf("config: %s must not be empty", selectedKey)
|
||||
}
|
||||
if empty == "ignore" {
|
||||
present = false
|
||||
}
|
||||
}
|
||||
if present {
|
||||
parsed := reflect.New(dst.Type()).Elem()
|
||||
if err := parseEnvValue(parsed, raw); err != nil {
|
||||
return fmt.Errorf("config: %s must be a valid %s", selectedKey, dst.Type())
|
||||
}
|
||||
dst.Set(parsed)
|
||||
}
|
||||
if required && !defaults && emptyRequired(dst) {
|
||||
return fmt.Errorf("config: %s is required", key)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := walk(result, options.Prefix, 0); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result.Interface().(T), nil
|
||||
}
|
||||
|
||||
func emptyRequired(v reflect.Value) bool {
|
||||
switch v.Kind() {
|
||||
case reflect.String:
|
||||
return v.Len() == 0
|
||||
case reflect.Pointer, reflect.Map, reflect.Slice:
|
||||
return v.IsNil()
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func parseEnvValue(v reflect.Value, raw string) (err error) {
|
||||
func parseTextValue(v reflect.Value, raw string) (err error) {
|
||||
defer func() {
|
||||
if recover() != nil {
|
||||
err = errors.New("invalid custom value")
|
||||
@@ -223,7 +107,7 @@ func parseEnvValue(v reflect.Value, raw string) (err error) {
|
||||
}()
|
||||
if v.Kind() == reflect.Pointer {
|
||||
v.Set(reflect.New(v.Type().Elem()))
|
||||
return parseEnvValue(v.Elem(), raw)
|
||||
return parseTextValue(v.Elem(), raw)
|
||||
}
|
||||
if v.CanAddr() && v.Addr().Type().Implements(textType) {
|
||||
return v.Addr().Interface().(encoding.TextUnmarshaler).UnmarshalText([]byte(raw))
|
||||
@@ -266,37 +150,6 @@ func parseEnvValue(v reflect.Value, raw string) (err error) {
|
||||
v.SetFloat(x)
|
||||
}
|
||||
return err
|
||||
case reflect.Slice, reflect.Map:
|
||||
var value any
|
||||
if err := apierrors.DecodeCatalogJSON([]byte(raw), &value); err != nil {
|
||||
return err
|
||||
}
|
||||
value, err := durations(value, v.Type())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(encoded))
|
||||
decoder.DisallowUnknownFields()
|
||||
return decoder.Decode(v.Addr().Interface())
|
||||
}
|
||||
return errors.New("unsupported environment value")
|
||||
}
|
||||
|
||||
func envName(s string) string {
|
||||
r := []rune(s)
|
||||
var out strings.Builder
|
||||
for i, c := range r {
|
||||
if i > 0 && unicode.IsUpper(c) && (unicode.IsLower(r[i-1]) || unicode.IsDigit(r[i-1]) || (i+1 < len(r) && unicode.IsLower(r[i+1]))) {
|
||||
out.WriteByte('_')
|
||||
}
|
||||
if c == '-' {
|
||||
c = '_'
|
||||
}
|
||||
out.WriteRune(unicode.ToUpper(c))
|
||||
}
|
||||
return out.String()
|
||||
return errors.New("expected scalar text")
|
||||
}
|
||||
|
||||
@@ -42,20 +42,3 @@ func TestConfigFilePreservesDefaultsWithoutSharingMutableData(t *testing.T) {
|
||||
t.Fatal("decode error leaked contents", err)
|
||||
}
|
||||
}
|
||||
func TestEnvCollectionPrecisionAndDurationMaps(t *testing.T) {
|
||||
type cfg struct {
|
||||
IDs []int64 `json:"ids"`
|
||||
Timers map[string]time.Duration `json:"timers"`
|
||||
}
|
||||
lookup := func(key string) (string, bool) {
|
||||
v, ok := map[string]string{"IDS": "[9007199254740993]", "TIMERS": `{"backoff":"250ms"}`}[key]
|
||||
return v, ok
|
||||
}
|
||||
c, err := OverlayEnv(cfg{}, EnvOptions{Lookup: lookup})
|
||||
if err != nil || c.IDs[0] != 9007199254740993 || c.Timers["backoff"] != 250*time.Millisecond {
|
||||
t.Fatal(c, err)
|
||||
}
|
||||
if _, err = OverlayEnv(cfg{}, EnvOptions{Lookup: func(key string) (string, bool) { return `{"x":"1s","x":"2s"}`, key == "TIMERS" }}); err == nil {
|
||||
t.Fatal("duplicate map key accepted")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,124 +0,0 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type projectNested struct {
|
||||
ReadTimeout time.Duration `json:"readTimeout"`
|
||||
Names []string `json:"names"`
|
||||
}
|
||||
type projectConfig struct {
|
||||
Nested *projectNested `json:"nested"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Count int `json:"count" envDefault:"12"`
|
||||
Password string `json:"-" env:"ETAX_PASSWORD,secret,required,empty=reject"`
|
||||
Label string `json:"label" env:",empty=ignore"`
|
||||
}
|
||||
|
||||
func (c projectConfig) Validate() error {
|
||||
if c.Count < 0 {
|
||||
return errors.New("count must be nonnegative")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func TestProjectConfigurationOrderAndValues(t *testing.T) {
|
||||
f := filepath.Join(t.TempDir(), "config.yaml")
|
||||
if err := os.WriteFile(f, []byte("count: 3\nenabled: true\nnested:\n readTimeout: 2s\n names: [file]\nlabel: keep\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
env := map[string]string{"ETAX_PASSWORD": " password ", "APP_ENABLED": "false", "APP_COUNT": "0", "APP_NESTED_READ_TIMEOUT": "7s", "APP_NESTED_NAMES": "[]", "APP_LABEL": ""}
|
||||
options := EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
|
||||
cfg, err := LoadProject(f, projectConfig{}, options)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Enabled || cfg.Count != 0 || cfg.Password != " password " || cfg.Nested.ReadTimeout != 7*time.Second || cfg.Nested.Names == nil || len(cfg.Nested.Names) != 0 || cfg.Label != "keep" {
|
||||
t.Fatal("overlay mismatch")
|
||||
}
|
||||
delete(env, "APP_COUNT")
|
||||
cfg, err = LoadProject("", projectConfig{}, options)
|
||||
if err != nil || cfg.Count != 12 {
|
||||
t.Fatal(cfg.Count, err)
|
||||
}
|
||||
env["APP_COUNT"] = "-1"
|
||||
if _, err = LoadProject(f, projectConfig{}, options); err == nil {
|
||||
t.Fatal("project Validate not called")
|
||||
}
|
||||
}
|
||||
|
||||
func TestProjectOverlayIsTransactionalAndRedacted(t *testing.T) {
|
||||
base := projectConfig{Nested: &projectNested{ReadTimeout: time.Second, Names: []string{"old"}}, Password: "original"}
|
||||
env := map[string]string{"APP_NESTED_READ_TIMEOUT": "2s", "APP_NESTED_NAMES": "[\"new\"]", "APP_COUNT": "secret-value"}
|
||||
_, err := OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }})
|
||||
if err == nil || strings.Contains(err.Error(), "secret-value") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if base.Nested.ReadTimeout != time.Second || !reflect.DeepEqual(base.Nested.Names, []string{"old"}) {
|
||||
t.Fatal("default pointer modified")
|
||||
}
|
||||
delete(env, "APP_COUNT")
|
||||
env["ETAX_PASSWORD"] = ""
|
||||
if _, err = OverlayEnv(base, EnvOptions{Prefix: "APP_", Lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}); err == nil {
|
||||
t.Fatal("empty secret accepted")
|
||||
}
|
||||
type required struct {
|
||||
Token string `env:"TOKEN,required"`
|
||||
}
|
||||
if _, err = OverlayEnv(required{}, EnvOptions{Lookup: func(string) (string, bool) { return "", false }}); err == nil {
|
||||
t.Fatal("required absent")
|
||||
}
|
||||
type duplicate struct {
|
||||
A string `env:"SAME"`
|
||||
B string `env:"SAME"`
|
||||
}
|
||||
if _, err = OverlayEnv(duplicate{}, EnvOptions{}); err == nil {
|
||||
t.Fatal("duplicate env binding accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestEnvironmentCollectionAndEmptySemantics(t *testing.T) {
|
||||
type sample struct {
|
||||
Values []time.Duration
|
||||
Flag bool
|
||||
Text string
|
||||
Optional *int
|
||||
}
|
||||
cases := []struct {
|
||||
raw string
|
||||
valid bool
|
||||
}{{`["1s","2s"]`, true}, {`["invalid-secret"]`, false}}
|
||||
for _, tc := range cases {
|
||||
cfg, err := OverlayEnv(sample{}, EnvOptions{Lookup: func(k string) (string, bool) {
|
||||
if k == "VALUES" {
|
||||
return tc.raw, true
|
||||
}
|
||||
return "", false
|
||||
}})
|
||||
if (err == nil) != tc.valid {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if tc.valid && !reflect.DeepEqual(cfg.Values, []time.Duration{time.Second, 2 * time.Second}) {
|
||||
t.Fatal(cfg)
|
||||
}
|
||||
if err != nil && strings.Contains(err.Error(), "secret") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, mode := range []string{"allow", "ignore", "reject"} {
|
||||
cfg, err := OverlayEnv(sample{Text: "old"}, EnvOptions{Empty: mode, Lookup: func(k string) (string, bool) { return "", k == "TEXT" }})
|
||||
if mode == "reject" {
|
||||
if err == nil {
|
||||
t.Fatal("empty not rejected")
|
||||
}
|
||||
} else if err != nil || (mode == "allow" && cfg.Text != "") || (mode == "ignore" && cfg.Text != "old") {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
210
config/references.go
Normal file
210
config/references.go
Normal file
@@ -0,0 +1,210 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
)
|
||||
|
||||
// WithLookupEnv supplies an explicit-reference lookup for tests or an embedding
|
||||
// host. Ordinary projects need no loader options. The function must support
|
||||
// concurrent loads; each load caches each referenced name independently.
|
||||
func WithLookupEnv(lookup func(string) (string, bool)) LoaderOption {
|
||||
return func(l *Loader) error {
|
||||
if l == nil || l.sealed || lookup == nil {
|
||||
return errors.New("config: expected a non-nil environment lookup on a new loader")
|
||||
}
|
||||
l.lookup = lookup
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
// resolvedText is textual input bound according to the target scalar type. It
|
||||
// never becomes an object, collection or null by interpreting environment data.
|
||||
type resolvedText string
|
||||
|
||||
const maxExpandedBytes = 4 << 20
|
||||
|
||||
type environmentValue struct {
|
||||
value string
|
||||
present bool
|
||||
}
|
||||
type referenceResolver struct {
|
||||
lookup func(string) (string, bool)
|
||||
cache map[string]environmentValue
|
||||
doc Document
|
||||
nodes, bytes, cacheBytes int
|
||||
}
|
||||
|
||||
func resolveDocument(doc Document, lookup func(string) (string, bool)) (Document, error) {
|
||||
r := referenceResolver{lookup: lookup, cache: map[string]environmentValue{}, doc: Document{Positions: doc.Positions, environments: map[string][]string{}}}
|
||||
v, err := r.walk(doc.Value, "", 0)
|
||||
if err != nil {
|
||||
return Document{}, err
|
||||
}
|
||||
r.doc.Value = v
|
||||
return r.doc, nil
|
||||
}
|
||||
func (r *referenceResolver) walk(v any, path string, depth int) (any, error) {
|
||||
r.nodes++
|
||||
if depth > maxConfigDepth || r.nodes > maxConfigNodes {
|
||||
return nil, r.doc.diagnostic(path, "configuration nesting or node limit exceeded", nil)
|
||||
}
|
||||
switch x := v.(type) {
|
||||
case string:
|
||||
return r.text(x, path, false)
|
||||
case Text:
|
||||
return r.text(string(x), path, true)
|
||||
case map[string]any:
|
||||
out := make(map[string]any, len(x))
|
||||
for _, key := range sortedKeys(x) {
|
||||
value, err := r.walk(x[key], joinField(path, key), depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out[key] = value
|
||||
}
|
||||
return out, nil
|
||||
case []any:
|
||||
out := make([]any, len(x))
|
||||
for i, value := range x {
|
||||
var err error
|
||||
out[i], err = r.walk(value, fmt.Sprintf("%s[%d]", path, i), depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
default:
|
||||
return v, nil
|
||||
}
|
||||
}
|
||||
func (r *referenceResolver) fail(path, name, reason string) error {
|
||||
d := r.doc.diagnostic(path, reason, nil)
|
||||
d.Env = name
|
||||
return d
|
||||
}
|
||||
func validEnvironmentName(name string) bool {
|
||||
if len(name) == 0 {
|
||||
return false
|
||||
}
|
||||
for i := 0; i < len(name); i++ {
|
||||
c := name[i]
|
||||
if c == '_' || c >= 'A' && c <= 'Z' || c >= 'a' && c <= 'z' || i > 0 && c >= '0' && c <= '9' {
|
||||
continue
|
||||
}
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
func (r *referenceResolver) lookupValue(name, path string) (environmentValue, error) {
|
||||
if v, ok := r.cache[name]; ok {
|
||||
return v, nil
|
||||
}
|
||||
value, present := r.lookup(name)
|
||||
if len(value) > maxConfigBytes || r.cacheBytes+len(value) > maxExpandedBytes {
|
||||
return environmentValue{}, r.fail(path, name, "environment value exceeds configuration size limit")
|
||||
}
|
||||
if !utf8.ValidString(value) {
|
||||
return environmentValue{}, r.fail(path, name, "environment value is not valid UTF-8")
|
||||
}
|
||||
|
||||
r.cacheBytes += len(value)
|
||||
v := environmentValue{value, present}
|
||||
r.cache[name] = v
|
||||
return v, nil
|
||||
}
|
||||
func (r *referenceResolver) text(raw, path string, xmlText bool) (any, error) {
|
||||
var out strings.Builder
|
||||
var names []string
|
||||
seen := map[string]bool{}
|
||||
if !strings.Contains(raw, "$") {
|
||||
if len(raw) > maxConfigBytes || r.bytes+len(raw) > maxExpandedBytes {
|
||||
return nil, r.fail(path, "", "expanded configuration exceeds size limit")
|
||||
}
|
||||
r.bytes += len(raw)
|
||||
if xmlText {
|
||||
return Text(raw), nil
|
||||
}
|
||||
return raw, nil
|
||||
}
|
||||
appendText := func(s string) error {
|
||||
if out.Len()+len(s) > maxConfigBytes || r.bytes+len(s) > maxExpandedBytes {
|
||||
return r.fail(path, strings.Join(names, ", "), "expanded configuration exceeds size limit")
|
||||
}
|
||||
out.WriteString(s)
|
||||
r.bytes += len(s)
|
||||
return nil
|
||||
}
|
||||
// Scan only the file's decoded scalar. Inserted environment values are never
|
||||
// scanned again, including dollars, quotes, separators or newline characters.
|
||||
for i := 0; i < len(raw); {
|
||||
j := strings.IndexByte(raw[i:], '$')
|
||||
if j < 0 {
|
||||
if err := appendText(raw[i:]); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
break
|
||||
}
|
||||
j += i
|
||||
if err := appendText(raw[i:j]); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch {
|
||||
case j+1 < len(raw) && raw[j+1] == '$':
|
||||
if err := appendText("$"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
i = j + 2
|
||||
case j+1 < len(raw) && raw[j+1] == '{':
|
||||
end := strings.IndexByte(raw[j+2:], '}')
|
||||
if end < 0 {
|
||||
return nil, r.fail(path, "", "unclosed environment reference; use ${NAME}, ${NAME:-fallback} or $${...} for a literal")
|
||||
}
|
||||
end += j + 2
|
||||
expression := raw[j+2 : end]
|
||||
name, fallback, hasDefault := strings.Cut(expression, ":-")
|
||||
if !validEnvironmentName(name) || strings.Contains(fallback, "${") {
|
||||
return nil, r.fail(path, "", "invalid environment reference; use ${NAME} or ${NAME:-fallback}; nested references are not supported")
|
||||
}
|
||||
r.nodes++
|
||||
if r.nodes > maxConfigNodes {
|
||||
return nil, r.fail(path, name, "environment reference count exceeds configuration limit")
|
||||
}
|
||||
if !seen[name] {
|
||||
names = append(names, name)
|
||||
seen[name] = true
|
||||
}
|
||||
|
||||
value, err := r.lookupValue(name, path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
selected := value.value
|
||||
if !value.present || selected == "" {
|
||||
if !hasDefault {
|
||||
return nil, r.fail(path, name, "required environment variable is unset or empty")
|
||||
}
|
||||
selected = fallback
|
||||
}
|
||||
if err := appendText(selected); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
i = end + 1
|
||||
default:
|
||||
if err := appendText("$"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
i = j + 1
|
||||
}
|
||||
}
|
||||
if len(names) > 0 {
|
||||
r.doc.environments[path] = names
|
||||
return resolvedText(out.String()), nil
|
||||
}
|
||||
if xmlText {
|
||||
return Text(out.String()), nil
|
||||
}
|
||||
return out.String(), nil
|
||||
}
|
||||
230
config/references_test.go
Normal file
230
config/references_test.go
Normal file
@@ -0,0 +1,230 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type referenceConfig struct {
|
||||
Port int `json:"port"`
|
||||
BatchSize int `json:"batchSize"`
|
||||
Enabled bool `json:"enabled"`
|
||||
Password string `json:"password"`
|
||||
Address string `json:"address"`
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
Optional *int `json:"optional"`
|
||||
Labels map[string]string `json:"labels"`
|
||||
Numbers []int64 `json:"numbers"`
|
||||
Dynamic any `json:"dynamic"`
|
||||
}
|
||||
|
||||
func (c referenceConfig) Validate() error {
|
||||
if c.Port < 0 {
|
||||
return Invalid("port", "must not be negative")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func TestExplicitReferencesAcrossFormats(t *testing.T) {
|
||||
// Delimiters, quotes and newlines belong to the credential, never file syntax.
|
||||
secret := " DDSFLKJlsf:saf:-123\"\n${DO_NOT_READ}&<>\\ "
|
||||
sources := map[string]string{
|
||||
"json": `{"port":"${PORT:-3306}","enabled":"${ENABLED:-true}","password":"${PASSWORD}","address":"https://${HOST}:${PORT:-3306}/v1","timeout":"${TIMEOUT:-3s}","optional":"${ZERO}","numbers":["${BIG}"],"dynamic":"${ZERO}","labels":{"literal":"$${NOT_READ}","empty":"${OPTIONAL:-}","dollars":"$$cost"}}`,
|
||||
"yaml": "port: \"${PORT:-3306}\"\nenabled: \"${ENABLED:-true}\"\npassword: \"${PASSWORD}\"\naddress: \"https://${HOST}:${PORT:-3306}/v1\"\ntimeout: \"${TIMEOUT:-3s}\"\noptional: \"${ZERO}\"\nnumbers: [\"${BIG}\"]\ndynamic: \"${ZERO}\"\nlabels:\n literal: \"$${NOT_READ}\"\n empty: \"${OPTIONAL:-}\"\n dollars: \"$$cost\"\n",
|
||||
"xml": `<config><port>${PORT:-3306}</port><enabled>${ENABLED:-true}</enabled><password>${PASSWORD}</password><address>https://${HOST}:${PORT:-3306}/v1</address><timeout>${TIMEOUT:-3s}</timeout><optional>${ZERO}</optional><numbers type="array"><item>${BIG}</item></numbers><dynamic>${ZERO}</dynamic><labels><literal>$${NOT_READ}</literal><empty>${OPTIONAL:-}</empty><dollars>$$cost</dollars></labels></config>`,
|
||||
}
|
||||
for ext, source := range sources {
|
||||
t.Run(ext, func(t *testing.T) {
|
||||
calls := map[string]int{}
|
||||
values := map[string]string{"PORT": "0", "ENABLED": "false", "PASSWORD": secret, "HOST": "example.test", "ZERO": "0", "BIG": "9007199254740993"}
|
||||
l := testLoader(func(name string) (string, bool) { calls[name]++; v, ok := values[name]; return v, ok })
|
||||
got, err := LoadProject(configFile(t, ext, source), referenceConfig{Port: 77, BatchSize: 100, Enabled: true, Password: "old"}, l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Port != 0 || got.BatchSize != 100 || got.Enabled || got.Password != secret || got.Address != "https://example.test:0/v1" || got.Timeout != 3*time.Second || got.Optional == nil || *got.Optional != 0 || got.Numbers[0] != 9007199254740993 || got.Dynamic != "0" {
|
||||
t.Fatal("binding changed values")
|
||||
}
|
||||
if got.Labels["literal"] != "${NOT_READ}" || got.Labels["empty"] != "" || got.Labels["dollars"] != "$cost" {
|
||||
t.Fatal(got.Labels)
|
||||
}
|
||||
for name, n := range calls {
|
||||
if n != 1 {
|
||||
t.Fatal("not cached", name, n)
|
||||
}
|
||||
}
|
||||
if calls["DO_NOT_READ"] != 0 || calls["NOT_READ"] != 0 {
|
||||
t.Fatal("recursive expansion")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestReferenceFallbackAndFixedValues(t *testing.T) {
|
||||
for _, present := range []bool{false, true} {
|
||||
l := testLoader(func(string) (string, bool) { return "", present })
|
||||
got, err := LoadProject(configFile(t, "yaml", "port: \"${PORT:-3306}\"\npassword: \"${PASSWORD:-DDSFLKJlsf:saf:-123}\""), referenceConfig{}, l)
|
||||
if err != nil || got.Port != 3306 || got.Password != "DDSFLKJlsf:saf:-123" {
|
||||
t.Fatal(got, err)
|
||||
}
|
||||
for _, ext := range []string{"json", "yaml", "xml"} {
|
||||
raw := `{"password":"${PASSWORD}"}`
|
||||
if ext == "yaml" {
|
||||
raw = "password: '${PASSWORD}'"
|
||||
}
|
||||
if ext == "xml" {
|
||||
raw = "<config><password>${PASSWORD}</password></config>"
|
||||
}
|
||||
path := configFile(t, ext, raw)
|
||||
_, err = LoadProject(path, referenceConfig{Password: "initial"}, l)
|
||||
var d *Error
|
||||
if !errors.As(err, &d) || d.Env != "PASSWORD" || d.File != path || d.Field != "password" || d.Line < 1 {
|
||||
t.Fatal("missing required-reference diagnostic", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
calls := 0
|
||||
l := testLoader(func(string) (string, bool) { calls++; return "999", true })
|
||||
got, err := LoadProject(configFile(t, "json", `{"port":0,"enabled":false,"password":""}`), referenceConfig{Port: 77, BatchSize: 100, Enabled: true, Password: "old"}, l)
|
||||
if err != nil || got.Port != 0 || got.BatchSize != 100 || got.Enabled || got.Password != "" || calls != 0 {
|
||||
t.Fatal(got, err, calls)
|
||||
}
|
||||
got, err = LoadProject(configFile(t, "yaml", "password: '${VALUE}'"), referenceConfig{}, testLoader(func(string) (string, bool) { return " ", true }))
|
||||
if err != nil || got.Password != " " {
|
||||
t.Fatal("trimmed credential", err)
|
||||
}
|
||||
}
|
||||
func TestReferenceDiagnosticsAndScalarLimits(t *testing.T) {
|
||||
for _, value := range []string{"private-not-int", "92233720368547758080", "false", "null", "-1"} {
|
||||
path := configFile(t, "yaml", "port: '${PORT}'")
|
||||
_, err := LoadProject(path, referenceConfig{}, testLoader(func(string) (string, bool) { return value, true }))
|
||||
var d *Error
|
||||
if !errors.As(err, &d) || d.File != path || d.Env != "PORT" || d.Field != "port" || d.Line != 1 || d.Column < 1 || strings.Contains(err.Error(), value) {
|
||||
t.Fatalf("invalid diagnostic: %v", err)
|
||||
}
|
||||
}
|
||||
for _, raw := range []string{"${BAD-123}", "${}", "${X", "${1BAD}", "${BAD NAME}", "${X:-${Y}}"} {
|
||||
encoded, _ := json.Marshal(map[string]string{"password": raw})
|
||||
if _, err := LoadProject(configFile(t, "json", string(encoded)), referenceConfig{}, testLoader(noEnvironment)); err == nil {
|
||||
t.Fatal("invalid syntax accepted", raw)
|
||||
}
|
||||
}
|
||||
for _, value := range []string{string([]byte{0xff}), strings.Repeat("x", maxConfigBytes+1)} {
|
||||
if _, err := LoadProject(configFile(t, "yaml", "password: '${SECRET}'"), referenceConfig{}, testLoader(func(string) (string, bool) { return value, true })); err == nil {
|
||||
t.Fatal("unbounded/invalid environment")
|
||||
}
|
||||
}
|
||||
path := configFile(t, "yaml", "labels:\n a: '${BIG}'\n b: '${BIG}'\n c: '${BIG}'\n d: '${BIG}'\n e: '${BIG}'")
|
||||
if _, err := LoadProject(path, referenceConfig{}, testLoader(func(string) (string, bool) { return strings.Repeat("x", maxConfigBytes), true })); err == nil {
|
||||
t.Fatal("expansion budget bypassed")
|
||||
}
|
||||
// An environment value cannot inject an array/object or coerce dynamic null.
|
||||
for _, raw := range []string{"[1,2]", "{\"x\":1}", "null"} {
|
||||
_, err := LoadProject(configFile(t, "yaml", "numbers: '${VALUE}'"), referenceConfig{}, testLoader(func(string) (string, bool) { return raw, true }))
|
||||
if err == nil {
|
||||
t.Fatal("structure injected")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestReferenceCacheIsolationAndRemovedRules(t *testing.T) {
|
||||
path := configFile(t, "yaml", "password: '${TOKEN}'")
|
||||
for i := 0; i < 24; i++ {
|
||||
t.Run(fmt.Sprint(i), func(t *testing.T) {
|
||||
t.Parallel()
|
||||
v := fmt.Sprint(i)
|
||||
l := testLoader(func(string) (string, bool) { return v, true })
|
||||
for n := 0; n < 4; n++ {
|
||||
got, err := LoadProject(path, referenceConfig{}, l)
|
||||
if err != nil || got.Password != v {
|
||||
t.Fatal("load leaked", err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestReferenceLookupPerLoadAndConcurrentLoader(t *testing.T) {
|
||||
var mu sync.Mutex
|
||||
reads := 0
|
||||
l := testLoader(func(string) (string, bool) { mu.Lock(); defer mu.Unlock(); reads++; return "secret", true })
|
||||
path := configFile(t, "yaml", "password: '${TOKEN}${TOKEN}'")
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 24; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
got, err := LoadProject(path, referenceConfig{}, l)
|
||||
if err != nil || got.Password != "secretsecret" {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
if reads != 24 {
|
||||
t.Fatal(reads)
|
||||
}
|
||||
}
|
||||
func TestRemovedEnvTagsAndFieldSuggestion(t *testing.T) {
|
||||
type old struct {
|
||||
Value int `json:"value" envDefault:"9"`
|
||||
}
|
||||
if _, err := LoadProject("", old{}); err == nil || !strings.Contains(err.Error(), "removed") {
|
||||
t.Fatal(err)
|
||||
}
|
||||
type oldSecret struct {
|
||||
Token string `json:"-" env:"TOKEN"`
|
||||
}
|
||||
if _, err := DecodeFile(configFile(t, "json", `{}`), oldSecret{}); err == nil {
|
||||
t.Fatal("old env tag ignored")
|
||||
}
|
||||
_, err := LoadProject(configFile(t, "yaml", "batchSiz: 10"), referenceConfig{})
|
||||
if err == nil || !strings.Contains(err.Error(), `did you mean "batchSize"`) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_BATCH_SIZE", "999")
|
||||
got, err := LoadProject("", referenceConfig{BatchSize: 100})
|
||||
if err != nil || got.BatchSize != 100 {
|
||||
t.Fatal("implicit overlay", err)
|
||||
}
|
||||
}
|
||||
func FuzzConfigurationReferences(f *testing.F) {
|
||||
for _, s := range []string{"${X}", "$${X}", "${X:-DDS:abc:-123}", "${X:-}", "https://${X}/${X}", "${X:-${Y}}"} {
|
||||
f.Add(s, " secret${NO}\n\" ")
|
||||
}
|
||||
f.Fuzz(func(t *testing.T, raw, value string) {
|
||||
if len(raw) > 8192 || len(value) > 8192 {
|
||||
return
|
||||
}
|
||||
doc := Document{Value: map[string]any{"password": raw}}
|
||||
resolved, err := resolveDocument(doc, func(string) (string, bool) { return value, true })
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
_, _ = bindDocument(resolved, referenceConfig{})
|
||||
})
|
||||
}
|
||||
func BenchmarkExplicitConfiguration(b *testing.B) {
|
||||
for _, refs := range []bool{false, true} {
|
||||
b.Run(fmt.Sprint(refs), func(b *testing.B) {
|
||||
source := `{"port":3306,"enabled":false,"password":"fixed","batchSize":0}`
|
||||
if refs {
|
||||
source = `{"port":"${PORT:-3306}","enabled":"${ENABLED:-false}","password":"${PASSWORD}","batchSize":0}`
|
||||
}
|
||||
path := b.TempDir() + "/config.json"
|
||||
if err := os.WriteFile(path, []byte(source), 0600); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
l := testLoader(func(k string) (string, bool) { return "private", k == "PASSWORD" })
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
if _, err := LoadProject(path, referenceConfig{BatchSize: 100}, l); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
51
config/suggestions.go
Normal file
51
config/suggestions.go
Normal file
@@ -0,0 +1,51 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Suggest only a unique, nearby declared field. Input values are never used.
|
||||
func unknownFieldReason(name string, fields map[string]reflect.StructField) string {
|
||||
if len(name) < 3 || len(name) > 128 {
|
||||
return "unknown field"
|
||||
}
|
||||
best, match, tied := 3, "", false
|
||||
for candidate := range fields {
|
||||
if len(candidate) > 128 {
|
||||
continue
|
||||
}
|
||||
d := fieldDistance(strings.ToLower(name), strings.ToLower(candidate))
|
||||
if d < best {
|
||||
best, match, tied = d, candidate, false
|
||||
} else if d == best {
|
||||
tied = true
|
||||
}
|
||||
}
|
||||
if match != "" && !tied && best <= 2 {
|
||||
return "unknown field; did you mean " + strconv.Quote(match) + "?"
|
||||
}
|
||||
return "unknown field"
|
||||
}
|
||||
func fieldDistance(a, b string) int {
|
||||
prev := make([]int, len(b)+1)
|
||||
for i := range prev {
|
||||
prev[i] = i
|
||||
}
|
||||
for i := 0; i < len(a); i++ {
|
||||
left, diagonal := i+1, prev[0]
|
||||
prev[0] = left
|
||||
for j := 0; j < len(b); j++ {
|
||||
cost := 0
|
||||
if a[i] != b[j] {
|
||||
cost = 1
|
||||
}
|
||||
old := prev[j+1]
|
||||
left = min(left+1, old+1, diagonal+cost)
|
||||
prev[j+1] = left
|
||||
diagonal = old
|
||||
}
|
||||
}
|
||||
return prev[len(b)]
|
||||
}
|
||||
@@ -380,7 +380,10 @@ func decodeValue(dec *json.Decoder, depth int) (any, error) {
|
||||
object[name] = value
|
||||
}
|
||||
end, err := dec.Token()
|
||||
if err != nil || end != json.Delim('}') {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if end != json.Delim('}') {
|
||||
return nil, errors.New("unterminated object")
|
||||
}
|
||||
return object, nil
|
||||
@@ -394,7 +397,10 @@ func decodeValue(dec *json.Decoder, depth int) (any, error) {
|
||||
items = append(items, v)
|
||||
}
|
||||
end, err := dec.Token()
|
||||
if err != nil || end != json.Delim(']') {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if end != json.Delim(']') {
|
||||
return nil, errors.New("unterminated array")
|
||||
}
|
||||
return items, nil
|
||||
|
||||
@@ -3,9 +3,11 @@ package contract
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/configdiag"
|
||||
)
|
||||
|
||||
// RequestPolicy describes protocol strictness, independently of business
|
||||
@@ -35,7 +37,7 @@ func (p RequestPolicy) Validate() error {
|
||||
ok = ok || value == item.value
|
||||
}
|
||||
if !ok {
|
||||
return fmt.Errorf("request policy: invalid %s", item.name)
|
||||
return configdiag.New(item.name, "must be one of "+strings.Join(item.allowed, ", "))
|
||||
}
|
||||
}
|
||||
return nil
|
||||
|
||||
@@ -3,6 +3,7 @@ package contract
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"io"
|
||||
@@ -10,6 +11,7 @@ import (
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
"testing/iotest"
|
||||
)
|
||||
|
||||
type byteReader struct{ r io.Reader }
|
||||
@@ -34,6 +36,17 @@ func TestBodyPolicyPropagatesOuterLimit(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestJSONClosingDelimiterPreservesReaderCause(t *testing.T) {
|
||||
cause := errors.New("body reader failed")
|
||||
for _, prefix := range []string{"{", "[", `{"value":[]`, `[{}`} {
|
||||
reader := io.MultiReader(byteReader{strings.NewReader(prefix)}, iotest.ErrReader(cause))
|
||||
_, err := decodeValue(json.NewDecoder(reader), 0)
|
||||
if !errors.Is(err, cause) {
|
||||
t.Fatalf("prefix %q: lost reader cause: %v", prefix, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestStrictRequestPolicy(t *testing.T) {
|
||||
type request struct {
|
||||
Name string `json:"name"`
|
||||
|
||||
117
copyx/benchmark_test.go
Normal file
117
copyx/benchmark_test.go
Normal file
@@ -0,0 +1,117 @@
|
||||
package copyx_test
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/copyx"
|
||||
)
|
||||
|
||||
func BenchmarkScenarios(b *testing.B) {
|
||||
type request struct {
|
||||
Name string
|
||||
Count int64
|
||||
Active bool
|
||||
}
|
||||
type result struct {
|
||||
Label string
|
||||
Count int64
|
||||
Active bool
|
||||
}
|
||||
source := request{"sample", 9007199254740993, true}
|
||||
option := copyx.WithFieldNames[request, result](map[string]string{"Name": "Label"})
|
||||
copier, err := copyx.New(option)
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
for _, reuse := range []bool{false, true} {
|
||||
b.Run(fmt.Sprintf("struct/reuse%t", reuse), func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var target result
|
||||
var err error
|
||||
if reuse {
|
||||
err = copier.Copy(&target, source)
|
||||
} else {
|
||||
err = copyx.Copy(&target, source, option)
|
||||
}
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
deep, _ := copyx.New(copyx.WithDeepCopy())
|
||||
b.Run("bytes64KiB", func(b *testing.B) {
|
||||
src := make([]byte, 64<<10)
|
||||
b.SetBytes(int64(len(src)))
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst []byte
|
||||
if err := deep.Copy(&dst, src); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
src := map[string][]int{}
|
||||
for i := 0; i < 100; i++ {
|
||||
src[fmt.Sprint(i)] = []int{1, 2, 3}
|
||||
}
|
||||
b.Run("map100", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst map[string][]int
|
||||
if err := deep.Copy(&dst, src); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
cycle := &detail{Values: []int{1, 2, 3}}
|
||||
cycle.Next = cycle
|
||||
b.Run("cycle", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst *detail
|
||||
if err := deep.Copy(&dst, cycle); err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
b.Run("parallelCycle", func(b *testing.B) {
|
||||
b.ReportAllocs()
|
||||
b.RunParallel(func(pb *testing.PB) {
|
||||
for pb.Next() {
|
||||
var dst *detail
|
||||
if err := deep.Copy(&dst, cycle); err != nil {
|
||||
b.Error(err)
|
||||
return
|
||||
}
|
||||
}
|
||||
})
|
||||
})
|
||||
for _, fresh := range []bool{false, true} {
|
||||
b.Run(fmt.Sprintf("converter/fresh%t", fresh), func(b *testing.B) {
|
||||
c, err := copyx.New(copyx.WithDeepCopy(), copyx.WithConverter(func(v []int) ([]int, error) {
|
||||
if fresh {
|
||||
return append([]int(nil), v...), nil
|
||||
}
|
||||
return v, nil
|
||||
}))
|
||||
if err != nil {
|
||||
b.Fatal(err)
|
||||
}
|
||||
src := []int{1, 2, 3}
|
||||
b.ReportAllocs()
|
||||
b.ResetTimer()
|
||||
for i := 0; i < b.N; i++ {
|
||||
var dst []int
|
||||
err := c.Copy(&dst, src)
|
||||
if fresh && err != nil || !fresh && !errors.Is(err, copyx.ErrSharedReference) {
|
||||
b.Fatal(err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
192
copyx/check.go
Normal file
192
copyx/check.go
Normal file
@@ -0,0 +1,192 @@
|
||||
package copyx
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"sort"
|
||||
)
|
||||
|
||||
// ErrDynamicType means a source interface requires a concrete runtime value.
|
||||
// An exact converter can explicitly own that dynamic boundary.
|
||||
var ErrDynamicType = errors.New("copyx: dynamic source type requires runtime validation or an exact converter")
|
||||
|
||||
// Check validates a source/target type pair using the same options as Copy.
|
||||
// It never executes converters and requires no example values. Nil policies,
|
||||
// converter failures, map-key collisions and deep reference isolation still
|
||||
// require runtime checks. Options remain local; no mapping files or registration.
|
||||
func Check[S, T any](options ...Option) error {
|
||||
c, err := New(options...)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Check(reflect.TypeFor[S](), reflect.TypeFor[T]())
|
||||
}
|
||||
|
||||
// Check validates types with an existing Copier's immutable settings. The target
|
||||
// is the result type (not Copy's extra output pointer). Safe for concurrent use.
|
||||
func (c *Copier) Check(source, target reflect.Type) error {
|
||||
if c == nil {
|
||||
return errors.New("copyx: nil Copier")
|
||||
}
|
||||
if source == nil || target == nil {
|
||||
return errors.New("copyx: source and target types are required")
|
||||
}
|
||||
return newTypeChecker(&c.options, false).check(source, target, "$", 0)
|
||||
}
|
||||
|
||||
type typeChecker struct {
|
||||
options *settings
|
||||
allowDynamic bool
|
||||
seen map[pair]bool
|
||||
nodes int
|
||||
}
|
||||
|
||||
func newTypeChecker(options *settings, allowDynamic bool) *typeChecker {
|
||||
return &typeChecker{options: options, allowDynamic: allowDynamic, seen: map[pair]bool{}}
|
||||
}
|
||||
|
||||
// Absent elements can reveal type/field mismatches, but contain no actual
|
||||
// resource handles or dynamic interface values to validate. Runtime checks stay
|
||||
// responsible for those value-dependent cases.
|
||||
func (s *state) checkAbsent(from, to reflect.Type, path string) error {
|
||||
if s.checkedTypes == nil {
|
||||
s.checkedTypes = map[pair]bool{}
|
||||
}
|
||||
checker := typeChecker{options: s.options, allowDynamic: true, seen: s.checkedTypes, nodes: s.checkedNodes}
|
||||
err := checker.check(from, to, path, 0)
|
||||
s.checkedNodes = checker.nodes
|
||||
return err
|
||||
}
|
||||
func (c *typeChecker) check(from, to reflect.Type, path string, depth int) error {
|
||||
c.nodes++
|
||||
if depth > maxDepth || c.nodes > maxNodes {
|
||||
return at(path, ErrLimit)
|
||||
}
|
||||
key := pair{from, to}
|
||||
if c.seen[key] {
|
||||
return nil
|
||||
}
|
||||
c.seen[key] = true
|
||||
if !c.allowDynamic && !shallowSafe(to) {
|
||||
return at(path, ErrUnsupported)
|
||||
}
|
||||
if _, ok := c.options.converters[key]; ok {
|
||||
return nil
|
||||
}
|
||||
if from == to && len(c.options.converters) == 0 && len(c.options.fields) == 0 && !c.options.skipZero && c.options.nilPolicy != NilSkip {
|
||||
if !c.options.deep || simpleValue(to) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if from.Kind() == reflect.Interface {
|
||||
if c.allowDynamic {
|
||||
return nil
|
||||
}
|
||||
return at(path, ErrDynamicType)
|
||||
}
|
||||
if to.Kind() == reflect.Interface {
|
||||
if !from.Implements(to) {
|
||||
return at(path, ErrType)
|
||||
}
|
||||
return c.check(from, from, path, depth+1)
|
||||
}
|
||||
if !c.allowDynamic && c.options.deep && forbidden(to) {
|
||||
return at(path, ErrUnsupported)
|
||||
}
|
||||
if from.Kind() == reflect.Pointer && to.Kind() != reflect.Pointer {
|
||||
return c.check(from.Elem(), to, path, depth+1)
|
||||
}
|
||||
if to.Kind() == reflect.Pointer {
|
||||
if !c.options.deep && from == to {
|
||||
return nil
|
||||
}
|
||||
if from.Kind() == reflect.Pointer {
|
||||
from = from.Elem()
|
||||
}
|
||||
return c.check(from, to.Elem(), path, depth+1)
|
||||
}
|
||||
if from.Kind() != to.Kind() {
|
||||
return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to))
|
||||
}
|
||||
if to == timeType && from == to {
|
||||
return nil
|
||||
}
|
||||
switch to.Kind() {
|
||||
case reflect.Struct:
|
||||
for i := 0; i < to.NumField(); i++ {
|
||||
f := to.Field(i)
|
||||
if !c.allowDynamic && f.PkgPath != "" && (from != to || c.options.deep && privateMutable(f.Type)) {
|
||||
return at(path+"."+f.Name, ErrUnsupported)
|
||||
}
|
||||
}
|
||||
sources, targets := exportedFields(from), exportedFields(to)
|
||||
rules := c.options.fields[key]
|
||||
if rules == nil {
|
||||
rules = &fields{}
|
||||
}
|
||||
reverse, used := map[string]string{}, map[string]bool{}
|
||||
for a, b := range rules.names {
|
||||
reverse[b] = a
|
||||
}
|
||||
for _, name := range sortedFieldNames(targets) {
|
||||
if rules.ignoreTarget[name] {
|
||||
continue
|
||||
}
|
||||
sourceName := name
|
||||
if n, ok := reverse[name]; ok {
|
||||
sourceName = n
|
||||
}
|
||||
source, ok := sources[sourceName]
|
||||
if !ok || rules.ignoreSource[sourceName] {
|
||||
return at(path+"."+name, ErrField)
|
||||
}
|
||||
used[sourceName] = true
|
||||
if err := c.check(source.Type, targets[name].Type, path+"."+name, depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if c.options.strict {
|
||||
for _, name := range sortedFieldNames(sources) {
|
||||
if !used[name] && !rules.ignoreSource[name] {
|
||||
return at(path+"."+name, ErrField)
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
case reflect.Slice:
|
||||
if !c.options.deep && from == to {
|
||||
return nil
|
||||
}
|
||||
return c.check(from.Elem(), to.Elem(), path+"[]", depth+1)
|
||||
case reflect.Array:
|
||||
if from.Len() != to.Len() {
|
||||
return at(path, ErrType)
|
||||
}
|
||||
return c.check(from.Elem(), to.Elem(), path+"[]", depth+1)
|
||||
case reflect.Map:
|
||||
if !c.options.deep && from == to {
|
||||
return nil
|
||||
}
|
||||
if err := c.check(from.Key(), to.Key(), path+"[key]", depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
return c.check(from.Elem(), to.Elem(), path+"[value]", depth+1)
|
||||
case reflect.Func, reflect.Chan, reflect.UnsafePointer:
|
||||
if c.options.deep && !c.allowDynamic {
|
||||
return at(path, ErrUnsupported)
|
||||
}
|
||||
}
|
||||
if from != to {
|
||||
return at(path, fmt.Errorf("%w: %s -> %s", ErrType, from, to))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func sortedFieldNames(fields map[string]reflect.StructField) []string {
|
||||
names := make([]string, 0, len(fields))
|
||||
for name := range fields {
|
||||
names = append(names, name)
|
||||
}
|
||||
sort.Strings(names)
|
||||
return names
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user