feat: complete API-driven projects with infrastructure and business acceptance
This commit is contained in:
10
ADVANCED.md
10
ADVANCED.md
@@ -11,15 +11,17 @@
|
|||||||
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
|
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
|
||||||
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
|
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
|
||||||
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
|
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
|
||||||
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
|
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别、轮转/脱敏/堆栈/慢请求 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
|
||||||
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
|
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
|
||||||
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
|
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
|
||||||
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
|
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
|
||||||
| `storage` | Local、S3 兼容(含 R2 配置)、OSS 官方 SDK 适配、临时下载链接 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
|
| `storage` | Local、S3/R2、OSS、临时下载、受限直传、云端分片 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
|
||||||
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
|
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
|
||||||
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
|
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
|
||||||
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
|
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
|
||||||
|
|
||||||
|
日志的配置、脱敏边界、轮转和慢请求记录见 [logx 使用说明](logx/README.md)。框架保留通用 Context/结构化字段扩展,不新增固定业务身份上下文。
|
||||||
|
|
||||||
## 对象存储
|
## 对象存储
|
||||||
|
|
||||||
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
|
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
|
||||||
@@ -51,9 +53,9 @@ files, err := oss.New(client, bucket, 32<<20)
|
|||||||
|
|
||||||
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
|
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
|
||||||
|
|
||||||
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,调用方保证与内容一致。默认单对象 32 MiB,流式读写;OSS 本适配器只做 simple upload,最大配置 5 GiB。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
|
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,并检查实际读取长度。默认单对象 32 MiB,流式读写;OSS 简单 Put 上限 5 GiB,更大使用独立分片能力。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
|
||||||
|
|
||||||
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。首版未封装前端直传、分片续传、生命周期、ACL、CDN 自定义域名和版本管理。
|
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。受限直传和云端分片现已提供可选接口,见 [上传与续传说明](storage/README.md)。生命周期、ACL、CDN 自定义域名和版本管理仍由平台/部署管理。
|
||||||
|
|
||||||
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。
|
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。
|
||||||
|
|
||||||
|
|||||||
169
API.md
Normal file
169
API.md
Normal file
@@ -0,0 +1,169 @@
|
|||||||
|
# API 契约与项目生成
|
||||||
|
|
||||||
|
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
|
||||||
|
|
||||||
|
## 与 go-zero 的对应关系
|
||||||
|
|
||||||
|
调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式:统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写,不引入 go-zero 或 Gin 运行时依赖。
|
||||||
|
|
||||||
|
| 使用体验 | go-omni 实现 |
|
||||||
|
| --- | --- |
|
||||||
|
| 新建接口项目 | `go-omni api new <directory>` |
|
||||||
|
| 按契约生成、更新 | `go-omni api generate --api user.api --dir .` |
|
||||||
|
| 熟悉的命令别名 | `api go -api user.api -dir .` |
|
||||||
|
| 校验契约 | `api validate --api user.api` |
|
||||||
|
| CI 检查生成产物是否过期 | `api generate --api user.api --dir . --check` |
|
||||||
|
| 业务逻辑目录 | `internal/service/<group>`,沿用 Omni 的 Service 分层 |
|
||||||
|
| 依赖装配 | `internal/servicecontext` |
|
||||||
|
| 接口文档 | 自动生成 `docs/openapi.json`,同时提供 `/openapi.json` |
|
||||||
|
|
||||||
|
这是明确限定的 go-zero 风格方言,**不是完整 goctl 兼容实现**。不支持的声明和校验标签会在生成前报错。
|
||||||
|
|
||||||
|
## 立即试用
|
||||||
|
|
||||||
|
源码开发时,从 go-omni 仓库运行,显式引用本地框架:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni api new \
|
||||||
|
--module example.com/omniDemo --framework-dir . ../omniDemo
|
||||||
|
|
||||||
|
cd ../omniDemo
|
||||||
|
go mod tidy
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
访问 `http://127.0.0.1:8080/api/greet/Alice` 可得到带 `data.message` 的统一响应。`api new` 的问候示例包含可运行逻辑;自定义契约新生成的 Service 会明确返回 501,等待业务实现。
|
||||||
|
|
||||||
|
用现成的多分组定义生成项目,在 go-omni 仓库执行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni new --api examples/contracts/user.api \
|
||||||
|
--module example.com/userService --framework-dir . \
|
||||||
|
--log-output file --log-format json ../userService
|
||||||
|
```
|
||||||
|
|
||||||
|
`api generate` 在新目录中也会生成完整项目。默认只监听 localhost;`GO_OMNI_HTTP_ADDR` 修改地址,`GO_OMNI_TRANSPORT=native` 切换网络层。配置顺序为默认值 → `etc/config.yaml` → 环境变量。文件日志支持现有轮转配置。远程试用使用与 CLI 一致的固定版本,见 [发布说明](RELEASE.md)。
|
||||||
|
|
||||||
|
API 项目现在支持组合数据库、存储和日志:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni new --api user.api --module example.com/userService \
|
||||||
|
--database mysql --storage local --log-output file ./userService
|
||||||
|
```
|
||||||
|
|
||||||
|
- `--database none|memory|mysql`:API 默认 `none`,`memory` 是不创建数据库连接的别名;MySQL 生成连接池、配置、事务入口和 readiness 探针。
|
||||||
|
- `--storage none|local|s3|r2|oss`:默认 none,生成统一 Files / Uploads / Downloads / UploadLinks 依赖;本地签名对象端点为 `/storage/object`。
|
||||||
|
- `--log-output stdout|stderr|file` 与 `--log-format json|text`:配置和无配置文件启动使用一致默认值。
|
||||||
|
|
||||||
|
初次选项记录在 `.goOmni/apiManifest.json`,以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错,因为初始化代码属于业务方;可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。旧式 CRUD 脚手架仍保留。
|
||||||
|
|
||||||
|
MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
|
||||||
|
|
||||||
|
HTTP 请求类型不能推导数据库表结构;迁移和 Repository 由业务自己维护。完整案例见 [用户管理示例](examples/userService/README.md),覆盖真实 MySQL、事务审计、鉴权、文件导出和日志。
|
||||||
|
|
||||||
|
## 接口定义
|
||||||
|
|
||||||
|
```go
|
||||||
|
syntax = "v1"
|
||||||
|
|
||||||
|
type UpdateUserReq {
|
||||||
|
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||||
|
Notify bool `form:"notify,default=false"`
|
||||||
|
Name string `json:"name" binding:"required,max=50"`
|
||||||
|
Age *int `json:"age,optional,range=[0:150]"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type UserResp {
|
||||||
|
UserID int64 `json:"userId"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
|
||||||
|
@server (
|
||||||
|
group: user
|
||||||
|
prefix: /api/v1
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@doc "Update a user"
|
||||||
|
@handler updateUser
|
||||||
|
put /users/:userId (UpdateUserReq) returns (UserResp)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
类型和 Go 字段使用导出命名;JSON、query、path 名称使用 lowerCamelCase;Header 使用 HTTP 标准名称。响应类型只定义 `data`,统一响应的 `code`、`message`、`requestId` 由框架添加。
|
||||||
|
|
||||||
|
支持:
|
||||||
|
|
||||||
|
- 可省略的 `syntax = "v1"`;`info` 中的 title、version 和描述性元数据。
|
||||||
|
- 单独的 `type X { ... }` 与 `type (...)`;基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。
|
||||||
|
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
|
||||||
|
- 同一 service 名称的多个块;`@server` 的 group、prefix、middleware,group 可以写 `admin/user`。
|
||||||
|
- 每个路由必须有 `@handler`;可选 `@doc "..."`;GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。
|
||||||
|
- 可省略请求或响应;有响应默认 200,无响应默认 204。Omni 扩展 `@status 201`、`@status 202`;204 不允许响应类型,HEAD 不允许响应体,GET/HEAD 不允许 JSON 请求体。
|
||||||
|
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query,不解析 multipart 或表单 body。query 切片用重复键;header/path 使用标量。
|
||||||
|
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
|
||||||
|
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
|
||||||
|
|
||||||
|
暂不支持匿名嵌入、别名、递归类型、外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证请用命名中间件接入。`@doc` 块等未列出的语法也会明确报错。
|
||||||
|
|
||||||
|
## 参数语义
|
||||||
|
|
||||||
|
无 optional/default 的字段按“是否出现”判断必填,`false`、`0`、`""` 是已提供的值。额外声明 `binding:"required"` 才会拒绝标量零值。默认值只在字段缺失时使用,不替换显式的空值。
|
||||||
|
|
||||||
|
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;非指针拒绝 null;`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
|
||||||
|
|
||||||
|
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
|
||||||
|
|
||||||
|
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;非 nullable 集合请返回空集合而不是 nil。
|
||||||
|
|
||||||
|
## 生成目录和业务边界
|
||||||
|
|
||||||
|
```text
|
||||||
|
api/ # 开发者维护的契约及导入
|
||||||
|
cmd/server/main.go # 启动、日志、优雅退出
|
||||||
|
etc/config.yaml
|
||||||
|
internal/settings/
|
||||||
|
internal/servicecontext/ # 依赖初始化和释放
|
||||||
|
internal/types/typesGen.go
|
||||||
|
internal/handler/routesGen.go
|
||||||
|
internal/handler/openapiGen.go
|
||||||
|
internal/handler/<group>/handlersGen.go
|
||||||
|
internal/service/<group>/service.go
|
||||||
|
internal/service/<group>/<operation>.go
|
||||||
|
internal/repository/<group>/repository.go
|
||||||
|
internal/middleware/registryGen.go
|
||||||
|
internal/middleware/<name>.go
|
||||||
|
docs/openapi.json
|
||||||
|
.goOmni/apiManifest.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Service 方法接收标准 `context.Context` 与生成的请求 DTO;Handler 调用 Service 并完成响应包装。网络适配器仍经 Omni/httpx 接入。Repository 是业务所有的扩展位置,不自动生成虚构 CRUD 查询。
|
||||||
|
|
||||||
|
命名中间件按声明顺序组合,第一个在最外层。新 hook 初始返回 501,必须实现后才放行;registry 自动维护,hook 文件只创建一次。缺失 hook 在注册任何路由前报错。文档通过 `x-omni-middleware` 标出命名 hook,不把名称误当成已实现的认证协议。
|
||||||
|
|
||||||
|
## 安全更新与 CI
|
||||||
|
|
||||||
|
在生成项目中安装/使用与框架版本相符的 go-omni CLI:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni api validate --api api/user.api
|
||||||
|
go-omni api generate --api api/user.api --dir .
|
||||||
|
go-omni api generate --api api/user.api --dir . --check
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
`Gen.go` 和 OpenAPI 由生成器管理,manifest 保存其 SHA-256。手动修改后更新会报冲突。Service、Repository、main、配置、依赖装配和中间件实现按字节保留;只补缺失文件。删除接口/分组时只移除旧的生成文件,保留业务代码;已删除类型仍被旧业务引用时,需要明确迁移该业务文件,编译器会提示。
|
||||||
|
|
||||||
|
`--check` 只读,检查产物是否过期,以及所需骨架是否缺失。契约复制到项目后,请编辑项目内的 API;外部定义与已存在副本不一致时会拒绝覆盖。一个项目由一个 API 入口管理,可以通过导入拆分文件。
|
||||||
|
|
||||||
|
生成器拒绝输出路径穿越、受管理路径上的符号链接、大小写路径冲突和无所有权的覆盖。生成前统一校验/格式化,锁文件防止两个生成器同时修改;文件以原子替换写入,普通写入失败会回滚。本版不是整个目录的崩溃原子事务:进程被强杀或机器断电后,应按 Git 恢复生成产物和 manifest,再重试。请将 manifest 与生成产物一并提交,也不要在生成过程中编辑这些文件。
|
||||||
|
|
||||||
|
生成器不自动安装依赖、不访问数据库、不推送仓库、不替业务实现权限规则。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test -race -p=2 ./contract ./openapi ./cmd/go-omni/...
|
||||||
|
python3 scripts/verifyAPI.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Go 测试覆盖契约拒绝规则、多源绑定、零值/缺失、默认/范围/嵌套集合、文档约束、幂等更新、只读检查、业务保留、删除路由、路径边界及生成项目编译。Python 验收会在临时目录生成项目,分别启动 nethttp/native,验证真实 HTTP 请求、中间件、日志 requestId 和优雅退出;结束后清理自己创建的进程和临时文件。结果记录在 `performance/apiGenerationLocal.json`。
|
||||||
12
CHANGELOG.md
12
CHANGELOG.md
@@ -1,6 +1,16 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
## Unreleased
|
## v0.7.0-preview
|
||||||
|
|
||||||
|
- 新增 `.api` 契约驱动生成器:DTO、严格多来源绑定与校验、路由、分组 Handler / Service / Repository、命名中间件及 OpenAPI;生成文件哈希管理、业务文件保护和只读 `--check`。
|
||||||
|
- `new --api` / `api new` / `api generate` 可以组合 MySQL、日志以及 Local / S3 / R2 / OSS。保存初始基础设施选项,重生成自动沿用;显式切换已有项目选项会拒绝,避免静默丢失初始化代码。
|
||||||
|
- 新增 `examples/userService` 独立业务示例与 `verifyAPIWorkflow.py`:用户 CRUD / 分页、管理员 Token 鉴权、MySQL 事务审计、故障回滚、文件导出、requestId 日志、数据库 readiness 和优雅退出;支持从远程版本验收。
|
||||||
|
- 日志新增文件轮转、结构化敏感字段脱敏、异常堆栈及慢请求配置;存储新增统一上传策略、直传签名和云端分片能力。
|
||||||
|
- 新增缓存、HTTP 客户端、限流 / 幂等 / 超时、后台任务、Redis 队列、事务 outbox、分布式锁、审计、迁移与 telemetry 等可选模块;业务身份与 Orbit 保持由应用自行组合。
|
||||||
|
- `database.ApplyEnv` 支持文件/默认配置叠加环境变量;数据库配置使用 lowerCamelCase,DSN 不参与 JSON/YAML 序列化。
|
||||||
|
- 修复 native 在业务层提前拒绝请求体为 413 时等待继续读取 body 的问题,拒绝后关闭未读连接。
|
||||||
|
|
||||||
|
以下是自上一版本以来的网络层变化与历史验收记录:
|
||||||
|
|
||||||
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
||||||
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
|
|
||||||
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
|
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
|
||||||
|
|
||||||
这批能力位于 main,尚未包含在 `v0.6.0-preview` 标签中。统一示例见
|
这批能力从 `v0.7.0-preview` 提供。统一示例见
|
||||||
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
||||||
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
||||||
|
|
||||||
|
|||||||
20
README.md
20
README.md
@@ -5,7 +5,7 @@ GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用
|
|||||||
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
go get git.yuntuo.hk/k2safe/go-omni@v0.7.0-preview
|
||||||
```
|
```
|
||||||
|
|
||||||
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
||||||
@@ -13,7 +13,7 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
|||||||
创建新业务项目:
|
创建新业务项目:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||||
go-omni new --module example.com/billing ./billing
|
go-omni new --module example.com/billing ./billing
|
||||||
cd billing
|
cd billing
|
||||||
go mod tidy
|
go mod tidy
|
||||||
@@ -27,6 +27,17 @@ go run ./cmd/server
|
|||||||
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
|
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
|
||||||
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
|
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
|
||||||
|
|
||||||
|
## API 定义与项目生成
|
||||||
|
|
||||||
|
当前源码新增 **API 定义驱动项目生成**:写 `.api` 定义请求、响应和路由,自动生成项目目录、DTO、Handler、Service 骨架及 OpenAPI。重复生成保留业务代码,支持只读 CI 检查,完整语法和示例见 [API 开发指南](API.md)。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni api new --framework-dir . \
|
||||||
|
--module example.com/omniDemo ../omniDemo
|
||||||
|
```
|
||||||
|
|
||||||
|
API 生成支持 `--database mysql --storage local --log-output file` 组合初始化。上面的命令使用本地源码;远程安装见 [发布说明](RELEASE.md),完整业务案例见 [userService](examples/userService/README.md)。
|
||||||
|
|
||||||
## 目录结构
|
## 目录结构
|
||||||
|
|
||||||
```text
|
```text
|
||||||
@@ -41,6 +52,7 @@ go-omni/
|
|||||||
├── metrics/ # 请求指标与状态统计
|
├── metrics/ # 请求指标与状态统计
|
||||||
├── config/ # 配置管理
|
├── config/ # 配置管理
|
||||||
├── validate/ # 参数校验
|
├── validate/ # 参数校验
|
||||||
|
├── contract/ # API 字段契约、严格多源绑定与文档约束
|
||||||
├── auth/ # 认证和权限接口
|
├── auth/ # 认证和权限接口
|
||||||
├── database/ # database/sql 生命周期和事务
|
├── database/ # database/sql 生命周期和事务
|
||||||
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
||||||
@@ -62,7 +74,7 @@ go-omni/
|
|||||||
|
|
||||||
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
||||||
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
||||||
可以直接运行 [组合示例](examples/modules/README.md)。这些新增能力尚未包含在上面的历史版本标签中。
|
可以直接运行 [组合示例](examples/modules/README.md)。这些能力从 `v0.7.0-preview` 提供。
|
||||||
|
|
||||||
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
|
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
|
||||||
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
|
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
|
||||||
@@ -167,7 +179,7 @@ app.GET("/metrics", requestMetrics.Handler())
|
|||||||
|
|
||||||
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
||||||
|
|
||||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。
|
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。文件轮转、敏感字段脱敏、异常堆栈与慢请求配置见 [logx 文档](logx/README.md);文件直传与云端分片上传见 [storage 文档](storage/README.md)。 创建新业务项目时可通过 `go-omni new --log-output file --storage local` 按需生成配置、装配代码与可运行示例,当前源码使用方式见 [CLI 文档](cmd/go-omni/README.md)。
|
||||||
|
|
||||||
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
||||||
|
|
||||||
|
|||||||
63
RELEASE.md
Normal file
63
RELEASE.md
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
# v0.7.0-preview
|
||||||
|
|
||||||
|
这一版把 API 定义、项目生成、基础模块初始化和完整业务示例接通。
|
||||||
|
预览版用于固定版本试用;API 方言有明确范围,HTTP/2 仍是独立实验适配器。
|
||||||
|
|
||||||
|
## 安装与开始
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOPRIVATE=git.yuntuo.hk go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||||
|
|
||||||
|
go-omni api new --module example.com/demo ./demo
|
||||||
|
cd demo
|
||||||
|
GOPRIVATE=git.yuntuo.hk go mod tidy
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
需要该 Git 仓库的读取权限时,通过自己的 Git 凭据管理器配置;不要将凭据写入 module path 或 go.mod。
|
||||||
|
CLI 会将自身下载版本固定到生成项目的 go.mod。业务项目正常使用远程模块,不需要本地 replace。
|
||||||
|
|
||||||
|
使用已有定义,并一次选择基础模块:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni new --api user.api --module example.com/userService \
|
||||||
|
--database mysql --storage local --log-output file ./userService
|
||||||
|
```
|
||||||
|
|
||||||
|
MySQL 与存储需要按生成 README 设置环境变量。任意契约生成的业务方法先返回 501,
|
||||||
|
由业务方实现;可直接参考 [完整用户示例](examples/userService/README.md)。
|
||||||
|
|
||||||
|
## 主要变化
|
||||||
|
|
||||||
|
- `.api` 生成项目、绑定/校验、路由、业务骨架和 OpenAPI;支持分组、导入、命名中间件。
|
||||||
|
- 同一入口初始化 MySQL、日志、Local / S3 / R2 / OSS;标准 context 与 httpx 保持业务/网络分离。
|
||||||
|
- 生成器保护手写业务文件,校验自有文件哈希,支持只读 --check;基础设施选项持久化到 manifest。
|
||||||
|
- 完整用户示例包含数据库迁移、CRUD / 分页、事务审计、鉴权、文件导出与 requestId 日志。
|
||||||
|
- 日志文件轮转/结构化脱敏、上传策略/直传/分片,以及此前 main 分支的通用业务模块一并提供。
|
||||||
|
|
||||||
|
所有变化及网络层验收记录见 [CHANGELOG](CHANGELOG.md)。
|
||||||
|
|
||||||
|
## 升级注意
|
||||||
|
|
||||||
|
1. 将 CLI 和项目框架依赖固定为相同版本。旧版 CRUD 项目继续使用现有目录;不能直接用 API 生成器接管已有 Go 目录。
|
||||||
|
2. Gen.go 由生成器维护,业务实现/配置/迁移由项目维护。新增 API 后执行 generate 和 go test;修改或删除类型需要同步处理自己的业务代码。
|
||||||
|
3. `.goOmni/apiManifest.json` 需随代码提交。切换数据库或存储选项时,在新目录生成对照后迁移;重新生成不会改写现有基础设施文件。
|
||||||
|
4. 配置使用 lowerCamelCase。DSN、Token、存储密钥由环境变量提供;`database.Config` 的 DSN 不会被 JSON/YAML 序列化。
|
||||||
|
5. API 的 none/memory 数据库选项仅表示不打开连接,不会从 HTTP DTO 自动推导内存仓库或数据表。
|
||||||
|
|
||||||
|
## 验证入口
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=2 -count=1 -timeout=300s ./...
|
||||||
|
go vet ./...
|
||||||
|
go build ./...
|
||||||
|
python3 scripts/verifyAPIWorkflow.py
|
||||||
|
python3 scripts/verifyAPIWorkflow.py --version v0.7.0-preview \
|
||||||
|
--report performance/apiWorkflowRemote.json
|
||||||
|
```
|
||||||
|
|
||||||
|
最后一项在全新 module cache 安装远程 CLI,并验证无本地 replace 的项目生成、
|
||||||
|
真实 MySQL、nethttp/native 启动与业务流程。容器只使用本机临时资源,结束时清理。
|
||||||
|
本轮验收见 [API 业务闭环报告](performance/apiWorkflowAcceptance.md)。
|
||||||
|
|
||||||
|
OSS/R2 云端仍需项目自己的账号做上线环境验收;当前云端适配器检查不等同于这些账号的实网测试。
|
||||||
@@ -65,13 +65,14 @@ func configure(options ...config.Option) config.Config {
|
|||||||
|
|
||||||
func newApp(router httpx.Router, settings config.Config) *App {
|
func newApp(router httpx.Router, settings config.Config) *App {
|
||||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
panic("go-omni: " + err.Error())
|
panic("go-omni: " + err.Error())
|
||||||
}
|
}
|
||||||
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
||||||
if settings.EnableLogger {
|
if settings.EnableLogger {
|
||||||
middlewares = append(middlewares, middleware.AccessLog())
|
middlewares = append(middlewares, middleware.AccessLog(middleware.WithSlowRequestThreshold(settings.SlowRequestThreshold)))
|
||||||
}
|
}
|
||||||
middlewares = append(middlewares, middleware.Recovery())
|
middlewares = append(middlewares, middleware.Recovery())
|
||||||
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
|
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
|
||||||
@@ -93,6 +94,7 @@ func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
|
|||||||
func (a *App) Config() config.Config {
|
func (a *App) Config() config.Config {
|
||||||
settings := a.config
|
settings := a.config
|
||||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
return settings
|
return settings
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,16 +1,79 @@
|
|||||||
# go-omni CLI
|
# go-omni CLI
|
||||||
|
|
||||||
|
## API 契约驱动项目
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# 创建包含可运行问候接口的新项目
|
||||||
|
go run ./cmd/go-omni api new --module example.com/demo --framework-dir . ../demo
|
||||||
|
|
||||||
|
# 从已有定义生成对应目录、类型、路由和业务骨架
|
||||||
|
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
|
||||||
|
--dir ../userService --module example.com/userService --framework-dir .
|
||||||
|
|
||||||
|
# 校验定义;也接受 goctl 风格的 api go -api ... -dir ...
|
||||||
|
go run ./cmd/go-omni api validate --api examples/contracts/user.api
|
||||||
|
```
|
||||||
|
|
||||||
|
`new --api <file>` 是契约项目的另一个入口。可以组合 `--database mysql --storage local --log-output file`,生成配置、依赖初始化与资源关闭。API 默认无数据库、无对象存储;`--database memory` 等同于不打开连接。`--force` 不允许覆盖 API 项目。初始配置选择写入 manifest,重新生成自动沿用。
|
||||||
|
|
||||||
|
后续在生成项目内执行 `go-omni api generate --api api/user.api --dir .`。
|
||||||
|
`--check` 只读检查生成产物是否过期。生成器记录所有权,保留 Service / Repository / 配置 / main 的手写内容,遇到生成文件被手动修改则报冲突。新 Service 与中间件占位实现返回 501。
|
||||||
|
|
||||||
|
支持范围、参数语义、目录及安全更新策略见 [API 开发指南](../../API.md)。本版借鉴 go-zero 的工作流,不宣称兼容全部 goctl 语法;仍通过 Omni/httpx 保留网络层替换能力。
|
||||||
|
|
||||||
|
## 日志与对象存储按需装配
|
||||||
|
|
||||||
|
新增 `--log-output stdout|stderr|file`、`--log-format json|text`、`--storage none|local|s3|r2|oss`。
|
||||||
|
指定任意一个新选项后,生成严格的配置文件、初始化代码和项目自身的测试;不指定则保留原有轻量骨架。
|
||||||
|
对象存储独立于 `--database memory|mysql`。只有选择文件存储才会生成文件 Handler / Service 和对应适配器。
|
||||||
|
|
||||||
|
开发阶段可以使用本地源码;固定远程版本的安装和验收见 [发布说明](../../RELEASE.md):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# 在 go-omni 仓库中运行;目标目录须为空。
|
||||||
|
go run ./cmd/go-omni new --framework-dir . \
|
||||||
|
--module example.com/fileService --log-output file --log-format json \
|
||||||
|
--storage local ../fileService
|
||||||
|
```
|
||||||
|
|
||||||
|
生成项目包含 `etc/config.yaml`、`internal/settings`、`internal/servicecontext/storage.go`、
|
||||||
|
文件 Service/Handler、测试和 `MODULES.md`。按生成的 MODULES.md 配置示例 Token 和本地签名密钥后,
|
||||||
|
`go mod tidy && go run ./cmd/server` 即可体验。云存储分别选择 `--storage s3`、`r2`、`oss`,
|
||||||
|
再设置已有桶/凭据等环境变量;不自动创建桶或修改云资源。
|
||||||
|
|
||||||
|
配置优先级为默认值 → YAML/JSON → 环境变量;密钥仅允许环境变量。Service 依赖统一存储接口,
|
||||||
|
组合处负责创建客户端与有界连接池,退出时关闭自己创建的资源。文件接口提供服务端上传、直传签名、
|
||||||
|
临时下载链接及统一错误处理;日志示例携带 requestId,文件输出默认带轮转。没有固定业务身份上下文。
|
||||||
|
|
||||||
|
新模板要求 `v0.7.0-preview` 或更新版本;明确指定旧版本并启用新模块会被拒绝,
|
||||||
|
避免模板引用旧框架不存在的接口。本地 `--framework-dir` 验证不受此限制。`--force` 可重新生成兼容配置,
|
||||||
|
但如果切换配置会遗留引用已移除依赖的旧文件,则拒绝操作,要求使用新目录;不会自动删除修改过的代码。
|
||||||
|
|
||||||
|
验证当前生成器:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./cmd/go-omni/...
|
||||||
|
python3 scripts/verifyScaffoldModules.py
|
||||||
|
```
|
||||||
|
|
||||||
|
前者覆盖两种数据库与五种文件存储选择、日志方式/格式,以及生成项目的配置、资源关闭和接口测试。
|
||||||
|
后者实际编译启动服务,验证日志、上传/下载/直传链接、错误状态、nethttp/native 下的 SIGINT/SIGTERM 和监听端口释放。
|
||||||
|
云端选项的测试使用本地替身,不代表 OSS/R2 实网验收;本批不访问业务服务器。报告见
|
||||||
|
[脚手架模块验收](../../performance/scaffoldModulesLocal.json)。
|
||||||
|
|
||||||
|
## 其他命令与远程安装
|
||||||
|
|
||||||
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
||||||
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
||||||
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
||||||
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
||||||
该命令尚未包含在下方历史标签中;当前源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
该命令从 `v0.7.0-preview` 提供;源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
||||||
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
||||||
|
|
||||||
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||||
go-omni new --module example.com/billing ./billing
|
go-omni new --module example.com/billing ./billing
|
||||||
cd billing
|
cd billing
|
||||||
go mod tidy
|
go mod tidy
|
||||||
|
|||||||
127
cmd/go-omni/api.go
Normal file
127
cmd/go-omni/api.go
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||||
|
)
|
||||||
|
|
||||||
|
func runAPI(args []string) error {
|
||||||
|
if len(args) == 0 || args[0] == "help" || args[0] == "--help" || args[0] == "-h" {
|
||||||
|
usage()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
command := args[0]
|
||||||
|
if command == "go" {
|
||||||
|
command = "generate"
|
||||||
|
}
|
||||||
|
if command == "check" {
|
||||||
|
command = "validate"
|
||||||
|
}
|
||||||
|
if command != "new" && command != "generate" && command != "validate" {
|
||||||
|
return fmt.Errorf("unknown api command %q", command)
|
||||||
|
}
|
||||||
|
o := apigen.Options{}
|
||||||
|
for i := 1; i < len(args); i++ {
|
||||||
|
arg := args[i]
|
||||||
|
if arg == "--help" || arg == "-h" {
|
||||||
|
usage()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if arg == "--check" {
|
||||||
|
if command != "generate" {
|
||||||
|
return errors.New("--check belongs to api generate")
|
||||||
|
}
|
||||||
|
o.Check = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(arg, "-") {
|
||||||
|
if command == "new" && o.Dir == "" {
|
||||||
|
o.Dir = arg
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return fmt.Errorf("unexpected argument %q", arg)
|
||||||
|
}
|
||||||
|
name, value, inline := strings.Cut(arg, "=")
|
||||||
|
// goctl-style -api/-dir aliases ease migration; long forms are canonical.
|
||||||
|
if name == "-api" {
|
||||||
|
name = "--api"
|
||||||
|
}
|
||||||
|
if name == "-dir" {
|
||||||
|
name = "--dir"
|
||||||
|
}
|
||||||
|
switch name {
|
||||||
|
case "--api", "--dir", "--module", "--framework-dir", "--framework-version", "--log-output", "--log-format", "--database", "--storage":
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("unknown api option %q", name)
|
||||||
|
}
|
||||||
|
if !inline {
|
||||||
|
if i+1 >= len(args) || strings.HasPrefix(args[i+1], "-") {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
i++
|
||||||
|
value = args[i]
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(value) == "" {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
if command == "validate" && name != "--api" {
|
||||||
|
return fmt.Errorf("api validate does not accept %s", name)
|
||||||
|
}
|
||||||
|
switch name {
|
||||||
|
case "--api":
|
||||||
|
o.API = value
|
||||||
|
case "--dir":
|
||||||
|
o.Dir = value
|
||||||
|
case "--module":
|
||||||
|
o.Module = value
|
||||||
|
case "--framework-dir":
|
||||||
|
o.FrameworkDir = value
|
||||||
|
case "--framework-version":
|
||||||
|
o.FrameworkVersion = value
|
||||||
|
case "--database":
|
||||||
|
o.Database = value
|
||||||
|
case "--storage":
|
||||||
|
o.Storage = value
|
||||||
|
case "--log-output":
|
||||||
|
o.LogOutput = value
|
||||||
|
case "--log-format":
|
||||||
|
o.LogFormat = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if command == "validate" {
|
||||||
|
if o.API == "" {
|
||||||
|
return errors.New("--api is required")
|
||||||
|
}
|
||||||
|
spec, err := apigen.Parse(o.API)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err = apigen.Compile(spec); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Printf("valid API: %d types, %d routes\n", len(spec.Types), len(spec.Routes))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var out apigen.Output
|
||||||
|
var err error
|
||||||
|
if command == "new" {
|
||||||
|
out, err = apigen.New(o)
|
||||||
|
} else {
|
||||||
|
if o.Dir == "" {
|
||||||
|
o.Dir = "."
|
||||||
|
}
|
||||||
|
out, err = apigen.Generate(o)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if o.Check {
|
||||||
|
fmt.Println("API generated files are current")
|
||||||
|
} else {
|
||||||
|
fmt.Printf("API project ready: %d generated, %d created, %d preserved, %d removed\n", len(out.Generated), len(out.Created), len(out.Preserved), len(out.Removed))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
44
cmd/go-omni/api_test.go
Normal file
44
cmd/go-omni/api_test.go
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAPICommands(t *testing.T) {
|
||||||
|
root, err := filepath.Abs("../..")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
project := filepath.Join(t.TempDir(), "demo")
|
||||||
|
if err = runAPI([]string{"new", project, "--module=example.com/demo", "--framework-dir", root}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
api := filepath.Join(project, "api/service.api")
|
||||||
|
if err = runAPI([]string{"validate", "-api", api}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = runAPI([]string{"go", "-api", api, "-dir", project, "--check"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = runNew([]string{"--force", project}); err == nil {
|
||||||
|
t.Fatal("CRUD starter overwrote API project")
|
||||||
|
}
|
||||||
|
target := filepath.Join(t.TempDir(), "other")
|
||||||
|
if err = runNew([]string{"--api", api, "--module", "example.com/other", "--framework-dir", root, target}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = os.Stat(filepath.Join(target, "internal/service/greet/greet.go")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, args := range [][]string{{"validate"}, {"generate", "--api"}, {"new", "--api=x", "somewhere"}, {"generate", "--dir="}, {"generate", "--force"}, {"validate", "--api", api, "--dir", project}, {"validate", "--check"}} {
|
||||||
|
if err = runAPI(args); err == nil {
|
||||||
|
t.Fatalf("invalid args accepted: %v", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err = runNew([]string{"--api", api, "--force", target}); err == nil || !strings.Contains(err.Error(), "does not accept --force") {
|
||||||
|
t.Fatalf("incompatible profile: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
360
cmd/go-omni/apigen/apigen_test.go
Normal file
360
cmd/go-omni/apigen/apigen_test.go
Normal file
@@ -0,0 +1,360 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func frameworkRoot(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
p, err := filepath.Abs("../../..")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
func writeFile(t *testing.T, name, body string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.MkdirAll(filepath.Dir(name), 0755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(name, []byte(body), 0644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func readFile(t *testing.T, name string) string {
|
||||||
|
t.Helper()
|
||||||
|
b, err := os.ReadFile(name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return string(b)
|
||||||
|
}
|
||||||
|
func parseText(t *testing.T, source string) (*Spec, error) {
|
||||||
|
t.Helper()
|
||||||
|
file := filepath.Join(t.TempDir(), "service.api")
|
||||||
|
writeFile(t, file, source)
|
||||||
|
return Parse(file)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompileExample(t *testing.T) {
|
||||||
|
s, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/user.api"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
c, err := Compile(s)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(c.Groups) != 2 || len(s.Routes) != 3 || len(s.Sources) != 2 {
|
||||||
|
t.Fatalf("wrong spec: %+v", s)
|
||||||
|
}
|
||||||
|
var doc map[string]any
|
||||||
|
if err = json.Unmarshal(c.Document, &doc); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
route := doc["paths"].(map[string]any)["/api/v1/users/{userId}"].(map[string]any)["put"].(map[string]any)
|
||||||
|
if len(route["parameters"].([]any)) != 3 {
|
||||||
|
t.Fatal("missing mixed parameters")
|
||||||
|
}
|
||||||
|
schema := route["requestBody"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
|
||||||
|
if schema["additionalProperties"] != false || len(schema["required"].([]any)) != 2 {
|
||||||
|
t.Fatalf("wrong body schema: %+v", schema)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(c.Document), `"user_id"`) {
|
||||||
|
t.Fatal("snake case wire name")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestInvalidDefinitions(t *testing.T) {
|
||||||
|
cases := []string{
|
||||||
|
`syntax="v2"`,
|
||||||
|
`@server ( jwt: Auth ) service user { @handler go get /go }`,
|
||||||
|
"type Request { Name string `json:\"name\" binding:\"typo\"` }\nservice user { @handler run\n post /go (Request) }",
|
||||||
|
"type A { Other *A `json:\"other,optional\"` }\nservice user { @handler run\n post /go (A) }",
|
||||||
|
"type A { Other Missing `json:\"other\"` }\nservice user { @handler run\n post /go (A) }",
|
||||||
|
"type A { UserID int `json:\"user_id\"` }\nservice user { @handler run\n post /go (A) }",
|
||||||
|
"service user { @handler go\n get /go\n @handler other\n get /go\n}",
|
||||||
|
"service user { @handler go\n get /:id\n}",
|
||||||
|
"service user { @handler service\n get /go\n}",
|
||||||
|
"type A { ID int `path:\"id\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||||
|
"type A { Name string `json:\"name\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||||
|
"service user { @handler go\n @status 201\n post /go\n}",
|
||||||
|
"type A { Name string `header:\"Name\"` }\nservice user { @handler go\n get /go returns (A)\n}",
|
||||||
|
"@server (group: ../../bad)\nservice user { @handler go\n get /go\n}",
|
||||||
|
"service user { @handler go\n get /health/ready\n}",
|
||||||
|
"type A { Name string `json:\"name\"`\n Name int `json:\"other\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||||
|
"service user { @handler go\n get /go\n @handler Go\n get /other\n}",
|
||||||
|
"type A { Value []byte `json:\"value\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||||
|
}
|
||||||
|
for i, source := range cases {
|
||||||
|
s, err := parseText(t, source)
|
||||||
|
if err == nil {
|
||||||
|
_, err = Compile(s)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("case %d accepted: %s", i, source)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestImports(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
entry := filepath.Join(root, "api", "entry.api")
|
||||||
|
writeFile(t, entry, "import \"types.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||||
|
writeFile(t, filepath.Join(root, "api/types.api"), `import "entry.api"`)
|
||||||
|
if _, err := Parse(entry); err == nil || !strings.Contains(err.Error(), "circular") {
|
||||||
|
t.Fatalf("cycle: %v", err)
|
||||||
|
}
|
||||||
|
writeFile(t, entry, "import \"../outside.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||||
|
writeFile(t, filepath.Join(root, "outside.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||||
|
if _, err := Parse(entry); err == nil {
|
||||||
|
t.Fatal("escaped import root")
|
||||||
|
}
|
||||||
|
writeFile(t, entry, "import (\n \"shared.api\"\n \"./shared.api\"\n)\nservice user { @handler ping\n get /ping\n}")
|
||||||
|
writeFile(t, filepath.Join(root, "api/shared.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||||
|
s, err := Parse(entry)
|
||||||
|
if err != nil || len(s.Types) != 1 {
|
||||||
|
t.Fatalf("duplicate import: %+v %v", s, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSharedPrefixConflict(t *testing.T) {
|
||||||
|
source := "type A { ID int `path:\"id\"` }\ntype B { Name string `path:\"name\"` }\nservice user {\n @handler first\n get /users/:id (A)\n @handler second\n get /users/:name/items (B)\n}"
|
||||||
|
s, err := parseText(t, source)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = Compile(s); err == nil {
|
||||||
|
t.Fatal("shared prefix conflict was accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func FuzzParser(f *testing.F) {
|
||||||
|
for _, s := range []string{starterAPI, "syntax = \"v1\"", "type (", "@server (prefix: /api)"} {
|
||||||
|
f.Add(s)
|
||||||
|
}
|
||||||
|
f.Fuzz(func(t *testing.T, source string) {
|
||||||
|
if len(source) > 16384 {
|
||||||
|
t.Skip()
|
||||||
|
}
|
||||||
|
tokens, err := lex("fuzz.api", source)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
spec := &Spec{Title: "Fuzz", Version: "1"}
|
||||||
|
p := parser{tokens: tokens, spec: spec}
|
||||||
|
if p.parse() == nil {
|
||||||
|
_, _ = Compile(spec)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
func TestRegenerationAndConflicts(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "app")
|
||||||
|
if _, err := New(Options{Dir: dir, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
manifestBefore := readFile(t, filepath.Join(dir, manifestPath))
|
||||||
|
typesBefore := readFile(t, filepath.Join(dir, "internal/types/typesGen.go"))
|
||||||
|
o.Check = true
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
servicePath := filepath.Join(dir, "internal/service/greet/greet.go")
|
||||||
|
service := readFile(t, servicePath) + "\n// user edit must survive\n"
|
||||||
|
writeFile(t, servicePath, service)
|
||||||
|
for _, file := range []string{"cmd/server/main.go", "etc/config.yaml", "internal/servicecontext/context.go", "internal/repository/greet/repository.go"} {
|
||||||
|
body := readFile(t, filepath.Join(dir, file))
|
||||||
|
if strings.HasSuffix(file, ".go") {
|
||||||
|
body += "\n// handwritten extension\n"
|
||||||
|
} else {
|
||||||
|
body += "\n# handwritten extension\n"
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, file), body)
|
||||||
|
}
|
||||||
|
api := readFile(t, o.API)
|
||||||
|
api += "\n@server (group: extra)\nservice greet-api {\n @handler ping\n get /ping\n}\n"
|
||||||
|
writeFile(t, o.API, api)
|
||||||
|
if _, err := Generate(o); err == nil {
|
||||||
|
t.Fatal("check accepted stale output")
|
||||||
|
}
|
||||||
|
if readFile(t, filepath.Join(dir, manifestPath)) != manifestBefore {
|
||||||
|
t.Fatal("--check wrote manifest")
|
||||||
|
}
|
||||||
|
o.Check = false
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if readFile(t, servicePath) != service {
|
||||||
|
t.Fatal("business code overwritten")
|
||||||
|
}
|
||||||
|
// Retiring a group removes only owned handlers, keeping its business files.
|
||||||
|
writeFile(t, o.API, strings.Split(api, "\n@server (group: extra)")[0])
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "internal/handler/extra/handlersGen.go")); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("stale handler remains")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "internal/service/extra/ping.go")); err != nil {
|
||||||
|
t.Fatal("business stub removed")
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), typesBefore+"\n// edited generated file\n")
|
||||||
|
before := snapshot(t, dir)
|
||||||
|
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "was edited") {
|
||||||
|
t.Fatalf("did not protect edited output: %v", err)
|
||||||
|
}
|
||||||
|
after := snapshot(t, dir)
|
||||||
|
if !sameFiles(before, after) {
|
||||||
|
t.Fatal("failed regeneration changed files")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestOutputBoundaries(t *testing.T) {
|
||||||
|
for _, kind := range []string{"symlink", "unowned", "lock", "caseCollision"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
dir := filepath.Join(root, "app")
|
||||||
|
api := filepath.Join(root, "input/service.api")
|
||||||
|
writeFile(t, api, starterAPI)
|
||||||
|
switch kind {
|
||||||
|
case "symlink":
|
||||||
|
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
outside := filepath.Join(root, "outside")
|
||||||
|
os.MkdirAll(outside, 0755)
|
||||||
|
if err := os.Symlink(outside, filepath.Join(dir, "internal")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "unowned":
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), "package types // mine\n")
|
||||||
|
case "lock":
|
||||||
|
writeFile(t, filepath.Join(dir, ".goOmni/api.lock"), "someone else")
|
||||||
|
case "caseCollision":
|
||||||
|
writeFile(t, api, starterAPI+"\n@server (group: gReet)\nservice greet-api {\n @handler extra\n get /extra\n}\n")
|
||||||
|
}
|
||||||
|
before := snapshot(t, root)
|
||||||
|
if _, err := Generate(Options{Dir: dir, API: api, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err == nil {
|
||||||
|
t.Fatal("unsafe output accepted")
|
||||||
|
}
|
||||||
|
if !sameFiles(before, snapshot(t, root)) {
|
||||||
|
t.Fatal("conflict changed files")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func snapshot(t *testing.T, root string) map[string]string {
|
||||||
|
t.Helper()
|
||||||
|
out := map[string]string{}
|
||||||
|
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if d.Type().IsRegular() {
|
||||||
|
data, e := os.ReadFile(path)
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
rel, _ := filepath.Rel(root, path)
|
||||||
|
out[rel] = string(data)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
func sameFiles(a, b map[string]string) bool {
|
||||||
|
if len(a) != len(b) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for k, v := range a {
|
||||||
|
if b[k] != v {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGeneratedProjectsBuildAndRun(t *testing.T) {
|
||||||
|
if testing.Short() {
|
||||||
|
t.Skip("consumer module integration")
|
||||||
|
}
|
||||||
|
root := frameworkRoot(t)
|
||||||
|
for _, starter := range []bool{true, false} {
|
||||||
|
name := "contract"
|
||||||
|
if starter {
|
||||||
|
name = "starter"
|
||||||
|
}
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "project")
|
||||||
|
o := Options{Dir: dir, Module: "example.com/consumer", FrameworkDir: root, LogOutput: "file", LogFormat: "text"}
|
||||||
|
var err error
|
||||||
|
if starter {
|
||||||
|
_, err = New(o)
|
||||||
|
} else {
|
||||||
|
o.API = filepath.Join(root, "examples/contracts/user.api")
|
||||||
|
_, err = Generate(o)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
source := consumerContractTest
|
||||||
|
if starter {
|
||||||
|
source = consumerStarterTest
|
||||||
|
} else {
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/service/user/updateUser.go"), `package user
|
||||||
|
import("context";"example.com/consumer/internal/types")
|
||||||
|
func(s *Service) UpdateUser(ctx context.Context,req *types.UpdateUserReq)(*types.UserResp,error){return &types.UserResp{UserID:req.UserID,Name:req.Name,Active:req.Active},nil}
|
||||||
|
`)
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, "contract_test.go"), source)
|
||||||
|
o.API = filepath.Join(dir, "api/service.api")
|
||||||
|
if !starter {
|
||||||
|
o.API = filepath.Join(dir, "api/user.api")
|
||||||
|
}
|
||||||
|
if _, err = Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||||
|
cmd.Dir = dir
|
||||||
|
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("consumer: %v\n%s", err, output)
|
||||||
|
}
|
||||||
|
t.Log(string(output))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const consumerStarterTest = `package consumer_test
|
||||||
|
import("context";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext";"example.com/consumer/internal/settings")
|
||||||
|
func TestGreeting(t *testing.T){cfg,err:=settings.Load("etc/config.yaml");if err!=nil{t.Fatal(err)};cfg.Server.Log.Writer=io.Discard;cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{};logger,err:=logx.New(cfg.Server.Log);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)};w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,httptest.NewRequest("GET","/api/greet/Alice",nil));if w.Code!=200||!strings.Contains(w.Body.String(),"Hello Alice")||w.Header().Get("X-Request-ID")==""{t.Fatalf("%d %s",w.Code,w.Body.String())}}
|
||||||
|
`
|
||||||
|
const consumerContractTest = `package consumer_test
|
||||||
|
import("context";"encoding/json";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext")
|
||||||
|
func TestContract(t *testing.T){cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
|
||||||
|
for _,tc:=range []struct{method,path,body string;status int}{
|
||||||
|
{"PUT","/api/v1/users/7?notify=true",` + "`" + `{"name":"Alice","active":false,"age":0}` + "`" + `,200},
|
||||||
|
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice"}` + "`" + `,400},
|
||||||
|
{"PUT","/api/v1/users/7?notify=true¬ify=false",` + "`" + `{"name":"Alice","active":false}` + "`" + `,400},
|
||||||
|
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice","active":false,"age":151}` + "`" + `,400},
|
||||||
|
{"GET","/api/v1/users?page=0","",400},
|
||||||
|
{"GET","/api/v1/users?page=1","",501},
|
||||||
|
{"DELETE","/admin/v1/users/7","",501},
|
||||||
|
}{req:=httptest.NewRequest(tc.method,tc.path,strings.NewReader(tc.body));req.Header.Set("Content-Type","application/json");w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,req);if w.Code!=tc.status{t.Fatalf("%s: %d %s",tc.path,w.Code,w.Body.String())};if tc.status==200{var v struct{Data struct{UserID int64 ` + "`json:\"userId\"`" + `;Name string ` + "`json:\"name\"`" + `}};if err:=json.Unmarshal(w.Body.Bytes(),&v);err!=nil||v.Data.UserID!=7||v.Data.Name!="Alice"{t.Fatal(w.Body.String())}}}
|
||||||
|
delete(deps.Middlewares,"Auth");if err:=handler.RegisterRoutes(omni.New(),deps);err==nil{t.Fatal("missing auth hook accepted")}
|
||||||
|
}
|
||||||
|
`
|
||||||
332
cmd/go-omni/apigen/compile.go
Normal file
332
cmd/go-omni/apigen/compile.go
Normal file
@@ -0,0 +1,332 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"go/ast"
|
||||||
|
goparser "go/parser"
|
||||||
|
gotoken "go/token"
|
||||||
|
"reflect"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Compiled struct {
|
||||||
|
Spec *Spec
|
||||||
|
Bindings map[string]*contract.Binder
|
||||||
|
Document []byte
|
||||||
|
Groups []string
|
||||||
|
}
|
||||||
|
|
||||||
|
var exportedName = regexp.MustCompile(`^[A-Z][A-Za-z0-9]*$`)
|
||||||
|
var goName = regexp.MustCompile(`^[a-zA-Z][A-Za-z0-9]*$`)
|
||||||
|
var lowerName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||||
|
var serviceName = regexp.MustCompile(`^[a-zA-Z][a-zA-Z0-9-]*$`)
|
||||||
|
var staticSegment = regexp.MustCompile(`^[A-Za-z0-9._~-]+$`)
|
||||||
|
|
||||||
|
func Compile(spec *Spec) (*Compiled, error) {
|
||||||
|
if spec == nil || len(spec.Routes) == 0 || !serviceName.MatchString(spec.Service) {
|
||||||
|
return nil, errors.New("api: valid service and routes required")
|
||||||
|
}
|
||||||
|
definitions := map[string]Type{}
|
||||||
|
for _, t := range spec.Types {
|
||||||
|
if !exportedName.MatchString(t.Name) {
|
||||||
|
return nil, fmt.Errorf("%s: type names must be UpperCamelCase", t.Position)
|
||||||
|
}
|
||||||
|
if _, ok := definitions[t.Name]; ok {
|
||||||
|
return nil, fmt.Errorf("%s: duplicate type %s", t.Position, t.Name)
|
||||||
|
}
|
||||||
|
definitions[t.Name] = t
|
||||||
|
}
|
||||||
|
resolved := map[string]reflect.Type{}
|
||||||
|
resolving := map[string]bool{}
|
||||||
|
var resolve func(string, int) (reflect.Type, error)
|
||||||
|
var expression func(ast.Expr, int) (reflect.Type, error)
|
||||||
|
primitives := map[string]reflect.Type{"string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||||
|
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
|
||||||
|
if depth > 32 {
|
||||||
|
return nil, errors.New("type nesting exceeds 32")
|
||||||
|
}
|
||||||
|
switch e := e.(type) {
|
||||||
|
case *ast.Ident:
|
||||||
|
if t, ok := primitives[e.Name]; ok {
|
||||||
|
return t, nil
|
||||||
|
}
|
||||||
|
return resolve(e.Name, depth+1)
|
||||||
|
case *ast.StarExpr:
|
||||||
|
if _, double := e.X.(*ast.StarExpr); double {
|
||||||
|
return nil, errors.New("multiple pointer levels unsupported")
|
||||||
|
}
|
||||||
|
t, err := expression(e.X, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return reflect.PointerTo(t), nil
|
||||||
|
case *ast.ArrayType:
|
||||||
|
if e.Len != nil {
|
||||||
|
return nil, errors.New("use slices, not fixed arrays")
|
||||||
|
}
|
||||||
|
t, err := expression(e.Elt, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return reflect.SliceOf(t), nil
|
||||||
|
case *ast.MapType:
|
||||||
|
key, ok := e.Key.(*ast.Ident)
|
||||||
|
if !ok || key.Name != "string" {
|
||||||
|
return nil, errors.New("map keys must be string")
|
||||||
|
}
|
||||||
|
t, err := expression(e.Value, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return reflect.MapOf(reflect.TypeOf(""), t), nil
|
||||||
|
default:
|
||||||
|
return nil, errors.New("unsupported type expression")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resolve = func(name string, depth int) (reflect.Type, error) {
|
||||||
|
if depth > 32 {
|
||||||
|
return nil, errors.New("type nesting exceeds 32")
|
||||||
|
}
|
||||||
|
if t, ok := resolved[name]; ok {
|
||||||
|
return t, nil
|
||||||
|
}
|
||||||
|
if resolving[name] {
|
||||||
|
return nil, fmt.Errorf("recursive type %s is unsupported", name)
|
||||||
|
}
|
||||||
|
typ, ok := definitions[name]
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("unknown type %s", name)
|
||||||
|
}
|
||||||
|
resolving[name] = true
|
||||||
|
defer delete(resolving, name)
|
||||||
|
var fields []reflect.StructField
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, f := range typ.Fields {
|
||||||
|
if !exportedName.MatchString(f.Name) || seen[f.Name] {
|
||||||
|
return nil, fmt.Errorf("%s: invalid or duplicate field %s", name, f.Name)
|
||||||
|
}
|
||||||
|
seen[f.Name] = true
|
||||||
|
expr, err := goparser.ParseExpr(f.Type)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
t, err := expression(expr, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err)
|
||||||
|
}
|
||||||
|
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)})
|
||||||
|
}
|
||||||
|
t := reflect.StructOf(fields)
|
||||||
|
resolved[name] = t
|
||||||
|
return t, nil
|
||||||
|
}
|
||||||
|
c := &Compiled{Spec: spec, Bindings: map[string]*contract.Binder{}}
|
||||||
|
for _, t := range spec.Types {
|
||||||
|
rt, err := resolve(t.Name, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := contract.ForType(rt)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", t.Position, err)
|
||||||
|
}
|
||||||
|
c.Bindings[t.Name] = b
|
||||||
|
}
|
||||||
|
api, err := openapi.New(spec.Title, spec.Version)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{}
|
||||||
|
middlewareSymbols := map[string]string{}
|
||||||
|
for _, r := range spec.Routes {
|
||||||
|
if !goName.MatchString(r.Handler) {
|
||||||
|
return nil, fmt.Errorf("%s: invalid handler name", r.Position)
|
||||||
|
}
|
||||||
|
if strings.EqualFold(r.Handler, "service") {
|
||||||
|
return nil, errors.New("handler name service conflicts with the service constructor file")
|
||||||
|
}
|
||||||
|
if r.Group == "" {
|
||||||
|
return nil, errors.New("group required")
|
||||||
|
}
|
||||||
|
for _, part := range strings.Split(r.Group, "/") {
|
||||||
|
if !lowerName.MatchString(part) || gotoken.Lookup(part).IsKeyword() {
|
||||||
|
return nil, fmt.Errorf("invalid lowerCamelCase group %q", r.Group)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
h := r.Group + "/" + strings.ToLower(r.Handler)
|
||||||
|
if handlers[h] {
|
||||||
|
return nil, fmt.Errorf("duplicate handler %s in group %s", r.Handler, r.Group)
|
||||||
|
}
|
||||||
|
handlers[h] = true
|
||||||
|
groups[r.Group] = true
|
||||||
|
mseen := map[string]bool{}
|
||||||
|
for _, m := range r.Middleware {
|
||||||
|
if !goName.MatchString(m) || mseen[m] {
|
||||||
|
return nil, fmt.Errorf("invalid or duplicate middleware %q", m)
|
||||||
|
}
|
||||||
|
mseen[m] = true
|
||||||
|
if strings.EqualFold(m, "defaults") || strings.EqualFold(m, "registryGen") {
|
||||||
|
return nil, fmt.Errorf("reserved middleware name %q", m)
|
||||||
|
}
|
||||||
|
if prior, ok := middlewareSymbols[strings.ToLower(m)]; ok && prior != m {
|
||||||
|
return nil, fmt.Errorf("middleware names %s and %s collide", prior, m)
|
||||||
|
}
|
||||||
|
middlewareSymbols[strings.ToLower(m)] = m
|
||||||
|
}
|
||||||
|
shape, err := routeShape(r.Path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||||
|
}
|
||||||
|
for _, prior := range spec.Routes {
|
||||||
|
if prior.Method != r.Method || prior.Path == r.Path {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if conflictPrefix(prior.Path, r.Path) {
|
||||||
|
return nil, fmt.Errorf("conflicting parameter names at a shared route prefix: %s and %s", prior.Path, r.Path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if prior, exists := shapes[shape]; exists && prior != r.Path {
|
||||||
|
return nil, fmt.Errorf("ambiguous route parameter names: %s and %s", prior, r.Path)
|
||||||
|
}
|
||||||
|
shapes[shape] = r.Path
|
||||||
|
pathKey := r.Method + " " + shape
|
||||||
|
if paths[pathKey] {
|
||||||
|
return nil, fmt.Errorf("duplicate route %s", pathKey)
|
||||||
|
}
|
||||||
|
paths[pathKey] = true
|
||||||
|
if r.Path == "/health/live" || r.Path == "/health/ready" || r.Path == "/openapi.json" {
|
||||||
|
return nil, fmt.Errorf("route %s is reserved by the generated server", r.Path)
|
||||||
|
}
|
||||||
|
if r.Method != strings.ToLower(r.Method) {
|
||||||
|
return nil, errors.New("HTTP methods must be lowercase")
|
||||||
|
}
|
||||||
|
if r.Status != 200 && r.Status != 201 && r.Status != 202 && r.Status != 204 {
|
||||||
|
return nil, errors.New("@status supports 200, 201, 202 or 204")
|
||||||
|
}
|
||||||
|
if (r.Status == 204) != (r.Response == "") {
|
||||||
|
return nil, errors.New("204 requires no response; other success statuses require a response type")
|
||||||
|
}
|
||||||
|
if r.Method == "head" && r.Response != "" {
|
||||||
|
return nil, errors.New("HEAD must not declare a response body")
|
||||||
|
}
|
||||||
|
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
|
||||||
|
if r.Request != "" {
|
||||||
|
b, ok := c.Bindings[r.Request]
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("unknown request %s", r.Request)
|
||||||
|
}
|
||||||
|
op.Parameters = b.Parameters()
|
||||||
|
op.Request = b.BodySchema()
|
||||||
|
if op.Request != nil && (r.Method == "get" || r.Method == "head") {
|
||||||
|
return nil, errors.New("GET/HEAD requests cannot declare JSON bodies; use form/path/header tags")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var response *openapi.Schema
|
||||||
|
if r.Response != "" {
|
||||||
|
b, ok := c.Bindings[r.Response]
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("unknown response %s", r.Response)
|
||||||
|
}
|
||||||
|
s, err := b.ResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
response = openapi.Envelope(s)
|
||||||
|
response.Required = append(response.Required, "data")
|
||||||
|
}
|
||||||
|
op.Responses[r.Status] = openapi.Response{Description: "Success", Schema: response}
|
||||||
|
errorSchema := &openapi.Schema{Type: "object", Required: []string{"code", "message"}, Properties: map[string]*openapi.Schema{"code": {Type: "string"}, "message": {Type: "string"}, "requestId": {Type: "string"}}}
|
||||||
|
for _, code := range []int{400, 413, 415, 500, 501} {
|
||||||
|
op.Responses[code] = openapi.Response{Description: "Error", Schema: errorSchema}
|
||||||
|
}
|
||||||
|
if err = api.Handle(discardRoutes{}, op, func(*httpx.Context) error { return nil }); err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for group := range groups {
|
||||||
|
c.Groups = append(c.Groups, group)
|
||||||
|
}
|
||||||
|
sort.Strings(c.Groups)
|
||||||
|
c.Document, err = api.JSON()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
// Show named hooks in documentation without claiming they implement a
|
||||||
|
// particular authentication scheme. Applications own that policy.
|
||||||
|
var doc map[string]any
|
||||||
|
decoder := json.NewDecoder(bytes.NewReader(c.Document))
|
||||||
|
decoder.UseNumber()
|
||||||
|
if err = decoder.Decode(&doc); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, r := range spec.Routes {
|
||||||
|
if len(r.Middleware) > 0 {
|
||||||
|
path := r.Path
|
||||||
|
for _, part := range strings.Split(r.Path, "/") {
|
||||||
|
if strings.HasPrefix(part, ":") {
|
||||||
|
path = strings.ReplaceAll(path, part, "{"+part[1:]+"}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
doc["paths"].(map[string]any)[path].(map[string]any)[r.Method].(map[string]any)["x-omni-middleware"] = r.Middleware
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c.Document, err = json.MarshalIndent(doc, "", " ")
|
||||||
|
return c, err
|
||||||
|
}
|
||||||
|
|
||||||
|
type discardRoutes struct{}
|
||||||
|
|
||||||
|
func (discardRoutes) Handle(string, string, httpx.Handler, ...httpx.Middleware) {}
|
||||||
|
func operationID(r Route) string {
|
||||||
|
return strings.ReplaceAll(r.Group, "/", "_") + "_" + upper(r.Handler)
|
||||||
|
}
|
||||||
|
func upper(s string) string { return strings.ToUpper(s[:1]) + s[1:] }
|
||||||
|
func lower(s string) string { return strings.ToLower(s[:1]) + s[1:] }
|
||||||
|
func routeShape(path string) (string, error) {
|
||||||
|
if path == "/" {
|
||||||
|
return path, nil
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(path, "/") {
|
||||||
|
return "", errors.New("route must start with /")
|
||||||
|
}
|
||||||
|
parts := strings.Split(path[1:], "/")
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for i, part := range parts {
|
||||||
|
if strings.HasPrefix(part, ":") {
|
||||||
|
name := part[1:]
|
||||||
|
if !lowerName.MatchString(name) || seen[name] {
|
||||||
|
return "", errors.New("invalid/duplicate path parameter")
|
||||||
|
}
|
||||||
|
seen[name] = true
|
||||||
|
parts[i] = ":"
|
||||||
|
} else if !staticSegment.MatchString(part) || part == "." || part == ".." {
|
||||||
|
return "", errors.New("invalid route segment")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "/" + strings.Join(parts, "/"), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func conflictPrefix(left, right string) bool {
|
||||||
|
a, b := strings.Split(left, "/"), strings.Split(right, "/")
|
||||||
|
for i := 0; i < len(a) && i < len(b); i++ {
|
||||||
|
if strings.HasPrefix(a[i], ":") && strings.HasPrefix(b[i], ":") {
|
||||||
|
if a[i] != b[i] {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if a[i] != b[i] {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
458
cmd/go-omni/apigen/generate.go
Normal file
458
cmd/go-omni/apigen/generate.go
Normal file
@@ -0,0 +1,458 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"go/format"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime/debug"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||||
|
"golang.org/x/mod/modfile"
|
||||||
|
"golang.org/x/mod/module"
|
||||||
|
"golang.org/x/mod/semver"
|
||||||
|
)
|
||||||
|
|
||||||
|
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
const minimumVersion = "v0.7.0-preview"
|
||||||
|
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||||
|
|
||||||
|
type Options struct {
|
||||||
|
Database, Storage string
|
||||||
|
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
|
||||||
|
Check bool
|
||||||
|
}
|
||||||
|
type Output struct{ Generated, Created, Preserved, Removed []string }
|
||||||
|
|
||||||
|
// Generate creates or safely updates one contract-driven project. It renders
|
||||||
|
// and validates every output before applying an ownership-checked transaction.
|
||||||
|
func Generate(o Options) (Output, error) {
|
||||||
|
if o.API == "" || o.Dir == "" {
|
||||||
|
return Output{}, errors.New("--api and --dir are required")
|
||||||
|
}
|
||||||
|
spec, err := Parse(o.API)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
compiled, err := Compile(spec)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
return generate(o, compiled, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
const starterAPI = "syntax = \"v1\"\n\ninfo (\n title: \"Greeting API\"\n version: \"1.0.0\"\n)\n\ntype GreetReq {\n Name string `path:\"name\" binding:\"required,max=80\"`\n}\ntype GreetResp {\n Message string `json:\"message\"`\n}\n\n@server (\n group: greet\n prefix: /api\n)\nservice greet-api {\n @doc \"Greet a visitor\"\n @handler greet\n get /greet/:name (GreetReq) returns (GreetResp)\n}\n"
|
||||||
|
|
||||||
|
// New creates a runnable greeting project, including its editable API file.
|
||||||
|
func New(o Options) (Output, error) {
|
||||||
|
if o.API != "" || o.Check {
|
||||||
|
return Output{}, errors.New("api new does not accept --api or --check")
|
||||||
|
}
|
||||||
|
if o.Dir == "" {
|
||||||
|
return Output{}, errors.New("project directory required")
|
||||||
|
}
|
||||||
|
if entries, err := os.ReadDir(o.Dir); err == nil && len(entries) > 0 {
|
||||||
|
return Output{}, errors.New("api new requires an empty directory")
|
||||||
|
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
tokens, err := lex("service.api", starterAPI)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
spec := &Spec{Version: "1.0.0", Entry: "service.api", Sources: map[string][]byte{"service.api": []byte(starterAPI)}}
|
||||||
|
p := parser{tokens: tokens, spec: spec}
|
||||||
|
if err = p.parse(); err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
c, err := Compile(spec)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
return generate(o, c, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||||
|
directory, err := filepath.Abs(o.Dir)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
o.Dir = directory
|
||||||
|
if err := resolveProfile(&o); err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
seeds := map[string][]byte{}
|
||||||
|
moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod"))
|
||||||
|
if readErr == nil {
|
||||||
|
mf, err := modfile.Parse("go.mod", moduleData, nil)
|
||||||
|
if err != nil || mf.Module == nil {
|
||||||
|
return Output{}, errors.New("invalid project go.mod")
|
||||||
|
}
|
||||||
|
if o.Module != "" && o.Module != mf.Module.Mod.Path {
|
||||||
|
return Output{}, errors.New("--module differs from existing go.mod")
|
||||||
|
}
|
||||||
|
o.Module = mf.Module.Mod.Path
|
||||||
|
found := false
|
||||||
|
requiredVersion := ""
|
||||||
|
for _, r := range mf.Require {
|
||||||
|
if r.Mod.Path == frameworkModule {
|
||||||
|
found = true
|
||||||
|
requiredVersion = r.Mod.Version
|
||||||
|
if o.FrameworkVersion != "" && o.FrameworkVersion != r.Mod.Version {
|
||||||
|
return Output{}, errors.New("existing framework version differs; update go.mod explicitly")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
return Output{}, errors.New("existing go.mod must require go-omni")
|
||||||
|
}
|
||||||
|
if semver.Compare(requiredVersion, minimumVersion) < 0 {
|
||||||
|
localCurrent := false
|
||||||
|
for _, r := range mf.Replace {
|
||||||
|
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||||
|
path := r.New.Path
|
||||||
|
if !filepath.IsAbs(path) {
|
||||||
|
path = filepath.Join(directory, path)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(path, "contract", "contract.go")); err == nil {
|
||||||
|
localCurrent = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !localCurrent {
|
||||||
|
return Output{}, fmt.Errorf("existing framework dependency predates API support; use %s or a current local replacement", minimumVersion)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if o.FrameworkDir != "" {
|
||||||
|
matched := false
|
||||||
|
for _, r := range mf.Replace {
|
||||||
|
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||||
|
replacement := r.New.Path
|
||||||
|
if !filepath.IsAbs(replacement) {
|
||||||
|
replacement = filepath.Join(directory, replacement)
|
||||||
|
}
|
||||||
|
matched = canonical(replacement) == canonical(o.FrameworkDir)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !matched {
|
||||||
|
return Output{}, errors.New("existing framework replacement differs; update go.mod explicitly")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if !errors.Is(readErr, os.ErrNotExist) {
|
||||||
|
return Output{}, readErr
|
||||||
|
} else {
|
||||||
|
if o.Module == "" {
|
||||||
|
o.Module = filepath.Base(directory)
|
||||||
|
}
|
||||||
|
data, err := moduleFile(o)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
seeds["go.mod"] = data
|
||||||
|
}
|
||||||
|
if err = module.CheckImportPath(o.Module); err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
generated, base, err := renderProject(o, c, starter)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
for name, data := range base {
|
||||||
|
seeds[name] = data
|
||||||
|
}
|
||||||
|
// The API source remains developer-owned. A relocated project regenerates
|
||||||
|
// from its own api/ tree, including all imports.
|
||||||
|
entry := "api/" + c.Spec.Entry
|
||||||
|
for name, data := range c.Spec.Sources {
|
||||||
|
seeds["api/"+name] = data
|
||||||
|
}
|
||||||
|
if o.API != "" {
|
||||||
|
source, _ := filepath.Abs(o.API)
|
||||||
|
sourceReal, _ := filepath.EvalSymlinks(source)
|
||||||
|
targetReal := canonical(directory)
|
||||||
|
rel, e := filepath.Rel(targetReal, sourceReal)
|
||||||
|
if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
|
entry = filepath.ToSlash(rel)
|
||||||
|
for name := range c.Spec.Sources {
|
||||||
|
delete(seeds, "api/"+name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Source collisions must never leave a copied contract different from the
|
||||||
|
// generated code. On updates, use the developer-owned API in the project.
|
||||||
|
for name, data := range seeds {
|
||||||
|
if strings.HasSuffix(name, ".api") {
|
||||||
|
current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name)))
|
||||||
|
if e == nil && string(current) != string(data) {
|
||||||
|
return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
seeds["README.md"] = []byte(projectREADME(o.Module, entry) + infrastructureREADME(o))
|
||||||
|
for _, files := range []map[string][]byte{generated, seeds} {
|
||||||
|
for name, data := range files {
|
||||||
|
if strings.HasSuffix(name, ".go") {
|
||||||
|
data, err = format.Source(data)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, fmt.Errorf("format %s: %w", name, err)
|
||||||
|
}
|
||||||
|
files[name] = data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return apply(directory, o.Module, entry, generated, seeds, o.Check, o.profile())
|
||||||
|
}
|
||||||
|
|
||||||
|
func moduleFile(o Options) ([]byte, error) {
|
||||||
|
if err := module.CheckImportPath(o.Module); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
root := o.FrameworkDir
|
||||||
|
version := o.FrameworkVersion
|
||||||
|
if root == "" && version == "" {
|
||||||
|
wd, _ := os.Getwd()
|
||||||
|
for wd != "" {
|
||||||
|
data, err := os.ReadFile(filepath.Join(wd, "go.mod"))
|
||||||
|
if err == nil {
|
||||||
|
mf, e := modfile.Parse("go.mod", data, nil)
|
||||||
|
if e == nil && mf.Module != nil && mf.Module.Mod.Path == frameworkModule {
|
||||||
|
root = wd
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
parent := filepath.Dir(wd)
|
||||||
|
if parent == wd {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
wd = parent
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if version == "" {
|
||||||
|
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
|
||||||
|
version = info.Main.Version
|
||||||
|
} else {
|
||||||
|
version = minimumVersion
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {
|
||||||
|
return nil, errors.New("invalid framework version")
|
||||||
|
}
|
||||||
|
if root == "" && semver.Compare(version, minimumVersion) < 0 {
|
||||||
|
return nil, fmt.Errorf("API generation needs %s or newer; use --framework-dir for unpublished source", minimumVersion)
|
||||||
|
}
|
||||||
|
text := "module " + o.Module + "\n\ngo 1.22.0\n\nrequire " + frameworkModule + " " + version + "\n"
|
||||||
|
if root != "" {
|
||||||
|
root = canonical(root)
|
||||||
|
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
mf, err := modfile.Parse("go.mod", data, nil)
|
||||||
|
if err != nil || mf.Module == nil || mf.Module.Mod.Path != frameworkModule {
|
||||||
|
return nil, errors.New("--framework-dir must contain go-omni")
|
||||||
|
}
|
||||||
|
if _, err = os.Stat(filepath.Join(root, "contract", "contract.go")); err != nil {
|
||||||
|
return nil, errors.New("local framework lacks API contract support")
|
||||||
|
}
|
||||||
|
relative, err := filepath.Rel(canonical(o.Dir), root)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
text += "\nreplace " + frameworkModule + " => " + strconv.Quote(filepath.ToSlash(relative)) + "\n"
|
||||||
|
}
|
||||||
|
return []byte(text), nil
|
||||||
|
}
|
||||||
|
func canonical(path string) string {
|
||||||
|
path, _ = filepath.Abs(path)
|
||||||
|
candidate := path
|
||||||
|
var tail []string
|
||||||
|
for {
|
||||||
|
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
|
||||||
|
for i := len(tail) - 1; i >= 0; i-- {
|
||||||
|
resolved = filepath.Join(resolved, tail[i])
|
||||||
|
}
|
||||||
|
return resolved
|
||||||
|
}
|
||||||
|
parent := filepath.Dir(candidate)
|
||||||
|
if parent == candidate {
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
tail = append(tail, filepath.Base(candidate))
|
||||||
|
candidate = parent
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map[string][]byte, error) {
|
||||||
|
generated, seeds := map[string][]byte{}, map[string][]byte{}
|
||||||
|
put := func(files map[string][]byte, name, content string) { files[name] = []byte(content) }
|
||||||
|
types := "package types\n\n"
|
||||||
|
for _, t := range c.Spec.Types {
|
||||||
|
types += "type " + t.Name + " struct {\n"
|
||||||
|
for _, f := range t.Fields {
|
||||||
|
types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag)
|
||||||
|
}
|
||||||
|
types += "}\n"
|
||||||
|
}
|
||||||
|
put(generated, "internal/types/typesGen.go", generatedHeader+types)
|
||||||
|
put(generated, "docs/openapi.json", string(c.Document)+"\n")
|
||||||
|
put(generated, "internal/handler/openapiGen.go", generatedHeader+fmt.Sprintf("package handler\nimport (\"net/http\"; %q)\nconst OpenAPIDocument = %q\nfunc OpenAPI(c *httpx.Context) error { return c.WriteResponse(200,http.Header{\"Content-Type\": {\"application/json; charset=utf-8\"}},[]byte(OpenAPIDocument)) }\n", frameworkModule+"/httpx", string(c.Document)+"\n"))
|
||||||
|
imports := fmt.Sprintf("import (\"fmt\"; %q; %q; %q;", frameworkModule+"/httpx", frameworkModule+"/openapi", o.Module+"/internal/servicecontext")
|
||||||
|
routes := "func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {\n if router==nil || deps==nil {return fmt.Errorf(\"router and dependencies required\")}\n"
|
||||||
|
middlewareSet := map[string]bool{}
|
||||||
|
for _, r := range c.Spec.Routes {
|
||||||
|
for _, m := range r.Middleware {
|
||||||
|
middlewareSet[m] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var middlewareNames []string
|
||||||
|
for m := range middlewareSet {
|
||||||
|
middlewareNames = append(middlewareNames, m)
|
||||||
|
}
|
||||||
|
sort.Strings(middlewareNames)
|
||||||
|
for _, m := range middlewareNames {
|
||||||
|
routes += fmt.Sprintf("if deps.Middlewares[%q]==nil {return fmt.Errorf(%q)}\n", m, "missing middleware "+m)
|
||||||
|
}
|
||||||
|
for i, group := range c.Groups {
|
||||||
|
imports += fmt.Sprintf("h%d %q; s%d %q;", i, o.Module+"/internal/handler/"+group, i, o.Module+"/internal/service/"+group)
|
||||||
|
routes += fmt.Sprintf("h%d := h%d.New(s%d.New(deps))\n", i, i, i)
|
||||||
|
groupRoutes := []Route{}
|
||||||
|
for _, r := range c.Spec.Routes {
|
||||||
|
if r.Group == group {
|
||||||
|
groupRoutes = append(groupRoutes, r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
packageName := filepath.Base(group)
|
||||||
|
text := "package " + packageName + "\nimport(\"context\"; \"fmt\";"
|
||||||
|
needsTypes, needsBinder := false, false
|
||||||
|
for _, r := range groupRoutes {
|
||||||
|
needsTypes = needsTypes || r.Request != "" || r.Response != ""
|
||||||
|
needsBinder = needsBinder || r.Request != ""
|
||||||
|
}
|
||||||
|
text += fmt.Sprintf("%q;", frameworkModule+"/httpx")
|
||||||
|
if needsTypes {
|
||||||
|
text += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||||
|
}
|
||||||
|
if needsBinder {
|
||||||
|
text += fmt.Sprintf("%q;", frameworkModule+"/contract")
|
||||||
|
}
|
||||||
|
text += ")\n"
|
||||||
|
text += "type Service interface {\n"
|
||||||
|
for _, r := range groupRoutes {
|
||||||
|
text += signature(r) + "\n"
|
||||||
|
}
|
||||||
|
text += "}\ntype Handler struct {service Service}\nfunc New(s Service)*Handler {return &Handler{service:s}}\n"
|
||||||
|
for _, r := range groupRoutes {
|
||||||
|
name := upper(r.Handler)
|
||||||
|
if r.Request != "" {
|
||||||
|
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s]()\n", lower(r.Handler), r.Request)
|
||||||
|
}
|
||||||
|
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
|
||||||
|
call := "c.Context()"
|
||||||
|
if r.Request != "" {
|
||||||
|
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req);err!=nil{return err}\n", r.Request, lower(r.Handler))
|
||||||
|
call += ", &req"
|
||||||
|
}
|
||||||
|
if r.Response != "" {
|
||||||
|
text += fmt.Sprintf("resp,err:=h.service.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name)
|
||||||
|
method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status]
|
||||||
|
text += "return c." + method + "(resp)\n}\n"
|
||||||
|
} else {
|
||||||
|
text += fmt.Sprintf("if err:=h.service.%s(%s);err!=nil{return err};return c.NoContent()\n}\n", name, call)
|
||||||
|
}
|
||||||
|
routes += fmt.Sprintf("router.Handle(%q,%q,h%d.%s", strings.ToUpper(r.Method), r.Path, i, name)
|
||||||
|
for _, m := range r.Middleware {
|
||||||
|
routes += fmt.Sprintf(",deps.Middlewares[%q]", m)
|
||||||
|
}
|
||||||
|
routes += ")\n"
|
||||||
|
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
|
||||||
|
if r.Request != "" || r.Response != "" {
|
||||||
|
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||||
|
}
|
||||||
|
service += ")\n"
|
||||||
|
service += "// " + name + " implements the business operation. This file is never overwritten.\nfunc(s *Service) " + signature(r) + " {\n"
|
||||||
|
if starter && r.Handler == "greet" {
|
||||||
|
service = strings.Replace(service, "apierrors "+strconv.Quote(frameworkModule+"/errors")+";", "\"fmt\";", 1)
|
||||||
|
service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}"
|
||||||
|
} else {
|
||||||
|
service += "// TODO: implement business behavior and use s.deps for dependencies.\n"
|
||||||
|
if r.Response != "" {
|
||||||
|
service += "return nil, "
|
||||||
|
} else {
|
||||||
|
service += "return "
|
||||||
|
}
|
||||||
|
service += "apierrors.New(501,\"NOT_IMPLEMENTED\",\"operation is not implemented\")\n}"
|
||||||
|
}
|
||||||
|
put(seeds, "internal/service/"+group+"/"+lower(r.Handler)+".go", service)
|
||||||
|
}
|
||||||
|
if !hasResponse(groupRoutes) {
|
||||||
|
text = strings.Replace(text, "\"fmt\";", "", 1)
|
||||||
|
}
|
||||||
|
put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text)
|
||||||
|
put(seeds, "internal/service/"+group+"/service.go", fmt.Sprintf("package %s\nimport %q\ntype Service struct {deps *servicecontext.Context}\nfunc New(deps *servicecontext.Context)*Service{return &Service{deps:deps}}\n", packageName, o.Module+"/internal/servicecontext"))
|
||||||
|
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
|
||||||
|
}
|
||||||
|
// Avoid unused imports when no route uses a named middleware.
|
||||||
|
imports = strings.Replace(imports, strconv.Quote(frameworkModule+"/httpx")+";", "", 1)
|
||||||
|
put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n")
|
||||||
|
registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n"
|
||||||
|
for _, m := range middlewareNames {
|
||||||
|
registry += fmt.Sprintf("%q:%s,\n", m, upper(m))
|
||||||
|
put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m)))
|
||||||
|
}
|
||||||
|
put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n")
|
||||||
|
for name, source := range baseTemplates {
|
||||||
|
content := strings.ReplaceAll(source, "MODULE", o.Module)
|
||||||
|
content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat)
|
||||||
|
content = strings.ReplaceAll(content, "LOGOUTPUT", o.LogOutput)
|
||||||
|
if name == "etc/config.yaml" && o.LogOutput == "file" {
|
||||||
|
content = strings.Replace(content, "output: file", "output: stderr\n file: ./var/log/server.log\n rotation:\n enabled: true\n maxSizeMb: 100\n maxBackups: 10\n maxAgeDays: 7\n compress: true", 1)
|
||||||
|
}
|
||||||
|
put(seeds, name, content)
|
||||||
|
}
|
||||||
|
|
||||||
|
infra, err := scaffold.APIInfrastructure(o.Module, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
for name, data := range infra {
|
||||||
|
seeds[name] = data
|
||||||
|
}
|
||||||
|
if o.Storage == "local" {
|
||||||
|
for _, r := range c.Spec.Routes {
|
||||||
|
if r.Path == "/storage/object" {
|
||||||
|
return nil, nil, errors.New("/storage/object is reserved by local storage")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
main := string(seeds["cmd/server/main.go"])
|
||||||
|
main = strings.Replace(main, " if err=handler.RegisterRoutes", ` for _,method:=range []string{"GET","HEAD","PUT"}{app.Handle(method,"/storage/object",func(c *httpx.Context)error{deps.FileHTTP.ServeHTTP(c.Raw(),c.Request());return nil})}
|
||||||
|
if err=handler.RegisterRoutes`, 1)
|
||||||
|
main = strings.Replace(main, `"git.yuntuo.hk/k2safe/go-omni/logx"`, `"git.yuntuo.hk/k2safe/go-omni/logx"; "git.yuntuo.hk/k2safe/go-omni/httpx"`, 1)
|
||||||
|
seeds["cmd/server/main.go"] = []byte(main)
|
||||||
|
}
|
||||||
|
return generated, seeds, nil
|
||||||
|
}
|
||||||
|
func hasResponse(routes []Route) bool {
|
||||||
|
for _, r := range routes {
|
||||||
|
if r.Response != "" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
func signature(r Route) string {
|
||||||
|
s := upper(r.Handler) + "(ctx context.Context"
|
||||||
|
if r.Request != "" {
|
||||||
|
s += ",req *types." + r.Request
|
||||||
|
}
|
||||||
|
s += ") "
|
||||||
|
if r.Response != "" {
|
||||||
|
s += "(*types." + r.Response + ",error)"
|
||||||
|
} else {
|
||||||
|
s += "error"
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
460
cmd/go-omni/apigen/parse.go
Normal file
460
cmd/go-omni/apigen/parse.go
Normal file
@@ -0,0 +1,460 @@
|
|||||||
|
// Package apigen implements go-omni's deliberately bounded go-zero-style API
|
||||||
|
// dialect. It generates Omni handlers, never go-zero or transport dependencies.
|
||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"text/scanner"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Spec struct {
|
||||||
|
Title, Version, Service string
|
||||||
|
Types []Type
|
||||||
|
Routes []Route
|
||||||
|
Sources map[string][]byte // relative to the entry API's directory
|
||||||
|
Entry string
|
||||||
|
}
|
||||||
|
type Type struct {
|
||||||
|
Name string
|
||||||
|
Fields []Field
|
||||||
|
Position string
|
||||||
|
}
|
||||||
|
type Field struct{ Name, Type, Tag string }
|
||||||
|
type Route struct {
|
||||||
|
Method, Path, Request, Response, Handler, Group, Summary string
|
||||||
|
Middleware []string
|
||||||
|
Status int
|
||||||
|
Position string
|
||||||
|
}
|
||||||
|
type token struct {
|
||||||
|
text, position string
|
||||||
|
kind rune
|
||||||
|
}
|
||||||
|
type parser struct {
|
||||||
|
tokens []token
|
||||||
|
at int
|
||||||
|
spec *Spec
|
||||||
|
imports []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func Parse(path string) (*Spec, error) {
|
||||||
|
entry, err := filepath.Abs(path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
entry, err = filepath.EvalSymlinks(entry)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
root := filepath.Dir(entry)
|
||||||
|
spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||||
|
visiting, visited := map[string]bool{}, map[string]bool{}
|
||||||
|
total := 0
|
||||||
|
var visit func(string, int) error
|
||||||
|
visit = func(name string, depth int) error {
|
||||||
|
if depth > 32 || len(visited)+len(visiting) >= 64 {
|
||||||
|
return errors.New("api: import limit exceeded")
|
||||||
|
}
|
||||||
|
real, err := filepath.EvalSymlinks(name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
rel, err := filepath.Rel(root, real)
|
||||||
|
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
|
return errors.New("api: imports must stay within the entry API directory")
|
||||||
|
}
|
||||||
|
if filepath.Ext(real) != ".api" {
|
||||||
|
return errors.New("api: expected .api file")
|
||||||
|
}
|
||||||
|
if visiting[real] {
|
||||||
|
return fmt.Errorf("api: circular import %s", rel)
|
||||||
|
}
|
||||||
|
if visited[real] {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
visiting[real] = true
|
||||||
|
defer delete(visiting, real)
|
||||||
|
f, err := os.Open(real)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||||
|
_ = f.Close()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
total += len(data)
|
||||||
|
if len(data) > 1<<20 || total > 8<<20 {
|
||||||
|
return errors.New("api: source size limit exceeded")
|
||||||
|
}
|
||||||
|
tokens, err := lex(rel, string(data))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p := &parser{tokens: tokens, spec: spec}
|
||||||
|
if err = p.parse(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
spec.Sources[filepath.ToSlash(rel)] = data
|
||||||
|
for _, imp := range p.imports {
|
||||||
|
if filepath.IsAbs(imp) || strings.Contains(imp, "\\") {
|
||||||
|
return errors.New("api: imports must use relative slash paths")
|
||||||
|
}
|
||||||
|
if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
visited[real] = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err = visit(entry, 0); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if spec.Service == "" || len(spec.Routes) == 0 {
|
||||||
|
return nil, errors.New("api: at least one service route required")
|
||||||
|
}
|
||||||
|
if spec.Title == "" {
|
||||||
|
spec.Title = spec.Service
|
||||||
|
}
|
||||||
|
return spec, nil
|
||||||
|
}
|
||||||
|
func lex(name, source string) ([]token, error) {
|
||||||
|
var s scanner.Scanner
|
||||||
|
s.Init(strings.NewReader(source))
|
||||||
|
s.Filename = name
|
||||||
|
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments
|
||||||
|
s.Whitespace ^= 1 << '\n'
|
||||||
|
var scanErr error
|
||||||
|
s.Error = func(s *scanner.Scanner, message string) {
|
||||||
|
if scanErr == nil {
|
||||||
|
scanErr = fmt.Errorf("%s: %s", s.Position, message)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var out []token
|
||||||
|
for {
|
||||||
|
kind := s.Scan()
|
||||||
|
out = append(out, token{s.TokenText(), s.Position.String(), kind})
|
||||||
|
if kind == scanner.EOF {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, scanErr
|
||||||
|
}
|
||||||
|
func (p *parser) peek() string { return p.tokens[p.at].text }
|
||||||
|
func (p *parser) next() token {
|
||||||
|
t := p.tokens[p.at]
|
||||||
|
if t.kind != scanner.EOF {
|
||||||
|
p.at++
|
||||||
|
}
|
||||||
|
return t
|
||||||
|
}
|
||||||
|
func (p *parser) skip() {
|
||||||
|
for p.peek() == "\n" || p.peek() == ";" {
|
||||||
|
p.next()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (p *parser) err(message string) error {
|
||||||
|
return fmt.Errorf("%s: %s (got %q)", p.tokens[p.at].position, message, p.peek())
|
||||||
|
}
|
||||||
|
func (p *parser) expect(s string) error {
|
||||||
|
if p.peek() != s {
|
||||||
|
return p.err("expected " + strconv.Quote(s))
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) ident() (string, error) {
|
||||||
|
if p.tokens[p.at].kind != scanner.Ident {
|
||||||
|
return "", p.err("expected identifier")
|
||||||
|
}
|
||||||
|
return p.next().text, nil
|
||||||
|
}
|
||||||
|
func (p *parser) quoted() (string, error) {
|
||||||
|
if p.tokens[p.at].kind != scanner.String {
|
||||||
|
return "", p.err("expected quoted string")
|
||||||
|
}
|
||||||
|
return strconv.Unquote(p.next().text)
|
||||||
|
}
|
||||||
|
func (p *parser) parse() error {
|
||||||
|
server := map[string]string{}
|
||||||
|
syntaxSeen := false
|
||||||
|
for p.skip(); p.tokens[p.at].kind != scanner.EOF; p.skip() {
|
||||||
|
switch p.peek() {
|
||||||
|
case "syntax":
|
||||||
|
if syntaxSeen {
|
||||||
|
return p.err("duplicate syntax")
|
||||||
|
}
|
||||||
|
syntaxSeen = true
|
||||||
|
p.next()
|
||||||
|
if err := p.expect("="); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
v, err := p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if v != "v1" {
|
||||||
|
return p.err("only syntax v1 supported")
|
||||||
|
}
|
||||||
|
case "info":
|
||||||
|
p.next()
|
||||||
|
m, err := p.metadata()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for k, v := range m {
|
||||||
|
switch k {
|
||||||
|
case "title":
|
||||||
|
p.spec.Title = v
|
||||||
|
case "version":
|
||||||
|
p.spec.Version = v
|
||||||
|
case "desc", "description", "author", "email":
|
||||||
|
default:
|
||||||
|
return p.err("unsupported info property " + k)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "import":
|
||||||
|
p.next()
|
||||||
|
if p.peek() == "(" {
|
||||||
|
p.next()
|
||||||
|
for p.skip(); p.peek() != ")"; p.skip() {
|
||||||
|
v, err := p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.imports = append(p.imports, v)
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
} else {
|
||||||
|
v, err := p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.imports = append(p.imports, v)
|
||||||
|
}
|
||||||
|
case "type":
|
||||||
|
p.next()
|
||||||
|
if p.peek() == "(" {
|
||||||
|
p.next()
|
||||||
|
for p.skip(); p.peek() != ")"; p.skip() {
|
||||||
|
if err := p.parseType(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
} else {
|
||||||
|
if err := p.parseType(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "@":
|
||||||
|
p.next()
|
||||||
|
if err := p.expect("server"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(server) != 0 {
|
||||||
|
return p.err("duplicate server block")
|
||||||
|
}
|
||||||
|
m, err := p.metadata()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
server = m
|
||||||
|
for k := range m {
|
||||||
|
if k != "prefix" && k != "group" && k != "middleware" {
|
||||||
|
return p.err("unsupported @server option " + k + "; wire authentication through named middleware")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "service":
|
||||||
|
if err := p.parseService(server); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
server = map[string]string{}
|
||||||
|
default:
|
||||||
|
return p.err("unsupported declaration")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(server) > 0 {
|
||||||
|
return p.err("@server must precede a service")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) metadata() (map[string]string, error) {
|
||||||
|
if err := p.expect("("); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out := map[string]string{}
|
||||||
|
for p.skip(); p.peek() != ")"; p.skip() {
|
||||||
|
key, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err = p.expect(":"); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
value := ""
|
||||||
|
if p.tokens[p.at].kind == scanner.String {
|
||||||
|
value, err = p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
for p.peek() != "\n" && p.peek() != ";" && p.peek() != ")" && p.tokens[p.at].kind != scanner.EOF {
|
||||||
|
value += p.next().text
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if value == "" {
|
||||||
|
return nil, p.err("metadata value required")
|
||||||
|
}
|
||||||
|
if _, exists := out[key]; exists {
|
||||||
|
return nil, p.err("duplicate metadata " + key)
|
||||||
|
}
|
||||||
|
out[key] = value
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
func (p *parser) parseType() error {
|
||||||
|
pos := p.tokens[p.at].position
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.skip()
|
||||||
|
if err = p.expect("{"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
typ := Type{Name: name, Position: pos}
|
||||||
|
for p.skip(); p.peek() != "}"; p.skip() {
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
expression := ""
|
||||||
|
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" && p.tokens[p.at].kind != scanner.RawString && p.tokens[p.at].kind != scanner.EOF {
|
||||||
|
expression += p.next().text
|
||||||
|
}
|
||||||
|
if expression == "" {
|
||||||
|
return p.err("field type required; embedded types are not supported")
|
||||||
|
}
|
||||||
|
if p.tokens[p.at].kind != scanner.RawString {
|
||||||
|
return p.err("field source tag required")
|
||||||
|
}
|
||||||
|
tag, err := strconv.Unquote(p.next().text)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag})
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
p.spec.Types = append(p.spec.Types, typ)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) parseService(server map[string]string) error {
|
||||||
|
p.next()
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for p.peek() == "-" {
|
||||||
|
p.next()
|
||||||
|
part, e := p.ident()
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
name += "-" + part
|
||||||
|
}
|
||||||
|
if p.spec.Service != "" && p.spec.Service != name {
|
||||||
|
return p.err("one service name per entry API is supported")
|
||||||
|
}
|
||||||
|
p.spec.Service = name
|
||||||
|
p.skip()
|
||||||
|
if err = p.expect("{"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for p.skip(); p.peek() != "}"; p.skip() {
|
||||||
|
route := Route{Group: server["group"], Status: 200, Position: p.tokens[p.at].position}
|
||||||
|
if route.Group == "" {
|
||||||
|
route.Group = "api"
|
||||||
|
}
|
||||||
|
if server["middleware"] != "" {
|
||||||
|
route.Middleware = strings.Split(server["middleware"], ",")
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for p.peek() == "@" {
|
||||||
|
p.next()
|
||||||
|
annotation, e := p.ident()
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
if seen[annotation] {
|
||||||
|
return p.err("duplicate route annotation")
|
||||||
|
}
|
||||||
|
seen[annotation] = true
|
||||||
|
switch annotation {
|
||||||
|
case "handler":
|
||||||
|
route.Handler, e = p.ident()
|
||||||
|
case "doc":
|
||||||
|
route.Summary, e = p.quoted()
|
||||||
|
case "status":
|
||||||
|
route.Status, e = strconv.Atoi(p.next().text)
|
||||||
|
default:
|
||||||
|
return p.err("unsupported route annotation " + annotation)
|
||||||
|
}
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
p.skip()
|
||||||
|
}
|
||||||
|
if route.Handler == "" {
|
||||||
|
return p.err("@handler required")
|
||||||
|
}
|
||||||
|
route.Method, err = p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
path := ""
|
||||||
|
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "(" && p.peek() != "returns" && p.peek() != "}" && p.tokens[p.at].kind != scanner.EOF {
|
||||||
|
path += p.next().text
|
||||||
|
}
|
||||||
|
route.Path = server["prefix"] + path
|
||||||
|
if p.peek() == "(" {
|
||||||
|
route.Request, err = p.routeType()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if p.peek() == "returns" {
|
||||||
|
p.next()
|
||||||
|
route.Response, err = p.routeType()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" {
|
||||||
|
return p.err("unexpected route suffix")
|
||||||
|
}
|
||||||
|
if route.Response == "" && !seen["status"] {
|
||||||
|
route.Status = 204
|
||||||
|
}
|
||||||
|
p.spec.Routes = append(p.spec.Routes, route)
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) routeType() (string, error) {
|
||||||
|
if err := p.expect("("); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return name, p.expect(")")
|
||||||
|
}
|
||||||
73
cmd/go-omni/apigen/profile.go
Normal file
73
cmd/go-omni/apigen/profile.go
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
)
|
||||||
|
|
||||||
|
type projectProfile struct {
|
||||||
|
Database string `json:"database"`
|
||||||
|
Storage string `json:"storage"`
|
||||||
|
LogOutput string `json:"logOutput"`
|
||||||
|
LogFormat string `json:"logFormat"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o Options) profile() projectProfile {
|
||||||
|
return projectProfile{o.Database, o.Storage, o.LogOutput, o.LogFormat}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Infrastructure files belong to the developer. Persist the original selection
|
||||||
|
// so a plain regeneration cannot silently change adapters or repair a missing
|
||||||
|
// file with settings from another profile.
|
||||||
|
func resolveProfile(o *Options) error {
|
||||||
|
defaults := projectProfile{"none", "none", "stderr", "json"}
|
||||||
|
data, err := os.ReadFile(filepath.Join(o.Dir, manifestPath))
|
||||||
|
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
existing := err == nil
|
||||||
|
if existing {
|
||||||
|
var previous manifest
|
||||||
|
if err = json.Unmarshal(data, &previous); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if previous.Profile != nil {
|
||||||
|
defaults = *previous.Profile
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if o.Database == "memory" {
|
||||||
|
o.Database = "none"
|
||||||
|
}
|
||||||
|
for _, item := range []struct {
|
||||||
|
name string
|
||||||
|
value *string
|
||||||
|
fallback string
|
||||||
|
}{
|
||||||
|
{"database", &o.Database, defaults.Database}, {"storage", &o.Storage, defaults.Storage},
|
||||||
|
{"log-output", &o.LogOutput, defaults.LogOutput}, {"log-format", &o.LogFormat, defaults.LogFormat},
|
||||||
|
} {
|
||||||
|
if *item.value == "" {
|
||||||
|
*item.value = item.fallback
|
||||||
|
} else if existing && *item.value != item.fallback {
|
||||||
|
return fmt.Errorf("--%s differs from the original project profile; infrastructure is developer-owned, generate a new project to compare and migrate it explicitly", item.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if o.Database != "none" && o.Database != "mysql" {
|
||||||
|
return errors.New("--database must be none, memory or mysql")
|
||||||
|
}
|
||||||
|
switch o.Storage {
|
||||||
|
case "none", "local", "s3", "r2", "oss":
|
||||||
|
default:
|
||||||
|
return errors.New("--storage must be none, local, s3, r2 or oss")
|
||||||
|
}
|
||||||
|
if o.LogOutput != "stderr" && o.LogOutput != "stdout" && o.LogOutput != "file" {
|
||||||
|
return errors.New("--log-output must be stdout, stderr or file")
|
||||||
|
}
|
||||||
|
if o.LogFormat != "json" && o.LogFormat != "text" {
|
||||||
|
return errors.New("--log-format must be json or text")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
98
cmd/go-omni/apigen/profile_test.go
Normal file
98
cmd/go-omni/apigen/profile_test.go
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestInfrastructureProfileOwnership(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "project")
|
||||||
|
o := Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: "mysql", Storage: "local", LogOutput: "file"}
|
||||||
|
if _, err := New(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
o = Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before := snapshot(t, dir)
|
||||||
|
for _, changed := range []Options{
|
||||||
|
{Database: "none"}, {Storage: "s3"}, {LogOutput: "stdout"}, {LogFormat: "text"},
|
||||||
|
} {
|
||||||
|
changed.Dir, changed.API = dir, o.API
|
||||||
|
if _, err := Generate(changed); err == nil {
|
||||||
|
t.Fatal("profile switch accepted")
|
||||||
|
}
|
||||||
|
if !sameFiles(before, snapshot(t, dir)) {
|
||||||
|
t.Fatal("profile rejection changed project")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Restoring a missing seed must use the saved profile, not current defaults.
|
||||||
|
if err := os.Remove(filepath.Join(dir, "internal/servicecontext/storage.go")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := Generate(o); err == nil {
|
||||||
|
t.Fatal("check accepted missing infrastructure")
|
||||||
|
}
|
||||||
|
o.Check = false
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !sameFiles(before, snapshot(t, dir)) {
|
||||||
|
t.Fatal("profile reconstruction changed project")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPIInfrastructureConsumers(t *testing.T) {
|
||||||
|
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
|
||||||
|
t.Skip("consumer module integration")
|
||||||
|
}
|
||||||
|
for _, db := range []string{"none", "mysql"} {
|
||||||
|
for _, storage := range []string{"none", "local", "s3", "r2", "oss"} {
|
||||||
|
t.Run(db+"/"+storage, func(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "project")
|
||||||
|
if _, err := New(Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: db, Storage: storage, LogOutput: "file"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
test := `package settings
|
||||||
|
import("testing";"os")
|
||||||
|
func TestConfiguration(t *testing.T){
|
||||||
|
t.Setenv("GO_OMNI_DB_DSN","test:test@tcp(127.0.0.1:1)/test")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY","01234567890123456789012345678901")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","test");t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","test")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_BUCKET","test");t.Setenv("GO_OMNI_STORAGE_ENDPOINT","ENDPOINT")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID","01234567890123456789012345678901")
|
||||||
|
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||||
|
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
|
||||||
|
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
|
||||||
|
_=os.Getenv("unused")
|
||||||
|
}`
|
||||||
|
endpoint := ""
|
||||||
|
if storage == "s3" {
|
||||||
|
endpoint = "127.0.0.1:9000"
|
||||||
|
}
|
||||||
|
if storage == "oss" {
|
||||||
|
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/settings/profile_test.go"), strings.ReplaceAll(test, `"ENDPOINT"`, `"`+endpoint+`"`))
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||||
|
cmd.Dir = dir
|
||||||
|
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||||
|
out, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%v\n%s", err, out)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
160
cmd/go-omni/apigen/templates.go
Normal file
160
cmd/go-omni/apigen/templates.go
Normal file
@@ -0,0 +1,160 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import "fmt"
|
||||||
|
|
||||||
|
var baseTemplates = map[string]string{
|
||||||
|
"cmd/server/main.go": `package main
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"MODULE/internal/handler"
|
||||||
|
"MODULE/internal/servicecontext"
|
||||||
|
"MODULE/internal/settings"
|
||||||
|
)
|
||||||
|
func main(){
|
||||||
|
path:=flag.String("config","etc/config.yaml","configuration file; empty uses defaults and environment")
|
||||||
|
flag.Parse()
|
||||||
|
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||||
|
}
|
||||||
|
func run(path string)(runErr error){
|
||||||
|
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
cfg,err:=settings.Load(path);if err!=nil{return err}
|
||||||
|
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||||
|
logx.SetDefault(logger)
|
||||||
|
deps,err:=servicecontext.New(ctx,logger,cfg);if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,deps.Close())}()
|
||||||
|
app:=omni.New(omni.WithConfig(cfg.Server))
|
||||||
|
app.OnShutdown(logger.Close)
|
||||||
|
app.OnShutdown(deps.Close)
|
||||||
|
if err=handler.RegisterRoutes(app,deps);err!=nil{return err}
|
||||||
|
app.GET("/openapi.json",handler.OpenAPI)
|
||||||
|
app.GET("/health/live",deps.Health.LivenessHandler())
|
||||||
|
app.GET("/health/ready",deps.Health.ReadinessHandler())
|
||||||
|
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||||
|
return app.RunContext(ctx,cfg.Server.Address)
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
".gitignore": "/server\n/var/\n.env\n.env.*\n!.env.example\n/.goOmni/api.lock\n",
|
||||||
|
}
|
||||||
|
|
||||||
|
func projectREADME(module, entry string) string {
|
||||||
|
return fmt.Sprintf(`# %s
|
||||||
|
|
||||||
|
This project is generated from an editable API contract. Business code uses
|
||||||
|
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
`+"```sh"+`
|
||||||
|
go mod tidy
|
||||||
|
go run ./cmd/server
|
||||||
|
`+"```"+`
|
||||||
|
|
||||||
|
GET /health/live and /health/ready check the running service. GET /openapi.json
|
||||||
|
serves the same document as docs/openapi.json. Bind address defaults to localhost.
|
||||||
|
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
|
||||||
|
The greeting starter works immediately; other new operation stubs return 501 until
|
||||||
|
their business logic is implemented. Authentication hooks also fail closed with 501.
|
||||||
|
|
||||||
|
## Develop
|
||||||
|
|
||||||
|
1. Edit %s and its relative imports.
|
||||||
|
2. Run the same go-omni CLI version as the framework dependency:
|
||||||
|
|
||||||
|
`+"```sh"+`
|
||||||
|
go-omni api validate --api %s
|
||||||
|
go-omni api generate --api %s --dir .
|
||||||
|
go-omni api generate --api %s --dir . --check
|
||||||
|
go test ./...
|
||||||
|
`+"```"+`
|
||||||
|
|
||||||
|
The --check command is read-only and suitable for CI. It fails on stale or missing
|
||||||
|
generated files and missing business scaffolds. go.mod, configuration, main,
|
||||||
|
Service, Repository, servicecontext, middleware hook bodies and API files are
|
||||||
|
created once and then preserved byte for byte. Gen.go files and OpenAPI are owned
|
||||||
|
by the generator; manual edits cause a conflict instead of being overwritten.
|
||||||
|
Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it.
|
||||||
|
Avoid editing files while generation is running; api.lock serializes generators.
|
||||||
|
|
||||||
|
Routes and types that disappear are removed from generated output. Handwritten
|
||||||
|
business files remain: remove/adapt obsolete implementations explicitly if an
|
||||||
|
API change makes their old types or signatures fail to compile. Every regeneration
|
||||||
|
must be followed by go test ./... . Do not delete service files to change contracts.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
`+"```text"+`
|
||||||
|
api/ # editable contract and imported types
|
||||||
|
cmd/server/main.go # startup, lifecycle and route registration
|
||||||
|
etc/config.yaml # lowerCamelCase configuration
|
||||||
|
internal/settings/ # defaults -> file -> environment
|
||||||
|
internal/servicecontext/ # dependency initialization and cleanup
|
||||||
|
internal/types/typesGen.go # generated request/response types
|
||||||
|
internal/handler/<group>/ # generated binding and service adapters
|
||||||
|
internal/service/<group>/ # developer-owned business operations
|
||||||
|
internal/repository/<group>/ # developer-owned persistence contracts
|
||||||
|
internal/middleware/ # named hooks and generated registry
|
||||||
|
docs/openapi.json # generated OpenAPI 3.0.3
|
||||||
|
.goOmni/apiManifest.json # ownership and content hashes
|
||||||
|
`+"```"+`
|
||||||
|
|
||||||
|
The API response describes data only. Omni adds code, message and requestId.
|
||||||
|
Do not define the envelope again in each response. Handlers reject nil successful
|
||||||
|
responses. Body DTOs use application/json and have a 1 MiB limit by default.
|
||||||
|
Use contract.Binder.Bind's optional limit in a custom adapter for larger payloads.
|
||||||
|
|
||||||
|
Fields require exactly one source tag: json, path, form (URL query only), header.
|
||||||
|
All are required by presence unless optional, omitempty or default is specified.
|
||||||
|
Zero and false are valid present values; binding:"required" rejects scalar zero.
|
||||||
|
Use *T with optional when business logic needs to distinguish missing from zero.
|
||||||
|
Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON
|
||||||
|
fields and duplicate scalar query/header values are rejected. Arrays in the query
|
||||||
|
use repeated keys. Enum/range/default and nested fields share the documentation
|
||||||
|
and binding compiler. Responses are expected to honor their declared constraints;
|
||||||
|
they are not validated on every response. Do not return nil slices/maps for a
|
||||||
|
non-nullable collection (initialize empty collections instead).
|
||||||
|
|
||||||
|
Named middleware is composed in declaration order (first is outermost). Implement
|
||||||
|
the generated hooks or replace entries in servicecontext; never bypass an auth
|
||||||
|
hook merely to make the server start. Middleware names do not imply a JWT policy.
|
||||||
|
|
||||||
|
This CLI supports a documented go-zero-style subset, not all goctl features.
|
||||||
|
Database tables and repositories cannot be inferred from an HTTP contract. Add
|
||||||
|
business repositories through servicecontext. Database and storage options are
|
||||||
|
composed at creation; the original profile is retained on regeneration.
|
||||||
|
`, module, entry, entry, entry, entry)
|
||||||
|
}
|
||||||
|
|
||||||
|
func infrastructureREADME(o Options) string {
|
||||||
|
return fmt.Sprintf(`
|
||||||
|
## Infrastructure
|
||||||
|
|
||||||
|
Selected database: %s; storage: %s; logging: %s/%s.
|
||||||
|
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
|
||||||
|
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||||
|
No tables are invented or migrated automatically from request/response DTOs.
|
||||||
|
The none/memory selection opens no database; implement repositories per business.
|
||||||
|
|
||||||
|
With storage enabled, services receive deps.Files, Uploads, Downloads and
|
||||||
|
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
|
||||||
|
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
|
||||||
|
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
|
||||||
|
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
|
||||||
|
and authorize before issuing them. No public upload-link route is added.
|
||||||
|
Storage failures during startup close an already-open database; Close is idempotent.
|
||||||
|
|
||||||
|
Database/storage/logging selection is recorded in .goOmni/apiManifest.json.
|
||||||
|
Changing selection through generation flags is rejected on an existing project;
|
||||||
|
edit your owned configuration and initialization explicitly, or generate a fresh
|
||||||
|
project for comparison. New routes still regenerate without repeating these flags.
|
||||||
|
`, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||||
|
}
|
||||||
295
cmd/go-omni/apigen/write.go
Normal file
295
cmd/go-omni/apigen/write.go
Normal file
@@ -0,0 +1,295 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
const manifestPath = ".goOmni/apiManifest.json"
|
||||||
|
|
||||||
|
type manifest struct {
|
||||||
|
Profile *projectProfile `json:"profile,omitempty"`
|
||||||
|
Version int `json:"version"`
|
||||||
|
Module string `json:"module"`
|
||||||
|
Entry string `json:"entry"`
|
||||||
|
Generated map[string]string `json:"generated"`
|
||||||
|
}
|
||||||
|
type change struct {
|
||||||
|
name string
|
||||||
|
before, after []byte
|
||||||
|
existed, remove bool
|
||||||
|
mode os.FileMode
|
||||||
|
}
|
||||||
|
|
||||||
|
func apply(root, module, entry string, generated, seeds map[string][]byte, check bool, profile projectProfile) (Output, error) {
|
||||||
|
var result Output
|
||||||
|
if info, err := os.Lstat(root); err == nil {
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() {
|
||||||
|
return result, errors.New("api: output root must be a real directory")
|
||||||
|
}
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
root = canonical(root)
|
||||||
|
names := map[string]bool{}
|
||||||
|
caseNames := map[string]string{}
|
||||||
|
for _, files := range []map[string][]byte{generated, seeds} {
|
||||||
|
for name := range files {
|
||||||
|
if names[name] {
|
||||||
|
return result, fmt.Errorf("duplicate output %s", name)
|
||||||
|
}
|
||||||
|
names[name] = true
|
||||||
|
folded := strings.ToLower(name)
|
||||||
|
if old, ok := caseNames[folded]; ok {
|
||||||
|
return result, fmt.Errorf("case-insensitive path collision: %s and %s", old, name)
|
||||||
|
}
|
||||||
|
caseNames[folded] = name
|
||||||
|
if err := safePath(root, name); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := safePath(root, manifestPath); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
if !check {
|
||||||
|
if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
lock := filepath.Join(root, ".goOmni", "api.lock")
|
||||||
|
f, err := os.OpenFile(lock, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600)
|
||||||
|
if err != nil {
|
||||||
|
return result, fmt.Errorf("api: acquire generation lock (remove stale api.lock only after checking no generator is running): %w", err)
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(f, "pid=%d\n", os.Getpid())
|
||||||
|
_ = f.Close()
|
||||||
|
defer os.Remove(lock)
|
||||||
|
}
|
||||||
|
old := manifest{Generated: map[string]string{}}
|
||||||
|
oldData, err := os.ReadFile(filepath.Join(root, manifestPath))
|
||||||
|
if err == nil {
|
||||||
|
if err = json.Unmarshal(oldData, &old); err != nil {
|
||||||
|
return result, fmt.Errorf("invalid ownership manifest: %w", err)
|
||||||
|
}
|
||||||
|
if old.Profile != nil && *old.Profile != profile {
|
||||||
|
return result, errors.New("api: profile changed during generation; retry")
|
||||||
|
}
|
||||||
|
if old.Version != 1 || old.Module != module || old.Entry != entry {
|
||||||
|
return result, errors.New("api: manifest version/module/entry differs; one API entry owns this project")
|
||||||
|
}
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
if oldData == nil {
|
||||||
|
if err = filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
|
||||||
|
if errors.Is(walkErr, os.ErrNotExist) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if walkErr != nil {
|
||||||
|
return walkErr
|
||||||
|
}
|
||||||
|
if d.IsDir() && d.Name() == ".git" {
|
||||||
|
return filepath.SkipDir
|
||||||
|
}
|
||||||
|
if !d.IsDir() && strings.HasSuffix(d.Name(), ".go") {
|
||||||
|
return fmt.Errorf("api: %s belongs to an existing Go layout; generate into a new directory", path)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
next := manifest{Profile: &profile, Version: 1, Module: module, Entry: entry, Generated: map[string]string{}}
|
||||||
|
for name, data := range generated {
|
||||||
|
next.Generated[name] = digest(data)
|
||||||
|
}
|
||||||
|
var changes []change
|
||||||
|
// Check ALL previous outputs, even stale ones, before writing any new file.
|
||||||
|
for name, hash := range old.Generated {
|
||||||
|
if !ownedPath(name) {
|
||||||
|
return result, fmt.Errorf("api: invalid owned path %q", name)
|
||||||
|
}
|
||||||
|
if err = safePath(root, name); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
data, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(name)))
|
||||||
|
if errors.Is(e, os.ErrNotExist) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if e != nil {
|
||||||
|
return result, e
|
||||||
|
}
|
||||||
|
if digest(data) != hash {
|
||||||
|
return result, fmt.Errorf("api: generated file %s was edited; move custom code into developer-owned files before regenerating", name)
|
||||||
|
}
|
||||||
|
if _, exists := generated[name]; !exists {
|
||||||
|
info, _ := os.Stat(filepath.Join(root, filepath.FromSlash(name)))
|
||||||
|
changes = append(changes, change{name: name, before: data, existed: true, remove: true, mode: info.Mode().Perm()})
|
||||||
|
result.Removed = append(result.Removed, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
paths := make([]string, 0, len(names))
|
||||||
|
for name := range names {
|
||||||
|
paths = append(paths, name)
|
||||||
|
}
|
||||||
|
sort.Strings(paths)
|
||||||
|
for _, name := range paths {
|
||||||
|
target := filepath.Join(root, filepath.FromSlash(name))
|
||||||
|
current, err := os.ReadFile(target)
|
||||||
|
exists := err == nil
|
||||||
|
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
content, isGenerated := generated[name]
|
||||||
|
if !isGenerated {
|
||||||
|
content = seeds[name]
|
||||||
|
}
|
||||||
|
if exists && !isGenerated {
|
||||||
|
result.Preserved = append(result.Preserved, name)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if exists && isGenerated {
|
||||||
|
if _, owned := old.Generated[name]; !owned {
|
||||||
|
return result, fmt.Errorf("api: refusing to adopt existing unowned file %s", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if isGenerated {
|
||||||
|
result.Generated = append(result.Generated, name)
|
||||||
|
} else {
|
||||||
|
result.Created = append(result.Created, name)
|
||||||
|
}
|
||||||
|
if exists && bytes.Equal(current, content) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
mode := os.FileMode(0644)
|
||||||
|
if exists {
|
||||||
|
info, e := os.Stat(target)
|
||||||
|
if e != nil {
|
||||||
|
return result, e
|
||||||
|
}
|
||||||
|
mode = info.Mode().Perm()
|
||||||
|
}
|
||||||
|
changes = append(changes, change{name: name, before: current, after: content, existed: exists, mode: mode})
|
||||||
|
}
|
||||||
|
data, err := json.MarshalIndent(next, "", " ")
|
||||||
|
if err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
data = append(data, '\n')
|
||||||
|
if !bytes.Equal(oldData, data) {
|
||||||
|
changes = append(changes, change{name: manifestPath, before: oldData, after: data, existed: oldData != nil, mode: 0644})
|
||||||
|
}
|
||||||
|
sort.Strings(result.Removed)
|
||||||
|
if check {
|
||||||
|
if len(changes) > 0 {
|
||||||
|
var stale []string
|
||||||
|
for _, c := range changes {
|
||||||
|
stale = append(stale, c.name)
|
||||||
|
}
|
||||||
|
sort.Strings(stale)
|
||||||
|
return result, fmt.Errorf("api: generated project is stale: %s", strings.Join(stale, ", "))
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
// A rollback restores bytes/modes if an ordinary filesystem write fails.
|
||||||
|
// File replacement is atomic; the whole directory is not crash-atomic.
|
||||||
|
var applied []change
|
||||||
|
for _, c := range changes {
|
||||||
|
if err = safePath(root, c.name); err == nil {
|
||||||
|
current, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||||
|
if c.existed {
|
||||||
|
if e != nil || !bytes.Equal(current, c.before) {
|
||||||
|
err = fmt.Errorf("file changed during generation: %s", c.name)
|
||||||
|
}
|
||||||
|
} else if !errors.Is(e, os.ErrNotExist) {
|
||||||
|
err = fmt.Errorf("file appeared during generation: %s", c.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
if c.remove {
|
||||||
|
err = os.Remove(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||||
|
} else {
|
||||||
|
err = atomicWrite(root, c.name, c.after, c.mode)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
for i := len(applied) - 1; i >= 0; i-- {
|
||||||
|
previous := applied[i]
|
||||||
|
var rollback error
|
||||||
|
if previous.existed {
|
||||||
|
rollback = atomicWrite(root, previous.name, previous.before, previous.mode)
|
||||||
|
} else {
|
||||||
|
rollback = os.Remove(filepath.Join(root, filepath.FromSlash(previous.name)))
|
||||||
|
}
|
||||||
|
err = errors.Join(err, rollback)
|
||||||
|
}
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
applied = append(applied, c)
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
|
||||||
|
func ownedPath(name string) bool {
|
||||||
|
if name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return strings.HasPrefix(name, "internal/handler/") && strings.HasSuffix(name, "/handlersGen.go")
|
||||||
|
}
|
||||||
|
func safePath(root, name string) error {
|
||||||
|
if filepath.IsAbs(name) || strings.Contains(name, "\\") || filepath.ToSlash(filepath.Clean(name)) != name || name == "." || strings.HasPrefix(name, "../") {
|
||||||
|
return fmt.Errorf("api: unsafe output path %q", name)
|
||||||
|
}
|
||||||
|
path := root
|
||||||
|
parts := strings.Split(name, "/")
|
||||||
|
for i, part := range parts {
|
||||||
|
path = filepath.Join(path, part)
|
||||||
|
info, err := os.Lstat(path)
|
||||||
|
if errors.Is(err, os.ErrNotExist) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return fmt.Errorf("api: refusing symlink %s", path)
|
||||||
|
}
|
||||||
|
if i < len(parts)-1 && !info.IsDir() {
|
||||||
|
return fmt.Errorf("api: expected directory %s", path)
|
||||||
|
}
|
||||||
|
if i == len(parts)-1 && !info.Mode().IsRegular() {
|
||||||
|
return fmt.Errorf("api: expected regular file %s", path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func atomicWrite(root, name string, data []byte, mode os.FileMode) error {
|
||||||
|
target := filepath.Join(root, filepath.FromSlash(name))
|
||||||
|
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
f, err := os.CreateTemp(filepath.Dir(target), ".apiGen-*")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer os.Remove(f.Name())
|
||||||
|
if err = f.Chmod(mode); err == nil {
|
||||||
|
_, err = f.Write(data)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
err = f.Sync()
|
||||||
|
}
|
||||||
|
err = errors.Join(err, f.Close())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.Rename(f.Name(), target)
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -17,6 +18,11 @@ func main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
switch os.Args[1] {
|
switch os.Args[1] {
|
||||||
|
case "api":
|
||||||
|
if err := runAPI(os.Args[2:]); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, "go-omni api:", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
case "migrate":
|
case "migrate":
|
||||||
if err := runMigrate(os.Args[2:]); err != nil {
|
if err := runMigrate(os.Args[2:]); err != nil {
|
||||||
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
|
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
|
||||||
@@ -44,9 +50,43 @@ func main() {
|
|||||||
func runNew(args []string) error {
|
func runNew(args []string) error {
|
||||||
options := scaffold.Options{}
|
options := scaffold.Options{}
|
||||||
var target string
|
var target string
|
||||||
|
var apiFile string
|
||||||
for index := 0; index < len(args); index++ {
|
for index := 0; index < len(args); index++ {
|
||||||
arg := args[index]
|
arg := args[index]
|
||||||
switch {
|
switch {
|
||||||
|
case arg == "--api" || strings.HasPrefix(arg, "--api="):
|
||||||
|
_, value, inline := strings.Cut(arg, "=")
|
||||||
|
if !inline {
|
||||||
|
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "-") {
|
||||||
|
return errors.New("--api requires a file")
|
||||||
|
}
|
||||||
|
index++
|
||||||
|
value = args[index]
|
||||||
|
}
|
||||||
|
if value == "" {
|
||||||
|
return errors.New("--api requires a file")
|
||||||
|
}
|
||||||
|
apiFile = value
|
||||||
|
case arg == "--log-output" || arg == "--log-format" || arg == "--storage" || strings.HasPrefix(arg, "--log-output=") || strings.HasPrefix(arg, "--log-format=") || strings.HasPrefix(arg, "--storage="):
|
||||||
|
name, value, hasValue := strings.Cut(arg, "=")
|
||||||
|
if !hasValue {
|
||||||
|
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "--") {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
index++
|
||||||
|
value = args[index]
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(value) == "" {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
switch name {
|
||||||
|
case "--log-output":
|
||||||
|
options.LogOutput = value
|
||||||
|
case "--log-format":
|
||||||
|
options.LogFormat = value
|
||||||
|
case "--storage":
|
||||||
|
options.Storage = value
|
||||||
|
}
|
||||||
case arg == "--force":
|
case arg == "--force":
|
||||||
options.Force = true
|
options.Force = true
|
||||||
case arg == "--database":
|
case arg == "--database":
|
||||||
@@ -98,10 +138,26 @@ func runNew(args []string) error {
|
|||||||
if target == "" {
|
if target == "" {
|
||||||
return errors.New("project directory is required")
|
return errors.New("project directory is required")
|
||||||
}
|
}
|
||||||
if options.Module == "" {
|
if options.Module == "" && apiFile == "" {
|
||||||
options.Module = filepath.Base(filepath.Clean(target))
|
options.Module = filepath.Base(filepath.Clean(target))
|
||||||
}
|
}
|
||||||
options.Dir = target
|
options.Dir = target
|
||||||
|
if apiFile != "" {
|
||||||
|
if options.Force {
|
||||||
|
return errors.New("--api protects existing business code and does not accept --force")
|
||||||
|
}
|
||||||
|
_, err := apigen.Generate(apigen.Options{API: apiFile, Dir: target, Module: options.Module, FrameworkDir: options.FrameworkDir, FrameworkVersion: options.FrameworkVersion, LogOutput: options.LogOutput, LogFormat: options.LogFormat, Database: options.Database, Storage: options.Storage})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Printf("created API project %s; implement internal/service operations\n", filepath.Clean(target))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(filepath.Join(target, ".goOmni", "apiManifest.json")); err == nil {
|
||||||
|
return errors.New("this is an API project; use api generate instead of the CRUD/module starter")
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := scaffold.Generate(options); err != nil {
|
if err := scaffold.Generate(options); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -115,10 +171,17 @@ func usage() {
|
|||||||
fmt.Fprintln(os.Stderr, "Usage:")
|
fmt.Fprintln(os.Stderr, "Usage:")
|
||||||
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
||||||
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni api new [--module <module>] <directory> Create a runnable API project")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni api generate --api <file.api> --dir <project> [--check]")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni api validate --api <file.api> Validate definitions and OpenAPI")
|
||||||
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
||||||
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
||||||
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
||||||
fmt.Fprintln(os.Stderr, " --database <memory|mysql> Select generated storage (default memory)")
|
fmt.Fprintln(os.Stderr, " --database <none|memory|mysql> API defaults to none; CRUD defaults to memory")
|
||||||
|
fmt.Fprintln(os.Stderr, " --log-output <stdout|stderr|file> Generate configured logging (default stderr)")
|
||||||
|
fmt.Fprintln(os.Stderr, " --log-format <json|text> Select structured log format (default json)")
|
||||||
|
fmt.Fprintln(os.Stderr, " --storage <none|local|s3|r2|oss> Select optional storage adapter (default none)")
|
||||||
}
|
}
|
||||||
|
|
||||||
func runAdd(args []string) error {
|
func runAdd(args []string) error {
|
||||||
|
|||||||
@@ -73,3 +73,28 @@ func TestNewMySQLDatabaseOptions(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestModuleOptionCLI(t *testing.T) {
|
||||||
|
for _, flags := range [][]string{
|
||||||
|
{"--log-output", "file", "--log-format", "json", "--storage", "local"},
|
||||||
|
{"--log-output=stdout", "--log-format=text", "--storage=none"},
|
||||||
|
} {
|
||||||
|
project := filepath.Join(t.TempDir(), "demo")
|
||||||
|
args := append([]string{"--module", "example.com/demo"}, flags...)
|
||||||
|
if err := runNew(append(args, project)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(project, "etc/config.yaml")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, args := range [][]string{
|
||||||
|
{"--storage"}, {"--storage="}, {"--storage", "--force"},
|
||||||
|
{"--log-output"}, {"--log-output="}, {"--log-format="},
|
||||||
|
{"--storage", "unknown", "demo"}, {"--log-format", "xml", "demo"},
|
||||||
|
} {
|
||||||
|
if err := runNew(args); err == nil {
|
||||||
|
t.Fatal("invalid module option accepted", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
26
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
26
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
package scaffold
|
||||||
|
|
||||||
|
import "fmt"
|
||||||
|
|
||||||
|
// APIInfrastructure renders shared infrastructure without inventing business
|
||||||
|
// tables or routes. The API generator applies these as developer-owned seeds.
|
||||||
|
func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
|
||||||
|
data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true}
|
||||||
|
files := map[string]string{
|
||||||
|
"internal/settings/config.go": "templates/modules-settings.go.tmpl",
|
||||||
|
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
|
||||||
|
"internal/servicecontext/context.go": "templates/api-context.go.tmpl",
|
||||||
|
}
|
||||||
|
if storage != "none" {
|
||||||
|
files["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||||
|
}
|
||||||
|
result := make(map[string][]byte, len(files))
|
||||||
|
for name, source := range files {
|
||||||
|
content, err := render(source, data)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("render %s: %w", name, err)
|
||||||
|
}
|
||||||
|
result[name] = []byte(content)
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
package scaffold
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type moduleProfile struct{ database, storage, output, format string }
|
||||||
|
|
||||||
|
var moduleProfiles = []moduleProfile{
|
||||||
|
{"memory", "none", "stdout", "json"},
|
||||||
|
{"memory", "local", "file", "json"},
|
||||||
|
{"memory", "s3", "stdout", "text"},
|
||||||
|
{"memory", "r2", "stderr", "json"},
|
||||||
|
{"memory", "oss", "stderr", "text"},
|
||||||
|
{"mysql", "none", "file", "text"},
|
||||||
|
{"mysql", "local", "stdout", "json"},
|
||||||
|
{"mysql", "s3", "file", "text"},
|
||||||
|
{"mysql", "r2", "stderr", "text"},
|
||||||
|
{"mysql", "oss", "file", "json"},
|
||||||
|
}
|
||||||
|
|
||||||
|
func generateProfile(t *testing.T, p moduleProfile) string {
|
||||||
|
t.Helper()
|
||||||
|
project := filepath.Join(t.TempDir(), "project")
|
||||||
|
if err := Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: p.database, Storage: p.storage, LogOutput: p.output, LogFormat: p.format}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return project
|
||||||
|
}
|
||||||
|
func TestGenerateModuleProfiles(t *testing.T) {
|
||||||
|
for _, p := range moduleProfiles {
|
||||||
|
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||||
|
project := generateProfile(t, p)
|
||||||
|
for _, name := range []string{"etc/config.yaml", "internal/settings/config.go", "MODULES.md", ".gitignore"} {
|
||||||
|
if _, err := os.Stat(filepath.Join(project, name)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, err := os.Stat(filepath.Join(project, "internal/handler/files.go"))
|
||||||
|
if p.storage == "none" && !os.IsNotExist(err) {
|
||||||
|
t.Fatal("unselected storage was emitted")
|
||||||
|
}
|
||||||
|
if p.storage != "none" && err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
module, err := os.ReadFile(filepath.Join(project, "go.mod"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
pieces := strings.Split(string(module), "=> ")
|
||||||
|
if len(pieces) != 2 {
|
||||||
|
t.Fatal("local replace missing")
|
||||||
|
}
|
||||||
|
relative, err := strconv.Unquote(strings.TrimSpace(pieces[1]))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = os.Stat(filepath.Join(project, relative, "go.mod")); err != nil {
|
||||||
|
t.Fatal("replacement does not resolve", err)
|
||||||
|
}
|
||||||
|
cfg, _ := os.ReadFile(filepath.Join(project, "etc/config.yaml"))
|
||||||
|
if strings.Contains(string(cfg), "secretKey:") || strings.Contains(string(cfg), "filesToken:") {
|
||||||
|
t.Fatal("secret defaults emitted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestModuleOptionsRejectBeforeWriting(t *testing.T) {
|
||||||
|
for _, options := range []Options{
|
||||||
|
{LogOutput: "invalid"}, {LogFormat: "xml"}, {Storage: "ftp"},
|
||||||
|
{Storage: "local", FrameworkVersion: "v0.6.0-preview"},
|
||||||
|
{Module: "example.com/x\nrequire malicious v1.0.0"},
|
||||||
|
} {
|
||||||
|
options.Dir = filepath.Join(t.TempDir(), "project")
|
||||||
|
if options.Module == "" {
|
||||||
|
options.Module = "example.com/profile"
|
||||||
|
}
|
||||||
|
if err := Generate(options); err == nil {
|
||||||
|
t.Fatalf("invalid options accepted: %+v", options)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(options.Dir); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("invalid options touched target")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestCanonicalPathBeforeDirectoryExists(t *testing.T) {
|
||||||
|
parent := t.TempDir()
|
||||||
|
alias := filepath.Join(t.TempDir(), "alias")
|
||||||
|
if err := os.Symlink(parent, alias); err != nil {
|
||||||
|
t.Skip(err)
|
||||||
|
}
|
||||||
|
path := filepath.Join(alias, "new", "nested")
|
||||||
|
if got, want := canonicalPath(path), filepath.Join(canonicalPath(parent), "new", "nested"); got != want {
|
||||||
|
t.Fatalf("%s != %s", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Compile consuming modules, then run their generated tests. No provider account
|
||||||
|
// is used: cloud constructors are checked offline, file workflows use Local.
|
||||||
|
func TestModuleProfilesIntegration(t *testing.T) {
|
||||||
|
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||||
|
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-profile tests")
|
||||||
|
}
|
||||||
|
for _, p := range moduleProfiles {
|
||||||
|
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||||
|
project := generateProfile(t, p)
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-p=2", "-count=1", "-timeout=60s", "./...")
|
||||||
|
cmd.Dir = project
|
||||||
|
cmd.Env = append(os.Environ(), "GOWORK=off", "GOMAXPROCS=2", "GO_OMNI_TEST_MYSQL_DSN=")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
t.Fatalf("generated %s/%s: %v\n%s", p.database, p.storage, err, output)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestForceRejectsRemovedDependenciesWithoutTouchingCode(t *testing.T) {
|
||||||
|
project := generateProfile(t, moduleProfile{"mysql", "local", "file", "json"})
|
||||||
|
path := filepath.Join(project, "cmd/server/main.go")
|
||||||
|
before, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Storage: "none"}); err == nil {
|
||||||
|
t.Fatal("incompatible force overwrite accepted")
|
||||||
|
}
|
||||||
|
after, _ := os.ReadFile(path)
|
||||||
|
if string(after) != string(before) {
|
||||||
|
t.Fatal("failed profile switch changed code")
|
||||||
|
}
|
||||||
|
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Database: "mysql", Storage: "local", LogOutput: "stdout"}); err != nil {
|
||||||
|
t.Fatal("compatible force overwrite rejected", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -12,6 +12,9 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"text/template"
|
"text/template"
|
||||||
|
|
||||||
|
"golang.org/x/mod/module"
|
||||||
|
"golang.org/x/mod/semver"
|
||||||
)
|
)
|
||||||
|
|
||||||
//go:embed templates moduleTemplates
|
//go:embed templates moduleTemplates
|
||||||
@@ -24,6 +27,9 @@ type Options struct {
|
|||||||
FrameworkDir string
|
FrameworkDir string
|
||||||
FrameworkVersion string
|
FrameworkVersion string
|
||||||
Database string
|
Database string
|
||||||
|
LogOutput string
|
||||||
|
LogFormat string
|
||||||
|
Storage string
|
||||||
Force bool
|
Force bool
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -48,6 +54,31 @@ func Generate(options Options) error {
|
|||||||
if strings.TrimSpace(options.Module) == "" {
|
if strings.TrimSpace(options.Module) == "" {
|
||||||
return errors.New("module path is required")
|
return errors.New("module path is required")
|
||||||
}
|
}
|
||||||
|
if err := module.CheckImportPath(options.Module); err != nil {
|
||||||
|
return fmt.Errorf("invalid module path: %w", err)
|
||||||
|
}
|
||||||
|
modules := options.LogOutput != "" || options.LogFormat != "" || options.Storage != ""
|
||||||
|
logOutput, logFormat, storage := options.LogOutput, options.LogFormat, options.Storage
|
||||||
|
if logOutput == "" {
|
||||||
|
logOutput = "stderr"
|
||||||
|
}
|
||||||
|
if logFormat == "" {
|
||||||
|
logFormat = "json"
|
||||||
|
}
|
||||||
|
if storage == "" {
|
||||||
|
storage = "none"
|
||||||
|
}
|
||||||
|
if logOutput != "stdout" && logOutput != "stderr" && logOutput != "file" {
|
||||||
|
return fmt.Errorf("unsupported log output %q: choose stdout, stderr or file", logOutput)
|
||||||
|
}
|
||||||
|
if logFormat != "json" && logFormat != "text" {
|
||||||
|
return fmt.Errorf("unsupported log format %q: choose json or text", logFormat)
|
||||||
|
}
|
||||||
|
switch storage {
|
||||||
|
case "none", "local", "s3", "r2", "oss":
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("unsupported object storage %q: choose none, local, s3, r2 or oss", storage)
|
||||||
|
}
|
||||||
database := strings.ToLower(strings.TrimSpace(options.Database))
|
database := strings.ToLower(strings.TrimSpace(options.Database))
|
||||||
if database == "" {
|
if database == "" {
|
||||||
database = "memory"
|
database = "memory"
|
||||||
@@ -58,6 +89,9 @@ func Generate(options Options) error {
|
|||||||
version := strings.TrimSpace(options.FrameworkVersion)
|
version := strings.TrimSpace(options.FrameworkVersion)
|
||||||
if version == "" {
|
if version == "" {
|
||||||
version = currentFrameworkVersion()
|
version = currentFrameworkVersion()
|
||||||
|
if modules && version == defaultFrameworkVersion {
|
||||||
|
version = modulesFrameworkVersion
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if !validFrameworkVersion(version) {
|
if !validFrameworkVersion(version) {
|
||||||
return fmt.Errorf("framework version %q must be a complete v0 or v1 semantic version", version)
|
return fmt.Errorf("framework version %q must be a complete v0 or v1 semantic version", version)
|
||||||
@@ -70,20 +104,20 @@ func Generate(options Options) error {
|
|||||||
if err := checkTarget(directory, options.Force); err != nil {
|
if err := checkTarget(directory, options.Force); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
|
||||||
return fmt.Errorf("create project directory: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// An explicit remote version opts out of automatic local development mode.
|
// An explicit remote version opts out of automatic local development mode.
|
||||||
replace := ""
|
replace := ""
|
||||||
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
|
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
|
||||||
replace = frameworkReplace(directory, options.FrameworkDir)
|
replace = frameworkReplace(directory, options.FrameworkDir)
|
||||||
}
|
}
|
||||||
|
if modules && replace == "" && semver.Compare(version, modulesFrameworkVersion) < 0 && version != installedFrameworkVersion() {
|
||||||
|
return fmt.Errorf("module templates require %s or newer; use --framework-dir for unpublished source", modulesFrameworkVersion)
|
||||||
|
}
|
||||||
data := templateData{
|
data := templateData{
|
||||||
Module: options.Module,
|
Module: options.Module,
|
||||||
FrameworkVersion: version,
|
FrameworkVersion: version,
|
||||||
FrameworkReplace: replace,
|
FrameworkReplace: replace,
|
||||||
Database: database,
|
Database: database,
|
||||||
|
Modules: modules, LogOutput: logOutput, LogFormat: logFormat, Storage: storage,
|
||||||
}
|
}
|
||||||
projectFiles := make(map[string]string, len(files)+3)
|
projectFiles := make(map[string]string, len(files)+3)
|
||||||
for path, source := range files {
|
for path, source := range files {
|
||||||
@@ -94,11 +128,42 @@ func Generate(options Options) error {
|
|||||||
projectFiles["schema.sql"] = "templates/schema.sql.tmpl"
|
projectFiles["schema.sql"] = "templates/schema.sql.tmpl"
|
||||||
projectFiles["internal/servicecontext/databaseIntegration_test.go"] = "templates/databaseIntegration_test.go.tmpl"
|
projectFiles["internal/servicecontext/databaseIntegration_test.go"] = "templates/databaseIntegration_test.go.tmpl"
|
||||||
}
|
}
|
||||||
|
if modules {
|
||||||
|
projectFiles["cmd/server/main.go"] = "templates/modules-main.go.tmpl"
|
||||||
|
projectFiles["internal/settings/config.go"] = "templates/modules-settings.go.tmpl"
|
||||||
|
projectFiles["internal/settings/config_test.go"] = "templates/modules-settings_test.go.tmpl"
|
||||||
|
projectFiles["etc/config.yaml"] = "templates/modules-config.yaml.tmpl"
|
||||||
|
projectFiles[".gitignore"] = "templates/modules-gitignore.tmpl"
|
||||||
|
projectFiles["MODULES.md"] = "templates/modules-README.md.tmpl"
|
||||||
|
if storage != "none" {
|
||||||
|
projectFiles["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||||
|
projectFiles["internal/servicecontext/storage_test.go"] = "templates/modules-storage_test.go.tmpl"
|
||||||
|
projectFiles["internal/service/files.go"] = "templates/modules-filesService.go.tmpl"
|
||||||
|
projectFiles["internal/handler/files.go"] = "templates/modules-filesHandler.go.tmpl"
|
||||||
|
projectFiles["internal/handler/files_test.go"] = "templates/modules-filesHandler_test.go.tmpl"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if options.Force {
|
||||||
|
// Do not leave old generated Go files that reference dependencies omitted
|
||||||
|
// by this profile, and do not silently delete potentially edited code.
|
||||||
|
for _, path := range []string{"internal/repository/mysql.go", "internal/servicecontext/databaseIntegration_test.go", "schema.sql", "internal/settings/config.go", "internal/settings/config_test.go", "etc/config.yaml", "MODULES.md", "internal/servicecontext/storage.go", "internal/servicecontext/storage_test.go", "internal/service/files.go", "internal/handler/files.go", "internal/handler/files_test.go"} {
|
||||||
|
if _, emitted := projectFiles[path]; emitted {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(filepath.Join(directory, path)); err == nil {
|
||||||
|
return fmt.Errorf("existing %s conflicts with the selected profile; generate into a new directory instead of removing existing code", path)
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
paths := make([]string, 0, len(projectFiles))
|
paths := make([]string, 0, len(projectFiles))
|
||||||
for path := range projectFiles {
|
for path := range projectFiles {
|
||||||
paths = append(paths, path)
|
paths = append(paths, path)
|
||||||
}
|
}
|
||||||
sort.Strings(paths)
|
sort.Strings(paths)
|
||||||
|
// Render and format the whole project before touching the target directory.
|
||||||
|
contents := make(map[string]string, len(paths))
|
||||||
for _, path := range paths {
|
for _, path := range paths {
|
||||||
content, err := render(projectFiles[path], data)
|
content, err := render(projectFiles[path], data)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -111,6 +176,9 @@ func Generate(options Options) error {
|
|||||||
}
|
}
|
||||||
content = string(formatted)
|
content = string(formatted)
|
||||||
}
|
}
|
||||||
|
contents[path] = content
|
||||||
|
}
|
||||||
|
for _, path := range paths {
|
||||||
target := filepath.Join(directory, filepath.FromSlash(path))
|
target := filepath.Join(directory, filepath.FromSlash(path))
|
||||||
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
||||||
return fmt.Errorf("create %s: %w", filepath.Dir(path), err)
|
return fmt.Errorf("create %s: %w", filepath.Dir(path), err)
|
||||||
@@ -122,7 +190,7 @@ func Generate(options Options) error {
|
|||||||
return fmt.Errorf("check %s: %w", target, err)
|
return fmt.Errorf("check %s: %w", target, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(target, []byte(content), 0o644); err != nil {
|
if err := os.WriteFile(target, []byte(contents[path]), 0o644); err != nil {
|
||||||
return fmt.Errorf("write %s: %w", target, err)
|
return fmt.Errorf("write %s: %w", target, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -130,10 +198,15 @@ func Generate(options Options) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type templateData struct {
|
type templateData struct {
|
||||||
|
API bool
|
||||||
Module string
|
Module string
|
||||||
FrameworkVersion string
|
FrameworkVersion string
|
||||||
FrameworkReplace string
|
FrameworkReplace string
|
||||||
Database string
|
Database string
|
||||||
|
Modules bool
|
||||||
|
LogOutput string
|
||||||
|
LogFormat string
|
||||||
|
Storage string
|
||||||
}
|
}
|
||||||
|
|
||||||
func render(name string, data templateData) (string, error) {
|
func render(name string, data templateData) (string, error) {
|
||||||
@@ -202,12 +275,29 @@ func canonicalPath(path string) string {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return path
|
return path
|
||||||
}
|
}
|
||||||
resolved, err := filepath.EvalSymlinks(abs)
|
// Resolve the nearest existing ancestor too: generation now renders before
|
||||||
|
// creating directories, and macOS /var commonly resolves through /private.
|
||||||
|
candidate := abs
|
||||||
|
var tail []string
|
||||||
|
for {
|
||||||
|
resolved, err := filepath.EvalSymlinks(candidate)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
|
for i := len(tail) - 1; i >= 0; i-- {
|
||||||
|
resolved = filepath.Join(resolved, tail[i])
|
||||||
|
}
|
||||||
return resolved
|
return resolved
|
||||||
}
|
}
|
||||||
|
if !errors.Is(err, os.ErrNotExist) {
|
||||||
return abs
|
return abs
|
||||||
}
|
}
|
||||||
|
parent := filepath.Dir(candidate)
|
||||||
|
if parent == candidate {
|
||||||
|
return abs
|
||||||
|
}
|
||||||
|
tail = append(tail, filepath.Base(candidate))
|
||||||
|
candidate = parent
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func findFrameworkRoot() string {
|
func findFrameworkRoot() string {
|
||||||
directory, err := os.Getwd()
|
directory, err := os.Getwd()
|
||||||
|
|||||||
@@ -1,10 +1,13 @@
|
|||||||
# GoOmni service
|
# GoOmni service
|
||||||
|
|
||||||
This service was generated with `go-omni new`.
|
This service was generated with `go-omni new`.
|
||||||
|
{{if .Modules}}
|
||||||
|
**本项目启用了模块化配置,启动步骤和环境变量请先看 [MODULES.md](MODULES.md)。** 配置来自 `etc/config.yaml`,文件模块需要配置环境密钥后启动。
|
||||||
|
{{end}}
|
||||||
|
|
||||||
Framework dependency: `git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}`.
|
Framework dependency: `git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}`.
|
||||||
|
|
||||||
Storage: `{{.Database}}`.
|
Database: `{{.Database}}`.
|
||||||
|
|
||||||
{{if eq .Database "mysql"}}
|
{{if eq .Database "mysql"}}
|
||||||
先创建业务数据库,再执行项目的建表脚本(替换示例数据库名、账号和地址):
|
先创建业务数据库,再执行项目的建表脚本(替换示例数据库名、账号和地址):
|
||||||
|
|||||||
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
package servicecontext
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"sync"
|
||||||
|
{{if ne .Storage "none"}}"net/http"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||||
|
{{if eq .Database "mysql"}}"time"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"github.com/go-sql-driver/mysql"{{end}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
"{{.Module}}/internal/middleware"
|
||||||
|
)
|
||||||
|
// Context owns infrastructure, while Logger is borrowed from main.
|
||||||
|
// Add business repositories here; HTTP contracts do not define database tables.
|
||||||
|
type Context struct {
|
||||||
|
Logger *logx.Logger
|
||||||
|
Health *health.Checker
|
||||||
|
Middlewares map[string]httpx.Middleware
|
||||||
|
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
Files storage.Store
|
||||||
|
Uploads *storage.Uploader
|
||||||
|
Downloads storage.Presigner
|
||||||
|
UploadLinks storage.UploadPresigner
|
||||||
|
FileHTTP http.Handler
|
||||||
|
storageClose func()error
|
||||||
|
{{end}}
|
||||||
|
closeMu sync.Mutex
|
||||||
|
closed bool
|
||||||
|
closeOnce sync.Once
|
||||||
|
closeErr error
|
||||||
|
}
|
||||||
|
// Pass the settings loaded by main. The optional form supports lightweight
|
||||||
|
// tests without infrastructure; selected adapters still require valid settings.
|
||||||
|
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
|
||||||
|
if ctx==nil{ctx=context.Background()}
|
||||||
|
if err=ctx.Err();err!=nil{return nil,err}
|
||||||
|
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
|
||||||
|
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||||
|
_=cfg
|
||||||
|
checker,err:=health.New();if err!=nil{return nil,err}
|
||||||
|
c:=&Context{Logger:logger,Health:checker,Middlewares:middleware.Defaults()}
|
||||||
|
defer func(){if err!=nil{err=errors.Join(err,c.Close())}}()
|
||||||
|
{{if eq .Database "mysql"}}
|
||||||
|
if err=cfg.Validate();err!=nil{return nil,err}
|
||||||
|
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
|
||||||
|
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
|
||||||
|
dsn.ParseTime=true;dsn.Loc=time.UTC
|
||||||
|
cfg.Database.DSN=dsn.FormatDSN()
|
||||||
|
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
|
||||||
|
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
|
||||||
|
{{end}}
|
||||||
|
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
|
||||||
|
return c,nil
|
||||||
|
}
|
||||||
|
// Close is idempotent, including startup failure and graceful shutdown.
|
||||||
|
func(c *Context)Close()error{
|
||||||
|
if c==nil{return nil}
|
||||||
|
c.closeMu.Lock();defer c.closeMu.Unlock();c.closed=true
|
||||||
|
c.closeOnce.Do(func(){
|
||||||
|
{{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}}
|
||||||
|
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||||
|
})
|
||||||
|
return c.closeErr
|
||||||
|
}
|
||||||
@@ -23,7 +23,10 @@ func (s *Users) Create(ctx context.Context, input model.UserInput) (model.User,
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return model.User{}, err
|
return model.User{}, err
|
||||||
}
|
}
|
||||||
return s.services.Users.Create(ctx, input)
|
user, err := s.services.Users.Create(ctx, input)
|
||||||
|
if err != nil { s.services.Logger.ErrorContext(ctx,"user.create.failed","error",err); return model.User{},err }
|
||||||
|
s.services.Logger.InfoContext(ctx,"user.created","userId",user.ID)
|
||||||
|
return user,nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Users) Get(ctx context.Context, id int64) (model.User, error) {
|
func (s *Users) Get(ctx context.Context, id int64) (model.User, error) {
|
||||||
|
|||||||
@@ -2,6 +2,10 @@ package servicecontext
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
{{if and .Modules (ne .Storage "none")}}"errors"
|
||||||
|
"net/http"
|
||||||
|
"sync"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||||
{{if eq .Database "mysql"}}"fmt"
|
{{if eq .Database "mysql"}}"fmt"
|
||||||
"time"
|
"time"
|
||||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
@@ -17,6 +21,18 @@ type Context struct {
|
|||||||
Logger *logx.Logger
|
Logger *logx.Logger
|
||||||
Users repository.UserRepository
|
Users repository.UserRepository
|
||||||
Health *health.Checker
|
Health *health.Checker
|
||||||
|
{{if and .Modules (ne .Storage "none")}}
|
||||||
|
Files storage.Store
|
||||||
|
Uploads *storage.Uploader
|
||||||
|
Downloads storage.Presigner
|
||||||
|
UploadLinks storage.UploadPresigner
|
||||||
|
FileHTTP http.Handler
|
||||||
|
storageClose func() error
|
||||||
|
closeMu sync.Mutex
|
||||||
|
closed bool
|
||||||
|
closeOnce sync.Once
|
||||||
|
closeErr error
|
||||||
|
{{end}}
|
||||||
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -61,6 +77,15 @@ func New(ctx context.Context, logger *logx.Logger) (*Context, error) {
|
|||||||
|
|
||||||
// Close is safe to call on startup failure and again during application exit.
|
// Close is safe to call on startup failure and again during application exit.
|
||||||
func (c *Context) Close() error {
|
func (c *Context) Close() error {
|
||||||
{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}
|
{{if and .Modules (ne .Storage "none")}}
|
||||||
return nil
|
if c==nil{return nil}
|
||||||
|
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||||
|
c.closed=true
|
||||||
|
c.closeOnce.Do(func(){
|
||||||
|
if c.storageClose!=nil {c.closeErr=errors.Join(c.closeErr,c.storageClose())}
|
||||||
|
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||||
|
})
|
||||||
|
return c.closeErr
|
||||||
|
{{else}}{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}{{end}}
|
||||||
|
{{if not (and .Modules (ne .Storage "none"))}}return nil{{end}}
|
||||||
}
|
}
|
||||||
|
|||||||
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
# 按需装配的日志与文件模块
|
||||||
|
|
||||||
|
本项目选用日志输出 `{{.LogOutput}}`、格式 `{{.LogFormat}}`、对象存储 `{{.Storage}}`。数据库选择 `{{.Database}}` 独立于对象存储。
|
||||||
|
|
||||||
|
配置入口为 `etc/config.yaml`,通过 `go run ./cmd/server -config /path/to/config.yaml` 指定其他 JSON/YAML 文件;`-config ''` 只用代码默认值和环境变量。优先级:默认值 → 文件 → 非空环境变量。未知字段、无效参数会在监听前拒绝。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go mod tidy
|
||||||
|
{{if ne .Storage "none"}}# 示例文件接口使用单一共享 Token,不是用户/租户权限系统。
|
||||||
|
export GO_OMNI_FILES_TOKEN="$(openssl rand -hex 32)"
|
||||||
|
{{if eq .Storage "local"}}# 本地签名密钥需要稳定保存;换密钥后旧链接失效。
|
||||||
|
export GO_OMNI_STORAGE_SIGNING_KEY="$(openssl rand -hex 32)"
|
||||||
|
{{else}}# 配置当前存储平台已有的桶和凭据,不会自动创建桶。
|
||||||
|
export GO_OMNI_STORAGE_BUCKET='your-bucket'
|
||||||
|
export GO_OMNI_STORAGE_ACCESS_KEY='your-access-key'
|
||||||
|
export GO_OMNI_STORAGE_SECRET_KEY='your-secret-key'
|
||||||
|
{{if eq .Storage "s3"}}export GO_OMNI_STORAGE_ENDPOINT='s3.us-east-1.amazonaws.com'
|
||||||
|
export GO_OMNI_STORAGE_REGION='us-east-1'
|
||||||
|
{{else if eq .Storage "r2"}}export GO_OMNI_STORAGE_ACCOUNT_ID='your-32-character-account-id'
|
||||||
|
{{else}}export GO_OMNI_STORAGE_REGION='cn-hangzhou'
|
||||||
|
{{end}}{{end}}{{end}}
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
{{if eq .Database "mysql"}}还需按 README 配置独立 MySQL 数据库并应用 schema.sql;应用不自动迁移。{{end}}
|
||||||
|
|
||||||
|
默认仅监听 `127.0.0.1:8080`。`GO_OMNI_HTTP_ADDR` 覆盖地址;本地签名 URL 的外部地址通过 `GO_OMNI_STORAGE_PUBLIC_URL` 单独配置,始终以 `/storage/object` 结尾,不使用请求 Host 自动拼接可信链接。
|
||||||
|
|
||||||
|
## 日志
|
||||||
|
|
||||||
|
业务示例在创建用户/上传文件成功时通过 logx 写结构化日志。Context 原样传递 requestId,框架不自动增加用户、租户等业务字段。文件模式已配置轮转:100 MiB、10 份、7 天、压缩。调整 `server.log` 即可更改;`redactKeys` 可增加业务敏感键。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -H 'Content-Type: application/json' -H 'X-Request-ID: example-request' \
|
||||||
|
-d '{"name":"Example","email":"example@example.com"}' http://127.0.0.1:8080/users
|
||||||
|
```
|
||||||
|
|
||||||
|
`GO_OMNI_LOG_LEVEL`、`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_FILE` 覆盖对应项;显式设置 `GO_OMNI_LOG_OUTPUT=stdout` 且未设置 LOG_FILE 时,切换到控制台并关闭文件轮转。慢请求阈值为 500ms,可通过 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 调整。字段脱敏不扫描 msg/error 中的自由文本;不要把凭据或签名 URL 拼到日志中。
|
||||||
|
|
||||||
|
{{if ne .Storage "none"}}## 文件示例
|
||||||
|
|
||||||
|
Handler → Service → storage 接口;底层构造与连接池集中在 `internal/servicecontext/storage.go`。示例接口要求 `Authorization: Bearer $GO_OMNI_FILES_TOKEN`,它授权这个示例的全部文件,实际业务应替换认证和对象归属检查。存储适配器不承担业务授权。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
printf 'hello go-omni' > /tmp/goOmniExample.txt
|
||||||
|
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" \
|
||||||
|
-H 'Content-Type: text/plain' -H 'X-Request-ID: file-example' \
|
||||||
|
--data-binary @/tmp/goOmniExample.txt http://127.0.0.1:8080/files
|
||||||
|
```
|
||||||
|
|
||||||
|
保存响应中 `data.key`,再请求下载链接(把示例 key 替换为返回值):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||||
|
-d '{"key":"uploads/returned-key"}' http://127.0.0.1:8080/files/downloadLink
|
||||||
|
```
|
||||||
|
|
||||||
|
返回 `data.url`,在有效期内 GET 即可下载,无需再携带 Token。签发前会检查对象存在;URL 默认 5 分钟有效,不写日志,响应设置 no-store。
|
||||||
|
|
||||||
|
需要直传时先申请链接:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||||
|
-d '{"size":13,"contentType":"text/plain"}' http://127.0.0.1:8080/files/uploadLink
|
||||||
|
```
|
||||||
|
|
||||||
|
响应包括随机 `data.key` 和 `data.request`(method/url/headers/expiresAt)。按返回的 method 和 headers 用原始文件字节上传到 URL;不能额外包 FormData。浏览器自动填写 Content-Length,其余签名 Header 原样发送;云桶需配置允许业务域名的 CORS。云端直传只绑定声明类型,实际内容需上传后验证;本地和 API 中转上传会检查允许类型及前缀嗅探结果。未完成内容验证前不要把暂存文件当作可信文件公开。
|
||||||
|
|
||||||
|
| 参数 | 默认值 / 环境变量 |
|
||||||
|
| --- | --- |
|
||||||
|
| 大小 | 32 MiB / GO_OMNI_STORAGE_MAX_BYTES |
|
||||||
|
| 类型 | text/plain、image/png、image/jpeg、application/pdf / GO_OMNI_STORAGE_CONTENT_TYPES(逗号分隔) |
|
||||||
|
| 链接有效期 | 5m / GO_OMNI_STORAGE_LINK_TTL(1s..1h) |
|
||||||
|
| 请求超时 | 30s / GO_OMNI_STORAGE_REQUEST_TIMEOUT(>0 且 <=5m) |
|
||||||
|
| 本地目录 | var/files / GO_OMNI_STORAGE_ROOT |
|
||||||
|
| 本地外部 URL | http://127.0.0.1:8080/storage/object / GO_OMNI_STORAGE_PUBLIC_URL |
|
||||||
|
| 云配置 | GO_OMNI_STORAGE_ENDPOINT、REGION、BUCKET、ACCOUNT_ID、SECURE |
|
||||||
|
| 环境密钥 | GO_OMNI_FILES_TOKEN、GO_OMNI_STORAGE_SIGNING_KEY、GO_OMNI_STORAGE_ACCESS_KEY、SECRET_KEY、SESSION_TOKEN |
|
||||||
|
|
||||||
|
全部 JSON/YAML 字段使用 lowerCamelCase。密钥字段仅允许环境变量,配置文件中写入会报未知字段;不要把运行环境文件提交到仓库。S3 的 endpoint 为 host[:port],OSS 的可选 endpoint 为 HTTPS origin;R2 从 accountId 派生官方 endpoint,region 为 auto。仅 S3/MinIO 的本地测试可设置 SECURE=false。
|
||||||
|
|
||||||
|
错误响应:未授权 401、无 Content-Length 411、超限 413、类型不允许 415、长度不符/非法 key 400、对象不存在 404、存储超时 504,其余内部错误统一隐藏细节。底层签名入口遵循存储协议返回 201/403 等状态,流式传输且访问日志保留实际状态。
|
||||||
|
|
||||||
|
云客户端只在组合处创建,使用专用有界连接池;配置正确不代表桶和网络已经可用,首次真实调用仍可能失败。不创建桶、不修改桶权限、不初始化任何业务数据库。更换平台时替换组合处适配器,业务 Service 接口保持不变。
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
## 生命周期与验证
|
||||||
|
|
||||||
|
启动失败会清理已创建资源;SIGINT/SIGTERM 后先等待 HTTP 请求退出,再关闭服务依赖,最后关闭日志。只关闭项目自己创建的资源,不关闭全局默认 HTTP Transport。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test -race ./...
|
||||||
|
go build -o server ./cmd/server
|
||||||
|
./server
|
||||||
|
```
|
||||||
|
|
||||||
|
生成的测试覆盖配置优先级/校验{{if ne .Storage "none"}}、资源关闭幂等性和借用日志所有权、上传与临时链接、签名篡改、限额/类型错误。业务测试使用临时 Local,不访问云账号;云端配置需另做实网验收{{end}}。基础用户 CRUD 示例仍未启用认证,部署前按业务需要替换。
|
||||||
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
# Non-secret settings only. Environment variables override these values.
|
||||||
|
server:
|
||||||
|
address: '127.0.0.1:8080'
|
||||||
|
enableLogger: true
|
||||||
|
slowRequestThreshold: 500ms
|
||||||
|
shutdownTimeout: 10s
|
||||||
|
log:
|
||||||
|
service: {{printf "%q" .Module}}
|
||||||
|
level: info
|
||||||
|
format: {{.LogFormat}}
|
||||||
|
{{if eq .LogOutput "file"}} file: './var/log/server.log'
|
||||||
|
rotation:
|
||||||
|
enabled: true
|
||||||
|
maxSizeMb: 100
|
||||||
|
maxBackups: 10
|
||||||
|
maxAgeDays: 7
|
||||||
|
compress: true
|
||||||
|
{{else}} output: {{.LogOutput}}
|
||||||
|
{{end}} redactKeys: []
|
||||||
|
{{if ne .Storage "none"}}storage:
|
||||||
|
provider: {{.Storage}}
|
||||||
|
maxBytes: 33554432
|
||||||
|
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||||
|
linkTtl: 5m
|
||||||
|
requestTimeout: 30s
|
||||||
|
{{if eq .Storage "local"}} root: './var/files'
|
||||||
|
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||||
|
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
|
||||||
|
secure: true
|
||||||
|
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
|
||||||
|
region: auto
|
||||||
|
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
|
||||||
|
region: us-east-1
|
||||||
|
{{else}} region: cn-hangzhou
|
||||||
|
endpoint: '' # empty uses the official regional endpoint
|
||||||
|
{{end}}{{end}}{{end}}
|
||||||
|
|
||||||
|
{{if and .API (eq .Database "mysql")}}database:
|
||||||
|
driver: mysql
|
||||||
|
maxOpenConns: 10
|
||||||
|
maxIdleConns: 5
|
||||||
|
connMaxLifetime: 30m
|
||||||
|
connMaxIdleTime: 5m
|
||||||
|
pingTimeout: 5s
|
||||||
|
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||||
|
{{end}}
|
||||||
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
package handler
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/subtle"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"{{.Module}}/internal/service"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Files struct {service *service.Files; maxBytes int64; timeout time.Duration}
|
||||||
|
func NewFiles(service *service.Files,maxBytes int64,timeout time.Duration)*Files{return &Files{service:service,maxBytes:maxBytes,timeout:timeout}}
|
||||||
|
|
||||||
|
// RegisterRoutes protects example control endpoints with one shared demo token.
|
||||||
|
// Replace this gate with project authentication and object ownership checks.
|
||||||
|
// Signed object URLs carry their own short-lived authorization.
|
||||||
|
func (h *Files) RegisterRoutes(application *omni.App,token string,signed http.Handler)error {
|
||||||
|
if len(token)<32{return errors.New("file example requires a token of at least 32 bytes")}
|
||||||
|
bounded:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||||
|
ctx,cancel:=context.WithTimeout(c.Context(),h.timeout);defer cancel();c.WithContext(ctx);return next(c)
|
||||||
|
}}
|
||||||
|
authorized:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||||
|
if len(c.Request().Header.Values("Authorization"))!=1 || subtle.ConstantTimeCompare([]byte(c.Header("Authorization")),[]byte("Bearer "+token))!=1{return omni.Unauthorized("UNAUTHORIZED","invalid file access token")}
|
||||||
|
return next(c)
|
||||||
|
}}
|
||||||
|
group:=application.Group("/files",authorized,bounded)
|
||||||
|
group.POST("",h.Upload)
|
||||||
|
group.POST("/uploadLink",h.UploadLink)
|
||||||
|
group.POST("/downloadLink",h.DownloadLink)
|
||||||
|
if signed!=nil {application.Any("/storage/object",func(c *omni.Context)error{
|
||||||
|
signed.ServeHTTP(contextWriter{c},c.Request());return nil
|
||||||
|
},bounded)}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (h *Files) Upload(c *omni.Context)error {
|
||||||
|
size:=c.Request().ContentLength
|
||||||
|
if size<0{return omni.NewError(http.StatusLengthRequired,"LENGTH_REQUIRED","Content-Length is required")}
|
||||||
|
if size>h.maxBytes{return omni.NewError(http.StatusRequestEntityTooLarge,"FILE_TOO_LARGE","file exceeds upload limit")}
|
||||||
|
body:=http.MaxBytesReader(c.Raw(),c.Request().Body,h.maxBytes)
|
||||||
|
defer body.Close()
|
||||||
|
info,err:=h.service.Upload(c.Context(),body,size,c.Header("Content-Type"))
|
||||||
|
if err!=nil{return fileError(err)}
|
||||||
|
return c.Created(info)
|
||||||
|
}
|
||||||
|
func (h *Files) UploadLink(c *omni.Context)error {
|
||||||
|
var input struct{Size int64 `json:"size" binding:"gte=0"`;ContentType string `json:"contentType" binding:"required,max=256"`}
|
||||||
|
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||||
|
link,err:=h.service.UploadLink(c.Context(),input.Size,input.ContentType)
|
||||||
|
if err!=nil{return fileError(err)}
|
||||||
|
c.SetResponseHeader("Cache-Control","no-store")
|
||||||
|
return c.OK(link)
|
||||||
|
}
|
||||||
|
func (h *Files) DownloadLink(c *omni.Context)error {
|
||||||
|
var input struct{Key string `json:"key" binding:"required,max=1024"`}
|
||||||
|
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||||
|
if err:=storage.Check(c.Context(),input.Key);err!=nil{return omni.BadRequest("INVALID_KEY","invalid object key")}
|
||||||
|
link,err:=h.service.DownloadLink(c.Context(),input.Key)
|
||||||
|
if err!=nil{return fileError(err)}
|
||||||
|
c.SetResponseHeader("Cache-Control","no-store")
|
||||||
|
return c.OK(map[string]string{"url":link})
|
||||||
|
}
|
||||||
|
func fileError(err error)error {
|
||||||
|
var limit *http.MaxBytesError
|
||||||
|
switch {
|
||||||
|
case errors.Is(err,storage.ErrTooLarge),errors.As(err,&limit):return omni.NewError(413,"FILE_TOO_LARGE","file exceeds upload limit").WithCause(err)
|
||||||
|
case errors.Is(err,storage.ErrContentType):return omni.NewError(415,"CONTENT_TYPE_NOT_ALLOWED","file content type is not allowed").WithCause(err)
|
||||||
|
case errors.Is(err,storage.ErrSizeMismatch):return omni.BadRequest("SIZE_MISMATCH","file size does not match Content-Length").WithCause(err)
|
||||||
|
case errors.Is(err,storage.ErrNotFound):return omni.NotFound("FILE_NOT_FOUND","file was not found").WithCause(err)
|
||||||
|
case errors.Is(err,context.DeadlineExceeded):return omni.NewError(504,"STORAGE_TIMEOUT","storage operation timed out").WithCause(err)
|
||||||
|
default:return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Track raw streaming handler responses through Omni so access logs retain the
|
||||||
|
// actual status (including failed signatures), without buffering object bodies.
|
||||||
|
type contextWriter struct{ctx *omni.Context}
|
||||||
|
func (w contextWriter) Header()http.Header{return w.ctx.Raw().Header()}
|
||||||
|
func (w contextWriter) WriteHeader(status int){_ = w.ctx.WriteResponse(status,nil,nil)}
|
||||||
|
func (w contextWriter) Write(data []byte)(int,error){
|
||||||
|
if !w.ctx.Responded(){if err:=w.ctx.WriteResponse(http.StatusOK,nil,nil);err!=nil{return 0,err}}
|
||||||
|
return w.ctx.Raw().Write(data)
|
||||||
|
}
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
package handler_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
"{{.Module}}/internal/handler"
|
||||||
|
"{{.Module}}/internal/service"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Exercise business code using Local even when production selects a cloud
|
||||||
|
// adapter. These tests do not use cloud accounts or network credentials.
|
||||||
|
func TestFileUploadAndLinks(t *testing.T){
|
||||||
|
files,err:=local.New(t.TempDir(),64);if err!=nil{t.Fatal(err)}
|
||||||
|
policy:=storage.UploadPolicy{MaxBytes:64,ContentTypes:[]string{"text/plain"}}
|
||||||
|
signer,err:=storage.NewUploadSigner(files,"http://localhost/storage/object",bytes.Repeat([]byte{1},32),policy);if err!=nil{t.Fatal(err)}
|
||||||
|
uploader,err:=storage.NewUploader(files,policy);if err!=nil{t.Fatal(err)}
|
||||||
|
var logs bytes.Buffer
|
||||||
|
logger,err:=logx.New(logx.Config{Writer:&logs});if err!=nil{t.Fatal(err)}
|
||||||
|
logx.SetDefault(logger);defer logx.SetDefault(nil)
|
||||||
|
app:=omni.New(omni.WithLogger(true))
|
||||||
|
token:=strings.Repeat("t",32)
|
||||||
|
svc:=service.NewFiles(files,uploader,signer,signer,logger,time.Minute)
|
||||||
|
if err=handler.NewFiles(svc,64,time.Second).RegisterRoutes(app,token,signer);err!=nil{t.Fatal(err)}
|
||||||
|
request:=func(method,path,body,contentType string,authorized bool)*httptest.ResponseRecorder{
|
||||||
|
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type",contentType);r.Header.Set("X-Request-ID","upload-test")
|
||||||
|
if authorized{r.Header.Set("Authorization","Bearer "+token)}
|
||||||
|
w:=httptest.NewRecorder();app.ServeHTTP(w,r);return w
|
||||||
|
}
|
||||||
|
if w:=request("POST","/files","hello","text/plain",false);w.Code!=401{t.Fatal(w.Code)}
|
||||||
|
w:=request("POST","/files","hello","text/plain",true)
|
||||||
|
var saved struct{Data storage.Info}
|
||||||
|
if w.Code!=201 || json.Unmarshal(w.Body.Bytes(),&saved)!=nil || saved.Data.Size!=5{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
raw,_:=json.Marshal(map[string]string{"key":saved.Data.Key})
|
||||||
|
w=request("POST","/files/downloadLink",string(raw),"application/json",true)
|
||||||
|
var download struct{Data struct{URL string}}
|
||||||
|
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&download)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
w=request("GET",download.Data.URL,"","",false)
|
||||||
|
if w.Code!=200 || w.Body.String()!="hello"{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
u,_:=url.Parse(download.Data.URL);q:=u.Query();q.Set("signature","tampered");u.RawQuery=q.Encode()
|
||||||
|
if w=request("GET",u.String(),"","",false);w.Code!=403{t.Fatal(w.Code)}
|
||||||
|
for _,tc:=range []struct{path,body,contentType string;status int}{
|
||||||
|
{"/files",strings.Repeat("x",65),"text/plain",413},
|
||||||
|
{"/files","hello","image/png",415},
|
||||||
|
{"/files","<html>hello</html>","text/plain",415},
|
||||||
|
{"/files/downloadLink",`{"key":"../bad"}`,"application/json",400},
|
||||||
|
{"/files/downloadLink",`{"key":"missing"}`,"application/json",404},
|
||||||
|
{"/files/uploadLink",`{"size":65,"contentType":"text/plain"}`,"application/json",413},
|
||||||
|
{"/files/uploadLink",`{"size":5,"contentType":"image/png"}`,"application/json",415},
|
||||||
|
}{if w=request("POST",tc.path,tc.body,tc.contentType,true);w.Code!=tc.status{t.Fatal(tc.path,w.Code,w.Body.String())}}
|
||||||
|
w=request("POST","/files/uploadLink",`{"size":5,"contentType":"text/plain"}`,"application/json",true)
|
||||||
|
var direct struct{Data service.FileUploadLink}
|
||||||
|
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&direct)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
if w=request("PUT",direct.Data.Request.URL,"world","text/plain",false);w.Code!=201{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
object,err:=files.Get(context.Background(),direct.Data.Key);if err!=nil{t.Fatal(err)};body,err:=io.ReadAll(object.Body);object.Body.Close();if err!=nil || string(body)!="world"{t.Fatal(err,string(body))}
|
||||||
|
decoder:=json.NewDecoder(&logs);sawBusiness,sawDenied:=false,false
|
||||||
|
for {var entry map[string]any;err:=decoder.Decode(&entry);if err==io.EOF{break};if err!=nil{t.Fatal(err)}
|
||||||
|
if entry["requestId"]!="upload-test"{t.Fatal("lost request correlation")}
|
||||||
|
if entry["msg"]=="file.uploaded"{sawBusiness=true}
|
||||||
|
if entry["msg"]=="http.request" && entry["path"]=="/storage/object" && entry["status"]==float64(http.StatusForbidden){sawDenied=true}
|
||||||
|
}
|
||||||
|
if !sawBusiness || !sawDenied{t.Fatal("business log or signed-handler status missing")}
|
||||||
|
}
|
||||||
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"io"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Files borrows provider-neutral capabilities; it never creates or closes clients.
|
||||||
|
// Authentication/ownership rules belong to this project, outside storage adapters.
|
||||||
|
type Files struct {
|
||||||
|
store storage.Store
|
||||||
|
uploader *storage.Uploader
|
||||||
|
downloads storage.Presigner
|
||||||
|
uploadLinks storage.UploadPresigner
|
||||||
|
logger *logx.Logger
|
||||||
|
ttl time.Duration
|
||||||
|
}
|
||||||
|
func NewFiles(store storage.Store,uploader *storage.Uploader,downloads storage.Presigner,uploadLinks storage.UploadPresigner,logger *logx.Logger,ttl time.Duration)*Files{
|
||||||
|
return &Files{store:store,uploader:uploader,downloads:downloads,uploadLinks:uploadLinks,logger:logger.With("component","files"),ttl:ttl}
|
||||||
|
}
|
||||||
|
func newFileKey()(string,error){
|
||||||
|
var id [16]byte
|
||||||
|
if _,err:=rand.Read(id[:]);err!=nil{return "",err}
|
||||||
|
return "uploads/"+hex.EncodeToString(id[:]),nil
|
||||||
|
}
|
||||||
|
func (s *Files) Upload(ctx context.Context,body io.Reader,size int64,contentType string)(storage.Info,error){
|
||||||
|
key,err:=newFileKey();if err!=nil{return storage.Info{},err}
|
||||||
|
info,err:=s.uploader.Put(ctx,key,body,size,contentType)
|
||||||
|
if err!=nil{s.logger.WarnContext(ctx,"file.upload.failed","error",err);return storage.Info{},err}
|
||||||
|
s.logger.InfoContext(ctx,"file.uploaded","key",info.Key,"size",info.Size,"contentType",info.ContentType)
|
||||||
|
return info,nil
|
||||||
|
}
|
||||||
|
type FileUploadLink struct {
|
||||||
|
Key string `json:"key"`
|
||||||
|
Request storage.UploadRequest `json:"request"`
|
||||||
|
}
|
||||||
|
func (s *Files) UploadLink(ctx context.Context,size int64,contentType string)(FileUploadLink,error){
|
||||||
|
key,err:=newFileKey();if err!=nil{return FileUploadLink{},err}
|
||||||
|
// Check the same policy before issuing a local or cloud capability.
|
||||||
|
// Cloud byte-level checks must happen in a post-upload validation workflow.
|
||||||
|
request,err:=s.uploadLinks.PresignPut(ctx,key,size,contentType,s.ttl)
|
||||||
|
if err!=nil{return FileUploadLink{},err}
|
||||||
|
s.logger.InfoContext(ctx,"file.uploadLink.issued","key",key)
|
||||||
|
return FileUploadLink{Key:key,Request:request},nil
|
||||||
|
}
|
||||||
|
func (s *Files) DownloadLink(ctx context.Context,key string)(string,error){
|
||||||
|
if err:=storage.Check(ctx,key);err!=nil{return "",err}
|
||||||
|
// Confirm the object exists without buffering its contents.
|
||||||
|
object,err:=s.store.Get(ctx,key);if err!=nil{return "",err}
|
||||||
|
if err=object.Body.Close();err!=nil{return "",err}
|
||||||
|
link,err:=s.downloads.PresignGet(ctx,key,s.ttl)
|
||||||
|
if err!=nil{return "",err}
|
||||||
|
s.logger.InfoContext(ctx,"file.downloadLink.issued","key",key)
|
||||||
|
return link.String(),nil
|
||||||
|
}
|
||||||
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
/var/
|
||||||
|
/server
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
*.local.yaml
|
||||||
|
*.local.json
|
||||||
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"{{.Module}}/internal/handler"
|
||||||
|
"{{.Module}}/internal/service"
|
||||||
|
"{{.Module}}/internal/servicecontext"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main(){
|
||||||
|
path:=flag.String("config","etc/config.yaml","JSON/YAML settings; empty uses defaults and environment")
|
||||||
|
flag.Parse()
|
||||||
|
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||||
|
}
|
||||||
|
func run(path string)(runErr error){
|
||||||
|
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
cfg,err:=settings.Load(path)
|
||||||
|
if err!=nil{return err}
|
||||||
|
logger,err:=logx.New(cfg.Server.Log)
|
||||||
|
if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||||
|
logx.SetDefault(logger)
|
||||||
|
services,err:=servicecontext.New(ctx,logger)
|
||||||
|
if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,services.Close())}()
|
||||||
|
{{if ne .Storage "none"}}if err=services.InitStorage(ctx,cfg.Storage);err!=nil{return err}{{end}}
|
||||||
|
application:=omni.New(omni.WithConfig(cfg.Server))
|
||||||
|
application.OnShutdown(logger.Close)
|
||||||
|
application.OnShutdown(services.Close)
|
||||||
|
application.GET("/health/live",services.Health.LivenessHandler())
|
||||||
|
application.GET("/health/ready",services.Health.ReadinessHandler())
|
||||||
|
handler.NewUsers(service.NewUsers(services)).RegisterRoutes(application)
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
files:=service.NewFiles(services.Files,services.Uploads,services.Downloads,services.UploadLinks,logger,cfg.Storage.LinkTTL)
|
||||||
|
if err=handler.NewFiles(files,cfg.Storage.MaxBytes,cfg.Storage.RequestTimeout).RegisterRoutes(application,cfg.Storage.FilesToken,services.FileHTTP);err!=nil{return err}
|
||||||
|
{{end}}
|
||||||
|
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||||
|
return application.RunContext(ctx,cfg.Server.Address)
|
||||||
|
}
|
||||||
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
// Package settings owns this project's configuration; it adds no business identity model.
|
||||||
|
package settings
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
{{if ne .Storage "none"}}"fmt"
|
||||||
|
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||||
|
"strconv"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||||
|
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Config struct {
|
||||||
|
Server config.Config `json:"server"`
|
||||||
|
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
|
||||||
|
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func Default() Config {
|
||||||
|
server:=config.Default()
|
||||||
|
server.Address="127.0.0.1:8080"
|
||||||
|
server.EnableLogger=true
|
||||||
|
server.SlowRequestThreshold=500*time.Millisecond
|
||||||
|
server.Log.Service={{printf "%q" .Module}}
|
||||||
|
server.Log.Format={{printf "%q" .LogFormat}}
|
||||||
|
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
|
||||||
|
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
|
||||||
|
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
|
||||||
|
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||||
|
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
|
||||||
|
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
|
||||||
|
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
|
||||||
|
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
|
||||||
|
{{if eq .Storage "r2"}}Region:"auto",{{end}}
|
||||||
|
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
|
||||||
|
}{{end}}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||||
|
// Pass an empty path to use only defaults and environment.
|
||||||
|
func Load(path string)(Config,error) {
|
||||||
|
cfg:=Default()
|
||||||
|
var err error
|
||||||
|
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
|
||||||
|
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
|
||||||
|
if err!=nil{return Config{},err}
|
||||||
|
// A stream explicitly selected by environment overrides a file profile.
|
||||||
|
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
|
||||||
|
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
|
||||||
|
}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
|
||||||
|
{{end}}
|
||||||
|
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
|
||||||
|
if err=cfg.Validate();err!=nil{return Config{},err}
|
||||||
|
return cfg,nil
|
||||||
|
}
|
||||||
|
func (c Config) Validate()error {
|
||||||
|
{{if and .API (eq .Database "mysql")}}
|
||||||
|
if err:=c.Database.Validate();err!=nil{return err}
|
||||||
|
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
|
||||||
|
{{end}}
|
||||||
|
if err:=c.Server.Validate();err!=nil{return err}
|
||||||
|
if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")}
|
||||||
|
switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")}
|
||||||
|
{{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}}
|
||||||
|
}
|
||||||
|
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
type StorageConfig struct {
|
||||||
|
Provider string `json:"provider"`
|
||||||
|
Root string `json:"root"`
|
||||||
|
PublicURL string `json:"publicUrl"`
|
||||||
|
Endpoint string `json:"endpoint"`
|
||||||
|
Region string `json:"region"`
|
||||||
|
Bucket string `json:"bucket"`
|
||||||
|
AccountID string `json:"accountId"`
|
||||||
|
Secure bool `json:"secure"`
|
||||||
|
MaxBytes int64 `json:"maxBytes"`
|
||||||
|
ContentTypes []string `json:"contentTypes"`
|
||||||
|
LinkTTL time.Duration `json:"linkTtl"`
|
||||||
|
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||||
|
// Secrets are environment-only and cannot be populated by config files.
|
||||||
|
FilesToken string `json:"-"`
|
||||||
|
SigningKey string `json:"-"`
|
||||||
|
AccessKey string `json:"-"`
|
||||||
|
SecretKey string `json:"-"`
|
||||||
|
SessionToken string `json:"-"`
|
||||||
|
}
|
||||||
|
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||||
|
func (s *StorageConfig) applyEnv()error {
|
||||||
|
values:=map[string]*string{
|
||||||
|
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
|
||||||
|
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
|
||||||
|
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
|
||||||
|
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
|
||||||
|
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
|
||||||
|
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
|
||||||
|
}
|
||||||
|
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
|
||||||
|
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
|
||||||
|
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
|
||||||
|
}
|
||||||
|
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
|
||||||
|
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
|
||||||
|
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (s StorageConfig) Validate()error {
|
||||||
|
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
|
||||||
|
if err:=s.Policy().Validate();err!=nil{return err}
|
||||||
|
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||||
|
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||||
|
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||||
|
{{if eq .Storage "local"}}
|
||||||
|
if s.Root=="" {return errors.New("storage.root is required")}
|
||||||
|
u,err:=url.Parse(s.PublicURL)
|
||||||
|
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||||
|
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||||
|
{{else}}
|
||||||
|
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||||
|
{{if eq .Storage "r2"}}
|
||||||
|
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||||
|
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||||
|
{{else if eq .Storage "s3"}}
|
||||||
|
u,err:=url.Parse("https://"+s.Endpoint)
|
||||||
|
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
|
||||||
|
{{else if eq .Storage "oss"}}
|
||||||
|
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
|
||||||
|
if !s.Secure{return errors.New("OSS requires secure=true")}
|
||||||
|
{{end}}
|
||||||
|
{{end}}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
{{end}}
|
||||||
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
package settings
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
func environment(t *testing.T){
|
||||||
|
t.Helper()
|
||||||
|
for _,entry:=range os.Environ(){key,_,_:=strings.Cut(entry,"=");if strings.HasPrefix(key,"GO_OMNI_"){t.Setenv(key,"")}}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
t.Setenv("GO_OMNI_FILES_TOKEN",strings.Repeat("t",32))
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY",strings.Repeat("s",32))
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","fixtureAccess")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","fixtureSecret")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_BUCKET","testbucket")
|
||||||
|
{{if eq .Storage "s3"}}t.Setenv("GO_OMNI_STORAGE_ENDPOINT","127.0.0.1:9000"){{end}}
|
||||||
|
{{if eq .Storage "r2"}}t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID",strings.Repeat("a",32)){{end}}
|
||||||
|
{{end}}
|
||||||
|
}
|
||||||
|
func TestSettingsPrecedenceAndStrictness(t *testing.T){
|
||||||
|
environment(t)
|
||||||
|
path:=filepath.Join(t.TempDir(),"config.yaml")
|
||||||
|
if err:=os.WriteFile(path,[]byte("server:\n address: '127.0.0.1:8123'\n log:\n level: debug\n"),0600);err!=nil{t.Fatal(err)}
|
||||||
|
t.Setenv("GO_OMNI_HTTP_ADDR","127.0.0.1:9123")
|
||||||
|
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout")
|
||||||
|
cfg,err:=Load(path);if err!=nil{t.Fatal(err)}
|
||||||
|
if cfg.Server.Address!="127.0.0.1:9123" || cfg.Server.Log.Level!="debug" || cfg.Server.Log.Output!="stdout" || cfg.Server.Log.File!="" || cfg.Server.Log.Rotation.Enabled {t.Fatal("configuration precedence failed")}
|
||||||
|
for _,body:=range []string{"unexpected: true","server:\n slowRequestThreshold: -1s","server:\n log:\n format: invalid"{{if ne .Storage "none"}},"storage:\n filesToken: forbidden","storage:\n maxBytes: -1","storage:\n provider: unsupported"{{end}}}{
|
||||||
|
if err:=os.WriteFile(path,[]byte(body),0600);err!=nil{t.Fatal(err)}
|
||||||
|
if _,err:=Load(path);err==nil{t.Fatal("invalid configuration accepted")}
|
||||||
|
}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
t.Setenv("GO_OMNI_FILES_TOKEN","")
|
||||||
|
if _,err:=Load("");err==nil{t.Fatal("missing file token accepted")}
|
||||||
|
{{end}}
|
||||||
|
}
|
||||||
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
{{if eq .Storage "local"}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
{{else}}
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
{{if eq .Storage "oss"}}
|
||||||
|
ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss"
|
||||||
|
ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
|
||||||
|
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
|
||||||
|
{{else}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/s3"
|
||||||
|
"github.com/minio/minio-go/v7/pkg/credentials"
|
||||||
|
{{end}}{{end}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InitStorage is called once, before serving requests. The composition root
|
||||||
|
// owns provider clients; business services receive only capability interfaces.
|
||||||
|
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
|
||||||
|
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
|
||||||
|
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||||
|
if c.closed{return errors.New("service dependencies are closed")}
|
||||||
|
if err:=ctx.Err();err!=nil{return err}
|
||||||
|
if c.Files!=nil{return errors.New("storage already initialized")}
|
||||||
|
if err:=cfg.Validate();err!=nil{return err}
|
||||||
|
{{if eq .Storage "local"}}
|
||||||
|
files,err:=local.New(cfg.Root,cfg.MaxBytes)
|
||||||
|
if err!=nil{return err}
|
||||||
|
signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy())
|
||||||
|
if err!=nil{return err}
|
||||||
|
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||||
|
if err!=nil{return err}
|
||||||
|
c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer
|
||||||
|
{{else}}
|
||||||
|
// Dedicated bounded pool, never the process-wide default transport.
|
||||||
|
transport:=&http.Transport{
|
||||||
|
Proxy:http.ProxyFromEnvironment,
|
||||||
|
DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext,
|
||||||
|
ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32,
|
||||||
|
IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second,
|
||||||
|
ResponseHeaderTimeout:cfg.RequestTimeout,
|
||||||
|
}
|
||||||
|
initialized:=false
|
||||||
|
defer func(){if !initialized{transport.CloseIdleConnections()}}()
|
||||||
|
{{if eq .Storage "oss"}}
|
||||||
|
sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region).
|
||||||
|
WithSignatureVersion(ossSDK.SignatureVersionV4).
|
||||||
|
WithAdditionalHeaders([]string{"content-length"}).
|
||||||
|
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)).
|
||||||
|
WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout})
|
||||||
|
if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)}
|
||||||
|
files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes)
|
||||||
|
{{else}}
|
||||||
|
creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)
|
||||||
|
{{if eq .Storage "r2"}}
|
||||||
|
options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds)
|
||||||
|
if err!=nil{return err}
|
||||||
|
options.MaxBytes,options.Transport=cfg.MaxBytes,transport
|
||||||
|
{{else}}
|
||||||
|
options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport}
|
||||||
|
{{end}}
|
||||||
|
files,err:=s3.Open(options)
|
||||||
|
{{end}}
|
||||||
|
if err!=nil{return err}
|
||||||
|
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||||
|
if err!=nil{return err}
|
||||||
|
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
|
||||||
|
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
|
||||||
|
initialized=true
|
||||||
|
{{end}}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
)
|
||||||
|
func TestStorageInitializationAndOwnership(t *testing.T){
|
||||||
|
cfg:=settings.Default().Storage
|
||||||
|
cfg.FilesToken=strings.Repeat("t",32)
|
||||||
|
cfg.SigningKey=strings.Repeat("s",32)
|
||||||
|
cfg.Root=t.TempDir()
|
||||||
|
cfg.AccessKey,cfg.SecretKey,cfg.Bucket="fixtureAccess","fixtureSecret","testbucket"
|
||||||
|
{{if eq .Storage "s3"}}cfg.Endpoint="127.0.0.1:9000"{{end}}
|
||||||
|
{{if eq .Storage "r2"}}cfg.AccountID=strings.Repeat("a",32){{end}}
|
||||||
|
var output bytes.Buffer
|
||||||
|
logger,err:=logx.New(logx.Config{Writer:&output});if err!=nil{t.Fatal(err)};defer logger.Close()
|
||||||
|
c:=&Context{Logger:logger}
|
||||||
|
canceled,cancel:=context.WithCancel(context.Background());cancel()
|
||||||
|
if err=c.InitStorage(canceled,cfg);err==nil || c.Files!=nil{t.Fatal("canceled startup initialized storage")}
|
||||||
|
invalid:=cfg;invalid.FilesToken=""
|
||||||
|
if err=c.InitStorage(context.Background(),invalid);err==nil || c.Files!=nil{t.Fatal("invalid settings initialized storage")}
|
||||||
|
if err=c.InitStorage(context.Background(),cfg);err!=nil{t.Fatal(err)}
|
||||||
|
if c.Files==nil || c.Uploads==nil || c.Downloads==nil || c.UploadLinks==nil{t.Fatal("dependencies not assembled")}
|
||||||
|
if err=c.InitStorage(context.Background(),cfg);err==nil{t.Fatal("duplicate initialization accepted")}
|
||||||
|
cleanup:=c.storageClose;count:=0
|
||||||
|
c.storageClose=func()error{count++;if cleanup!=nil{return cleanup()};return nil}
|
||||||
|
if err=c.Close();err!=nil{t.Fatal(err)};if err=c.Close();err!=nil{t.Fatal(err)}
|
||||||
|
logger.Info("still usable")
|
||||||
|
if count!=1 || !strings.Contains(output.String(),"still usable"){t.Fatal("resource ownership violated")}
|
||||||
|
closed:=&Context{Logger:logger}
|
||||||
|
if err=closed.Close();err!=nil{t.Fatal(err)}
|
||||||
|
if err=closed.InitStorage(context.Background(),cfg);err==nil || closed.Files!=nil{t.Fatal("closed context reopened resources")}
|
||||||
|
}
|
||||||
@@ -10,7 +10,19 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
|||||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||||
// downloaded module version, which also has a module checksum.
|
// downloaded module version, which also has a module checksum.
|
||||||
const defaultFrameworkVersion = "v0.6.0-preview"
|
const defaultFrameworkVersion = "v0.7.0-preview"
|
||||||
|
|
||||||
|
// This target is not a claim that the release already exists. Source users
|
||||||
|
// select a local framework checkout until the module templates are published.
|
||||||
|
const modulesFrameworkVersion = "v0.7.0-preview"
|
||||||
|
|
||||||
|
func installedFrameworkVersion() string {
|
||||||
|
info, _ := debug.ReadBuildInfo()
|
||||||
|
if info != nil && info.Main.Path == frameworkModule && info.Main.Sum != "" && validFrameworkVersion(info.Main.Version) {
|
||||||
|
return info.Main.Version
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
||||||
|
|
||||||
|
|||||||
@@ -29,6 +29,7 @@ type Config struct {
|
|||||||
TLS TLSConfig `json:"tls"`
|
TLS TLSConfig `json:"tls"`
|
||||||
RequestIDHeader string `json:"requestIdHeader"`
|
RequestIDHeader string `json:"requestIdHeader"`
|
||||||
EnableLogger bool `json:"enableLogger"`
|
EnableLogger bool `json:"enableLogger"`
|
||||||
|
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||||
TrustedProxies []string `json:"trustedProxies"`
|
TrustedProxies []string `json:"trustedProxies"`
|
||||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||||
Log logx.Config `json:"log"`
|
Log logx.Config `json:"log"`
|
||||||
@@ -46,6 +47,9 @@ func Default() Config {
|
|||||||
|
|
||||||
// Validate checks configuration values before an application starts.
|
// Validate checks configuration values before an application starts.
|
||||||
func (c Config) Validate() error {
|
func (c Config) Validate() error {
|
||||||
|
if c.SlowRequestThreshold < 0 {
|
||||||
|
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
|
||||||
|
}
|
||||||
if strings.TrimSpace(c.Address) == "" {
|
if strings.TrimSpace(c.Address) == "" {
|
||||||
return &ConfigError{Field: "address", Message: "is required"}
|
return &ConfigError{Field: "address", Message: "is required"}
|
||||||
}
|
}
|
||||||
@@ -114,6 +118,11 @@ func WithLogger(enabled bool) Option {
|
|||||||
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// WithSlowRequestThreshold enables access logs and warns after this duration.
|
||||||
|
func WithSlowRequestThreshold(threshold time.Duration) Option {
|
||||||
|
return func(cfg *Config) { cfg.SlowRequestThreshold = threshold; cfg.EnableLogger = true }
|
||||||
|
}
|
||||||
|
|
||||||
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
||||||
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
||||||
// by Config.Validate and panic when passed directly to App.New.
|
// by Config.Validate and panic when passed directly to App.New.
|
||||||
@@ -126,15 +135,21 @@ func WithTrustedProxies(proxies ...string) Option {
|
|||||||
// environment. Later options can still override individual fields.
|
// environment. Later options can still override individual fields.
|
||||||
func WithConfig(settings Config) Option {
|
func WithConfig(settings Config) Option {
|
||||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
return func(cfg *Config) {
|
return func(cfg *Config) {
|
||||||
*cfg = settings
|
*cfg = settings
|
||||||
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
cfg.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// WithLogConfig configures the logger used by the application runtime.
|
// WithLogConfig configures the logger used by the application runtime.
|
||||||
func WithLogConfig(settings logx.Config) Option {
|
func WithLogConfig(settings logx.Config) Option {
|
||||||
return func(cfg *Config) { cfg.Log = settings }
|
settings.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||||
|
return func(cfg *Config) {
|
||||||
|
cfg.Log = settings
|
||||||
|
cfg.Log.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// WithShutdownTimeout controls how long RunContext waits for active requests.
|
// WithShutdownTimeout controls how long RunContext waits for active requests.
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
|||||||
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
||||||
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
||||||
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
||||||
|
"GO_OMNI_SLOW_REQUEST_THRESHOLD",
|
||||||
} {
|
} {
|
||||||
t.Setenv(key, "")
|
t.Setenv(key, "")
|
||||||
}
|
}
|
||||||
@@ -80,6 +81,7 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
|||||||
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
|
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
|
||||||
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
|
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
|
||||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
|
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
|
||||||
|
t.Setenv("GO_OMNI_SLOW_REQUEST_THRESHOLD", "500ms")
|
||||||
|
|
||||||
cfg, err := LoadFromEnv("")
|
cfg, err := LoadFromEnv("")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -91,6 +93,9 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
|||||||
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
||||||
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
|
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
|
||||||
}
|
}
|
||||||
|
if cfg.SlowRequestThreshold != 500*time.Millisecond {
|
||||||
|
t.Fatal(cfg.SlowRequestThreshold)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||||
@@ -142,3 +147,23 @@ func TestWithConfigAndWithLogConfig(t *testing.T) {
|
|||||||
t.Fatalf("log config was not replaced: %+v", cfg.Log)
|
t.Fatalf("log config was not replaced: %+v", cfg.Log)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestApplyEnvPreservesProjectDefaults(t *testing.T) {
|
||||||
|
cfg := Default()
|
||||||
|
cfg.Address = "127.0.0.1:8123"
|
||||||
|
cfg.Log.RedactKeys = []string{"bankAccount"}
|
||||||
|
cfg.TrustedProxies = []string{"127.0.0.1"}
|
||||||
|
t.Setenv("OVERLAY_TEST_LOG_LEVEL", "debug")
|
||||||
|
got, err := ApplyEnv(cfg, "OVERLAY_TEST_")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.Address != cfg.Address || got.Log.Level != "debug" {
|
||||||
|
t.Fatal(got)
|
||||||
|
}
|
||||||
|
got.Log.RedactKeys[0] = "changed"
|
||||||
|
got.TrustedProxies[0] = "192.0.2.1"
|
||||||
|
if cfg.Log.RedactKeys[0] != "bankAccount" || cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||||
|
t.Fatal("caller-owned defaults mutated")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -16,6 +16,14 @@ func LoadFromEnv(prefix string) (Config, error) {
|
|||||||
return applyEnv(Default(), prefix)
|
return applyEnv(Default(), prefix)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ApplyEnv overlays nonempty environment values on a caller's defaults or file
|
||||||
|
// settings, then validates the result. Caller-owned slices are not retained.
|
||||||
|
func ApplyEnv(defaults Config, prefix string) (Config, error) {
|
||||||
|
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
|
||||||
|
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
|
||||||
|
return applyEnv(defaults, prefix)
|
||||||
|
}
|
||||||
|
|
||||||
func applyEnv(cfg Config, prefix string) (Config, error) {
|
func applyEnv(cfg Config, prefix string) (Config, error) {
|
||||||
prefix = strings.TrimSpace(prefix)
|
prefix = strings.TrimSpace(prefix)
|
||||||
if prefix == "" {
|
if prefix == "" {
|
||||||
@@ -46,6 +54,12 @@ func applyEnv(cfg Config, prefix string) (Config, error) {
|
|||||||
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
|
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
|
||||||
return Config{}, err
|
return Config{}, err
|
||||||
}
|
}
|
||||||
|
if value := strings.TrimSpace(os.Getenv(prefix + "SLOW_REQUEST_THRESHOLD")); value != "" {
|
||||||
|
cfg.SlowRequestThreshold, err = time.ParseDuration(value)
|
||||||
|
if err != nil {
|
||||||
|
return Config{}, fmt.Errorf("%sSLOW_REQUEST_THRESHOLD must be a duration: %w", prefix, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
|
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
|
||||||
cfg.ShutdownTimeout, err = time.ParseDuration(value)
|
cfg.ShutdownTimeout, err = time.ParseDuration(value)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -182,6 +182,7 @@ func (r *Reloader) Snapshot() Config {
|
|||||||
defer r.mu.Unlock()
|
defer r.mu.Unlock()
|
||||||
cfg := r.current
|
cfg := r.current
|
||||||
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
|
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
|
||||||
|
cfg.Log.RedactKeys = append([]string(nil), cfg.Log.RedactKeys...)
|
||||||
return cfg
|
return cfg
|
||||||
}
|
}
|
||||||
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
|
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
|
||||||
|
|||||||
30
contract/README.md
Normal file
30
contract/README.md
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
# contract
|
||||||
|
|
||||||
|
Generated API projects use this package to compile Go field tags once and bind
|
||||||
|
each request without depending on a specific HTTP server. `New[T]` validates a
|
||||||
|
struct and returns an immutable Binder. `ForType` provides the same compiler to
|
||||||
|
the CLI. `BodySchema`, `Parameters`, and `ResponseSchema` return detached OpenAPI
|
||||||
|
snapshots. `MustNew[T]` is intended for generated, already checked declarations.
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Request struct {
|
||||||
|
ID int64 `path:"id,range=[1:100]"`
|
||||||
|
Name string `json:"name" binding:"required,max=50"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var request = contract.MustNew[Request]()
|
||||||
|
|
||||||
|
func Handle(c *httpx.Context) error {
|
||||||
|
var req Request
|
||||||
|
if err := request.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return c.OK(req.Name)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The default body limit is 1 MiB. An optional positive final argument to Bind
|
||||||
|
overrides it. Application transport read timeouts still control slow clients.
|
||||||
|
See [API.md](../API.md) for source tags, validation rules, null/presence behavior,
|
||||||
|
strict decoding and supported type limits. This API leaves the existing
|
||||||
|
`Context.BindJSON/BindQuery/BindURI` behavior unchanged.
|
||||||
310
contract/bind.go
Normal file
310
contract/bind.go
Normal file
@@ -0,0 +1,310 @@
|
|||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"math"
|
||||||
|
"mime"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"reflect"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Bind fills a fresh DTO transactionally. A JSON body, if declared, is required,
|
||||||
|
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
|
||||||
|
// parameters never overwrite path/body/header values. form means URL query only.
|
||||||
|
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
|
||||||
|
if b == nil || r == nil {
|
||||||
|
return apierrors.Internal(errors.New("contract: nil binder or request"))
|
||||||
|
}
|
||||||
|
v := reflect.ValueOf(target)
|
||||||
|
if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ {
|
||||||
|
return apierrors.Internal(errors.New("contract: target type mismatch"))
|
||||||
|
}
|
||||||
|
limit := DefaultMaxBodyBytes
|
||||||
|
if len(maxBytes) > 1 || (len(maxBytes) == 1 && (maxBytes[0] <= 0 || maxBytes[0] == math.MaxInt64)) {
|
||||||
|
return apierrors.Internal(errors.New("contract: invalid body limit"))
|
||||||
|
}
|
||||||
|
if len(maxBytes) == 1 {
|
||||||
|
limit = maxBytes[0]
|
||||||
|
}
|
||||||
|
fresh := reflect.New(b.typ).Elem()
|
||||||
|
body := map[string]any{}
|
||||||
|
if len(b.root.schema.Properties) > 0 {
|
||||||
|
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
|
||||||
|
if err != nil || ct != "application/json" {
|
||||||
|
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json")
|
||||||
|
}
|
||||||
|
if r.Body == nil {
|
||||||
|
return invalid(errors.New("body required"))
|
||||||
|
}
|
||||||
|
if r.ContentLength > limit {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
reader := &io.LimitedReader{R: r.Body, N: limit + 1}
|
||||||
|
dec := json.NewDecoder(reader)
|
||||||
|
dec.UseNumber()
|
||||||
|
value, err := decodeValue(dec, 0)
|
||||||
|
if reader.N == 0 {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
var maxErr *http.MaxBytesError
|
||||||
|
if errors.As(err, &maxErr) {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
return invalid(err)
|
||||||
|
}
|
||||||
|
_, err = dec.Token()
|
||||||
|
if reader.N == 0 {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
if err != io.EOF {
|
||||||
|
return invalid(errors.New("exactly one JSON value required"))
|
||||||
|
}
|
||||||
|
var ok bool
|
||||||
|
body, ok = value.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return invalid(errors.New("body must be an object"))
|
||||||
|
}
|
||||||
|
for key := range body {
|
||||||
|
if _, ok = b.root.schema.Properties[key]; !ok {
|
||||||
|
return invalid(fmt.Errorf("unknown body field %s", key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
query := url.Values{}
|
||||||
|
if r.URL != nil {
|
||||||
|
var err error
|
||||||
|
query, err = url.ParseQuery(r.URL.RawQuery)
|
||||||
|
if err != nil {
|
||||||
|
return invalid(errors.New("malformed query"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, f := range b.root.fields {
|
||||||
|
var value any
|
||||||
|
var present bool
|
||||||
|
if f.source == "json" {
|
||||||
|
value, present = body[f.name]
|
||||||
|
} else {
|
||||||
|
var raw []string
|
||||||
|
switch f.source {
|
||||||
|
case "path":
|
||||||
|
if s, ok := params[f.name]; ok {
|
||||||
|
raw = []string{s}
|
||||||
|
}
|
||||||
|
case "form":
|
||||||
|
raw = query[f.name]
|
||||||
|
case "header":
|
||||||
|
raw = r.Header.Values(f.name)
|
||||||
|
}
|
||||||
|
present = len(raw) > 0
|
||||||
|
if present {
|
||||||
|
var err error
|
||||||
|
value, err = fromStrings(f.value, raw)
|
||||||
|
if err != nil {
|
||||||
|
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := assignField(f, fresh.Field(f.index), value, present); err != nil {
|
||||||
|
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
v.Elem().Set(fresh)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func invalid(err error) error {
|
||||||
|
return apierrors.BadRequest("INVALID_REQUEST", "request parameters are invalid").WithCause(err)
|
||||||
|
}
|
||||||
|
func tooLarge() error { return apierrors.New(413, "BODY_TOO_LARGE", "request body exceeds limit") }
|
||||||
|
|
||||||
|
func fromStrings(n *node, values []string) (any, error) {
|
||||||
|
n = base(n)
|
||||||
|
if n.typ.Kind() == reflect.Slice {
|
||||||
|
out := make([]any, 0, len(values))
|
||||||
|
for _, s := range values {
|
||||||
|
v, err := parseScalar(s, base(n.elem).typ)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, v)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
if len(values) != 1 {
|
||||||
|
return nil, errors.New("duplicate scalar parameter")
|
||||||
|
}
|
||||||
|
return parseScalar(values[0], n.typ)
|
||||||
|
}
|
||||||
|
func assignField(f field, dst reflect.Value, value any, present bool) error {
|
||||||
|
if !present {
|
||||||
|
if f.hasDefault {
|
||||||
|
value = f.fallback
|
||||||
|
} else if f.optional {
|
||||||
|
return nil
|
||||||
|
} else {
|
||||||
|
return errors.New("required field missing")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := setValue(f.value, dst, value); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return checkField(f, dst)
|
||||||
|
}
|
||||||
|
func setValue(n *node, dst reflect.Value, value any) error {
|
||||||
|
if value == nil {
|
||||||
|
if n.typ.Kind() == reflect.Pointer {
|
||||||
|
dst.SetZero()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return errors.New("null requires a pointer type")
|
||||||
|
}
|
||||||
|
if n.typ.Kind() == reflect.Pointer {
|
||||||
|
dst.Set(reflect.New(n.typ.Elem()))
|
||||||
|
return setValue(n.elem, dst.Elem(), value)
|
||||||
|
}
|
||||||
|
switch n.typ.Kind() {
|
||||||
|
case reflect.Struct:
|
||||||
|
object, ok := value.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected object")
|
||||||
|
}
|
||||||
|
for key := range object {
|
||||||
|
if _, ok = n.schema.Properties[key]; !ok {
|
||||||
|
return fmt.Errorf("unknown field %s", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, f := range n.fields {
|
||||||
|
item, exists := object[f.name]
|
||||||
|
if err := assignField(f, dst.Field(f.index), item, exists); err != nil {
|
||||||
|
return fmt.Errorf("%s: %w", f.name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case reflect.Slice:
|
||||||
|
items, ok := value.([]any)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected array")
|
||||||
|
}
|
||||||
|
dst.Set(reflect.MakeSlice(n.typ, len(items), len(items)))
|
||||||
|
for i, item := range items {
|
||||||
|
if err := setValue(n.elem, dst.Index(i), item); err != nil {
|
||||||
|
return fmt.Errorf("item %d: %w", i, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case reflect.Map:
|
||||||
|
items, ok := value.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected object")
|
||||||
|
}
|
||||||
|
dst.Set(reflect.MakeMapWithSize(n.typ, len(items)))
|
||||||
|
for key, item := range items {
|
||||||
|
v := reflect.New(n.typ.Elem()).Elem()
|
||||||
|
if err := setValue(n.elem, v, item); err != nil {
|
||||||
|
return fmt.Errorf("map value: %w", err)
|
||||||
|
}
|
||||||
|
dst.SetMapIndex(reflect.ValueOf(key).Convert(n.typ.Key()), v)
|
||||||
|
}
|
||||||
|
case reflect.String:
|
||||||
|
s, ok := value.(string)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected string")
|
||||||
|
}
|
||||||
|
dst.SetString(s)
|
||||||
|
case reflect.Bool:
|
||||||
|
v, ok := value.(bool)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected boolean")
|
||||||
|
}
|
||||||
|
dst.SetBool(v)
|
||||||
|
default:
|
||||||
|
num, ok := value.(json.Number)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected number")
|
||||||
|
}
|
||||||
|
switch n.typ.Kind() {
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
v, err := strconv.ParseFloat(string(num), n.typ.Bits())
|
||||||
|
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||||
|
return errors.New("number out of range")
|
||||||
|
}
|
||||||
|
dst.SetFloat(v)
|
||||||
|
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
v, err := strconv.ParseUint(string(num), 10, n.typ.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("invalid unsigned integer")
|
||||||
|
}
|
||||||
|
dst.SetUint(v)
|
||||||
|
default:
|
||||||
|
v, err := strconv.ParseInt(string(num), 10, n.typ.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("invalid integer")
|
||||||
|
}
|
||||||
|
dst.SetInt(v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Token decoding rejects duplicate keys at every level (encoding/json normally
|
||||||
|
// keeps the last value). Depth and body limits bound client-controlled work.
|
||||||
|
func decodeValue(dec *json.Decoder, depth int) (any, error) {
|
||||||
|
if depth > 64 {
|
||||||
|
return nil, errors.New("JSON nesting exceeds 64")
|
||||||
|
}
|
||||||
|
token, err := dec.Token()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
switch token {
|
||||||
|
case json.Delim('{'):
|
||||||
|
object := map[string]any{}
|
||||||
|
for dec.More() {
|
||||||
|
key, err := dec.Token()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
name, ok := key.(string)
|
||||||
|
if !ok {
|
||||||
|
return nil, errors.New("invalid object key")
|
||||||
|
}
|
||||||
|
if _, exists := object[name]; exists {
|
||||||
|
return nil, errors.New("duplicate JSON field")
|
||||||
|
}
|
||||||
|
value, err := decodeValue(dec, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
object[name] = value
|
||||||
|
}
|
||||||
|
end, err := dec.Token()
|
||||||
|
if err != nil || end != json.Delim('}') {
|
||||||
|
return nil, errors.New("unterminated object")
|
||||||
|
}
|
||||||
|
return object, nil
|
||||||
|
case json.Delim('['):
|
||||||
|
items := []any{}
|
||||||
|
for dec.More() {
|
||||||
|
v, err := decodeValue(dec, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
items = append(items, v)
|
||||||
|
}
|
||||||
|
end, err := dec.Token()
|
||||||
|
if err != nil || end != json.Delim(']') {
|
||||||
|
return nil, errors.New("unterminated array")
|
||||||
|
}
|
||||||
|
return items, nil
|
||||||
|
default:
|
||||||
|
if _, bad := token.(json.Delim); bad {
|
||||||
|
return nil, errors.New("unexpected JSON delimiter")
|
||||||
|
}
|
||||||
|
return token, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
394
contract/contract.go
Normal file
394
contract/contract.go
Normal file
@@ -0,0 +1,394 @@
|
|||||||
|
// Package contract compiles API field tags into a reusable request binder and
|
||||||
|
// OpenAPI schemas. Generated services and the CLI use the same compiler.
|
||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"reflect"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
"github.com/go-playground/validator/v10"
|
||||||
|
)
|
||||||
|
|
||||||
|
const DefaultMaxBodyBytes int64 = 1 << 20
|
||||||
|
|
||||||
|
var wireName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||||
|
var headerName = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9-]*$`)
|
||||||
|
var valuesValidator = validator.New()
|
||||||
|
|
||||||
|
// Binder is immutable after compilation and safe for concurrent requests.
|
||||||
|
type Binder struct {
|
||||||
|
root *node
|
||||||
|
typ reflect.Type
|
||||||
|
}
|
||||||
|
type node struct {
|
||||||
|
typ reflect.Type
|
||||||
|
schema *openapi.Schema
|
||||||
|
fields []field
|
||||||
|
elem *node
|
||||||
|
}
|
||||||
|
type field struct {
|
||||||
|
index int
|
||||||
|
name, source string
|
||||||
|
value *node
|
||||||
|
optional bool
|
||||||
|
fallback any
|
||||||
|
hasDefault bool
|
||||||
|
binding string
|
||||||
|
}
|
||||||
|
|
||||||
|
func New[T any]() (*Binder, error) { return ForType(reflect.TypeOf((*T)(nil)).Elem()) }
|
||||||
|
func MustNew[T any]() *Binder {
|
||||||
|
b, err := New[T]()
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
// ForType also lets generators check the exact runtime contract before writing files.
|
||||||
|
// The root must be a struct; recursive and custom-marshaling types are rejected.
|
||||||
|
func ForType(t reflect.Type) (*Binder, error) {
|
||||||
|
if t == nil || t.Kind() != reflect.Struct {
|
||||||
|
return nil, errors.New("contract: root must be a struct")
|
||||||
|
}
|
||||||
|
budget := 10000
|
||||||
|
n, err := compile(t, true, map[reflect.Type]bool{}, 0, &budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &Binder{root: n, typ: t}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int, budget *int) (*node, error) {
|
||||||
|
*budget--
|
||||||
|
if *budget < 0 {
|
||||||
|
return nil, errors.New("contract: schema expansion exceeds 10000 nodes")
|
||||||
|
}
|
||||||
|
if depth > 32 {
|
||||||
|
return nil, errors.New("contract: type nesting exceeds 32")
|
||||||
|
}
|
||||||
|
if stack[t] {
|
||||||
|
return nil, errors.New("contract: recursive types are not supported")
|
||||||
|
}
|
||||||
|
stack[t] = true
|
||||||
|
defer delete(stack, t)
|
||||||
|
n := &node{typ: t, schema: &openapi.Schema{}}
|
||||||
|
// Wire behavior must be described completely by these tags.
|
||||||
|
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||||
|
unmarshaler := reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
|
||||||
|
if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler) {
|
||||||
|
return nil, errors.New("contract: custom JSON marshalers are unsupported")
|
||||||
|
}
|
||||||
|
switch t.Kind() {
|
||||||
|
case reflect.Pointer:
|
||||||
|
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
n.elem = e
|
||||||
|
*n.schema = *e.schema
|
||||||
|
n.schema.Nullable = true
|
||||||
|
case reflect.String:
|
||||||
|
n.schema.Type = "string"
|
||||||
|
case reflect.Bool:
|
||||||
|
n.schema.Type = "boolean"
|
||||||
|
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
n.schema.Type = "integer"
|
||||||
|
if t.Bits() <= 32 {
|
||||||
|
n.schema.Format = "int32"
|
||||||
|
} else {
|
||||||
|
n.schema.Format = "int64"
|
||||||
|
}
|
||||||
|
if t.Kind() >= reflect.Uint && t.Kind() <= reflect.Uint64 {
|
||||||
|
v := float64(0)
|
||||||
|
n.schema.Minimum = &v
|
||||||
|
if t.Bits() < 64 {
|
||||||
|
max := float64((uint64(1) << uint(t.Bits())) - 1)
|
||||||
|
n.schema.Maximum = &max
|
||||||
|
if t.Bits() == 32 {
|
||||||
|
n.schema.Format = "int64"
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
n.schema.Format = ""
|
||||||
|
}
|
||||||
|
} else if t.Bits() < 32 {
|
||||||
|
min, max := -float64(int64(1)<<uint(t.Bits()-1)), float64((int64(1)<<uint(t.Bits()-1))-1)
|
||||||
|
n.schema.Minimum, n.schema.Maximum = &min, &max
|
||||||
|
}
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
n.schema.Type = "number"
|
||||||
|
if t.Kind() == reflect.Float32 {
|
||||||
|
n.schema.Format = "float"
|
||||||
|
} else {
|
||||||
|
n.schema.Format = "double"
|
||||||
|
}
|
||||||
|
case reflect.Slice, reflect.Map:
|
||||||
|
if t.Kind() == reflect.Map && t.Key().Kind() != reflect.String {
|
||||||
|
return nil, errors.New("contract: map keys must be strings")
|
||||||
|
}
|
||||||
|
if t.Kind() == reflect.Slice && t.Elem().Kind() == reflect.Uint8 {
|
||||||
|
return nil, errors.New("contract: byte slices use base64 in Go; use string for binary data")
|
||||||
|
}
|
||||||
|
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
n.elem = e
|
||||||
|
if t.Kind() == reflect.Slice {
|
||||||
|
n.schema.Type = "array"
|
||||||
|
n.schema.Items = e.schema
|
||||||
|
} else {
|
||||||
|
n.schema.Type = "object"
|
||||||
|
n.schema.AdditionalProperties = e.schema
|
||||||
|
}
|
||||||
|
case reflect.Struct:
|
||||||
|
n.schema.Type = "object"
|
||||||
|
allowed := false
|
||||||
|
n.schema.AdditionalPropertiesAllowed = &allowed
|
||||||
|
n.schema.Properties = map[string]*openapi.Schema{}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for i := 0; i < t.NumField(); i++ {
|
||||||
|
sf := t.Field(i)
|
||||||
|
if sf.PkgPath != "" || sf.Anonymous {
|
||||||
|
return nil, fmt.Errorf("contract: %s must be an exported named field", sf.Name)
|
||||||
|
}
|
||||||
|
f, err := compileField(sf, root, stack, depth+1, budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("contract: %s: %w", sf.Name, err)
|
||||||
|
}
|
||||||
|
f.index = i
|
||||||
|
key := f.source + ":" + strings.ToLower(f.name)
|
||||||
|
if seen[key] {
|
||||||
|
return nil, fmt.Errorf("contract: duplicate field %s", f.name)
|
||||||
|
}
|
||||||
|
seen[key] = true
|
||||||
|
n.fields = append(n.fields, f)
|
||||||
|
if f.source == "json" {
|
||||||
|
n.schema.Properties[f.name] = f.value.schema
|
||||||
|
if !f.optional && !f.hasDefault {
|
||||||
|
n.schema.Required = append(n.schema.Required, f.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("contract: unsupported type %s", t)
|
||||||
|
}
|
||||||
|
return n, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool, depth int, budget *int) (field, error) {
|
||||||
|
f := field{}
|
||||||
|
// Reject typos and unsupported tags instead of silently losing validation.
|
||||||
|
tags, err := ParseTags(string(sf.Tag))
|
||||||
|
if err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
var sourceTag string
|
||||||
|
for key, value := range tags {
|
||||||
|
switch key {
|
||||||
|
case "json", "path", "form", "header":
|
||||||
|
if value == "-" {
|
||||||
|
return f, errors.New("ignored fields are not part of an API contract")
|
||||||
|
}
|
||||||
|
if f.source != "" {
|
||||||
|
return f, errors.New("exactly one source tag is required")
|
||||||
|
}
|
||||||
|
f.source = key
|
||||||
|
sourceTag = value
|
||||||
|
case "binding":
|
||||||
|
f.binding = value
|
||||||
|
default:
|
||||||
|
return f, fmt.Errorf("unsupported tag %q", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if f.source == "" {
|
||||||
|
return f, errors.New("json, path, form or header tag required")
|
||||||
|
}
|
||||||
|
if !root && f.source != "json" {
|
||||||
|
return f, errors.New("nested fields must use json tags")
|
||||||
|
}
|
||||||
|
parts := strings.Split(sourceTag, ",")
|
||||||
|
f.name = parts[0]
|
||||||
|
if (f.source == "header" && !headerName.MatchString(f.name)) || (f.source != "header" && !wireName.MatchString(f.name)) {
|
||||||
|
return f, errors.New("wire names must be lowerCamelCase (headers use HTTP names)")
|
||||||
|
}
|
||||||
|
n, err := compile(sf.Type, false, stack, depth, budget)
|
||||||
|
if err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
f.value = n
|
||||||
|
scalar := base(n).typ.Kind()
|
||||||
|
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
|
||||||
|
return f, errors.New("path/query/header values must be scalars or scalar slices")
|
||||||
|
}
|
||||||
|
if f.source == "path" && !isScalar(scalar) {
|
||||||
|
return f, errors.New("path values must be scalars")
|
||||||
|
}
|
||||||
|
if f.source == "header" && !isScalar(scalar) {
|
||||||
|
return f, errors.New("header values must be scalars")
|
||||||
|
}
|
||||||
|
options := map[string]bool{}
|
||||||
|
for _, option := range parts[1:] {
|
||||||
|
key, value, _ := strings.Cut(option, "=")
|
||||||
|
if options[key] {
|
||||||
|
return f, fmt.Errorf("duplicate option %q", key)
|
||||||
|
}
|
||||||
|
options[key] = true
|
||||||
|
switch key {
|
||||||
|
case "optional", "omitempty":
|
||||||
|
if option != key {
|
||||||
|
return f, errors.New("optional takes no value")
|
||||||
|
}
|
||||||
|
f.optional = true
|
||||||
|
case "default":
|
||||||
|
if !isScalar(scalar) {
|
||||||
|
return f, errors.New("default requires a scalar")
|
||||||
|
}
|
||||||
|
v, e := parseScalar(value, base(n).typ)
|
||||||
|
if e != nil {
|
||||||
|
return f, fmt.Errorf("invalid default: %w", e)
|
||||||
|
}
|
||||||
|
f.fallback = v
|
||||||
|
f.hasDefault = true
|
||||||
|
n.schema.Default = v
|
||||||
|
case "options":
|
||||||
|
if !isScalar(scalar) || value == "" {
|
||||||
|
return f, errors.New("options requires scalar values")
|
||||||
|
}
|
||||||
|
for _, s := range strings.Split(value, "|") {
|
||||||
|
v, e := parseScalar(s, base(n).typ)
|
||||||
|
if e != nil {
|
||||||
|
return f, e
|
||||||
|
}
|
||||||
|
n.schema.Enum = append(n.schema.Enum, v)
|
||||||
|
}
|
||||||
|
case "range":
|
||||||
|
if !isNumber(scalar) {
|
||||||
|
return f, errors.New("range requires numeric type")
|
||||||
|
}
|
||||||
|
if err = applyRange(n.schema, value); err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return f, fmt.Errorf("unsupported field option %q", option)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err = applyBinding(&f); err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
if n.schema.Nullable && len(n.schema.Enum) > 0 {
|
||||||
|
n.schema.Enum = append(n.schema.Enum, nil)
|
||||||
|
}
|
||||||
|
if f.source == "path" && (f.optional || f.hasDefault || sf.Type.Kind() == reflect.Pointer) {
|
||||||
|
return f, errors.New("path fields must be required non-pointer values")
|
||||||
|
}
|
||||||
|
if f.hasDefault {
|
||||||
|
v := reflect.New(sf.Type).Elem()
|
||||||
|
if err = setValue(n, v, f.fallback); err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
if err = checkField(f, v); err != nil {
|
||||||
|
return f, fmt.Errorf("default violates validation: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return f, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseTags strictly parses Go struct tags, including duplicate/unknown key checks
|
||||||
|
// performed by the compiler. It is also used to normalize generated Go JSON tags.
|
||||||
|
func ParseTags(raw string) (map[string]string, error) {
|
||||||
|
out := map[string]string{}
|
||||||
|
for strings.TrimSpace(raw) != "" {
|
||||||
|
raw = strings.TrimLeft(raw, " ")
|
||||||
|
i := strings.Index(raw, ":")
|
||||||
|
if i < 1 {
|
||||||
|
return nil, errors.New("invalid struct tag")
|
||||||
|
}
|
||||||
|
key := raw[:i]
|
||||||
|
raw = raw[i+1:]
|
||||||
|
if strings.ContainsAny(key, " \t\r\n\"`") || len(raw) == 0 || raw[0] != '"' {
|
||||||
|
return nil, errors.New("invalid struct tag")
|
||||||
|
}
|
||||||
|
end := 1
|
||||||
|
for end < len(raw) {
|
||||||
|
if raw[end] == '\\' {
|
||||||
|
end += 2
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if raw[end] == '"' {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
end++
|
||||||
|
}
|
||||||
|
if end >= len(raw) {
|
||||||
|
return nil, errors.New("unterminated struct tag")
|
||||||
|
}
|
||||||
|
v, err := strconv.Unquote(raw[:end+1])
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if _, ok := out[key]; ok {
|
||||||
|
return nil, fmt.Errorf("duplicate tag %q", key)
|
||||||
|
}
|
||||||
|
out[key] = v
|
||||||
|
raw = raw[end+1:]
|
||||||
|
if raw != "" && raw[0] != ' ' {
|
||||||
|
return nil, errors.New("struct tags require spaces")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func base(n *node) *node {
|
||||||
|
for n.typ.Kind() == reflect.Pointer {
|
||||||
|
n = n.elem
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
func isNumber(k reflect.Kind) bool {
|
||||||
|
return k >= reflect.Int && k <= reflect.Float64 && k != reflect.Uintptr
|
||||||
|
}
|
||||||
|
func isScalar(k reflect.Kind) bool { return k == reflect.String || k == reflect.Bool || isNumber(k) }
|
||||||
|
|
||||||
|
func (b *Binder) BodySchema() *openapi.Schema {
|
||||||
|
if len(b.root.schema.Properties) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return copySchema(b.root.schema)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResponseSchema rejects request-only fields to keep response docs honest.
|
||||||
|
func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
|
||||||
|
for _, f := range b.root.fields {
|
||||||
|
if f.source != "json" {
|
||||||
|
return nil, errors.New("contract: response fields must use json tags")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return copySchema(b.root.schema), nil
|
||||||
|
}
|
||||||
|
func (b *Binder) Parameters() []openapi.Parameter {
|
||||||
|
var out []openapi.Parameter
|
||||||
|
for _, f := range b.root.fields {
|
||||||
|
if f.source == "json" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
in := f.source
|
||||||
|
if in == "form" {
|
||||||
|
in = "query"
|
||||||
|
}
|
||||||
|
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: copySchema(f.value.schema)})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
func copySchema(s *openapi.Schema) *openapi.Schema {
|
||||||
|
data, _ := json.Marshal(s)
|
||||||
|
var out openapi.Schema
|
||||||
|
_ = json.Unmarshal(data, &out)
|
||||||
|
return &out
|
||||||
|
}
|
||||||
244
contract/contract_test.go
Normal file
244
contract/contract_test.go
Normal file
@@ -0,0 +1,244 @@
|
|||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net/http/httptest"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
)
|
||||||
|
|
||||||
|
type address struct {
|
||||||
|
City string `json:"city" binding:"required,max=30"`
|
||||||
|
}
|
||||||
|
type mixed struct {
|
||||||
|
ID int64 `path:"userId,range=[1:100]"`
|
||||||
|
Page int `form:"page,default=1,range=[1:50]"`
|
||||||
|
Tags []string `form:"tag,optional" binding:"max=3"`
|
||||||
|
Client string `header:"X-Client"`
|
||||||
|
Name string `json:"name" binding:"required,max=20"`
|
||||||
|
Active bool `json:"active"`
|
||||||
|
Count *int `json:"count,optional"`
|
||||||
|
Contact address `json:"contact"`
|
||||||
|
Items []address `json:"items,optional"`
|
||||||
|
Labels map[string]address `json:"labels,optional"`
|
||||||
|
}
|
||||||
|
|
||||||
|
const validBody = `{"name":"小明","active":false,"count":0,"contact":{"city":"杭州"}}`
|
||||||
|
|
||||||
|
func bindMixed(body, query string) (mixed, error) {
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
r := httptest.NewRequest("POST", "/users/7"+query, strings.NewReader(body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
var v mixed
|
||||||
|
return v, b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||||
|
}
|
||||||
|
func TestMixedSourcesAndPresence(t *testing.T) {
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
r := httptest.NewRequest("POST", "/users/7?tag=a&tag=b&userId=99&name=query", strings.NewReader(validBody))
|
||||||
|
r.Header.Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
v := mixed{Name: "previous"}
|
||||||
|
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if v.ID != 7 || v.Page != 1 || v.Name != "小明" || v.Active || v.Count == nil || *v.Count != 0 || len(v.Tags) != 2 || v.Client != "web" || v.Contact.City != "杭州" {
|
||||||
|
t.Fatalf("bad binding: %+v", v)
|
||||||
|
}
|
||||||
|
r = httptest.NewRequest("POST", "/", strings.NewReader(strings.Replace(validBody, `,"count":0`, "", 1)))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil || v.Count != nil {
|
||||||
|
t.Fatalf("missing != zero: %+v %v", v, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRejectInvalidRequests(t *testing.T) {
|
||||||
|
cases := []struct{ name, body, query string }{
|
||||||
|
{"missing", `{"name":"a","contact":{"city":"杭州"}}`, ""},
|
||||||
|
{"empty", strings.Replace(validBody, `"小明"`, `""`, 1), ""},
|
||||||
|
{"caseSensitive", strings.Replace(validBody, `"name"`, `"Name"`, 1), ""},
|
||||||
|
{"unknown", strings.Replace(validBody, `"active":false`, `"active":false,"extra":1`, 1), ""},
|
||||||
|
{"duplicate", strings.Replace(validBody, `"active":false`, `"active":false,"active":true`, 1), ""},
|
||||||
|
{"nestedDuplicate", strings.Replace(validBody, `"city":"杭州"`, `"city":"杭州","city":"北京"`, 1), ""},
|
||||||
|
{"nestedRequired", strings.Replace(validBody, `{"city":"杭州"}`, `{}`, 1), ""},
|
||||||
|
{"nestedUnknown", strings.Replace(validBody, `{"city":"杭州"}`, `{"city":"杭州","other":0}`, 1), ""},
|
||||||
|
{"trailing", validBody + ` {}`, ""},
|
||||||
|
{"null", strings.Replace(validBody, `false`, `null`, 1), ""},
|
||||||
|
{"numberOverflow", strings.Replace(validBody, `"count":0`, `"count":99999999999999999999`, 1), ""},
|
||||||
|
{"arrayValidation", strings.TrimSuffix(validBody, "}") + `,"items":[{"city":""}]}`, ""},
|
||||||
|
{"mapValidation", strings.TrimSuffix(validBody, "}") + `,"labels":{"home":{"city":""}}}`, ""},
|
||||||
|
{"queryDuplicate", validBody, "?page=1&page=2"},
|
||||||
|
{"queryDefaultNotOnEmpty", validBody, "?page="},
|
||||||
|
{"range", validBody, "?page=51"},
|
||||||
|
{"malformedQuery", validBody, "?page=%zz"},
|
||||||
|
{"arrayLimit", validBody, "?tag=a&tag=b&tag=c&tag=d"},
|
||||||
|
}
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
r := httptest.NewRequest("POST", "/"+tc.query, strings.NewReader(tc.body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
v := mixed{Name: "unchanged"}
|
||||||
|
err := b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||||
|
if err == nil || apierrors.From(err).Status != 400 {
|
||||||
|
t.Fatalf("wanted 400: %v", err)
|
||||||
|
}
|
||||||
|
if v.Name != "unchanged" || v.Page != 0 {
|
||||||
|
t.Fatal("failed binding modified target")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestBodyLimitsAndMediaType(t *testing.T) {
|
||||||
|
type request struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
b := MustNew[request]()
|
||||||
|
for _, tc := range []struct {
|
||||||
|
body, ct string
|
||||||
|
limit int64
|
||||||
|
status int
|
||||||
|
}{
|
||||||
|
{`{"name":"x"}`, "text/plain", 100, 415},
|
||||||
|
{`{"name":"long"}`, "application/json", 10, 413},
|
||||||
|
{`{"name":"x"}` + strings.Repeat(" ", 2048), "application/json", 1024, 413},
|
||||||
|
{`{"name":"x"}`, "application/json", 12, 0},
|
||||||
|
} {
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||||
|
r.ContentLength = -1
|
||||||
|
r.Header.Set("Content-Type", tc.ct)
|
||||||
|
var req request
|
||||||
|
err := b.Bind(r, nil, &req, tc.limit)
|
||||||
|
if tc.status == 0 {
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
} else if err == nil || apierrors.From(err).Status != tc.status {
|
||||||
|
t.Fatalf("status %d wanted: %v", tc.status, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestSchemaUsesSameRules(t *testing.T) {
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
s := b.BodySchema()
|
||||||
|
if len(s.Required) != 3 || s.Properties["count"].Nullable != true || s.Properties["name"].MaxLength == nil || *s.Properties["name"].MaxLength != 20 {
|
||||||
|
t.Fatalf("wrong schema: %+v", s)
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(s)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(encoded), `"additionalProperties":false`) {
|
||||||
|
t.Fatal(string(encoded))
|
||||||
|
}
|
||||||
|
params := b.Parameters()
|
||||||
|
if len(params) != 4 || params[1].Required || params[1].Schema.Default != json.Number("1") {
|
||||||
|
t.Fatalf("wrong parameters: %+v", params)
|
||||||
|
}
|
||||||
|
// Returned documentation snapshots do not mutate concurrent binders.
|
||||||
|
s.Properties["name"].MaxLength = nil
|
||||||
|
if b.BodySchema().Properties["name"].MaxLength == nil {
|
||||||
|
t.Fatal("shared schema")
|
||||||
|
}
|
||||||
|
if _, err = b.ResponseSchema(); err == nil {
|
||||||
|
t.Fatal("request sources in response accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRulesAndConcurrentReuse(t *testing.T) {
|
||||||
|
type req struct {
|
||||||
|
Score float64 `json:"score,options=1.0|2,range=(0:2]"`
|
||||||
|
Email string `json:"email" binding:"email"`
|
||||||
|
Value *int `json:"value" binding:"required"`
|
||||||
|
}
|
||||||
|
b := MustNew[req]()
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 24; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"score":1,"email":"a@example.com","value":3}`))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
var v req
|
||||||
|
if err := b.Bind(r, nil, &v); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
for _, body := range []string{`{"score":0,"email":"a@example.com","value":1}`, `{"score":2,"email":"bad","value":1}`, `{"score":2,"email":"a@example.com","value":null}`, `{"score":2,"email":"a@example.com","value":0}`} {
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
var v req
|
||||||
|
if err := b.Bind(r, nil, &v); err == nil {
|
||||||
|
t.Fatal(body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestInvalidContracts(t *testing.T) {
|
||||||
|
for _, tag := range []string{`json:"bad_name"`, `json:"a" form:"a"`, `json:"a,optionl"`, `json:"a,range=[5:2]"`, `json:"a,default=20,range=[1:10]"`, `json:"a,optional" binding:"required"`, `json:"a" binding:"unknown"`, `path:"a,optional"`, `json:"a" json:"b"`, `jsoon:"a"`, `json:"a,options=x"`} {
|
||||||
|
typ := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: reflect.TypeOf(0), Tag: reflect.StructTag(tag)}})
|
||||||
|
if _, err := ForType(typ); err == nil {
|
||||||
|
t.Fatalf("accepted %s", tag)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExactLargeIntegerConstraints(t *testing.T) {
|
||||||
|
type req struct {
|
||||||
|
Value int64 `json:"value,range=[:9007199254740992]"`
|
||||||
|
}
|
||||||
|
b := MustNew[req]()
|
||||||
|
for _, tc := range []struct {
|
||||||
|
body string
|
||||||
|
valid bool
|
||||||
|
}{{`{"value":9007199254740992}`, true}, {`{"value":9007199254740993}`, false}} {
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
var v req
|
||||||
|
if err := b.Bind(r, nil, &v); (err == nil) != tc.valid {
|
||||||
|
t.Fatalf("%s: %v", tc.body, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
type defaults struct {
|
||||||
|
Value uint64 `form:"value,default=18446744073709551615,options=18446744073709551615"`
|
||||||
|
}
|
||||||
|
d := MustNew[defaults]()
|
||||||
|
data, err := json.Marshal(d.Parameters())
|
||||||
|
if err != nil || !strings.Contains(string(data), "18446744073709551615") {
|
||||||
|
t.Fatalf("lost precision: %s %v", data, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIntrinsicIntegerBounds(t *testing.T) {
|
||||||
|
type req struct {
|
||||||
|
Small uint8 `json:"small,range=[-1:500]"`
|
||||||
|
Wide uint64 `json:"wide"`
|
||||||
|
}
|
||||||
|
b := MustNew[req]()
|
||||||
|
s := b.BodySchema()
|
||||||
|
if *s.Properties["small"].Minimum != 0 || *s.Properties["small"].Maximum != 255 || s.Properties["wide"].Format != "" {
|
||||||
|
t.Fatalf("wrong integer bounds: %+v", s.Properties)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func FuzzStrictBody(f *testing.F) {
|
||||||
|
for _, seed := range []string{validBody, `{}`, `{"name":"a","name":"b"}`, `[null]`, `{"x":[]}`} {
|
||||||
|
f.Add(seed)
|
||||||
|
}
|
||||||
|
binder := MustNew[mixed]()
|
||||||
|
f.Fuzz(func(t *testing.T, body string) {
|
||||||
|
if len(body) > 16384 {
|
||||||
|
t.Skip()
|
||||||
|
}
|
||||||
|
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
req.Header.Set("X-Client", "fuzz")
|
||||||
|
var value mixed
|
||||||
|
_ = binder.Bind(req, map[string]string{"userId": "1"}, &value, 16384)
|
||||||
|
})
|
||||||
|
}
|
||||||
259
contract/rules.go
Normal file
259
contract/rules.go
Normal file
@@ -0,0 +1,259 @@
|
|||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"math"
|
||||||
|
"reflect"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
)
|
||||||
|
|
||||||
|
func applyRange(s *openapi.Schema, value string) error {
|
||||||
|
if len(value) < 3 || (value[0] != '[' && value[0] != '(') || (value[len(value)-1] != ']' && value[len(value)-1] != ')') {
|
||||||
|
return errors.New("invalid range: use [min:max], (min:max), or mixed bounds")
|
||||||
|
}
|
||||||
|
lo, hi, ok := strings.Cut(value[1:len(value)-1], ":")
|
||||||
|
if !ok || (lo == "" && hi == "") {
|
||||||
|
return errors.New("invalid numeric range")
|
||||||
|
}
|
||||||
|
if lo != "" {
|
||||||
|
v, err := bound(lo)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if s.Minimum == nil || v > *s.Minimum || (v == *s.Minimum && value[0] == '(') {
|
||||||
|
s.Minimum = &v
|
||||||
|
s.ExclusiveMinimum = value[0] == '('
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if hi != "" {
|
||||||
|
v, err := bound(hi)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if s.Maximum == nil || v < *s.Maximum || (v == *s.Maximum && value[len(value)-1] == ')') {
|
||||||
|
s.Maximum = &v
|
||||||
|
s.ExclusiveMaximum = value[len(value)-1] == ')'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return checkBounds(s)
|
||||||
|
}
|
||||||
|
func bound(s string) (float64, error) {
|
||||||
|
v, err := strconv.ParseFloat(s, 64)
|
||||||
|
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) || math.Abs(v) > 1<<53 {
|
||||||
|
return 0, errors.New("constraint bound must be finite and within ±2^53")
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
func checkBounds(s *openapi.Schema) error {
|
||||||
|
if s.Minimum != nil && s.Maximum != nil && (*s.Minimum > *s.Maximum || (*s.Minimum == *s.Maximum && (s.ExclusiveMinimum || s.ExclusiveMaximum))) {
|
||||||
|
return errors.New("empty numeric range")
|
||||||
|
}
|
||||||
|
if s.MinLength != nil && s.MaxLength != nil && *s.MinLength > *s.MaxLength {
|
||||||
|
return errors.New("empty string length range")
|
||||||
|
}
|
||||||
|
if s.MinItems != nil && s.MaxItems != nil && *s.MinItems > *s.MaxItems {
|
||||||
|
return errors.New("empty array length range")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func applyBinding(f *field) error {
|
||||||
|
if f.binding == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
n := base(f.value)
|
||||||
|
k := n.typ.Kind()
|
||||||
|
s := f.value.schema
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, rule := range strings.Split(f.binding, ",") {
|
||||||
|
name, value, _ := strings.Cut(rule, "=")
|
||||||
|
if seen[name] {
|
||||||
|
return fmt.Errorf("duplicate binding rule %s", name)
|
||||||
|
}
|
||||||
|
seen[name] = true
|
||||||
|
switch name {
|
||||||
|
case "required":
|
||||||
|
if rule != name || f.optional || f.hasDefault {
|
||||||
|
return errors.New("required conflicts with optional/default")
|
||||||
|
}
|
||||||
|
if !isScalar(k) {
|
||||||
|
return errors.New("binding required supports scalars; other fields are already required by presence")
|
||||||
|
}
|
||||||
|
s.Nullable = false
|
||||||
|
switch k {
|
||||||
|
case reflect.String:
|
||||||
|
one := 1
|
||||||
|
s.MinLength = &one
|
||||||
|
case reflect.Bool:
|
||||||
|
s.Not = &openapi.Schema{Enum: []any{false}}
|
||||||
|
default:
|
||||||
|
s.Not = &openapi.Schema{Enum: []any{0}}
|
||||||
|
}
|
||||||
|
case "email":
|
||||||
|
if rule != name || k != reflect.String {
|
||||||
|
return errors.New("email requires a string")
|
||||||
|
}
|
||||||
|
s.Format = "email"
|
||||||
|
case "min", "max", "len":
|
||||||
|
if k == reflect.String || k == reflect.Slice {
|
||||||
|
v, err := strconv.Atoi(value)
|
||||||
|
if err != nil || v < 0 {
|
||||||
|
return errors.New("length constraints must be nonnegative integers")
|
||||||
|
}
|
||||||
|
if k == reflect.String {
|
||||||
|
if name == "min" || name == "len" {
|
||||||
|
if s.MinLength == nil || v > *s.MinLength {
|
||||||
|
s.MinLength = &v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if name == "max" || name == "len" {
|
||||||
|
s.MaxLength = &v
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if name == "min" || name == "len" {
|
||||||
|
s.MinItems = &v
|
||||||
|
}
|
||||||
|
if name == "max" || name == "len" {
|
||||||
|
s.MaxItems = &v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if isNumber(k) && name != "len" {
|
||||||
|
v, err := bound(value)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if name == "min" {
|
||||||
|
if s.Minimum == nil || v > *s.Minimum {
|
||||||
|
s.Minimum = &v
|
||||||
|
s.ExclusiveMinimum = false
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if s.Maximum == nil || v < *s.Maximum {
|
||||||
|
s.Maximum = &v
|
||||||
|
s.ExclusiveMaximum = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
return fmt.Errorf("%s requires a number, string or slice", name)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("unsupported binding rule %q; use required,min,max,len,email or source-tag options/range", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return checkBounds(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseScalar(raw string, t reflect.Type) (any, error) {
|
||||||
|
switch t.Kind() {
|
||||||
|
case reflect.String:
|
||||||
|
return raw, nil
|
||||||
|
case reflect.Bool:
|
||||||
|
if raw == "true" {
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
if raw == "false" {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
return nil, errors.New("expected true or false")
|
||||||
|
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
|
||||||
|
v, err := strconv.ParseInt(raw, 10, t.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New("invalid integer")
|
||||||
|
}
|
||||||
|
return json.Number(strconv.FormatInt(v, 10)), nil
|
||||||
|
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
v, err := strconv.ParseUint(raw, 10, t.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New("invalid unsigned integer")
|
||||||
|
}
|
||||||
|
return json.Number(strconv.FormatUint(v, 10)), nil
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
v, err := strconv.ParseFloat(raw, t.Bits())
|
||||||
|
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||||
|
return nil, errors.New("invalid finite number")
|
||||||
|
}
|
||||||
|
return json.Number(strconv.FormatFloat(v, 'g', -1, t.Bits())), nil
|
||||||
|
}
|
||||||
|
return nil, errors.New("expected scalar")
|
||||||
|
}
|
||||||
|
|
||||||
|
func checkField(f field, v reflect.Value) error {
|
||||||
|
if v.Kind() == reflect.Pointer {
|
||||||
|
if v.IsNil() {
|
||||||
|
if strings.Contains(","+f.binding+",", ",required,") {
|
||||||
|
return errors.New("must not be null")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
v = v.Elem()
|
||||||
|
}
|
||||||
|
if strings.Contains(","+f.binding+",", ",required,") && v.IsZero() {
|
||||||
|
return errors.New("must not be zero or empty")
|
||||||
|
}
|
||||||
|
s := f.value.schema
|
||||||
|
if s.Format == "email" && valuesValidator.Var(v.Interface(), "email") != nil {
|
||||||
|
return errors.New("invalid email")
|
||||||
|
}
|
||||||
|
if len(s.Enum) > 0 {
|
||||||
|
found := false
|
||||||
|
for _, item := range s.Enum {
|
||||||
|
option := reflect.New(v.Type()).Elem()
|
||||||
|
if setValue(base(f.value), option, item) == nil && reflect.DeepEqual(option.Interface(), v.Interface()) {
|
||||||
|
found = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
return errors.New("not an allowed option")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if isNumber(v.Kind()) {
|
||||||
|
var number float64
|
||||||
|
switch v.Kind() {
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
number = v.Float()
|
||||||
|
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
number = float64(v.Uint())
|
||||||
|
if v.Uint() > 1<<53 {
|
||||||
|
number = math.Inf(1)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
number = float64(v.Int())
|
||||||
|
if v.Int() > 1<<53 {
|
||||||
|
number = math.Inf(1)
|
||||||
|
} else if v.Int() < -(1 << 53) {
|
||||||
|
number = math.Inf(-1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s.Minimum != nil && (number < *s.Minimum || (s.ExclusiveMinimum && number == *s.Minimum)) {
|
||||||
|
return errors.New("below minimum")
|
||||||
|
}
|
||||||
|
if s.Maximum != nil && (number > *s.Maximum || (s.ExclusiveMaximum && number == *s.Maximum)) {
|
||||||
|
return errors.New("above maximum")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v.Kind() == reflect.String {
|
||||||
|
size := utf8.RuneCountInString(v.String())
|
||||||
|
if s.MinLength != nil && size < *s.MinLength {
|
||||||
|
return errors.New("string too short")
|
||||||
|
}
|
||||||
|
if s.MaxLength != nil && size > *s.MaxLength {
|
||||||
|
return errors.New("string too long")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v.Kind() == reflect.Slice {
|
||||||
|
size := v.Len()
|
||||||
|
if s.MinItems != nil && size < *s.MinItems {
|
||||||
|
return errors.New("too few items")
|
||||||
|
}
|
||||||
|
if s.MaxItems != nil && size > *s.MaxItems {
|
||||||
|
return errors.New("too many items")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
35
database/configOverlay_test.go
Normal file
35
database/configOverlay_test.go
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
package database
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestApplyEnvPreservesFilePoolAndHidesDSN(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.MaxOpenConns = 3
|
||||||
|
cfg.MaxIdleConns = 1
|
||||||
|
cfg.PingTimeout = time.Second
|
||||||
|
t.Setenv("OMNI_TEST_DSN", "user:secret@tcp(localhost:3306)/test")
|
||||||
|
t.Setenv("OMNI_TEST_MAX_IDLE_CONNS", "0")
|
||||||
|
got, err := ApplyEnv(cfg, "OMNI_TEST_")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.MaxOpenConns != 3 || got.MaxIdleConns != 0 || got.PingTimeout != time.Second {
|
||||||
|
t.Fatalf("lost file defaults: %+v", got)
|
||||||
|
}
|
||||||
|
data, err := json.Marshal(got)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(data), "secret") || !strings.Contains(string(data), "maxOpenConns") {
|
||||||
|
t.Fatalf("unsafe config encoding: %s", data)
|
||||||
|
}
|
||||||
|
t.Setenv("OMNI_TEST_MAX_OPEN_CONNS", "bad")
|
||||||
|
if _, err = ApplyEnv(cfg, "OMNI_TEST_"); err == nil {
|
||||||
|
t.Fatal("invalid override accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -14,13 +14,13 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
type Config struct {
|
type Config struct {
|
||||||
Driver string
|
Driver string `json:"driver"`
|
||||||
DSN string
|
DSN string `json:"-"`
|
||||||
MaxOpenConns int
|
MaxOpenConns int `json:"maxOpenConns"`
|
||||||
MaxIdleConns int
|
MaxIdleConns int `json:"maxIdleConns"`
|
||||||
ConnMaxLifetime time.Duration
|
ConnMaxLifetime time.Duration `json:"connMaxLifetime"`
|
||||||
ConnMaxIdleTime time.Duration
|
ConnMaxIdleTime time.Duration `json:"connMaxIdleTime"`
|
||||||
PingTimeout time.Duration
|
PingTimeout time.Duration `json:"pingTimeout"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func DefaultConfig() Config {
|
func DefaultConfig() Config {
|
||||||
@@ -35,16 +35,25 @@ func DefaultConfig() Config {
|
|||||||
|
|
||||||
// ConfigFromEnv reads GO_OMNI_DB_* style settings. DSN is required.
|
// ConfigFromEnv reads GO_OMNI_DB_* style settings. DSN is required.
|
||||||
func ConfigFromEnv(prefix string) (Config, error) {
|
func ConfigFromEnv(prefix string) (Config, error) {
|
||||||
|
return ApplyEnv(DefaultConfig(), prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ApplyEnv overlays nonempty environment settings on file/default configuration.
|
||||||
|
// DSN is environment-only when Config is decoded from JSON or YAML.
|
||||||
|
func ApplyEnv(cfg Config, prefix string) (Config, error) {
|
||||||
prefix = strings.TrimSpace(prefix)
|
prefix = strings.TrimSpace(prefix)
|
||||||
if prefix == "" {
|
if prefix == "" {
|
||||||
prefix = "GO_OMNI_DB_"
|
prefix = "GO_OMNI_DB_"
|
||||||
}
|
}
|
||||||
cfg := DefaultConfig()
|
if value := strings.TrimSpace(os.Getenv(prefix + "DRIVER")); value != "" {
|
||||||
cfg.Driver = strings.TrimSpace(os.Getenv(prefix + "DRIVER"))
|
cfg.Driver = value
|
||||||
|
}
|
||||||
if cfg.Driver == "" {
|
if cfg.Driver == "" {
|
||||||
cfg.Driver = "mysql"
|
cfg.Driver = "mysql"
|
||||||
}
|
}
|
||||||
cfg.DSN = strings.TrimSpace(os.Getenv(prefix + "DSN"))
|
if value := strings.TrimSpace(os.Getenv(prefix + "DSN")); value != "" {
|
||||||
|
cfg.DSN = value
|
||||||
|
}
|
||||||
if cfg.DSN == "" {
|
if cfg.DSN == "" {
|
||||||
return Config{}, fmt.Errorf("%sDSN is required", prefix)
|
return Config{}, fmt.Errorf("%sDSN is required", prefix)
|
||||||
}
|
}
|
||||||
|
|||||||
15
examples/contracts/README.md
Normal file
15
examples/contracts/README.md
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
# 接口定义示例
|
||||||
|
|
||||||
|
`user.api` 是入口,导入 `shared.api`。包含 JSON/path/query/header 混合请求、分页与重复 query 参数、嵌套对象、optional 指针、user 与 admin/user 分组以及 Auth/Audit 命名中间件。
|
||||||
|
|
||||||
|
在 go-omni 根目录执行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni api validate --api examples/contracts/user.api
|
||||||
|
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
|
||||||
|
--dir ../userService --module example.com/userService --framework-dir .
|
||||||
|
```
|
||||||
|
|
||||||
|
项目中补充 `internal/service` 的业务实现和 `internal/middleware` 的 Auth/Audit hook,再启动服务。初始占位实现返回 501。
|
||||||
|
|
||||||
|
需要立即能返回成功结果的最小项目,用 `go-omni api new` 创建问候示例。完整语法、目录、更新策略见 [API.md](../../API.md)。
|
||||||
11
examples/contracts/shared.api
Normal file
11
examples/contracts/shared.api
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
syntax = "v1"
|
||||||
|
|
||||||
|
type Address {
|
||||||
|
City string `json:"city" binding:"required,max=50"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type UserResp {
|
||||||
|
UserID int64 `json:"userId"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Active bool `json:"active"`
|
||||||
|
}
|
||||||
58
examples/contracts/user.api
Normal file
58
examples/contracts/user.api
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
syntax = "v1"
|
||||||
|
|
||||||
|
info (
|
||||||
|
title: "User API"
|
||||||
|
version: "1.0.0"
|
||||||
|
)
|
||||||
|
|
||||||
|
import "shared.api"
|
||||||
|
|
||||||
|
type (
|
||||||
|
UpdateUserReq {
|
||||||
|
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||||
|
Notify bool `form:"notify,default=false"`
|
||||||
|
Client string `header:"X-Client,optional"`
|
||||||
|
Name string `json:"name" binding:"required,max=50"`
|
||||||
|
Active bool `json:"active"`
|
||||||
|
Age *int `json:"age,optional,range=[0:150]"`
|
||||||
|
Address *Address `json:"address,optional"`
|
||||||
|
}
|
||||||
|
ListUsersReq {
|
||||||
|
Page int `form:"page,default=1,range=[1:1000]"`
|
||||||
|
PageSize int `form:"pageSize,default=20,range=[1:100]"`
|
||||||
|
Tags []string `form:"tag,optional" binding:"max=10"`
|
||||||
|
}
|
||||||
|
ListUsersResp {
|
||||||
|
Items []UserResp `json:"items"`
|
||||||
|
Page int `json:"page"`
|
||||||
|
Total int64 `json:"total"`
|
||||||
|
}
|
||||||
|
DeleteUserReq {
|
||||||
|
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
@server (
|
||||||
|
group: user
|
||||||
|
prefix: /api/v1
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@doc "Update a user"
|
||||||
|
@handler updateUser
|
||||||
|
put /users/:userId (UpdateUserReq) returns (UserResp)
|
||||||
|
|
||||||
|
@doc "List users"
|
||||||
|
@handler listUsers
|
||||||
|
get /users (ListUsersReq) returns (ListUsersResp)
|
||||||
|
}
|
||||||
|
|
||||||
|
@server (
|
||||||
|
group: admin/user
|
||||||
|
prefix: /admin/v1
|
||||||
|
middleware: Auth,Audit
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@doc "Delete a user; business-owned authentication and auditing required"
|
||||||
|
@handler deleteUser
|
||||||
|
delete /users/:userId (DeleteUserReq)
|
||||||
|
}
|
||||||
6
examples/userService/.gitignore
vendored
Normal file
6
examples/userService/.gitignore
vendored
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
/server
|
||||||
|
/var/
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
/.goOmni/api.lock
|
||||||
19
examples/userService/.goOmni/apiManifest.json
Normal file
19
examples/userService/.goOmni/apiManifest.json
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
{
|
||||||
|
"profile": {
|
||||||
|
"database": "mysql",
|
||||||
|
"storage": "local",
|
||||||
|
"logOutput": "file",
|
||||||
|
"logFormat": "json"
|
||||||
|
},
|
||||||
|
"version": 1,
|
||||||
|
"module": "example.com/userService",
|
||||||
|
"entry": "api/user.api",
|
||||||
|
"generated": {
|
||||||
|
"docs/openapi.json": "0675e4eb1278b36828aa31b12587b0d33d2f384112f67a43179eb72b52c3a0ea",
|
||||||
|
"internal/handler/openapiGen.go": "8ddc60c338630db915b9f074ade776f526eabf60fc99b33e425359fd415c4926",
|
||||||
|
"internal/handler/routesGen.go": "7f45030e7eb9f055c4b4d2ff12ad855d7c726aad4aee63c5b3c3f97559a64257",
|
||||||
|
"internal/handler/user/handlersGen.go": "06f3c2798ff323812ff3eb2d8b933a569af3ce1b86a67b66036374add98c0d7e",
|
||||||
|
"internal/middleware/registryGen.go": "a6122a5453dedc6a4415860108814c50e155128cfe5412b39dd33ce2d01b4b00",
|
||||||
|
"internal/types/typesGen.go": "3ab2ce9ffe467c3ea53ac3d1fec283ecebc87cef57fd592d08dca991fd964876"
|
||||||
|
}
|
||||||
|
}
|
||||||
79
examples/userService/README.md
Normal file
79
examples/userService/README.md
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
# API 驱动的用户管理示例
|
||||||
|
|
||||||
|
这个独立 Go module 从 `api/user.api` 生成,随后在开发者拥有的文件中实现业务。
|
||||||
|
使用 MySQL 持久化、事务审计、环境变量 Token 鉴权、文件日志和 local 存储;业务层只接收标准 context 和基础模块接口。
|
||||||
|
|
||||||
|
## 启动
|
||||||
|
|
||||||
|
先准备空的业务数据库,再设置环境变量(替换占位值,密钥不写进配置或 Git):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
export GO_OMNI_DB_DSN='user:password@tcp(127.0.0.1:3306)/user_service?parseTime=true'
|
||||||
|
export USER_SERVICE_API_TOKEN='<至少32字节的随机值>'
|
||||||
|
export GO_OMNI_STORAGE_SIGNING_KEY='<另一个至少32字节的随机值>'
|
||||||
|
export GO_OMNI_STORAGE_PUBLIC_URL='http://127.0.0.1:8080/storage/object'
|
||||||
|
go mod tidy
|
||||||
|
# 在本目录运行与框架版本一致的 CLI;迁移必须显式执行。
|
||||||
|
go-omni migrate --dir migrations --apply
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
源码示例的 go.mod 使用 `replace ... => ../..`。复制到独立仓库时,删除这条 replace,
|
||||||
|
保留已发布的固定依赖版本,再执行 go mod tidy。远程版本验证脚本会自动检查没有本地替换。
|
||||||
|
`GO_OMNI_TRANSPORT=native` 可切换为 native,业务代码无需调整。
|
||||||
|
|
||||||
|
配置是默认值 → etc/config.yaml → 环境变量。日志默认写入 var/log/server.log 并轮转,
|
||||||
|
`GO_OMNI_LOG_OUTPUT=stderr` 可以切换为标准错误输出。MySQL 默认最多 10 条连接。
|
||||||
|
应用检查 users 和 user_audit 表,缺失时拒绝启动,不自动改表。
|
||||||
|
|
||||||
|
## 接口
|
||||||
|
|
||||||
|
除健康检查、OpenAPI 和持有效签名的对象下载外,业务接口需要 `Authorization: Bearer <Token>`。
|
||||||
|
这个 Token 代表示例的管理员;可通过同一个命名中间件槽位替换为 JWT/OIDC 和业务授权策略。
|
||||||
|
框架不会推断租户模型或自动给接口放行。
|
||||||
|
|
||||||
|
| 接口 | 行为 |
|
||||||
|
| --- | --- |
|
||||||
|
| POST /api/v1/users | 创建,返回 201;邮箱规范化并唯一 |
|
||||||
|
| GET /api/v1/users/:id | 查询,缺失返回 404 |
|
||||||
|
| GET /api/v1/users?page=1&pageSize=20 | 分页,pageSize 最大 100,空结果返回 [] |
|
||||||
|
| PUT /api/v1/users/:id | 完整更新;邮箱冲突返回 409 |
|
||||||
|
| DELETE /api/v1/users/:id | 删除,返回 204 |
|
||||||
|
| POST /api/v1/users/:id/export | 私有文本导出,返回 5 分钟下载链接 |
|
||||||
|
| GET /health/ready | 数据库不可用时返回 503,恢复后重新就绪 |
|
||||||
|
| GET /openapi.json | 与 docs/openapi.json 一致 |
|
||||||
|
|
||||||
|
参数不合法返回 400,认证失败 401,邮箱重复 409,内部故障只返回通用 500。
|
||||||
|
OpenAPI 的业务数据类型来自契约;401/404/409 的业务错误码见上表。
|
||||||
|
|
||||||
|
创建、更新、删除都会在**同一个 MySQL 事务**写入 user_audit;审计写入失败时业务变更一起回滚。
|
||||||
|
事务提交后才写成功业务日志,requestId 同时出现在 HTTP 响应、业务日志和审计记录。
|
||||||
|
数据库异常日志不记录 SQL、DSN 或用户输入。
|
||||||
|
|
||||||
|
文件导出演示 `deps.Uploads` 和 `deps.Downloads`。对象使用随机 key,签名 URL 不进入日志。
|
||||||
|
签名过期只限制下载,不删除文件;实际项目应给 exports/ 配置保留期限或清理任务。
|
||||||
|
只签发下载链接,不对外开放任意对象上传或签名接口。
|
||||||
|
|
||||||
|
## 开发与验证
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni api generate --api api/user.api --dir .
|
||||||
|
go-omni api generate --api api/user.api --dir . --check
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
Gen.go 和 docs/openapi.json 由生成器维护;Service、Repository、迁移、鉴权、配置和 main 保留。
|
||||||
|
修改 .api 的类型/方法签名后,需要同步修改自己的业务实现并运行测试。
|
||||||
|
|
||||||
|
在框架仓库运行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 scripts/verifyAPIWorkflow.py
|
||||||
|
# 已发布版本:全新 module cache 安装 CLI,无本地 replace 消费框架。
|
||||||
|
python3 scripts/verifyAPIWorkflow.py --version v0.7.0-preview \
|
||||||
|
--report performance/apiWorkflowRemote.json
|
||||||
|
```
|
||||||
|
|
||||||
|
验收创建只绑定 localhost 的临时 MySQL 容器(1 CPU / 768 MiB),使用随机密钥,结束时删除。
|
||||||
|
覆盖 nethttp/native 的 CRUD、校验、鉴权、事务错误/异常/取消回滚、数据库停机恢复、
|
||||||
|
重启持久化、签名文件下载、文件日志与优雅退出。不会连接业务服务器。
|
||||||
60
examples/userService/api/user.api
Normal file
60
examples/userService/api/user.api
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
syntax = "v1"
|
||||||
|
info (
|
||||||
|
title: "User service reference"
|
||||||
|
version: "1.0.0"
|
||||||
|
)
|
||||||
|
type (
|
||||||
|
UserResp {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Email string `json:"email"`
|
||||||
|
CreatedAt string `json:"createdAt"`
|
||||||
|
UpdatedAt string `json:"updatedAt"`
|
||||||
|
}
|
||||||
|
CreateUserReq {
|
||||||
|
Name string `json:"name" binding:"required,max=100"`
|
||||||
|
Email string `json:"email" binding:"required,email,max=254"`
|
||||||
|
}
|
||||||
|
UserIDReq {
|
||||||
|
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||||
|
}
|
||||||
|
UpdateUserReq {
|
||||||
|
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||||
|
Name string `json:"name" binding:"required,max=100"`
|
||||||
|
Email string `json:"email" binding:"required,email,max=254"`
|
||||||
|
}
|
||||||
|
ListUsersReq {
|
||||||
|
Page int `form:"page,default=1,range=[1:1000000]"`
|
||||||
|
PageSize int `form:"pageSize,default=20,range=[1:100]"`
|
||||||
|
}
|
||||||
|
UserPageResp {
|
||||||
|
Items []UserResp `json:"items"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
Page int `json:"page"`
|
||||||
|
PageSize int `json:"pageSize"`
|
||||||
|
}
|
||||||
|
ExportResp {
|
||||||
|
DownloadURL string `json:"downloadUrl"`
|
||||||
|
}
|
||||||
|
)
|
||||||
|
@server (
|
||||||
|
group: user
|
||||||
|
prefix: /api/v1
|
||||||
|
middleware: Auth
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@handler createUser
|
||||||
|
@status 201
|
||||||
|
post /users (CreateUserReq) returns (UserResp)
|
||||||
|
@handler getUser
|
||||||
|
get /users/:id (UserIDReq) returns (UserResp)
|
||||||
|
@handler listUsers
|
||||||
|
get /users (ListUsersReq) returns (UserPageResp)
|
||||||
|
@handler updateUser
|
||||||
|
put /users/:id (UpdateUserReq) returns (UserResp)
|
||||||
|
@handler deleteUser
|
||||||
|
delete /users/:id (UserIDReq)
|
||||||
|
@doc "Export one user as a private text object with a short-lived download link"
|
||||||
|
@handler exportUser
|
||||||
|
post /users/:id/export (UserIDReq) returns (ExportResp)
|
||||||
|
}
|
||||||
58
examples/userService/cmd/server/main.go
Normal file
58
examples/userService/cmd/server/main.go
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"example.com/userService/internal/handler"
|
||||||
|
"example.com/userService/internal/servicecontext"
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"flag"
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
path := flag.String("config", "etc/config.yaml", "configuration file; empty uses defaults and environment")
|
||||||
|
flag.Parse()
|
||||||
|
if err := run(*path); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func run(path string) (runErr error) {
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
cfg, err := settings.Load(path)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
logger, err := logx.New(cfg.Server.Log)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer func() { runErr = errors.Join(runErr, logger.Close()) }()
|
||||||
|
logx.SetDefault(logger)
|
||||||
|
deps, err := servicecontext.New(ctx, logger, cfg)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer func() { runErr = errors.Join(runErr, deps.Close()) }()
|
||||||
|
app := omni.New(omni.WithConfig(cfg.Server))
|
||||||
|
app.OnShutdown(logger.Close)
|
||||||
|
app.OnShutdown(deps.Close)
|
||||||
|
for _, method := range []string{"GET", "HEAD", "PUT"} {
|
||||||
|
app.Handle(method, "/storage/object", func(c *httpx.Context) error { deps.FileHTTP.ServeHTTP(c.Raw(), c.Request()); return nil })
|
||||||
|
}
|
||||||
|
if err = handler.RegisterRoutes(app, deps); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
app.GET("/openapi.json", handler.OpenAPI)
|
||||||
|
app.GET("/health/live", deps.Health.LivenessHandler())
|
||||||
|
app.GET("/health/ready", deps.Health.ReadinessHandler())
|
||||||
|
logger.InfoContext(ctx, "service.starting", "address", cfg.Server.Address)
|
||||||
|
return app.RunContext(ctx, cfg.Server.Address)
|
||||||
|
}
|
||||||
1249
examples/userService/docs/openapi.json
Normal file
1249
examples/userService/docs/openapi.json
Normal file
File diff suppressed because it is too large
Load Diff
37
examples/userService/etc/config.yaml
Normal file
37
examples/userService/etc/config.yaml
Normal file
@@ -0,0 +1,37 @@
|
|||||||
|
# Non-secret settings only. Environment variables override these values.
|
||||||
|
server:
|
||||||
|
address: '127.0.0.1:8080'
|
||||||
|
enableLogger: true
|
||||||
|
slowRequestThreshold: 500ms
|
||||||
|
shutdownTimeout: 10s
|
||||||
|
log:
|
||||||
|
service: "example.com/userService"
|
||||||
|
level: info
|
||||||
|
format: json
|
||||||
|
file: './var/log/server.log'
|
||||||
|
rotation:
|
||||||
|
enabled: true
|
||||||
|
maxSizeMb: 100
|
||||||
|
maxBackups: 10
|
||||||
|
maxAgeDays: 7
|
||||||
|
compress: true
|
||||||
|
redactKeys: []
|
||||||
|
storage:
|
||||||
|
provider: local
|
||||||
|
maxBytes: 33554432
|
||||||
|
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||||
|
linkTtl: 5m
|
||||||
|
requestTimeout: 30s
|
||||||
|
root: './var/files'
|
||||||
|
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||||
|
|
||||||
|
|
||||||
|
database:
|
||||||
|
driver: mysql
|
||||||
|
maxOpenConns: 10
|
||||||
|
maxIdleConns: 5
|
||||||
|
connMaxLifetime: 30m
|
||||||
|
connMaxIdleTime: 5m
|
||||||
|
pingTimeout: 5s
|
||||||
|
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||||
|
|
||||||
26
examples/userService/go.mod
Normal file
26
examples/userService/go.mod
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
module example.com/userService
|
||||||
|
|
||||||
|
go 1.22.0
|
||||||
|
|
||||||
|
require (
|
||||||
|
git.yuntuo.hk/k2safe/go-omni v0.7.0-preview
|
||||||
|
github.com/go-sql-driver/mysql v1.9.3
|
||||||
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
filippo.io/edwards25519 v1.1.0 // indirect
|
||||||
|
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||||
|
github.com/go-playground/locales v0.14.1 // indirect
|
||||||
|
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||||
|
github.com/go-playground/validator/v10 v10.20.0 // indirect
|
||||||
|
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
|
||||||
|
github.com/leodido/go-urn v1.4.0 // indirect
|
||||||
|
golang.org/x/crypto v0.31.0 // indirect
|
||||||
|
golang.org/x/net v0.33.0 // indirect
|
||||||
|
golang.org/x/sys v0.28.0 // indirect
|
||||||
|
golang.org/x/text v0.21.0 // indirect
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
|
)
|
||||||
|
|
||||||
|
replace git.yuntuo.hk/k2safe/go-omni => ../..
|
||||||
38
examples/userService/go.sum
Normal file
38
examples/userService/go.sum
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
|
||||||
|
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
||||||
|
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
||||||
|
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
||||||
|
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
||||||
|
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||||
|
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
|
||||||
|
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
|
||||||
|
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
||||||
|
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
|
||||||
|
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
||||||
|
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
|
||||||
|
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
|
||||||
|
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||||
|
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||||
|
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||||
|
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
|
||||||
|
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||||
|
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
|
||||||
|
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
|
||||||
|
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
|
||||||
|
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
|
||||||
|
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
|
||||||
|
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||||
|
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
|
||||||
|
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
13
examples/userService/internal/handler/openapiGen.go
Normal file
13
examples/userService/internal/handler/openapiGen.go
Normal file
File diff suppressed because one or more lines are too long
27
examples/userService/internal/handler/routesGen.go
Normal file
27
examples/userService/internal/handler/routesGen.go
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
// Code generated by go-omni api. DO NOT EDIT.
|
||||||
|
package handler
|
||||||
|
|
||||||
|
import (
|
||||||
|
h0 "example.com/userService/internal/handler/user"
|
||||||
|
s0 "example.com/userService/internal/service/user"
|
||||||
|
"example.com/userService/internal/servicecontext"
|
||||||
|
"fmt"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
)
|
||||||
|
|
||||||
|
func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {
|
||||||
|
if router == nil || deps == nil {
|
||||||
|
return fmt.Errorf("router and dependencies required")
|
||||||
|
}
|
||||||
|
if deps.Middlewares["Auth"] == nil {
|
||||||
|
return fmt.Errorf("missing middleware Auth")
|
||||||
|
}
|
||||||
|
h0 := h0.New(s0.New(deps))
|
||||||
|
router.Handle("POST", "/api/v1/users", h0.CreateUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("GET", "/api/v1/users/:id", h0.GetUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("GET", "/api/v1/users", h0.ListUsers, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("PUT", "/api/v1/users/:id", h0.UpdateUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("DELETE", "/api/v1/users/:id", h0.DeleteUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("POST", "/api/v1/users/:id/export", h0.ExportUser, deps.Middlewares["Auth"])
|
||||||
|
return nil
|
||||||
|
}
|
||||||
120
examples/userService/internal/handler/user/handlersGen.go
Normal file
120
examples/userService/internal/handler/user/handlersGen.go
Normal file
@@ -0,0 +1,120 @@
|
|||||||
|
// Code generated by go-omni api. DO NOT EDIT.
|
||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
"fmt"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Service interface {
|
||||||
|
CreateUser(ctx context.Context, req *types.CreateUserReq) (*types.UserResp, error)
|
||||||
|
GetUser(ctx context.Context, req *types.UserIDReq) (*types.UserResp, error)
|
||||||
|
ListUsers(ctx context.Context, req *types.ListUsersReq) (*types.UserPageResp, error)
|
||||||
|
UpdateUser(ctx context.Context, req *types.UpdateUserReq) (*types.UserResp, error)
|
||||||
|
DeleteUser(ctx context.Context, req *types.UserIDReq) error
|
||||||
|
ExportUser(ctx context.Context, req *types.UserIDReq) (*types.ExportResp, error)
|
||||||
|
}
|
||||||
|
type Handler struct{ service Service }
|
||||||
|
|
||||||
|
func New(s Service) *Handler { return &Handler{service: s} }
|
||||||
|
|
||||||
|
var createUserRequest = contract.MustNew[types.CreateUserReq]()
|
||||||
|
|
||||||
|
func (h *Handler) CreateUser(c *httpx.Context) error {
|
||||||
|
var req types.CreateUserReq
|
||||||
|
if err := createUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.CreateUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("CreateUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.Created(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var getUserRequest = contract.MustNew[types.UserIDReq]()
|
||||||
|
|
||||||
|
func (h *Handler) GetUser(c *httpx.Context) error {
|
||||||
|
var req types.UserIDReq
|
||||||
|
if err := getUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.GetUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("GetUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var listUsersRequest = contract.MustNew[types.ListUsersReq]()
|
||||||
|
|
||||||
|
func (h *Handler) ListUsers(c *httpx.Context) error {
|
||||||
|
var req types.ListUsersReq
|
||||||
|
if err := listUsersRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.ListUsers(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("ListUsers returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var updateUserRequest = contract.MustNew[types.UpdateUserReq]()
|
||||||
|
|
||||||
|
func (h *Handler) UpdateUser(c *httpx.Context) error {
|
||||||
|
var req types.UpdateUserReq
|
||||||
|
if err := updateUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.UpdateUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("UpdateUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var deleteUserRequest = contract.MustNew[types.UserIDReq]()
|
||||||
|
|
||||||
|
func (h *Handler) DeleteUser(c *httpx.Context) error {
|
||||||
|
var req types.UserIDReq
|
||||||
|
if err := deleteUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := h.service.DeleteUser(c.Context(), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return c.NoContent()
|
||||||
|
}
|
||||||
|
|
||||||
|
var exportUserRequest = contract.MustNew[types.UserIDReq]()
|
||||||
|
|
||||||
|
func (h *Handler) ExportUser(c *httpx.Context) error {
|
||||||
|
var req types.UserIDReq
|
||||||
|
if err := exportUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.ExportUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("ExportUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
40
examples/userService/internal/middleware/auth.go
Normal file
40
examples/userService/internal/middleware/auth.go
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
package middleware
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"crypto/subtle"
|
||||||
|
"errors"
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Auth is the fail-closed registry default. Startup replaces it with configured
|
||||||
|
// authentication. This administrative example uses one environment-only key;
|
||||||
|
// production applications can inject JWT/OIDC through the same middleware slot.
|
||||||
|
func Auth(next httpx.Handler) httpx.Handler {
|
||||||
|
return func(c *httpx.Context) error {
|
||||||
|
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func NewAuthFromEnv() (httpx.Middleware, error) {
|
||||||
|
token := os.Getenv("USER_SERVICE_API_TOKEN")
|
||||||
|
if len(token) < 32 || strings.ContainsAny(token, " \t\r\n") {
|
||||||
|
return nil, errors.New("USER_SERVICE_API_TOKEN requires at least 32 bytes without whitespace")
|
||||||
|
}
|
||||||
|
expected := sha256.Sum256([]byte(token))
|
||||||
|
return func(next httpx.Handler) httpx.Handler {
|
||||||
|
return func(c *httpx.Context) error {
|
||||||
|
parts := strings.Fields(c.Header("Authorization"))
|
||||||
|
if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") {
|
||||||
|
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||||
|
}
|
||||||
|
actual := sha256.Sum256([]byte(parts[1]))
|
||||||
|
if subtle.ConstantTimeCompare(expected[:], actual[:]) != 1 {
|
||||||
|
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||||
|
}
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
35
examples/userService/internal/middleware/auth_test.go
Normal file
35
examples/userService/internal/middleware/auth_test.go
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
package middleware
|
||||||
|
|
||||||
|
import (
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAuthFailsClosedAndDoesNotLeakKey(t *testing.T) {
|
||||||
|
t.Setenv("USER_SERVICE_API_TOKEN", "")
|
||||||
|
if _, err := NewAuthFromEnv(); err == nil {
|
||||||
|
t.Fatal("missing credential accepted")
|
||||||
|
}
|
||||||
|
token := "test-key-012345678901234567890123456789"
|
||||||
|
t.Setenv("USER_SERVICE_API_TOKEN", token)
|
||||||
|
middleware, err := NewAuthFromEnv()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, header := range []string{"", "Basic " + token, "Bearer wrong", "Bearer " + token} {
|
||||||
|
called := false
|
||||||
|
r := httptest.NewRequest("GET", "/", nil)
|
||||||
|
r.Header.Set("Authorization", header)
|
||||||
|
c := httpx.NewContext(r, httptest.NewRecorder(), nil)
|
||||||
|
err = middleware(func(*httpx.Context) error { called = true; return nil })(c)
|
||||||
|
if header == "Bearer "+token {
|
||||||
|
if err != nil || !called {
|
||||||
|
t.Fatal("valid token rejected")
|
||||||
|
}
|
||||||
|
} else if err == nil || called || apierrors.From(err).Status != 401 {
|
||||||
|
t.Fatal("unauthenticated request reached service")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
10
examples/userService/internal/middleware/registryGen.go
Normal file
10
examples/userService/internal/middleware/registryGen.go
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
// Code generated by go-omni api. DO NOT EDIT.
|
||||||
|
package middleware
|
||||||
|
|
||||||
|
import "git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
|
||||||
|
func Defaults() map[string]httpx.Middleware {
|
||||||
|
return map[string]httpx.Middleware{
|
||||||
|
"Auth": Auth,
|
||||||
|
}
|
||||||
|
}
|
||||||
11
examples/userService/internal/model/errors.go
Normal file
11
examples/userService/internal/model/errors.go
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
package model
|
||||||
|
|
||||||
|
import "errors"
|
||||||
|
|
||||||
|
// Domain errors are translated into HTTP responses only by the handler layer.
|
||||||
|
var (
|
||||||
|
ErrUserNotFound = errors.New("user not found")
|
||||||
|
ErrEmailTaken = errors.New("email already in use")
|
||||||
|
ErrInvalidUser = errors.New("invalid user")
|
||||||
|
ErrInvalidPage = errors.New("invalid pagination")
|
||||||
|
)
|
||||||
25
examples/userService/internal/model/user.go
Normal file
25
examples/userService/internal/model/user.go
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
package model
|
||||||
|
|
||||||
|
import "time"
|
||||||
|
|
||||||
|
// User is the business entity; it has no dependency on HTTP or a database driver.
|
||||||
|
type User struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Email string `json:"email"`
|
||||||
|
CreatedAt time.Time `json:"createdAt"`
|
||||||
|
UpdatedAt time.Time `json:"updatedAt"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// UserInput is shared by creation and full replacement (PUT).
|
||||||
|
type UserInput struct {
|
||||||
|
Name string
|
||||||
|
Email string
|
||||||
|
}
|
||||||
|
|
||||||
|
type UserPage struct {
|
||||||
|
Items []User `json:"items"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
Page int `json:"page"`
|
||||||
|
PageSize int `json:"pageSize"`
|
||||||
|
}
|
||||||
143
examples/userService/internal/repository/user/mysql.go
Normal file
143
examples/userService/internal/repository/user/mysql.go
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
"github.com/go-sql-driver/mysql"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SQLExecutor is implemented by both *sql.DB and *sql.Tx. A repository created
|
||||||
|
// with a transaction keeps all of its queries on that same transaction.
|
||||||
|
type SQLExecutor interface {
|
||||||
|
ExecContext(context.Context, string, ...any) (sql.Result, error)
|
||||||
|
QueryContext(context.Context, string, ...any) (*sql.Rows, error)
|
||||||
|
QueryRowContext(context.Context, string, ...any) *sql.Row
|
||||||
|
}
|
||||||
|
|
||||||
|
// MySQLUsers is the persistent implementation of UserRepository.
|
||||||
|
// The schema is documented in schema.sql; migrations stay outside the framework.
|
||||||
|
type MySQLUsers struct {
|
||||||
|
db SQLExecutor
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewMySQLUsers(db SQLExecutor) *MySQLUsers {
|
||||||
|
return &MySQLUsers{db: db}
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckSchema validates that the application's migration has been applied.
|
||||||
|
// Runtime startup never creates or alters tables.
|
||||||
|
func (r *MySQLUsers) CheckSchema(ctx context.Context) error {
|
||||||
|
rows, err := r.db.QueryContext(ctx, `SELECT id, name, email, created_at, updated_at FROM users LIMIT 0`)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return rows.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Create(ctx context.Context, input model.UserInput) (model.User, error) {
|
||||||
|
result, err := r.db.ExecContext(ctx, `
|
||||||
|
INSERT INTO users (name, email, created_at, updated_at)
|
||||||
|
VALUES (?, ?, UTC_TIMESTAMP(6), UTC_TIMESTAMP(6))`, input.Name, input.Email)
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, mapMySQLError(err)
|
||||||
|
}
|
||||||
|
id, err := result.LastInsertId()
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, err
|
||||||
|
}
|
||||||
|
return r.Get(ctx, id)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Get(ctx context.Context, id int64) (model.User, error) {
|
||||||
|
var user model.User
|
||||||
|
err := r.db.QueryRowContext(ctx, `
|
||||||
|
SELECT id, name, email, created_at, updated_at
|
||||||
|
FROM users WHERE id = ?`, id).Scan(
|
||||||
|
&user.ID, &user.Name, &user.Email, &user.CreatedAt, &user.UpdatedAt,
|
||||||
|
)
|
||||||
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
|
return model.User{}, model.ErrUserNotFound
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, err
|
||||||
|
}
|
||||||
|
return user, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) List(ctx context.Context, page, pageSize int) (model.UserPage, error) {
|
||||||
|
if page < 1 || pageSize < 1 || pageSize > 100 {
|
||||||
|
return model.UserPage{}, model.ErrInvalidPage
|
||||||
|
}
|
||||||
|
var total int
|
||||||
|
if err := r.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM users`).Scan(&total); err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
result := model.UserPage{Items: []model.User{}, Total: total, Page: page, PageSize: pageSize}
|
||||||
|
if total == 0 || page-1 > (total-1)/pageSize {
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
offset := (page - 1) * pageSize
|
||||||
|
rows, err := r.db.QueryContext(ctx, `
|
||||||
|
SELECT id, name, email, created_at, updated_at
|
||||||
|
FROM users ORDER BY id LIMIT ? OFFSET ?`, pageSize, offset)
|
||||||
|
if err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
for rows.Next() {
|
||||||
|
var user model.User
|
||||||
|
if err := rows.Scan(&user.ID, &user.Name, &user.Email, &user.CreatedAt, &user.UpdatedAt); err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
result.Items = append(result.Items, user)
|
||||||
|
}
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Update(ctx context.Context, id int64, input model.UserInput) (model.User, error) {
|
||||||
|
result, err := r.db.ExecContext(ctx, `
|
||||||
|
UPDATE users SET name = ?, email = ?, updated_at = UTC_TIMESTAMP(6)
|
||||||
|
WHERE id = ?`, input.Name, input.Email, id)
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, mapMySQLError(err)
|
||||||
|
}
|
||||||
|
changed, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, err
|
||||||
|
}
|
||||||
|
if changed == 0 {
|
||||||
|
// MySQL may report zero affected rows when the replacement is identical;
|
||||||
|
// distinguish that from an ID that does not exist.
|
||||||
|
return r.Get(ctx, id)
|
||||||
|
}
|
||||||
|
return r.Get(ctx, id)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Delete(ctx context.Context, id int64) error {
|
||||||
|
result, err := r.db.ExecContext(ctx, `DELETE FROM users WHERE id = ?`, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
deleted, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if deleted == 0 {
|
||||||
|
return model.ErrUserNotFound
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func mapMySQLError(err error) error {
|
||||||
|
var mysqlErr *mysql.MySQLError
|
||||||
|
if errors.As(err, &mysqlErr) && mysqlErr.Number == 1062 {
|
||||||
|
return model.ErrEmailTaken
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
16
examples/userService/internal/repository/user/repository.go
Normal file
16
examples/userService/internal/repository/user/repository.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// UserRepository describes the storage operations the business needs.
|
||||||
|
// Implementations must atomically enforce unique, normalized email addresses.
|
||||||
|
type UserRepository interface {
|
||||||
|
Create(context.Context, model.UserInput) (model.User, error)
|
||||||
|
Get(context.Context, int64) (model.User, error)
|
||||||
|
List(context.Context, int, int) (model.UserPage, error)
|
||||||
|
Update(context.Context, int64, model.UserInput) (model.User, error)
|
||||||
|
Delete(context.Context, int64) error
|
||||||
|
}
|
||||||
41
examples/userService/internal/repository/user/store.go
Normal file
41
examples/userService/internal/repository/user/store.go
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type UnitOfWork interface {
|
||||||
|
UserRepository
|
||||||
|
WithTx(context.Context, func(UserRepository, func(string, int64, string) error) error) error
|
||||||
|
}
|
||||||
|
type Store struct {
|
||||||
|
*MySQLUsers
|
||||||
|
db *database.DB
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewStore(db *database.DB) *Store { return &Store{MySQLUsers: NewMySQLUsers(db.DB), db: db} }
|
||||||
|
func (s *Store) WithTx(ctx context.Context, work func(UserRepository, func(string, int64, string) error) error) error {
|
||||||
|
return s.db.InTx(ctx, nil, func(tx *sql.Tx) error {
|
||||||
|
audit := func(operation string, id int64, requestID string) error {
|
||||||
|
_, err := tx.ExecContext(ctx, "INSERT INTO user_audit (user_id,operation,request_id,created_at) VALUES (?,?,?,?)", id, operation, requestID, time.Now().UTC())
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return work(NewMySQLUsers(tx), audit)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckSchema validates columns; it never mutates database schema.
|
||||||
|
func (s *Store) CheckSchema(ctx context.Context) error {
|
||||||
|
if err := s.MySQLUsers.CheckSchema(ctx); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
rows, err := s.db.QueryContext(ctx, "SELECT id,user_id,operation,request_id,created_at FROM user_audit LIMIT 0")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return errors.Join(rows.Err(), rows.Close())
|
||||||
|
}
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This suite runs only against the disposable database created by verifyAPIWorkflow.
|
||||||
|
func TestTransactions(t *testing.T) {
|
||||||
|
if os.Getenv("USER_SERVICE_TEST_DATABASE") != "1" {
|
||||||
|
t.Skip("isolated MySQL required")
|
||||||
|
}
|
||||||
|
cfg, err := database.ConfigFromEnv("")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
db, err := database.Open(context.Background(), cfg)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer db.Close()
|
||||||
|
store := NewStore(db)
|
||||||
|
for _, kind := range []string{"error", "panic", "cancel", "commit"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
email := kind + "@transaction.example"
|
||||||
|
var recovered any
|
||||||
|
func() {
|
||||||
|
defer func() { recovered = recover() }()
|
||||||
|
err = store.WithTx(ctx, func(r UserRepository, audit func(string, int64, string) error) error {
|
||||||
|
u, e := r.Create(ctx, model.UserInput{Name: "Transaction", Email: email})
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
if e = audit("created", u.ID, "transactionCheck"); e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
switch kind {
|
||||||
|
case "error":
|
||||||
|
return errors.New("rollback sentinel")
|
||||||
|
case "panic":
|
||||||
|
panic("rollback sentinel")
|
||||||
|
case "cancel":
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}()
|
||||||
|
if kind == "panic" && recovered == nil {
|
||||||
|
t.Fatal("panic lost")
|
||||||
|
}
|
||||||
|
if kind == "commit" && err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if kind != "commit" && kind != "panic" && err == nil {
|
||||||
|
t.Fatal("failed transaction committed")
|
||||||
|
}
|
||||||
|
var count int
|
||||||
|
if e := db.QueryRow("SELECT COUNT(*) FROM users WHERE email=?", email).Scan(&count); e != nil {
|
||||||
|
t.Fatal(e)
|
||||||
|
}
|
||||||
|
want := 0
|
||||||
|
if kind == "commit" {
|
||||||
|
want = 1
|
||||||
|
}
|
||||||
|
if count != want {
|
||||||
|
t.Fatalf("rows=%d want=%d", count, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
var audits int
|
||||||
|
if err = db.QueryRow("SELECT COUNT(*) FROM user_audit WHERE request_id='transactionCheck'").Scan(&audits); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if audits != 1 {
|
||||||
|
t.Fatalf("audit transaction mismatch: %d", audits)
|
||||||
|
}
|
||||||
|
}
|
||||||
16
examples/userService/internal/service/user/createUser.go
Normal file
16
examples/userService/internal/service/user/createUser.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) CreateUser(ctx context.Context, req *types.CreateUserReq) (*types.UserResp, error) {
|
||||||
|
input, err := normalize(req.Name, req.Email)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
return s.mutate(ctx, "created", func(r repo.UserRepository) (model.User, error) { return r.Create(ctx, input) })
|
||||||
|
}
|
||||||
16
examples/userService/internal/service/user/deleteUser.go
Normal file
16
examples/userService/internal/service/user/deleteUser.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) DeleteUser(ctx context.Context, req *types.UserIDReq) error {
|
||||||
|
_, err := s.mutate(ctx, "deleted", func(r repo.UserRepository) (model.User, error) {
|
||||||
|
err := r.Delete(ctx, req.ID)
|
||||||
|
return model.User{ID: req.ID}, err
|
||||||
|
})
|
||||||
|
return err
|
||||||
|
}
|
||||||
40
examples/userService/internal/service/user/exportUser.go
Normal file
40
examples/userService/internal/service/user/exportUser.go
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) ExportUser(ctx context.Context, req *types.UserIDReq) (*types.ExportResp, error) {
|
||||||
|
u, err := s.GetUser(ctx, req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
payload, err := json.Marshal(u)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
var nonce [16]byte
|
||||||
|
if _, err = rand.Read(nonce[:]); err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
key := fmt.Sprintf("exports/users/%d/%s.txt", u.ID, hex.EncodeToString(nonce[:]))
|
||||||
|
if _, err = s.deps.Uploads.Put(ctx, key, strings.NewReader(string(payload)), int64(len(payload)), "text/plain"); err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
url, err := s.deps.Downloads.PresignGet(ctx, key, 5*time.Minute)
|
||||||
|
if err != nil {
|
||||||
|
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = s.deps.Files.Delete(cleanup, key)
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
s.deps.Logger.InfoContext(ctx, "user.exported", "userId", u.ID)
|
||||||
|
return &types.ExportResp{DownloadURL: url.String()}, nil
|
||||||
|
}
|
||||||
14
examples/userService/internal/service/user/getUser.go
Normal file
14
examples/userService/internal/service/user/getUser.go
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) GetUser(ctx context.Context, req *types.UserIDReq) (*types.UserResp, error) {
|
||||||
|
u, err := s.deps.Users.Get(ctx, req.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
return response(u), nil
|
||||||
|
}
|
||||||
18
examples/userService/internal/service/user/listUsers.go
Normal file
18
examples/userService/internal/service/user/listUsers.go
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) ListUsers(ctx context.Context, req *types.ListUsersReq) (*types.UserPageResp, error) {
|
||||||
|
page, err := s.deps.Users.List(ctx, req.Page, req.PageSize)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
result := &types.UserPageResp{Items: []types.UserResp{}, Total: page.Total, Page: page.Page, PageSize: page.PageSize}
|
||||||
|
for _, item := range page.Items {
|
||||||
|
result.Items = append(result.Items, *response(item))
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
65
examples/userService/internal/service/user/service.go
Normal file
65
examples/userService/internal/service/user/service.go
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/servicecontext"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"net/mail"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Service struct{ deps *servicecontext.Context }
|
||||||
|
|
||||||
|
func New(deps *servicecontext.Context) *Service { return &Service{deps: deps} }
|
||||||
|
func normalize(name, email string) (model.UserInput, error) {
|
||||||
|
input := model.UserInput{Name: strings.TrimSpace(name), Email: strings.ToLower(strings.TrimSpace(email))}
|
||||||
|
address, err := mail.ParseAddress(input.Email)
|
||||||
|
if input.Name == "" || utf8.RuneCountInString(input.Name) > 100 || len(input.Email) > 254 || err != nil || address.Address != input.Email {
|
||||||
|
return model.UserInput{}, model.ErrInvalidUser
|
||||||
|
}
|
||||||
|
return input, nil
|
||||||
|
}
|
||||||
|
func response(u model.User) *types.UserResp {
|
||||||
|
return &types.UserResp{ID: u.ID, Name: u.Name, Email: u.Email, CreatedAt: u.CreatedAt.UTC().Format(time.RFC3339Nano), UpdatedAt: u.UpdatedAt.UTC().Format(time.RFC3339Nano)}
|
||||||
|
}
|
||||||
|
func (s *Service) failure(ctx context.Context, err error) error {
|
||||||
|
switch {
|
||||||
|
case err == nil:
|
||||||
|
return nil
|
||||||
|
case errors.Is(err, model.ErrUserNotFound):
|
||||||
|
return apierrors.NotFound("USER_NOT_FOUND", "user not found")
|
||||||
|
case errors.Is(err, model.ErrEmailTaken):
|
||||||
|
return apierrors.New(409, "EMAIL_TAKEN", "email already in use")
|
||||||
|
case errors.Is(err, model.ErrInvalidUser):
|
||||||
|
return apierrors.BadRequest("INVALID_USER", "invalid user")
|
||||||
|
case errors.Is(err, model.ErrInvalidPage):
|
||||||
|
return apierrors.BadRequest("INVALID_PAGE", "invalid pagination")
|
||||||
|
default:
|
||||||
|
// Do not log SQL/DSN or user input; requestId correlates the failed operation.
|
||||||
|
s.deps.Logger.ErrorContext(ctx, "user.operationFailed", "category", "persistence")
|
||||||
|
return apierrors.Internal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (s *Service) mutate(ctx context.Context, operation string, fn func(repo.UserRepository) (model.User, error)) (*types.UserResp, error) {
|
||||||
|
var u model.User
|
||||||
|
err := s.deps.Users.WithTx(ctx, func(r repo.UserRepository, audit func(string, int64, string) error) error {
|
||||||
|
var err error
|
||||||
|
u, err = fn(r)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return audit(operation, u.ID, logx.RequestID(ctx))
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
s.deps.Logger.InfoContext(ctx, "user."+operation, "userId", u.ID)
|
||||||
|
return response(u), nil
|
||||||
|
}
|
||||||
16
examples/userService/internal/service/user/updateUser.go
Normal file
16
examples/userService/internal/service/user/updateUser.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) UpdateUser(ctx context.Context, req *types.UpdateUserReq) (*types.UserResp, error) {
|
||||||
|
input, err := normalize(req.Name, req.Email)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
return s.mutate(ctx, "updated", func(r repo.UserRepository) (model.User, error) { return r.Update(ctx, req.ID, input) })
|
||||||
|
}
|
||||||
122
examples/userService/internal/servicecontext/context.go
Normal file
122
examples/userService/internal/servicecontext/context.go
Normal file
@@ -0,0 +1,122 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"example.com/userService/internal/middleware"
|
||||||
|
userrepo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"github.com/go-sql-driver/mysql"
|
||||||
|
"net/http"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Context owns infrastructure, while Logger is borrowed from main.
|
||||||
|
// Add business repositories here; HTTP contracts do not define database tables.
|
||||||
|
type Context struct {
|
||||||
|
Users userrepo.UnitOfWork
|
||||||
|
Logger *logx.Logger
|
||||||
|
Health *health.Checker
|
||||||
|
Middlewares map[string]httpx.Middleware
|
||||||
|
DB *database.DB
|
||||||
|
|
||||||
|
Files storage.Store
|
||||||
|
Uploads *storage.Uploader
|
||||||
|
Downloads storage.Presigner
|
||||||
|
UploadLinks storage.UploadPresigner
|
||||||
|
FileHTTP http.Handler
|
||||||
|
storageClose func() error
|
||||||
|
|
||||||
|
closeMu sync.Mutex
|
||||||
|
closed bool
|
||||||
|
closeOnce sync.Once
|
||||||
|
closeErr error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pass the settings loaded by main. The optional form supports lightweight
|
||||||
|
// tests without infrastructure; selected adapters still require valid settings.
|
||||||
|
func New(ctx context.Context, logger *logx.Logger, configs ...settings.Config) (result *Context, err error) {
|
||||||
|
if ctx == nil {
|
||||||
|
ctx = context.Background()
|
||||||
|
}
|
||||||
|
if err = ctx.Err(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if len(configs) > 1 {
|
||||||
|
return nil, errors.New("expected at most one configuration")
|
||||||
|
}
|
||||||
|
cfg := settings.Default()
|
||||||
|
if len(configs) == 1 {
|
||||||
|
cfg = configs[0]
|
||||||
|
}
|
||||||
|
_ = cfg
|
||||||
|
checker, err := health.New()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
c := &Context{Logger: logger, Health: checker, Middlewares: middleware.Defaults()}
|
||||||
|
defer func() {
|
||||||
|
if err != nil {
|
||||||
|
err = errors.Join(err, c.Close())
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
if err = cfg.Validate(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
dsn, parseErr := mysql.ParseDSN(cfg.Database.DSN)
|
||||||
|
if parseErr != nil || dsn.DBName == "" {
|
||||||
|
return nil, errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")
|
||||||
|
}
|
||||||
|
dsn.ParseTime = true
|
||||||
|
dsn.Loc = time.UTC
|
||||||
|
cfg.Database.DSN = dsn.FormatDSN()
|
||||||
|
c.DB, err = database.Open(ctx, cfg.Database)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err = checker.Add(health.Probe{Name: "database", Timeout: cfg.Database.PingTimeout, Check: c.DB.Health}); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if err = c.InitStorage(ctx, cfg.Storage); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
c.Users = userrepo.NewStore(c.DB)
|
||||||
|
checkCtx, cancel := context.WithTimeout(ctx, cfg.Database.PingTimeout)
|
||||||
|
defer cancel()
|
||||||
|
if err = userrepo.NewStore(c.DB).CheckSchema(checkCtx); err != nil {
|
||||||
|
return nil, errors.New("user schema is missing or incompatible; apply migrations first")
|
||||||
|
}
|
||||||
|
authHook, authErr := middleware.NewAuthFromEnv()
|
||||||
|
if authErr != nil {
|
||||||
|
return nil, authErr
|
||||||
|
}
|
||||||
|
c.Middlewares["Auth"] = authHook
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close is idempotent, including startup failure and graceful shutdown.
|
||||||
|
func (c *Context) Close() error {
|
||||||
|
if c == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
c.closeMu.Lock()
|
||||||
|
defer c.closeMu.Unlock()
|
||||||
|
c.closed = true
|
||||||
|
c.closeOnce.Do(func() {
|
||||||
|
if c.storageClose != nil {
|
||||||
|
c.closeErr = errors.Join(c.closeErr, c.storageClose())
|
||||||
|
}
|
||||||
|
if c.DB != nil {
|
||||||
|
c.closeErr = errors.Join(c.closeErr, c.DB.Close())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
return c.closeErr
|
||||||
|
}
|
||||||
81
examples/userService/internal/servicecontext/context_test.go
Normal file
81
examples/userService/internal/servicecontext/context_test.go
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestStartupFailureAndIdempotentCleanup(t *testing.T) {
|
||||||
|
if os.Getenv("USER_SERVICE_TEST_DATABASE") != "1" {
|
||||||
|
t.Skip("isolated MySQL required")
|
||||||
|
}
|
||||||
|
t.Setenv("USER_SERVICE_API_TOKEN", "test-012345678901234567890123456789")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY", "test-012345678901234567890123456789")
|
||||||
|
cfg, err := settings.Load("../../etc/config.yaml")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
observer, err := database.Open(context.Background(), cfg.Database)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer observer.Close()
|
||||||
|
count := func() int {
|
||||||
|
var n int
|
||||||
|
if err := observer.QueryRow("SELECT COUNT(*) FROM information_schema.processlist WHERE USER=SUBSTRING_INDEX(CURRENT_USER(),'@',1) AND DB=DATABASE()").Scan(&n); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
baseline := count()
|
||||||
|
cfg.Server.Log.File = ""
|
||||||
|
cfg.Server.Log.Writer = io.Discard
|
||||||
|
cfg.Server.Log.Rotation = logx.RotationConfig{}
|
||||||
|
logger, err := logx.New(cfg.Server.Log)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer logger.Close()
|
||||||
|
root := t.TempDir()
|
||||||
|
file := filepath.Join(root, "notADirectory")
|
||||||
|
if err = os.WriteFile(file, []byte("x"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg.Storage.Root = file
|
||||||
|
if c, err := New(context.Background(), logger, cfg); err == nil {
|
||||||
|
c.Close()
|
||||||
|
t.Fatal("invalid storage root accepted")
|
||||||
|
}
|
||||||
|
deadline := time.Now().Add(3 * time.Second)
|
||||||
|
for count() != baseline {
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
t.Fatal("failed startup leaked MySQL connection")
|
||||||
|
}
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
cfg.Storage.Root = filepath.Join(root, "files")
|
||||||
|
c, err := New(context.Background(), logger, cfg)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = c.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = c.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = c.DB.Ping(); err == nil {
|
||||||
|
t.Fatal("closed application retained database")
|
||||||
|
}
|
||||||
|
if err = c.InitStorage(context.Background(), cfg.Storage); err == nil {
|
||||||
|
t.Fatal("closed dependencies reopened storage")
|
||||||
|
}
|
||||||
|
}
|
||||||
49
examples/userService/internal/servicecontext/storage.go
Normal file
49
examples/userService/internal/servicecontext/storage.go
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InitStorage is called once, before serving requests. The composition root
|
||||||
|
// owns provider clients; business services receive only capability interfaces.
|
||||||
|
func (c *Context) InitStorage(ctx context.Context, cfg settings.StorageConfig) error {
|
||||||
|
if c == nil || ctx == nil {
|
||||||
|
return errors.New("storage initialization requires context")
|
||||||
|
}
|
||||||
|
c.closeMu.Lock()
|
||||||
|
defer c.closeMu.Unlock()
|
||||||
|
if c.closed {
|
||||||
|
return errors.New("service dependencies are closed")
|
||||||
|
}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if c.Files != nil {
|
||||||
|
return errors.New("storage already initialized")
|
||||||
|
}
|
||||||
|
if err := cfg.Validate(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
files, err := local.New(cfg.Root, cfg.MaxBytes)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
signer, err := storage.NewUploadSigner(files, cfg.PublicURL, []byte(cfg.SigningKey), cfg.Policy())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
uploader, err := storage.NewUploader(files, cfg.Policy())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
c.Files, c.Uploads, c.Downloads, c.UploadLinks, c.FileHTTP = files, uploader, signer, signer, signer
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
193
examples/userService/internal/settings/config.go
Normal file
193
examples/userService/internal/settings/config.go
Normal file
@@ -0,0 +1,193 @@
|
|||||||
|
// Package settings owns this project's configuration; it adds no business identity model.
|
||||||
|
package settings
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Config struct {
|
||||||
|
Server config.Config `json:"server"`
|
||||||
|
Database database.Config `json:"database"`
|
||||||
|
Storage StorageConfig `json:"storage"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func Default() Config {
|
||||||
|
server := config.Default()
|
||||||
|
server.Address = "127.0.0.1:8080"
|
||||||
|
server.EnableLogger = true
|
||||||
|
server.SlowRequestThreshold = 500 * time.Millisecond
|
||||||
|
server.Log.Service = "example.com/userService"
|
||||||
|
server.Log.Format = "json"
|
||||||
|
server.Log.File = "./var/log/server.log"
|
||||||
|
server.Log.Rotation = logx.RotationConfig{Enabled: true, MaxSizeMB: 100, MaxBackups: 10, MaxAgeDays: 7, Compress: true}
|
||||||
|
|
||||||
|
return Config{Server: server, Database: database.DefaultConfig(), Storage: StorageConfig{
|
||||||
|
Provider: "local", Root: "./var/files", PublicURL: "http://127.0.0.1:8080/storage/object",
|
||||||
|
Secure: true, MaxBytes: 32 << 20, ContentTypes: []string{"text/plain", "image/png", "image/jpeg", "application/pdf"},
|
||||||
|
LinkTTL: 5 * time.Minute, RequestTimeout: 30 * time.Second,
|
||||||
|
}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||||
|
// Pass an empty path to use only defaults and environment.
|
||||||
|
func Load(path string) (Config, error) {
|
||||||
|
cfg := Default()
|
||||||
|
var err error
|
||||||
|
if path != "" {
|
||||||
|
cfg, err = config.DecodeFile(path, cfg)
|
||||||
|
if err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cfg.Server, err = config.ApplyEnv(cfg.Server, "")
|
||||||
|
if err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
// A stream explicitly selected by environment overrides a file profile.
|
||||||
|
if output := strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")); output != "" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE")) == "" {
|
||||||
|
cfg.Server.Log.File = ""
|
||||||
|
cfg.Server.Log.Rotation = logx.RotationConfig{}
|
||||||
|
}
|
||||||
|
|
||||||
|
if err = cfg.Storage.applyEnv(); err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg.Database, err = database.ApplyEnv(cfg.Database, "")
|
||||||
|
if err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
if err = cfg.Validate(); err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
|
func (c Config) Validate() error {
|
||||||
|
|
||||||
|
if err := c.Database.Validate(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if c.Database.Driver != "mysql" || c.Database.MaxOpenConns < 1 {
|
||||||
|
return errors.New("database requires driver mysql and a positive maxOpenConns")
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := c.Server.Validate(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if c.Server.Log.Format != "json" && c.Server.Log.Format != "text" {
|
||||||
|
return errors.New("log.format must be json or text")
|
||||||
|
}
|
||||||
|
switch strings.ToLower(c.Server.Log.Level) {
|
||||||
|
case "debug", "info", "warn", "warning", "error":
|
||||||
|
default:
|
||||||
|
return errors.New("invalid log.level")
|
||||||
|
}
|
||||||
|
return c.Storage.Validate()
|
||||||
|
}
|
||||||
|
|
||||||
|
type StorageConfig struct {
|
||||||
|
Provider string `json:"provider"`
|
||||||
|
Root string `json:"root"`
|
||||||
|
PublicURL string `json:"publicUrl"`
|
||||||
|
Endpoint string `json:"endpoint"`
|
||||||
|
Region string `json:"region"`
|
||||||
|
Bucket string `json:"bucket"`
|
||||||
|
AccountID string `json:"accountId"`
|
||||||
|
Secure bool `json:"secure"`
|
||||||
|
MaxBytes int64 `json:"maxBytes"`
|
||||||
|
ContentTypes []string `json:"contentTypes"`
|
||||||
|
LinkTTL time.Duration `json:"linkTtl"`
|
||||||
|
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||||
|
// Secrets are environment-only and cannot be populated by config files.
|
||||||
|
FilesToken string `json:"-"`
|
||||||
|
SigningKey string `json:"-"`
|
||||||
|
AccessKey string `json:"-"`
|
||||||
|
SecretKey string `json:"-"`
|
||||||
|
SessionToken string `json:"-"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s StorageConfig) Policy() storage.UploadPolicy {
|
||||||
|
return storage.UploadPolicy{MaxBytes: s.MaxBytes, ContentTypes: s.ContentTypes}
|
||||||
|
}
|
||||||
|
func (s *StorageConfig) applyEnv() error {
|
||||||
|
values := map[string]*string{
|
||||||
|
"GO_OMNI_STORAGE_ROOT": &s.Root, "GO_OMNI_STORAGE_PUBLIC_URL": &s.PublicURL,
|
||||||
|
"GO_OMNI_STORAGE_ENDPOINT": &s.Endpoint, "GO_OMNI_STORAGE_REGION": &s.Region,
|
||||||
|
"GO_OMNI_STORAGE_BUCKET": &s.Bucket, "GO_OMNI_STORAGE_ACCOUNT_ID": &s.AccountID,
|
||||||
|
"GO_OMNI_STORAGE_SIGNING_KEY": &s.SigningKey, "GO_OMNI_FILES_TOKEN": &s.FilesToken,
|
||||||
|
"GO_OMNI_STORAGE_ACCESS_KEY": &s.AccessKey, "GO_OMNI_STORAGE_SECRET_KEY": &s.SecretKey,
|
||||||
|
"GO_OMNI_STORAGE_SESSION_TOKEN": &s.SessionToken,
|
||||||
|
}
|
||||||
|
for name, target := range values {
|
||||||
|
if value := strings.TrimSpace(os.Getenv(name)); value != "" {
|
||||||
|
*target = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for name, target := range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL": &s.LinkTTL, "GO_OMNI_STORAGE_REQUEST_TIMEOUT": &s.RequestTimeout} {
|
||||||
|
if value := strings.TrimSpace(os.Getenv(name)); value != "" {
|
||||||
|
v, err := time.ParseDuration(value)
|
||||||
|
if err != nil {
|
||||||
|
return fmt.Errorf("%s must be a duration", name)
|
||||||
|
}
|
||||||
|
*target = v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if value := os.Getenv("GO_OMNI_STORAGE_MAX_BYTES"); value != "" {
|
||||||
|
v, err := strconv.ParseInt(value, 10, 64)
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")
|
||||||
|
}
|
||||||
|
s.MaxBytes = v
|
||||||
|
}
|
||||||
|
if value := os.Getenv("GO_OMNI_STORAGE_SECURE"); value != "" {
|
||||||
|
v, err := strconv.ParseBool(value)
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")
|
||||||
|
}
|
||||||
|
s.Secure = v
|
||||||
|
}
|
||||||
|
if value := strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES")); value != "" {
|
||||||
|
s.ContentTypes = strings.Split(value, ",")
|
||||||
|
for i := range s.ContentTypes {
|
||||||
|
s.ContentTypes[i] = strings.TrimSpace(s.ContentTypes[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (s StorageConfig) Validate() error {
|
||||||
|
if s.Provider != "local" {
|
||||||
|
return errors.New("storage.provider must match the adapter selected at generation")
|
||||||
|
}
|
||||||
|
if err := s.Policy().Validate(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if s.LinkTTL < time.Second || s.LinkTTL > time.Hour {
|
||||||
|
return errors.New("storage.linkTtl must be 1s..1h")
|
||||||
|
}
|
||||||
|
if s.RequestTimeout <= 0 || s.RequestTimeout > 5*time.Minute {
|
||||||
|
return errors.New("storage.requestTimeout must be >0 and <=5m")
|
||||||
|
}
|
||||||
|
|
||||||
|
if s.Root == "" {
|
||||||
|
return errors.New("storage.root is required")
|
||||||
|
}
|
||||||
|
u, err := url.Parse(s.PublicURL)
|
||||||
|
if err != nil || u.Host == "" || u.Hostname() == "" || (u.Scheme != "http" && u.Scheme != "https") || u.User != nil || u.RawQuery != "" || u.Fragment != "" || u.Path != "/storage/object" {
|
||||||
|
return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")
|
||||||
|
}
|
||||||
|
if len(s.SigningKey) < 32 {
|
||||||
|
return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
35
examples/userService/internal/types/typesGen.go
Normal file
35
examples/userService/internal/types/typesGen.go
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
// Code generated by go-omni api. DO NOT EDIT.
|
||||||
|
package types
|
||||||
|
|
||||||
|
type UserResp struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Email string `json:"email"`
|
||||||
|
CreatedAt string `json:"createdAt"`
|
||||||
|
UpdatedAt string `json:"updatedAt"`
|
||||||
|
}
|
||||||
|
type CreateUserReq struct {
|
||||||
|
Name string `json:"name" binding:"required,max=100"`
|
||||||
|
Email string `json:"email" binding:"required,email,max=254"`
|
||||||
|
}
|
||||||
|
type UserIDReq struct {
|
||||||
|
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||||
|
}
|
||||||
|
type UpdateUserReq struct {
|
||||||
|
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||||
|
Name string `json:"name" binding:"required,max=100"`
|
||||||
|
Email string `json:"email" binding:"required,email,max=254"`
|
||||||
|
}
|
||||||
|
type ListUsersReq struct {
|
||||||
|
Page int `form:"page,default=1,range=[1:1000000]"`
|
||||||
|
PageSize int `form:"pageSize,default=20,range=[1:100]"`
|
||||||
|
}
|
||||||
|
type UserPageResp struct {
|
||||||
|
Items []UserResp `json:"items"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
Page int `json:"page"`
|
||||||
|
PageSize int `json:"pageSize"`
|
||||||
|
}
|
||||||
|
type ExportResp struct {
|
||||||
|
DownloadURL string `json:"downloadUrl"`
|
||||||
|
}
|
||||||
2
examples/userService/migrations/000001_users.down.sql
Normal file
2
examples/userService/migrations/000001_users.down.sql
Normal file
@@ -0,0 +1,2 @@
|
|||||||
|
DROP TABLE user_audit;
|
||||||
|
DROP TABLE users;
|
||||||
18
examples/userService/migrations/000001_users.up.sql
Normal file
18
examples/userService/migrations/000001_users.up.sql
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
CREATE TABLE users (
|
||||||
|
id BIGINT NOT NULL AUTO_INCREMENT,
|
||||||
|
name VARCHAR(100) NOT NULL,
|
||||||
|
email VARCHAR(254) NOT NULL,
|
||||||
|
created_at DATETIME(6) NOT NULL,
|
||||||
|
updated_at DATETIME(6) NOT NULL,
|
||||||
|
PRIMARY KEY (id),
|
||||||
|
UNIQUE KEY uq_users_email (email)
|
||||||
|
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||||
|
|
||||||
|
CREATE TABLE user_audit (
|
||||||
|
id BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY,
|
||||||
|
user_id BIGINT NOT NULL,
|
||||||
|
operation VARCHAR(16) NOT NULL,
|
||||||
|
request_id VARCHAR(128) NOT NULL,
|
||||||
|
created_at DATETIME(6) NOT NULL,
|
||||||
|
INDEX ix_user_audit_user (user_id)
|
||||||
|
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||||
1
go.mod
1
go.mod
@@ -17,6 +17,7 @@ require (
|
|||||||
go.opentelemetry.io/otel/trace v1.31.0
|
go.opentelemetry.io/otel/trace v1.31.0
|
||||||
golang.org/x/mod v0.21.0
|
golang.org/x/mod v0.21.0
|
||||||
golang.org/x/net v0.33.0
|
golang.org/x/net v0.33.0
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1
|
||||||
gopkg.in/yaml.v3 v3.0.1
|
gopkg.in/yaml.v3 v3.0.1
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
2
go.sum
2
go.sum
@@ -130,5 +130,7 @@ golang.org/x/time v0.5.0 h1:o7cqy6amK/52YcAKIPlM3a+Fpj35zvRj2TP+e1xFSfk=
|
|||||||
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
|
golang.org/x/time v0.5.0/go.mod h1:3BpzKBy/shNhVucY/MWOyx10tF3SFh9QdLuxbVysPQM=
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
|
||||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
|
|||||||
161
internal/storagefixture/multipart.go
Normal file
161
internal/storagefixture/multipart.go
Normal file
@@ -0,0 +1,161 @@
|
|||||||
|
package storagefixture
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/md5"
|
||||||
|
"encoding/xml"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// MultipartHandler checks protocol sequencing, not provider authentication.
|
||||||
|
// The fixture is memory bounded and should only be mounted on httptest servers.
|
||||||
|
func MultipartHandler() http.Handler {
|
||||||
|
type part struct {
|
||||||
|
Number int `xml:"PartNumber"`
|
||||||
|
ETag string `xml:"ETag"`
|
||||||
|
Size int64 `xml:"Size"`
|
||||||
|
data []byte
|
||||||
|
}
|
||||||
|
type upload struct {
|
||||||
|
key, contentType string
|
||||||
|
parts map[int]part
|
||||||
|
}
|
||||||
|
var mu sync.Mutex
|
||||||
|
uploads := map[string]*upload{}
|
||||||
|
objects := map[string]value{}
|
||||||
|
sequence := 0
|
||||||
|
simple := Handler()
|
||||||
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
q := r.URL.Query()
|
||||||
|
id := q.Get("uploadId")
|
||||||
|
if _, start := q["uploads"]; !start && id == "" {
|
||||||
|
mu.Lock()
|
||||||
|
v, ok := objects[r.URL.Path]
|
||||||
|
if ok && r.Method == "DELETE" {
|
||||||
|
delete(objects, r.URL.Path)
|
||||||
|
mu.Unlock()
|
||||||
|
w.WriteHeader(204)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
mu.Unlock()
|
||||||
|
if ok && (r.Method == "GET" || r.Method == "HEAD") {
|
||||||
|
w.Header().Set("Content-Length", strconv.Itoa(len(v.data)))
|
||||||
|
w.Header().Set("Content-Type", v.contentType)
|
||||||
|
w.Header().Set("ETag", v.etag)
|
||||||
|
w.Header().Set("Last-Modified", time.Now().UTC().Format(http.TimeFormat))
|
||||||
|
if r.Method == "GET" {
|
||||||
|
w.Write(v.data)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
simple.ServeHTTP(w, r)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
mu.Lock()
|
||||||
|
defer mu.Unlock()
|
||||||
|
fail := func(code string, status int) {
|
||||||
|
w.Header().Set("Content-Type", "application/xml")
|
||||||
|
w.WriteHeader(status)
|
||||||
|
fmt.Fprintf(w, "<Error><Code>%s</Code><Message>fixture rejection</Message></Error>", code)
|
||||||
|
}
|
||||||
|
if r.Header.Get("Authorization") == "" && q.Get("X-Amz-Signature") == "" && q.Get("x-oss-signature") == "" {
|
||||||
|
fail("AccessDenied", 403)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if _, ok := q["uploads"]; ok && r.Method == "POST" {
|
||||||
|
if len(uploads) >= 8 {
|
||||||
|
fail("TooManyUploads", 503)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sequence++
|
||||||
|
id = strconv.Itoa(sequence)
|
||||||
|
uploads[id] = &upload{key: r.URL.Path, contentType: r.Header.Get("Content-Type"), parts: map[int]part{}}
|
||||||
|
fmt.Fprintf(w, "<InitiateMultipartUploadResult><Bucket>testbucket</Bucket><Key>file</Key><UploadId>%s</UploadId></InitiateMultipartUploadResult>", id)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
u, ok := uploads[id]
|
||||||
|
if !ok || u.key != r.URL.Path {
|
||||||
|
fail("NoSuchUpload", 404)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
switch r.Method {
|
||||||
|
case "PUT":
|
||||||
|
number, _ := strconv.Atoi(q.Get("partNumber"))
|
||||||
|
if number < 1 || number > 3 {
|
||||||
|
fail("InvalidPart", 400)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(r.Body, (6<<20)+1))
|
||||||
|
if err != nil || len(data) > 6<<20 {
|
||||||
|
fail("EntityTooLarge", 413)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
p := part{Number: number, ETag: fmt.Sprintf("\"%x\"", md5.Sum(data)), Size: int64(len(data)), data: data}
|
||||||
|
u.parts[number] = p
|
||||||
|
w.Header().Set("ETag", p.ETag)
|
||||||
|
case "GET":
|
||||||
|
marker, _ := strconv.Atoi(q.Get("part-number-marker"))
|
||||||
|
limit, _ := strconv.Atoi(q.Get("max-parts"))
|
||||||
|
if limit < 1 {
|
||||||
|
limit = 1000
|
||||||
|
}
|
||||||
|
numbers := []int{}
|
||||||
|
for n := range u.parts {
|
||||||
|
if n > marker {
|
||||||
|
numbers = append(numbers, n)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Ints(numbers)
|
||||||
|
var result struct {
|
||||||
|
XMLName xml.Name `xml:"ListPartsResult"`
|
||||||
|
Truncated bool `xml:"IsTruncated"`
|
||||||
|
Next int `xml:"NextPartNumberMarker"`
|
||||||
|
Parts []part `xml:"Part"`
|
||||||
|
}
|
||||||
|
if len(numbers) > limit {
|
||||||
|
result.Truncated = true
|
||||||
|
numbers = numbers[:limit]
|
||||||
|
result.Next = numbers[len(numbers)-1]
|
||||||
|
}
|
||||||
|
for _, n := range numbers {
|
||||||
|
result.Parts = append(result.Parts, u.parts[n])
|
||||||
|
}
|
||||||
|
xml.NewEncoder(w).Encode(result)
|
||||||
|
case "POST":
|
||||||
|
var request struct {
|
||||||
|
Parts []part `xml:"Part"`
|
||||||
|
}
|
||||||
|
if xml.NewDecoder(io.LimitReader(r.Body, 65537)).Decode(&request) != nil || len(request.Parts) == 0 {
|
||||||
|
fail("InvalidPart", 400)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var data []byte
|
||||||
|
for _, p := range request.Parts {
|
||||||
|
actual, ok := u.parts[p.Number]
|
||||||
|
if !ok || strings.Trim(p.ETag, "\"") != strings.Trim(actual.ETag, "\"") {
|
||||||
|
fail("InvalidPart", 400)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(data)+len(actual.data) > 7<<20 {
|
||||||
|
fail("EntityTooLarge", 413)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
data = append(data, actual.data...)
|
||||||
|
}
|
||||||
|
objects[u.key] = value{data: data, contentType: u.contentType, etag: fmt.Sprintf("\"%x\"", md5.Sum(data))}
|
||||||
|
delete(uploads, id)
|
||||||
|
fmt.Fprint(w, "<CompleteMultipartUploadResult><Bucket>testbucket</Bucket><Key>file</Key><ETag>\"completed\"</ETag></CompleteMultipartUploadResult>")
|
||||||
|
case "DELETE":
|
||||||
|
delete(uploads, id)
|
||||||
|
w.WriteHeader(204)
|
||||||
|
default:
|
||||||
|
fail("InvalidRequest", 400)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
@@ -1,31 +1,62 @@
|
|||||||
# logx
|
# logx
|
||||||
|
|
||||||
`logx` 是 GoOmni 的业务层结构化日志入口。业务代码不需要依赖具体网络库:
|
业务通过 `logx` 记录结构化日志,独立于 HTTP、队列和具体业务字段。普通 `context.Context` 继续由业务管理;框架不会约定或自动注入 userId、tenantId。
|
||||||
|
|
||||||
```go
|
```go
|
||||||
import "git.yuntuo.hk/k2safe/go-omni/logx"
|
logx.InfoContext(ctx, "order.created", "orderId", orderID)
|
||||||
|
logx.ErrorContext(ctx, "order.failed", "error", err)
|
||||||
func (s *UserService) Create(ctx context.Context, input CreateInput) error {
|
// 可选:附上本次记录位置的当前 goroutine 堆栈。
|
||||||
logx.InfoContext(ctx, "user.create.start", "email", input.Email)
|
logx.ErrorStackContext(ctx, "unexpected.failure", "error", err)
|
||||||
// ...
|
// 固定字段可使用子日志对象,无需重复传入。
|
||||||
return nil
|
logger := logx.With("component", "invoice")
|
||||||
}
|
logger.InfoContext(ctx, "invoice.saved", "invoiceId", invoiceID)
|
||||||
```
|
```
|
||||||
|
|
||||||
应用启动时可以替换默认输出:
|
`Debug/Info/Warn/Error` 有包级和实例方法,也有带 Context 的方法。`requestId` 自动跟随 HTTP Context;接入 telemetry 后可带 `traceId`。不要在 Service 中改用 `context.Background()`,否则会丢失取消信号和关联字段。
|
||||||
|
|
||||||
|
## 文件轮转
|
||||||
|
|
||||||
```go
|
```go
|
||||||
closeLog, err := logx.Setup(logx.Config{
|
closeLog, err := logx.Setup(logx.Config{
|
||||||
Service: "billing-api",
|
Service: "billing-api", Environment: "production",
|
||||||
Environment: "production",
|
Level: "info", Format: "json", File: "./var/log/billing-api.log",
|
||||||
Level: "info",
|
Rotation: logx.RotationConfig{
|
||||||
Format: "json",
|
Enabled: true, MaxSizeMB: 100, MaxBackups: 10,
|
||||||
File: "./var/log/billing-api.log",
|
MaxAgeDays: 7, Compress: true,
|
||||||
|
},
|
||||||
|
RedactKeys: []string{"bankAccount", "identityNumber"},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil { return err }
|
||||||
return err
|
|
||||||
}
|
|
||||||
defer closeLog()
|
defer closeLog()
|
||||||
```
|
```
|
||||||
|
|
||||||
HTTP 请求的 `X-Request-ID` 会被放进标准 `context.Context`,之后的 `InfoContext`、`ErrorContext` 会自动带上 `requestId`。后续可以在不改变业务调用方式的前提下增加文件轮转、采样和 OpenTelemetry 导出。
|
也可用 `omni.WithLogConfig(...)` 交给应用运行生命周期管理。子日志共用输出、级别和关闭状态。独立 `logx.New` 返回的实例由创建者负责关闭。
|
||||||
|
|
||||||
|
轮转显式开启;三个数值为 0 时分别取 100 MiB、10 份、7 天。超过大小后轮转,旧文件按份数/年龄清理,可 gzip 压缩;保留天数不表示每天定时切割。压缩和历史清理由底层异步执行,不承诺 Close 等待压缩完成。轮转使用 [lumberjack v2](https://github.com/natefinch/lumberjack/tree/v2.2.1),**一个日志文件只能由一个进程拥有**。容器通常直接用 stdout/stderr,让运行平台管理日志文件。
|
||||||
|
|
||||||
|
`rotation.enabled` 与自定义 Writer 不能同时使用;只设轮转但未提供 File 会报错。开启轮转时会创建父目录,并在初始化阶段检查文件可打开。自定义 Writer 的并发和生命周期由接入方负责。普通文件输出仍然是追加写入;未开启轮转不自动清理。
|
||||||
|
|
||||||
|
## 结构化脱敏
|
||||||
|
|
||||||
|
默认匹配 password、passwd、secret、clientSecret、token、accessToken、refreshToken、authorization、cookie、setCookie、apiKey、privateKey、signedUrl。匹配忽略大小写、下划线和连字符;`RedactKeys` 追加业务自行定义的键,不替换默认列表。
|
||||||
|
|
||||||
|
覆盖普通属性、With 属性、slog Group/LogValuer、嵌套字符串键 map、切片和导出 struct 字段。敏感值变为 `[REDACTED]`,不修改调用者的数据;循环/过深结构截断。嵌套字节数组隐藏为 `[BINARY]`,非字符串键 map 隐藏为 `[UNSUPPORTED MAP]`,不调用对象自定义 JSON 序列化方法。
|
||||||
|
|
||||||
|
**脱敏按字段名处理,不解析自由文本。** 把密码拼进 msg/error/URL 字符串,或使用不在列表中的字段名,仍可能泄露。签名链接、完整请求体和认证头不应写入日志。业务隐私字段自行放进 `RedactKeys`;日志中不会自动添加身份/租户约定。
|
||||||
|
|
||||||
|
## HTTP 异常与慢请求
|
||||||
|
|
||||||
|
```go
|
||||||
|
application := omni.New(
|
||||||
|
omni.WithLogger(true),
|
||||||
|
omni.WithSlowRequestThreshold(500*time.Millisecond),
|
||||||
|
)
|
||||||
|
```
|
||||||
|
|
||||||
|
`WithSlowRequestThreshold` 同时开启访问日志,0 禁用慢请求识别。也可以在 JSON/YAML 中配 `enableLogger: true`、`slowRequestThreshold: "500ms"`,或使用 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 配合 `GO_OMNI_ENABLE_LOGGER=true`。
|
||||||
|
|
||||||
|
每个请求记录一条 `http.request`:方法、路径、匹配路由、实际状态码、耗时、客户端 IP 和关联字段。正常请求 INFO,超过阈值 WARN(`slow: true`),5xx ERROR;返回的 error 会记录到结构化字段。计时覆盖 Handler 调用,不包含框架之外的代理排队,也不自动计入 Handler 返回后继续写入的自管流。
|
||||||
|
|
||||||
|
Recovery 自动记录单独的 ERROR `http.panic` 和当前堆栈,再走原有统一错误响应;如果响应已提交,不改写已发送状态。客户端不会收到日志堆栈。普通业务 error 默认不抓堆栈,需要时显式使用 `ErrorStackContext`,它记录调用位置,不回溯 error 最初创建位置。
|
||||||
|
|
||||||
|
运行时可通过 `logger.SetLevel("debug")` 调整级别;文件轮转和脱敏配置属于静态设置,变更后重建实例/重启。当前不包含日志采样、异步缓冲、日志平台导出或慢 SQL 记录。
|
||||||
|
|||||||
27
logx/logx.go
27
logx/logx.go
@@ -10,6 +10,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"os"
|
"os"
|
||||||
|
"runtime/debug"
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
@@ -29,6 +30,8 @@ type Config struct {
|
|||||||
Environment string `json:"environment"`
|
Environment string `json:"environment"`
|
||||||
AddSource bool `json:"addSource"`
|
AddSource bool `json:"addSource"`
|
||||||
Writer io.Writer `json:"-"`
|
Writer io.Writer `json:"-"`
|
||||||
|
Rotation RotationConfig `json:"rotation"`
|
||||||
|
RedactKeys []string `json:"redactKeys"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// Default returns the default production-friendly logger configuration.
|
// Default returns the default production-friendly logger configuration.
|
||||||
@@ -62,13 +65,17 @@ var (
|
|||||||
func init() {
|
func init() {
|
||||||
// Keep package-level logging useful without requiring every service to
|
// Keep package-level logging useful without requiring every service to
|
||||||
// repeat setup. Applications can replace this logger with Setup.
|
// repeat setup. Applications can replace this logger with Setup.
|
||||||
defaultLogger.Store(&Logger{logger: slog.New(slog.NewJSONHandler(os.Stderr, nil))})
|
logger, _ := New(Default())
|
||||||
|
defaultLogger.Store(logger)
|
||||||
}
|
}
|
||||||
|
|
||||||
// New creates an independent logger. The caller owns its lifecycle and should
|
// New creates an independent logger. The caller owns its lifecycle and should
|
||||||
// call Close when the process is shutting down.
|
// call Close when the process is shutting down.
|
||||||
func New(cfg Config) (*Logger, error) {
|
func New(cfg Config) (*Logger, error) {
|
||||||
cfg = normalizeConfig(cfg)
|
cfg = normalizeConfig(cfg)
|
||||||
|
if err := cfg.Rotation.validate(cfg); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
level, err := parseLevel(cfg.Level)
|
level, err := parseLevel(cfg.Level)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -81,7 +88,7 @@ func New(cfg Config) (*Logger, error) {
|
|||||||
|
|
||||||
dynamicLevel := new(slog.LevelVar)
|
dynamicLevel := new(slog.LevelVar)
|
||||||
dynamicLevel.Set(level)
|
dynamicLevel.Set(level)
|
||||||
opts := &slog.HandlerOptions{Level: dynamicLevel, AddSource: cfg.AddSource}
|
opts := &slog.HandlerOptions{Level: dynamicLevel, AddSource: cfg.AddSource, ReplaceAttr: redactor(cfg.RedactKeys)}
|
||||||
var handler slog.Handler
|
var handler slog.Handler
|
||||||
if cfg.Format == "text" {
|
if cfg.Format == "text" {
|
||||||
handler = slog.NewTextHandler(writer, opts)
|
handler = slog.NewTextHandler(writer, opts)
|
||||||
@@ -208,6 +215,19 @@ func (l *Logger) ErrorContext(ctx context.Context, msg string, args ...any) {
|
|||||||
l.logger.Log(ctx, slog.LevelError, msg, contextAttrs(ctx, args)...)
|
l.logger.Log(ctx, slog.LevelError, msg, contextAttrs(ctx, args)...)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ErrorStackContext records the current goroutine's stack at the logging call,
|
||||||
|
// not the historical location where an error was originally constructed.
|
||||||
|
func (l *Logger) ErrorStackContext(ctx context.Context, msg string, args ...any) {
|
||||||
|
attrs := make([]any, 0, len(args)+2)
|
||||||
|
attrs = append(attrs, args...)
|
||||||
|
attrs = append(attrs, "stack", string(debug.Stack()))
|
||||||
|
l.ErrorContext(ctx, msg, attrs...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func ErrorStackContext(ctx context.Context, msg string, args ...any) {
|
||||||
|
defaultLogger.Load().ErrorStackContext(ctx, msg, args...)
|
||||||
|
}
|
||||||
|
|
||||||
func (l *Logger) Debug(msg string, args ...any) { l.DebugContext(context.Background(), msg, args...) }
|
func (l *Logger) Debug(msg string, args ...any) { l.DebugContext(context.Background(), msg, args...) }
|
||||||
func (l *Logger) Info(msg string, args ...any) { l.InfoContext(context.Background(), msg, args...) }
|
func (l *Logger) Info(msg string, args ...any) { l.InfoContext(context.Background(), msg, args...) }
|
||||||
func (l *Logger) Warn(msg string, args ...any) { l.WarnContext(context.Background(), msg, args...) }
|
func (l *Logger) Warn(msg string, args ...any) { l.WarnContext(context.Background(), msg, args...) }
|
||||||
@@ -324,6 +344,9 @@ func openWriter(cfg Config) (io.Writer, func() error, error) {
|
|||||||
return cfg.Writer, nil, nil
|
return cfg.Writer, nil, nil
|
||||||
}
|
}
|
||||||
if cfg.File != "" {
|
if cfg.File != "" {
|
||||||
|
if cfg.Rotation.Enabled {
|
||||||
|
return rotatingWriter(cfg)
|
||||||
|
}
|
||||||
file, err := os.OpenFile(cfg.File, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
|
file, err := os.OpenFile(cfg.File, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0o644)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, nil, err
|
return nil, nil, err
|
||||||
|
|||||||
157
logx/production_test.go
Normal file
157
logx/production_test.go
Normal file
@@ -0,0 +1,157 @@
|
|||||||
|
package logx
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"compress/gzip"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type credentialsValue struct{}
|
||||||
|
|
||||||
|
func (credentialsValue) LogValue() slog.Value {
|
||||||
|
return slog.GroupValue(slog.String("token", "do-not-print"), slog.String("label", "keep"))
|
||||||
|
}
|
||||||
|
func TestStructuredRedaction(t *testing.T) {
|
||||||
|
for _, format := range []string{"json", "text"} {
|
||||||
|
t.Run(format, func(t *testing.T) {
|
||||||
|
var out bytes.Buffer
|
||||||
|
logger, err := New(Config{Writer: &out, Format: format, RedactKeys: []string{"bankAccount"}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer logger.Close()
|
||||||
|
type account struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
Password string `json:"password"`
|
||||||
|
Account string `json:"bankAccount"`
|
||||||
|
}
|
||||||
|
nested := map[string]any{"profile": account{"visible", "do-not-print", "do-not-print"}, "Authorization": "do-not-print", "items": []any{map[string]string{"API_KEY": "do-not-print", "label": "visible"}}}
|
||||||
|
logger.With("refreshToken", "do-not-print").WithGroup("metadata").Info("event", "body", nested, slog.Group("password", "value", "do-not-print"), "valuer", credentialsValue{})
|
||||||
|
if strings.Contains(out.String(), "do-not-print") || !strings.Contains(out.String(), "visible") || !strings.Contains(out.String(), redacted) {
|
||||||
|
t.Fatal(out.String())
|
||||||
|
}
|
||||||
|
if format == "json" {
|
||||||
|
var value map[string]any
|
||||||
|
if err := json.Unmarshal(out.Bytes(), &value); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRedactionDoesNotMutateOrLoop(t *testing.T) {
|
||||||
|
var out bytes.Buffer
|
||||||
|
logger, _ := New(Config{Writer: &out})
|
||||||
|
m := map[string]any{"password": "original"}
|
||||||
|
m["self"] = m
|
||||||
|
logger.Info("event", "map", m)
|
||||||
|
if m["password"] != "original" || strings.Contains(out.String(), "original") || !strings.Contains(out.String(), "TRUNCATED") {
|
||||||
|
t.Fatal(out.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRotationCompressRetentionAndClose(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
file := filepath.Join(dir, "logs", "app.log")
|
||||||
|
logger, err := New(Config{File: file, Rotation: RotationConfig{Enabled: true, MaxSizeMB: 1, MaxBackups: 2, MaxAgeDays: 1, Compress: true}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
payload := strings.Repeat("x", 600<<10)
|
||||||
|
for i := 0; i < 5; i++ {
|
||||||
|
logger.Info("rotation", "value", payload)
|
||||||
|
time.Sleep(3 * time.Millisecond)
|
||||||
|
}
|
||||||
|
deadline := time.Now().Add(3 * time.Second)
|
||||||
|
var backups []string
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
backups, _ = filepath.Glob(filepath.Join(dir, "logs", "app-*.gz"))
|
||||||
|
plain, _ := filepath.Glob(filepath.Join(dir, "logs", "app-*.log"))
|
||||||
|
if len(backups) == 2 && len(plain) == 0 {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
if len(backups) != 2 {
|
||||||
|
t.Fatalf("backups: %v", backups)
|
||||||
|
}
|
||||||
|
for _, path := range backups {
|
||||||
|
f, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
z, err := gzip.NewReader(f)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(z)
|
||||||
|
z.Close()
|
||||||
|
f.Close()
|
||||||
|
if err != nil || !json.Valid(bytes.TrimSpace(data)) {
|
||||||
|
t.Fatal("corrupt backup", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
child := logger.With("component", "test")
|
||||||
|
if err = logger.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before, _ := os.Stat(file)
|
||||||
|
child.Info("must not reopen")
|
||||||
|
after, _ := os.Stat(file)
|
||||||
|
if before.Size() != after.Size() {
|
||||||
|
t.Fatal("closed logger wrote")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRotationConcurrentWritesAndValidation(t *testing.T) {
|
||||||
|
for _, cfg := range []Config{{Rotation: RotationConfig{Enabled: true}}, {File: "unused", Writer: io.Discard, Rotation: RotationConfig{Enabled: true}}, {Rotation: RotationConfig{MaxBackups: -1}}} {
|
||||||
|
if _, err := New(cfg); err == nil {
|
||||||
|
t.Fatal("invalid rotation accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
file := filepath.Join(t.TempDir(), "app.log")
|
||||||
|
logger, err := New(Config{File: file, Rotation: RotationConfig{Enabled: true}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 8; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
for j := 0; j < 100; j++ {
|
||||||
|
logger.Info("concurrent", "n", j)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
logger.Close()
|
||||||
|
raw, _ := os.ReadFile(file)
|
||||||
|
if len(bytes.Split(bytes.TrimSpace(raw), []byte("\n"))) != 800 {
|
||||||
|
t.Fatal("lost logs")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestBusinessErrorStack(t *testing.T) {
|
||||||
|
var out bytes.Buffer
|
||||||
|
logger, _ := New(Config{Writer: &out, AddSource: true})
|
||||||
|
ctx := WithRequestID(context.Background(), "request")
|
||||||
|
logger.ErrorStackContext(ctx, "failure", "password", "do-not-print")
|
||||||
|
var entry map[string]any
|
||||||
|
if err := json.Unmarshal(out.Bytes(), &entry); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(entry["stack"].(string), "TestBusinessErrorStack") || entry["requestId"] != "request" || strings.Contains(out.String(), "do-not-print") {
|
||||||
|
t.Fatal(out.String())
|
||||||
|
}
|
||||||
|
if entry["source"].(map[string]any)["file"] == nil {
|
||||||
|
t.Fatal("source schema changed")
|
||||||
|
}
|
||||||
|
}
|
||||||
133
logx/redaction.go
Normal file
133
logx/redaction.go
Normal file
@@ -0,0 +1,133 @@
|
|||||||
|
package logx
|
||||||
|
|
||||||
|
import (
|
||||||
|
"log/slog"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode"
|
||||||
|
)
|
||||||
|
|
||||||
|
const redacted = "[REDACTED]"
|
||||||
|
|
||||||
|
// Redaction applies to structured keys, including nested maps and structs.
|
||||||
|
// Free text (messages, errors, URLs embedded in strings) is not parsed.
|
||||||
|
func redactor(extra []string) func([]string, slog.Attr) slog.Attr {
|
||||||
|
keys := make(map[string]bool)
|
||||||
|
for _, key := range append([]string{"password", "passwd", "secret", "clientSecret", "token", "accessToken", "refreshToken", "authorization", "cookie", "setCookie", "apiKey", "privateKey", "signedUrl"}, extra...) {
|
||||||
|
keys[normalizedKey(key)] = true
|
||||||
|
}
|
||||||
|
return func(groups []string, a slog.Attr) slog.Attr {
|
||||||
|
for _, group := range groups {
|
||||||
|
if keys[normalizedKey(group)] {
|
||||||
|
return slog.String(a.Key, redacted)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if keys[normalizedKey(a.Key)] {
|
||||||
|
return slog.String(a.Key, redacted)
|
||||||
|
}
|
||||||
|
a.Value = a.Value.Resolve()
|
||||||
|
if a.Value.Kind() == slog.KindAny {
|
||||||
|
a.Value = slog.AnyValue(sanitize(reflect.ValueOf(a.Value.Any()), keys, 0))
|
||||||
|
}
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func normalizedKey(key string) string {
|
||||||
|
return strings.Map(func(r rune) rune {
|
||||||
|
if unicode.IsLetter(r) || unicode.IsDigit(r) {
|
||||||
|
return unicode.ToLower(r)
|
||||||
|
}
|
||||||
|
return -1
|
||||||
|
}, key)
|
||||||
|
}
|
||||||
|
|
||||||
|
func sanitize(v reflect.Value, keys map[string]bool, depth int) any {
|
||||||
|
if !v.IsValid() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if depth >= 16 {
|
||||||
|
return "[TRUNCATED]"
|
||||||
|
}
|
||||||
|
if (v.Kind() == reflect.Pointer || v.Kind() == reflect.Interface) && v.IsNil() {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if v.CanInterface() {
|
||||||
|
switch x := v.Interface().(type) {
|
||||||
|
case *slog.Source:
|
||||||
|
return sanitize(reflect.ValueOf(map[string]any{"function": x.Function, "file": x.File, "line": x.Line}), keys, depth+1)
|
||||||
|
case time.Time:
|
||||||
|
return x
|
||||||
|
case error:
|
||||||
|
return x.Error()
|
||||||
|
case []slog.Attr:
|
||||||
|
out := make(map[string]any, len(x))
|
||||||
|
for _, a := range x {
|
||||||
|
if keys[normalizedKey(a.Key)] {
|
||||||
|
out[a.Key] = redacted
|
||||||
|
} else {
|
||||||
|
out[a.Key] = sanitize(reflect.ValueOf(a.Value.Resolve().Any()), keys, depth+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
case slog.LogValuer:
|
||||||
|
value := slog.AnyValue(x).Resolve()
|
||||||
|
return sanitize(reflect.ValueOf(value.Any()), keys, depth+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch v.Kind() {
|
||||||
|
case reflect.Interface, reflect.Pointer:
|
||||||
|
return sanitize(v.Elem(), keys, depth+1)
|
||||||
|
case reflect.Map:
|
||||||
|
if v.Type().Key().Kind() != reflect.String {
|
||||||
|
return "[UNSUPPORTED MAP]"
|
||||||
|
}
|
||||||
|
out := make(map[string]any, v.Len())
|
||||||
|
it := v.MapRange()
|
||||||
|
for it.Next() {
|
||||||
|
key := it.Key().String()
|
||||||
|
if keys[normalizedKey(key)] {
|
||||||
|
out[key] = redacted
|
||||||
|
} else {
|
||||||
|
out[key] = sanitize(it.Value(), keys, depth+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
case reflect.Struct:
|
||||||
|
out := make(map[string]any)
|
||||||
|
for i := 0; i < v.NumField(); i++ {
|
||||||
|
field := v.Type().Field(i)
|
||||||
|
if !field.IsExported() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
name := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||||
|
if name == "-" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if name == "" {
|
||||||
|
name = field.Name
|
||||||
|
}
|
||||||
|
if keys[normalizedKey(name)] || keys[normalizedKey(field.Name)] {
|
||||||
|
out[name] = redacted
|
||||||
|
} else {
|
||||||
|
out[name] = sanitize(v.Field(i), keys, depth+1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
case reflect.Slice, reflect.Array:
|
||||||
|
if v.Type().Elem().Kind() == reflect.Uint8 {
|
||||||
|
return "[BINARY]"
|
||||||
|
}
|
||||||
|
out := make([]any, v.Len())
|
||||||
|
for i := range out {
|
||||||
|
out[i] = sanitize(v.Index(i), keys, depth+1)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
default:
|
||||||
|
if v.CanInterface() {
|
||||||
|
return v.Interface()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
83
logx/rotation.go
Normal file
83
logx/rotation.go
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
package logx
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"gopkg.in/natefinch/lumberjack.v2"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RotationConfig opts a file logger into size-based rotation. Zero limits use
|
||||||
|
// 100 MiB, 10 backups and 7 days. Each file must have a single process owner.
|
||||||
|
type RotationConfig struct {
|
||||||
|
Enabled bool `json:"enabled"`
|
||||||
|
MaxSizeMB int `json:"maxSizeMb"`
|
||||||
|
MaxBackups int `json:"maxBackups"`
|
||||||
|
MaxAgeDays int `json:"maxAgeDays"`
|
||||||
|
Compress bool `json:"compress"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r RotationConfig) validate(cfg Config) error {
|
||||||
|
if r.MaxSizeMB < 0 || r.MaxSizeMB > 1024*1024 || r.MaxBackups < 0 || r.MaxBackups > 10000 || r.MaxAgeDays < 0 || r.MaxAgeDays > 36500 {
|
||||||
|
return errors.New("logx: invalid rotation limits")
|
||||||
|
}
|
||||||
|
if r.Enabled && (cfg.File == "" || cfg.Writer != nil) {
|
||||||
|
return errors.New("logx: rotation requires a file and no custom writer")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func rotatingWriter(cfg Config) (io.Writer, func() error, error) {
|
||||||
|
r := cfg.Rotation
|
||||||
|
if r.MaxSizeMB == 0 {
|
||||||
|
r.MaxSizeMB = 100
|
||||||
|
}
|
||||||
|
if r.MaxBackups == 0 {
|
||||||
|
r.MaxBackups = 10
|
||||||
|
}
|
||||||
|
if r.MaxAgeDays == 0 {
|
||||||
|
r.MaxAgeDays = 7
|
||||||
|
}
|
||||||
|
if err := os.MkdirAll(filepath.Dir(cfg.File), 0700); err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
// Fail at setup, rather than silently dropping the first log record.
|
||||||
|
f, err := os.OpenFile(cfg.File, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0600)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
if err = f.Close(); err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
w := &rotationWriter{writer: &lumberjack.Logger{Filename: cfg.File, MaxSize: r.MaxSizeMB, MaxBackups: r.MaxBackups, MaxAge: r.MaxAgeDays, Compress: r.Compress}}
|
||||||
|
return w, w.Close, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Lumberjack can reopen after Close; prevent children of a closed logger from
|
||||||
|
// resurrecting its file. The lock also serializes shutdown with pending writes.
|
||||||
|
type rotationWriter struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
writer *lumberjack.Logger
|
||||||
|
closed bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *rotationWriter) Write(p []byte) (int, error) {
|
||||||
|
w.mu.Lock()
|
||||||
|
defer w.mu.Unlock()
|
||||||
|
if w.closed {
|
||||||
|
return 0, io.ErrClosedPipe
|
||||||
|
}
|
||||||
|
return w.writer.Write(p)
|
||||||
|
}
|
||||||
|
func (w *rotationWriter) Close() error {
|
||||||
|
w.mu.Lock()
|
||||||
|
defer w.mu.Unlock()
|
||||||
|
if w.closed {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
w.closed = true
|
||||||
|
return w.writer.Close()
|
||||||
|
}
|
||||||
@@ -2,6 +2,7 @@ package middleware
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"runtime/debug"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
@@ -37,6 +38,7 @@ func Recovery() httpx.Middleware {
|
|||||||
return func(ctx *httpx.Context) (err error) {
|
return func(ctx *httpx.Context) (err error) {
|
||||||
defer func() {
|
defer func() {
|
||||||
if recovered := recover(); recovered != nil {
|
if recovered := recover(); recovered != nil {
|
||||||
|
logx.ErrorContext(ctx.Context(), "http.panic", "panic", recovered, "stack", string(debug.Stack()), "method", ctx.Request().Method, "route", httpx.RoutePattern(ctx.Context()))
|
||||||
err = apierrors.Internal(fmt.Errorf("panic recovered: %v", recovered))
|
err = apierrors.Internal(fmt.Errorf("panic recovered: %v", recovered))
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
@@ -48,7 +50,25 @@ func Recovery() httpx.Middleware {
|
|||||||
// AccessLog records one structured event after each request. It deliberately
|
// AccessLog records one structured event after each request. It deliberately
|
||||||
// lives in GoOmni instead of using a router-specific logger so business
|
// lives in GoOmni instead of using a router-specific logger so business
|
||||||
// applications get the same log format with every transport adapter.
|
// applications get the same log format with every transport adapter.
|
||||||
func AccessLog() httpx.Middleware {
|
type AccessLogOption func(*accessLogConfig)
|
||||||
|
type accessLogConfig struct{ slowThreshold time.Duration }
|
||||||
|
|
||||||
|
// WithSlowRequestThreshold logs slow requests at WARN. Zero disables detection.
|
||||||
|
func WithSlowRequestThreshold(threshold time.Duration) AccessLogOption {
|
||||||
|
return func(c *accessLogConfig) {
|
||||||
|
if threshold >= 0 {
|
||||||
|
c.slowThreshold = threshold
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func AccessLog(options ...AccessLogOption) httpx.Middleware {
|
||||||
|
settings := accessLogConfig{}
|
||||||
|
for _, option := range options {
|
||||||
|
if option != nil {
|
||||||
|
option(&settings)
|
||||||
|
}
|
||||||
|
}
|
||||||
return func(next httpx.Handler) httpx.Handler {
|
return func(next httpx.Handler) httpx.Handler {
|
||||||
return func(ctx *httpx.Context) error {
|
return func(ctx *httpx.Context) error {
|
||||||
started := time.Now()
|
started := time.Now()
|
||||||
@@ -57,14 +77,30 @@ func AccessLog() httpx.Middleware {
|
|||||||
if err != nil && !ctx.Responded() {
|
if err != nil && !ctx.Responded() {
|
||||||
status = apierrors.From(err).Status
|
status = apierrors.From(err).Status
|
||||||
}
|
}
|
||||||
logx.InfoContext(ctx.Context(), "http.request",
|
duration := time.Since(started)
|
||||||
|
slow := settings.slowThreshold > 0 && duration >= settings.slowThreshold
|
||||||
|
attrs := []any{
|
||||||
"method", ctx.Request().Method,
|
"method", ctx.Request().Method,
|
||||||
"path", ctx.Request().URL.Path,
|
"path", ctx.Request().URL.Path,
|
||||||
"route", httpx.RoutePattern(ctx.Context()),
|
"route", httpx.RoutePattern(ctx.Context()),
|
||||||
"status", status,
|
"status", status,
|
||||||
"durationMs", time.Since(started).Seconds()*1000,
|
"durationMs", duration.Seconds() * 1000,
|
||||||
"clientIp", ctx.ClientIP(),
|
"clientIp", ctx.ClientIP(),
|
||||||
)
|
}
|
||||||
|
if slow {
|
||||||
|
attrs = append(attrs, "slow", true)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
attrs = append(attrs, "error", err)
|
||||||
|
}
|
||||||
|
switch {
|
||||||
|
case status >= 500:
|
||||||
|
logx.ErrorContext(ctx.Context(), "http.request", attrs...)
|
||||||
|
case slow:
|
||||||
|
logx.WarnContext(ctx.Context(), "http.request", attrs...)
|
||||||
|
default:
|
||||||
|
logx.InfoContext(ctx.Context(), "http.request", attrs...)
|
||||||
|
}
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,8 +4,11 @@ import (
|
|||||||
"bytes"
|
"bytes"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
|
"io"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
@@ -42,13 +45,20 @@ func TestAccessLogReportsCommittedStatus(t *testing.T) {
|
|||||||
ctx.WriteError(err)
|
ctx.WriteError(err)
|
||||||
}
|
}
|
||||||
var entry struct {
|
var entry struct {
|
||||||
|
Message string `json:"msg"`
|
||||||
Status int
|
Status int
|
||||||
Route string
|
Route string
|
||||||
RequestID string `json:"requestId"`
|
RequestID string `json:"requestId"`
|
||||||
}
|
}
|
||||||
if err := json.Unmarshal(output.Bytes(), &entry); err != nil {
|
decoder := json.NewDecoder(&output)
|
||||||
|
for {
|
||||||
|
if err := decoder.Decode(&entry); err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
|
if entry.Message == "http.request" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
if entry.Status != tc.status || w.Code != tc.status || entry.Route != "/users/:id" || entry.RequestID != "logRequest" {
|
if entry.Status != tc.status || w.Code != tc.status || entry.Route != "/users/:id" || entry.RequestID != "logRequest" {
|
||||||
t.Fatalf("log=%s response=%d; expected status %d and request metadata", &output, w.Code, tc.status)
|
t.Fatalf("log=%s response=%d; expected status %d and request metadata", &output, w.Code, tc.status)
|
||||||
}
|
}
|
||||||
@@ -86,3 +96,48 @@ func TestAccessLogUsesVerifiedClientIP(t *testing.T) {
|
|||||||
t.Fatalf("expected verified client IP, got %q", entry.ClientIP)
|
t.Fatalf("expected verified client IP, got %q", entry.ClientIP)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestSlowAndPanicLogs(t *testing.T) {
|
||||||
|
var out bytes.Buffer
|
||||||
|
logger, _ := logx.New(logx.Config{Writer: &out})
|
||||||
|
logx.SetDefault(logger)
|
||||||
|
defer logx.SetDefault(nil)
|
||||||
|
ctx := httpx.NewContext(httptest.NewRequest("GET", "/panic?secret=hidden", nil), httptest.NewRecorder(), nil)
|
||||||
|
chain := middleware.RequestID("X-Request-ID")(middleware.AccessLog(middleware.WithSlowRequestThreshold(time.Nanosecond))(middleware.Recovery()(func(c *httpx.Context) error { panic("boom") })))
|
||||||
|
err := chain(ctx)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("panic not recovered")
|
||||||
|
}
|
||||||
|
decoder := json.NewDecoder(&out)
|
||||||
|
count := 0
|
||||||
|
for {
|
||||||
|
var entry map[string]any
|
||||||
|
err := decoder.Decode(&entry)
|
||||||
|
if err == io.EOF {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
count++
|
||||||
|
if entry["requestId"] == "" || entry["level"] != "ERROR" {
|
||||||
|
t.Fatal(entry)
|
||||||
|
}
|
||||||
|
if entry["msg"] == "http.panic" && !strings.Contains(entry["stack"].(string), "TestSlowAndPanicLogs") {
|
||||||
|
t.Fatal(entry)
|
||||||
|
}
|
||||||
|
if entry["msg"] == "http.request" && (entry["slow"] != true || entry["status"] != float64(500)) {
|
||||||
|
t.Fatal(entry)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if count != 2 {
|
||||||
|
t.Fatal(count)
|
||||||
|
}
|
||||||
|
out.Reset()
|
||||||
|
ctx = httpx.NewContext(httptest.NewRequest("GET", "/slow", nil), httptest.NewRecorder(), nil)
|
||||||
|
err = middleware.AccessLog(middleware.WithSlowRequestThreshold(time.Nanosecond))(func(c *httpx.Context) error { return c.OK("done") })(ctx)
|
||||||
|
var entry map[string]any
|
||||||
|
if err != nil || json.Unmarshal(out.Bytes(), &entry) != nil || entry["level"] != "WARN" || entry["slow"] != true {
|
||||||
|
t.Fatal(out.String(), err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
4
omni.go
4
omni.go
@@ -48,6 +48,10 @@ func WithRequestIDHeader(header string) Option { return config.WithRequestIDHead
|
|||||||
|
|
||||||
func WithLogger(enabled bool) Option { return config.WithLogger(enabled) }
|
func WithLogger(enabled bool) Option { return config.WithLogger(enabled) }
|
||||||
|
|
||||||
|
func WithSlowRequestThreshold(threshold time.Duration) Option {
|
||||||
|
return config.WithSlowRequestThreshold(threshold)
|
||||||
|
}
|
||||||
|
|
||||||
func WithTrustedProxies(proxies ...string) Option { return config.WithTrustedProxies(proxies...) }
|
func WithTrustedProxies(proxies ...string) Option { return config.WithTrustedProxies(proxies...) }
|
||||||
|
|
||||||
func WithConfig(settings Config) Option { return config.WithConfig(settings) }
|
func WithConfig(settings Config) Option { return config.WithConfig(settings) }
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
package openapi
|
package openapi
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
@@ -22,7 +23,19 @@ type Schema struct {
|
|||||||
Required []string `json:"required,omitempty"`
|
Required []string `json:"required,omitempty"`
|
||||||
Items *Schema `json:"items,omitempty"`
|
Items *Schema `json:"items,omitempty"`
|
||||||
AdditionalProperties *Schema `json:"additionalProperties,omitempty"`
|
AdditionalProperties *Schema `json:"additionalProperties,omitempty"`
|
||||||
|
AdditionalPropertiesAllowed *bool `json:"-"`
|
||||||
Nullable bool `json:"nullable,omitempty"`
|
Nullable bool `json:"nullable,omitempty"`
|
||||||
|
Minimum *float64 `json:"minimum,omitempty"`
|
||||||
|
Maximum *float64 `json:"maximum,omitempty"`
|
||||||
|
ExclusiveMinimum bool `json:"exclusiveMinimum,omitempty"`
|
||||||
|
ExclusiveMaximum bool `json:"exclusiveMaximum,omitempty"`
|
||||||
|
MinLength *int `json:"minLength,omitempty"`
|
||||||
|
MaxLength *int `json:"maxLength,omitempty"`
|
||||||
|
MinItems *int `json:"minItems,omitempty"`
|
||||||
|
MaxItems *int `json:"maxItems,omitempty"`
|
||||||
|
Enum []any `json:"enum,omitempty"`
|
||||||
|
Default any `json:"default,omitempty"`
|
||||||
|
Not *Schema `json:"not,omitempty"`
|
||||||
}
|
}
|
||||||
type Parameter struct {
|
type Parameter struct {
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
@@ -135,7 +148,9 @@ func (a *API) Handle(router Registrar, op Operation, handler httpx.Handler, midd
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
var frozen any
|
var frozen any
|
||||||
if err = json.Unmarshal(encoded, &frozen); err != nil {
|
decoder := json.NewDecoder(bytes.NewReader(encoded))
|
||||||
|
decoder.UseNumber()
|
||||||
|
if err = decoder.Decode(&frozen); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
a.mu.Lock()
|
a.mu.Lock()
|
||||||
|
|||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user