Compare commits
79 Commits
v0.1.0
...
v0.8.0-pre
| Author | SHA1 | Date | |
|---|---|---|---|
| 7670b222ac | |||
| bef7ea2641 | |||
| 2913e94a0a | |||
| ab8eac5fc4 | |||
| 0ce3efb6fc | |||
| 2c74dc0859 | |||
| 04a1f7999d | |||
| 208a9c3fbd | |||
| f91034477b | |||
| b9259e628e | |||
| fe66221a9c | |||
| 467c890363 | |||
| be620d2243 | |||
| 8e18528fc2 | |||
| 59bf04a461 | |||
| b54a303ccd | |||
| 84670f7be1 | |||
| fa2be5e016 | |||
| dfe8a1a9af | |||
| 830cb3a7ad | |||
| d94df4fc54 | |||
| 1a534c7976 | |||
| 2b93545507 | |||
| 15825f0ab8 | |||
| 1b9215114f | |||
| 73a05b8c3c | |||
| 64a9fde204 | |||
| ca0e1d6b92 | |||
| 98f17660c8 | |||
| c8660026ce | |||
| 695bdb7a00 | |||
| 7e9e8aa67c | |||
| de33eda846 | |||
| dd7d3fef0a | |||
| 46c5c9406c | |||
| 1487a96f67 | |||
| 1a20ab53ac | |||
| a072da4a31 | |||
| fcd6f48b53 | |||
| 83633f724e | |||
| a523e6ba3f | |||
| 4a8feaca36 | |||
| 95dc4568f6 | |||
| 2259606f72 | |||
| 35b7470be2 | |||
| 4f97aa0f3d | |||
| 3b4d0a3fad | |||
| a4d843b8f4 | |||
| fadd17a4d4 | |||
| ac46071dfe | |||
| 52d603c31e | |||
| 42021538b2 | |||
| ca707761fc | |||
| 30fdad44c2 | |||
| b7dc36fc99 | |||
| 8e474a24d9 | |||
| ef98d1dd8d | |||
| 5ab906098f | |||
| c5e415ca52 | |||
| 9c9c8ab59a | |||
| e6c82a01fe | |||
| 847621d218 | |||
| 4daf3b211c | |||
| 62b0f09546 | |||
| 694374fde8 | |||
| 22f1c9ebfb | |||
| b3cb9e8857 | |||
| efd3163902 | |||
| 241a568c83 | |||
| 22939fdb0b | |||
| 3d5f225fb2 | |||
| c36bb70f96 | |||
| 05b4088a25 | |||
| 3bbf4627a1 | |||
| 61c37e0dba | |||
| 5561795d80 | |||
| affd02d5f7 | |||
| ff95692edb | |||
| 0bd71b17b4 |
1
.gitignore
vendored
1
.gitignore
vendored
@@ -5,3 +5,4 @@ bin/
|
||||
dist/
|
||||
*.log
|
||||
coverage.out
|
||||
__pycache__/
|
||||
|
||||
151
ADVANCED.md
Normal file
151
ADVANCED.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# 扩展业务模块
|
||||
|
||||
本批提供八类扩展,以及分页、审计和显式数据库迁移。模块按需导入;`omni.New` 与业务 Handler 不绑定存储、队列或网络实现。JSON、配置和新示例使用 lowerCamelCase;Go 导出标识符和 `_test.go` 保持语言约定。
|
||||
|
||||
按本次范围决定,**不集成 Orbit,不引入 Orbit SDK,也不实现工作流编排**。通用消息队列与 Orbit 的持久流程、执行授权、结果订阅和人工恢复不是等价能力。业务项目以后需要 Orbit,可独立依赖它的 SDK;当前没有承诺只换队列适配器就能迁移其全部语义。
|
||||
|
||||
## 能力与边界
|
||||
|
||||
| 模块 | 本批实现 | 明确边界 |
|
||||
| --- | --- | --- |
|
||||
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
|
||||
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
|
||||
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
|
||||
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别、轮转/脱敏/堆栈/慢请求 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
|
||||
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
|
||||
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
|
||||
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
|
||||
| `storage` | Local、S3/R2、OSS、临时下载、受限直传、云端分片 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
|
||||
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
|
||||
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
|
||||
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
|
||||
|
||||
日志的配置、脱敏边界、轮转和慢请求记录见 [logx 使用说明](logx/README.md)。框架保留通用 Context/结构化字段扩展,不新增固定业务身份上下文。
|
||||
|
||||
## 对象存储
|
||||
|
||||
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
|
||||
|
||||
```go
|
||||
type FileService struct { Files storage.Store }
|
||||
|
||||
// Local:私有目录,流式临时文件 + 原子替换。
|
||||
files, err := local.New("/srv/app/objects", 32<<20)
|
||||
|
||||
// S3:AWS/MinIO 或其他经过契约测试的 S3 兼容服务。
|
||||
files, err := s3.Open(s3.Config{
|
||||
Endpoint: "s3.us-east-1.amazonaws.com", Region: "us-east-1",
|
||||
Bucket: "example-bucket", Secure: true, MaxBytes: 32<<20,
|
||||
Credentials: credentialProvider, Transport: transport,
|
||||
})
|
||||
|
||||
// Cloudflare R2:复用 S3 适配器,签名区域为 auto。
|
||||
cfg, err := s3.R2Config(accountID, bucket, credentialProvider)
|
||||
cfg.Transport = transport
|
||||
files, err := s3.Open(cfg)
|
||||
|
||||
// OSS:单独适配官方 Go SDK v2,保留平台签名与凭据提供器。
|
||||
client := ossSdk.NewClient(ossSdk.LoadDefaultConfig().
|
||||
WithRegion("cn-hangzhou").
|
||||
WithCredentialsProvider(ossCredentials.NewEnvironmentVariableCredentialsProvider()))
|
||||
files, err := oss.New(client, bucket, 32<<20)
|
||||
```
|
||||
|
||||
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
|
||||
|
||||
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,并检查实际读取长度。默认单对象 32 MiB,流式读写;OSS 简单 Put 上限 5 GiB,更大使用独立分片能力。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
|
||||
|
||||
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。受限直传和云端分片现已提供可选接口,见 [上传与续传说明](storage/README.md)。生命周期、ACL、CDN 自定义域名和版本管理仍由平台/部署管理。
|
||||
|
||||
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。
|
||||
|
||||
验证区分:Local 契约、OSS SDK HTTP fixture、S3 SDK fixture 和 R2 签名参数属于本地测试;Linux runner 再跑真实 MinIO 上传/读取/删除与签名拒绝。**MinIO 通过不能代替 OSS/R2 实网验证。**迁移存储平台还需要复制已有对象和调整部署权限,不只是切配置。
|
||||
|
||||
## 可靠队列
|
||||
|
||||
```go
|
||||
backend, err := queueRedis.New(redisClient, queueRedis.Config{
|
||||
Namespace: "serviceA:jobs:v1", MaxRecords: 1000,
|
||||
MaxPayloadBytes: 64 << 10, DedupTTL: 24*time.Hour,
|
||||
})
|
||||
consumer, err := queueRedis.NewConsumer(backend, queueRedis.ConsumerConfig{
|
||||
Workers: 4, MaxAttempts: 3,
|
||||
TaskTimeout: 30*time.Second, VisibilityTimeout: time.Minute,
|
||||
})
|
||||
|
||||
// ID 与内容在业务端持久化;超时后复用原 ID 和原内容。
|
||||
receipt, err := backend.Publish(ctx, queue.Message{
|
||||
ID: operationID, Name: "generateReport", Payload: payload,
|
||||
})
|
||||
|
||||
// 在服务生命周期中运行;返回错误须让监控/进程管理感知。
|
||||
err = consumer.Run(serviceContext, func(ctx context.Context, item queue.Delivery) error {
|
||||
return generateReportIdempotently(ctx, item.ID, item.Payload)
|
||||
})
|
||||
```
|
||||
|
||||
业务依赖 `queue.Publisher` / `queue.Consumer`,Redis 类型留在组合处。receipt 表示受理,不代表业务成功。普通 error/panic 按配置重试,`queue.Reject(code)` 保留为失败消息;超过次数同样保留。`DeadLetters(after, limit)` 是有界分页,`DiscardDead` 仅在人工核对后删除匹配的失败消息,并继续保留一段提交去重窗口。错误正文不写入失败记录。需要再次处理时由业务决定新操作身份及去重策略,框架不自动无限重投。
|
||||
|
||||
一个 namespace 使用一个受管 consumer group;多个 consumer 可竞争消费同一队列。配置一致性检查拒绝不同的容量/去重策略和执行超时/重试策略混用。确认检查当前 pending owner,旧执行不能确认新执行;但不能阻止旧处理器已发出的外部副作用,因此仍必须业务幂等。处理器需响应 context;Go 无法强杀不配合的函数,超过 shutdownTimeout 返回错误时,最多 Workers 个处理器可能尚未退出。
|
||||
|
||||
容量包含待处理、失败及未过期的已完成身份,满额拒绝新 ID,保留原 ID 重报回执。完成/人工丢弃后的身份按 DedupTTL 过期;未完成/失败消息不自动清除,不用 MAXLEN 静默裁掉未消费消息。Redis 需启用合适的 AOF/复制策略与 noeviction;本批验证的是投递/接管协议,未证明 Redis 主从故障切换后零丢失。异常进程留下的无 pending consumer 元数据可在运维核实后清理,不能用 DELCONSUMER 丢掉尚未接管的 pending。
|
||||
|
||||
`task.Runner` 仍适用于本进程短任务和定时轮询,重启会丢失;需要持久交付时使用 `queue`。这里没有分布式 Cron、业务工作流、事务 Outbox 或 exactly-once 承诺。
|
||||
|
||||
## 认证与授权
|
||||
|
||||
`auth.NewJWT` 要求固定 issuer/audience 和可信 kid→公钥列表;只开放 RS256 / EdDSA,必须有 exp/sub,检查有效期、nbf 与 iat。`ReplaceKeys` 原子替换列表,轮换期可同时保留新旧 kid。JWT 签发、私钥托管、登录口令验证留给身份服务。
|
||||
|
||||
`auth.NewSessions(cacheStore, "sessions:", time.Hour)` 签发随机 opaque bearer token;存储仅使用 token 的 SHA-256 键,可 `Revoke`。Redis 缓存可共享 Session,内存缓存只在单实例生效且可能因容量淘汰提前登出。不要把 Session token 当普通缓存值写日志。
|
||||
|
||||
`RequireAuth(authenticator)` 在前,`RequirePermission(policy, action, resource)` 在后。Policy 接收已验证 Principal,业务自行校验租户、角色、资源归属。示例 `examples/platform` 验证了无身份 401、角色不足 403 和授权写入。
|
||||
|
||||
## 配置与日志热更新
|
||||
|
||||
`config.LoadFile(path, prefix)` 按默认值→文件→非空环境变量加载并验证。两种文件格式都使用 JSON tag 的 lowerCamelCase,duration 可以写 `"5s"`;文件上限 1 MiB,拒绝未知字段、多文档和非法值。自定义 prefix 要包含期望的尾部下划线,例如 `BILLING_`。空环境变量沿用文件值,不清空已配置字段。
|
||||
|
||||
```yaml
|
||||
address: '127.0.0.1:8080'
|
||||
transport: native
|
||||
shutdownTimeout: 10s
|
||||
log:
|
||||
level: info
|
||||
format: json
|
||||
```
|
||||
|
||||
先按配置创建 `logx.New(cfg.Log)`,再 `config.NewReloader(path, prefix, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`;后者阻塞,生命周期由应用负责。只有 `log.level` 允许变化,校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
|
||||
|
||||
## 熔断与观测
|
||||
|
||||
为每个上游创建 `resilience.New(Config{MaxConcurrent: 16, FailureThreshold: 5})`;通用调用用 `Do`,HTTP 用 `resilience.Transport{Base: transport, Breaker: breaker}`。HTTP 槽位覆盖响应体读取,必须关闭 body;网络错误/5xx 计失败,冷却后只放行一个探针。不要对效果未知的写操作自行换身份重试。
|
||||
|
||||
`telemetry.New` 接受 OTel `sdktrace.SpanExporter` 和 `sdkmetric.Reader`;可注入官方 OTLP exporter、PeriodicReader 或测试用 exporter,基础包不强制选择 collector 协议。`app.Use(runtime.Middleware())` 采集入站,`telemetry.Transport` 采集出站并传播 W3C trace context;Redis 使用 `client.AddHook(telemetry.RedisHook{Runtime: runtime})`,SQL 使用 `telemetry.SQL{DB: db, Runtime: runtime}`。业务操作/事务可用 `runtime.Do(ctx, "fixedOperationName", fn)` 包裹。
|
||||
|
||||
创建者负责在停止请求/Worker 后调用 `runtime.Shutdown(ctx)` 刷新并关闭 exporter/reader。指标按注册路由模板和固定方法分类;自动采集不记录 URL query、SQL 正文、Redis key、token 或业务载荷。业务自定义 span 名称同样应固定,避免用用户 ID 或对象 key。
|
||||
|
||||
## 分布式限流与幂等
|
||||
|
||||
`middleware/redis.NewRateLimiter(client, namespace, cfg)` 跨副本使用 Redis 时间与原子令牌桶;满额/存储失败返回 503,额度不足返回 429 和 Retry-After。`NewIdempotencyStore` 注入 `middleware.IdempotencyConfig.Store`;Scope 必须来自认证后的租户/用户,先认证再幂等。不同请求内容复用同一身份返回 409,已完成请求回放原响应和 requestId。
|
||||
|
||||
claim 在开始业务前持久保存,完成按 owner 原子检查。进程崩溃留下的 running claim 不自动按租约过期;返回 409 等待核查,`Resolve` 仅由运维核对后显式调用。已保存的完成/失败记录按配置 TTL 清理,TTL 过后不再提供业务去重保证。失败结果、含 Set-Cookie/no-store 的响应不用于成功回放。共享 Redis 需要 noeviction 和独立 namespace,数据丢失时不存在框架层面的重复执行保证。
|
||||
|
||||
## 迁移、审计与分页
|
||||
|
||||
迁移命名:`0001_createUsers.up.sql`,可附 `.down.sql`,但第一版 CLI 只执行 up。`go-omni migrate --dir migrations` 默认打印版本/方向/SHA-256 清单,不连接数据库;部署步骤显式传 `--apply` 才读取 `GO_OMNI_DB_DSN` 执行。迁移使用独立连接与数据库锁;MySQL DDL 可能部分提交,失败保留 dirty,禁止自动 force 或回滚。当前基于 golang-migrate,数据库版本表不保存每个历史文件校验和,部署系统应保留并校对预览清单。取消在当前有界 SQL 操作结束后停止,重试需新建 Runner。
|
||||
|
||||
审计:`audit.New(sink, "approvedField")` 建白名单,再 `Record(ctx, Event{ActorID, Action, Resource, Outcome, Fields})`;补入服务端时间、requestId/traceId,只保留明确允许的属性,Sink 失败原样返回。JSONSink 不保证与业务提交原子一致,需要这种语义时由业务在同一数据库事务内提供 Sink/Outbox。
|
||||
|
||||
分页:`pagination.Parse(request.URL.Query(), pagination.Config{})` 默认 page=1/pageSize=20,上限 100,最大 offset=1,000,000;拒绝重复参数、溢出和越界。`pagination.NewResult(request, items, total)` 输出 items/page/pageSize/total/totalPages,空列表为 `[]`。查询与租户过滤仍由 Repository 负责。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --advanced --goproxy https://goproxy.cn,direct --output performance/advancedLinux.json
|
||||
```
|
||||
|
||||
Linux runner 顺序运行编译、Redis、MinIO、MySQL 四个容器,各限 2 CPU、2 GiB、256 pids;数据阶段无外网、无发布端口,不连接现有数据库。原始结果和源码摘要以生成的 JSON/TXT 为准,未提供云凭据的 OSS/R2 不标为实网通过。
|
||||
|
||||
本批已通过上述 Linux 验证,源代码提交、98 个文件摘要、服务版本和边界见 [验收记录](performance/advancedAcceptance.md)。
|
||||
275
API.md
Normal file
275
API.md
Normal file
@@ -0,0 +1,275 @@
|
||||
# API 契约与项目生成
|
||||
|
||||
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
|
||||
|
||||
## 与 go-zero 的对应关系
|
||||
|
||||
调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式:统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写,不引入 go-zero 或 Gin 运行时依赖。
|
||||
|
||||
| 使用体验 | go-omni 实现 |
|
||||
| --- | --- |
|
||||
| 新建接口项目 | `go-omni api new <directory>` |
|
||||
| 按契约生成、更新 | `go-omni api generate --api user.api --dir .` |
|
||||
| 熟悉的命令别名 | `api go -api user.api -dir .` |
|
||||
| 校验契约 | `api validate --api user.api` |
|
||||
| CI 检查生成产物是否过期 | `api generate --api user.api --dir . --check` |
|
||||
| 业务逻辑目录 | `internal/service/<group>`,沿用 Omni 的 Service 分层 |
|
||||
| 依赖装配 | `internal/servicecontext` |
|
||||
| 接口文档 | 自动生成 `docs/openapi.json`,同时提供 `/openapi.json` |
|
||||
|
||||
这是明确限定的 go-zero 风格方言,**不是完整 goctl 兼容实现**。不支持的声明和校验标签会在生成前报错。
|
||||
|
||||
## 立即试用
|
||||
|
||||
源码开发时,从 go-omni 仓库运行,显式引用本地框架:
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni api new \
|
||||
--module example.com/omniDemo --framework-dir . ../omniDemo
|
||||
|
||||
cd ../omniDemo
|
||||
go mod tidy
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
访问 `http://127.0.0.1:8080/api/greet/Alice` 可得到带 `data.message` 的统一响应。`api new` 的问候示例包含可运行逻辑;自定义契约新生成的 Service 会明确返回 501,等待业务实现。
|
||||
|
||||
用现成的多分组定义生成项目,在 go-omni 仓库执行:
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni new --api examples/contracts/user.api \
|
||||
--module example.com/userService --framework-dir . \
|
||||
--log-output file --log-format json ../userService
|
||||
```
|
||||
|
||||
`api generate` 在新目录中也会生成完整项目。默认只监听 localhost;`GO_OMNI_HTTP_ADDR` 修改地址,`GO_OMNI_TRANSPORT=native` 切换网络层。配置顺序为默认值 → `etc/config.yaml` → 环境变量。文件日志支持现有轮转配置。远程试用使用与 CLI 一致的固定版本,见 [发布说明](RELEASE.md)。
|
||||
|
||||
API 项目现在支持组合数据库、存储和日志:
|
||||
|
||||
```sh
|
||||
go-omni new --api user.api --module example.com/userService \
|
||||
--database mysql --storage local --log-output file ./userService
|
||||
```
|
||||
|
||||
- `--database none|memory|mysql`:API 默认 `none`,`memory` 是不创建数据库连接的别名;MySQL 生成连接池、配置、事务入口和 readiness 探针。
|
||||
- `--storage none|local|s3|r2|oss`:默认 none,生成统一 Files / Uploads / Downloads / UploadLinks 依赖;本地签名对象端点为 `/storage/object`。
|
||||
- `--log-output stdout|stderr|file` 与 `--log-format json|text`:配置和无配置文件启动使用一致默认值。
|
||||
|
||||
初次选项记录在 `.goOmni/apiManifest.json`,以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错,因为初始化代码属于业务方;可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。旧式 CRUD 脚手架仍保留。
|
||||
|
||||
MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
|
||||
|
||||
HTTP 请求类型不能推导数据库表结构;迁移和 Repository 由业务自己维护。完整案例见 [用户管理示例](examples/userService/README.md),覆盖真实 MySQL、事务审计、鉴权、文件导出和日志。
|
||||
|
||||
## 接口定义
|
||||
|
||||
```go
|
||||
syntax = "v1"
|
||||
|
||||
type UpdateUserReq {
|
||||
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||
Notify bool `form:"notify,default=false"`
|
||||
Name string `json:"name" binding:"required,max=50"`
|
||||
Age *int `json:"age,optional,range=[0:150]"`
|
||||
}
|
||||
|
||||
type UserResp {
|
||||
UserID int64 `json:"userId"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
@server (
|
||||
group: user
|
||||
prefix: /api/v1
|
||||
)
|
||||
service user-api {
|
||||
@doc "Update a user"
|
||||
@handler updateUser
|
||||
put /users/:userId (UpdateUserReq) returns (UserResp)
|
||||
}
|
||||
```
|
||||
|
||||
类型和 Go 字段使用导出命名;JSON、query、path 名称使用 lowerCamelCase;Header 使用 HTTP 标准名称。响应类型定义业务返回值。未配置时使用默认公共体;当前源码支持在 `.api` 中声明公共响应模板,选择字段和结构,并按分组复用或由单接口覆盖,详见 [公共响应体](RESPONSE.md)。
|
||||
|
||||
支持:
|
||||
|
||||
- 可省略的 `syntax = "v1"`;`info` 中的 title、version 和描述性元数据。
|
||||
- 单独的 `type X { ... }` 与 `type (...)`;基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。
|
||||
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
|
||||
- 同一 service 名称的多个块;`@server` 的 group、prefix、middleware、auth、errors、response,group 可以写 `admin/user`。
|
||||
- 每个路由必须有 `@handler`;可选单行/多行 `@doc`、`@auth`、`@errors`、`@response`;GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。
|
||||
- 可省略请求或响应;有响应默认 200,无响应默认 204。Omni 扩展 `@status 201`、`@status 202`;204 不允许响应类型,HEAD 不允许响应体,GET/HEAD 不允许 JSON 请求体。
|
||||
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query,不解析 multipart 或表单 body。query 切片用重复键;header/path 使用标量。
|
||||
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
|
||||
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
|
||||
|
||||
暂不支持匿名嵌入、别名、递归类型、外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证验证通过声明关联的命名中间件实现。未列出的语法会明确报错。
|
||||
|
||||
业务错误、具名鉴权、多行文档及下述开发命令从 CLI `v0.7.0-preview.1` 提供。公共响应模板从 `v0.8.0-preview` 提供,CLI 与业务项目中的框架依赖需一起升级。
|
||||
|
||||
## 业务错误:声明、代码和文档共用一份定义
|
||||
|
||||
```go
|
||||
error (
|
||||
Unauthenticated 401 "UNAUTHENTICATED" "authentication required"
|
||||
Forbidden 403 "FORBIDDEN" "permission denied"
|
||||
UserNotFound 404 "USER_NOT_FOUND" "user not found"
|
||||
EmailTaken 409 "EMAIL_TAKEN" "email already in use"
|
||||
)
|
||||
```
|
||||
|
||||
也可写单条 `error EmailTaken 409 "EMAIL_TAKEN" "email already in use"`,或把错误目录放进相对导入的 `errors.api`。
|
||||
名称用 UpperCamelCase,HTTP 状态范围 400–599,错误码用大写字母/数字/下划线且全局唯一,message 是可公开返回的稳定提示。
|
||||
|
||||
`@server (errors: Unauthenticated)` 声明组级错误;路由 `@errors UserNotFound, EmailTaken` 追加到组级列表并去重。
|
||||
`@errors none` 清空继承的业务错误。未声明、重复声明或生成符号冲突会报告源码位置。
|
||||
|
||||
生成 `internal/apierrors/errorsGen.go`:
|
||||
|
||||
```go
|
||||
return nil, apierrors.EmailTaken()
|
||||
// 可用 apierrors.CodeEmailTaken 引用同一个错误码。
|
||||
```
|
||||
|
||||
每次调用返回新错误,避免并发修改共享错误对象。生成器不会自动把任意数据库错误转成业务错误;由 Service/适配层调用这些函数。
|
||||
内部 Cause 可能进入访问日志,敏感驱动报错应像 userService 示例一样先归类再转换。
|
||||
|
||||
OpenAPI 对每个状态按选定的响应模板生成结构和命名 examples,同一 409 下多个错误不会相互覆盖;平铺模式的纯业务状态还列出 code enum。
|
||||
默认框架错误状态与自动添加的鉴权 401 保留开放的 code 字段,容纳校验器/中间件实际错误。
|
||||
`x-omni-errors` 提供错误目录与路由关联。文档声明不拦截、改写或穷举运行时的所有错误。
|
||||
|
||||
## 鉴权:声明与路由注册一起变化
|
||||
|
||||
```go
|
||||
auth UserAuth (
|
||||
type: bearer
|
||||
middleware: Auth
|
||||
description: "Application access token"
|
||||
)
|
||||
|
||||
@server (
|
||||
group: user
|
||||
prefix: /api/v1
|
||||
auth: UserAuth
|
||||
errors: Unauthenticated
|
||||
)
|
||||
service user-api {
|
||||
@errors Forbidden, EmailTaken
|
||||
@handler createUser
|
||||
@status 201
|
||||
post /users (CreateUserReq) returns (UserResp)
|
||||
|
||||
@auth none
|
||||
@errors none
|
||||
@handler publicStatus
|
||||
get /status
|
||||
}
|
||||
```
|
||||
|
||||
`auth` 是具名契约,`middleware` 指向实际 hook;生成器把它放在普通 middleware 前面执行,并在注册任何路由前检查 hook 非空。
|
||||
新 hook 未实现时返回 501,已有手写 hook 保留。需要在 hook 中验证凭证、失效时间、签发者、权限等;声明不会自动实现登录或 JWT 校验。
|
||||
|
||||
- `type: bearer`:对应 HTTP Bearer,可选 `bearerFormat: JWT`,这只是文档提示。
|
||||
- `type: apiKey`:必须同时声明 `in: header` 或 `in: cookie` 和 `name`,例如 `X-API-Key` 或 `sessionId`。
|
||||
- `auth: UserAuth` 可在组级使用;路由 `@auth OtherAuth` 替换,`@auth none` 撤销继承的声明鉴权。普通 middleware 继续保留。
|
||||
- 不要再把该 auth 的 hook 重复写进普通 middleware。不支持 query token、多个方案 AND/OR、OAuth2 scopes 或隐含认证策略。
|
||||
- 被保护路由自动描述 401;403 和具体错误码由 `@errors` 明确声明。未声明 auth 的旧项目仍按其原有 middleware 执行。
|
||||
|
||||
文档生成 `components.securitySchemes` 与逐接口 `security`;显式公共接口使用空数组。
|
||||
结构遵循 [OpenAPI 3.0.3 Security Requirement](https://spec.openapis.org/oas/v3.0.3.html#security-requirement-object)。
|
||||
|
||||
## 文档和日常开发命令
|
||||
|
||||
保留单行 `@doc "摘要"`,也可以:
|
||||
|
||||
```go
|
||||
@doc (
|
||||
summary: "Create a user"
|
||||
description: "Writes user data and audit in the same transaction"
|
||||
tags: users, administration
|
||||
deprecated: false
|
||||
)
|
||||
```
|
||||
|
||||
未设置 tags 时使用 group 分组。`info` 的 description/desc 也会进入 OpenAPI;两者不能同时声明。
|
||||
|
||||
在生成项目目录内,不必重复输入 API 路径:
|
||||
|
||||
```sh
|
||||
go-omni api validate # 发现 manifest 中的入口并校验
|
||||
go-omni api inspect # 方法、路径、Handler、鉴权与业务错误总览
|
||||
go-omni api generate --dry-run # 预览新增/更新/移除,完全不写文件
|
||||
go-omni api generate # 实际生成,保留手写业务
|
||||
go-omni api generate --check # 只读检查,可用于 CI
|
||||
go-omni api export > openapi.json # 独立导出纯 JSON,无需启动服务
|
||||
go test ./...
|
||||
```
|
||||
|
||||
`--dry-run` 和 `--check` 不能组合。新项目预览也不创建目录;预览不是审批或锁定,实际生成会重新检查文件状态。
|
||||
在项目外可显式传 `--api`;生成时传 `--dir` 可定位其他项目。不会猜测多个 .api 中哪个是入口。
|
||||
路由总览的 `(undeclared)` 仅表示没有声明 auth;旧式中间件仍可能做认证。
|
||||
外发文档可用 export;项目内受生成器管理的 `docs/openapi.json` 请通过 generate 更新,以同步文件哈希。
|
||||
|
||||
## 参数语义
|
||||
|
||||
无 optional/default 的字段按“是否出现”判断必填,`false`、`0`、`""` 是已提供的值。额外声明 `binding:"required"` 才会拒绝标量零值。默认值只在字段缺失时使用,不替换显式的空值。
|
||||
|
||||
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;非指针拒绝 null;`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
|
||||
|
||||
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
|
||||
|
||||
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;非 nullable 集合请返回空集合而不是 nil。
|
||||
|
||||
## 生成目录和业务边界
|
||||
|
||||
```text
|
||||
api/ # 开发者维护的契约及导入
|
||||
cmd/server/main.go # 启动、日志、优雅退出
|
||||
etc/config.yaml
|
||||
internal/settings/
|
||||
internal/servicecontext/ # 依赖初始化和释放
|
||||
internal/types/typesGen.go
|
||||
internal/apierrors/errorsGen.go # 声明错误后生成
|
||||
internal/handler/routesGen.go
|
||||
internal/handler/openapiGen.go
|
||||
internal/handler/<group>/handlersGen.go
|
||||
internal/service/<group>/service.go
|
||||
internal/service/<group>/<operation>.go
|
||||
internal/repository/<group>/repository.go
|
||||
internal/middleware/registryGen.go
|
||||
internal/middleware/<name>.go
|
||||
docs/openapi.json
|
||||
.goOmni/apiManifest.json
|
||||
```
|
||||
|
||||
Service 方法接收标准 `context.Context` 与生成的请求 DTO;Handler 调用 Service 并完成响应包装。网络适配器仍经 Omni/httpx 接入。Repository 是业务所有的扩展位置,不自动生成虚构 CRUD 查询。
|
||||
|
||||
命名中间件按声明顺序组合,第一个在最外层。新 hook 初始返回 501,必须实现后才放行;registry 自动维护,hook 文件只创建一次。缺失 hook 在注册任何路由前报错。文档通过 `x-omni-middleware` 标出命名 hook,不把名称误当成已实现的认证协议。
|
||||
|
||||
## 安全更新与 CI
|
||||
|
||||
在生成项目中安装/使用与框架版本相符的 go-omni CLI:
|
||||
|
||||
```sh
|
||||
go-omni api validate --api api/user.api
|
||||
go-omni api generate --api api/user.api --dir .
|
||||
go-omni api generate --api api/user.api --dir . --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
`Gen.go` 和 OpenAPI 由生成器管理,manifest 保存其 SHA-256。手动修改后更新会报冲突。Service、Repository、main、配置、依赖装配和中间件实现按字节保留;只补缺失文件。删除接口/分组时只移除旧的生成文件,保留业务代码;已删除类型仍被旧业务引用时,需要明确迁移该业务文件,编译器会提示。
|
||||
|
||||
`--check` 只读,检查产物是否过期,以及所需骨架是否缺失。契约复制到项目后,请编辑项目内的 API;外部定义与已存在副本不一致时会拒绝覆盖。一个项目由一个 API 入口管理,可以通过导入拆分文件。
|
||||
|
||||
生成器拒绝输出路径穿越、受管理路径上的符号链接、大小写路径冲突和无所有权的覆盖。生成前统一校验/格式化,锁文件防止两个生成器同时修改;文件以原子替换写入,普通写入失败会回滚。本版不是整个目录的崩溃原子事务:进程被强杀或机器断电后,应按 Git 恢复生成产物和 manifest,再重试。请将 manifest 与生成产物一并提交,也不要在生成过程中编辑这些文件。
|
||||
|
||||
生成器不自动安装依赖、不访问数据库、不推送仓库、不替业务实现权限规则。
|
||||
|
||||
## 验证
|
||||
|
||||
```sh
|
||||
go test -race -p=2 ./contract ./openapi ./cmd/go-omni/...
|
||||
python3 scripts/verifyAPI.py
|
||||
```
|
||||
|
||||
Go 测试覆盖契约拒绝规则、多源绑定、零值/缺失、默认/范围/嵌套集合、文档约束、幂等更新、只读检查、业务保留、删除路由、路径边界及生成项目编译。Python 验收会在临时目录生成项目,分别启动 nethttp/native,验证真实 HTTP 请求、中间件、日志 requestId 和优雅退出;结束后清理自己创建的进程和临时文件。结果记录在 `performance/apiGenerationLocal.json`。
|
||||
124
CHANGELOG.md
Normal file
124
CHANGELOG.md
Normal file
@@ -0,0 +1,124 @@
|
||||
# Changelog
|
||||
|
||||
## v0.8.0-preview
|
||||
|
||||
- 公共响应 `codeType` 支持 `int32` / `int64` / `string`,校验默认码及映射范围,OpenAPI 标注整数位宽;JSON/YAML 和生成代码保留 int64 精度,旧 `integer` 兼容。
|
||||
- 公共 JSON 响应体改为可配置模板:支持嵌套结构、独立错误体、错误码类型/映射、提示与空值行为;旧 DTO 仅保留源代码兼容。
|
||||
- `.api` 支持 `response Name (...)`、分组引用和单路由覆盖;`@response raw` 输出原始业务 JSON。生成的模板、路由和 OpenAPI 使用同一份配置。
|
||||
- 提供应用级/接口级编码器,以及 Text / XML / Bytes 直接输出,允许三方回调独立控制成功和失败的状态、Header、内容。
|
||||
- 接口响应策略可在全局中间件之前挂载,覆盖提前鉴权/限流失败和 Context 克隆;内置路由 404/405 使用应用配置。
|
||||
- 详见 [公共响应体](RESPONSE.md)。CLI 与项目框架依赖需要一起升级到本版。
|
||||
|
||||
## v0.7.0-preview.1
|
||||
|
||||
- `.api` 支持集中业务错误目录、组级与路由级错误声明,生成状态/错误码/提示一致的 `internal/apierrors` 函数,以及 OpenAPI 命名响应示例。
|
||||
- 具名鉴权声明支持 Bearer、header/cookie API key,关联真实中间件;支持组级继承、单路由覆盖与公共接口,认证 hook 优先执行并在启动时校验。
|
||||
- 新增多行 `@doc` 的摘要、描述、标签与弃用标记,`info.description` 同步文档。
|
||||
- 新增 `api inspect`、`api export`、`api generate --dry-run`;项目内自动识别 manifest 的 API 入口,减少重复参数。
|
||||
- 用户示例用真实 401/403/409 验证声明与行为,业务直接使用生成的错误函数;生成不会改写现有手写 hook 和 Service。
|
||||
- 纠正旧文档误将事务 Outbox、分布式锁列为已实现的描述;这两个模块当前没有实现。
|
||||
|
||||
## v0.7.0-preview
|
||||
|
||||
- 新增 `.api` 契约驱动生成器:DTO、严格多来源绑定与校验、路由、分组 Handler / Service / Repository、命名中间件及 OpenAPI;生成文件哈希管理、业务文件保护和只读 `--check`。
|
||||
- `new --api` / `api new` / `api generate` 可以组合 MySQL、日志以及 Local / S3 / R2 / OSS。保存初始基础设施选项,重生成自动沿用;显式切换已有项目选项会拒绝,避免静默丢失初始化代码。
|
||||
- 新增 `examples/userService` 独立业务示例与 `verifyAPIWorkflow.py`:用户 CRUD / 分页、管理员 Token 鉴权、MySQL 事务审计、故障回滚、文件导出、requestId 日志、数据库 readiness 和优雅退出;支持从远程版本验收。
|
||||
- 日志新增文件轮转、结构化敏感字段脱敏、异常堆栈及慢请求配置;存储新增统一上传策略、直传签名和云端分片能力。
|
||||
- 新增缓存、HTTP 客户端、限流 / 幂等 / 超时、后台任务、Redis 队列、审计、迁移与 telemetry 等可选模块;业务身份与 Orbit 保持由应用自行组合。
|
||||
- `database.ApplyEnv` 支持文件/默认配置叠加环境变量;数据库配置使用 lowerCamelCase,DSN 不参与 JSON/YAML 序列化。
|
||||
- 修复 native 在业务层提前拒绝请求体为 413 时等待继续读取 body 的问题,拒绝后关闭未读连接。
|
||||
|
||||
以下是自上一版本以来的网络层变化与历史验收记录:
|
||||
|
||||
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
||||
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
||||
- 新增 `TestTLSMixedSoak` 与 `verifyHTTP2.py --scenario mixed`:8 条 TLS 连接、128 个 worker 持续混跑普通请求、慢上传和慢下载,校验 TLS/HTTP2、requestId、响应内容及传输摘要;分别统计三类请求与每条连接覆盖情况,并记录退出/GC 后的回收数据。
|
||||
- 完成 Linux amd64 30 分钟多连接慢客户端混合长测:53,191,687 次请求(含 106,971 次慢上传、5,588 次慢下载),0 失败与协议错误;采样 RSS 峰值约 116.6 MiB,连接/请求、goroutine 和 FD 回收,释放测试数据后存活堆约 450 KiB。保留完整日志、120 份采样及每条连接的分类计数。
|
||||
- 新增独立 `transport/http2` 实验 Runner:支持 h2c/TLS h2、连接上限、全局请求并发闸门、GOAWAY 排空、RST_STREAM 单流取消和生命周期统计;不改变 native HTTP/1.1 默认路径。
|
||||
- HTTP/2 实验 Runner 增加标准 1 MiB 头部上限、默认 8 MiB 请求体上限、128 路同连接多路复用、2 MiB 慢读取流控验收、协议错误、RST_STREAM 和 GOAWAY 帧计数;已声明超限请求在进入业务 handler 前返回 413。
|
||||
- 新增可配置时长的 HTTP/2 TLS soak:默认 3 秒,Linux 验收可通过 `GO_OMNI_HTTP2_SOAK_DURATION=30m` 覆盖,混合验证 128 路 stream、慢 handler、请求体、Keep-Alive 和退出回收。
|
||||
- HTTP/2 soak 每 15 秒输出请求/失败数、Go 堆、Linux RSS、goroutine、FD 和协议统计;校验响应协议、内容和独立请求 ID,不再忽略压测期限附近已发出请求的失败。
|
||||
- 补充 HTTP/2 慢响应流窗口为零时的隔离与恢复验收:同连接 204、另一连接带 body 响应、WINDOW_UPDATE 恢复发送及 RST_STREAM 释放请求槽位;补充 TLS 握手未完成时的连接上限拒绝、握手超时回收和正常请求恢复。
|
||||
- 补充连接级接收窗口耗尽测试:保持 stream 信用充足,消费完 65,535 字节连接窗口后确认大响应暂停,同连接 204 和另一条连接仍可完成,连接级 WINDOW_UPDATE 恢复后继续复用;补充零窗口阻塞写在关闭超时后解除阻塞、取消请求与连接回收的验收。
|
||||
- 新增 `scripts/verifyHTTP2.py`,固定在 Go 1.25 Linux 容器中运行 HTTP/2 TLS 长测;默认使用普通模式,`--race` 默认运行 60 秒。容器默认限制为 2 核、2 GiB 内存、256 个进程/线程,带总时限、中断清理和日志大小限制。
|
||||
- 完成 Linux amd64 单 TLS 连接、128 个 worker 的 30 分钟 HTTP/2 长测:58,102,476 次请求,0 失败与协议错误,采样 RSS 峰值约 22.6 MiB(包含客户端和服务端),退出后连接/请求/FD 回收;保留原始日志和 120 份采样。
|
||||
- native TLS 明确限制 ALPN 为 `http/1.1`:缺省自动宣告该协议,误配置 `h2` 等未实现协议时在启动阶段报错。
|
||||
- native `Server.Stats()` 新增连接和请求过载拒绝计数;新增可信代理配置,访问日志只在 socket 对端属于配置网段时解析 `X-Forwarded-For`。
|
||||
- 新增 transport-neutral `metrics` 包,支持请求数、错误数、in-flight、累计耗时和状态分布,并可直接注册 `/metrics` 接口。
|
||||
- native 新增 `Server.Stats()`,提供 accepted connections、active connections 和 active requests 生命周期指标。
|
||||
- native 请求体改为流式读取,Content-Length 与 chunked body 不再在进入 Handler 前整体复制到内存;保留大小限制、读超时、trailer 校验和 Keep-Alive 排空。
|
||||
- 统一标准库与 native 的真实 TCP benchmark,增加固定并发档位、响应状态与读写错误检查。
|
||||
- 新增独立进程并发与长连接内存测量工具,记录成功吞吐、P95/P99、服务端分配、空闲连接堆内存和回收情况;修正并行 `ns/op` 与客户端/服务端混合分配的解释。
|
||||
- 加固 native HTTP/1.1 解析边界:校验 Host、重复 Content-Length、Connection token、Transfer-Encoding、控制字符和响应头名称;补充 HTTP/1.0 Keep-Alive、HEAD、204 以及异常请求测试。
|
||||
- native 支持带 trailers 的 chunked 请求体,并继续通过 `MaxBodyBytes`、读超时和头部上限控制解析内存与连接占用。
|
||||
- native 响应实现 `http.Flusher`:HTTP/1.1 流式响应自动使用 chunked framing,普通响应继续自动生成 `Content-Length`。
|
||||
- native 在长 Handler 运行期间监测客户端断开并取消请求 context,流式写失败同步取消;短请求延迟启动监测,管线数据保留给下一次请求。补充断开、监测退出和长连接复用验收。
|
||||
- native 增加 `ServeTLS` / `RunTLS` 和 `TLSConfig` 启动入口,App.RunWith 可以直接启用 TLS;补充真实 TLS 客户端握手、请求和优雅退出验收。
|
||||
- native 请求补齐 `Request.TLS` 连接状态;Linux/arm64 容器通过全量 race、vet、build,以及 TLS 并发、Keep-Alive、慢握手和连接上限回收验收。
|
||||
- 补充慢请求体超时、大请求体和同连接管线请求验收,确认流式 body 读取超时会取消请求并关闭连接,连续请求不会串线。
|
||||
- 补充 `MaxConnections` 满载拒绝与槽位释放验收,确认请求完成后连接容量可再次使用。
|
||||
- 补充 `MaxConcurrentRequests` 过载恢复验收,确认 503 拒绝后请求槽位释放并可继续处理新请求。
|
||||
- 增加高频建连/断连的状态池 churn 验收,覆盖连接对象回收与容量复用。
|
||||
- 新增 Linux Nginx 反向代理验收脚本,覆盖 native TLS、Keep-Alive、上游超时和优雅关闭。
|
||||
|
||||
## v0.6.0-preview
|
||||
|
||||
- native 网络层完成流式 Content-Length / chunked body、慢请求头、慢 body、慢 TLS 握手和 TLS 请求元数据验收;Handler 可在 body 未完全到达时开始读取,超时、畸形和超限 body 会被拒绝并关闭连接。
|
||||
- 将 native 连接 churn race 验收扩展为 16 个 worker、3,200 次建连/断连,并覆盖明文与 TLS 的连接上限、Keep-Alive 和槽位回收。
|
||||
- perfcheck 支持最长 30 分钟持续场景,新增 Linux 文件描述符采样;固定记录服务端堆、goroutine、连接数、accept 数和 FD 回收状态。
|
||||
- 新增 `scripts/verifyLinux.py`,在 `golang:1.25` Linux 容器内统一执行 race、vet、build 和可选 TLS / 非 TLS 压测。
|
||||
- 完成 Linux/arm64 30 秒高并发回归:128 并发流量、512 条空闲连接,TLS 与非 TLS 均为 0 错误,连接、goroutine 和 FD 回收正常;原始报告见 `performance/linuxSoak.json` 和 `performance/linuxSoakTLS.json`。
|
||||
|
||||
## v0.5.0
|
||||
|
||||
- 新增 `transport/native` 第一版自研 HTTP/1.1 服务端,接管 TCP 监听、连接、Keep-Alive、请求解析、响应写回和优雅关闭。
|
||||
- 新增 `App.RunWith` / `omni.Runner` 启动边界;native 通过标准 `http.Handler` 接入,现有 Handler、Service 和 Repository 无需修改。
|
||||
- native 支持 `Content-Length` / chunked 请求体、请求头 / 请求体限制、读写超时和服务关闭时的请求取消;通过对象池和 `MaxConnections` / `MaxConcurrentRequests` 控制高并发内存。
|
||||
- `omni.New()` 默认仍使用只依赖标准库的 `transport/nethttp` 路由,native 作为显式启用的网络层预览实现。
|
||||
|
||||
## v0.4.0
|
||||
|
||||
- **破坏性变更**:移除 Gin 适配器及其全部依赖,`omni.New()` 默认使用只依赖标准库的 `transport/nethttp`。
|
||||
- 移除 Gin 专属的 `Config.Mode` / `WithMode` API;网络层运行模式由具体适配器或启动层自行配置。
|
||||
- 公共 `httpx.Router`、`transport/transporttest.Run` 和 `omni.NewWithRouter` 保留,自研网络层无需修改业务 Handler / Service 即可接入。
|
||||
- benchmark 改为标准库路由和完整 GoOmni 请求链,新增 `scripts/verifyPerformance.py` 作为 race、vet、build 和性能基线入口。
|
||||
- 远程业务试用、脚手架、适配器契约和 MySQL 验收全部迁移到 `v0.4.0`。
|
||||
|
||||
## v0.3.1
|
||||
|
||||
- 新增独立业务模块验收 `scripts/verifyBusiness.py`:从远程 `v0.2.0` 升级到 `v0.3.1`,验证无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
|
||||
- 脚手架与各项远程验收脚本默认跟随 `v0.3.1`,新项目会固定到最新补丁版本。
|
||||
|
||||
## v0.3.0
|
||||
|
||||
- 新增公共适配器验收套件 `transport/transporttest.Run` 和远程消费验收脚本 `scripts/verifyTransports.py`。Gin、nethttp 使用同一套路由与业务处理链测试,并通过完整用户 CRUD 与 race 验证。
|
||||
- 统一参数、分组、方法、URL 解码、路由优先级及 404/405 规则;Gin 405 的 Allow 由适配层计算,避免方法树回退状态带入无关方法。
|
||||
- 统一路由注册校验;非法路径、重复路由、参数名冲突和 catch-all 冲突在注册时 panic。
|
||||
- `App.Handle` / `RouterGroup.Handle` 支持在完整业务中间件链内显式注册 HEAD、OPTIONS 与扩展方法。
|
||||
- JSON 在提交响应前编码,编码失败可正确返回统一 500;通过 Context 已提交的响应不会因后续错误、panic 或重复响应追加内容,访问日志保留实际提交的状态。
|
||||
- 验证 requestId 与业务日志关联、认证授权短路、请求并发隔离、context 值与 deadline 保留,以及真实客户端取消传入 Service。
|
||||
- **升级行为调整**:Gin 不再自动重定向尾部斜杠;nethttp 不再忽略尾部斜杠,通配参数包含开头的 `/`,Any 收敛为九种标准方法。HEAD 不自动回退 GET,其他方法需显式注册。404/405 为统一纯文本响应,405 附排序后的 Allow。
|
||||
- `gin.NewWithEngine` 应用 GoOmni 路由策略;原生 Gin 扩展与 Raw 流式响应不属于可移植契约。详见 `transport/README.md`。
|
||||
- 新增业务试用验收 `scripts/verifyBusiness.py`:独立目录内固定远程版本,执行 `v0.2.0` → `v0.3.0` 升级,验证生成业务服务的配置、日志、requestId、CRUD、参数校验、错误映射和优雅退出。
|
||||
|
||||
## v0.2.0
|
||||
|
||||
- 新增 `go-omni new --database mysql`,生成数据库配置、MySQL Repository、ServiceContext、`schema.sql` 和数据库就绪探针;默认仍使用内存存储。
|
||||
- 两种存储模式的脚手架均提供完整 CRUD、分页、输入校验与统一响应。新生成项目的 `POST /users` 从直接返回用户对象改为标准响应封装,用户对象位于 `data`。
|
||||
- MySQL 启动时检查连接和表结构,失败时清理连接池;数据库停机后 readiness 返回 503,恢复后自动重新检查。
|
||||
- MySQL Repository 支持 `*sql.DB` 和 `*sql.Tx`,可在同一事务中复用业务查询。
|
||||
- 修复 `MaxIdleConns=0` 未生效的问题,数据库健康检查增加超时限制。
|
||||
- 新增隔离 MySQL 容器验收脚本,覆盖真实 CRUD、事务提交和错误/panic/取消回滚、持久化、资源清理及数据库停机恢复。
|
||||
|
||||
## v0.1.1
|
||||
|
||||
- 修复脚手架生成项目固定依赖不存在的 `v0.0.0`;远程安装的 CLI 默认使用自身版本,支持 `--framework-version` 指定完整版本号。
|
||||
- 显式远程版本禁用自动本地替换;本地替换路径支持空格,并按精确 module path 识别框架目录。
|
||||
- 生成项目通过 `signal.NotifyContext` 接入优雅退出,启动过程中的错误会先执行 defer 清理再报告。
|
||||
- `App.Run` 自动处理 `SIGINT` / `SIGTERM`;`RunContext` 仍由调用方控制,退出超时会关闭活跃连接。
|
||||
- 增加请求排空、超时取消、启动失败清理测试,以及远程脚手架和真实进程信号验收脚本。
|
||||
|
||||
## v0.1.0
|
||||
|
||||
- 首个可试用版本:应用生命周期、HTTP 抽象、Gin 与标准库路由适配器、统一响应与错误。
|
||||
- 提供配置、日志、校验、认证接口、数据库基础能力、健康探针和项目脚手架。
|
||||
172
MODULES.md
Normal file
172
MODULES.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# 业务基础模块
|
||||
|
||||
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
|
||||
|
||||
这批能力从 `v0.7.0-preview` 提供。统一示例见
|
||||
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
||||
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
||||
|
||||
## 缓存:cache 与 cache/redis
|
||||
|
||||
```go
|
||||
store, err := cache.NewMemory(cache.MemoryConfig{MaxEntries: 4096, MaxBytes: 32 << 20})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(store.Close)
|
||||
loader, err := cache.NewLoader(store, cache.LoaderConfig{MaxLoads: 128, Timeout: 5 * time.Second})
|
||||
if err != nil { return err }
|
||||
value, err := loader.GetOrLoad(ctx, "tenant:42:product:7", time.Minute,
|
||||
func(ctx context.Context) ([]byte, error) { return repository.LoadProduct(ctx, "42", "7") })
|
||||
```
|
||||
|
||||
- `Store` 统一 `Get/Set/Delete`,返回字节切片;业务可用 JSON 编解码。
|
||||
- 内存缓存使用 LRU,同时限制条目数和 key/value 总字节数;字节上限不包含 Go 对象开销。
|
||||
读写复制切片,过期时懒清理,没有每个条目一个 goroutine。
|
||||
- TTL 为零表示不过期,负值拒绝;缺失返回 `cache.ErrMiss`。删除不存在的 key 成功。
|
||||
- Loader 合并**同进程、同 key**的并发回源,最多同时运行 `MaxLoads` 个不同 key 的加载。
|
||||
单个调用者取消只停止等待;共享加载有独立超时,加载函数必须响应 context。
|
||||
超限返回 `ErrBusy`,缓存故障直接返回错误,不自动当成 miss 冲击数据库。
|
||||
- 同 key 的调用必须具有相同数据含义、权限范围和 TTL;加载采用首个调用者的 context 值。
|
||||
多租户 key 必须含租户标识。进程间合并、分布式锁、缓存更新与数据库事务一致性由业务另外设计。
|
||||
|
||||
Redis 适配器借用调用方创建的 go-redis 客户端和连接池:
|
||||
|
||||
```go
|
||||
client := goredis.NewClient(&goredis.Options{
|
||||
Addr: "127.0.0.1:6379", PoolSize: 16, MaxRetries: -1,
|
||||
DialTimeout: time.Second, ReadTimeout: time.Second,
|
||||
WriteTimeout: time.Second, PoolTimeout: time.Second,
|
||||
ContextTimeoutEnabled: true,
|
||||
})
|
||||
// 启动时先使用有截止时间的 context 执行 client.Ping(ctx)。
|
||||
store, err := rediscache.New(client, "billing:") // cache/redis 的导入别名
|
||||
if err != nil { _ = client.Close(); return err }
|
||||
application.OnShutdown(client.Close)
|
||||
// store 可直接传入 cache.NewLoader。
|
||||
```
|
||||
|
||||
连接失败与 Redis 错误保持原样;只有 Redis nil 映射为 `ErrMiss`。Redis TTL 精度为毫秒,
|
||||
正的亚毫秒 TTL 提升到 1ms。适配器不负责关闭借用的客户端,Redis 服务端容量策略由部署配置。
|
||||
连接池和超时配置参考 [go-redis 官方生产使用说明](https://redis.io/docs/latest/develop/clients/go/produsage/)。
|
||||
|
||||
## 出站 HTTP:httpc
|
||||
|
||||
```go
|
||||
client, err := httpc.New(httpc.Config{
|
||||
Timeout: 3*time.Second, MaxRetries: 2, MaxConnsPerHost: 32,
|
||||
MaxResponseBytes: 1 << 20,
|
||||
})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(client.Close)
|
||||
err = client.JSON(ctx, "GET", serviceURL, nil, &result)
|
||||
```
|
||||
|
||||
客户端应在服务启动时创建并复用。默认使用独立的 `net/http.Transport` 连接池,每个目标主机
|
||||
有连接上限;该上限不是整个进程的全局连接数上限。也可以注入自有 `http.RoundTripper`,
|
||||
其连接限制和生命周期由调用方负责。这样出站实现同样可以替换。
|
||||
|
||||
总超时覆盖建连、重试、等待和读取响应体。`Do` 返回后调用方必须读取/关闭 `response.Body`;
|
||||
`JSON` 自动关闭,并限制解码前读取的字节数。`Do` 允许流式读取,本身不限制响应体大小。
|
||||
`Close` 拒绝新请求、关闭自有池的空闲连接;在途请求应先排空。
|
||||
|
||||
`MaxRetries` 默认 0,最多 5。封装只对可重放的 GET/HEAD/OPTIONS 重试网络错误及
|
||||
429/502/503/504;POST/PUT/PATCH/DELETE 不做应用层自动重试。标准库 Transport 和借用的
|
||||
RoundTripper 自身的重放规则仍生效,不能据此承诺“只执行一次”。退避带抖动,尊重
|
||||
`Retry-After`;服务器要求等待超过 `MaxBackoff` 时直接返回该响应,不提前重试。
|
||||
重定向交给业务处理,3xx 不会自动跟随。
|
||||
|
||||
默认从 context 传递 `X-Request-ID`,日志只包含方法、主机、状态、尝试次数和收到响应头的耗时,
|
||||
不记录完整 URL、查询参数、凭证和响应体。非 2xx 的 JSON 调用返回只带状态码的 `StatusError`。
|
||||
|
||||
## 请求保护:middleware
|
||||
|
||||
```go
|
||||
limiter, err := middleware.NewRateLimiter(middleware.RateLimitConfig{Rate: 20, Burst: 40})
|
||||
if err != nil { return err }
|
||||
idempotency, err := middleware.NewIdempotency(middleware.IdempotencyConfig{
|
||||
Scope: func(c *httpx.Context) (string, error) {
|
||||
principal, _ := auth.FromContext(c)
|
||||
return principal.UserID, nil // 多租户应用应组合租户和用户 ID
|
||||
},
|
||||
})
|
||||
if err != nil { return err }
|
||||
api := application.Group("/api", middleware.Timeout(3*time.Second),
|
||||
auth.RequireAuth(authenticator), limiter.Middleware())
|
||||
api.POST("/orders", createOrder, idempotency.Middleware())
|
||||
```
|
||||
|
||||
限流为进程内令牌桶,`Rate` 为每秒额度,`Burst` 为突发容量。默认按已解析的可信客户端 IP,
|
||||
可用 `Key` 按已认证身份分桶。额度不足返回 **429 + Retry-After**;key 数超过 `MaxKeys`
|
||||
返回 **503**。满容量时不会踢掉活跃桶让攻击者绕过限流;空闲桶经过完整补充周期后才允许回收。
|
||||
|
||||
Timeout 设置协作式 deadline,不额外启动 Handler goroutine。业务及下游调用必须响应 context;
|
||||
超时后尚未提交响应时返回 **504**,已经提交的响应保持原状态。它不能强制中断任意 Go 代码,
|
||||
也不能替代网络层读写超时。不要在业务代码中异步持有 `httpx.Context`。
|
||||
|
||||
幂等保护适用于小型、非流式写接口:
|
||||
|
||||
- 必须先认证/鉴权,再执行幂等中间件;`Scope` 必须取自服务端验证过的身份。
|
||||
GET/HEAD/OPTIONS 直接通过,写请求要求 `Idempotency-Key`,长度 1..128 个非空格 ASCII 字符。
|
||||
- 同 Scope + key + 方法/URL/Content-Type/body 重放首次响应;内容改变返回 **409**。
|
||||
正在执行或结果不可重放的相同请求也返回 **409**,不并发执行第二次。
|
||||
- 默认最多 32 个处理中请求、256 条记录、16 MiB 保存的响应头/body,单次请求体 1 MiB、
|
||||
响应 body 64 KiB。请求超限 **413**,容量满 **503**,响应超限 **500**。
|
||||
- panic、5xx、超时、响应超限、Set-Cookie、Cache-Control:no-store 或保存容量不足时保留失败记录,
|
||||
阻止在 TTL 内自动重执行业务。业务可能已经提交,调用方应查询实际结果后处理。
|
||||
- 完成记录默认保留 10 分钟;在途记录不会因 TTL 到期而允许重复执行。
|
||||
重放保留原响应 body、原 `requestId` 与原 `X-Request-ID`,增加 `Idempotency-Replayed:true`;
|
||||
本次访问日志仍使用本次请求 ID。
|
||||
- 记录仅在当前进程有效,重启、TTL 到期和多副本切换后均无法保证去重。支付、库存等关键写入
|
||||
应在数据库事务内配合唯一业务键。该实现不提供跨进程 exactly-once,也不支持 SSE/流式/升级连接。
|
||||
|
||||
## 后台执行:task
|
||||
|
||||
```go
|
||||
jobs, err := task.New(task.Config{Workers: 4, QueueSize: 128, TaskTimeout: 30*time.Second})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(jobs.Close) // 在注册数据库/缓存关闭之后注册,先排空任务
|
||||
future, err := jobs.Submit(serviceContext, "refreshCatalog", refreshCatalog)
|
||||
if err != nil { return err } // 满队列立即返回 ErrQueueFull
|
||||
err = future.Wait(ctx) // 可选;不等待时,任务失败仍通过 OnError/logx 报告
|
||||
err = jobs.Every("refreshCatalog", time.Minute, refreshCatalog)
|
||||
```
|
||||
|
||||
固定 worker 和有界队列控制资源。任务保留提交 context;需要独立于 HTTP 请求执行时使用服务
|
||||
context,或明确使用 `context.WithoutCancel` 保留日志关联值。任务仍受 Runner 的超时/退出控制。
|
||||
任务函数与 `OnError` 回调应及时返回,尤其要响应取消;panic 被转为失败,worker 继续工作。
|
||||
|
||||
`Every` 使用固定间隔,第一次在一个间隔后执行;同一计划上轮尚未完成、排队或队列满时跳过。
|
||||
默认最多 128 个计划,名称不可重复。`Stats` 提供 active/queued/completed/failed/skipped。
|
||||
它是进程内任务执行器,没有任务持久化、失败自动重试、cron 表达式、跨副本选主或崩溃恢复。
|
||||
|
||||
`Shutdown(ctx)` 停止接收和调度,排空已接受任务;截止时间到后取消任务并及时返回。
|
||||
忽略取消的任务不能被 Go 强杀,可能继续运行。`Close` 默认给排空 10 秒。
|
||||
|
||||
## 增加业务模块:go-omni add
|
||||
|
||||
```sh
|
||||
go build -o /tmp/go-omni ./cmd/go-omni
|
||||
/tmp/go-omni add orderItems --dir /path/to/billing
|
||||
```
|
||||
|
||||
生成 `internal/orderItems/{module.go,model,repository,service,handler,README.md}`,包含 name/createdAt
|
||||
示例模型、内存 CRUD、输入校验、错误映射及可注入 Repository。模块名使用 lowerCamelCase。
|
||||
|
||||
在现有启动代码中导入模块后调用 `orderItems.Register(application, repository, authMiddleware)`。
|
||||
脚手架不自动修改现有路由、main、go.mod 或数据库,已有目录不会覆盖。
|
||||
内存 Repository 仅为可运行示例:重启丢数据,列表未分页;正式业务应实现自己的持久化 Repository。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --goproxy https://goproxy.cn,direct
|
||||
```
|
||||
|
||||
Linux runner 顺序运行两个容器:全量 race/vet/build/独立生成项目,随后真实 Redis 适配验证。
|
||||
每次最多一个容器,限制 2 CPU、2 GiB(不额外使用 swap)、256 pids;Redis 测试容器禁外网,
|
||||
只用内部 loopback,不发布端口。Redis 集成二进制为静态非 race,适配器单测在全量 race 中覆盖。
|
||||
结束清理本轮容器,并记录提交、源文件摘要、镜像 ID、日志摘要和原有容器是否仍运行。
|
||||
|
||||
本轮通过结果和原始证据见 [业务模块验收](performance/modulesAcceptance.md)。
|
||||
89
PERFORMANCE.md
Normal file
89
PERFORMANCE.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# GoOmni 性能基线
|
||||
|
||||
这轮测量显示 native 与标准库处于相同吞吐量级,在本机短时测试中,native 的空闲长连接堆内存更少。当前 native 仅实现 HTTP 的部分能力,尚不能据此判断生产容量或替代完整标准库服务端。
|
||||
|
||||
## 运行方式
|
||||
|
||||
```sh
|
||||
# race、vet、build,以及进程内和真实 TCP benchmark
|
||||
python3 scripts/verifyPerformance.py
|
||||
|
||||
# 另加独立服务进程的并发、延迟和空闲长连接内存矩阵
|
||||
python3 scripts/verifyPerformance.py --load --output /tmp/goOmniPerf.json
|
||||
|
||||
# 只运行独立进程矩阵,默认每档 2 秒、3 轮
|
||||
go run ./internal/perfcheck --duration 2s --count 3 --output /tmp/goOmniPerf.json
|
||||
|
||||
# 在 Linux Docker 容器内固定运行 race、vet、build 和 TLS 压测
|
||||
python3 scripts/verifyLinux.py --load --tls --duration 30s \
|
||||
--concurrency 128 --idle 512 --count 1 \
|
||||
--output performance/linuxSoakTLS.json
|
||||
|
||||
# 发布候选验收可把持续流量延长到 30 分钟
|
||||
python3 scripts/verifyLinux.py --load --tls --duration 30m \
|
||||
--concurrency 128 --idle 512 --count 1 \
|
||||
--output performance/linuxSoakTLS-30m.json
|
||||
|
||||
# Linux 上用 Nginx 反向代理接入 native TLS 服务,检查 Keep-Alive、超时和优雅关闭
|
||||
python3 scripts/verifyProxy.py
|
||||
```
|
||||
|
||||
可通过 `--concurrency 1,32,128` / `--idle 128,1024` 调整矩阵;传空字符串可跳过对应场景。工具只监听本机随机端口,每个场景启动独立服务进程,并在结束后关闭客户端、检查连接排空、关闭服务。响应状态、完整响应体、requestId 或传输错误均计入失败,不能把快速返回的错误页算成成功吞吐。
|
||||
|
||||
Linux 验收脚本使用 `golang:1.25` 容器,固定执行 `go test -race ./...`、`go vet ./...`、`go build ./...`,再按参数运行 perfcheck。perfcheck 在 Linux 读取 `/proc/self/fd`,报告连接建立期间的文件描述符峰值以及回收后的数量;同时记录服务端堆、goroutine、连接数和累计 accept 数。30 秒报告用于快速回归,30 分钟命令用于发布前持续压测。
|
||||
|
||||
## 2026-09-21 本机测量
|
||||
|
||||
环境:Go 1.25.0、darwin/arm64、Apple M2 Max、GOMAXPROCS=12。网络实现来自 `3d5f225` 工作树(native 与 v0.6.0-preview 相同);本次新增测量工具,不修改服务端实现。原始测量值见 [tcpBaseline.json](performance/tcpBaseline.json)。报告时间为 UTC,北京时间是 9 月 21 日。
|
||||
|
||||
每个并发档位持续约 2 秒、重复 3 轮,每轮交替标准库/native 的先后顺序。下表为各指标的三轮中位数;全部流量场景共完成 **2,660,828** 次请求,错误 **0**。另有 128 / 1,024 条空闲长连接的建立、保持和回收测试。
|
||||
|
||||
| 并发请求 | 标准库成功请求/秒 | native 成功请求/秒 | 标准库 P95 | native P95 | 标准库 P99 | native P99 |
|
||||
| --- | ---: | ---: | ---: | ---: | ---: | ---: |
|
||||
| 1 | 19,731 | 21,497 | 0.074 ms | 0.070 ms | 0.139 ms | 0.119 ms |
|
||||
| 32 | 90,099 | 88,812 | 1.016 ms | 0.983 ms | 1.507 ms | 1.507 ms |
|
||||
| 128 | 112,422 | 112,105 | 3.277 ms | 3.408 ms | 4.719 ms | 4.981 ms |
|
||||
|
||||
128 并发时,两者吞吐差约 0.3%,native 的尾延迟略高。基于这组结果,暂时没有理由仅为健康接口吞吐就改成手写事件循环。
|
||||
|
||||
| 空闲 Keep-Alive 连接 | 标准库新增堆内存 | native 新增堆内存 | 标准库约每连接 | native 约每连接 |
|
||||
| --- | ---: | ---: | ---: | ---: |
|
||||
| 128 | 1.275 MiB | 0.658 MiB | 10,444 B | 5,389 B |
|
||||
| 1,024 | 9.871 MiB | 5.137 MiB | 10,108 B | 5,260 B |
|
||||
|
||||
1,024 条空闲连接时,native 的新增堆内存约少 48%。这是**服务端 Go 堆内存增量**,不包括 goroutine 栈、进程其他内存和内核 socket 缓冲区,不能解释为整个进程内存减半。同档服务端 `StackInuse` 分别约为标准库 8.75 MiB、native 8.69 MiB,原始报告单独记录。
|
||||
|
||||
连接断开后,两者连接数都回到 0,goroutine 分别回到 2 / 3 的初始基线。一次 GC 后仍有对象池缓存;第二次 GC 后,总堆内存中位数回到约 2.10 / 2.08 MiB(启动基线约 1.54 MiB)。堆内存不会立刻精确回到启动数值,不把缓存保留解释为泄漏,也不把这轮短测解释为长期无泄漏证明。
|
||||
|
||||
128 并发时,**服务端单独累计分配**约为标准库 4,454 B/请求、47 次/请求,native 4,297 B/请求、55 次/请求。native 的分配字节较少、次数较多;具体来源需用分配 profile 确认,不能仅凭总量归因于 HTTP 桥接或 JSON。
|
||||
|
||||
## 统计边界
|
||||
|
||||
- 同一台机器上,压测客户端和服务端是两个独立进程。两者使用同一健康接口、相同小型 JSON 响应、固定 `requestId=perfCheck` 和 Keep-Alive;不启用访问日志、TLS 或数据库。
|
||||
- 并发模型为闭环:每个 worker 等上一个请求完成后发下一个。实际经过时间包含最后请求排空;吞吐只计算成功响应。不能据此推断开放到达流量下的排队或过载表现。
|
||||
- 延迟从客户端发起请求到完整响应校验完成,包含失败请求。直方图使用每个 2 的幂区间内 16 档,报告桶上界,舍入误差至多约 6.25%。小幅差别不应过度解释。
|
||||
- 服务端 `runtime.MemStats` 通过独立 stdin/stdout 控制通道读取,客户端内存不计入。累计分配差值包含少量采样开销。堆峰值每 200 ms 采样,可能错过瞬时峰值;这些指标不是进程 RSS。
|
||||
- Linux 文件描述符来自服务端子进程的 `/proc/self/fd`,包含标准输入输出、监听 socket 和连接;macOS 等没有该路径的平台会省略该字段。回收后的数量应回到启动附近,不能代替内核 socket 缓冲或 RSS 测量。
|
||||
- 空闲场景在所有连接完成健康请求后做 GC 再采样。新增堆内存包括连接、缓冲区及分摊的运行时开销。`after` 为客户端断开并排空后一次 GC,`afterReclaim` 为第二次 GC,用于观察对象池缓存释放。
|
||||
- native 配置上限为 8,192 连接 / 4,096 活跃请求;本轮并发低于上限,未验证过载保护容量。标准库没有同样的并发闸门;比较包括各自真实实现的开销。
|
||||
- 可用 `python3 scripts/verifyPerformance.py --load --tls --duration 2s` 运行 TLS 版本的独立进程矩阵。该模式在每个子进程生成短期自签名证书,客户端仅用于本机基准验证;本报告的历史表格仍是无 TLS 数据。
|
||||
|
||||
## benchmark 口径修正
|
||||
|
||||
## TLS 验收记录
|
||||
|
||||
macOS arm64 本机的 5 秒 TLS 矩阵见 [tlsBaseline.json](performance/tlsBaseline.json):1 / 32 / 128 并发和 128 / 1,024 条空闲连接均为 0 错误;native 在 128 并发约 114,232 req/s,1,024 条空闲 TLS 连接的服务端堆增量约 8.94 MiB,连接关闭后 goroutine 回到基线。
|
||||
|
||||
Linux arm64 容器使用 Go 1.25.14、GOMAXPROCS=4 运行 3 秒矩阵,见 [tlsBaselineLinux.json](performance/tlsBaselineLinux.json):1 / 32 / 128 并发和 128 / 512 条空闲连接均为 0 错误;native 在 128 并发约 99,876 req/s,512 条空闲 TLS 连接的服务端堆增量约 4.47 MiB,连接关闭后 goroutine 回到基线。容器验证同时通过全量 race、vet、build。
|
||||
|
||||
Linux arm64 的 30 秒高并发回归见 [linuxSoak.json](performance/linuxSoak.json) 和 [linuxSoakTLS.json](performance/linuxSoakTLS.json):两份报告均使用 128 并发流量和 512 条空闲连接,所有场景错误为 0;native 非 TLS 约 146,614 req/s、TLS 约 149,720 req/s,空闲连接堆增量分别约 2.57 MiB 和 4.47 MiB,FD 均由 7 增至 519 并回到 7,goroutine 均回到启动附近。
|
||||
|
||||
目标 Linux amd64 服务器的 30 分钟 TLS 验收使用 1 个 CPU、1 GiB 内存的隔离 Docker 容器完成:net/http 约 11,865 req/s,native 约 13,098 req/s,两个流量场景均为 0 错误;native 的 P95/P99 为 44.040/52.429 ms。512 条空闲 TLS 连接时,native 服务端堆增量约 4.50 MiB(约 9,215 B/连接),文件描述符由 7 增至 519 后回到 7,goroutine 和连接均正常回收。原始报告保存在服务器的 `performance/linuxServerSoakTLS.json`。
|
||||
|
||||
`BenchmarkTransportTCPApplicationRequest/{nethttp,native}/concurrency{1,32,128}` 使用同一测试入口、客户端、应用与响应检查。原先独立的 `BenchmarkNativeApplicationRequest` 已合并,避免两个入口配置漂移。`verifyPerformance.py` 会一并运行。
|
||||
|
||||
并行 benchmark 的 **`ns/op` 是总经过时间除以请求数,反映吞吐成本,不是单请求延迟**。`B/op` / `allocs/op` 统计同一进程中的客户端与服务端总分配。此前报告的 native 15.4–16.7 µs/op、约 7.8 KB/op、100 allocs/op 应按这个含义理解,不能作为服务端延迟或独立内存指标。
|
||||
|
||||
`BenchmarkTransportRouteDispatch` 和 `BenchmarkTransportApplicationRequest` 继续保留,分别测进程内路由分发和完整框架处理链;不包含 TCP,不能直接与上述网络延迟比较。
|
||||
|
||||
后续通过 30 分钟命令完成发布前持续压测,并在目标反向代理和部署环境继续观察内核 socket 缓冲、FD 上限与真实业务流量;再通过 profile 决定是否优化对象分配或事件状态机。
|
||||
189
README.md
189
README.md
@@ -1,33 +1,84 @@
|
||||
# GoOmni
|
||||
|
||||
GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认用 Gin 作为网络适配器,业务代码只依赖 GoOmni 的 transport-neutral API,后续可以替换为自研网络层。
|
||||
GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用只依赖标准库的网络适配器,业务代码只依赖 GoOmni 的 transport-neutral API,后续可以替换为自研网络层。
|
||||
|
||||
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.1.0
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.8.0-preview
|
||||
```
|
||||
|
||||
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
||||
|
||||
创建新业务项目:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.8.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
远程安装的 CLI 默认引用自身版本,生成的项目不需要本地 `replace`。
|
||||
具体版本和本地开发选项见 [CLI 文档](cmd/go-omni/README.md)。
|
||||
|
||||
需要数据库持久化时,使用 `go-omni new --database mysql --module example.com/billing ./billing`。
|
||||
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
|
||||
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
|
||||
|
||||
## API 定义与项目生成
|
||||
|
||||
当前源码新增 **API 定义驱动项目生成**:写 `.api` 定义请求、响应和路由,自动生成项目目录、DTO、Handler、Service 骨架及 OpenAPI。重复生成保留业务代码,支持只读 CI 检查,完整语法和示例见 [API 开发指南](API.md)。
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni api new --framework-dir . \
|
||||
--module example.com/omniDemo ../omniDemo
|
||||
```
|
||||
|
||||
API 生成支持 `--database mysql --storage local --log-output file` 组合初始化。上面的命令使用本地源码;远程安装见 [发布说明](RELEASE.md),完整业务案例见 [userService](examples/userService/README.md)。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```text
|
||||
go-omni/
|
||||
├── app/ # 应用启动和生命周期
|
||||
├── httpx/ # 与网络框架无关的 HTTP 抽象
|
||||
├── response/ # 统一响应
|
||||
├── response/ # 可配置公共体和响应编码器
|
||||
├── errors/ # 统一错误
|
||||
├── health/ # 存活和就绪检查
|
||||
├── middleware/ # 请求 ID 等基础中间件
|
||||
├── logx/ # 业务层结构化日志
|
||||
├── metrics/ # 请求指标与状态统计
|
||||
├── config/ # 配置管理
|
||||
├── validate/ # 参数校验
|
||||
├── contract/ # API 字段契约、严格多源绑定与文档约束
|
||||
├── auth/ # 认证和权限接口
|
||||
├── database/ # database/sql 生命周期和事务
|
||||
├── transport/gin/ # Gin 网络适配器,可替换
|
||||
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
||||
├── httpc/ # 出站 HTTP 连接池、超时和重试
|
||||
├── task/ # 有界后台任务与间隔调度
|
||||
├── queue/ # 持久队列接口与 Redis Streams 适配
|
||||
├── storage/ # Local、S3/R2、OSS 与临时下载链接
|
||||
├── openapi/ # 路由和 OpenAPI 3.0.3 契约
|
||||
├── telemetry/ # 可注入 OpenTelemetry 观测
|
||||
├── resilience/ # 熔断与上游并发隔离
|
||||
├── audit/ # 显式业务审计与可注入 Sink
|
||||
├── pagination/ # 有界分页和统一结果
|
||||
├── transport/nethttp/ # 标准库网络适配器
|
||||
├── transport/native/ # 自研 HTTP/1.1 服务端预览
|
||||
├── transport/http2/ # opt-in HTTP/2 实验适配器
|
||||
├── examples/ # 示例项目
|
||||
└── cmd/ # 后续 CLI
|
||||
└── cmd/ # go-omni 项目脚手架
|
||||
```
|
||||
|
||||
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
||||
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
||||
可以直接运行 [组合示例](examples/modules/README.md)。这些能力从 `v0.7.0-preview` 提供。
|
||||
|
||||
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
|
||||
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
|
||||
|
||||
## 快速开始
|
||||
|
||||
```go
|
||||
@@ -63,17 +114,53 @@ func main() {
|
||||
return omni.NotFound("USER_NOT_FOUND", "user was not found")
|
||||
```
|
||||
|
||||
默认 Gin 适配器位于 `transport/gin`。业务层不需要导入 Gin;如果后续实现自己的网络层,只需要实现 `httpx.Router` / `httpx.RouterGroup`,然后通过 `omni.NewWithRouter` 启动应用:
|
||||
默认网络适配器位于 `transport/nethttp`。业务层不需要导入具体网络库;如果后续实现自己的网络层,只需要实现 `httpx.Router` / `httpx.RouterGroup`,然后通过 `omni.NewWithRouter` 启动应用:
|
||||
|
||||
```go
|
||||
application := omni.NewWithRouter(myRouter)
|
||||
```
|
||||
|
||||
仓库还提供了只依赖标准库的 `transport/nethttp` 参考实现,可以直接验证这条替换路径。
|
||||
仓库还提供了使用标准库路由的 `transport/nethttp` 参考实现。两种适配器已通过同一套路由、业务处理链和取消传播测试,完整用户 CRUD 复用同一份 Handler / Service。
|
||||
|
||||
`app.Engine()` 现在返回标准 `http.Handler`,`ctx.Raw()` 返回适配器提供的 `http.ResponseWriter`,避免业务 API 锁定 Gin 类型。
|
||||
自研适配器可以直接运行 `transport/transporttest.Run` 验收。具体规则、v0.4.0 的斜杠 / Any / 405 行为调整,以及自研网络内核与当前 HTTP 接口的边界,见 [适配层契约](transport/README.md)。
|
||||
|
||||
业务代码可以直接使用 `git.yuntuo.hk/k2safe/go-omni/logx` 打结构化日志,不需要依赖 Gin:
|
||||
`app.Engine()` 返回标准 `http.Handler`,`ctx.Raw()` 返回适配器提供的 `http.ResponseWriter`,业务 API 不锁定具体网络实现。
|
||||
|
||||
需要试用第一版自研网络层时,业务注册代码无需改变,只替换启动器:
|
||||
|
||||
```go
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer cancel()
|
||||
|
||||
app := omni.New()
|
||||
app.GET("/health", func(ctx *omni.Context) error {
|
||||
return ctx.OK(map[string]string{"status": "ok"})
|
||||
})
|
||||
if err := app.RunWith(ctx, native.New(nil), ":8080"); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
```
|
||||
|
||||
`transport/native` 当前支持 HTTP/1.1、Keep-Alive、`Content-Length` 和 chunked 请求体、`http.Flusher` 流式响应、可选 TLS、读写超时和优雅关闭;不支持的传输编码会被拒绝。它会复用连接状态和响应对象,并可通过 `MaxConnections`、`MaxConcurrentRequests` 限制内存峰值。它仍通过标准 HTTP 请求/响应接口连接业务层,适合在现有 Handler / Service 上做网络层验证。
|
||||
|
||||
native TLS 当前只支持并宣告 ALPN `http/1.1`;配置 `h2` 会在启动时快速失败。需要 HTTP/2 时,可由 Nginx/Cloudflare 终止后回源 HTTP/1.1,或使用默认的 `transport/nethttp` 适配器。
|
||||
|
||||
`transport/http2` 是独立的 opt-in 实验适配器,使用 `x/net/http2` 处理帧和流控,同时保留 GoOmni 的连接上限、全局请求并发闸门和生命周期边界。它不会改变 native 默认路径;实验入口、限制和验收结果见 [HTTP/2 进入条件](transport/native/HTTP2.md)。
|
||||
|
||||
自研网络层还提供 `server.Stats()`,可读取 accepted、rejected、active connections 和 active requests,与上面的业务请求 metrics 组合后能区分连接堆积、过载拒绝和 Handler 慢请求。`rejectedConnections` 表示连接闸门拒绝数,`rejectedRequests` 表示并发请求闸门返回 503 的次数。
|
||||
|
||||
业务代码可以直接使用 `git.yuntuo.hk/k2safe/go-omni/logx` 打结构化日志,不需要依赖具体网络库:
|
||||
|
||||
```go
|
||||
func (s *UserService) Create(ctx *httpx.Context) error {
|
||||
@@ -82,7 +169,17 @@ func (s *UserService) Create(ctx *httpx.Context) error {
|
||||
}
|
||||
```
|
||||
|
||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。
|
||||
需要基础运行指标时,可以注册 transport-neutral metrics middleware 和接口:
|
||||
|
||||
```go
|
||||
requestMetrics := omni.NewMetrics()
|
||||
app.Use(requestMetrics.Middleware())
|
||||
app.GET("/metrics", requestMetrics.Handler())
|
||||
```
|
||||
|
||||
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
||||
|
||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。文件轮转、敏感字段脱敏、异常堆栈与慢请求配置见 [logx 文档](logx/README.md);文件直传与云端分片上传见 [storage 文档](storage/README.md)。 创建新业务项目时可通过 `go-omni new --log-output file --storage local` 按需生成配置、装配代码与可运行示例,当前源码使用方式见 [CLI 文档](cmd/go-omni/README.md)。
|
||||
|
||||
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
||||
|
||||
@@ -101,10 +198,42 @@ application.OnShutdown(logger.Close)
|
||||
|
||||
常用变量包括 `GO_OMNI_ENABLE_LOGGER`、`GO_OMNI_LOG_LEVEL`、
|
||||
`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_OUTPUT`、`GO_OMNI_SERVICE`、
|
||||
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR` 和 `GO_OMNI_SHUTDOWN_TIMEOUT`。数据库连接仍由
|
||||
`GO_OMNI_ENVIRONMENT`、`GO_OMNI_HTTP_ADDR`、`GO_OMNI_SHUTDOWN_TIMEOUT` 和
|
||||
`GO_OMNI_TRUSTED_PROXIES`。可信代理变量使用逗号分隔的 IP/CIDR,例如
|
||||
`GO_OMNI_TRUSTED_PROXIES=127.0.0.1,10.0.0.0/8`;只有来自这些地址的连接才会解析
|
||||
`X-Forwarded-For`,默认只记录 socket 对端地址。数据库连接仍由
|
||||
`database.ConfigFromEnv` 从 `GO_OMNI_DB_*` 加载。
|
||||
|
||||
`App` 同时实现标准 `http.Handler`,并提供 `RunContext`:收到 `SIGINT` / `SIGTERM` 后可以先停止接收新请求,再在超时时间内等待活跃请求完成。数据库连接池由 `database.Open` 创建,在服务退出时调用 `Close`。
|
||||
网络层也由同一份配置选择,Handler / Service / Repository 不需要导入具体适配器:
|
||||
|
||||
| 配置 | 默认值 / 用途 |
|
||||
| --- | --- |
|
||||
| `GO_OMNI_TRANSPORT` | `nethttp`;可选 `native`、`http2` |
|
||||
| `GO_OMNI_TLS_CERT_FILE` | TLS 证书 PEM 文件路径 |
|
||||
| `GO_OMNI_TLS_KEY_FILE` | TLS 私钥 PEM 文件路径,必须与证书一起配置 |
|
||||
| `GO_OMNI_SHUTDOWN_TIMEOUT` | `10s`,三种适配器共用的请求排空时间 |
|
||||
|
||||
例如运行自研 HTTP/1.1,或显式试用独立 HTTP/2 适配器:
|
||||
|
||||
```sh
|
||||
GO_OMNI_TRANSPORT=native go run ./examples/reference
|
||||
GO_OMNI_TRANSPORT=http2 GO_OMNI_HTTP_ADDR=127.0.0.1:8443 \
|
||||
GO_OMNI_TLS_CERT_FILE=./server.crt GO_OMNI_TLS_KEY_FILE=./server.key \
|
||||
go run ./examples/reference
|
||||
```
|
||||
|
||||
`http2` 必须配置 TLS;native TLS 只协商 `http/1.1`,nethttp TLS 支持标准库的 HTTP/1.1 和 HTTP/2。
|
||||
未知适配器、缺少配对证书、无法加载证书时,`Run` / `RunContext` 在监听前返回错误并执行资源清理,不会退回其他协议。
|
||||
HTTP/2 仍为实验适配器,尚未合并到 native 多协议监听器。
|
||||
|
||||
代码配置可写成 `omni.New(omni.WithTransport(omni.TransportNative), omni.WithTLS(certFile, keyFile))`。
|
||||
`ServeContext(ctx, listener)` 使用相同配置并接管已有 listener;失败或退出时也会关闭它。
|
||||
`RunWith(ctx, runner)` 保留自定义网络层入口,由 runner 自己管理 TLS、超时和网络参数;连接/请求上限等适配器专属选项仍通过这个入口设置。
|
||||
`NewServer` 仅创建标准库明文服务器,配置选择应用于 `Run`、`RunContext` 和 `ServeContext`。
|
||||
三种适配器的配置、真实 TLS 业务和关闭行为已通过 Linux 全量 race 及 10 次重复回归,见 [统一启动验收](performance/transportStartup.md)。
|
||||
|
||||
`App` 同时实现标准 `http.Handler`。`Run` 监听 `SIGINT` / `SIGTERM`;`RunContext` 则由调用方的 context 控制退出,生成项目已使用 `signal.NotifyContext` 接好信号。
|
||||
退出时先停止接收新请求,等待活跃请求完成,再执行资源清理。超过 `ShutdownTimeout` 时强制关闭连接、取消请求 context,并返回超时错误。业务代码应响应 context 取消;框架无法强制结束忽略取消的业务 goroutine。数据库连接池由 `database.Open` 创建,在服务退出时调用 `Close`。
|
||||
|
||||
应用资源可以直接挂到 `omni.New` 返回的生命周期上:
|
||||
|
||||
@@ -134,7 +263,11 @@ application.GET("/health/ready", checker.ReadinessHandler())
|
||||
新项目脚手架会生成项目自己的 `internal/servicecontext`。它集中保存 Logger、数据库、缓存和外部客户端,Service 只依赖这个项目上下文,不把基础设施初始化散落到各个 Handler:
|
||||
|
||||
```go
|
||||
services := servicecontext.New(logger, users)
|
||||
services, err := servicecontext.New(ctx, logger)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer services.Close()
|
||||
userService := service.NewUsers(services)
|
||||
```
|
||||
|
||||
@@ -142,6 +275,28 @@ userService := service.NewUsers(services)
|
||||
|
||||
[`examples/reference`](examples/reference/README.md) 提供按 Handler / Service / Repository / Model 分层的用户 CRUD 示例。默认使用并发安全的内存存储,支持分页、邮箱唯一校验、结构化请求日志和统一错误,无需配置数据库。
|
||||
|
||||
从远程版本生成并升级一个独立业务模块,可运行:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyBusiness.py --version v0.6.0-preview
|
||||
```
|
||||
|
||||
该验收会在临时目录中先固定 `v0.2.0`,再升级到 `v0.6.0-preview`,检查无本地 replace、配置加载、结构化日志、requestId、CRUD、校验错误、业务错误和优雅退出。
|
||||
|
||||
标准库与 native 的并发、延迟和长连接内存对照见 [PERFORMANCE.md](PERFORMANCE.md)。`python3 scripts/verifyPerformance.py` 运行检查与 benchmark;增加 `--load --output /tmp/goOmniPerf.json` 可测量独立服务进程并保存报告。
|
||||
|
||||
HTTP/2 仍是显式 opt-in 实验适配器,不会改变 native 的 HTTP/1.1 默认路径。Linux TLS 长测使用 `python3 scripts/verifyHTTP2.py --duration 30m`,默认使用普通 Go 运行模式;`--race` 默认改为 60 秒,用于并发检查,其额外内存开销不能当成生产运行基线。
|
||||
|
||||
多连接混合场景使用 `python3 scripts/verifyHTTP2.py --scenario mixed --duration 30m`:8 条独立 TLS 连接,每条连接 14 个普通请求 worker、1 个慢上传 worker、1 个慢下载 worker,共 128 个 worker。上传为 256 KiB、每 4 KiB 等待 2 ms;下载为 8 MiB、每次最多读取 64 KiB 后等待 20 ms。日志分别记录三类成功数、传输字节数、每条连接的覆盖情况,以及停止并释放测试数据后两次 GC 的采样。
|
||||
|
||||
该脚本默认把测试容器限制为 2 核、2 GiB 内存(禁用 swap)、256 个进程/线程,整个命令最多运行 40 分钟,中断和超时会清理本次容器。这些限制可用 `--cpus`、`--memory-mb`、`--pids-limit`、`--wall-timeout` 调整;共享服务器应先用 `--duration 60s` 短测,再逐步延长。每 15 秒输出的堆、RSS 和 FD 统计包含同一进程内的客户端与服务端。
|
||||
|
||||
Linux amd64 已完成 30 分钟、单 TLS 连接、128 个 worker 的普通模式长测:58,102,476 次请求,0 失败、0 协议错误,采样 RSS 峰值约 22.6 MiB,连接/活动请求归零且 FD 回到基线。完整采样和原始日志见 [HTTP/2 长测报告](performance/http2SoakLinux.json) 与 [原始日志](performance/http2SoakLinux.txt),此前的 race 对照见 [短测记录](performance/http2SmokeLinux.json)。这项结果覆盖单连接复用。
|
||||
|
||||
多连接慢客户端场景也已完成 Linux amd64 的 30 分钟长测:8 条 TLS 连接、128 个 worker,53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,共 53,191,687 次,0 失败与协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端、服务端和下载缓冲);退出后连接/请求归零,goroutine 回到 2、FD 回到 6;释放测试数据并两次 GC 后存活堆约 450 KiB,RSS 不要求立即归还操作系统。完整 120 份采样见 [混合长测报告](performance/http2MixedSoakLinux.json) 和 [原始日志](performance/http2MixedSoakLinux.txt)。连接级窗口耗尽、WINDOW_UPDATE 恢复及零窗口写阻塞的关闭回收也已有独立测试;Nginx 的 HTTP/2 回源验收结果见下方,适配器保持实验状态。
|
||||
|
||||
`python3 scripts/verifyHTTP2Proxy.py --count 3` 验证真实 Nginx 双段 TLS/h2、回源连接复用、证书拒绝、超时/取消恢复与新旧实例的 GOAWAY 排空;增加 `--race` 可运行并发检查。Linux amd64、Nginx 1.30.5 的普通与 race 模式各 3 轮均已通过。编译和测试容器顺序运行,均限制为 2 核、2 GiB;测试容器禁用外部网络,全部请求在容器内完成。使用方法、报告及当前边界见 [HTTP/2 代理回源验收](transport/http2/PROXY.md)。
|
||||
|
||||
在框架根目录运行:
|
||||
|
||||
```sh
|
||||
@@ -150,8 +305,12 @@ go run ./examples/reference
|
||||
|
||||
默认地址为 `http://127.0.0.1:8080`。示例文档提供完整的 curl 操作流程。数据只保存在内存中,重启后清空。
|
||||
|
||||
框架的 `App` 也实现了标准 `http.Handler`,可以直接用于 `http.Server` 和 `httptest.NewServer`;业务路由可通过 `*omni.RouterGroup` 注册,无需导入 Gin。
|
||||
框架的 `App` 也实现了标准 `http.Handler`,可以直接用于 `http.Server` 和 `httptest.NewServer`;业务路由可通过 `*omni.RouterGroup` 注册,无需导入网络库。
|
||||
|
||||
## 当前边界
|
||||
|
||||
缓存、消息队列、定时任务和 CLI 暂时作为后续可选模块。认证层只定义 `Authenticator`、`Principal` 和路由保护中间件,具体 JWT、Session 或 OAuth 由项目实现。核心数据库层只依赖 `database/sql`,具体驱动、迁移和表结构由项目选择。
|
||||
缓存、Redis 消息队列和进程内定时任务已作为可选模块提供。认证层提供 `Authenticator`、JWT/Session 适配器和 Policy 接口,登录流程与业务授权规则由项目实现。核心数据库层只依赖 `database/sql`,具体驱动、迁移和表结构由项目选择。
|
||||
|
||||
当前成熟度与后续验收标准见 [ROADMAP.md](ROADMAP.md),版本变更见 [CHANGELOG.md](CHANGELOG.md)。
|
||||
|
||||
`v0.8.0-preview` 新增 [公共响应模板](RESPONSE.md):在 `.api` 中定义一次返回公共体,分组与接口引用,运行时和 OpenAPI 共用。支持数字错误码、嵌套结构、原始 JSON,以及独立的文本/XML回调编码器。源码开发可通过 `--framework-dir` 显式引用本地框架。
|
||||
|
||||
55
RELEASE.md
Normal file
55
RELEASE.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# v0.8.0-preview
|
||||
|
||||
这一版让业务自行定义公共返回体:在 `.api` 中配置字段、结构、成功/失败提示和错误码类型,按分组复用或由单接口覆盖,运行时与 OpenAPI 共用定义。需要特殊协议的回调可独立控制状态、Header 和内容。
|
||||
|
||||
本版继续作为预览版供固定版本接入;HTTP/2 仍是独立实验适配器。
|
||||
|
||||
## 安装与升级
|
||||
|
||||
```sh
|
||||
GOPRIVATE=git.yuntuo.hk go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.8.0-preview
|
||||
|
||||
# 已有业务项目中执行:
|
||||
GOPRIVATE=git.yuntuo.hk go get git.yuntuo.hk/k2safe/go-omni@v0.8.0-preview
|
||||
go-omni api generate
|
||||
go test ./...
|
||||
```
|
||||
|
||||
新项目可以直接 `go-omni api new --module example.com/demo ./demo`,或用 `go-omni new --api service.api --module example.com/demo ./demo`。远程安装的 CLI 会固定生成项目的框架版本,无需本地 replace。需要仓库权限时使用自己的 Git 凭据管理器,不把凭据写入 module path 或 go.mod。
|
||||
|
||||
## 本版功能
|
||||
|
||||
- `response Name (...)` 定义公共 JSON 体,支持嵌套结构、独立 errorBody、常量及类型保持的占位符;通过 import 共享配置。
|
||||
- `@server (response: Name)` 为整个 service 块选择模板,`@response Name` 单接口覆盖;`@response raw` 成功返回业务 JSON,错误保留组级格式。
|
||||
- `codeType: int32 / int64 / string` 使用与业务字段一致的名称;默认码和映射都校验,int64 配置及生成代码保留整数精度。旧 `integer` 写法仍兼容。
|
||||
- `successMessage` / `errorMessage` 配置默认提示,`codes` / `messages` 按逻辑错误码覆盖。业务只返回数据或错误,公共体由框架组装。
|
||||
- `omni.WithResponse` 设置手写路由的应用默认格式,`WithResponseEncoder` 与 `HandleWithResponse` 支持特殊协议;Text / XML / Bytes / JSON / NoContent 可直接输出。
|
||||
- 生成路由在全局中间件之前选定策略,提前鉴权拒绝、限流、panic 和 Context 克隆沿用正确格式。OpenAPI 同步模板结构与业务错误示例。
|
||||
|
||||
详见 [公共响应体](RESPONSE.md)、[完整定义示例](examples/contracts/responses.api) 和 [更新记录](CHANGELOG.md)。
|
||||
|
||||
## 升级行为
|
||||
|
||||
1. CLI 与框架依赖一起升级到本版。新生成路由使用新的响应策略入口,旧运行时不能编译。旧 `.api` 可继续使用;生成器不会覆盖已有 Service、Repository、middleware、配置和 main。
|
||||
2. 生成接口以 `.api` 为响应契约来源,即使没有声明模板也固定使用默认格式;应用 YAML 的 response 用于手写路由和应用兜底。需要变更生成接口的格式,请修改 `.api` 后重新生成。
|
||||
3. `response.Response` DTO 保留为兼容类型,运行时不再依赖它固定结构。未配置的正常业务响应继续使用默认 code/message/data/requestId。
|
||||
4. 内置 App 的 404/405 现在经过应用响应策略和中间件,默认输出 JSON;原先依赖纯文本兜底的客户端需调整。直接使用 nethttp.Router 的兜底行为不变。
|
||||
5. `int32` / `int64` 按有符号整数范围校验并输出 JSON 数值;消费端需支持相应精度。需要文本码时使用 string。旧 integer 继续保留原有 ±(2^53−1) 范围。
|
||||
6. `.api` 的模板和 raw 当前描述 JSON。文本/XML回调用 Go 编码器并维护对应文档;TLS/HTTP解析错误、连接级过载等发生在进入应用前的错误仍按网络协议处理。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=1 -count=1 -timeout=300s ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyAPIWorkflow.py --report performance/responsePoliciesWorkflowLocal.json
|
||||
python3 scripts/verifyResponseRelease.py --version v0.8.0-preview \
|
||||
--report performance/responsePoliciesRemote.json
|
||||
python3 scripts/verifyAPIWorkflow.py --version v0.8.0-preview \
|
||||
--report performance/responsePoliciesWorkflowRemote.json
|
||||
```
|
||||
|
||||
远程验收分别从全新 module cache 安装 CLI,验证生成项目固定依赖且没有本地 replace,以及完整用户示例的真实 MySQL、nethttp/native、事务、日志和生命周期。数据库容器只使用本机临时资源(1 CPU / 768 MiB),结束清理,不连接业务服务器。
|
||||
|
||||
验收记录见 [响应模板验收](performance/responsePoliciesAcceptance.md)。OSS/R2 云账号实网验收与长时间压力测试不属于本次发布范围。
|
||||
140
RESPONSE.md
Normal file
140
RESPONSE.md
Normal file
@@ -0,0 +1,140 @@
|
||||
# 公共响应体与独立接口协议
|
||||
|
||||
这些功能从 `v0.8.0-preview` 提供。CLI 与项目框架依赖请固定到同一版本;源码开发可显式使用 `--framework-dir`。`v0.7.0-preview.1` 不包含响应模板声明。
|
||||
|
||||
go-omni 的响应编码由应用或接口配置。没有配置时保留旧版默认值;业务可以定义自己的字段、嵌套结构、错误码类型、提示和空值行为。`response.Response` 仅保留为兼容 DTO,运行时与生成器不再通过它固定响应结构。
|
||||
|
||||
## 在 .api 中定义一次、到处引用
|
||||
|
||||
```go
|
||||
response CommonResponse (
|
||||
body: {
|
||||
"code": "${code}",
|
||||
"msg": "${message}",
|
||||
"data": "${data}"
|
||||
}
|
||||
codeType: int32
|
||||
successCode: 200
|
||||
errorCode: 9999
|
||||
successMessage: "操作成功"
|
||||
dataMode: always
|
||||
codes: {"USER_NOT_FOUND":404001}
|
||||
messages: {"USER_NOT_FOUND":"用户不存在"}
|
||||
)
|
||||
|
||||
@server (
|
||||
group: user
|
||||
response: CommonResponse
|
||||
)
|
||||
service user-api {
|
||||
@handler getUser
|
||||
get /users/:userId (GetUserReq) returns (UserResp)
|
||||
|
||||
@response raw
|
||||
@handler rawUser
|
||||
get /rawUsers/:userId (GetUserReq) returns (UserResp)
|
||||
}
|
||||
```
|
||||
|
||||
请求/业务返回类型照常在 `type` 中定义。上例 `UserResp` 是业务数据,由模板中的 `${data}` 放入公共体。请求失败时,同一模板输出映射后的业务错误。错误本身的 HTTP 状态保持不变,业务 `code` 与 HTTP 状态相互独立。
|
||||
|
||||
- 公共声明可以放进 `common.api`,其他契约通过 `import "common.api"` 复用。
|
||||
- `@server (response: CommonResponse)` 指定整个 service 块的公共体;不同块可以引用同一个模板。
|
||||
- `@response PartnerResponse` 为单接口替换模板,成功和错误一起生效。
|
||||
- `@response raw` 让成功 JSON 直接返回业务对象,错误沿用组级模板;组本身为 raw 时,错误使用默认模板。
|
||||
- 模板名使用 UpperCamelCase;未知模板、未知配置、错误的占位符和冲突设置在生成前报错。
|
||||
|
||||
**生成接口以 `.api` 为响应契约来源。** 生成器把选定模板编译进 `internal/handler/responseGen.go`,在全局鉴权、限流等应用中间件之前选定,然后再执行路由中间件和业务。OpenAPI 与运行时共用配置和映射规则。即使 `.api` 没有声明模板,生成接口也明确使用默认契约,不会被应用的 YAML 设置静默改变。
|
||||
|
||||
业务只需返回业务数据或业务错误,无需每个 Handler 手工组装公共体。生成器维护模板、路由及文档文件,重复生成不覆盖手写 Service、Repository、middleware 和配置。
|
||||
|
||||
## 自定义结构与占位符
|
||||
|
||||
公共体没有规定必须出现的字段,也可以把成功和失败分开:
|
||||
|
||||
```go
|
||||
response PartnerResponse (
|
||||
body: {
|
||||
"accepted": "${ok}",
|
||||
"payload": "${data}",
|
||||
"meta": {"trace":"${requestId}","httpStatus":"${status}"},
|
||||
"version": 1
|
||||
}
|
||||
errorBody: {
|
||||
"accepted": "${ok}",
|
||||
"error": {"reason":"${message}","id":"${code}"}
|
||||
}
|
||||
)
|
||||
```
|
||||
|
||||
`body` 是 JSON 对象,可嵌套对象、数组和常量;`errorBody` 可选,需要同时定义 `body`。空对象 `{}` 也是有效模板。
|
||||
|
||||
| 整值占位符 | 输出值 |
|
||||
| --- | --- |
|
||||
| `${data}` | 当前接口的业务返回值,保留对象/数组/数值等原类型 |
|
||||
| `${code}` | 映射后的 string / int32 / int64 业务错误码 |
|
||||
| `${message}` | 公开提示,经 message 配置映射 |
|
||||
| `${requestId}` | 请求标识;为空时省略所在对象字段 |
|
||||
| `${ok}` | 成功为 true;有错误或 HTTP 状态 ≥400 时为 false |
|
||||
| `${status}` | HTTP 状态整数 |
|
||||
|
||||
占位符只替换整个值,不执行表达式或字符串拼接。字面量 `${data}` 写成 `$${data}`。数据为 nil 时,`dataMode: omitNil`(默认)省略所在对象字段;`always` 输出 null。数组中的占位符保留位置。模板最多嵌套 24 层。
|
||||
|
||||
`codeType` 使用与业务字段一致的类型名,默认 `string`:
|
||||
|
||||
| 类型 | 含义 | OpenAPI |
|
||||
| --- | --- | --- |
|
||||
| `string` | 字符串业务码,例如 `"OK"`、`"USER_NOT_FOUND"` | string |
|
||||
| `int32` | 32 位有符号整数,−2147483648 到 2147483647 | integer / int32 |
|
||||
| `int64` | 64 位有符号整数,−9223372036854775808 到 9223372036854775807 | integer / int64 |
|
||||
|
||||
整数类型必须提供 `successCode` 和 `errorCode`,支持 0 和负数;默认码和 `codes` 映射都会校验范围,越界或小数会报错。`int64` 的配置读取和生成代码保留完整整数精度,并按 JSON 数值输出;消费端也需要支持对应的整数精度。需要文本形式时可用 `string` 并配置字符串码。
|
||||
|
||||
旧写法 `integer` 保留兼容,沿用 ±(2^53−1) 的数值范围;新配置优先写 `int32` 或 `int64`。
|
||||
|
||||
`codes` 按逻辑错误码覆盖默认码;`messages` 优先于 `successMessage` / `errorMessage`。未配置的公开提示沿用原错误消息;内部 cause 不会被默认编码器序列化。
|
||||
|
||||
简单的平铺响应也可使用 `codeField` / `messageField` / `dataField` / `requestIdField` 快捷配置,`"-"` 禁用字段。空名称沿用旧默认值。这些字段名快捷配置不能和 `body` 混用,以免两个定义冲突。
|
||||
|
||||
## 手写路由、配置文件与三方回调
|
||||
|
||||
手写路由的应用默认配置可以放在 JSON/YAML 的 `response` 字段,也可用 Go 选项:
|
||||
|
||||
```go
|
||||
app := omni.New(omni.WithResponse(response.Config{
|
||||
Body: map[string]any{"code":"${code}", "msg":"${message}", "data":"${data}"},
|
||||
CodeType: "int32", SuccessCode: 200, ErrorCode: 9999, DataMode: "always",
|
||||
}))
|
||||
```
|
||||
|
||||
应用之间的配置独立,模板在创建时验证并复制;没有全局可变的响应格式。应用配置快照也会复制模板和映射。响应配置属于静态配置,变更后需重启。
|
||||
|
||||
单次显式输出可使用 `c.JSON(status, value)`、`c.Text(status, text)`、`c.XML(status, value)`、`c.Bytes(status, contentType, bytes)` 或 `c.NoContent()`,都绕过公共体。已经提交的响应不会再追加错误 JSON。
|
||||
|
||||
三方回调可能要求成功与失败都返回 HTTP 200,并分别输出 `success` / `fail`。通过接口编码器统一处理成功、错误和 panic:
|
||||
|
||||
```go
|
||||
ack := response.EncoderFunc(func(r response.Result) (response.Payload, error) {
|
||||
if r.Err != nil || r.Status >= 400 {
|
||||
return response.Text(200, "fail"), nil
|
||||
}
|
||||
return response.Text(200, "success"), nil
|
||||
})
|
||||
app.HandleWithResponse("POST", "/hooks/payment", ack, paymentHandler)
|
||||
```
|
||||
|
||||
编码器也可以设置 HTTP 状态、Header、XML 或任意字节。`omni.WithResponseEncoder` 设置应用默认编码器;`httpx.WithResponse` 是普通路由中间件,按中间件顺序生效。需要覆盖全局中间件提前拒绝时,应使用 `HandleWithResponse`。自定义 Registrar 可以实现 `httpx.ResponseRegistrar` 提供相同的早期挂载能力;未实现时辅助函数退化为普通路由中间件。
|
||||
|
||||
`.api` 当前声明的是 JSON 协议,`@response raw` 也是原始 JSON,不代表任意文本/XML协议。自定义文本/XML回调使用上述 Go 入口,并由应用维护相应文档;生成器不会猜测自定义 Go 编码器的输出。如果在命名中间件里替换编码器,业务也必须同步维护文档。
|
||||
|
||||
框架内置路由的 404/405 使用应用默认格式;它们不属于某个 `.api` 分组。TLS 握手、畸形 HTTP、连接级过载等发生在进入应用前,仍按网络层协议处理。自定义编码器失败、panic 或返回非法状态/204 body 时,最终返回空 500,避免递归编码或泄露另一种响应体。
|
||||
|
||||
## 运行示例与验证
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni api validate --api examples/contracts/responses.api
|
||||
go run ./cmd/go-omni api generate --api examples/contracts/responses.api \
|
||||
--module example.com/responseDemo --framework-dir . --dir ../responseDemo
|
||||
```
|
||||
|
||||
示例业务方法起初为 501,完成 Service 后就会套用声明的模板。参考 [完整 .api](examples/contracts/responses.api)。验收测试会在临时目录生成真实项目,写入业务方法,执行已生成路由,检查数字错误码、原始 JSON、鉴权错误、路由覆盖与重复生成保护。
|
||||
105
ROADMAP.md
Normal file
105
ROADMAP.md
Normal file
@@ -0,0 +1,105 @@
|
||||
# GoOmni 成熟度计划
|
||||
|
||||
## 1. 新项目启动与发布流程(已完成,v0.1.1)
|
||||
|
||||
- 修正脚手架固定版本引用,保留显式本地开发选项。
|
||||
- 生成服务处理退出信号,等待请求完成后清理资源。
|
||||
- 验收:源码目录外安装远程 CLI → 生成无 replace 的项目 → 编译 → 请求健康接口和业务接口 → SIGINT/SIGTERM 正常退出。
|
||||
- 自动验收入口:`python3 scripts/verifyScaffold.py --version v0.1.1`。
|
||||
- 已通过全量 race 测试、vet、build,以及从远程提交安装 CLI 后的无 replace 端到端验收;覆盖生成服务和 `App.Run` 的两种退出信号。
|
||||
|
||||
## 2. 业务数据接入(已完成,v0.2.0)
|
||||
|
||||
- 数据库初始化、Repository 和 ServiceContext 组合成完整业务启动流程。
|
||||
- Readiness 注册数据库探针,启动失败与服务退出时清理连接池。
|
||||
- 验收:真实数据库 CRUD、事务回滚、数据库不可用时 readiness 返回 503。
|
||||
- `go-omni new --database mysql` 已生成完整数据接入流程,默认内存模式仍然可用。
|
||||
- 独立 MySQL 8.4 容器已验证 CRUD、唯一邮箱、分页、服务重启持久化、事务提交与错误/panic/取消回滚。
|
||||
- 已验证初始化失败、缺少表结构、正常退出时关闭连接池,以及数据库停止后的 readiness 503、恢复后的 200。
|
||||
- 自动验收入口:`python3 scripts/verifyDatabase.py --version v0.2.0`;结束后清理本次测试容器、匿名卷和临时项目。
|
||||
|
||||
## 3. 网络适配一致性(已完成,v0.4.0)
|
||||
|
||||
- 标准库适配器运行公共契约测试,并为自研网络层保留相同的接入边界。
|
||||
- 覆盖路由参数、分组、中间件、错误响应和取消传播。
|
||||
- 验收:切换适配器后,业务 Handler 和 Service 无需修改,约定的 HTTP 行为一致。
|
||||
- 新增公共 `transport/transporttest.Run`,自研适配器可直接复用;覆盖静态 / 参数 / 通配路由、分组、405 Allow、鉴权短路、错误与 panic、响应提交、requestId、并发隔离、截止时间及真实客户端取消。
|
||||
- 标准库路由实现明确的路径、方法、尾部斜杠、通配参数、Any 和 404/405 约定;不再引入 Gin 依赖。
|
||||
- 完整用户 CRUD 已用标准库适配器验证;自研适配器可直接复用同一公共契约测试包。
|
||||
- 自动验收入口:`python3 scripts/verifyTransports.py --version v0.4.0`,从源码目录外消费远程版本与公共契约测试包。
|
||||
|
||||
## 4. 实际业务试用(已完成,v0.4.0)
|
||||
|
||||
- 在独立业务目录生成 billing service,先固定远程 `v0.2.0`,再升级到 `v0.4.0`,全程无本地 replace。
|
||||
- 验证环境配置、结构化访问日志、requestId 关联、参数校验、业务错误映射、CRUD、优雅退出和升级后的 race 测试。
|
||||
- 业务试用没有要求修改 Handler / Service;现有公共 API 足以支撑一套分层业务模块。
|
||||
- 自动验收入口:`python3 scripts/verifyBusiness.py --version v0.4.0`。
|
||||
|
||||
## 5. 自研网络层、稳定性与性能基线(v0.6.0-preview 验收完成)
|
||||
|
||||
- 已加入标准库路由和完整 GoOmni 请求链 benchmark,输出分配次数和耗时,作为未来自研网络层的对照基线。
|
||||
- 新增 `transport/native` 第一版自研 HTTP/1.1 服务端:自行管理 TCP listener、Keep-Alive、Content-Length / chunked 请求解析、响应写回、读写超时和优雅关闭;通过 `App.RunWith` 接入,业务 Handler / Service 保持不变。
|
||||
- 标准库与 native 共用 TCP benchmark,固定 1 / 32 / 128 并发;相同应用、客户端、超时和响应检查,避免两个测试入口逐渐失去可比性。
|
||||
- 新增独立服务进程压测 `internal/perfcheck`,记录成功吞吐、P50/P95/P99、服务端分配、采样堆峰值,以及 128 / 1,024 条空闲长连接的堆内存、goroutine 和断开后的回收情况。原始报告保存在 `performance/tcpBaseline.json`。
|
||||
- native 已支持受 `MaxBodyBytes` 约束的 `Content-Length` / chunked 请求体,仍使用标准 `http.Request` / `http.ResponseWriter` 作为桥接;解析器、连接抽象和生产压测将在后续阶段继续下沉。
|
||||
- native 请求体已改为流式读取,Handler 可边到边处理,服务端只在 Handler 返回后排空剩余 body,超时、畸形和超限 body 会取消请求并关闭连接。
|
||||
- native 响应已支持标准 `http.Flusher`;HTTP/1.1 流式响应使用 chunked framing,普通响应继续走 `Content-Length`,HTTP/1.0 回退到关闭连接分帧。
|
||||
- native 长请求已补客户端断开及流式写失败取消,监测退出后再复用 Reader;已验收普通/流式断开、慢 Handler 后连续 Keep-Alive 和管线数据不被误消费。短请求延迟启动监测,已有管线数据时取消依赖后续写失败或 Handler 返回。
|
||||
- 已补充慢请求头、慢 Content-Length body、慢 chunked body、慢 TLS 握手、大请求体、HTTP/1.0 Keep-Alive、HEAD/204 和管线请求验收;Linux/arm64 容器已通过全量 race、vet、build,以及 3 秒 TLS 并发/空闲连接矩阵。原始报告见 [tlsBaselineLinux.json](performance/tlsBaselineLinux.json)。
|
||||
- 已验证 `MaxConnections` 满载时快速拒绝新连接、请求完成后释放容量;后续仍需在目标部署环境观察文件描述符、内核 socket 缓冲和反向代理联动。
|
||||
- 已验证 `MaxConcurrentRequests` 满载返回 503,活动请求结束后槽位释放,后续请求可以恢复处理。
|
||||
- native `Stats()` 已记录连接和请求过载拒绝数;访问日志已支持可信代理网段配置,默认拒绝伪造的 `X-Forwarded-For`。
|
||||
- native TLS 已明确只宣告 HTTP/1.1 ALPN;HTTP/2 暂由边缘代理终止或使用 `transport/nethttp`,已整理独立协议设计、流控和压测门槛,见 [native HTTP/2 进入条件](transport/native/HTTP2.md),完成后再接入 native。
|
||||
- 已加入独立 `transport/http2` 实验 Runner,验证 h2c/TLS h2、连接上限、全局请求并发拒绝和恢复,以及 GOAWAY 排空、RST_STREAM 单流取消和服务关闭超时;它使用 `x/net/http2` 作为临时帧/流控实现,已完成下述 Nginx HTTP/2 回源联调,尚未接入 native 默认路径。
|
||||
- HTTP/2 实验已补齐头部/请求体上限、128 路同连接多路复用、慢读取流控、协议错误、RST_STREAM 与 GOAWAY 统计,并提供 `scripts/verifyHTTP2.py --duration 30m` 的 Linux TLS 长测入口;长测默认不启用 race,短时并发检查使用 `--race`(默认 60 秒)。测试容器默认限制为 2 核、2 GiB 内存、256 个进程/线程,并设总时限和退出清理;受限长测和 Nginx 回源联调结果见下方。
|
||||
- 新 Linux amd64 环境在上述硬上限内完成 HTTP/2 普通模式 2 分钟与 race 模式 60 秒短测,分别为 3,885,273 和 377,788 次请求,均为 0 失败;连接/请求归零、FD 回到基线。SIGTERM、SIGHUP 和总时限退出清理已实测。报告见 [HTTP/2 短测记录](performance/http2SmokeLinux.json);此前无资源上限的 race 长测因 RSS 增长而停止,不作为通过依据。
|
||||
- 已完成受限的 HTTP/2 单连接 TLS 30 分钟长测:128 个 worker、58,102,476 次请求、0 失败及协议错误;包含客户端与服务端的采样 RSS 峰值约 22.6 MiB,连接/活动请求归零,FD 由 9 回到 6。120 份定时采样见 [HTTP/2 长测报告](performance/http2SoakLinux.json),测试提交和原始日志校验值保存在报告中。Nginx HTTP/2 回源的 GOAWAY/排空结果见下方。
|
||||
- 已完成 8 条 TLS 连接、128 个 worker 的 30 分钟混合长测:53,079,128 次普通请求、106,971 次慢上传、5,588 次慢下载,0 失败及协议错误。采样 RSS 峰值约 116.6 MiB(包含客户端和服务端),前段与最后 5 分钟 RSS 中位数分别约 111.3/110.4 MiB;退出后活动连接/请求归零,goroutine 2、FD 6 均回到基线,两次 GC 后存活堆约 450 KiB。原始日志、120 份采样、测试提交和校验值见 [混合长测报告](performance/http2MixedSoakLinux.json)。
|
||||
- 已补连接级窗口耗尽时的暂停、同连接 204/跨连接隔离、WINDOW_UPDATE 恢复和连接复用;已补零窗口写阻塞在关闭超时后的解除阻塞、请求取消与回收。Linux amd64 的 HTTP/2 全包 race、vet、build 已通过,见 [边界回归报告](performance/http2FlowControlLinux.json)。
|
||||
- 新增 HTTP/2 零 stream 窗口时的慢响应隔离、WINDOW_UPDATE 恢复、RST_STREAM 取消与请求槽位回收测试;补充 TLS 半开握手占满连接上限时的及时拒绝、超时回收及正常请求恢复测试。
|
||||
- 已完成 Nginx 1.30.5 → HTTP/2 的 TLS 回源联调:普通与 race 模式各 3 轮,共 18 个子场景通过;覆盖两段证书校验、32 次 POST 复用一条回源连接、不可信 CA/主机名拒绝、504 超时、客户端取消、GOAWAY 后旧 POST 完成及 20 个新 POST 转入新实例,无重复业务执行。见 [验收说明与报告](transport/http2/PROXY.md)。
|
||||
- `scripts/verifyHTTP2Proxy.py` 顺序运行受限编译容器和禁用外部网络的测试容器;总时限、SIGTERM、SIGHUP 清理已在 Linux 实测,原有 7 个业务容器仍运行。当前结果不覆盖 Nginx master/worker 重载或 Cloudflare 实网部署。
|
||||
- 已接入 `omni.New` 的统一 transport / TLS / shutdownTimeout 配置:默认 nethttp,可显式选择 native 或实验 HTTP/2;证书在监听前加载,HTTP/2 配置必须启用 TLS。`RunWith` 继续支持自定义网络层,业务不依赖具体适配器。
|
||||
- 参考业务已增加五种真实 TCP/TLS 组合的 CRUD、认证权限、错误响应、requestId 日志与协议协商测试;三种适配器共用排空和超时退出测试。修复 native 退出时过早取消在途请求,并使关闭超时错误返回调用方。下一步仍为业务环境试用、native 多协议监听器设计与接入回归,HTTP/2 保持实验状态。
|
||||
- 统一启动已通过 Linux 全仓库 race/vet/build 及关键场景 10 次重复验证;修复了重复测试发现的 native 活动连接转为空闲时错过关闭的竞态,并保证不响应取消的 Handler 不阻塞关闭超时返回。脚手架独立项目的 native SIGINT/SIGTERM 验证通过;失败复现与最终验收证据见 [统一启动验收](performance/transportStartup.md)。
|
||||
- 已用 16 个并发 worker 完成 3,200 次建连/断连 race 验收;MaxConnections 满载拒绝、槽位释放和 TLS 连接上限均已覆盖。
|
||||
- 新增 `scripts/verifyLinux.py`,固定在 Go 1.25 Linux 容器内运行 race、vet、build 和可选 TLS/非 TLS perfcheck;perfcheck 记录堆、goroutine、连接和 Linux 文件描述符。
|
||||
- Linux/arm64 已完成 30 秒、128 并发、512 空闲连接的 TLS 与非 TLS 持续回归,报告见 `performance/linuxSoak.json` 和 `performance/linuxSoakTLS.json`,均为 0 错误且连接、goroutine、FD 回收正常。
|
||||
- 目标 Linux amd64 服务器已完成 30 分钟、128 并发、512 空闲连接的 TLS 验收:net/http 约 11,865 req/s、native 约 13,098 req/s,均为 0 错误;native P95/P99 为 44.040/52.429 ms,空闲连接堆增量约 4.50 MiB,FD 由 7 增至 519 后回到 7。
|
||||
- 新增 `scripts/verifyProxy.py`,在 Linux Docker 中用 Nginx 反向代理 native TLS 服务,检查真实 TLS、Keep-Alive、上游超时和 native 优雅关闭。
|
||||
- 自动验收入口:`python3 scripts/verifyPerformance.py`;Linux 固定验收入口:`python3 scripts/verifyLinux.py --load --tls --duration 30m`;代理联调入口:`python3 scripts/verifyProxy.py`。
|
||||
|
||||
## 6. 可复用业务基础模块(第一版)
|
||||
|
||||
- `cache`:有界 LRU 内存缓存、TTL、数据复制、同进程合并回源;`cache/redis` 适配 go-redis 客户端,保留借用连接池的生命周期边界。
|
||||
- `httpc`:复用出站连接、每目标主机连接上限、覆盖响应体的总超时、保守重试、requestId 传递、JSON 响应大小限制与安全日志字段。
|
||||
- `middleware`:有容量上限的令牌桶限流;按已认证身份隔离的幂等保护,覆盖冲突、处理中记录、失败记录、容量与请求/响应大小上限;协作式请求 deadline。
|
||||
- `task`:固定 worker、有界队列、任务超时、panic 隔离、间隔调度、不重叠执行、停止接收与排空/取消。
|
||||
- `go-omni add`:向已有项目生成 Handler / Service / Repository / Model 和模块注册入口;目标存在时拒绝覆盖,默认内存 CRUD 可注入持久化 Repository。
|
||||
- `examples/modules` 将以上运行时能力串联,使用真实 TLS 在 nethttp、native、实验 HTTP/2 下验证缓存、出站 requestId、认证、幂等、后台任务与退出。
|
||||
- 使用文档:[业务基础模块](MODULES.md)。验证入口:`GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...` 和 `python3 scripts/verifyModules.py`;Linux runner 顺序运行全量检查及独立 Redis 容器,硬限制 2 CPU / 2 GiB / 256 pids。
|
||||
- 最终代码已通过 Linux amd64 全仓库 race/vet/build、独立生成项目 CRUD、真实 Redis 7.4.11 与三种网络层 TLS 组合测试;测试容器清理完成、原有 7 个业务容器仍运行。提交、源码摘要与完整输出见 [验收记录](performance/modulesAcceptance.md)。
|
||||
- 当前幂等、限流、任务调度和合并回源均为单进程能力;Redis 仅提供缓存存储,不代表已实现分布式去重、任务持久化或跨副本选主。eTax 指定业务模块接入依用户要求暂缓。
|
||||
|
||||
## 7. 扩展业务模块(本批)
|
||||
|
||||
- 新增 OpenAPI、JWT/Session/Policy、OpenTelemetry、文件配置与日志级别热更新、熔断/并发隔离、Redis 共享限流/幂等、Redis Streams 持久队列、Local/S3/R2/OSS 存储;补充迁移、审计和分页。
|
||||
- 业务依赖独立接口,云平台 SDK 和 Redis 留在适配器中;按本次范围决定不集成 Orbit,不实现工作流编排。
|
||||
- 已完成本地模块测试及 race 检查,Linux 全仓库 race/vet/build、生成项目 CRUD、三个 Runner 的 TLS 示例和独立 Redis/MinIO/MySQL 均通过。共享状态、重复提交、失败消息、未确认消息接管、旧确认拒绝、对象签名和迁移 dirty 阻断有对应证据,见 [扩展模块验收](performance/advancedAcceptance.md)。OSS/R2 尚无云账号实网验收。
|
||||
- 契约、使用示例与限制见 [扩展模块](ADVANCED.md);验证入口为 `scripts/verifyModules.py --advanced`,继续限制单个测试容器 2 CPU / 2 GiB / 256 pids。
|
||||
|
||||
## 8. API 驱动项目与真实业务示例(v0.7.0-preview)
|
||||
|
||||
- `.api` → DTO / 绑定校验 / 路由 / Handler / Service / Repository / OpenAPI。
|
||||
- MySQL、日志与五种存储选择可组合生成;manifest 管理生成文件,保留业务文件。
|
||||
- 独立用户示例完成事务审计、鉴权、文件导出、数据库故障恢复与真实远程依赖验收。
|
||||
- 证据:[API 业务闭环](performance/apiWorkflowAcceptance.md)。
|
||||
|
||||
## 9. 接口开发体验(v0.7.0-preview.1)
|
||||
|
||||
- 错误目录与调用函数、401/403/409 文档、具名鉴权和运行时 hook 关联。
|
||||
- 组级继承/路由覆盖、多行接口描述、标签和弃用标记。
|
||||
- 路由总览、独立 OpenAPI 导出、生成预览与项目入口自动识别。
|
||||
- 证据:[接口开发体验验收](performance/apiDeveloperExperienceAcceptance.md)。
|
||||
|
||||
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,自研网络层已有可运行预览,稳定性与生产规模验证尚未完成。
|
||||
132
app/app.go
132
app/app.go
@@ -3,32 +3,46 @@ package app
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net"
|
||||
"net/http"
|
||||
"os"
|
||||
"os/signal"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
ginrouter "git.yuntuo.hk/k2safe/go-omni/transport/gin"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
netrouter "git.yuntuo.hk/k2safe/go-omni/transport/nethttp"
|
||||
)
|
||||
|
||||
// App is the top-level GoOmni application. Its public behavior depends on
|
||||
// httpx.Router, leaving the network implementation replaceable.
|
||||
type App struct {
|
||||
router httpx.Router
|
||||
config config.Config
|
||||
middlewares []httpx.Middleware
|
||||
closers []func() error
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
router httpx.Router
|
||||
config config.Config
|
||||
middlewares []httpx.Middleware
|
||||
closers []func() error
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
clientIPResolver *httpx.ClientIPResolver
|
||||
responseEncoder response.Encoder
|
||||
}
|
||||
|
||||
// New creates an application using the default Gin adapter.
|
||||
// Runner is the small lifecycle boundary used by an HTTP server
|
||||
// implementation. App.RunContext selects a configured built-in server; a custom runner
|
||||
// can own the listener, protocol parser and connection lifecycle.
|
||||
type Runner interface {
|
||||
Run(context.Context, http.Handler, string) error
|
||||
}
|
||||
|
||||
// New creates an application using GoOmni's standard-library router.
|
||||
func New(options ...config.Option) *App {
|
||||
settings := configure(options...)
|
||||
return newApp(ginrouter.New(settings.Mode), settings)
|
||||
return newApp(netrouter.New(), settings)
|
||||
}
|
||||
|
||||
// NewWithRouter creates an application with a custom network layer. A custom
|
||||
@@ -52,19 +66,43 @@ func configure(options ...config.Option) config.Config {
|
||||
}
|
||||
|
||||
func newApp(router httpx.Router, settings config.Config) *App {
|
||||
settings.Response = settings.Response.Clone()
|
||||
encoder, responseErr := response.New(settings.Response)
|
||||
if responseErr != nil {
|
||||
panic(responseErr)
|
||||
}
|
||||
var responseEncoder response.Encoder = encoder
|
||||
if settings.ResponseEncoder != nil {
|
||||
responseEncoder = settings.ResponseEncoder
|
||||
}
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
||||
if err != nil {
|
||||
panic("go-omni: " + err.Error())
|
||||
}
|
||||
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
||||
if settings.EnableLogger {
|
||||
middlewares = append(middlewares, middleware.AccessLog())
|
||||
middlewares = append(middlewares, middleware.AccessLog(middleware.WithSlowRequestThreshold(settings.SlowRequestThreshold)))
|
||||
}
|
||||
middlewares = append(middlewares, middleware.Recovery())
|
||||
return &App{router: router, config: settings, middlewares: middlewares}
|
||||
application := &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver, responseEncoder: responseEncoder}
|
||||
if fallback, ok := router.(httpx.FallbackRouter); ok {
|
||||
adaptError := func(status int, code, message string) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
application.adapt(func(*httpx.Context) error { return apierrors.New(status, code, message) }).ServeHTTP(w, r)
|
||||
})
|
||||
}
|
||||
fallback.SetFallbacks(adaptError(404, "NOT_FOUND", "route not found"), adaptError(405, "METHOD_NOT_ALLOWED", "method not allowed"))
|
||||
}
|
||||
return application
|
||||
}
|
||||
|
||||
// Router returns the transport contract used by this application.
|
||||
func (a *App) Router() httpx.Router { return a.router }
|
||||
|
||||
// Engine is kept as a compatibility name for the old API. It returns a
|
||||
// standard http.Handler, so callers no longer need a Gin type.
|
||||
// standard http.Handler, so callers do not depend on a router type.
|
||||
func (a *App) Engine() http.Handler { return a.router }
|
||||
|
||||
// ServeHTTP implements http.Handler for standard servers and HTTP tests.
|
||||
@@ -73,7 +111,13 @@ func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
|
||||
}
|
||||
|
||||
// Config returns the effective application configuration.
|
||||
func (a *App) Config() config.Config { return a.config }
|
||||
func (a *App) Config() config.Config {
|
||||
settings := a.config
|
||||
settings.Response = settings.Response.Clone()
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
return settings
|
||||
}
|
||||
|
||||
// Use adds middleware to routes registered after this call.
|
||||
func (a *App) Use(middleware ...httpx.Middleware) *App {
|
||||
@@ -104,12 +148,16 @@ func (a *App) Close() error {
|
||||
return a.closeErr
|
||||
}
|
||||
|
||||
// Run starts the HTTP server. The default address is :8080.
|
||||
// Run starts the HTTP server and handles SIGINT/SIGTERM. Use RunContext when
|
||||
// the caller owns cancellation. The default address is :8080.
|
||||
func (a *App) Run(address ...string) error {
|
||||
return a.RunContext(context.Background(), address...)
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
return a.RunContext(ctx, address...)
|
||||
}
|
||||
|
||||
// NewServer creates a standard HTTP server around the application.
|
||||
// NewServer creates a plain standard HTTP server around the application.
|
||||
// Use RunContext or ServeContext to apply the configured transport and TLS.
|
||||
func (a *App) NewServer(address string) *http.Server {
|
||||
if address == "" {
|
||||
address = a.config.Address
|
||||
@@ -125,6 +173,38 @@ func (a *App) RunContext(ctx context.Context, address ...string) error {
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
listener, err := net.Listen("tcp", a.listenAddress(address...))
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
// RunWith starts a custom server runner while keeping App-owned cleanup in the
|
||||
// same lifecycle. The runner receives the transport-neutral App handler and
|
||||
// owns transport/TLS settings; built-in transport selection is bypassed.
|
||||
func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) error {
|
||||
if runner == nil {
|
||||
return errors.Join(errors.New("go-omni: nil runner"), a.Close())
|
||||
}
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
return errors.Join(runner.Run(ctx, a, a.listenAddress(address...)), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) listenAddress(address ...string) string {
|
||||
listenOn := a.config.Address
|
||||
if listenOn == "" {
|
||||
listenOn = ":8080"
|
||||
@@ -132,9 +212,18 @@ func (a *App) RunContext(ctx context.Context, address ...string) error {
|
||||
if len(address) > 0 && address[0] != "" {
|
||||
listenOn = address[0]
|
||||
}
|
||||
server := a.NewServer(listenOn)
|
||||
return listenOn
|
||||
}
|
||||
|
||||
func (a *App) serveContext(ctx context.Context, server *http.Server, listener net.Listener) error {
|
||||
serveErr := make(chan error, 1)
|
||||
go func() { serveErr <- server.ListenAndServe() }()
|
||||
go func() {
|
||||
if server.TLSConfig != nil {
|
||||
serveErr <- server.ServeTLS(listener, "", "")
|
||||
} else {
|
||||
serveErr <- server.Serve(listener)
|
||||
}
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-serveErr:
|
||||
@@ -151,6 +240,11 @@ func (a *App) RunContext(ctx context.Context, address ...string) error {
|
||||
shutdownCtx, cancel := context.WithTimeout(context.Background(), shutdownTimeout)
|
||||
defer cancel()
|
||||
shutdownErr := server.Shutdown(shutdownCtx)
|
||||
if shutdownErr != nil {
|
||||
// Shutdown leaves active connections open on timeout. Close them so
|
||||
// request contexts are canceled before releasing shared resources.
|
||||
shutdownErr = errors.Join(shutdownErr, server.Close())
|
||||
}
|
||||
if err := <-serveErr; err != nil && err != http.ErrServerClosed {
|
||||
shutdownErr = errors.Join(shutdownErr, err)
|
||||
}
|
||||
@@ -162,5 +256,5 @@ func (a *App) writeError(ctx *httpx.Context, err error) {
|
||||
if err == nil || ctx.Responded() {
|
||||
return
|
||||
}
|
||||
ctx.WriteError(apierrors.From(err))
|
||||
ctx.WriteError(err)
|
||||
}
|
||||
|
||||
163
app/lifecycle_test.go
Normal file
163
app/lifecycle_test.go
Normal file
@@ -0,0 +1,163 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/nethttp"
|
||||
)
|
||||
|
||||
type closingListener struct {
|
||||
net.Listener
|
||||
closed chan struct{}
|
||||
once sync.Once
|
||||
}
|
||||
|
||||
func (l *closingListener) Close() error {
|
||||
err := l.Listener.Close()
|
||||
l.once.Do(func() { close(l.closed) })
|
||||
return err
|
||||
}
|
||||
|
||||
func startTestServer(t *testing.T, application *App) (*closingListener, context.CancelFunc, <-chan error) {
|
||||
t.Helper()
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wrapped := &closingListener{Listener: listener, closed: make(chan struct{})}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
t.Cleanup(cancel)
|
||||
t.Cleanup(func() { _ = listener.Close() })
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.serveContext(ctx, application.NewServer(""), wrapped) }()
|
||||
return wrapped, cancel, done
|
||||
}
|
||||
|
||||
func await(t *testing.T, event <-chan struct{}) {
|
||||
t.Helper()
|
||||
select {
|
||||
case <-event:
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("timed out waiting for server event")
|
||||
}
|
||||
}
|
||||
|
||||
func TestShutdownDrainsRequestBeforeCleanup(t *testing.T) {
|
||||
application := NewWithRouter(nethttp.New(), config.WithShutdownTimeout(3*time.Second))
|
||||
started, release, cleaned := make(chan struct{}), make(chan struct{}), make(chan struct{})
|
||||
var releaseOnce sync.Once
|
||||
unblock := func() { releaseOnce.Do(func() { close(release) }) }
|
||||
t.Cleanup(unblock)
|
||||
application.GET("/slow", func(ctx *httpx.Context) error {
|
||||
close(started)
|
||||
<-release
|
||||
return ctx.OK("finished")
|
||||
})
|
||||
application.OnShutdown(func() error { close(cleaned); return nil })
|
||||
listener, cancel, done := startTestServer(t, application)
|
||||
requestDone := make(chan error, 1)
|
||||
go func() {
|
||||
client := &http.Client{Timeout: 5 * time.Second}
|
||||
response, err := client.Get("http://" + listener.Addr().String() + "/slow")
|
||||
if err == nil {
|
||||
defer response.Body.Close()
|
||||
_, err = io.ReadAll(response.Body)
|
||||
if response.StatusCode != http.StatusOK {
|
||||
err = errors.New("request was not completed successfully")
|
||||
}
|
||||
}
|
||||
requestDone <- err
|
||||
}()
|
||||
await(t, started)
|
||||
cancel()
|
||||
await(t, listener.closed)
|
||||
select {
|
||||
case <-cleaned:
|
||||
t.Fatal("resources closed while a request was still running")
|
||||
default:
|
||||
}
|
||||
unblock()
|
||||
if err := <-requestDone; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("server did not finish shutdown")
|
||||
}
|
||||
await(t, cleaned)
|
||||
}
|
||||
|
||||
func TestShutdownTimeoutClosesActiveConnection(t *testing.T) {
|
||||
application := NewWithRouter(nethttp.New(), config.WithShutdownTimeout(25*time.Millisecond))
|
||||
started, canceled := make(chan struct{}), make(chan struct{})
|
||||
application.GET("/slow", func(ctx *httpx.Context) error {
|
||||
close(started)
|
||||
<-ctx.Context().Done()
|
||||
close(canceled)
|
||||
return nil
|
||||
})
|
||||
listener, cancel, done := startTestServer(t, application)
|
||||
requestDone := make(chan struct{})
|
||||
go func() {
|
||||
defer close(requestDone)
|
||||
client := &http.Client{Timeout: 5 * time.Second}
|
||||
response, err := client.Get("http://" + listener.Addr().String() + "/slow")
|
||||
if err == nil {
|
||||
response.Body.Close()
|
||||
}
|
||||
}()
|
||||
await(t, started)
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatalf("expected shutdown deadline error, got %v", err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("shutdown ignored timeout")
|
||||
}
|
||||
await(t, canceled)
|
||||
await(t, requestDone)
|
||||
}
|
||||
|
||||
func TestRunContextStartupFailureAndCanceledContextCloseResources(t *testing.T) {
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer listener.Close()
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := NewWithRouter(nethttp.New())
|
||||
closeCount := 0
|
||||
application.OnShutdown(func() error { closeCount++; return nil })
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
err := application.RunContext(ctx, listener.Addr().String())
|
||||
cancel()
|
||||
if !canceled && err == nil {
|
||||
t.Fatal("expected occupied address error")
|
||||
}
|
||||
if canceled && err != nil {
|
||||
t.Fatalf("canceled start: %v", err)
|
||||
}
|
||||
_ = application.Close()
|
||||
if closeCount != 1 {
|
||||
t.Fatalf("cleanup ran %d times", closeCount)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import (
|
||||
"net/http"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
func (a *App) handle(routes httpx.RouterGroup, method, path string, handler httpx.Handler, middleware []httpx.Middleware) {
|
||||
@@ -11,6 +12,10 @@ func (a *App) handle(routes httpx.RouterGroup, method, path string, handler http
|
||||
}
|
||||
|
||||
func (a *App) adapt(handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
|
||||
return a.adaptResponse(a.responseEncoder, handler, middleware...)
|
||||
}
|
||||
|
||||
func (a *App) adaptResponse(encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) http.Handler {
|
||||
wrapped := handler
|
||||
all := make([]httpx.Middleware, 0, len(a.middlewares)+len(middleware))
|
||||
all = append(all, a.middlewares...)
|
||||
@@ -21,13 +26,31 @@ func (a *App) adapt(handler httpx.Handler, middleware ...httpx.Middleware) http.
|
||||
}
|
||||
}
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
if len(a.config.TrustedProxies) > 0 {
|
||||
request = request.WithContext(httpx.WithClientIPResolver(request.Context(), a.clientIPResolver))
|
||||
}
|
||||
requestContext := httpx.NewContext(request, writer, httpx.RouteParams(request.Context()))
|
||||
requestContext.SetResponseEncoder(encoder)
|
||||
if err := wrapped(requestContext); err != nil {
|
||||
a.writeError(requestContext, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
// Handle registers an explicit method, including HEAD, OPTIONS and extension
|
||||
// methods, through the same business middleware and error handling as GET.
|
||||
func (a *App) Handle(method, path string, handler httpx.Handler, middleware ...httpx.Middleware) {
|
||||
a.handle(a.router, method, path, handler, middleware)
|
||||
}
|
||||
|
||||
// HandleWithResponse selects the route policy before all application middleware.
|
||||
func (a *App) HandleWithResponse(method, path string, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
|
||||
if encoder == nil {
|
||||
encoder = a.responseEncoder
|
||||
}
|
||||
a.router.Handle(method, path, a.adaptResponse(encoder, handler, middleware...))
|
||||
}
|
||||
|
||||
func (a *App) GET(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
|
||||
a.handle(a.router, http.MethodGet, path, handler, middleware)
|
||||
}
|
||||
@@ -80,6 +103,20 @@ func (g *RouterGroup) route(method, path string, handler httpx.Handler, middlewa
|
||||
g.app.handle(g.group, method, path, handler, combined)
|
||||
}
|
||||
|
||||
// Handle registers an explicit method within this group's middleware chain.
|
||||
func (g *RouterGroup) Handle(method, path string, handler httpx.Handler, middleware ...httpx.Middleware) {
|
||||
g.route(method, path, handler, middleware)
|
||||
}
|
||||
|
||||
func (g *RouterGroup) HandleWithResponse(method, path string, encoder response.Encoder, handler httpx.Handler, middleware ...httpx.Middleware) {
|
||||
if encoder == nil {
|
||||
encoder = g.app.responseEncoder
|
||||
}
|
||||
combined := append([]httpx.Middleware{}, g.middlewares...)
|
||||
combined = append(combined, middleware...)
|
||||
g.group.Handle(method, path, g.app.adaptResponse(encoder, handler, combined...))
|
||||
}
|
||||
|
||||
func (g *RouterGroup) GET(path string, handler httpx.Handler, middleware ...httpx.Middleware) {
|
||||
g.route(http.MethodGet, path, handler, middleware)
|
||||
}
|
||||
|
||||
79
app/transport.go
Normal file
79
app/transport.go
Normal file
@@ -0,0 +1,79 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
h2transport "git.yuntuo.hk/k2safe/go-omni/transport/http2"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
// ServeContext serves a caller-created listener using the configured transport
|
||||
// and TLS. App takes ownership of the listener, including on startup failure,
|
||||
// and runs resource cleanup after the server has stopped.
|
||||
func (a *App) ServeContext(ctx context.Context, listener net.Listener) error {
|
||||
if listener == nil {
|
||||
return errors.Join(errors.New("go-omni: nil listener"), a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
_ = listener.Close()
|
||||
return a.Close()
|
||||
}
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
_ = listener.Close()
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) prepareTransport() (func(context.Context, net.Listener) error, error) {
|
||||
if err := a.config.ValidateTransport(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
shutdownTimeout := a.config.ShutdownTimeout
|
||||
if shutdownTimeout <= 0 {
|
||||
shutdownTimeout = 10 * time.Second
|
||||
}
|
||||
var tlsConfig *tls.Config
|
||||
if strings.TrimSpace(a.config.TLS.CertFile) != "" {
|
||||
certificate, err := tls.LoadX509KeyPair(a.config.TLS.CertFile, a.config.TLS.KeyFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("go-omni: load TLS certificate: %w", err)
|
||||
}
|
||||
tlsConfig = &tls.Config{Certificates: []tls.Certificate{certificate}, MinVersion: tls.VersionTLS12}
|
||||
}
|
||||
switch a.config.Transport {
|
||||
case config.TransportNative:
|
||||
server := native.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
if tlsConfig != nil {
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
}
|
||||
return server.Serve, nil
|
||||
case config.TransportHTTP2:
|
||||
server := h2transport.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
default:
|
||||
server := a.NewServer("")
|
||||
server.TLSConfig = tlsConfig
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return a.serveContext(ctx, server, listener)
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
322
app/transport_test.go
Normal file
322
app/transport_test.go
Normal file
@@ -0,0 +1,322 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
func TestConfiguredTransportLifecycle(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
for _, timeout := range []bool{false, true} {
|
||||
name := transport + "/drain"
|
||||
if timeout {
|
||||
name = transport + "/timeout"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
limit := 3 * time.Second
|
||||
if timeout {
|
||||
limit = 80 * time.Millisecond
|
||||
}
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(limit))
|
||||
started, release, canceled := make(chan struct{}), make(chan struct{}), make(chan struct{})
|
||||
var once sync.Once
|
||||
unblock := func() { once.Do(func() { close(release) }) }
|
||||
defer unblock()
|
||||
var cleaned atomic.Int32
|
||||
application.OnShutdown(func() error { cleaned.Add(1); return nil })
|
||||
application.GET("/slow", func(ctx *httpx.Context) error {
|
||||
close(started)
|
||||
select {
|
||||
case <-ctx.Context().Done():
|
||||
close(canceled)
|
||||
return ctx.Context().Err()
|
||||
case <-release:
|
||||
return ctx.OK("finished")
|
||||
}
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wrapped := &closingListener{Listener: listener, closed: make(chan struct{})}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
defer listener.Close()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, wrapped) }()
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan error, 1)
|
||||
go func() {
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/slow")
|
||||
if err == nil {
|
||||
defer response.Body.Close()
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
err = readErr
|
||||
if response.StatusCode != 200 || !strings.Contains(string(body), "finished") {
|
||||
err = errors.New("drained response was lost")
|
||||
}
|
||||
}
|
||||
requestDone <- err
|
||||
}()
|
||||
await(t, started)
|
||||
began := time.Now()
|
||||
cancel()
|
||||
await(t, wrapped.closed)
|
||||
if !timeout {
|
||||
select {
|
||||
case <-canceled:
|
||||
t.Fatal("active request canceled before drain")
|
||||
default:
|
||||
}
|
||||
if cleaned.Load() != 0 {
|
||||
t.Fatal("cleanup preceded active request")
|
||||
}
|
||||
unblock()
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
if timeout {
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(began) < limit {
|
||||
t.Fatalf("timeout ignored: %v after %v", err, time.Since(began))
|
||||
}
|
||||
await(t, canceled)
|
||||
} else if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("server failed to stop")
|
||||
}
|
||||
select {
|
||||
case err := <-requestDone:
|
||||
if !timeout && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("client failed to finish")
|
||||
}
|
||||
_ = application.Close()
|
||||
if cleaned.Load() != 1 {
|
||||
t.Fatalf("cleanup count %d", cleaned.Load())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredStartupFailureClosesResources(t *testing.T) {
|
||||
occupied, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer occupied.Close()
|
||||
for _, test := range []struct {
|
||||
name, errorText string
|
||||
options []config.Option
|
||||
}{
|
||||
{"unknown", "transport", []config.Option{config.WithTransport("unknown")}},
|
||||
{"http2WithoutTLS", "tls", []config.Option{config.WithTransport("http2")}},
|
||||
{"partialTLS", "tls", []config.Option{config.WithTLS("missing.crt", "")}},
|
||||
{"nethttpCertificate", "load TLS certificate", []config.Option{config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"nativeCertificate", "load TLS certificate", []config.Option{config.WithTransport("native"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"http2Certificate", "load TLS certificate", []config.Option{config.WithTransport("http2"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
application := New(test.options...)
|
||||
cleanupError := errors.New("cleanup failure")
|
||||
count := 0
|
||||
application.OnShutdown(func() error { count++; return cleanupError })
|
||||
// The occupied address must not mask configuration/certificate errors.
|
||||
err := application.RunContext(context.Background(), occupied.Addr().String())
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorText) || !errors.Is(err, cleanupError) {
|
||||
t.Fatalf("startup: %v", err)
|
||||
}
|
||||
_ = application.Close()
|
||||
if count != 1 {
|
||||
t.Fatalf("cleanup count %d", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("unknown"))
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
err = application.ServeContext(ctx, listener)
|
||||
cancel()
|
||||
if canceled && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !canceled && err == nil {
|
||||
t.Fatal("invalid transport accepted")
|
||||
}
|
||||
if _, err := listener.Accept(); !errors.Is(err, net.ErrClosed) {
|
||||
t.Fatalf("listener leaked: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunContextConfiguredTLS(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
reserved, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
address := reserved.Addr().String()
|
||||
reserved.Close()
|
||||
settings := config.Default()
|
||||
settings.Address, settings.Transport = address, transport
|
||||
settings.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||
application := New(config.WithConfig(settings))
|
||||
application.GET("/health", func(ctx *httpx.Context) error { return ctx.OK("ok") })
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.RunContext(ctx) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("RunContext failed to stop")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for {
|
||||
response, err := client.Get("https://" + address + "/health")
|
||||
if err == nil {
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if readErr != nil || response.StatusCode != 200 || !strings.Contains(string(body), `"data":"ok"`) {
|
||||
t.Fatalf("response %s: %v", body, readErr)
|
||||
}
|
||||
want := "h2"
|
||||
if transport == config.TransportNative {
|
||||
want = "http/1.1"
|
||||
}
|
||||
if response.TLS.NegotiatedProtocol != want || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatalf("TLS negotiation: %+v", response.TLS)
|
||||
}
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type runnerFunc func(context.Context, http.Handler, string) error
|
||||
|
||||
func (f runnerFunc) Run(ctx context.Context, handler http.Handler, address string) error {
|
||||
return f(ctx, handler, address)
|
||||
}
|
||||
|
||||
func TestRunWithRemainsIndependentOfBuiltInTransport(t *testing.T) {
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("custom"), config.WithTLS("missing", "missing"))
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
calls, cleanups := 0, 0
|
||||
application.OnShutdown(func() error { cleanups++; return nil })
|
||||
err := application.RunWith(ctx, runnerFunc(func(_ context.Context, handler http.Handler, address string) error {
|
||||
calls++
|
||||
if handler != application || address != "127.0.0.1:1234" {
|
||||
t.Fatal("custom runner arguments changed")
|
||||
}
|
||||
return nil
|
||||
}), "127.0.0.1:1234")
|
||||
cancel()
|
||||
wantCalls := 1
|
||||
if canceled {
|
||||
wantCalls = 0
|
||||
}
|
||||
if err != nil || calls != wantCalls || cleanups != 1 {
|
||||
t.Fatalf("RunWith: %v calls=%d cleanups=%d", err, calls, cleanups)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredShutdownDoesNotWaitForUncooperativeHandler(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(80*time.Millisecond))
|
||||
started := make(chan context.Context, 1)
|
||||
release, finished := make(chan struct{}), make(chan struct{})
|
||||
application.GET("/blocked", func(ctx *httpx.Context) error {
|
||||
defer close(finished)
|
||||
started <- ctx.Context()
|
||||
<-release
|
||||
return nil
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer listener.Close()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan struct{})
|
||||
go func() {
|
||||
defer close(requestDone)
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/blocked")
|
||||
if err == nil {
|
||||
response.Body.Close()
|
||||
}
|
||||
}()
|
||||
var requestContext context.Context
|
||||
select {
|
||||
case requestContext = <-started:
|
||||
case <-time.After(3 * time.Second):
|
||||
close(release)
|
||||
t.Fatal("handler did not start")
|
||||
}
|
||||
defer func() { close(release); await(t, finished) }()
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("handler defeated shutdown deadline")
|
||||
}
|
||||
// net/http's HTTP/2 streams observe the closed socket asynchronously.
|
||||
await(t, requestContext.Done())
|
||||
await(t, requestDone)
|
||||
})
|
||||
}
|
||||
}
|
||||
43
app_test.go
43
app_test.go
@@ -8,12 +8,10 @@ import (
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
func TestAppReturnsStandardResponse(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
app.GET("/health", func(ctx *Context) error {
|
||||
return ctx.OK(map[string]string{"status": "ok"})
|
||||
})
|
||||
@@ -41,7 +39,7 @@ func TestAppReturnsStandardResponse(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppReturnsAPIError(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
app.GET("/users/:id", func(ctx *Context) error {
|
||||
return NotFound("USER_NOT_FOUND", "user was not found")
|
||||
})
|
||||
@@ -63,7 +61,7 @@ func TestAppReturnsAPIError(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppRecoversFromPanic(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
app.GET("/panic", func(ctx *Context) error {
|
||||
panic("boom")
|
||||
})
|
||||
@@ -85,7 +83,7 @@ func TestAppRecoversFromPanic(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppRunContextShutsDownGracefully(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode), WithShutdownTimeout(time.Second))
|
||||
app := New(WithShutdownTimeout(time.Second))
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- app.RunContext(ctx, "127.0.0.1:0") }()
|
||||
@@ -102,7 +100,7 @@ func TestAppRunContextShutsDownGracefully(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestAppClosesResourcesInReverseOrder(t *testing.T) {
|
||||
app := New(WithMode(gin.TestMode))
|
||||
app := New()
|
||||
var closed []string
|
||||
app.OnShutdown(func() error {
|
||||
closed = append(closed, "database")
|
||||
@@ -130,3 +128,34 @@ func TestAppUsesConfiguredAddressWhenNoOverrideIsProvided(t *testing.T) {
|
||||
t.Fatalf("expected configured address %q, got %q", settings.Address, got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppConfigCopiesTrustedProxies(t *testing.T) {
|
||||
proxies := []string{"127.0.0.1"}
|
||||
app := New(WithTrustedProxies(proxies...))
|
||||
proxies[0] = "10.0.0.0/8"
|
||||
settings := app.Config()
|
||||
if len(settings.TrustedProxies) != 1 || settings.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatalf("trusted proxy config was not copied: %+v", settings.TrustedProxies)
|
||||
}
|
||||
settings.TrustedProxies[0] = "192.0.2.0/24"
|
||||
if app.Config().TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("App.Config should return a copy")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAppTrustedProxyClientIPReachesHandlers(t *testing.T) {
|
||||
app := New(WithTrustedProxies("127.0.0.1"))
|
||||
app.GET("/client-ip", func(ctx *Context) error { return ctx.OK(ctx.ClientIP()) })
|
||||
request := httptest.NewRequest(http.MethodGet, "/client-ip", nil)
|
||||
request.RemoteAddr = "127.0.0.1:443"
|
||||
request.Header.Set("X-Forwarded-For", "198.51.100.7")
|
||||
recorder := httptest.NewRecorder()
|
||||
app.Engine().ServeHTTP(recorder, request)
|
||||
var response Response
|
||||
if err := json.Unmarshal(recorder.Body.Bytes(), &response); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.Data != "198.51.100.7" {
|
||||
t.Fatalf("expected trusted proxy client IP, got %#v", response.Data)
|
||||
}
|
||||
}
|
||||
|
||||
106
audit/audit.go
Normal file
106
audit/audit.go
Normal file
@@ -0,0 +1,106 @@
|
||||
// Package audit records explicitly selected business events without request bodies.
|
||||
package audit
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Event struct {
|
||||
Time time.Time `json:"time"`
|
||||
ActorID string `json:"actorId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Action string `json:"action"`
|
||||
Resource string `json:"resource"`
|
||||
Outcome string `json:"outcome"`
|
||||
RequestID string `json:"requestId,omitempty"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
Fields map[string]string `json:"fields,omitempty"`
|
||||
}
|
||||
type Sink interface {
|
||||
Write(context.Context, Event) error
|
||||
}
|
||||
type Recorder struct {
|
||||
sink Sink
|
||||
allowed map[string]bool
|
||||
}
|
||||
|
||||
func New(sink Sink, allowedFields ...string) (*Recorder, error) {
|
||||
if sink == nil || len(allowedFields) > 32 {
|
||||
return nil, errors.New("audit: sink and at most 32 fields required")
|
||||
}
|
||||
r := &Recorder{sink: sink, allowed: map[string]bool{}}
|
||||
for _, name := range allowedFields {
|
||||
if name == "" || len(name) > 64 {
|
||||
return nil, errors.New("audit: invalid field")
|
||||
}
|
||||
r.allowed[name] = true
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
func (r *Recorder) Record(ctx context.Context, event Event) error {
|
||||
if ctx == nil {
|
||||
return errors.New("audit: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if event.ActorID == "" || event.Action == "" || event.Resource == "" || (event.Outcome != "success" && event.Outcome != "failure" && event.Outcome != "denied") {
|
||||
return errors.New("audit: actor/action/resource and valid outcome required")
|
||||
}
|
||||
for _, value := range []string{event.ActorID, event.TenantID, event.Action, event.Resource} {
|
||||
if len(value) > 512 {
|
||||
return errors.New("audit: event field too large")
|
||||
}
|
||||
}
|
||||
event.Time = time.Now().UTC()
|
||||
event.RequestID = logx.RequestID(ctx)
|
||||
event.TraceID = logx.TraceID(ctx)
|
||||
filtered := map[string]string{}
|
||||
for name, value := range event.Fields {
|
||||
if r.allowed[name] {
|
||||
if len(value) > 1024 {
|
||||
return errors.New("audit: attribute too large")
|
||||
}
|
||||
filtered[name] = value
|
||||
}
|
||||
}
|
||||
event.Fields = filtered
|
||||
return r.sink.Write(ctx, event)
|
||||
}
|
||||
|
||||
// JSONSink borrows its writer and serializes complete JSON lines. Write errors
|
||||
// are returned; callers choose whether a failed audit must reject an operation.
|
||||
type JSONSink struct {
|
||||
mu sync.Mutex
|
||||
writer io.Writer
|
||||
}
|
||||
|
||||
func NewJSONSink(writer io.Writer) (*JSONSink, error) {
|
||||
if writer == nil {
|
||||
return nil, errors.New("audit: writer required")
|
||||
}
|
||||
return &JSONSink{writer: writer}, nil
|
||||
}
|
||||
func (s *JSONSink) Write(ctx context.Context, event Event) error {
|
||||
data, err := json.Marshal(event)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data = append(data, '\n')
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := s.writer.Write(data)
|
||||
if err == nil && n != len(data) {
|
||||
return io.ErrShortWrite
|
||||
}
|
||||
return err
|
||||
}
|
||||
50
audit/audit_test.go
Normal file
50
audit/audit_test.go
Normal file
@@ -0,0 +1,50 @@
|
||||
package audit
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type failedWriter struct{}
|
||||
|
||||
func (failedWriter) Write([]byte) (int, error) { return 0, io.ErrClosedPipe }
|
||||
func TestAllowlistCorrelationAndWriteFailure(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
sink, _ := NewJSONSink(&buf)
|
||||
recorder, _ := New(sink, "status")
|
||||
ctx := logx.WithRequestID(context.Background(), "r1")
|
||||
ctx = logx.WithTraceID(ctx, "t1")
|
||||
event := Event{ActorID: "u1", Action: "update", Resource: "object:1", Outcome: "success", Fields: map[string]string{"status": "ok", "password": "secret"}}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := recorder.Record(ctx, event); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
decoder := json.NewDecoder(&buf)
|
||||
for i := 0; i < 20; i++ {
|
||||
var got Event
|
||||
if err := decoder.Decode(&got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Fields["password"] != "" || got.RequestID != "r1" || got.TraceID != "t1" || got.Time.IsZero() {
|
||||
t.Fatal(got)
|
||||
}
|
||||
}
|
||||
bad, _ := NewJSONSink(failedWriter{})
|
||||
recorder, _ = New(bad)
|
||||
if err := recorder.Record(ctx, event); !errors.Is(err, io.ErrClosedPipe) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
105
auth/adapters_test.go
Normal file
105
auth/adapters_test.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestJWTValidationAndRotation(t *testing.T) {
|
||||
pub, private, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
j, err := NewJWT(JWTConfig{Issuer: "issuer", Audience: "api", Keys: map[string]JWTKey{"first": {Algorithm: "EdDSA", PublicKey: pub}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
claims := JWTClaims{RegisteredClaims: jwt.RegisteredClaims{Issuer: "issuer", Subject: "user1", Audience: jwt.ClaimStrings{"api"}, ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour))}, TenantID: "tenant1", Roles: []string{"reader"}}
|
||||
sign := func(c JWTClaims, kid string, method jwt.SigningMethod, key any) string {
|
||||
t.Helper()
|
||||
token := jwt.NewWithClaims(method, c)
|
||||
token.Header["kid"] = kid
|
||||
raw, err := token.SignedString(key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
raw := sign(claims, "first", jwt.SigningMethodEdDSA, private)
|
||||
// Constructor clones caller-owned key bytes.
|
||||
pub[0] ^= 1
|
||||
p, err := j.Authenticate(context.Background(), raw)
|
||||
if err != nil || p.TenantID != "tenant1" || !p.HasRole("reader") {
|
||||
t.Fatalf("%+v %v", p, err)
|
||||
}
|
||||
for name, change := range map[string]func(*JWTClaims){"issuer": func(c *JWTClaims) { c.Issuer = "bad" }, "audience": func(c *JWTClaims) { c.Audience = jwt.ClaimStrings{"other"} }, "expired": func(c *JWTClaims) { c.ExpiresAt = jwt.NewNumericDate(time.Now().Add(-time.Hour)) }, "noExpiry": func(c *JWTClaims) { c.ExpiresAt = nil }, "future": func(c *JWTClaims) { c.NotBefore = jwt.NewNumericDate(time.Now().Add(time.Hour)) }, "subject": func(c *JWTClaims) { c.Subject = "" }} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
bad := claims
|
||||
change(&bad)
|
||||
if _, err = j.Authenticate(context.Background(), sign(bad, "first", jwt.SigningMethodEdDSA, private)); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, invalid := range []string{sign(claims, "missing", jwt.SigningMethodEdDSA, private), sign(claims, "first", jwt.SigningMethodHS256, []byte("invalid secret")), strings.Repeat("x", 8193)} {
|
||||
if _, err = j.Authenticate(context.Background(), invalid); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err = j.ReplaceKeys(map[string]JWTKey{"second": {Algorithm: "EdDSA", PublicKey: private.Public()}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), raw); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), sign(claims, "second", jwt.SigningMethodEdDSA, private)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
if _, err = j.Authenticate(ctx, raw); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestSessionsRevokeAndIsolation(t *testing.T) {
|
||||
mem, _ := cache.NewMemory(cache.MemoryConfig{})
|
||||
defer mem.Close()
|
||||
s, err := NewSessions(mem, "session:", time.Minute)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
token, err := s.Issue(ctx, Principal{UserID: "u1", TenantID: "t1", Roles: []string{"reader"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
key, _ := s.key(token)
|
||||
if strings.Contains(key, token) {
|
||||
t.Fatal("plaintext session secret stored")
|
||||
}
|
||||
p, err := s.Authenticate(ctx, token)
|
||||
if err != nil || p.UserID != "u1" {
|
||||
t.Fatal(p, err)
|
||||
}
|
||||
p.Roles[0] = "admin"
|
||||
p, _ = s.Authenticate(ctx, token)
|
||||
if p.HasRole("admin") {
|
||||
t.Fatal("principal alias")
|
||||
}
|
||||
if err = s.Revoke(ctx, token); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, token); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, "bad"); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -21,9 +21,10 @@ var (
|
||||
// Principal is the authenticated identity made available to business handlers.
|
||||
// Claims are optional provider-specific values and should not be required by core business code.
|
||||
type Principal struct {
|
||||
UserID string
|
||||
Roles []string
|
||||
Claims map[string]any
|
||||
UserID string `json:"userId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
Claims map[string]any `json:"claims,omitempty"`
|
||||
}
|
||||
|
||||
func (p Principal) HasRole(role string) bool {
|
||||
|
||||
@@ -9,11 +9,10 @@ import (
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"github.com/gin-gonic/gin"
|
||||
)
|
||||
|
||||
func TestRequireAuthAndRole(t *testing.T) {
|
||||
app := omni.New(omni.WithMode(gin.TestMode))
|
||||
app := omni.New()
|
||||
authenticator := auth.StaticAuthenticator{"good-token": {UserID: "user-1", Roles: []string{"user"}}}
|
||||
private := app.Group("/private", auth.RequireAuth(authenticator))
|
||||
private.GET("/me", func(ctx *omni.Context) error {
|
||||
@@ -62,7 +61,7 @@ func TestRequireAuthAndRole(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestOptionalAuth(t *testing.T) {
|
||||
app := omni.New(omni.WithMode(gin.TestMode))
|
||||
app := omni.New()
|
||||
app.GET("/public", func(ctx *omni.Context) error {
|
||||
principal, authenticated := auth.FromContext(ctx)
|
||||
return ctx.OK(map[string]any{"authenticated": authenticated, "userId": principal.UserID})
|
||||
|
||||
127
auth/jwt.go
Normal file
127
auth/jwt.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rsa"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
)
|
||||
|
||||
// JWTKey binds a trusted key ID to exactly one asymmetric signing algorithm.
|
||||
// Keys come from server configuration, never a token's jku/x5u URL.
|
||||
type JWTKey struct {
|
||||
Algorithm string
|
||||
PublicKey any
|
||||
}
|
||||
type JWTConfig struct {
|
||||
Issuer string `json:"issuer"`
|
||||
Audience string `json:"audience"`
|
||||
Leeway time.Duration `json:"leeway"`
|
||||
Keys map[string]JWTKey `json:"-"`
|
||||
}
|
||||
type JWT struct {
|
||||
config JWTConfig
|
||||
mu sync.RWMutex
|
||||
keys map[string]JWTKey
|
||||
}
|
||||
type JWTClaims struct {
|
||||
jwt.RegisteredClaims
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
}
|
||||
|
||||
func NewJWT(cfg JWTConfig) (*JWT, error) {
|
||||
if cfg.Issuer == "" || cfg.Audience == "" || cfg.Leeway < 0 || cfg.Leeway > 2*time.Minute {
|
||||
return nil, errors.New("auth: issuer/audience required, leeway must be 0..2m")
|
||||
}
|
||||
j := &JWT{config: cfg}
|
||||
j.config.Keys = nil
|
||||
if err := j.ReplaceKeys(cfg.Keys); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return j, nil
|
||||
}
|
||||
|
||||
// ReplaceKeys atomically rotates the trusted key set. Retain old keys until
|
||||
// their tokens expire if a rotation should preserve existing sessions.
|
||||
func (j *JWT) ReplaceKeys(keys map[string]JWTKey) error {
|
||||
if len(keys) == 0 || len(keys) > 64 {
|
||||
return errors.New("auth: require 1..64 JWT keys")
|
||||
}
|
||||
copyKeys := make(map[string]JWTKey, len(keys))
|
||||
for id, key := range keys {
|
||||
if id == "" || len(id) > 128 {
|
||||
return errors.New("auth: invalid key ID")
|
||||
}
|
||||
switch key.Algorithm {
|
||||
case "RS256":
|
||||
k, ok := key.PublicKey.(*rsa.PublicKey)
|
||||
if !ok || k == nil || k.N == nil || k.N.BitLen() < 2048 || k.E < 3 {
|
||||
return errors.New("auth: RS256 requires a >=2048-bit RSA public key")
|
||||
}
|
||||
case "EdDSA":
|
||||
k, ok := key.PublicKey.(ed25519.PublicKey)
|
||||
if !ok || len(k) != ed25519.PublicKeySize {
|
||||
return errors.New("auth: EdDSA requires an Ed25519 public key")
|
||||
}
|
||||
default:
|
||||
return errors.New("auth: supported algorithms are RS256 and EdDSA")
|
||||
}
|
||||
encoded, err := x509.MarshalPKIXPublicKey(key.PublicKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
key.PublicKey, err = x509.ParsePKIXPublicKey(encoded)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
copyKeys[id] = key
|
||||
}
|
||||
j.mu.Lock()
|
||||
j.keys = copyKeys
|
||||
j.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (j *JWT) Authenticate(ctx context.Context, raw string) (Principal, error) {
|
||||
if ctx == nil {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
if len(raw) == 0 || len(raw) > 8192 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
claims := new(JWTClaims)
|
||||
token, err := jwt.ParseWithClaims(raw, claims, func(t *jwt.Token) (any, error) {
|
||||
id, ok := t.Header["kid"].(string)
|
||||
if !ok {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
j.mu.RLock()
|
||||
key, exists := j.keys[id]
|
||||
j.mu.RUnlock()
|
||||
if !exists || t.Method.Alg() != key.Algorithm {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
return key.PublicKey, nil
|
||||
}, jwt.WithValidMethods([]string{"RS256", "EdDSA"}), jwt.WithIssuer(j.config.Issuer), jwt.WithAudience(j.config.Audience), jwt.WithExpirationRequired(), jwt.WithIssuedAt(), jwt.WithLeeway(j.config.Leeway))
|
||||
if err != nil || !token.Valid || claims.Subject == "" || len(claims.Subject) > 256 || len(claims.TenantID) > 256 || len(claims.Roles) > 64 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
for _, role := range claims.Roles {
|
||||
if role == "" || len(role) > 128 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
return Principal{UserID: claims.Subject, TenantID: claims.TenantID, Roles: append([]string(nil), claims.Roles...)}, nil
|
||||
}
|
||||
34
auth/policy.go
Normal file
34
auth/policy.go
Normal file
@@ -0,0 +1,34 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type Policy interface {
|
||||
Authorize(context.Context, Principal, string, string) error
|
||||
}
|
||||
type PolicyFunc func(context.Context, Principal, string, string) error
|
||||
|
||||
func (f PolicyFunc) Authorize(ctx context.Context, p Principal, action, resource string) error {
|
||||
return f(ctx, p, action, resource)
|
||||
}
|
||||
func RequirePermission(policy Policy, action string, resource func(*httpx.Context) string) httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
p, ok := FromContext(c)
|
||||
if !ok {
|
||||
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||
}
|
||||
if policy == nil || action == "" || resource == nil {
|
||||
return apierrors.Internal(errors.New("auth: incomplete permission policy"))
|
||||
}
|
||||
if err := policy.Authorize(c.Context(), p, action, resource(c)); err != nil {
|
||||
return apierrors.Forbidden("FORBIDDEN", "permission denied").WithCause(err)
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
87
auth/session.go
Normal file
87
auth/session.go
Normal file
@@ -0,0 +1,87 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
)
|
||||
|
||||
// Sessions stores server-issued opaque bearer credentials as SHA-256 keys.
|
||||
// The caller verifies login credentials before Issue; Store owns persistence.
|
||||
type Sessions struct {
|
||||
store cache.Store
|
||||
prefix string
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func NewSessions(store cache.Store, prefix string, ttl time.Duration) (*Sessions, error) {
|
||||
if store == nil || len(prefix) > 512 || ttl < time.Second || ttl > 30*24*time.Hour {
|
||||
return nil, errors.New("auth: invalid session configuration")
|
||||
}
|
||||
return &Sessions{store: store, prefix: prefix, ttl: ttl}, nil
|
||||
}
|
||||
func (s *Sessions) Issue(ctx context.Context, p Principal) (string, error) {
|
||||
if p.UserID == "" || len(p.UserID) > 256 || len(p.TenantID) > 256 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
data, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(data) > 16384 {
|
||||
return "", errors.New("auth: session principal too large")
|
||||
}
|
||||
secret := make([]byte, 32)
|
||||
if _, err := rand.Read(secret); err != nil {
|
||||
return "", err
|
||||
}
|
||||
token := base64.RawURLEncoding.EncodeToString(secret)
|
||||
key, _ := s.key(token)
|
||||
if err := s.store.Set(ctx, key, data, s.ttl); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
func (s *Sessions) key(token string) (string, error) {
|
||||
if len(token) != 43 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
b, err := base64.RawURLEncoding.DecodeString(token)
|
||||
if err != nil || len(b) != 32 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
sum := sha256.Sum256(b)
|
||||
return s.prefix + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
func (s *Sessions) Authenticate(ctx context.Context, token string) (Principal, error) {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
data, err := s.store.Get(ctx, key)
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
var p Principal
|
||||
if len(data) > 16384 || json.Unmarshal(data, &p) != nil || p.UserID == "" {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
func (s *Sessions) Revoke(ctx context.Context, token string) error {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.store.Delete(ctx, key)
|
||||
}
|
||||
135
cache/cache.go
vendored
Normal file
135
cache/cache.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
// Package cache provides replaceable byte-oriented caches and bounded local
|
||||
// request coalescing. Storage errors are never silently treated as misses.
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrMiss = errors.New("cache: key not found")
|
||||
ErrClosed = errors.New("cache: closed")
|
||||
ErrTooLarge = errors.New("cache: entry exceeds capacity")
|
||||
ErrBusy = errors.New("cache: too many concurrent loads")
|
||||
)
|
||||
|
||||
// Store implementations copy values and accept ttl=0 for no expiration.
|
||||
// Negative TTLs are invalid. Delete is successful for missing keys.
|
||||
type Store interface {
|
||||
Get(context.Context, string) ([]byte, error)
|
||||
Set(context.Context, string, []byte, time.Duration) error
|
||||
Delete(context.Context, ...string) error
|
||||
}
|
||||
|
||||
func Check(ctx context.Context, key string, ttl time.Duration) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if key == "" || len(key) > 1024 {
|
||||
return errors.New("cache: key must contain 1..1024 bytes")
|
||||
}
|
||||
if ttl < 0 {
|
||||
return errors.New("cache: negative TTL")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type LoaderConfig struct {
|
||||
MaxLoads int `json:"maxLoads"`
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
}
|
||||
|
||||
type flight struct {
|
||||
done chan struct{}
|
||||
value []byte
|
||||
err error
|
||||
}
|
||||
|
||||
// Loader coalesces concurrent misses for the same key within this process.
|
||||
// Caller cancellation stops waiting without canceling other callers' load.
|
||||
// Load functions must observe their context, which has a bounded deadline.
|
||||
type Loader struct {
|
||||
store Store
|
||||
config LoaderConfig
|
||||
mu sync.Mutex
|
||||
flights map[string]*flight
|
||||
}
|
||||
|
||||
func NewLoader(store Store, cfg LoaderConfig) (*Loader, error) {
|
||||
if store == nil || cfg.MaxLoads < 0 || cfg.Timeout < 0 {
|
||||
return nil, errors.New("cache: invalid loader configuration")
|
||||
}
|
||||
if cfg.MaxLoads == 0 {
|
||||
cfg.MaxLoads = 128
|
||||
}
|
||||
if cfg.Timeout == 0 {
|
||||
cfg.Timeout = 5 * time.Second
|
||||
}
|
||||
return &Loader{store: store, config: cfg, flights: make(map[string]*flight)}, nil
|
||||
}
|
||||
|
||||
func (l *Loader) GetOrLoad(ctx context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error)) ([]byte, error) {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if load == nil {
|
||||
return nil, errors.New("cache: nil loader")
|
||||
}
|
||||
value, err := l.store.Get(ctx, key)
|
||||
if !errors.Is(err, ErrMiss) {
|
||||
return value, err
|
||||
}
|
||||
l.mu.Lock()
|
||||
f := l.flights[key]
|
||||
if f == nil {
|
||||
if len(l.flights) >= l.config.MaxLoads {
|
||||
l.mu.Unlock()
|
||||
return nil, ErrBusy
|
||||
}
|
||||
f = &flight{done: make(chan struct{})}
|
||||
l.flights[key] = f
|
||||
go l.run(context.WithoutCancel(ctx), key, ttl, load, f)
|
||||
}
|
||||
l.mu.Unlock()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
case <-f.done:
|
||||
return append([]byte(nil), f.value...), f.err
|
||||
}
|
||||
}
|
||||
|
||||
func (l *Loader) run(parent context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error), f *flight) {
|
||||
ctx, cancel := context.WithTimeout(parent, l.config.Timeout)
|
||||
defer cancel()
|
||||
defer func() {
|
||||
if value := recover(); value != nil {
|
||||
f.err = fmt.Errorf("cache: loader panic: %v", value)
|
||||
}
|
||||
l.mu.Lock()
|
||||
delete(l.flights, key)
|
||||
close(f.done)
|
||||
l.mu.Unlock()
|
||||
}()
|
||||
// Another request may have populated the key between the first Get and
|
||||
// flight acquisition. Recheck inside the coalesced load.
|
||||
f.value, f.err = l.store.Get(ctx, key)
|
||||
if !errors.Is(f.err, ErrMiss) {
|
||||
return
|
||||
}
|
||||
f.value, f.err = load(ctx)
|
||||
if f.err == nil {
|
||||
f.err = ctx.Err()
|
||||
}
|
||||
if f.err == nil {
|
||||
f.err = l.store.Set(ctx, key, f.value, ttl)
|
||||
}
|
||||
f.value = append([]byte(nil), f.value...)
|
||||
}
|
||||
118
cache/cache_test.go
vendored
Normal file
118
cache/cache_test.go
vendored
Normal file
@@ -0,0 +1,118 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestMemoryOwnershipCapacityExpiryAndClose(t *testing.T) {
|
||||
m, err := NewMemory(MemoryConfig{MaxEntries: 2, MaxBytes: 10})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now()
|
||||
m.now = func() time.Time { return now }
|
||||
ctx := context.Background()
|
||||
value := []byte("aa")
|
||||
if err := m.Set(ctx, "a", value, time.Second); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
value[0] = 'x'
|
||||
got, _ := m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("retained caller memory")
|
||||
}
|
||||
got[0] = 'z'
|
||||
got, _ = m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("returned mutable cache memory")
|
||||
}
|
||||
m.Set(ctx, "b", []byte("bb"), 0)
|
||||
m.Get(ctx, "a")
|
||||
m.Set(ctx, "c", []byte("cc"), 0)
|
||||
if _, err := m.Get(ctx, "b"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("LRU eviction failed")
|
||||
}
|
||||
now = now.Add(time.Second)
|
||||
if _, err := m.Get(ctx, "a"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("expiration boundary ignored")
|
||||
}
|
||||
if err := m.Set(ctx, "large", []byte("123456"), 0); !errors.Is(err, ErrTooLarge) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := m.Get(ctx, "c"); err != nil {
|
||||
t.Fatal("oversized write evicted valid data")
|
||||
}
|
||||
canceled, cancel := context.WithCancel(ctx)
|
||||
cancel()
|
||||
if err := m.Delete(canceled, "c"); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
m.Close()
|
||||
if _, err := m.Get(ctx, "c"); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderCoalescesAndSeparatesCallerCancellation(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
loader, _ := NewLoader(m, LoaderConfig{MaxLoads: 1, Timeout: time.Second})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
var calls atomic.Int32
|
||||
load := func(ctx context.Context) ([]byte, error) {
|
||||
calls.Add(1)
|
||||
close(started)
|
||||
select {
|
||||
case <-release:
|
||||
return []byte("value"), nil
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
first := make(chan error, 1)
|
||||
go func() { _, err := loader.GetOrLoad(ctx, "key", time.Minute, load); first <- err }()
|
||||
<-started
|
||||
second := make(chan error, 1)
|
||||
go func() {
|
||||
v, err := loader.GetOrLoad(context.Background(), "key", time.Minute, load)
|
||||
if err == nil && string(v) != "value" {
|
||||
err = errors.New("wrong value")
|
||||
}
|
||||
second <- err
|
||||
}()
|
||||
if _, err := loader.GetOrLoad(context.Background(), "other", 0, load); !errors.Is(err, ErrBusy) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cancel()
|
||||
if err := <-first; !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
close(release)
|
||||
if err := <-second; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 1 {
|
||||
t.Fatalf("loads=%d", calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderPanicDeadlineAndStoreFailure(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
l, _ := NewLoader(m, LoaderConfig{Timeout: 20 * time.Millisecond})
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(context.Context) ([]byte, error) { panic("failure") }); err == nil {
|
||||
t.Fatal("panic escaped")
|
||||
}
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(ctx context.Context) ([]byte, error) { <-ctx.Done(); return nil, ctx.Err() }); !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
if _, err := l.GetOrLoad(context.Background(), "closed", 0, func(context.Context) ([]byte, error) { t.Fatal("store outage triggered load"); return nil, nil }); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
135
cache/memory.go
vendored
Normal file
135
cache/memory.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type MemoryConfig struct {
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxBytes int `json:"maxBytes"` // Sum of key and value bytes, excluding Go object overhead.
|
||||
}
|
||||
|
||||
type entry struct {
|
||||
key string
|
||||
value []byte
|
||||
expires time.Time
|
||||
}
|
||||
|
||||
// Memory is an LRU cache with lazy expiration and no background goroutines.
|
||||
type Memory struct {
|
||||
mu sync.Mutex
|
||||
config MemoryConfig
|
||||
items map[string]*list.Element
|
||||
lru *list.List
|
||||
bytes int
|
||||
closed bool
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewMemory(cfg MemoryConfig) (*Memory, error) {
|
||||
if cfg.MaxEntries < 0 || cfg.MaxBytes < 0 {
|
||||
return nil, errors.New("cache: negative capacity")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 4096
|
||||
}
|
||||
if cfg.MaxBytes == 0 {
|
||||
cfg.MaxBytes = 32 << 20
|
||||
}
|
||||
return &Memory{config: cfg, items: make(map[string]*list.Element), lru: list.New(), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (m *Memory) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
e := m.items[key]
|
||||
if e == nil {
|
||||
return nil, ErrMiss
|
||||
}
|
||||
item := e.Value.(*entry)
|
||||
if !item.expires.IsZero() && !m.now().Before(item.expires) {
|
||||
m.remove(e)
|
||||
return nil, ErrMiss
|
||||
}
|
||||
m.lru.MoveToFront(e)
|
||||
return append([]byte(nil), item.value...), nil
|
||||
}
|
||||
|
||||
func (m *Memory) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(value) > m.config.MaxBytes-len(key) {
|
||||
return ErrTooLarge
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
for len(m.items) >= m.config.MaxEntries || m.bytes+len(key)+len(value) > m.config.MaxBytes {
|
||||
m.remove(m.lru.Back())
|
||||
}
|
||||
item := &entry{key: key, value: append([]byte(nil), value...)}
|
||||
if ttl > 0 {
|
||||
item.expires = m.now().Add(ttl)
|
||||
}
|
||||
m.items[key] = m.lru.PushFront(item)
|
||||
m.bytes += len(key) + len(value)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, key := range keys {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
for _, key := range keys {
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) remove(e *list.Element) {
|
||||
item := e.Value.(*entry)
|
||||
m.bytes -= len(item.key) + len(item.value)
|
||||
delete(m.items, item.key)
|
||||
m.lru.Remove(e)
|
||||
}
|
||||
|
||||
func (m *Memory) Close() error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.closed = true
|
||||
clear(m.items)
|
||||
m.lru.Init()
|
||||
m.bytes = 0
|
||||
return nil
|
||||
}
|
||||
68
cache/redis/redis.go
vendored
Normal file
68
cache/redis/redis.go
vendored
Normal file
@@ -0,0 +1,68 @@
|
||||
// Package redis adapts go-redis's pooled client to cache.Store.
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
// Store borrows the client; its owner must close the client after users stop.
|
||||
type Store struct {
|
||||
client goredis.Cmdable
|
||||
prefix string
|
||||
}
|
||||
|
||||
func New(client goredis.Cmdable, prefix string) (*Store, error) {
|
||||
if client == nil || len(prefix) > 1024 || strings.ContainsAny(prefix, "\r\n") {
|
||||
return nil, errors.New("cache/redis: invalid client or prefix")
|
||||
}
|
||||
return &Store{client: client, prefix: prefix}, nil
|
||||
}
|
||||
|
||||
func (s *Store) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
value, err := s.client.Get(ctx, s.prefix+key).Bytes()
|
||||
if errors.Is(err, goredis.Nil) {
|
||||
return nil, cache.ErrMiss
|
||||
}
|
||||
return value, err
|
||||
}
|
||||
|
||||
func (s *Store) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := cache.Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
// Redis expiration precision is milliseconds; round up, never truncate a
|
||||
// positive duration to an unbounded lifetime.
|
||||
if ttl > 0 && ttl < time.Millisecond {
|
||||
ttl = time.Millisecond
|
||||
}
|
||||
return s.client.Set(ctx, s.prefix+key, value, ttl).Err()
|
||||
}
|
||||
|
||||
func (s *Store) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache/redis: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
names := make([]string, len(keys))
|
||||
for i, key := range keys {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
names[i] = s.prefix + key
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return nil
|
||||
}
|
||||
return s.client.Del(ctx, names...).Err()
|
||||
}
|
||||
125
cache/redis/redis_test.go
vendored
Normal file
125
cache/redis/redis_test.go
vendored
Normal file
@@ -0,0 +1,125 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
type fakeClient struct {
|
||||
goredis.Cmdable
|
||||
key string
|
||||
ttl time.Duration
|
||||
failure error
|
||||
}
|
||||
|
||||
func (f *fakeClient) Get(ctx context.Context, key string) *goredis.StringCmd {
|
||||
f.key = key
|
||||
return goredis.NewStringResult("", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Set(ctx context.Context, key string, value interface{}, ttl time.Duration) *goredis.StatusCmd {
|
||||
f.key, f.ttl = key, ttl
|
||||
return goredis.NewStatusResult("OK", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Del(ctx context.Context, keys ...string) *goredis.IntCmd {
|
||||
f.key = keys[0]
|
||||
return goredis.NewIntResult(1, f.failure)
|
||||
}
|
||||
|
||||
func TestAdapterPreservesErrorsAndExpiration(t *testing.T) {
|
||||
f := &fakeClient{failure: goredis.Nil}
|
||||
store, _ := New(f, "app:")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, cache.ErrMiss) || f.key != "app:key" {
|
||||
t.Fatal(err, f.key)
|
||||
}
|
||||
f.failure = errors.New("unavailable")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, f.failure) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f.failure = nil
|
||||
if err := store.Set(context.Background(), "key", []byte("value"), time.Nanosecond); err != nil || f.ttl != time.Millisecond {
|
||||
t.Fatal(err, f.ttl)
|
||||
}
|
||||
if err := store.Set(context.Background(), "key", nil, -time.Second); err == nil {
|
||||
t.Fatal("negative TTL accepted")
|
||||
}
|
||||
if err := store.Delete(context.Background(), "key"); err != nil || f.key != "app:key" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisIntegration(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("set GO_OMNI_REDIS_TEST_ADDR to an isolated test Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 2, MaxRetries: -1, ContextTimeoutEnabled: true, DialTimeout: time.Second, ReadTimeout: time.Second, WriteTimeout: time.Second})
|
||||
defer client.Close()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := client.Ping(ctx).Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
prefix := fmt.Sprintf("goOmniTest:%d:", time.Now().UnixNano())
|
||||
store, _ := New(client, prefix)
|
||||
defer store.Delete(context.Background(), "key", "forever", "list", "canceled")
|
||||
if err := store.Set(ctx, "key", []byte{0, 1, 255}, 80*time.Millisecond); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v, err := store.Get(ctx, "key")
|
||||
if err != nil || len(v) != 3 || v[2] != 255 {
|
||||
t.Fatal(v, err)
|
||||
}
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for {
|
||||
_, err := store.Get(ctx, "key")
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
break
|
||||
}
|
||||
if err != nil || time.Now().After(deadline) {
|
||||
t.Fatal("TTL failed", err)
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
if err := store.Set(ctx, "forever", []byte("ok"), 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ttl := client.PTTL(ctx, prefix+"forever").Val(); ttl != -1 {
|
||||
t.Fatalf("unexpected persistent TTL %v", ttl)
|
||||
}
|
||||
if err := store.Delete(ctx, "forever"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "forever"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.RPush(ctx, prefix+"list", "value").Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "list"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("Redis error treated as miss", err)
|
||||
}
|
||||
canceled, stop := context.WithCancel(ctx)
|
||||
stop()
|
||||
if err := store.Set(canceled, "canceled", []byte("value"), 0); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "canceled"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("canceled write reached Redis", err)
|
||||
}
|
||||
if err := store.Delete(ctx, "list"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "key"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("closed client treated as miss", err)
|
||||
}
|
||||
}
|
||||
@@ -1,11 +1,154 @@
|
||||
# go-omni CLI
|
||||
|
||||
生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
||||
## API 契约驱动项目
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni new --module example.com/billing ./billing
|
||||
# 创建包含可运行问候接口的新项目
|
||||
go run ./cmd/go-omni api new --module example.com/demo --framework-dir . ../demo
|
||||
|
||||
# 从已有定义生成对应目录、类型、路由和业务骨架
|
||||
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
|
||||
--dir ../userService --module example.com/userService --framework-dir .
|
||||
|
||||
# 校验定义;也接受 goctl 风格的 api go -api ... -dir ...
|
||||
go run ./cmd/go-omni api validate --api examples/contracts/user.api
|
||||
```
|
||||
|
||||
`new --api <file>` 是契约项目的另一个入口。可以组合 `--database mysql --storage local --log-output file`,生成配置、依赖初始化与资源关闭。API 默认无数据库、无对象存储;`--database memory` 等同于不打开连接。`--force` 不允许覆盖 API 项目。初始配置选择写入 manifest,重新生成自动沿用。
|
||||
|
||||
后续在生成项目内执行 `go-omni api generate --api api/user.api --dir .`。
|
||||
项目内可省略 `--api` / `--dir`。`api inspect` 展示路由和鉴权/业务错误,`api export` 向 stdout 导出 OpenAPI,`api generate --dry-run` 预览文件变化。`--check` 只读检查生成产物是否过期。生成器记录所有权,保留 Service / Repository / 配置 / main 的手写内容,遇到生成文件被手动修改则报冲突。新 Service 与中间件占位实现返回 501。支持 `error (...)`、具名 `auth`、路由 `@errors` / `@auth` 和多行 `@doc`;错误目录同步生成 `internal/apierrors` 调用函数。
|
||||
|
||||
支持范围、参数语义、目录及安全更新策略见 [API 开发指南](../../API.md)。本版借鉴 go-zero 的工作流,不宣称兼容全部 goctl 语法;仍通过 Omni/httpx 保留网络层替换能力。
|
||||
|
||||
## 日志与对象存储按需装配
|
||||
|
||||
新增 `--log-output stdout|stderr|file`、`--log-format json|text`、`--storage none|local|s3|r2|oss`。
|
||||
指定任意一个新选项后,生成严格的配置文件、初始化代码和项目自身的测试;不指定则保留原有轻量骨架。
|
||||
对象存储独立于 `--database memory|mysql`。只有选择文件存储才会生成文件 Handler / Service 和对应适配器。
|
||||
|
||||
开发阶段可以使用本地源码;固定远程版本的安装和验收见 [发布说明](../../RELEASE.md):
|
||||
|
||||
```sh
|
||||
# 在 go-omni 仓库中运行;目标目录须为空。
|
||||
go run ./cmd/go-omni new --framework-dir . \
|
||||
--module example.com/fileService --log-output file --log-format json \
|
||||
--storage local ../fileService
|
||||
```
|
||||
|
||||
生成项目包含 `etc/config.yaml`、`internal/settings`、`internal/servicecontext/storage.go`、
|
||||
文件 Service/Handler、测试和 `MODULES.md`。按生成的 MODULES.md 配置示例 Token 和本地签名密钥后,
|
||||
`go mod tidy && go run ./cmd/server` 即可体验。云存储分别选择 `--storage s3`、`r2`、`oss`,
|
||||
再设置已有桶/凭据等环境变量;不自动创建桶或修改云资源。
|
||||
|
||||
配置优先级为默认值 → YAML/JSON → 环境变量;密钥仅允许环境变量。Service 依赖统一存储接口,
|
||||
组合处负责创建客户端与有界连接池,退出时关闭自己创建的资源。文件接口提供服务端上传、直传签名、
|
||||
临时下载链接及统一错误处理;日志示例携带 requestId,文件输出默认带轮转。没有固定业务身份上下文。
|
||||
|
||||
新模板要求 `v0.7.0-preview` 或更新版本;明确指定旧版本并启用新模块会被拒绝,
|
||||
避免模板引用旧框架不存在的接口。本地 `--framework-dir` 验证不受此限制。`--force` 可重新生成兼容配置,
|
||||
但如果切换配置会遗留引用已移除依赖的旧文件,则拒绝操作,要求使用新目录;不会自动删除修改过的代码。
|
||||
|
||||
验证当前生成器:
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./cmd/go-omni/...
|
||||
python3 scripts/verifyScaffoldModules.py
|
||||
```
|
||||
|
||||
前者覆盖两种数据库与五种文件存储选择、日志方式/格式,以及生成项目的配置、资源关闭和接口测试。
|
||||
后者实际编译启动服务,验证日志、上传/下载/直传链接、错误状态、nethttp/native 下的 SIGINT/SIGTERM 和监听端口释放。
|
||||
云端选项的测试使用本地替身,不代表 OSS/R2 实网验收;本批不访问业务服务器。报告见
|
||||
[脚手架模块验收](../../performance/scaffoldModulesLocal.json)。
|
||||
|
||||
## 其他命令与远程安装
|
||||
|
||||
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
||||
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
||||
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
||||
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
||||
该命令从 `v0.7.0-preview` 提供;源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
||||
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
||||
|
||||
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview.1
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
通过 `go install ...@版本` 安装的 CLI 从构建信息中读取版本,并把它固定到生成项目的
|
||||
`go.mod`。不会使用不存在的 `v0.0.0`,也不会在普通业务目录中添加本地 `replace`。
|
||||
|
||||
需要选择其他已发布版本时,使用完整版本号:
|
||||
|
||||
```sh
|
||||
go-omni new --framework-version v0.6.0-preview --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
`--framework-version` 禁用自动本地替换;`main`、`latest` 等浮动引用不接受为固定版本。
|
||||
源码构建的 CLI 使用代码中声明的目标发布版本;该版本未发布时请显式选择已发布版本,或用 `--framework-dir` 验证本地源码。
|
||||
|
||||
在 GoOmni 源码目录执行时,命令会自动在生成项目的 `go.mod` 中加入本地
|
||||
`replace git.yuntuo.hk/k2safe/go-omni => ...`,便于立即运行和测试。目标目录非空时命令会停止,明确传入
|
||||
`--force` 才会覆盖脚手架文件。
|
||||
|
||||
也可以在任意目录显式指定源码路径:
|
||||
|
||||
```sh
|
||||
go-omni new --framework-dir /path/to/go-omni --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
默认生成内存存储版本;选择 MySQL 时:
|
||||
|
||||
```sh
|
||||
go-omni new --database mysql --module example.com/billing ./billing
|
||||
```
|
||||
|
||||
MySQL 项目额外包含 `schema.sql`、MySQL Repository 和可选的数据库集成测试。
|
||||
先应用建表脚本,再设置 `GO_OMNI_DB_DSN` 启动。ServiceContext 负责连接池初始化、表结构检查、
|
||||
就绪探针注册和关闭。两种模式都提供用户 CRUD、分页、输入校验和统一错误响应。
|
||||
|
||||
生成服务默认包含 `/health/live`、`/health/ready` 和示例 `/users` CRUD。
|
||||
退出信号通过 `signal.NotifyContext` 传入 `RunContext`,Logger 的关闭函数注册到应用生命周期。
|
||||
|
||||
发布验收使用 Python 3 标准库脚本,不需要额外 Python 依赖:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyScaffold.py --version v0.6.0-preview
|
||||
python3 scripts/verifyDatabase.py --version v0.6.0-preview
|
||||
```
|
||||
|
||||
脚本在源码目录外安装远程 CLI、生成项目,确认固定版本且没有 `replace`,再下载依赖、
|
||||
编译、请求接口,并对真实二进制分别发送 `SIGINT` 和 `SIGTERM`。同时验证框架 `Run` 的
|
||||
信号处理和清理回调。每个进程都必须退出码为 0、释放监听端口;临时项目和测试进程会被清理。
|
||||
|
||||
发布前验证当前源码:
|
||||
|
||||
```sh
|
||||
go build -o /tmp/go-omni ./cmd/go-omni
|
||||
python3 scripts/verifyScaffold.py --cli /tmp/go-omni --framework-dir .
|
||||
python3 scripts/verifyDatabase.py --cli /tmp/go-omni --framework-dir .
|
||||
```
|
||||
|
||||
数据库验收需要本地 Docker,使用独立 MySQL 8.4 容器;不会连接已有业务数据库,结束后清理测试容器与数据卷。
|
||||
|
||||
## 显式迁移(main)
|
||||
|
||||
SQL 命名为 `0001_createUsers.up.sql`。先预览版本和 SHA-256 清单:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations
|
||||
```
|
||||
|
||||
部署步骤配置 `GO_OMNI_DB_DSN`,显式执行:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations --apply
|
||||
```
|
||||
|
||||
预览不连接数据库;应用启动不自动迁移。失败保留 dirty 状态,不提供自动 down/force。
|
||||
脚手架原有 `schema.sql` 不会自动变成有版本的迁移,已有库需要先确定基线。详见 [扩展模块](../../ADVANCED.md)。
|
||||
|
||||
176
cmd/go-omni/api.go
Normal file
176
cmd/go-omni/api.go
Normal file
@@ -0,0 +1,176 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
"text/tabwriter"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||
)
|
||||
|
||||
func runAPI(args []string) error {
|
||||
if len(args) == 0 || args[0] == "help" || args[0] == "--help" || args[0] == "-h" {
|
||||
usage()
|
||||
return nil
|
||||
}
|
||||
command := args[0]
|
||||
if command == "go" {
|
||||
command = "generate"
|
||||
}
|
||||
if command == "check" {
|
||||
command = "validate"
|
||||
}
|
||||
if command != "new" && command != "generate" && command != "validate" && command != "inspect" && command != "export" {
|
||||
return fmt.Errorf("unknown api command %q", command)
|
||||
}
|
||||
readOnly := command == "validate" || command == "inspect" || command == "export"
|
||||
o := apigen.Options{}
|
||||
for i := 1; i < len(args); i++ {
|
||||
arg := args[i]
|
||||
if arg == "--help" || arg == "-h" {
|
||||
usage()
|
||||
return nil
|
||||
}
|
||||
if arg == "--check" || arg == "--dry-run" {
|
||||
if command != "generate" {
|
||||
return fmt.Errorf("%s belongs to api generate", arg)
|
||||
}
|
||||
if arg == "--check" {
|
||||
o.Check = true
|
||||
} else {
|
||||
o.DryRun = true
|
||||
}
|
||||
continue
|
||||
}
|
||||
if !strings.HasPrefix(arg, "-") {
|
||||
if command == "new" && o.Dir == "" {
|
||||
o.Dir = arg
|
||||
continue
|
||||
}
|
||||
return fmt.Errorf("unexpected argument %q", arg)
|
||||
}
|
||||
name, value, inline := strings.Cut(arg, "=")
|
||||
// goctl-style -api/-dir aliases ease migration; long forms are canonical.
|
||||
if name == "-api" {
|
||||
name = "--api"
|
||||
}
|
||||
if name == "-dir" {
|
||||
name = "--dir"
|
||||
}
|
||||
switch name {
|
||||
case "--api", "--dir", "--module", "--framework-dir", "--framework-version", "--log-output", "--log-format", "--database", "--storage":
|
||||
default:
|
||||
return fmt.Errorf("unknown api option %q", name)
|
||||
}
|
||||
if !inline {
|
||||
if i+1 >= len(args) || strings.HasPrefix(args[i+1], "-") {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
i++
|
||||
value = args[i]
|
||||
}
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
if readOnly && name != "--api" {
|
||||
return fmt.Errorf("api %s does not accept %s", command, name)
|
||||
}
|
||||
switch name {
|
||||
case "--api":
|
||||
o.API = value
|
||||
case "--dir":
|
||||
o.Dir = value
|
||||
case "--module":
|
||||
o.Module = value
|
||||
case "--framework-dir":
|
||||
o.FrameworkDir = value
|
||||
case "--framework-version":
|
||||
o.FrameworkVersion = value
|
||||
case "--database":
|
||||
o.Database = value
|
||||
case "--storage":
|
||||
o.Storage = value
|
||||
case "--log-output":
|
||||
o.LogOutput = value
|
||||
case "--log-format":
|
||||
o.LogFormat = value
|
||||
}
|
||||
}
|
||||
if command != "new" && o.API == "" {
|
||||
dir := o.Dir
|
||||
if dir == "" {
|
||||
dir = "."
|
||||
}
|
||||
var err error
|
||||
o.API, err = apigen.ProjectEntry(dir)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if readOnly {
|
||||
spec, err := apigen.Parse(o.API)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
compiled, err := apigen.Compile(spec)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
switch command {
|
||||
case "validate":
|
||||
fmt.Printf("valid API: %d types, %d routes, %d errors, %d auth schemes\n", len(spec.Types), len(spec.Routes), len(spec.Errors), len(spec.Auth))
|
||||
case "export":
|
||||
fmt.Println(string(compiled.Document))
|
||||
case "inspect":
|
||||
writer := tabwriter.NewWriter(os.Stdout, 0, 4, 2, ' ', 0)
|
||||
fmt.Fprintln(writer, "METHOD\tPATH\tHANDLER\tAUTH\tRESPONSE\tERRORS")
|
||||
definitions := map[string]apigen.ErrorDefinition{}
|
||||
for _, e := range spec.Errors {
|
||||
definitions[e.Name] = e
|
||||
}
|
||||
for _, r := range compiled.Spec.Routes {
|
||||
auth := r.Auth
|
||||
if auth == "" {
|
||||
auth = "(undeclared)"
|
||||
}
|
||||
var errs []string
|
||||
for _, name := range r.Errors {
|
||||
e := definitions[name]
|
||||
errs = append(errs, fmt.Sprintf("%d:%s", e.Status, e.Code))
|
||||
}
|
||||
format := r.ResponseFormat
|
||||
if format == "" {
|
||||
format = "default"
|
||||
}
|
||||
fmt.Fprintf(writer, "%s\t%s\t%s/%s\t%s\t%s\t%s\n", strings.ToUpper(r.Method), r.Path, r.Group, r.Handler, auth, format, strings.Join(errs, ", "))
|
||||
}
|
||||
return writer.Flush()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
var out apigen.Output
|
||||
var err error
|
||||
if command == "new" {
|
||||
out, err = apigen.New(o)
|
||||
} else {
|
||||
if o.Dir == "" {
|
||||
o.Dir = "."
|
||||
}
|
||||
out, err = apigen.Generate(o)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if o.DryRun {
|
||||
for _, change := range out.Changes {
|
||||
fmt.Printf("%-7s %s\n", change.Action, change.Path)
|
||||
}
|
||||
fmt.Printf("Preview: %d changes; %d existing developer-owned files preserved; no files written\n", len(out.Changes), len(out.Preserved))
|
||||
} else if o.Check {
|
||||
fmt.Println("API generated files are current")
|
||||
} else {
|
||||
fmt.Printf("API project ready: %d generated, %d created, %d preserved, %d removed\n", len(out.Generated), len(out.Created), len(out.Preserved), len(out.Removed))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
107
cmd/go-omni/api_test.go
Normal file
107
cmd/go-omni/api_test.go
Normal file
@@ -0,0 +1,107 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestAPICommands(t *testing.T) {
|
||||
root, err := filepath.Abs("../..")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
project := filepath.Join(t.TempDir(), "demo")
|
||||
if err = runAPI([]string{"new", project, "--module=example.com/demo", "--framework-dir", root}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
api := filepath.Join(project, "api/service.api")
|
||||
if err = runAPI([]string{"validate", "-api", api}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runAPI([]string{"go", "-api", api, "-dir", project, "--check"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runNew([]string{"--force", project}); err == nil {
|
||||
t.Fatal("CRUD starter overwrote API project")
|
||||
}
|
||||
target := filepath.Join(t.TempDir(), "other")
|
||||
if err = runNew([]string{"--api", api, "--module", "example.com/other", "--framework-dir", root, target}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(target, "internal/service/greet/greet.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, args := range [][]string{{"validate"}, {"generate", "--api"}, {"new", "--api=x", "somewhere"}, {"generate", "--dir="}, {"generate", "--force"}, {"validate", "--api", api, "--dir", project}, {"validate", "--check"}} {
|
||||
if err = runAPI(args); err == nil {
|
||||
t.Fatalf("invalid args accepted: %v", args)
|
||||
}
|
||||
}
|
||||
if err = runNew([]string{"--api", api, "--force", target}); err == nil || !strings.Contains(err.Error(), "does not accept --force") {
|
||||
t.Fatalf("incompatible profile: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIDeveloperCommands(t *testing.T) {
|
||||
root, err := filepath.Abs("../..")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
if err = runAPI([]string{"new", dir, "--framework-dir", root, "--module", "example.com/commands"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Generation can resolve the entry from an explicit target directory.
|
||||
if err = runAPI([]string{"generate", "--dir", dir, "--dry-run"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wd, err := os.Getwd()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = os.Chdir(dir); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer os.Chdir(wd)
|
||||
for _, args := range [][]string{{"validate"}, {"inspect"}, {"generate", "--check"}} {
|
||||
if err = runAPI(args); err != nil {
|
||||
t.Fatalf("%v: %v", args, err)
|
||||
}
|
||||
}
|
||||
doc := captureAPIOutput(t, []string{"export"})
|
||||
var parsed map[string]any
|
||||
if err = json.Unmarshal([]byte(doc), &parsed); err != nil {
|
||||
t.Fatalf("export must be pure JSON: %v", err)
|
||||
}
|
||||
if parsed["openapi"] != "3.0.3" {
|
||||
t.Fatal("wrong export")
|
||||
}
|
||||
for _, args := range [][]string{{"generate", "--dry-run", "--check"}, {"new", "--dry-run", "other"}, {"inspect", "--dir", dir}, {"export", "--log-output", "file"}} {
|
||||
if err = runAPI(args); err == nil {
|
||||
t.Fatalf("accepted invalid flags: %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
func captureAPIOutput(t *testing.T, args []string) string {
|
||||
t.Helper()
|
||||
r, w, err := os.Pipe()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
previous := os.Stdout
|
||||
os.Stdout = w
|
||||
defer func() { os.Stdout = previous; r.Close(); w.Close() }()
|
||||
result := make(chan []byte, 1)
|
||||
go func() { data, _ := io.ReadAll(r); result <- data }()
|
||||
callErr := runAPI(args)
|
||||
w.Close()
|
||||
os.Stdout = previous
|
||||
data := <-result
|
||||
if callErr != nil {
|
||||
t.Fatal(callErr)
|
||||
}
|
||||
return string(data)
|
||||
}
|
||||
360
cmd/go-omni/apigen/apigen_test.go
Normal file
360
cmd/go-omni/apigen/apigen_test.go
Normal file
@@ -0,0 +1,360 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func frameworkRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
p, err := filepath.Abs("../../..")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return p
|
||||
}
|
||||
func writeFile(t *testing.T, name, body string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Dir(name), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(name, []byte(body), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func readFile(t *testing.T, name string) string {
|
||||
t.Helper()
|
||||
b, err := os.ReadFile(name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
func parseText(t *testing.T, source string) (*Spec, error) {
|
||||
t.Helper()
|
||||
file := filepath.Join(t.TempDir(), "service.api")
|
||||
writeFile(t, file, source)
|
||||
return Parse(file)
|
||||
}
|
||||
|
||||
func TestCompileExample(t *testing.T) {
|
||||
s, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/user.api"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(s)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(c.Groups) != 2 || len(s.Routes) != 3 || len(s.Sources) != 2 {
|
||||
t.Fatalf("wrong spec: %+v", s)
|
||||
}
|
||||
var doc map[string]any
|
||||
if err = json.Unmarshal(c.Document, &doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
route := doc["paths"].(map[string]any)["/api/v1/users/{userId}"].(map[string]any)["put"].(map[string]any)
|
||||
if len(route["parameters"].([]any)) != 3 {
|
||||
t.Fatal("missing mixed parameters")
|
||||
}
|
||||
schema := route["requestBody"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
|
||||
if schema["additionalProperties"] != false || len(schema["required"].([]any)) != 2 {
|
||||
t.Fatalf("wrong body schema: %+v", schema)
|
||||
}
|
||||
if strings.Contains(string(c.Document), `"user_id"`) {
|
||||
t.Fatal("snake case wire name")
|
||||
}
|
||||
}
|
||||
func TestInvalidDefinitions(t *testing.T) {
|
||||
cases := []string{
|
||||
`syntax="v2"`,
|
||||
`@server ( jwt: Auth ) service user { @handler go get /go }`,
|
||||
"type Request { Name string `json:\"name\" binding:\"typo\"` }\nservice user { @handler run\n post /go (Request) }",
|
||||
"type A { Other *A `json:\"other,optional\"` }\nservice user { @handler run\n post /go (A) }",
|
||||
"type A { Other Missing `json:\"other\"` }\nservice user { @handler run\n post /go (A) }",
|
||||
"type A { UserID int `json:\"user_id\"` }\nservice user { @handler run\n post /go (A) }",
|
||||
"service user { @handler go\n get /go\n @handler other\n get /go\n}",
|
||||
"service user { @handler go\n get /:id\n}",
|
||||
"service user { @handler service\n get /go\n}",
|
||||
"type A { ID int `path:\"id\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||
"type A { Name string `json:\"name\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||
"service user { @handler go\n @status 201\n post /go\n}",
|
||||
"type A { Name string `header:\"Name\"` }\nservice user { @handler go\n get /go returns (A)\n}",
|
||||
"@server (group: ../../bad)\nservice user { @handler go\n get /go\n}",
|
||||
"service user { @handler go\n get /health/ready\n}",
|
||||
"type A { Name string `json:\"name\"`\n Name int `json:\"other\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||
"service user { @handler go\n get /go\n @handler Go\n get /other\n}",
|
||||
"type A { Value []byte `json:\"value\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||
}
|
||||
for i, source := range cases {
|
||||
s, err := parseText(t, source)
|
||||
if err == nil {
|
||||
_, err = Compile(s)
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatalf("case %d accepted: %s", i, source)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestImports(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
entry := filepath.Join(root, "api", "entry.api")
|
||||
writeFile(t, entry, "import \"types.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||
writeFile(t, filepath.Join(root, "api/types.api"), `import "entry.api"`)
|
||||
if _, err := Parse(entry); err == nil || !strings.Contains(err.Error(), "circular") {
|
||||
t.Fatalf("cycle: %v", err)
|
||||
}
|
||||
writeFile(t, entry, "import \"../outside.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||
writeFile(t, filepath.Join(root, "outside.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||
if _, err := Parse(entry); err == nil {
|
||||
t.Fatal("escaped import root")
|
||||
}
|
||||
writeFile(t, entry, "import (\n \"shared.api\"\n \"./shared.api\"\n)\nservice user { @handler ping\n get /ping\n}")
|
||||
writeFile(t, filepath.Join(root, "api/shared.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||
s, err := Parse(entry)
|
||||
if err != nil || len(s.Types) != 1 {
|
||||
t.Fatalf("duplicate import: %+v %v", s, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSharedPrefixConflict(t *testing.T) {
|
||||
source := "type A { ID int `path:\"id\"` }\ntype B { Name string `path:\"name\"` }\nservice user {\n @handler first\n get /users/:id (A)\n @handler second\n get /users/:name/items (B)\n}"
|
||||
s, err := parseText(t, source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = Compile(s); err == nil {
|
||||
t.Fatal("shared prefix conflict was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func FuzzParser(f *testing.F) {
|
||||
for _, s := range []string{starterAPI, declaredAPI, "error Broken", "auth Auth (type: bearer)", "syntax = \"v1\"", "type (", "@server (prefix: /api)"} {
|
||||
f.Add(s)
|
||||
}
|
||||
f.Fuzz(func(t *testing.T, source string) {
|
||||
if len(source) > 16384 {
|
||||
t.Skip()
|
||||
}
|
||||
tokens, err := lex("fuzz.api", source)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
spec := &Spec{Title: "Fuzz", Version: "1"}
|
||||
p := parser{tokens: tokens, spec: spec}
|
||||
if p.parse() == nil {
|
||||
_, _ = Compile(spec)
|
||||
}
|
||||
})
|
||||
}
|
||||
func TestRegenerationAndConflicts(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
manifestBefore := readFile(t, filepath.Join(dir, manifestPath))
|
||||
typesBefore := readFile(t, filepath.Join(dir, "internal/types/typesGen.go"))
|
||||
o.Check = true
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
servicePath := filepath.Join(dir, "internal/service/greet/greet.go")
|
||||
service := readFile(t, servicePath) + "\n// user edit must survive\n"
|
||||
writeFile(t, servicePath, service)
|
||||
for _, file := range []string{"cmd/server/main.go", "etc/config.yaml", "internal/servicecontext/context.go", "internal/repository/greet/repository.go"} {
|
||||
body := readFile(t, filepath.Join(dir, file))
|
||||
if strings.HasSuffix(file, ".go") {
|
||||
body += "\n// handwritten extension\n"
|
||||
} else {
|
||||
body += "\n# handwritten extension\n"
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, file), body)
|
||||
}
|
||||
api := readFile(t, o.API)
|
||||
api += "\n@server (group: extra)\nservice greet-api {\n @handler ping\n get /ping\n}\n"
|
||||
writeFile(t, o.API, api)
|
||||
if _, err := Generate(o); err == nil {
|
||||
t.Fatal("check accepted stale output")
|
||||
}
|
||||
if readFile(t, filepath.Join(dir, manifestPath)) != manifestBefore {
|
||||
t.Fatal("--check wrote manifest")
|
||||
}
|
||||
o.Check = false
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if readFile(t, servicePath) != service {
|
||||
t.Fatal("business code overwritten")
|
||||
}
|
||||
// Retiring a group removes only owned handlers, keeping its business files.
|
||||
writeFile(t, o.API, strings.Split(api, "\n@server (group: extra)")[0])
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/handler/extra/handlersGen.go")); !os.IsNotExist(err) {
|
||||
t.Fatal("stale handler remains")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/service/extra/ping.go")); err != nil {
|
||||
t.Fatal("business stub removed")
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), typesBefore+"\n// edited generated file\n")
|
||||
before := snapshot(t, dir)
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "was edited") {
|
||||
t.Fatalf("did not protect edited output: %v", err)
|
||||
}
|
||||
after := snapshot(t, dir)
|
||||
if !sameFiles(before, after) {
|
||||
t.Fatal("failed regeneration changed files")
|
||||
}
|
||||
}
|
||||
func TestOutputBoundaries(t *testing.T) {
|
||||
for _, kind := range []string{"symlink", "unowned", "lock", "caseCollision"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
dir := filepath.Join(root, "app")
|
||||
api := filepath.Join(root, "input/service.api")
|
||||
writeFile(t, api, starterAPI)
|
||||
switch kind {
|
||||
case "symlink":
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
outside := filepath.Join(root, "outside")
|
||||
os.MkdirAll(outside, 0755)
|
||||
if err := os.Symlink(outside, filepath.Join(dir, "internal")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "unowned":
|
||||
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), "package types // mine\n")
|
||||
case "lock":
|
||||
writeFile(t, filepath.Join(dir, ".goOmni/api.lock"), "someone else")
|
||||
case "caseCollision":
|
||||
writeFile(t, api, starterAPI+"\n@server (group: gReet)\nservice greet-api {\n @handler extra\n get /extra\n}\n")
|
||||
}
|
||||
before := snapshot(t, root)
|
||||
if _, err := Generate(Options{Dir: dir, API: api, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err == nil {
|
||||
t.Fatal("unsafe output accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("conflict changed files")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func snapshot(t *testing.T, root string) map[string]string {
|
||||
t.Helper()
|
||||
out := map[string]string{}
|
||||
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.Type().IsRegular() {
|
||||
data, e := os.ReadFile(path)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
rel, _ := filepath.Rel(root, path)
|
||||
out[rel] = string(data)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
func sameFiles(a, b map[string]string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for k, v := range a {
|
||||
if b[k] != v {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func TestGeneratedProjectsBuildAndRun(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("consumer module integration")
|
||||
}
|
||||
root := frameworkRoot(t)
|
||||
for _, starter := range []bool{true, false} {
|
||||
name := "contract"
|
||||
if starter {
|
||||
name = "starter"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
o := Options{Dir: dir, Module: "example.com/consumer", FrameworkDir: root, LogOutput: "file", LogFormat: "text"}
|
||||
var err error
|
||||
if starter {
|
||||
_, err = New(o)
|
||||
} else {
|
||||
o.API = filepath.Join(root, "examples/contracts/user.api")
|
||||
_, err = Generate(o)
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
source := consumerContractTest
|
||||
if starter {
|
||||
source = consumerStarterTest
|
||||
} else {
|
||||
writeFile(t, filepath.Join(dir, "internal/service/user/updateUser.go"), `package user
|
||||
import("context";"example.com/consumer/internal/types")
|
||||
func(s *Service) UpdateUser(ctx context.Context,req *types.UpdateUserReq)(*types.UserResp,error){return &types.UserResp{UserID:req.UserID,Name:req.Name,Active:req.Active},nil}
|
||||
`)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "contract_test.go"), source)
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
if !starter {
|
||||
o.API = filepath.Join(dir, "api/user.api")
|
||||
}
|
||||
if _, err = Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("consumer: %v\n%s", err, output)
|
||||
}
|
||||
t.Log(string(output))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const consumerStarterTest = `package consumer_test
|
||||
import("context";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext";"example.com/consumer/internal/settings")
|
||||
func TestGreeting(t *testing.T){cfg,err:=settings.Load("etc/config.yaml");if err!=nil{t.Fatal(err)};cfg.Server.Log.Writer=io.Discard;cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{};logger,err:=logx.New(cfg.Server.Log);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)};w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,httptest.NewRequest("GET","/api/greet/Alice",nil));if w.Code!=200||!strings.Contains(w.Body.String(),"Hello Alice")||w.Header().Get("X-Request-ID")==""{t.Fatalf("%d %s",w.Code,w.Body.String())}}
|
||||
`
|
||||
const consumerContractTest = `package consumer_test
|
||||
import("context";"encoding/json";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext")
|
||||
func TestContract(t *testing.T){cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
|
||||
for _,tc:=range []struct{method,path,body string;status int}{
|
||||
{"PUT","/api/v1/users/7?notify=true",` + "`" + `{"name":"Alice","active":false,"age":0}` + "`" + `,200},
|
||||
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice"}` + "`" + `,400},
|
||||
{"PUT","/api/v1/users/7?notify=true¬ify=false",` + "`" + `{"name":"Alice","active":false}` + "`" + `,400},
|
||||
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice","active":false,"age":151}` + "`" + `,400},
|
||||
{"GET","/api/v1/users?page=0","",400},
|
||||
{"GET","/api/v1/users?page=1","",501},
|
||||
{"DELETE","/admin/v1/users/7","",501},
|
||||
}{req:=httptest.NewRequest(tc.method,tc.path,strings.NewReader(tc.body));req.Header.Set("Content-Type","application/json");w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,req);if w.Code!=tc.status{t.Fatalf("%s: %d %s",tc.path,w.Code,w.Body.String())};if tc.status==200{var v struct{Data struct{UserID int64 ` + "`json:\"userId\"`" + `;Name string ` + "`json:\"name\"`" + `}};if err:=json.Unmarshal(w.Body.Bytes(),&v);err!=nil||v.Data.UserID!=7||v.Data.Name!="Alice"{t.Fatal(w.Body.String())}}}
|
||||
delete(deps.Middlewares,"Auth");if err:=handler.RegisterRoutes(omni.New(),deps);err==nil{t.Fatal("missing auth hook accepted")}
|
||||
}
|
||||
`
|
||||
336
cmd/go-omni/apigen/compile.go
Normal file
336
cmd/go-omni/apigen/compile.go
Normal file
@@ -0,0 +1,336 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/ast"
|
||||
goparser "go/parser"
|
||||
gotoken "go/token"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
)
|
||||
|
||||
type Compiled struct {
|
||||
Spec *Spec
|
||||
Bindings map[string]*contract.Binder
|
||||
Document []byte
|
||||
Groups []string
|
||||
}
|
||||
|
||||
var exportedName = regexp.MustCompile(`^[A-Z][A-Za-z0-9]*$`)
|
||||
var goName = regexp.MustCompile(`^[a-zA-Z][A-Za-z0-9]*$`)
|
||||
var lowerName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||
var serviceName = regexp.MustCompile(`^[a-zA-Z][a-zA-Z0-9-]*$`)
|
||||
var staticSegment = regexp.MustCompile(`^[A-Za-z0-9._~-]+$`)
|
||||
|
||||
func Compile(spec *Spec) (*Compiled, error) {
|
||||
if spec == nil || len(spec.Routes) == 0 || !serviceName.MatchString(spec.Service) {
|
||||
return nil, errors.New("api: valid service and routes required")
|
||||
}
|
||||
var prepareErr error
|
||||
spec, prepareErr = prepareMetadata(spec)
|
||||
if prepareErr != nil {
|
||||
return nil, prepareErr
|
||||
}
|
||||
definitions := map[string]Type{}
|
||||
for _, t := range spec.Types {
|
||||
if !exportedName.MatchString(t.Name) {
|
||||
return nil, fmt.Errorf("%s: type names must be UpperCamelCase", t.Position)
|
||||
}
|
||||
if _, ok := definitions[t.Name]; ok {
|
||||
return nil, fmt.Errorf("%s: duplicate type %s", t.Position, t.Name)
|
||||
}
|
||||
definitions[t.Name] = t
|
||||
}
|
||||
resolved := map[string]reflect.Type{}
|
||||
resolving := map[string]bool{}
|
||||
var resolve func(string, int) (reflect.Type, error)
|
||||
var expression func(ast.Expr, int) (reflect.Type, error)
|
||||
primitives := map[string]reflect.Type{"string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
|
||||
if depth > 32 {
|
||||
return nil, errors.New("type nesting exceeds 32")
|
||||
}
|
||||
switch e := e.(type) {
|
||||
case *ast.Ident:
|
||||
if t, ok := primitives[e.Name]; ok {
|
||||
return t, nil
|
||||
}
|
||||
return resolve(e.Name, depth+1)
|
||||
case *ast.StarExpr:
|
||||
if _, double := e.X.(*ast.StarExpr); double {
|
||||
return nil, errors.New("multiple pointer levels unsupported")
|
||||
}
|
||||
t, err := expression(e.X, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reflect.PointerTo(t), nil
|
||||
case *ast.ArrayType:
|
||||
if e.Len != nil {
|
||||
return nil, errors.New("use slices, not fixed arrays")
|
||||
}
|
||||
t, err := expression(e.Elt, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reflect.SliceOf(t), nil
|
||||
case *ast.MapType:
|
||||
key, ok := e.Key.(*ast.Ident)
|
||||
if !ok || key.Name != "string" {
|
||||
return nil, errors.New("map keys must be string")
|
||||
}
|
||||
t, err := expression(e.Value, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reflect.MapOf(reflect.TypeOf(""), t), nil
|
||||
default:
|
||||
return nil, errors.New("unsupported type expression")
|
||||
}
|
||||
}
|
||||
resolve = func(name string, depth int) (reflect.Type, error) {
|
||||
if depth > 32 {
|
||||
return nil, errors.New("type nesting exceeds 32")
|
||||
}
|
||||
if t, ok := resolved[name]; ok {
|
||||
return t, nil
|
||||
}
|
||||
if resolving[name] {
|
||||
return nil, fmt.Errorf("recursive type %s is unsupported", name)
|
||||
}
|
||||
typ, ok := definitions[name]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown type %s", name)
|
||||
}
|
||||
resolving[name] = true
|
||||
defer delete(resolving, name)
|
||||
var fields []reflect.StructField
|
||||
seen := map[string]bool{}
|
||||
for _, f := range typ.Fields {
|
||||
if !exportedName.MatchString(f.Name) || seen[f.Name] {
|
||||
return nil, fmt.Errorf("%s: invalid or duplicate field %s", name, f.Name)
|
||||
}
|
||||
seen[f.Name] = true
|
||||
expr, err := goparser.ParseExpr(f.Type)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
t, err := expression(expr, depth+1)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err)
|
||||
}
|
||||
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)})
|
||||
}
|
||||
t := reflect.StructOf(fields)
|
||||
resolved[name] = t
|
||||
return t, nil
|
||||
}
|
||||
c := &Compiled{Spec: spec, Bindings: map[string]*contract.Binder{}}
|
||||
for _, t := range spec.Types {
|
||||
rt, err := resolve(t.Name, 0)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b, err := contract.ForType(rt)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", t.Position, err)
|
||||
}
|
||||
c.Bindings[t.Name] = b
|
||||
}
|
||||
api, err := openapi.New(spec.Title, spec.Version)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{}
|
||||
middlewareSymbols := map[string]string{}
|
||||
for _, r := range spec.Routes {
|
||||
if !goName.MatchString(r.Handler) {
|
||||
return nil, fmt.Errorf("%s: invalid handler name", r.Position)
|
||||
}
|
||||
if strings.EqualFold(r.Handler, "service") {
|
||||
return nil, errors.New("handler name service conflicts with the service constructor file")
|
||||
}
|
||||
if r.Group == "" {
|
||||
return nil, errors.New("group required")
|
||||
}
|
||||
for _, part := range strings.Split(r.Group, "/") {
|
||||
if !lowerName.MatchString(part) || gotoken.Lookup(part).IsKeyword() {
|
||||
return nil, fmt.Errorf("invalid lowerCamelCase group %q", r.Group)
|
||||
}
|
||||
}
|
||||
h := r.Group + "/" + strings.ToLower(r.Handler)
|
||||
if handlers[h] {
|
||||
return nil, fmt.Errorf("duplicate handler %s in group %s", r.Handler, r.Group)
|
||||
}
|
||||
handlers[h] = true
|
||||
groups[r.Group] = true
|
||||
mseen := map[string]bool{}
|
||||
for _, m := range r.Middleware {
|
||||
if !goName.MatchString(m) || mseen[m] {
|
||||
return nil, fmt.Errorf("invalid or duplicate middleware %q", m)
|
||||
}
|
||||
mseen[m] = true
|
||||
if strings.EqualFold(m, "defaults") || strings.EqualFold(m, "registryGen") {
|
||||
return nil, fmt.Errorf("reserved middleware name %q", m)
|
||||
}
|
||||
if prior, ok := middlewareSymbols[strings.ToLower(m)]; ok && prior != m {
|
||||
return nil, fmt.Errorf("middleware names %s and %s collide", prior, m)
|
||||
}
|
||||
middlewareSymbols[strings.ToLower(m)] = m
|
||||
}
|
||||
shape, err := routeShape(r.Path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||
}
|
||||
for _, prior := range spec.Routes {
|
||||
if prior.Method != r.Method || prior.Path == r.Path {
|
||||
continue
|
||||
}
|
||||
if conflictPrefix(prior.Path, r.Path) {
|
||||
return nil, fmt.Errorf("conflicting parameter names at a shared route prefix: %s and %s", prior.Path, r.Path)
|
||||
}
|
||||
}
|
||||
if prior, exists := shapes[shape]; exists && prior != r.Path {
|
||||
return nil, fmt.Errorf("ambiguous route parameter names: %s and %s", prior, r.Path)
|
||||
}
|
||||
shapes[shape] = r.Path
|
||||
pathKey := r.Method + " " + shape
|
||||
if paths[pathKey] {
|
||||
return nil, fmt.Errorf("duplicate route %s", pathKey)
|
||||
}
|
||||
paths[pathKey] = true
|
||||
if r.Path == "/health/live" || r.Path == "/health/ready" || r.Path == "/openapi.json" {
|
||||
return nil, fmt.Errorf("route %s is reserved by the generated server", r.Path)
|
||||
}
|
||||
if r.Method != strings.ToLower(r.Method) {
|
||||
return nil, errors.New("HTTP methods must be lowercase")
|
||||
}
|
||||
if r.Status != 200 && r.Status != 201 && r.Status != 202 && r.Status != 204 {
|
||||
return nil, errors.New("@status supports 200, 201, 202 or 204")
|
||||
}
|
||||
if (r.Status == 204) != (r.Response == "") {
|
||||
return nil, errors.New("204 requires no response; other success statuses require a response type")
|
||||
}
|
||||
if r.Method == "head" && r.Response != "" {
|
||||
return nil, errors.New("HEAD must not declare a response body")
|
||||
}
|
||||
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
|
||||
if r.Request != "" {
|
||||
b, ok := c.Bindings[r.Request]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown request %s", r.Request)
|
||||
}
|
||||
op.Parameters = b.Parameters()
|
||||
op.Request = b.BodySchema()
|
||||
if op.Request != nil && (r.Method == "get" || r.Method == "head") {
|
||||
return nil, errors.New("GET/HEAD requests cannot declare JSON bodies; use form/path/header tags")
|
||||
}
|
||||
}
|
||||
var response *openapi.Schema
|
||||
if r.Response != "" {
|
||||
b, ok := c.Bindings[r.Response]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown response %s", r.Response)
|
||||
}
|
||||
s, err := b.ResponseSchema()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if r.ResponseFormat == "raw" {
|
||||
response = s
|
||||
} else {
|
||||
response, err = openapi.EnvelopeWith(s, responseConfig(spec, r))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
}
|
||||
op.Responses[r.Status] = openapi.Response{Description: "Success", Schema: response}
|
||||
errorSchema, err := openapi.EnvelopeWith(nil, responseConfig(spec, r))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, code := range []int{400, 413, 415, 500, 501} {
|
||||
op.Responses[code] = openapi.Response{Description: "Error", Schema: errorSchema}
|
||||
}
|
||||
if err = api.Handle(discardRoutes{}, op, func(*httpx.Context) error { return nil }); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||
}
|
||||
}
|
||||
for group := range groups {
|
||||
c.Groups = append(c.Groups, group)
|
||||
}
|
||||
sort.Strings(c.Groups)
|
||||
c.Document, err = api.JSON()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Decorate the document with explicit auth/error contracts. Credential
|
||||
// validation remains in the linked application middleware.
|
||||
var doc map[string]any
|
||||
decoder := json.NewDecoder(bytes.NewReader(c.Document))
|
||||
decoder.UseNumber()
|
||||
if err = decoder.Decode(&doc); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
decorateDocument(doc, spec)
|
||||
c.Document, err = json.MarshalIndent(doc, "", " ")
|
||||
return c, err
|
||||
}
|
||||
|
||||
type discardRoutes struct{}
|
||||
|
||||
func (discardRoutes) Handle(string, string, httpx.Handler, ...httpx.Middleware) {}
|
||||
func operationID(r Route) string {
|
||||
return strings.ReplaceAll(r.Group, "/", "_") + "_" + upper(r.Handler)
|
||||
}
|
||||
func upper(s string) string { return strings.ToUpper(s[:1]) + s[1:] }
|
||||
func lower(s string) string { return strings.ToLower(s[:1]) + s[1:] }
|
||||
func routeShape(path string) (string, error) {
|
||||
if path == "/" {
|
||||
return path, nil
|
||||
}
|
||||
if !strings.HasPrefix(path, "/") {
|
||||
return "", errors.New("route must start with /")
|
||||
}
|
||||
parts := strings.Split(path[1:], "/")
|
||||
seen := map[string]bool{}
|
||||
for i, part := range parts {
|
||||
if strings.HasPrefix(part, ":") {
|
||||
name := part[1:]
|
||||
if !lowerName.MatchString(name) || seen[name] {
|
||||
return "", errors.New("invalid/duplicate path parameter")
|
||||
}
|
||||
seen[name] = true
|
||||
parts[i] = ":"
|
||||
} else if !staticSegment.MatchString(part) || part == "." || part == ".." {
|
||||
return "", errors.New("invalid route segment")
|
||||
}
|
||||
}
|
||||
return "/" + strings.Join(parts, "/"), nil
|
||||
}
|
||||
|
||||
func conflictPrefix(left, right string) bool {
|
||||
a, b := strings.Split(left, "/"), strings.Split(right, "/")
|
||||
for i := 0; i < len(a) && i < len(b); i++ {
|
||||
if strings.HasPrefix(a[i], ":") && strings.HasPrefix(b[i], ":") {
|
||||
if a[i] != b[i] {
|
||||
return true
|
||||
}
|
||||
continue
|
||||
}
|
||||
if a[i] != b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
220
cmd/go-omni/apigen/contractMetadata.go
Normal file
220
cmd/go-omni/apigen/contractMetadata.go
Normal file
@@ -0,0 +1,220 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
var errorCode = regexp.MustCompile(`^[A-Z][A-Z0-9_]*$`)
|
||||
var credentialName = regexp.MustCompile("^[!#$%&'*+.^_`|~0-9A-Za-z-]+$")
|
||||
|
||||
// prepareMetadata returns a detached spec; repeated compilation must not prepend
|
||||
// authentication middleware more than once or mutate the parser's output.
|
||||
func prepareMetadata(original *Spec) (*Spec, error) {
|
||||
spec := *original
|
||||
spec.Routes = append([]Route(nil), original.Routes...)
|
||||
if err := validateResponses(&spec); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
definitions := map[string]ErrorDefinition{}
|
||||
codes, symbols := map[string]bool{}, map[string]bool{}
|
||||
for _, e := range spec.Errors {
|
||||
if !exportedName.MatchString(e.Name) || e.Status < 400 || e.Status > 599 || !errorCode.MatchString(e.Code) || strings.TrimSpace(e.Message) == "" {
|
||||
return nil, fmt.Errorf("%s: error requires UpperCamelCase name, status 400..599, UPPER_SNAKE_CASE code and a nonempty public message", e.Position)
|
||||
}
|
||||
if codes[e.Code] {
|
||||
return nil, fmt.Errorf("%s: duplicate error code %s", e.Position, e.Code)
|
||||
}
|
||||
for _, symbol := range []string{e.Name, "Code" + e.Name} {
|
||||
if symbols[symbol] {
|
||||
return nil, fmt.Errorf("%s: duplicate error symbol %s", e.Position, symbol)
|
||||
}
|
||||
symbols[symbol] = true
|
||||
}
|
||||
codes[e.Code] = true
|
||||
definitions[e.Name] = e
|
||||
}
|
||||
auth := map[string]AuthDefinition{}
|
||||
for _, a := range spec.Auth {
|
||||
if !exportedName.MatchString(a.Name) || !goName.MatchString(a.Middleware) || strings.EqualFold(a.Middleware, "defaults") || strings.EqualFold(a.Middleware, "registryGen") {
|
||||
return nil, fmt.Errorf("%s: auth requires an UpperCamelCase name and a valid named middleware", a.Position)
|
||||
}
|
||||
if _, ok := auth[a.Name]; ok {
|
||||
return nil, fmt.Errorf("%s: duplicate auth %s", a.Position, a.Name)
|
||||
}
|
||||
switch a.Type {
|
||||
case "bearer":
|
||||
if a.In != "" || a.KeyName != "" {
|
||||
return nil, fmt.Errorf("%s: bearer auth does not accept in/name", a.Position)
|
||||
}
|
||||
case "apiKey":
|
||||
if (a.In != "header" && a.In != "cookie") || !credentialName.MatchString(a.KeyName) || a.BearerFormat != "" {
|
||||
return nil, fmt.Errorf("%s: apiKey requires in header/cookie and a valid name; bearerFormat is not supported", a.Position)
|
||||
}
|
||||
if a.In == "header" && (strings.EqualFold(a.KeyName, "Authorization") || strings.EqualFold(a.KeyName, "Cookie")) {
|
||||
return nil, fmt.Errorf("%s: use bearer for Authorization or in cookie for a session", a.Position)
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("%s: auth type must be bearer or apiKey", a.Position)
|
||||
}
|
||||
auth[a.Name] = a
|
||||
}
|
||||
for i := range spec.Routes {
|
||||
r := &spec.Routes[i]
|
||||
r.Middleware = append([]string(nil), r.Middleware...)
|
||||
if r.Auth != "" && r.Auth != "none" {
|
||||
a, ok := auth[r.Auth]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("%s: unknown auth %s; declare auth %s (...) first", r.Position, r.Auth, r.Auth)
|
||||
}
|
||||
for _, m := range r.Middleware {
|
||||
if strings.EqualFold(m, a.Middleware) {
|
||||
return nil, fmt.Errorf("%s: auth %s already installs %s; remove it from middleware", r.Position, r.Auth, a.Middleware)
|
||||
}
|
||||
}
|
||||
r.Middleware = append([]string{a.Middleware}, r.Middleware...)
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, name := range r.Errors {
|
||||
if _, ok := definitions[name]; !ok {
|
||||
return nil, fmt.Errorf("%s: unknown error %s; declare it in error (...)", r.Position, name)
|
||||
}
|
||||
if seen[name] {
|
||||
return nil, fmt.Errorf("%s: duplicate error %s", r.Position, name)
|
||||
}
|
||||
seen[name] = true
|
||||
}
|
||||
seen = map[string]bool{}
|
||||
for _, tag := range r.Tags {
|
||||
if strings.TrimSpace(tag) == "" || seen[tag] {
|
||||
return nil, fmt.Errorf("%s: @doc tags must be nonempty and unique", r.Position)
|
||||
}
|
||||
seen[tag] = true
|
||||
}
|
||||
}
|
||||
return &spec, nil
|
||||
}
|
||||
|
||||
func decorateDocument(doc map[string]any, spec *Spec) {
|
||||
if spec.Description != "" {
|
||||
doc["info"].(map[string]any)["description"] = spec.Description
|
||||
}
|
||||
schemes := map[string]any{}
|
||||
for _, a := range spec.Auth {
|
||||
scheme := map[string]any{}
|
||||
if a.Type == "bearer" {
|
||||
scheme["type"], scheme["scheme"] = "http", "bearer"
|
||||
if a.BearerFormat != "" {
|
||||
scheme["bearerFormat"] = a.BearerFormat
|
||||
}
|
||||
} else {
|
||||
scheme["type"], scheme["in"], scheme["name"] = "apiKey", a.In, a.KeyName
|
||||
}
|
||||
if a.Description != "" {
|
||||
scheme["description"] = a.Description
|
||||
}
|
||||
schemes[a.Name] = scheme
|
||||
}
|
||||
doc["components"].(map[string]any)["securitySchemes"] = schemes
|
||||
definitions := map[string]ErrorDefinition{}
|
||||
for _, e := range spec.Errors {
|
||||
definitions[e.Name] = e
|
||||
}
|
||||
if len(spec.Errors) > 0 {
|
||||
doc["x-omni-errors"] = spec.Errors
|
||||
}
|
||||
for _, r := range spec.Routes {
|
||||
policy := response.MustNew(responseConfig(spec, r))
|
||||
cfg := policy.Config()
|
||||
parts := strings.Split(r.Path, "/")
|
||||
for i, part := range parts {
|
||||
if strings.HasPrefix(part, ":") {
|
||||
parts[i] = "{" + part[1:] + "}"
|
||||
}
|
||||
}
|
||||
op := doc["paths"].(map[string]any)[strings.Join(parts, "/")].(map[string]any)[r.Method].(map[string]any)
|
||||
if r.ResponseFormat != "" {
|
||||
op["x-omni-response"] = r.ResponseFormat
|
||||
}
|
||||
if len(r.Middleware) > 0 {
|
||||
op["x-omni-middleware"] = r.Middleware
|
||||
}
|
||||
if r.Auth != "" && r.Auth != "none" {
|
||||
op["security"] = []map[string][]string{{r.Auth: {}}}
|
||||
} else {
|
||||
op["security"] = []any{}
|
||||
}
|
||||
if r.Description != "" {
|
||||
op["description"] = r.Description
|
||||
}
|
||||
if len(r.Tags) > 0 {
|
||||
op["tags"] = r.Tags
|
||||
} else {
|
||||
op["tags"] = []string{r.Group}
|
||||
}
|
||||
if r.Deprecated {
|
||||
op["deprecated"] = true
|
||||
}
|
||||
responses := op["responses"].(map[string]any)
|
||||
if r.Auth != "" && r.Auth != "none" {
|
||||
responses["401"] = genericErrorResponse("Authentication required", cfg)
|
||||
}
|
||||
grouped := map[int][]ErrorDefinition{}
|
||||
for _, name := range r.Errors {
|
||||
e := definitions[name]
|
||||
grouped[e.Status] = append(grouped[e.Status], e)
|
||||
}
|
||||
if len(r.Errors) > 0 {
|
||||
op["x-omni-errors"] = r.Errors
|
||||
}
|
||||
for status, errs := range grouped {
|
||||
key := fmt.Sprint(status)
|
||||
response, exists := responses[key].(map[string]any)
|
||||
if !exists {
|
||||
response = genericErrorResponse(http.StatusText(status), cfg)
|
||||
responses[key] = response
|
||||
}
|
||||
media := response["content"].(map[string]any)["application/json"].(map[string]any)
|
||||
examples := map[string]any{}
|
||||
codes := []any{}
|
||||
seenCodes := map[string]bool{}
|
||||
description := []string{}
|
||||
sort.Slice(errs, func(i, j int) bool { return errs[i].Name < errs[j].Name })
|
||||
for _, e := range errs {
|
||||
value := policy.Value(responseResult(e))
|
||||
examples[e.Name] = map[string]any{"summary": e.Message, "value": value}
|
||||
codeKey := fmt.Sprint(value[cfg.CodeField])
|
||||
if !seenCodes[codeKey] {
|
||||
codes = append(codes, value[cfg.CodeField])
|
||||
seenCodes[codeKey] = true
|
||||
}
|
||||
description = append(description, e.Code+": "+e.Message)
|
||||
}
|
||||
media["examples"] = examples
|
||||
if !exists && cfg.Body == nil && cfg.CodeField != "-" {
|
||||
media["schema"].(map[string]any)["properties"].(map[string]any)[cfg.CodeField].(map[string]any)["enum"] = codes
|
||||
}
|
||||
response["description"] = strings.Join(description, "\n")
|
||||
}
|
||||
}
|
||||
}
|
||||
func responseResult(e ErrorDefinition) response.Result {
|
||||
return response.Result{Status: e.Status, Code: e.Code, Message: e.Message, RequestID: "exampleRequest"}
|
||||
}
|
||||
func genericErrorResponse(description string, cfg response.Config) map[string]any {
|
||||
if description == "" {
|
||||
description = "Error"
|
||||
}
|
||||
schema, _ := openapi.EnvelopeWith(nil, cfg)
|
||||
data, _ := json.Marshal(schema)
|
||||
var value map[string]any
|
||||
_ = json.Unmarshal(data, &value)
|
||||
return map[string]any{"description": description, "content": map[string]any{"application/json": map[string]any{"schema": value}}}
|
||||
}
|
||||
165
cmd/go-omni/apigen/declarations.go
Normal file
165
cmd/go-omni/apigen/declarations.go
Normal file
@@ -0,0 +1,165 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// ErrorDefinition declares a public, stable error. Generated constructors return
|
||||
// a fresh error each time so WithCause cannot mutate a shared singleton.
|
||||
type ErrorDefinition struct {
|
||||
Name string `json:"name"`
|
||||
Status int `json:"status"`
|
||||
Code string `json:"code"`
|
||||
Message string `json:"message"`
|
||||
Position string `json:"-"`
|
||||
}
|
||||
type AuthDefinition struct {
|
||||
Name, Type, In, KeyName, Middleware, Description, BearerFormat, Position string
|
||||
}
|
||||
|
||||
func (p *parser) parseErrors() error {
|
||||
p.next()
|
||||
block := p.peek() == "("
|
||||
if block {
|
||||
p.next()
|
||||
p.skip()
|
||||
}
|
||||
for {
|
||||
pos := p.tokens[p.at].position
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
status, err := strconv.Atoi(p.next().text)
|
||||
if err != nil {
|
||||
return p.err("error HTTP status must be an integer")
|
||||
}
|
||||
code, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
message, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.spec.Errors = append(p.spec.Errors, ErrorDefinition{Name: name, Status: status, Code: code, Message: message, Position: pos})
|
||||
if p.peek() != "\n" && p.peek() != ";" && !(block && p.peek() == ")") && p.peek() != "" {
|
||||
return p.err("expected end of error declaration")
|
||||
}
|
||||
if !block {
|
||||
return nil
|
||||
}
|
||||
p.skip()
|
||||
if p.peek() == ")" {
|
||||
p.next()
|
||||
return nil
|
||||
}
|
||||
}
|
||||
}
|
||||
func (p *parser) parseAuth() error {
|
||||
pos := p.next().position
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.skip()
|
||||
props, err := p.metadata()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
a := AuthDefinition{Name: name, Position: pos}
|
||||
for key, value := range props {
|
||||
switch key {
|
||||
case "type":
|
||||
a.Type = value
|
||||
case "in":
|
||||
a.In = value
|
||||
case "name":
|
||||
a.KeyName = value
|
||||
case "middleware":
|
||||
a.Middleware = value
|
||||
case "description":
|
||||
a.Description = value
|
||||
case "bearerFormat":
|
||||
a.BearerFormat = value
|
||||
default:
|
||||
return fmt.Errorf("%s: unsupported auth property %q", pos, key)
|
||||
}
|
||||
}
|
||||
p.spec.Auth = append(p.spec.Auth, a)
|
||||
return nil
|
||||
}
|
||||
func (p *parser) names() ([]string, error) {
|
||||
var names []string
|
||||
for {
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, prior := range names {
|
||||
if prior == name {
|
||||
return nil, p.err("duplicate name " + name)
|
||||
}
|
||||
}
|
||||
names = append(names, name)
|
||||
if p.peek() != "," {
|
||||
return names, nil
|
||||
}
|
||||
p.next()
|
||||
}
|
||||
}
|
||||
func commaList(value string) []string {
|
||||
if value == "" {
|
||||
return nil
|
||||
}
|
||||
result := strings.Split(value, ",")
|
||||
for i := range result {
|
||||
result[i] = strings.TrimSpace(result[i])
|
||||
}
|
||||
return result
|
||||
}
|
||||
func mergeNames(base, extra []string) []string {
|
||||
result := append([]string(nil), base...)
|
||||
for _, name := range extra {
|
||||
found := false
|
||||
for _, prior := range result {
|
||||
found = found || prior == name
|
||||
}
|
||||
if !found {
|
||||
result = append(result, name)
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
func (p *parser) parseDoc(route *Route) error {
|
||||
if p.peek() != "(" {
|
||||
value, err := p.quoted()
|
||||
route.Summary = value
|
||||
return err
|
||||
}
|
||||
pos := p.tokens[p.at].position
|
||||
props, err := p.metadata()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for key, value := range props {
|
||||
switch key {
|
||||
case "summary":
|
||||
route.Summary = value
|
||||
case "description":
|
||||
route.Description = value
|
||||
case "tags":
|
||||
route.Tags = commaList(value)
|
||||
case "deprecated":
|
||||
if value != "true" && value != "false" {
|
||||
return fmt.Errorf("%s: deprecated must be true or false", pos)
|
||||
}
|
||||
route.Deprecated = value == "true"
|
||||
default:
|
||||
return fmt.Errorf("%s: unsupported @doc property %q", pos, key)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
485
cmd/go-omni/apigen/generate.go
Normal file
485
cmd/go-omni/apigen/generate.go
Normal file
@@ -0,0 +1,485 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime/debug"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||
"golang.org/x/mod/modfile"
|
||||
"golang.org/x/mod/module"
|
||||
"golang.org/x/mod/semver"
|
||||
)
|
||||
|
||||
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
const minimumVersion = "v0.8.0-preview"
|
||||
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||
|
||||
type Options struct {
|
||||
Database, Storage string
|
||||
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
|
||||
Check bool
|
||||
DryRun bool
|
||||
}
|
||||
type FileChange struct {
|
||||
Path string
|
||||
Action string
|
||||
}
|
||||
type Output struct {
|
||||
Generated, Created, Preserved, Removed []string
|
||||
Changes []FileChange
|
||||
}
|
||||
|
||||
// Generate creates or safely updates one contract-driven project. It renders
|
||||
// and validates every output before applying an ownership-checked transaction.
|
||||
func Generate(o Options) (Output, error) {
|
||||
if o.Check && o.DryRun {
|
||||
return Output{}, errors.New("--check and --dry-run cannot be combined")
|
||||
}
|
||||
if o.API == "" || o.Dir == "" {
|
||||
return Output{}, errors.New("--api and --dir are required")
|
||||
}
|
||||
spec, err := Parse(o.API)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
compiled, err := Compile(spec)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
return generate(o, compiled, false)
|
||||
}
|
||||
|
||||
const starterAPI = "syntax = \"v1\"\n\ninfo (\n title: \"Greeting API\"\n version: \"1.0.0\"\n)\n\ntype GreetReq {\n Name string `path:\"name\" binding:\"required,max=80\"`\n}\ntype GreetResp {\n Message string `json:\"message\"`\n}\n\n@server (\n group: greet\n prefix: /api\n)\nservice greet-api {\n @doc \"Greet a visitor\"\n @handler greet\n get /greet/:name (GreetReq) returns (GreetResp)\n}\n"
|
||||
|
||||
// New creates a runnable greeting project, including its editable API file.
|
||||
func New(o Options) (Output, error) {
|
||||
if o.API != "" || o.Check || o.DryRun {
|
||||
return Output{}, errors.New("api new does not accept --api, --check or --dry-run")
|
||||
}
|
||||
if o.Dir == "" {
|
||||
return Output{}, errors.New("project directory required")
|
||||
}
|
||||
if entries, err := os.ReadDir(o.Dir); err == nil && len(entries) > 0 {
|
||||
return Output{}, errors.New("api new requires an empty directory")
|
||||
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return Output{}, err
|
||||
}
|
||||
tokens, err := lex("service.api", starterAPI)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
spec := &Spec{Version: "1.0.0", Entry: "service.api", Sources: map[string][]byte{"service.api": []byte(starterAPI)}}
|
||||
p := parser{tokens: tokens, spec: spec}
|
||||
if err = p.parse(); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
return generate(o, c, true)
|
||||
}
|
||||
|
||||
func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
directory, err := filepath.Abs(o.Dir)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
o.Dir = directory
|
||||
if err := resolveProfile(&o); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
seeds := map[string][]byte{}
|
||||
moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod"))
|
||||
if readErr == nil {
|
||||
mf, err := modfile.Parse("go.mod", moduleData, nil)
|
||||
if err != nil || mf.Module == nil {
|
||||
return Output{}, errors.New("invalid project go.mod")
|
||||
}
|
||||
if o.Module != "" && o.Module != mf.Module.Mod.Path {
|
||||
return Output{}, errors.New("--module differs from existing go.mod")
|
||||
}
|
||||
o.Module = mf.Module.Mod.Path
|
||||
found := false
|
||||
requiredVersion := ""
|
||||
for _, r := range mf.Require {
|
||||
if r.Mod.Path == frameworkModule {
|
||||
found = true
|
||||
requiredVersion = r.Mod.Version
|
||||
if o.FrameworkVersion != "" && o.FrameworkVersion != r.Mod.Version {
|
||||
return Output{}, errors.New("existing framework version differs; update go.mod explicitly")
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return Output{}, errors.New("existing go.mod must require go-omni")
|
||||
}
|
||||
if semver.Compare(requiredVersion, minimumVersion) < 0 {
|
||||
localCurrent := false
|
||||
for _, r := range mf.Replace {
|
||||
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||
path := r.New.Path
|
||||
if !filepath.IsAbs(path) {
|
||||
path = filepath.Join(directory, path)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(path, "httpx", "response.go")); err == nil {
|
||||
localCurrent = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !localCurrent {
|
||||
return Output{}, fmt.Errorf("existing framework dependency predates configurable responses; use %s or a current local replacement", minimumVersion)
|
||||
}
|
||||
}
|
||||
if o.FrameworkDir != "" {
|
||||
matched := false
|
||||
for _, r := range mf.Replace {
|
||||
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||
replacement := r.New.Path
|
||||
if !filepath.IsAbs(replacement) {
|
||||
replacement = filepath.Join(directory, replacement)
|
||||
}
|
||||
matched = canonical(replacement) == canonical(o.FrameworkDir)
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
return Output{}, errors.New("existing framework replacement differs; update go.mod explicitly")
|
||||
}
|
||||
}
|
||||
} else if !errors.Is(readErr, os.ErrNotExist) {
|
||||
return Output{}, readErr
|
||||
} else {
|
||||
if o.Module == "" {
|
||||
o.Module = filepath.Base(directory)
|
||||
}
|
||||
data, err := moduleFile(o)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
seeds["go.mod"] = data
|
||||
}
|
||||
if err = module.CheckImportPath(o.Module); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
generated, base, err := renderProject(o, c, starter)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
for name, data := range base {
|
||||
seeds[name] = data
|
||||
}
|
||||
// The API source remains developer-owned. A relocated project regenerates
|
||||
// from its own api/ tree, including all imports.
|
||||
entry := "api/" + c.Spec.Entry
|
||||
for name, data := range c.Spec.Sources {
|
||||
seeds["api/"+name] = data
|
||||
}
|
||||
if o.API != "" {
|
||||
source, _ := filepath.Abs(o.API)
|
||||
sourceReal, _ := filepath.EvalSymlinks(source)
|
||||
targetReal := canonical(directory)
|
||||
rel, e := filepath.Rel(targetReal, sourceReal)
|
||||
if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
entry = filepath.ToSlash(rel)
|
||||
for name := range c.Spec.Sources {
|
||||
delete(seeds, "api/"+name)
|
||||
}
|
||||
}
|
||||
}
|
||||
// Source collisions must never leave a copied contract different from the
|
||||
// generated code. On updates, use the developer-owned API in the project.
|
||||
for name, data := range seeds {
|
||||
if strings.HasSuffix(name, ".api") {
|
||||
current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name)))
|
||||
if e == nil && string(current) != string(data) {
|
||||
return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
seeds["README.md"] = []byte(projectREADME(o.Module, entry) + infrastructureREADME(o))
|
||||
for _, files := range []map[string][]byte{generated, seeds} {
|
||||
for name, data := range files {
|
||||
if strings.HasSuffix(name, ".go") {
|
||||
data, err = format.Source(data)
|
||||
if err != nil {
|
||||
return Output{}, fmt.Errorf("format %s: %w", name, err)
|
||||
}
|
||||
files[name] = data
|
||||
}
|
||||
}
|
||||
}
|
||||
return apply(directory, o.Module, entry, generated, seeds, o.Check, o.DryRun, o.profile())
|
||||
}
|
||||
|
||||
func moduleFile(o Options) ([]byte, error) {
|
||||
if err := module.CheckImportPath(o.Module); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
root := o.FrameworkDir
|
||||
version := o.FrameworkVersion
|
||||
if root == "" && version == "" {
|
||||
wd, _ := os.Getwd()
|
||||
for wd != "" {
|
||||
data, err := os.ReadFile(filepath.Join(wd, "go.mod"))
|
||||
if err == nil {
|
||||
mf, e := modfile.Parse("go.mod", data, nil)
|
||||
if e == nil && mf.Module != nil && mf.Module.Mod.Path == frameworkModule {
|
||||
root = wd
|
||||
}
|
||||
break
|
||||
}
|
||||
parent := filepath.Dir(wd)
|
||||
if parent == wd {
|
||||
break
|
||||
}
|
||||
wd = parent
|
||||
}
|
||||
}
|
||||
if version == "" {
|
||||
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
|
||||
version = info.Main.Version
|
||||
} else {
|
||||
version = minimumVersion
|
||||
}
|
||||
}
|
||||
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {
|
||||
return nil, errors.New("invalid framework version")
|
||||
}
|
||||
if root == "" && semver.Compare(version, minimumVersion) < 0 {
|
||||
return nil, fmt.Errorf("API generation needs %s or newer; use --framework-dir for unpublished source", minimumVersion)
|
||||
}
|
||||
text := "module " + o.Module + "\n\ngo 1.22.0\n\nrequire " + frameworkModule + " " + version + "\n"
|
||||
if root != "" {
|
||||
root = canonical(root)
|
||||
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mf, err := modfile.Parse("go.mod", data, nil)
|
||||
if err != nil || mf.Module == nil || mf.Module.Mod.Path != frameworkModule {
|
||||
return nil, errors.New("--framework-dir must contain go-omni")
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(root, "contract", "contract.go")); err != nil {
|
||||
return nil, errors.New("local framework lacks API contract support")
|
||||
}
|
||||
relative, err := filepath.Rel(canonical(o.Dir), root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
text += "\nreplace " + frameworkModule + " => " + strconv.Quote(filepath.ToSlash(relative)) + "\n"
|
||||
}
|
||||
return []byte(text), nil
|
||||
}
|
||||
func canonical(path string) string {
|
||||
path, _ = filepath.Abs(path)
|
||||
candidate := path
|
||||
var tail []string
|
||||
for {
|
||||
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
|
||||
for i := len(tail) - 1; i >= 0; i-- {
|
||||
resolved = filepath.Join(resolved, tail[i])
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
parent := filepath.Dir(candidate)
|
||||
if parent == candidate {
|
||||
return path
|
||||
}
|
||||
tail = append(tail, filepath.Base(candidate))
|
||||
candidate = parent
|
||||
}
|
||||
}
|
||||
|
||||
func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map[string][]byte, error) {
|
||||
generated, seeds := map[string][]byte{}, map[string][]byte{}
|
||||
put := func(files map[string][]byte, name, content string) { files[name] = []byte(content) }
|
||||
if len(c.Spec.Errors) > 0 {
|
||||
content := "package apierrors\nimport omnierrors " + strconv.Quote(frameworkModule+"/errors") + "\n"
|
||||
for _, e := range c.Spec.Errors {
|
||||
content += fmt.Sprintf("const Code%s = %q\n// %s returns a fresh public error declared in the API contract.\nfunc %s()*omnierrors.APIError{return omnierrors.New(%d,Code%s,%q)}\n", e.Name, e.Code, e.Name, e.Name, e.Status, e.Name, e.Message)
|
||||
}
|
||||
put(generated, "internal/apierrors/errorsGen.go", generatedHeader+content)
|
||||
}
|
||||
types := "package types\n\n"
|
||||
for _, t := range c.Spec.Types {
|
||||
types += "type " + t.Name + " struct {\n"
|
||||
for _, f := range t.Fields {
|
||||
types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag)
|
||||
}
|
||||
types += "}\n"
|
||||
}
|
||||
put(generated, "internal/types/typesGen.go", generatedHeader+types)
|
||||
put(generated, "docs/openapi.json", string(c.Document)+"\n")
|
||||
put(generated, "internal/handler/openapiGen.go", generatedHeader+fmt.Sprintf("package handler\nimport (\"net/http\"; %q)\nconst OpenAPIDocument = %q\nfunc OpenAPI(c *httpx.Context) error { return c.WriteResponse(200,http.Header{\"Content-Type\": {\"application/json; charset=utf-8\"}},[]byte(OpenAPIDocument)) }\n", frameworkModule+"/httpx", string(c.Document)+"\n"))
|
||||
policySource := fmt.Sprintf("package handler\nimport(\"encoding/json\";\"strings\";%q)\nvar responseTemplates=map[string]response.Encoder{\"\":response.Default(),\n", frameworkModule+"/response")
|
||||
for _, definition := range c.Spec.Responses {
|
||||
data, err := json.Marshal(definition.Config)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
policySource += fmt.Sprintf("%q: newResponseTemplate(%q),\n", definition.Name, string(data))
|
||||
}
|
||||
policySource += "}\nfunc newResponseTemplate(raw string)response.Encoder{var cfg response.Config;decoder:=json.NewDecoder(strings.NewReader(raw));decoder.UseNumber();if err:=decoder.Decode(&cfg);err!=nil{panic(err)};return response.MustNew(cfg)}\nfunc responseEncoder(name string,raw bool)response.Encoder{encoder:=responseTemplates[name];if encoder==nil{panic(\"unknown response template\")};if raw{encoder=response.RawJSON(encoder)};return encoder}\n"
|
||||
put(generated, "internal/handler/responseGen.go", generatedHeader+policySource)
|
||||
imports := fmt.Sprintf("import (\"fmt\"; %q; %q; %q;", frameworkModule+"/httpx", frameworkModule+"/openapi", o.Module+"/internal/servicecontext")
|
||||
routes := "func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {\n if router==nil || deps==nil {return fmt.Errorf(\"router and dependencies required\")}\n"
|
||||
middlewareSet := map[string]bool{}
|
||||
for _, r := range c.Spec.Routes {
|
||||
for _, m := range r.Middleware {
|
||||
middlewareSet[m] = true
|
||||
}
|
||||
}
|
||||
var middlewareNames []string
|
||||
for m := range middlewareSet {
|
||||
middlewareNames = append(middlewareNames, m)
|
||||
}
|
||||
sort.Strings(middlewareNames)
|
||||
for _, m := range middlewareNames {
|
||||
routes += fmt.Sprintf("if deps.Middlewares[%q]==nil {return fmt.Errorf(%q)}\n", m, "missing middleware "+m)
|
||||
}
|
||||
for i, group := range c.Groups {
|
||||
imports += fmt.Sprintf("h%d %q; s%d %q;", i, o.Module+"/internal/handler/"+group, i, o.Module+"/internal/service/"+group)
|
||||
routes += fmt.Sprintf("h%d := h%d.New(s%d.New(deps))\n", i, i, i)
|
||||
groupRoutes := []Route{}
|
||||
for _, r := range c.Spec.Routes {
|
||||
if r.Group == group {
|
||||
groupRoutes = append(groupRoutes, r)
|
||||
}
|
||||
}
|
||||
packageName := filepath.Base(group)
|
||||
text := "package " + packageName + "\nimport(\"context\"; \"fmt\";"
|
||||
needsTypes, needsBinder := false, false
|
||||
for _, r := range groupRoutes {
|
||||
needsTypes = needsTypes || r.Request != "" || r.Response != ""
|
||||
needsBinder = needsBinder || r.Request != ""
|
||||
}
|
||||
text += fmt.Sprintf("%q;", frameworkModule+"/httpx")
|
||||
if needsTypes {
|
||||
text += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||
}
|
||||
if needsBinder {
|
||||
text += fmt.Sprintf("%q;", frameworkModule+"/contract")
|
||||
}
|
||||
text += ")\n"
|
||||
text += "type Service interface {\n"
|
||||
for _, r := range groupRoutes {
|
||||
text += signature(r) + "\n"
|
||||
}
|
||||
text += "}\ntype Handler struct {service Service}\nfunc New(s Service)*Handler {return &Handler{service:s}}\n"
|
||||
for _, r := range groupRoutes {
|
||||
name := upper(r.Handler)
|
||||
if r.Request != "" {
|
||||
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s]()\n", lower(r.Handler), r.Request)
|
||||
}
|
||||
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
|
||||
call := "c.Context()"
|
||||
if r.Request != "" {
|
||||
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req);err!=nil{return err}\n", r.Request, lower(r.Handler))
|
||||
call += ", &req"
|
||||
}
|
||||
if r.Response != "" {
|
||||
text += fmt.Sprintf("resp,err:=h.service.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name)
|
||||
method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status]
|
||||
text += "return c." + method + "(resp)\n}\n"
|
||||
} else {
|
||||
text += fmt.Sprintf("if err:=h.service.%s(%s);err!=nil{return err};return c.NoContent()\n}\n", name, call)
|
||||
}
|
||||
routes += fmt.Sprintf("httpx.HandleWithResponse(router,%q,%q,responseEncoder(%q,%t),h%d.%s", strings.ToUpper(r.Method), r.Path, responseName(r), r.ResponseFormat == "raw", i, name)
|
||||
for _, m := range r.Middleware {
|
||||
routes += fmt.Sprintf(",deps.Middlewares[%q]", m)
|
||||
}
|
||||
routes += ")\n"
|
||||
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
|
||||
if r.Request != "" || r.Response != "" {
|
||||
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||
}
|
||||
service += ")\n"
|
||||
service += "// " + name + " implements the business operation. This file is never overwritten.\nfunc(s *Service) " + signature(r) + " {\n"
|
||||
if starter && r.Handler == "greet" {
|
||||
service = strings.Replace(service, "apierrors "+strconv.Quote(frameworkModule+"/errors")+";", "\"fmt\";", 1)
|
||||
service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}"
|
||||
} else {
|
||||
service += "// TODO: implement business behavior and use s.deps for dependencies.\n"
|
||||
if r.Response != "" {
|
||||
service += "return nil, "
|
||||
} else {
|
||||
service += "return "
|
||||
}
|
||||
service += "apierrors.New(501,\"NOT_IMPLEMENTED\",\"operation is not implemented\")\n}"
|
||||
}
|
||||
put(seeds, "internal/service/"+group+"/"+lower(r.Handler)+".go", service)
|
||||
}
|
||||
if !hasResponse(groupRoutes) {
|
||||
text = strings.Replace(text, "\"fmt\";", "", 1)
|
||||
}
|
||||
put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text)
|
||||
put(seeds, "internal/service/"+group+"/service.go", fmt.Sprintf("package %s\nimport %q\ntype Service struct {deps *servicecontext.Context}\nfunc New(deps *servicecontext.Context)*Service{return &Service{deps:deps}}\n", packageName, o.Module+"/internal/servicecontext"))
|
||||
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
|
||||
}
|
||||
put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n")
|
||||
registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n"
|
||||
for _, m := range middlewareNames {
|
||||
registry += fmt.Sprintf("%q:%s,\n", m, upper(m))
|
||||
put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m)))
|
||||
}
|
||||
put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n")
|
||||
for name, source := range baseTemplates {
|
||||
content := strings.ReplaceAll(source, "MODULE", o.Module)
|
||||
content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat)
|
||||
content = strings.ReplaceAll(content, "LOGOUTPUT", o.LogOutput)
|
||||
if name == "etc/config.yaml" && o.LogOutput == "file" {
|
||||
content = strings.Replace(content, "output: file", "output: stderr\n file: ./var/log/server.log\n rotation:\n enabled: true\n maxSizeMb: 100\n maxBackups: 10\n maxAgeDays: 7\n compress: true", 1)
|
||||
}
|
||||
put(seeds, name, content)
|
||||
}
|
||||
|
||||
infra, err := scaffold.APIInfrastructure(o.Module, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
for name, data := range infra {
|
||||
seeds[name] = data
|
||||
}
|
||||
if o.Storage == "local" {
|
||||
for _, r := range c.Spec.Routes {
|
||||
if r.Path == "/storage/object" {
|
||||
return nil, nil, errors.New("/storage/object is reserved by local storage")
|
||||
}
|
||||
}
|
||||
main := string(seeds["cmd/server/main.go"])
|
||||
main = strings.Replace(main, " if err=handler.RegisterRoutes", ` for _,method:=range []string{"GET","HEAD","PUT"}{app.Handle(method,"/storage/object",func(c *httpx.Context)error{deps.FileHTTP.ServeHTTP(c.Raw(),c.Request());return nil})}
|
||||
if err=handler.RegisterRoutes`, 1)
|
||||
main = strings.Replace(main, `"git.yuntuo.hk/k2safe/go-omni/logx"`, `"git.yuntuo.hk/k2safe/go-omni/logx"; "git.yuntuo.hk/k2safe/go-omni/httpx"`, 1)
|
||||
seeds["cmd/server/main.go"] = []byte(main)
|
||||
}
|
||||
return generated, seeds, nil
|
||||
}
|
||||
func hasResponse(routes []Route) bool {
|
||||
for _, r := range routes {
|
||||
if r.Response != "" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
func signature(r Route) string {
|
||||
s := upper(r.Handler) + "(ctx context.Context"
|
||||
if r.Request != "" {
|
||||
s += ",req *types." + r.Request
|
||||
}
|
||||
s += ") "
|
||||
if r.Response != "" {
|
||||
s += "(*types." + r.Response + ",error)"
|
||||
} else {
|
||||
s += "error"
|
||||
}
|
||||
return s
|
||||
}
|
||||
305
cmd/go-omni/apigen/metadata_test.go
Normal file
305
cmd/go-omni/apigen/metadata_test.go
Normal file
@@ -0,0 +1,305 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
const declaredAPI = `syntax = "v1"
|
||||
info (
|
||||
title: "Declared API"
|
||||
description: "Errors and authentication share one contract"
|
||||
)
|
||||
error (
|
||||
Unauthenticated 401 "UNAUTHENTICATED" "authentication required"
|
||||
Forbidden 403 "FORBIDDEN" "permission denied"
|
||||
EmailTaken 409 "EMAIL_TAKEN" "email already in use"
|
||||
NameTaken 409 "NAME_TAKEN" "name already in use"
|
||||
)
|
||||
auth UserAuth (
|
||||
type: bearer
|
||||
middleware: Auth
|
||||
description: "Verified application token"
|
||||
)
|
||||
auth PartnerAuth (
|
||||
type: apiKey
|
||||
in: header
|
||||
name: X-API-Key
|
||||
middleware: Partner
|
||||
)
|
||||
auth SessionAuth (
|
||||
type: apiKey
|
||||
in: cookie
|
||||
name: sessionId
|
||||
middleware: Session
|
||||
)
|
||||
@server (
|
||||
group: user
|
||||
auth: UserAuth
|
||||
errors: Unauthenticated
|
||||
middleware: Audit
|
||||
)
|
||||
service declared-api {
|
||||
@doc (
|
||||
summary: "Create a user"
|
||||
description: "Uniqueness is enforced in a transaction"
|
||||
tags: users, administration
|
||||
deprecated: true
|
||||
)
|
||||
@errors Forbidden, EmailTaken, NameTaken
|
||||
@handler create
|
||||
post /users
|
||||
@auth none
|
||||
@errors none
|
||||
@handler public
|
||||
get /public
|
||||
@auth PartnerAuth
|
||||
@handler partner
|
||||
get /partner
|
||||
@auth SessionAuth
|
||||
@handler session
|
||||
get /session
|
||||
}
|
||||
`
|
||||
|
||||
func document(t *testing.T, c *Compiled) map[string]any {
|
||||
t.Helper()
|
||||
var doc map[string]any
|
||||
if err := json.Unmarshal(c.Document, &doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return doc
|
||||
}
|
||||
func operation(t *testing.T, doc map[string]any, path, method string) map[string]any {
|
||||
t.Helper()
|
||||
return doc["paths"].(map[string]any)[path].(map[string]any)[method].(map[string]any)
|
||||
}
|
||||
func TestDeclaredErrorsAndAuthentication(t *testing.T) {
|
||||
spec, err := parseText(t, declaredAPI)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
again, err := Compile(spec)
|
||||
if err != nil || string(c.Document) != string(again.Document) {
|
||||
t.Fatalf("compile is not repeatable: %v", err)
|
||||
}
|
||||
if strings.Join(spec.Routes[0].Middleware, ",") != "Audit" {
|
||||
t.Fatal("parser output mutated")
|
||||
}
|
||||
if strings.Join(c.Spec.Routes[0].Middleware, ",") != "Auth,Audit" {
|
||||
t.Fatal("authentication must be first")
|
||||
}
|
||||
if strings.Join(c.Spec.Routes[1].Middleware, ",") != "Audit" {
|
||||
t.Fatal("public override retained automatic auth")
|
||||
}
|
||||
doc := document(t, c)
|
||||
op := operation(t, doc, "/users", "post")
|
||||
if op["deprecated"] != true || op["description"] == nil || len(op["tags"].([]any)) != 2 {
|
||||
t.Fatalf("documentation missing: %v", op)
|
||||
}
|
||||
security := op["security"].([]any)
|
||||
if _, ok := security[0].(map[string]any)["UserAuth"]; !ok {
|
||||
t.Fatal("security requirement absent")
|
||||
}
|
||||
schemes := doc["components"].(map[string]any)["securitySchemes"].(map[string]any)
|
||||
if len(schemes) != 3 || schemes["UserAuth"].(map[string]any)["scheme"] != "bearer" || schemes["SessionAuth"].(map[string]any)["in"] != "cookie" {
|
||||
t.Fatalf("wrong security schemes: %v", schemes)
|
||||
}
|
||||
responses := op["responses"].(map[string]any)
|
||||
for _, status := range []string{"401", "403", "409"} {
|
||||
if responses[status] == nil {
|
||||
t.Fatal("missing status " + status)
|
||||
}
|
||||
}
|
||||
media := responses["409"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
|
||||
if len(media["examples"].(map[string]any)) != 2 {
|
||||
t.Fatal("same-status errors were overwritten")
|
||||
}
|
||||
codes := media["schema"].(map[string]any)["properties"].(map[string]any)["code"].(map[string]any)["enum"].([]any)
|
||||
if len(codes) != 2 {
|
||||
t.Fatal("business code enum absent")
|
||||
}
|
||||
public := operation(t, doc, "/public", "get")
|
||||
if len(public["security"].([]any)) != 0 || public["responses"].(map[string]any)["401"] != nil || public["x-omni-errors"] != nil {
|
||||
t.Fatal("public overrides did not clear group declarations")
|
||||
}
|
||||
}
|
||||
func TestInvalidDeclarations(t *testing.T) {
|
||||
replacements := [][2]string{
|
||||
{`"EMAIL_TAKEN"`, `"UNAUTHENTICATED"`},
|
||||
{`EmailTaken 409`, `EmailTaken 200`},
|
||||
{`EmailTaken 409`, `emailTaken 409`},
|
||||
{`NameTaken 409`, `CodeEmailTaken 409`},
|
||||
{`auth: UserAuth`, `auth: Unknown`},
|
||||
{`errors: Unauthenticated`, `errors: Unknown`},
|
||||
{`middleware: Audit`, `middleware: Auth`},
|
||||
{`type: bearer`, `type: oauth2`},
|
||||
{`in: header`, `in: query`},
|
||||
{`name: X-API-Key`, `name: Authorization`},
|
||||
{`deprecated: true`, `deprecated: yes`},
|
||||
{`tags: users, administration`, `tags: users, users`},
|
||||
{`@errors Forbidden, EmailTaken, NameTaken`, `@errors Forbidden, Forbidden`},
|
||||
{`middleware: Auth`, `middleware: Defaults`},
|
||||
}
|
||||
for _, change := range replacements {
|
||||
spec, err := parseText(t, strings.Replace(declaredAPI, change[0], change[1], 1))
|
||||
if err == nil {
|
||||
_, err = Compile(spec)
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatalf("accepted invalid %q", change[1])
|
||||
}
|
||||
if !strings.Contains(err.Error(), "service.api:") {
|
||||
t.Fatalf("source position missing: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestImportedErrorCatalog(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
writeFile(t, filepath.Join(root, "errors.api"), `error Conflict 409 "CONFLICT" "conflict"`)
|
||||
writeFile(t, filepath.Join(root, "service.api"), "import \"errors.api\"\nservice imported-api { @errors Conflict\n @handler create\n post /users\n }")
|
||||
spec, err := Parse(filepath.Join(root, "service.api"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = Compile(spec); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestDryRunAndEntryDiscovery(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
source := filepath.Join(root, "input/service.api")
|
||||
writeFile(t, source, declaredAPI)
|
||||
dir := filepath.Join(root, "project")
|
||||
o := Options{API: source, Dir: dir, Module: "example.com/dx", FrameworkDir: frameworkRoot(t), DryRun: true}
|
||||
out, err := Generate(o)
|
||||
if err != nil || len(out.Changes) == 0 {
|
||||
t.Fatalf("empty plan: %v", err)
|
||||
}
|
||||
if _, err = os.Stat(dir); !os.IsNotExist(err) {
|
||||
t.Fatal("dry-run created directory")
|
||||
}
|
||||
o.DryRun = false
|
||||
if _, err = Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
entry, err := ProjectEntry(dir)
|
||||
if err != nil || entry != filepath.Join(dir, "api/service.api") {
|
||||
t.Fatalf("entry: %s %v", entry, err)
|
||||
}
|
||||
o = Options{API: entry, Dir: dir, DryRun: true}
|
||||
before := snapshot(t, dir)
|
||||
out, err = Generate(o)
|
||||
if err != nil || len(out.Changes) != 0 {
|
||||
t.Fatalf("unchanged project plan: %v %+v", err, out.Changes)
|
||||
}
|
||||
writeFile(t, entry, strings.Replace(declaredAPI, "email already in use", "email is already registered", 1))
|
||||
before = snapshot(t, dir)
|
||||
out, err = Generate(o)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("dry-run changed project")
|
||||
}
|
||||
actions := map[string]string{}
|
||||
for _, c := range out.Changes {
|
||||
actions[c.Path] = c.Action
|
||||
}
|
||||
if actions["internal/apierrors/errorsGen.go"] != "update" || actions["docs/openapi.json"] != "update" {
|
||||
t.Fatalf("missing plan: %v", actions)
|
||||
}
|
||||
o.Check = true
|
||||
if _, err = Generate(o); err == nil {
|
||||
t.Fatal("check and dry-run combined")
|
||||
}
|
||||
// A malformed manifest cannot redirect the shorthand outside its project.
|
||||
writeFile(t, filepath.Join(dir, manifestPath), `{"version":1,"entry":"../input/service.api"}`)
|
||||
if _, err = ProjectEntry(dir); err == nil {
|
||||
t.Fatal("unsafe manifest entry accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeclaredContractConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
|
||||
t.Skip("consumer integration")
|
||||
}
|
||||
root := t.TempDir()
|
||||
source := filepath.Join(root, "input/service.api")
|
||||
writeFile(t, source, declaredAPI)
|
||||
dir := filepath.Join(root, "project")
|
||||
if _, err := Generate(Options{API: source, Dir: dir, Module: "example.com/dx", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "contract_test.go"), declaredConsumerTest)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||
if out, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("consumer: %v\n%s", err, out)
|
||||
}
|
||||
}
|
||||
|
||||
const declaredConsumerTest = `package dx_test
|
||||
import (
|
||||
"context";"errors";"io";"net/http/httptest";"strings";"testing"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"example.com/dx/internal/apierrors"
|
||||
"example.com/dx/internal/handler"
|
||||
"example.com/dx/internal/servicecontext"
|
||||
)
|
||||
func TestRuntimeAuthAndFreshErrors(t *testing.T){
|
||||
first:=apierrors.EmailTaken().WithCause(errors.New("private"));second:=apierrors.EmailTaken()
|
||||
if first==second||second.Cause!=nil||second.Code!=apierrors.CodeEmailTaken||second.Status!=409{t.Fatal("shared error instance")}
|
||||
cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close()
|
||||
deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close()
|
||||
app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
|
||||
w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,httptest.NewRequest("POST","/users",nil))
|
||||
if w.Code!=501||!strings.Contains(w.Body.String(),"MIDDLEWARE_NOT_IMPLEMENTED"){t.Fatal("unimplemented auth did not fail closed")}
|
||||
var called []string
|
||||
deps.Middlewares["Auth"]=func(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error{called=append(called,"auth");if c.Header("Authorization")!="Bearer valid"{return apierrors.Unauthenticated()};return next(c)}}
|
||||
deps.Middlewares["Audit"]=func(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error{called=append(called,"audit");return next(c)}}
|
||||
app2:=omni.New();defer app2.Close();if err=handler.RegisterRoutes(app2,deps);err!=nil{t.Fatal(err)}
|
||||
w=httptest.NewRecorder();app2.Engine().ServeHTTP(w,httptest.NewRequest("POST","/users",nil))
|
||||
if w.Code!=401||strings.Join(called,",")!="auth"{t.Fatal("unauthenticated request reached other middleware")}
|
||||
called=nil;req:=httptest.NewRequest("POST","/users",nil);req.Header.Set("Authorization","Bearer valid")
|
||||
w=httptest.NewRecorder();app2.Engine().ServeHTTP(w,req)
|
||||
if w.Code!=501||strings.Join(called,",")!="auth,audit"{t.Fatal("middleware order changed")}
|
||||
called=nil;w=httptest.NewRecorder();app2.Engine().ServeHTTP(w,httptest.NewRequest("GET","/public",nil))
|
||||
if w.Code!=501||strings.Join(called,",")!="audit"||strings.Contains(w.Body.String(),"MIDDLEWARE_NOT_IMPLEMENTED"){t.Fatal("public override retained auth")}
|
||||
delete(deps.Middlewares,"Auth");app3:=omni.New();defer app3.Close()
|
||||
if err=handler.RegisterRoutes(app3,deps);err==nil{t.Fatal("missing auth registration accepted")}
|
||||
w=httptest.NewRecorder();app3.Engine().ServeHTTP(w,httptest.NewRequest("GET","/public",nil))
|
||||
if w.Code!=404{t.Fatal("failed auth preflight registered partial routes")}
|
||||
}
|
||||
`
|
||||
|
||||
func TestAuthMetadataWithParameterNamePrefixes(t *testing.T) {
|
||||
source := "type Req {\n ID int `path:\"id\"`\n IDLong int `path:\"idLong\"`\n}\nauth Gate (\n type: bearer\n middleware: Auth\n)\n@server (auth: Gate)\nservice prefix-api {\n @handler read\n get /items/:id/children/:idLong (Req)\n}"
|
||||
spec, err := parseText(t, source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
op := operation(t, document(t, c), "/items/{id}/children/{idLong}", "get")
|
||||
if len(op["security"].([]any)) != 1 {
|
||||
t.Fatal("auth metadata lost")
|
||||
}
|
||||
}
|
||||
532
cmd/go-omni/apigen/parse.go
Normal file
532
cmd/go-omni/apigen/parse.go
Normal file
@@ -0,0 +1,532 @@
|
||||
// Package apigen implements go-omni's deliberately bounded go-zero-style API
|
||||
// dialect. It generates Omni handlers, never go-zero or transport dependencies.
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"text/scanner"
|
||||
)
|
||||
|
||||
type Spec struct {
|
||||
Title, Version, Service, Description string
|
||||
Errors []ErrorDefinition
|
||||
Auth []AuthDefinition
|
||||
Responses []ResponseDefinition
|
||||
Types []Type
|
||||
Routes []Route
|
||||
Sources map[string][]byte // relative to the entry API's directory
|
||||
Entry string
|
||||
}
|
||||
type Type struct {
|
||||
Name string
|
||||
Fields []Field
|
||||
Position string
|
||||
}
|
||||
type Field struct{ Name, Type, Tag string }
|
||||
type Route struct {
|
||||
Method, Path, Request, Response, Handler, Group, Summary string
|
||||
Middleware []string
|
||||
Auth string
|
||||
ResponseFormat string
|
||||
ErrorResponseFormat string
|
||||
Errors []string
|
||||
Description string
|
||||
Tags []string
|
||||
Deprecated bool
|
||||
Status int
|
||||
Position string
|
||||
}
|
||||
type token struct {
|
||||
text, position string
|
||||
kind rune
|
||||
space bool
|
||||
}
|
||||
type parser struct {
|
||||
tokens []token
|
||||
at int
|
||||
spec *Spec
|
||||
imports []string
|
||||
}
|
||||
|
||||
func Parse(path string) (*Spec, error) {
|
||||
entry, err := filepath.Abs(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entry, err = filepath.EvalSymlinks(entry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
root := filepath.Dir(entry)
|
||||
spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||
visiting, visited := map[string]bool{}, map[string]bool{}
|
||||
total := 0
|
||||
var visit func(string, int) error
|
||||
visit = func(name string, depth int) error {
|
||||
if depth > 32 || len(visited)+len(visiting) >= 64 {
|
||||
return errors.New("api: import limit exceeded")
|
||||
}
|
||||
real, err := filepath.EvalSymlinks(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel, err := filepath.Rel(root, real)
|
||||
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return errors.New("api: imports must stay within the entry API directory")
|
||||
}
|
||||
if filepath.Ext(real) != ".api" {
|
||||
return errors.New("api: expected .api file")
|
||||
}
|
||||
if visiting[real] {
|
||||
return fmt.Errorf("api: circular import %s", rel)
|
||||
}
|
||||
if visited[real] {
|
||||
return nil
|
||||
}
|
||||
visiting[real] = true
|
||||
defer delete(visiting, real)
|
||||
f, err := os.Open(real)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
_ = f.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
total += len(data)
|
||||
if len(data) > 1<<20 || total > 8<<20 {
|
||||
return errors.New("api: source size limit exceeded")
|
||||
}
|
||||
tokens, err := lex(rel, string(data))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p := &parser{tokens: tokens, spec: spec}
|
||||
if err = p.parse(); err != nil {
|
||||
return err
|
||||
}
|
||||
spec.Sources[filepath.ToSlash(rel)] = data
|
||||
for _, imp := range p.imports {
|
||||
if filepath.IsAbs(imp) || strings.Contains(imp, "\\") {
|
||||
return errors.New("api: imports must use relative slash paths")
|
||||
}
|
||||
if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
visited[real] = true
|
||||
return nil
|
||||
}
|
||||
if err = visit(entry, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if spec.Service == "" || len(spec.Routes) == 0 {
|
||||
return nil, errors.New("api: at least one service route required")
|
||||
}
|
||||
if spec.Title == "" {
|
||||
spec.Title = spec.Service
|
||||
}
|
||||
return spec, nil
|
||||
}
|
||||
func lex(name, source string) ([]token, error) {
|
||||
var s scanner.Scanner
|
||||
s.Init(strings.NewReader(source))
|
||||
s.Filename = name
|
||||
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments
|
||||
s.Whitespace ^= 1 << '\n'
|
||||
var scanErr error
|
||||
s.Error = func(s *scanner.Scanner, message string) {
|
||||
if scanErr == nil {
|
||||
scanErr = fmt.Errorf("%s: %s", s.Position, message)
|
||||
}
|
||||
}
|
||||
var out []token
|
||||
end := 0
|
||||
for {
|
||||
kind := s.Scan()
|
||||
out = append(out, token{s.TokenText(), s.Position.String(), kind, s.Position.Offset > end})
|
||||
end = s.Position.Offset + len(s.TokenText())
|
||||
if kind == scanner.EOF {
|
||||
break
|
||||
}
|
||||
}
|
||||
return out, scanErr
|
||||
}
|
||||
func (p *parser) peek() string { return p.tokens[p.at].text }
|
||||
func (p *parser) next() token {
|
||||
t := p.tokens[p.at]
|
||||
if t.kind != scanner.EOF {
|
||||
p.at++
|
||||
}
|
||||
return t
|
||||
}
|
||||
func (p *parser) skip() {
|
||||
for p.peek() == "\n" || p.peek() == ";" {
|
||||
p.next()
|
||||
}
|
||||
}
|
||||
func (p *parser) err(message string) error {
|
||||
return fmt.Errorf("%s: %s (got %q)", p.tokens[p.at].position, message, p.peek())
|
||||
}
|
||||
func (p *parser) expect(s string) error {
|
||||
if p.peek() != s {
|
||||
return p.err("expected " + strconv.Quote(s))
|
||||
}
|
||||
p.next()
|
||||
return nil
|
||||
}
|
||||
func (p *parser) ident() (string, error) {
|
||||
if p.tokens[p.at].kind != scanner.Ident {
|
||||
return "", p.err("expected identifier")
|
||||
}
|
||||
return p.next().text, nil
|
||||
}
|
||||
func (p *parser) quoted() (string, error) {
|
||||
if p.tokens[p.at].kind != scanner.String {
|
||||
return "", p.err("expected quoted string")
|
||||
}
|
||||
return strconv.Unquote(p.next().text)
|
||||
}
|
||||
func (p *parser) parse() error {
|
||||
server := map[string]string{}
|
||||
syntaxSeen := false
|
||||
for p.skip(); p.tokens[p.at].kind != scanner.EOF; p.skip() {
|
||||
switch p.peek() {
|
||||
case "syntax":
|
||||
if syntaxSeen {
|
||||
return p.err("duplicate syntax")
|
||||
}
|
||||
syntaxSeen = true
|
||||
p.next()
|
||||
if err := p.expect("="); err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if v != "v1" {
|
||||
return p.err("only syntax v1 supported")
|
||||
}
|
||||
case "info":
|
||||
p.next()
|
||||
m, err := p.metadata()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, a := m["desc"]; a {
|
||||
if _, b := m["description"]; b {
|
||||
return p.err("use only one of info desc and description")
|
||||
}
|
||||
}
|
||||
for k, v := range m {
|
||||
switch k {
|
||||
case "title":
|
||||
p.spec.Title = v
|
||||
case "version":
|
||||
p.spec.Version = v
|
||||
case "desc", "description":
|
||||
p.spec.Description = v
|
||||
case "author", "email":
|
||||
default:
|
||||
return p.err("unsupported info property " + k)
|
||||
}
|
||||
}
|
||||
case "import":
|
||||
p.next()
|
||||
if p.peek() == "(" {
|
||||
p.next()
|
||||
for p.skip(); p.peek() != ")"; p.skip() {
|
||||
v, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.imports = append(p.imports, v)
|
||||
}
|
||||
p.next()
|
||||
} else {
|
||||
v, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.imports = append(p.imports, v)
|
||||
}
|
||||
case "error":
|
||||
if err := p.parseErrors(); err != nil {
|
||||
return err
|
||||
}
|
||||
case "auth":
|
||||
if err := p.parseAuth(); err != nil {
|
||||
return err
|
||||
}
|
||||
case "response":
|
||||
if err := p.parseResponse(); err != nil {
|
||||
return err
|
||||
}
|
||||
case "type":
|
||||
p.next()
|
||||
if p.peek() == "(" {
|
||||
p.next()
|
||||
for p.skip(); p.peek() != ")"; p.skip() {
|
||||
if err := p.parseType(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
p.next()
|
||||
} else {
|
||||
if err := p.parseType(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case "@":
|
||||
p.next()
|
||||
if err := p.expect("server"); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(server) != 0 {
|
||||
return p.err("duplicate server block")
|
||||
}
|
||||
m, err := p.metadata()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
server = m
|
||||
for k := range m {
|
||||
if k != "prefix" && k != "group" && k != "middleware" && k != "auth" && k != "errors" && k != "response" {
|
||||
return p.err("unsupported @server option " + k + "; wire authentication through named middleware")
|
||||
}
|
||||
}
|
||||
case "service":
|
||||
if err := p.parseService(server); err != nil {
|
||||
return err
|
||||
}
|
||||
server = map[string]string{}
|
||||
default:
|
||||
return p.err("unsupported declaration")
|
||||
}
|
||||
}
|
||||
if len(server) > 0 {
|
||||
return p.err("@server must precede a service")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (p *parser) metadata() (map[string]string, error) {
|
||||
if err := p.expect("("); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := map[string]string{}
|
||||
for p.skip(); p.peek() != ")"; p.skip() {
|
||||
key, err := p.ident()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = p.expect(":"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
value := ""
|
||||
if p.tokens[p.at].kind == scanner.String {
|
||||
value, err = p.quoted()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else if p.peek() == "{" {
|
||||
depth := 0
|
||||
var object strings.Builder
|
||||
for {
|
||||
t := p.next()
|
||||
if t.kind == scanner.EOF {
|
||||
return nil, p.err("unterminated metadata object")
|
||||
}
|
||||
if t.text == "{" {
|
||||
depth++
|
||||
}
|
||||
if t.text == "}" {
|
||||
depth--
|
||||
}
|
||||
if t.space {
|
||||
object.WriteByte(' ')
|
||||
}
|
||||
object.WriteString(t.text)
|
||||
if depth == 0 {
|
||||
break
|
||||
}
|
||||
}
|
||||
value = object.String()
|
||||
} else {
|
||||
for p.peek() != "\n" && p.peek() != ";" && p.peek() != ")" && p.tokens[p.at].kind != scanner.EOF {
|
||||
value += p.next().text
|
||||
}
|
||||
}
|
||||
if value == "" {
|
||||
return nil, p.err("metadata value required")
|
||||
}
|
||||
if _, exists := out[key]; exists {
|
||||
return nil, p.err("duplicate metadata " + key)
|
||||
}
|
||||
out[key] = value
|
||||
}
|
||||
p.next()
|
||||
return out, nil
|
||||
}
|
||||
func (p *parser) parseType() error {
|
||||
pos := p.tokens[p.at].position
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.skip()
|
||||
if err = p.expect("{"); err != nil {
|
||||
return err
|
||||
}
|
||||
typ := Type{Name: name, Position: pos}
|
||||
for p.skip(); p.peek() != "}"; p.skip() {
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
expression := ""
|
||||
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" && p.tokens[p.at].kind != scanner.RawString && p.tokens[p.at].kind != scanner.EOF {
|
||||
expression += p.next().text
|
||||
}
|
||||
if expression == "" {
|
||||
return p.err("field type required; embedded types are not supported")
|
||||
}
|
||||
if p.tokens[p.at].kind != scanner.RawString {
|
||||
return p.err("field source tag required")
|
||||
}
|
||||
tag, err := strconv.Unquote(p.next().text)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag})
|
||||
}
|
||||
p.next()
|
||||
p.spec.Types = append(p.spec.Types, typ)
|
||||
return nil
|
||||
}
|
||||
func (p *parser) parseService(server map[string]string) error {
|
||||
p.next()
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for p.peek() == "-" {
|
||||
p.next()
|
||||
part, e := p.ident()
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
name += "-" + part
|
||||
}
|
||||
if p.spec.Service != "" && p.spec.Service != name {
|
||||
return p.err("one service name per entry API is supported")
|
||||
}
|
||||
p.spec.Service = name
|
||||
p.skip()
|
||||
if err = p.expect("{"); err != nil {
|
||||
return err
|
||||
}
|
||||
for p.skip(); p.peek() != "}"; p.skip() {
|
||||
route := Route{ResponseFormat: server["response"], Auth: server["auth"], Errors: commaList(server["errors"]), Group: server["group"], Status: 200, Position: p.tokens[p.at].position}
|
||||
if route.Group == "" {
|
||||
route.Group = "api"
|
||||
}
|
||||
if server["middleware"] != "" {
|
||||
route.Middleware = strings.Split(server["middleware"], ",")
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for p.peek() == "@" {
|
||||
p.next()
|
||||
annotation, e := p.ident()
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
if seen[annotation] {
|
||||
return p.err("duplicate route annotation")
|
||||
}
|
||||
seen[annotation] = true
|
||||
switch annotation {
|
||||
case "handler":
|
||||
route.Handler, e = p.ident()
|
||||
case "doc":
|
||||
e = p.parseDoc(&route)
|
||||
case "auth":
|
||||
route.Auth, e = p.ident()
|
||||
case "response":
|
||||
var format string
|
||||
format, e = p.ident()
|
||||
if format == "raw" && route.ResponseFormat != "raw" {
|
||||
route.ErrorResponseFormat = route.ResponseFormat
|
||||
}
|
||||
route.ResponseFormat = format
|
||||
case "errors":
|
||||
var names []string
|
||||
names, e = p.names()
|
||||
if len(names) == 1 && names[0] == "none" {
|
||||
route.Errors = nil
|
||||
} else {
|
||||
route.Errors = mergeNames(route.Errors, names)
|
||||
}
|
||||
case "status":
|
||||
route.Status, e = strconv.Atoi(p.next().text)
|
||||
default:
|
||||
return p.err("unsupported route annotation " + annotation)
|
||||
}
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
p.skip()
|
||||
}
|
||||
if route.Handler == "" {
|
||||
return p.err("@handler required")
|
||||
}
|
||||
route.Method, err = p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
path := ""
|
||||
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "(" && p.peek() != "returns" && p.peek() != "}" && p.tokens[p.at].kind != scanner.EOF {
|
||||
path += p.next().text
|
||||
}
|
||||
route.Path = server["prefix"] + path
|
||||
if p.peek() == "(" {
|
||||
route.Request, err = p.routeType()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if p.peek() == "returns" {
|
||||
p.next()
|
||||
route.Response, err = p.routeType()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" {
|
||||
return p.err("unexpected route suffix")
|
||||
}
|
||||
if route.Response == "" && !seen["status"] {
|
||||
route.Status = 204
|
||||
}
|
||||
p.spec.Routes = append(p.spec.Routes, route)
|
||||
}
|
||||
p.next()
|
||||
return nil
|
||||
}
|
||||
func (p *parser) routeType() (string, error) {
|
||||
if err := p.expect("("); err != nil {
|
||||
return "", err
|
||||
}
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, p.expect(")")
|
||||
}
|
||||
73
cmd/go-omni/apigen/profile.go
Normal file
73
cmd/go-omni/apigen/profile.go
Normal file
@@ -0,0 +1,73 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
type projectProfile struct {
|
||||
Database string `json:"database"`
|
||||
Storage string `json:"storage"`
|
||||
LogOutput string `json:"logOutput"`
|
||||
LogFormat string `json:"logFormat"`
|
||||
}
|
||||
|
||||
func (o Options) profile() projectProfile {
|
||||
return projectProfile{o.Database, o.Storage, o.LogOutput, o.LogFormat}
|
||||
}
|
||||
|
||||
// Infrastructure files belong to the developer. Persist the original selection
|
||||
// so a plain regeneration cannot silently change adapters or repair a missing
|
||||
// file with settings from another profile.
|
||||
func resolveProfile(o *Options) error {
|
||||
defaults := projectProfile{"none", "none", "stderr", "json"}
|
||||
data, err := os.ReadFile(filepath.Join(o.Dir, manifestPath))
|
||||
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
existing := err == nil
|
||||
if existing {
|
||||
var previous manifest
|
||||
if err = json.Unmarshal(data, &previous); err != nil {
|
||||
return err
|
||||
}
|
||||
if previous.Profile != nil {
|
||||
defaults = *previous.Profile
|
||||
}
|
||||
}
|
||||
if o.Database == "memory" {
|
||||
o.Database = "none"
|
||||
}
|
||||
for _, item := range []struct {
|
||||
name string
|
||||
value *string
|
||||
fallback string
|
||||
}{
|
||||
{"database", &o.Database, defaults.Database}, {"storage", &o.Storage, defaults.Storage},
|
||||
{"log-output", &o.LogOutput, defaults.LogOutput}, {"log-format", &o.LogFormat, defaults.LogFormat},
|
||||
} {
|
||||
if *item.value == "" {
|
||||
*item.value = item.fallback
|
||||
} else if existing && *item.value != item.fallback {
|
||||
return fmt.Errorf("--%s differs from the original project profile; infrastructure is developer-owned, generate a new project to compare and migrate it explicitly", item.name)
|
||||
}
|
||||
}
|
||||
if o.Database != "none" && o.Database != "mysql" {
|
||||
return errors.New("--database must be none, memory or mysql")
|
||||
}
|
||||
switch o.Storage {
|
||||
case "none", "local", "s3", "r2", "oss":
|
||||
default:
|
||||
return errors.New("--storage must be none, local, s3, r2 or oss")
|
||||
}
|
||||
if o.LogOutput != "stderr" && o.LogOutput != "stdout" && o.LogOutput != "file" {
|
||||
return errors.New("--log-output must be stdout, stderr or file")
|
||||
}
|
||||
if o.LogFormat != "json" && o.LogFormat != "text" {
|
||||
return errors.New("--log-format must be json or text")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
98
cmd/go-omni/apigen/profile_test.go
Normal file
98
cmd/go-omni/apigen/profile_test.go
Normal file
@@ -0,0 +1,98 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestInfrastructureProfileOwnership(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
o := Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: "mysql", Storage: "local", LogOutput: "file"}
|
||||
if _, err := New(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o = Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := snapshot(t, dir)
|
||||
for _, changed := range []Options{
|
||||
{Database: "none"}, {Storage: "s3"}, {LogOutput: "stdout"}, {LogFormat: "text"},
|
||||
} {
|
||||
changed.Dir, changed.API = dir, o.API
|
||||
if _, err := Generate(changed); err == nil {
|
||||
t.Fatal("profile switch accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("profile rejection changed project")
|
||||
}
|
||||
}
|
||||
// Restoring a missing seed must use the saved profile, not current defaults.
|
||||
if err := os.Remove(filepath.Join(dir, "internal/servicecontext/storage.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Generate(o); err == nil {
|
||||
t.Fatal("check accepted missing infrastructure")
|
||||
}
|
||||
o.Check = false
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("profile reconstruction changed project")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIInfrastructureConsumers(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
|
||||
t.Skip("consumer module integration")
|
||||
}
|
||||
for _, db := range []string{"none", "mysql"} {
|
||||
for _, storage := range []string{"none", "local", "s3", "r2", "oss"} {
|
||||
t.Run(db+"/"+storage, func(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: db, Storage: storage, LogOutput: "file"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
test := `package settings
|
||||
import("testing";"os")
|
||||
func TestConfiguration(t *testing.T){
|
||||
t.Setenv("GO_OMNI_DB_DSN","test:test@tcp(127.0.0.1:1)/test")
|
||||
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY","01234567890123456789012345678901")
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","test");t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","test")
|
||||
t.Setenv("GO_OMNI_STORAGE_BUCKET","test");t.Setenv("GO_OMNI_STORAGE_ENDPOINT","ENDPOINT")
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID","01234567890123456789012345678901")
|
||||
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
|
||||
_=os.Getenv("unused")
|
||||
}`
|
||||
endpoint := ""
|
||||
if storage == "s3" {
|
||||
endpoint = "127.0.0.1:9000"
|
||||
}
|
||||
if storage == "oss" {
|
||||
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "internal/settings/profile_test.go"), strings.ReplaceAll(test, `"ENDPOINT"`, `"`+endpoint+`"`))
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("%v\n%s", err, out)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
35
cmd/go-omni/apigen/projectEntry.go
Normal file
35
cmd/go-omni/apigen/projectEntry.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
// ProjectEntry discovers the API entry recorded by generation, never guessing
|
||||
// which of several .api files is the service entry.
|
||||
func ProjectEntry(directory string) (string, error) {
|
||||
root, err := filepath.Abs(directory)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if err = safePath(root, manifestPath); err != nil {
|
||||
return "", err
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(root, manifestPath))
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("--api is required outside a generated project (no readable %s)", manifestPath)
|
||||
}
|
||||
var m manifest
|
||||
if err = json.Unmarshal(data, &m); err != nil {
|
||||
return "", fmt.Errorf("invalid API project manifest: %w", err)
|
||||
}
|
||||
if m.Version != 1 || filepath.Ext(m.Entry) != ".api" {
|
||||
return "", fmt.Errorf("invalid API entry in %s; pass --api explicitly", manifestPath)
|
||||
}
|
||||
if err = safePath(root, m.Entry); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return filepath.Join(root, filepath.FromSlash(m.Entry)), nil
|
||||
}
|
||||
114
cmd/go-omni/apigen/response.go
Normal file
114
cmd/go-omni/apigen/response.go
Normal file
@@ -0,0 +1,114 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
type ResponseDefinition struct {
|
||||
Name string
|
||||
Config response.Config
|
||||
Position string
|
||||
}
|
||||
|
||||
func (p *parser) parseResponse() error {
|
||||
pos := p.next().position
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.skip()
|
||||
props, err := p.metadata()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
values := map[string]any{}
|
||||
for key, value := range props {
|
||||
switch key {
|
||||
case "successCode", "errorCode":
|
||||
if kind := props["codeType"]; kind == "integer" || kind == "int32" || kind == "int64" {
|
||||
var number json.Number
|
||||
if err = json.Unmarshal([]byte(value), &number); err != nil {
|
||||
return fmt.Errorf("%s: %s must be an integer", pos, key)
|
||||
}
|
||||
values[key] = number
|
||||
} else {
|
||||
values[key] = value
|
||||
}
|
||||
case "codes", "messages", "body", "errorBody":
|
||||
decoder := json.NewDecoder(strings.NewReader(value))
|
||||
decoder.UseNumber()
|
||||
var object map[string]any
|
||||
if err = decoder.Decode(&object); err != nil || object == nil {
|
||||
return fmt.Errorf("%s: %s must be a JSON object", pos, key)
|
||||
}
|
||||
var extra any
|
||||
if decoder.Decode(&extra) != io.EOF {
|
||||
return fmt.Errorf("%s: invalid %s mapping", pos, key)
|
||||
}
|
||||
values[key] = object
|
||||
default:
|
||||
values[key] = value
|
||||
}
|
||||
}
|
||||
data, err := json.Marshal(values)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
decoder := json.NewDecoder(bytes.NewReader(data))
|
||||
decoder.UseNumber()
|
||||
decoder.DisallowUnknownFields()
|
||||
var cfg response.Config
|
||||
if err = decoder.Decode(&cfg); err != nil {
|
||||
return fmt.Errorf("%s: response %s: %w", pos, name, err)
|
||||
}
|
||||
if err = cfg.Validate(); err != nil {
|
||||
return fmt.Errorf("%s: %w", pos, err)
|
||||
}
|
||||
p.spec.Responses = append(p.spec.Responses, ResponseDefinition{Name: name, Config: cfg, Position: pos})
|
||||
return nil
|
||||
}
|
||||
|
||||
func responseConfig(spec *Spec, route Route) response.Config {
|
||||
name := responseName(route)
|
||||
for _, definition := range spec.Responses {
|
||||
if definition.Name == name {
|
||||
return definition.Config
|
||||
}
|
||||
}
|
||||
return response.Config{}
|
||||
}
|
||||
|
||||
func responseName(route Route) string {
|
||||
if route.ResponseFormat == "raw" {
|
||||
return route.ErrorResponseFormat
|
||||
}
|
||||
return route.ResponseFormat
|
||||
}
|
||||
|
||||
func validateResponses(spec *Spec) error {
|
||||
seen := map[string]bool{}
|
||||
for _, definition := range spec.Responses {
|
||||
if !exportedName.MatchString(definition.Name) || seen[definition.Name] {
|
||||
return fmt.Errorf("%s: response template requires a unique UpperCamelCase name", definition.Position)
|
||||
}
|
||||
if err := definition.Config.Validate(); err != nil {
|
||||
return fmt.Errorf("%s: %w", definition.Position, err)
|
||||
}
|
||||
seen[definition.Name] = true
|
||||
}
|
||||
for _, r := range spec.Routes {
|
||||
if r.ResponseFormat != "" && r.ResponseFormat != "raw" && !seen[r.ResponseFormat] {
|
||||
return fmt.Errorf("%s: unknown response template %s", r.Position, r.ResponseFormat)
|
||||
}
|
||||
if r.ErrorResponseFormat != "" && !seen[r.ErrorResponseFormat] {
|
||||
return fmt.Errorf("%s: unknown error response template %s", r.Position, r.ErrorResponseFormat)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
357
cmd/go-omni/apigen/response_test.go
Normal file
357
cmd/go-omni/apigen/response_test.go
Normal file
@@ -0,0 +1,357 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
const configurableResponseAPI = `syntax = "v1"
|
||||
response CommonResponse (
|
||||
codeField: code
|
||||
messageField: msg
|
||||
dataField: data
|
||||
requestIdField: "-"
|
||||
codeType: integer
|
||||
successCode: 200
|
||||
errorCode: 9999
|
||||
successMessage: "操作成功"
|
||||
errorMessage: "系统错误"
|
||||
dataMode: always
|
||||
codes: {
|
||||
"UNAUTHENTICATED": 401001,
|
||||
"EMAIL_TAKEN": 409001
|
||||
}
|
||||
messages: {"UNAUTHENTICATED":"请先登录","EMAIL_TAKEN":"邮箱已存在"}
|
||||
)
|
||||
response PartnerResponse (
|
||||
codeField: result
|
||||
messageField: notice
|
||||
dataField: payload
|
||||
requestIdField: trace
|
||||
successCode: "PASS"
|
||||
)
|
||||
error (
|
||||
Unauthenticated 401 "UNAUTHENTICATED" "authentication required"
|
||||
EmailTaken 409 "EMAIL_TAKEN" "email already taken"
|
||||
)
|
||||
auth Access (type: bearer
|
||||
middleware: Auth
|
||||
)
|
||||
type ItemResponse {
|
||||
Name string ` + "`json:\"name\"`" + `
|
||||
}
|
||||
@server (
|
||||
group: items
|
||||
response: CommonResponse
|
||||
auth: Access
|
||||
errors: Unauthenticated
|
||||
)
|
||||
service response-api {
|
||||
@errors EmailTaken
|
||||
@handler item
|
||||
get /item returns (ItemResponse)
|
||||
@response raw
|
||||
@handler callback
|
||||
post /callback returns (ItemResponse)
|
||||
@response PartnerResponse
|
||||
@handler partner
|
||||
get /partner returns (ItemResponse)
|
||||
}
|
||||
`
|
||||
|
||||
func TestResponseTemplatesDocumentAndValidation(t *testing.T) {
|
||||
spec, err := parseText(t, configurableResponseAPI)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
doc := document(t, c)
|
||||
getSchema := func(path, status string) map[string]any {
|
||||
method := "get"
|
||||
if path == "/callback" {
|
||||
method = "post"
|
||||
}
|
||||
return operation(t, doc, path, method)["responses"].(map[string]any)[status].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
|
||||
}
|
||||
properties := getSchema("/item", "200")["properties"].(map[string]any)
|
||||
if len(properties) != 3 || properties["code"].(map[string]any)["type"] != "integer" || properties["msg"] == nil || properties["requestId"] != nil {
|
||||
t.Fatal(properties)
|
||||
}
|
||||
if getSchema("/callback", "200")["properties"].(map[string]any)["name"] == nil {
|
||||
t.Fatal("raw JSON was wrapped")
|
||||
}
|
||||
if getSchema("/callback", "401")["properties"].(map[string]any)["msg"] == nil {
|
||||
t.Fatal("raw override lost group error policy")
|
||||
}
|
||||
if getSchema("/partner", "200")["properties"].(map[string]any)["payload"] == nil {
|
||||
t.Fatal("route override not applied")
|
||||
}
|
||||
conflict := operation(t, doc, "/item", "get")["responses"].(map[string]any)["409"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
|
||||
value := conflict["examples"].(map[string]any)["EmailTaken"].(map[string]any)["value"].(map[string]any)
|
||||
if value["code"] != float64(409001) || value["msg"] != "邮箱已存在" || len(value) != 3 {
|
||||
t.Fatal(value)
|
||||
}
|
||||
if got := conflict["schema"].(map[string]any)["properties"].(map[string]any)["code"].(map[string]any)["enum"].([]any); got[0] != float64(409001) {
|
||||
t.Fatal(got)
|
||||
}
|
||||
for _, change := range [][2]string{{"response: CommonResponse", "response: Missing"}, {"messageField: msg", "messageField: code"}, {"codeType: integer", "codeType: boolean"}, {"successCode: 200", "successCode: 2.5"}, {"requestIdField: \"-\"", "unknownOption: value"}, {"@response raw", "@response missing"}} {
|
||||
invalid, err := parseText(t, strings.Replace(configurableResponseAPI, change[0], change[1], 1))
|
||||
if err == nil {
|
||||
_, err = Compile(invalid)
|
||||
}
|
||||
if err == nil || !strings.Contains(err.Error(), "service.api:") {
|
||||
t.Fatalf("bad declaration %s: %v", change[1], err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestImportedResponseTemplate(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
writeFile(t, filepath.Join(dir, "common.api"), "response Shared (messageField: msg)\n")
|
||||
writeFile(t, filepath.Join(dir, "service.api"), "import \"common.api\"\n@server (response: Shared)\nservice shared-api {\n@handler ping\nget /ping\n}")
|
||||
spec, err := Parse(filepath.Join(dir, "service.api"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = Compile(spec); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponseNestedBodyDocument(t *testing.T) {
|
||||
definition := `response Nested (
|
||||
body: {"success":"${ok}","result":"${data}","meta":{"status":"${status}","trace":"${requestId}"},"version":1}
|
||||
errorBody: {"success":"${ok}","error":{"id":"${code}","reason":"${message}"}}
|
||||
codeType: integer
|
||||
successCode: 0
|
||||
errorCode: 9999
|
||||
codes: {"EMAIL_TAKEN":409001}
|
||||
)
|
||||
`
|
||||
spec, err := parseText(t, definition+strings.Replace(configurableResponseAPI, "response: CommonResponse", "response: Nested", 1))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
compiled, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
doc := document(t, compiled)
|
||||
responses := operation(t, doc, "/item", "get")["responses"].(map[string]any)
|
||||
success := responses["200"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)["properties"].(map[string]any)
|
||||
if success["code"] != nil || success["result"] == nil || success["success"].(map[string]any)["type"] != "boolean" {
|
||||
t.Fatal(success)
|
||||
}
|
||||
media := responses["409"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)
|
||||
schema := media["schema"].(map[string]any)["properties"].(map[string]any)
|
||||
if schema["error"].(map[string]any)["properties"].(map[string]any)["id"].(map[string]any)["type"] != "integer" || schema["result"] != nil {
|
||||
t.Fatal(schema)
|
||||
}
|
||||
example := media["examples"].(map[string]any)["EmailTaken"].(map[string]any)["value"].(map[string]any)
|
||||
if example["success"] != false || example["error"].(map[string]any)["id"] != float64(409001) {
|
||||
t.Fatal(example)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponseTemplateJSONTokens(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
body string
|
||||
valid bool
|
||||
}{
|
||||
{`{"scale":-1.5,"exponent":1e-3,"items":[true,null,"${data}"],"label":"a b"}`, true},
|
||||
{`{"scale":1 2}`, false}, {`{"scale":- 12}`, false},
|
||||
{`{"scale":tru e}`, false}, {`{"scale":1. 2}`, false},
|
||||
} {
|
||||
text := "response Shared (\nbody: " + tc.body + "\n)\n@server (response: Shared)\nservice test-api {\n@handler ping\nget /ping\n}\n"
|
||||
spec, err := parseText(t, text)
|
||||
if err == nil {
|
||||
_, err = Compile(spec)
|
||||
}
|
||||
if (err == nil) != tc.valid {
|
||||
t.Fatalf("%s: %v", tc.body, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponseCodeTypeDeclarations(t *testing.T) {
|
||||
for _, kind := range []string{"int32", "int64"} {
|
||||
source := strings.Replace(configurableResponseAPI, "codeType: integer", "codeType: "+kind, 1)
|
||||
spec, err := parseText(t, source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
compiled, err := Compile(spec)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
doc := document(t, compiled)
|
||||
for _, status := range []string{"200", "401", "409", "500"} {
|
||||
responses := operation(t, doc, "/item", "get")["responses"].(map[string]any)
|
||||
schema := responses[status].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
|
||||
code := schema["properties"].(map[string]any)["code"].(map[string]any)
|
||||
if code["type"] != "integer" || code["format"] != kind {
|
||||
t.Fatal(code)
|
||||
}
|
||||
}
|
||||
for _, bad := range []string{"2.5", "9223372036854775808", "-9223372036854775809"} {
|
||||
invalid := strings.Replace(source, "successCode: 200", "successCode: "+bad, 1)
|
||||
if _, err := parseText(t, invalid); err == nil || !strings.Contains(err.Error(), "service.api:") {
|
||||
t.Fatalf("%s %s: %v", kind, bad, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
invalid := strings.Replace(configurableResponseAPI, "codeType: integer", "codeType: int32", 1)
|
||||
invalid = strings.Replace(invalid, `"EMAIL_TAKEN": 409001`, `"EMAIL_TAKEN": 2147483648`, 1)
|
||||
if _, err := parseText(t, invalid); err == nil {
|
||||
t.Fatal("overflowing int32 mapping accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResponseTemplateConsumer(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
|
||||
t.Skip("consumer integration")
|
||||
}
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
source := filepath.Join(t.TempDir(), "service.api")
|
||||
// Use an explicitly declared public body, rather than field-name shortcuts.
|
||||
sourceText := strings.Replace(configurableResponseAPI, " codeField: code\n messageField: msg\n dataField: data\n requestIdField: \"-\"", ` body: {"code":"${code}","msg":"${message}","data":"${data}"}`, 1)
|
||||
sourceText = strings.Replace(sourceText, "codeType: integer", "codeType: int64", 1)
|
||||
sourceText = strings.Replace(sourceText, "successCode: 200", "successCode: 9223372036854775807", 1)
|
||||
writeFile(t, source, sourceText)
|
||||
releaseCLI := os.Getenv("GO_OMNI_RELEASE_CLI")
|
||||
generate := func(api string, initial bool) {
|
||||
t.Helper()
|
||||
if releaseCLI != "" {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, releaseCLI, "api", "generate", "--api", api, "--dir", dir, "--module", "example.com/responseConsumer")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("released CLI: %v\n%s", err, output)
|
||||
}
|
||||
return
|
||||
}
|
||||
o := Options{API: api, Dir: dir}
|
||||
if initial {
|
||||
o.Module = "example.com/responseConsumer"
|
||||
o.FrameworkDir = frameworkRoot(t)
|
||||
}
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
generate(source, true)
|
||||
writeFile(t, filepath.Join(dir, "response_test.go"), responseConsumerTest)
|
||||
for _, method := range []string{"Item", "Callback", "Partner"} {
|
||||
body := `return &types.ItemResponse{Name:"` + strings.ToLower(method) + `"},nil`
|
||||
if method == "Partner" {
|
||||
body = `return nil,apierrors.EmailTaken()`
|
||||
}
|
||||
imports := `"context";"example.com/responseConsumer/internal/types"`
|
||||
if method == "Partner" {
|
||||
imports += `;"example.com/responseConsumer/internal/apierrors"`
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "internal/service/items", strings.ToLower(method)+".go"), "package items\nimport("+imports+")\nfunc(s *Service)"+method+"(ctx context.Context)(*types.ItemResponse,error){"+body+"}\n")
|
||||
}
|
||||
before := snapshot(t, dir)
|
||||
generate(filepath.Join(dir, "api/service.api"), false)
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("regeneration changed hand-written files")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("consumer: %v\n%s", err, output)
|
||||
}
|
||||
if releaseCLI != "" {
|
||||
cmd = exec.CommandContext(ctx, "go", "list", "-m", "-json", frameworkModule)
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("released module: %v\n%s", err, output)
|
||||
}
|
||||
var module struct {
|
||||
Version, Sum string
|
||||
Replace any
|
||||
}
|
||||
if err = json.Unmarshal(output, &module); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if module.Version != os.Getenv("GO_OMNI_RELEASE_VERSION") || module.Replace != nil || module.Sum == "" {
|
||||
t.Fatalf("expected fixed remote module without replacement: %+v", module)
|
||||
}
|
||||
t.Logf("remote framework %s checksum %s; no local replace", module.Version, module.Sum)
|
||||
}
|
||||
manifestBytes, err := os.ReadFile(filepath.Join(dir, manifestPath))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var manifest manifest
|
||||
if err = json.Unmarshal(manifestBytes, &manifest); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if manifest.Generated["internal/handler/responseGen.go"] == "" {
|
||||
t.Fatal("response policy not ownership-managed")
|
||||
}
|
||||
}
|
||||
|
||||
const responseConsumerTest = `package responseConsumer_test
|
||||
import (
|
||||
"bytes";"encoding/json";"io";"net/http/httptest";"testing"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
"example.com/responseConsumer/internal/apierrors"
|
||||
"example.com/responseConsumer/internal/handler"
|
||||
item "example.com/responseConsumer/internal/handler/items"
|
||||
"example.com/responseConsumer/internal/types"
|
||||
"context"
|
||||
"example.com/responseConsumer/internal/servicecontext"
|
||||
)
|
||||
type service struct{}
|
||||
func(service)Item(context.Context)(*types.ItemResponse,error){return &types.ItemResponse{Name:"item"},nil}
|
||||
func(service)Callback(context.Context)(*types.ItemResponse,error){return &types.ItemResponse{Name:"callback"},nil}
|
||||
func(service)Partner(context.Context)(*types.ItemResponse,error){return nil,apierrors.EmailTaken()}
|
||||
func TestResponseContract(t *testing.T){
|
||||
app:=omni.New();defer app.Close()
|
||||
cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close()
|
||||
deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close()
|
||||
deps.Middlewares["Auth"]=func(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error{if c.Header("Authorization")!="Bearer valid"{return apierrors.Unauthenticated()};return next(c)}}
|
||||
if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
|
||||
// Registered routes use the compiled policy before even running authentication.
|
||||
for _,path:=range []string{"/item","/callback"}{
|
||||
method:="GET";if path=="/callback"{method="POST"};w:=httptest.NewRecorder();app.ServeHTTP(w,httptest.NewRequest(method,path,nil))
|
||||
var value map[string]any;json.Unmarshal(w.Body.Bytes(),&value)
|
||||
if w.Code!=401||value["code"]!=float64(401001)||value["msg"]!="请先登录"||len(value)!=3{t.Fatalf("%s: %s",path,w.Body.String())}
|
||||
}
|
||||
// Execute the real generated registration and user-owned service files.
|
||||
for _,tc:=range []struct{method,path string;status int;key string}{{"GET","/item",200,"data"},{"POST","/callback",200,"name"},{"GET","/partner",409,"result"}}{
|
||||
w:=httptest.NewRecorder();r:=httptest.NewRequest(tc.method,tc.path,nil);r.Header.Set("Authorization","Bearer valid");app.ServeHTTP(w,r)
|
||||
var value map[string]any;decoder:=json.NewDecoder(bytes.NewReader(w.Body.Bytes()));decoder.UseNumber();if err:=decoder.Decode(&value);err!=nil{t.Fatal(err)}
|
||||
if w.Code!=tc.status||value[tc.key]==nil{t.Fatalf("generated %s: %d %s",tc.path,w.Code,w.Body.String())}
|
||||
if tc.path=="/item"&&value["code"]!=json.Number("9223372036854775807"){t.Fatalf("generated int64 lost precision: %s",w.Body.String())}
|
||||
}
|
||||
// Business handlers also honor the same runtime policy and raw override.
|
||||
h:=item.New(service{});runtime:=omni.New();defer runtime.Close()
|
||||
common:=response.MustNew(response.Config{CodeType:"integer",SuccessCode:200,ErrorCode:9999,MessageField:"msg",RequestIDField:"-",DataMode:"always"})
|
||||
runtime.GET("/item",h.Item,httpx.WithResponse(common))
|
||||
runtime.POST("/callback",h.Callback,httpx.WithResponse(response.RawJSON(common)))
|
||||
for _,tc:=range []struct{method,path,key string}{{"GET","/item","data"},{"POST","/callback","name"}}{
|
||||
w:=httptest.NewRecorder();runtime.ServeHTTP(w,httptest.NewRequest(tc.method,tc.path,nil));var value map[string]any;json.Unmarshal(w.Body.Bytes(),&value)
|
||||
if w.Code!=200||value[tc.key]==nil{t.Fatal(w.Body.String())}
|
||||
}
|
||||
}
|
||||
`
|
||||
176
cmd/go-omni/apigen/templates.go
Normal file
176
cmd/go-omni/apigen/templates.go
Normal file
@@ -0,0 +1,176 @@
|
||||
package apigen
|
||||
|
||||
import "fmt"
|
||||
|
||||
var baseTemplates = map[string]string{
|
||||
"cmd/server/main.go": `package main
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"MODULE/internal/handler"
|
||||
"MODULE/internal/servicecontext"
|
||||
"MODULE/internal/settings"
|
||||
)
|
||||
func main(){
|
||||
path:=flag.String("config","etc/config.yaml","configuration file; empty uses defaults and environment")
|
||||
flag.Parse()
|
||||
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||
}
|
||||
func run(path string)(runErr error){
|
||||
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||
defer stop()
|
||||
cfg,err:=settings.Load(path);if err!=nil{return err}
|
||||
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||
logx.SetDefault(logger)
|
||||
deps,err:=servicecontext.New(ctx,logger,cfg);if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,deps.Close())}()
|
||||
app:=omni.New(omni.WithConfig(cfg.Server))
|
||||
app.OnShutdown(logger.Close)
|
||||
app.OnShutdown(deps.Close)
|
||||
if err=handler.RegisterRoutes(app,deps);err!=nil{return err}
|
||||
app.GET("/openapi.json",handler.OpenAPI)
|
||||
app.GET("/health/live",deps.Health.LivenessHandler())
|
||||
app.GET("/health/ready",deps.Health.ReadinessHandler())
|
||||
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||
return app.RunContext(ctx,cfg.Server.Address)
|
||||
}
|
||||
`,
|
||||
".gitignore": "/server\n/var/\n.env\n.env.*\n!.env.example\n/.goOmni/api.lock\n",
|
||||
}
|
||||
|
||||
func projectREADME(module, entry string) string {
|
||||
return fmt.Sprintf(`# %s
|
||||
|
||||
This project is generated from an editable API contract. Business code uses
|
||||
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
|
||||
|
||||
## Run
|
||||
|
||||
`+"```sh"+`
|
||||
go mod tidy
|
||||
go run ./cmd/server
|
||||
`+"```"+`
|
||||
|
||||
GET /health/live and /health/ready check the running service. GET /openapi.json
|
||||
serves the same document as docs/openapi.json. Bind address defaults to localhost.
|
||||
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
|
||||
The greeting starter works immediately; other new operation stubs return 501 until
|
||||
their business logic is implemented. Authentication hooks also fail closed with 501.
|
||||
|
||||
## Develop
|
||||
|
||||
1. Edit %s and its relative imports.
|
||||
2. Run the same go-omni CLI version as the framework dependency:
|
||||
|
||||
`+"```sh"+`
|
||||
go-omni api validate
|
||||
go-omni api inspect
|
||||
go-omni api generate --dry-run
|
||||
go-omni api generate
|
||||
go-omni api generate --check
|
||||
go-omni api export > openapi.json
|
||||
go test ./...
|
||||
`+"```"+`
|
||||
|
||||
Commands inside a generated project discover the recorded API entry automatically.
|
||||
--dry-run previews create/update/remove actions without writing files.
|
||||
The --check command is read-only and suitable for CI. It fails on stale or missing
|
||||
generated files and missing business scaffolds. go.mod, configuration, main,
|
||||
Service, Repository, servicecontext, middleware hook bodies and API files are
|
||||
created once and then preserved byte for byte. Gen.go files and OpenAPI are owned
|
||||
by the generator; manual edits cause a conflict instead of being overwritten.
|
||||
Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it.
|
||||
Avoid editing files while generation is running; api.lock serializes generators.
|
||||
|
||||
Routes and types that disappear are removed from generated output. Handwritten
|
||||
business files remain: remove/adapt obsolete implementations explicitly if an
|
||||
API change makes their old types or signatures fail to compile. Every regeneration
|
||||
must be followed by go test ./... . Do not delete service files to change contracts.
|
||||
|
||||
## Layout
|
||||
|
||||
`+"```text"+`
|
||||
api/ # editable contract and imported types
|
||||
cmd/server/main.go # startup, lifecycle and route registration
|
||||
etc/config.yaml # lowerCamelCase configuration
|
||||
internal/settings/ # defaults -> file -> environment
|
||||
internal/servicecontext/ # dependency initialization and cleanup
|
||||
internal/types/typesGen.go # generated request/response types
|
||||
internal/apierrors/errorsGen.go # generated fresh error constructors, when declared
|
||||
internal/handler/<group>/ # generated binding and service adapters
|
||||
internal/service/<group>/ # developer-owned business operations
|
||||
internal/repository/<group>/ # developer-owned persistence contracts
|
||||
internal/middleware/ # named hooks and generated registry
|
||||
docs/openapi.json # generated OpenAPI 3.0.3
|
||||
.goOmni/apiManifest.json # ownership and content hashes
|
||||
`+"```"+`
|
||||
|
||||
The API response type describes business data. Declare response Name (...) and
|
||||
select it with @server(response: Name) or @response Name to share a public body;
|
||||
@response raw returns the business JSON directly. Without a template, the default
|
||||
envelope includes code, message, data and requestId. Generated routes and OpenAPI
|
||||
use the API contract, independently of the app's default response configuration.
|
||||
Handlers reject nil successful responses. Body DTOs use application/json and have a 1 MiB limit by default.
|
||||
Use contract.Binder.Bind's optional limit in a custom adapter for larger payloads.
|
||||
|
||||
Fields require exactly one source tag: json, path, form (URL query only), header.
|
||||
All are required by presence unless optional, omitempty or default is specified.
|
||||
Zero and false are valid present values; binding:"required" rejects scalar zero.
|
||||
Use *T with optional when business logic needs to distinguish missing from zero.
|
||||
Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON
|
||||
fields and duplicate scalar query/header values are rejected. Arrays in the query
|
||||
use repeated keys. Enum/range/default and nested fields share the documentation
|
||||
and binding compiler. Responses are expected to honor their declared constraints;
|
||||
they are not validated on every response. Do not return nil slices/maps for a
|
||||
non-nullable collection (initialize empty collections instead).
|
||||
|
||||
Declare public business errors in error (...) and reference them with @errors.
|
||||
Use internal/apierrors constructors to share status, code and message with OpenAPI.
|
||||
Auth declarations connect a bearer/header/cookie scheme to a real named middleware.
|
||||
The selected auth runs first; @auth none removes inherited declarative auth.
|
||||
Validation policy remains in the middleware implementation. Public messages must
|
||||
not contain secrets; error causes may be included in server logs.
|
||||
|
||||
Named middleware is composed in declaration order (first is outermost). Implement
|
||||
the generated hooks or replace entries in servicecontext; never bypass an auth
|
||||
hook merely to make the server start. Middleware names do not imply a JWT policy.
|
||||
|
||||
This CLI supports a documented go-zero-style subset, not all goctl features.
|
||||
Database tables and repositories cannot be inferred from an HTTP contract. Add
|
||||
business repositories through servicecontext. Database and storage options are
|
||||
composed at creation; the original profile is retained on regeneration.
|
||||
`, module, entry)
|
||||
}
|
||||
|
||||
func infrastructureREADME(o Options) string {
|
||||
return fmt.Sprintf(`
|
||||
## Infrastructure
|
||||
|
||||
Selected database: %s; storage: %s; logging: %s/%s.
|
||||
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
|
||||
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||
No tables are invented or migrated automatically from request/response DTOs.
|
||||
The none/memory selection opens no database; implement repositories per business.
|
||||
|
||||
With storage enabled, services receive deps.Files, Uploads, Downloads and
|
||||
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
|
||||
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
|
||||
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
|
||||
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
|
||||
and authorize before issuing them. No public upload-link route is added.
|
||||
Storage failures during startup close an already-open database; Close is idempotent.
|
||||
|
||||
Database/storage/logging selection is recorded in .goOmni/apiManifest.json.
|
||||
Changing selection through generation flags is rejected on an existing project;
|
||||
edit your owned configuration and initialization explicitly, or generate a fresh
|
||||
project for comparison. New routes still regenerate without repeating these flags.
|
||||
`, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||
}
|
||||
311
cmd/go-omni/apigen/write.go
Normal file
311
cmd/go-omni/apigen/write.go
Normal file
@@ -0,0 +1,311 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const manifestPath = ".goOmni/apiManifest.json"
|
||||
|
||||
type manifest struct {
|
||||
Profile *projectProfile `json:"profile,omitempty"`
|
||||
Version int `json:"version"`
|
||||
Module string `json:"module"`
|
||||
Entry string `json:"entry"`
|
||||
Generated map[string]string `json:"generated"`
|
||||
}
|
||||
type change struct {
|
||||
name string
|
||||
before, after []byte
|
||||
existed, remove bool
|
||||
mode os.FileMode
|
||||
}
|
||||
|
||||
func apply(root, module, entry string, generated, seeds map[string][]byte, check, dryRun bool, profile projectProfile) (Output, error) {
|
||||
var result Output
|
||||
if info, err := os.Lstat(root); err == nil {
|
||||
if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() {
|
||||
return result, errors.New("api: output root must be a real directory")
|
||||
}
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return result, err
|
||||
}
|
||||
root = canonical(root)
|
||||
names := map[string]bool{}
|
||||
caseNames := map[string]string{}
|
||||
for _, files := range []map[string][]byte{generated, seeds} {
|
||||
for name := range files {
|
||||
if names[name] {
|
||||
return result, fmt.Errorf("duplicate output %s", name)
|
||||
}
|
||||
names[name] = true
|
||||
folded := strings.ToLower(name)
|
||||
if old, ok := caseNames[folded]; ok {
|
||||
return result, fmt.Errorf("case-insensitive path collision: %s and %s", old, name)
|
||||
}
|
||||
caseNames[folded] = name
|
||||
if err := safePath(root, name); err != nil {
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := safePath(root, manifestPath); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if !check && !dryRun {
|
||||
if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
|
||||
return result, err
|
||||
}
|
||||
lock := filepath.Join(root, ".goOmni", "api.lock")
|
||||
f, err := os.OpenFile(lock, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600)
|
||||
if err != nil {
|
||||
return result, fmt.Errorf("api: acquire generation lock (remove stale api.lock only after checking no generator is running): %w", err)
|
||||
}
|
||||
_, _ = fmt.Fprintf(f, "pid=%d\n", os.Getpid())
|
||||
_ = f.Close()
|
||||
defer os.Remove(lock)
|
||||
}
|
||||
old := manifest{Generated: map[string]string{}}
|
||||
oldData, err := os.ReadFile(filepath.Join(root, manifestPath))
|
||||
if err == nil {
|
||||
if err = json.Unmarshal(oldData, &old); err != nil {
|
||||
return result, fmt.Errorf("invalid ownership manifest: %w", err)
|
||||
}
|
||||
if old.Profile != nil && *old.Profile != profile {
|
||||
return result, errors.New("api: profile changed during generation; retry")
|
||||
}
|
||||
if old.Version != 1 || old.Module != module || old.Entry != entry {
|
||||
return result, errors.New("api: manifest version/module/entry differs; one API entry owns this project")
|
||||
}
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return result, err
|
||||
}
|
||||
if oldData == nil {
|
||||
if err = filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
|
||||
if errors.Is(walkErr, os.ErrNotExist) {
|
||||
return nil
|
||||
}
|
||||
if walkErr != nil {
|
||||
return walkErr
|
||||
}
|
||||
if d.IsDir() && d.Name() == ".git" {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
if !d.IsDir() && strings.HasSuffix(d.Name(), ".go") {
|
||||
return fmt.Errorf("api: %s belongs to an existing Go layout; generate into a new directory", path)
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
next := manifest{Profile: &profile, Version: 1, Module: module, Entry: entry, Generated: map[string]string{}}
|
||||
for name, data := range generated {
|
||||
next.Generated[name] = digest(data)
|
||||
}
|
||||
var changes []change
|
||||
// Check ALL previous outputs, even stale ones, before writing any new file.
|
||||
for name, hash := range old.Generated {
|
||||
if !ownedPath(name) {
|
||||
return result, fmt.Errorf("api: invalid owned path %q", name)
|
||||
}
|
||||
if err = safePath(root, name); err != nil {
|
||||
return result, err
|
||||
}
|
||||
data, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(name)))
|
||||
if errors.Is(e, os.ErrNotExist) {
|
||||
continue
|
||||
}
|
||||
if e != nil {
|
||||
return result, e
|
||||
}
|
||||
if digest(data) != hash {
|
||||
return result, fmt.Errorf("api: generated file %s was edited; move custom code into developer-owned files before regenerating", name)
|
||||
}
|
||||
if _, exists := generated[name]; !exists {
|
||||
info, _ := os.Stat(filepath.Join(root, filepath.FromSlash(name)))
|
||||
changes = append(changes, change{name: name, before: data, existed: true, remove: true, mode: info.Mode().Perm()})
|
||||
result.Removed = append(result.Removed, name)
|
||||
}
|
||||
}
|
||||
paths := make([]string, 0, len(names))
|
||||
for name := range names {
|
||||
paths = append(paths, name)
|
||||
}
|
||||
sort.Strings(paths)
|
||||
for _, name := range paths {
|
||||
target := filepath.Join(root, filepath.FromSlash(name))
|
||||
current, err := os.ReadFile(target)
|
||||
exists := err == nil
|
||||
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return result, err
|
||||
}
|
||||
content, isGenerated := generated[name]
|
||||
if !isGenerated {
|
||||
content = seeds[name]
|
||||
}
|
||||
if exists && !isGenerated {
|
||||
result.Preserved = append(result.Preserved, name)
|
||||
continue
|
||||
}
|
||||
if exists && isGenerated {
|
||||
if _, owned := old.Generated[name]; !owned {
|
||||
return result, fmt.Errorf("api: refusing to adopt existing unowned file %s", name)
|
||||
}
|
||||
}
|
||||
if isGenerated {
|
||||
result.Generated = append(result.Generated, name)
|
||||
} else {
|
||||
result.Created = append(result.Created, name)
|
||||
}
|
||||
if exists && bytes.Equal(current, content) {
|
||||
continue
|
||||
}
|
||||
mode := os.FileMode(0644)
|
||||
if exists {
|
||||
info, e := os.Stat(target)
|
||||
if e != nil {
|
||||
return result, e
|
||||
}
|
||||
mode = info.Mode().Perm()
|
||||
}
|
||||
changes = append(changes, change{name: name, before: current, after: content, existed: exists, mode: mode})
|
||||
}
|
||||
data, err := json.MarshalIndent(next, "", " ")
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
data = append(data, '\n')
|
||||
if !bytes.Equal(oldData, data) {
|
||||
changes = append(changes, change{name: manifestPath, before: oldData, after: data, existed: oldData != nil, mode: 0644})
|
||||
}
|
||||
sort.Strings(result.Removed)
|
||||
for _, c := range changes {
|
||||
action := "create"
|
||||
if c.remove {
|
||||
action = "remove"
|
||||
} else if c.existed {
|
||||
action = "update"
|
||||
}
|
||||
result.Changes = append(result.Changes, FileChange{Path: c.name, Action: action})
|
||||
}
|
||||
sort.Slice(result.Changes, func(i, j int) bool { return result.Changes[i].Path < result.Changes[j].Path })
|
||||
if dryRun {
|
||||
return result, nil
|
||||
}
|
||||
if check {
|
||||
if len(changes) > 0 {
|
||||
var stale []string
|
||||
for _, c := range changes {
|
||||
stale = append(stale, c.name)
|
||||
}
|
||||
sort.Strings(stale)
|
||||
return result, fmt.Errorf("api: generated project is stale: %s", strings.Join(stale, ", "))
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
// A rollback restores bytes/modes if an ordinary filesystem write fails.
|
||||
// File replacement is atomic; the whole directory is not crash-atomic.
|
||||
var applied []change
|
||||
for _, c := range changes {
|
||||
if err = safePath(root, c.name); err == nil {
|
||||
current, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||
if c.existed {
|
||||
if e != nil || !bytes.Equal(current, c.before) {
|
||||
err = fmt.Errorf("file changed during generation: %s", c.name)
|
||||
}
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
err = fmt.Errorf("file appeared during generation: %s", c.name)
|
||||
}
|
||||
}
|
||||
if err == nil {
|
||||
if c.remove {
|
||||
err = os.Remove(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||
} else {
|
||||
err = atomicWrite(root, c.name, c.after, c.mode)
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
for i := len(applied) - 1; i >= 0; i-- {
|
||||
previous := applied[i]
|
||||
var rollback error
|
||||
if previous.existed {
|
||||
rollback = atomicWrite(root, previous.name, previous.before, previous.mode)
|
||||
} else {
|
||||
rollback = os.Remove(filepath.Join(root, filepath.FromSlash(previous.name)))
|
||||
}
|
||||
err = errors.Join(err, rollback)
|
||||
}
|
||||
return result, err
|
||||
}
|
||||
applied = append(applied, c)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
|
||||
func ownedPath(name string) bool {
|
||||
if name == "internal/handler/responseGen.go" {
|
||||
return true
|
||||
}
|
||||
if name == "internal/apierrors/errorsGen.go" || name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {
|
||||
return true
|
||||
}
|
||||
return strings.HasPrefix(name, "internal/handler/") && strings.HasSuffix(name, "/handlersGen.go")
|
||||
}
|
||||
func safePath(root, name string) error {
|
||||
if filepath.IsAbs(name) || strings.Contains(name, "\\") || filepath.ToSlash(filepath.Clean(name)) != name || name == "." || strings.HasPrefix(name, "../") {
|
||||
return fmt.Errorf("api: unsafe output path %q", name)
|
||||
}
|
||||
path := root
|
||||
parts := strings.Split(name, "/")
|
||||
for i, part := range parts {
|
||||
path = filepath.Join(path, part)
|
||||
info, err := os.Lstat(path)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
return fmt.Errorf("api: refusing symlink %s", path)
|
||||
}
|
||||
if i < len(parts)-1 && !info.IsDir() {
|
||||
return fmt.Errorf("api: expected directory %s", path)
|
||||
}
|
||||
if i == len(parts)-1 && !info.Mode().IsRegular() {
|
||||
return fmt.Errorf("api: expected regular file %s", path)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func atomicWrite(root, name string, data []byte, mode os.FileMode) error {
|
||||
target := filepath.Join(root, filepath.FromSlash(name))
|
||||
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
f, err := os.CreateTemp(filepath.Dir(target), ".apiGen-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(f.Name())
|
||||
if err = f.Chmod(mode); err == nil {
|
||||
_, err = f.Write(data)
|
||||
}
|
||||
if err == nil {
|
||||
err = f.Sync()
|
||||
}
|
||||
err = errors.Join(err, f.Close())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(f.Name(), target)
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||
)
|
||||
|
||||
@@ -17,6 +18,21 @@ func main() {
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "api":
|
||||
if err := runAPI(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni api:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "migrate":
|
||||
if err := runMigrate(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "add":
|
||||
if err := runAdd(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni add:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "new":
|
||||
if err := runNew(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni new:", err)
|
||||
@@ -34,11 +50,56 @@ func main() {
|
||||
func runNew(args []string) error {
|
||||
options := scaffold.Options{}
|
||||
var target string
|
||||
var apiFile string
|
||||
for index := 0; index < len(args); index++ {
|
||||
arg := args[index]
|
||||
switch {
|
||||
case arg == "--api" || strings.HasPrefix(arg, "--api="):
|
||||
_, value, inline := strings.Cut(arg, "=")
|
||||
if !inline {
|
||||
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "-") {
|
||||
return errors.New("--api requires a file")
|
||||
}
|
||||
index++
|
||||
value = args[index]
|
||||
}
|
||||
if value == "" {
|
||||
return errors.New("--api requires a file")
|
||||
}
|
||||
apiFile = value
|
||||
case arg == "--log-output" || arg == "--log-format" || arg == "--storage" || strings.HasPrefix(arg, "--log-output=") || strings.HasPrefix(arg, "--log-format=") || strings.HasPrefix(arg, "--storage="):
|
||||
name, value, hasValue := strings.Cut(arg, "=")
|
||||
if !hasValue {
|
||||
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "--") {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
index++
|
||||
value = args[index]
|
||||
}
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
switch name {
|
||||
case "--log-output":
|
||||
options.LogOutput = value
|
||||
case "--log-format":
|
||||
options.LogFormat = value
|
||||
case "--storage":
|
||||
options.Storage = value
|
||||
}
|
||||
case arg == "--force":
|
||||
options.Force = true
|
||||
case arg == "--database":
|
||||
if index+1 >= len(args) || strings.TrimSpace(args[index+1]) == "" {
|
||||
return errors.New("--database requires memory or mysql")
|
||||
}
|
||||
index++
|
||||
options.Database = args[index]
|
||||
case strings.HasPrefix(arg, "--database="):
|
||||
options.Database = strings.TrimPrefix(arg, "--database=")
|
||||
if strings.TrimSpace(options.Database) == "" {
|
||||
return errors.New("--database requires memory or mysql")
|
||||
}
|
||||
case arg == "--module":
|
||||
if index+1 >= len(args) {
|
||||
return errors.New("--module requires a value")
|
||||
@@ -55,6 +116,17 @@ func runNew(args []string) error {
|
||||
options.FrameworkDir = args[index]
|
||||
case strings.HasPrefix(arg, "--framework-dir="):
|
||||
options.FrameworkDir = strings.TrimPrefix(arg, "--framework-dir=")
|
||||
case arg == "--framework-version":
|
||||
if index+1 >= len(args) || strings.TrimSpace(args[index+1]) == "" {
|
||||
return errors.New("--framework-version requires a value")
|
||||
}
|
||||
index++
|
||||
options.FrameworkVersion = args[index]
|
||||
case strings.HasPrefix(arg, "--framework-version="):
|
||||
options.FrameworkVersion = strings.TrimPrefix(arg, "--framework-version=")
|
||||
if strings.TrimSpace(options.FrameworkVersion) == "" {
|
||||
return errors.New("--framework-version requires a value")
|
||||
}
|
||||
case strings.HasPrefix(arg, "-"):
|
||||
return fmt.Errorf("unknown option %q", arg)
|
||||
case target == "":
|
||||
@@ -66,10 +138,26 @@ func runNew(args []string) error {
|
||||
if target == "" {
|
||||
return errors.New("project directory is required")
|
||||
}
|
||||
if options.Module == "" {
|
||||
if options.Module == "" && apiFile == "" {
|
||||
options.Module = filepath.Base(filepath.Clean(target))
|
||||
}
|
||||
options.Dir = target
|
||||
if apiFile != "" {
|
||||
if options.Force {
|
||||
return errors.New("--api protects existing business code and does not accept --force")
|
||||
}
|
||||
_, err := apigen.Generate(apigen.Options{API: apiFile, Dir: target, Module: options.Module, FrameworkDir: options.FrameworkDir, FrameworkVersion: options.FrameworkVersion, LogOutput: options.LogOutput, LogFormat: options.LogFormat, Database: options.Database, Storage: options.Storage})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("created API project %s; implement internal/service operations\n", filepath.Clean(target))
|
||||
return nil
|
||||
}
|
||||
if _, err := os.Lstat(filepath.Join(target, ".goOmni", "apiManifest.json")); err == nil {
|
||||
return errors.New("this is an API project; use api generate instead of the CRUD/module starter")
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
if err := scaffold.Generate(options); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -81,5 +169,47 @@ func usage() {
|
||||
fmt.Fprintln(os.Stderr, "GoOmni project tool")
|
||||
fmt.Fprintln(os.Stderr, "")
|
||||
fmt.Fprintln(os.Stderr, "Usage:")
|
||||
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api new [--module <module>] <directory> Create a runnable API project")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api generate [--api <file.api>] [--dir <project>] [--check|--dry-run]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api inspect [--api <file.api>] Show routes, auth and business errors")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api export [--api <file.api>] Write OpenAPI JSON to stdout")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api validate --api <file.api> Validate definitions and OpenAPI")
|
||||
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
||||
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
||||
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
||||
fmt.Fprintln(os.Stderr, " --database <none|memory|mysql> API defaults to none; CRUD defaults to memory")
|
||||
fmt.Fprintln(os.Stderr, " --log-output <stdout|stderr|file> Generate configured logging (default stderr)")
|
||||
fmt.Fprintln(os.Stderr, " --log-format <json|text> Select structured log format (default json)")
|
||||
fmt.Fprintln(os.Stderr, " --storage <none|local|s3|r2|oss> Select optional storage adapter (default none)")
|
||||
}
|
||||
|
||||
func runAdd(args []string) error {
|
||||
directory, name := ".", ""
|
||||
for index := 0; index < len(args); index++ {
|
||||
arg := args[index]
|
||||
switch {
|
||||
case arg == "--dir":
|
||||
if index+1 >= len(args) {
|
||||
return errors.New("--dir requires a project directory")
|
||||
}
|
||||
index++
|
||||
directory = args[index]
|
||||
case strings.HasPrefix(arg, "--dir="):
|
||||
directory = strings.TrimPrefix(arg, "--dir=")
|
||||
case strings.HasPrefix(arg, "-"):
|
||||
return fmt.Errorf("unknown option %q", arg)
|
||||
case name == "":
|
||||
name = arg
|
||||
default:
|
||||
return fmt.Errorf("unexpected argument %q", arg)
|
||||
}
|
||||
}
|
||||
if err := scaffold.AddModule(directory, name); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("created internal/%s; see its README.md for route registration\n", name)
|
||||
return nil
|
||||
}
|
||||
|
||||
100
cmd/go-omni/main_test.go
Normal file
100
cmd/go-omni/main_test.go
Normal file
@@ -0,0 +1,100 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestNewWithRemoteFrameworkVersion(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--framework-version", "v0.1.0"}, {"--framework-version=v0.1.0"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
args := append([]string{"--module", "example.com/billing"}, flag...)
|
||||
if err := runNew(append(args, project)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
content, err := os.ReadFile(filepath.Join(project, "go.mod"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(content), "go-omni v0.1.0") || strings.Contains(string(content), "replace ") {
|
||||
t.Fatalf("CLI did not pin remote dependency: %s", content)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewRejectsEmptyFrameworkVersion(t *testing.T) {
|
||||
for _, args := range [][]string{{"--framework-version"}, {"--framework-version="}, {"--framework-version", " "}} {
|
||||
if err := runNew(args); err == nil {
|
||||
t.Fatalf("accepted missing framework version: %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleCLI(t *testing.T) {
|
||||
for _, equals := range []bool{false, true} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := runNew([]string{"--module", "example.com/billing", project}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
args := []string{"orders", "--dir", project}
|
||||
if equals {
|
||||
args = []string{"--dir=" + project, "orders"}
|
||||
}
|
||||
if err := runAdd(args); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(project, "internal/orders/module.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, args := range [][]string{nil, {"orders", "--dir"}, {"orders", "--dir="}, {"orders", "extra"}, {"orders", "--unknown"}} {
|
||||
if err := runAdd(args); err == nil {
|
||||
t.Fatalf("accepted invalid arguments %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewMySQLDatabaseOptions(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--database", "mysql"}, {"--database=mysql"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
args := append([]string{"--module", "example.com/billing"}, flag...)
|
||||
if err := runNew(append(args, project)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(project, "schema.sql")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, args := range [][]string{{"--database"}, {"--database="}, {"--database", " "}} {
|
||||
if err := runNew(args); err == nil {
|
||||
t.Fatalf("accepted missing database type: %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestModuleOptionCLI(t *testing.T) {
|
||||
for _, flags := range [][]string{
|
||||
{"--log-output", "file", "--log-format", "json", "--storage", "local"},
|
||||
{"--log-output=stdout", "--log-format=text", "--storage=none"},
|
||||
} {
|
||||
project := filepath.Join(t.TempDir(), "demo")
|
||||
args := append([]string{"--module", "example.com/demo"}, flags...)
|
||||
if err := runNew(append(args, project)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(project, "etc/config.yaml")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, args := range [][]string{
|
||||
{"--storage"}, {"--storage="}, {"--storage", "--force"},
|
||||
{"--log-output"}, {"--log-output="}, {"--log-format="},
|
||||
{"--storage", "unknown", "demo"}, {"--log-format", "xml", "demo"},
|
||||
} {
|
||||
if err := runNew(args); err == nil {
|
||||
t.Fatal("invalid module option accepted", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
60
cmd/go-omni/migrate.go
Normal file
60
cmd/go-omni/migrate.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
migration "git.yuntuo.hk/k2safe/go-omni/database/migrate"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func runMigrate(args []string) error {
|
||||
flags := flag.NewFlagSet("migrate", flag.ContinueOnError)
|
||||
flags.SetOutput(io.Discard)
|
||||
directory := flags.String("dir", "migrations", "SQL directory")
|
||||
dsnEnv := flags.String("dsn-env", "GO_OMNI_DB_DSN", "DSN environment variable")
|
||||
apply := flags.Bool("apply", false, "apply pending up migrations")
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() != 0 {
|
||||
return errors.New("migrate: unexpected argument")
|
||||
}
|
||||
_, manifest, err := migration.Snapshot(os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !*apply {
|
||||
data, err := json.MarshalIndent(manifest, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(data))
|
||||
return nil
|
||||
}
|
||||
dsn := os.Getenv(*dsnEnv)
|
||||
if dsn == "" {
|
||||
return errors.New("migrate: DSN environment variable is empty")
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
runner, err := migration.NewMySQL(ctx, dsn, os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer runner.Close()
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
version, dirty, err := runner.Version()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("migration version=%d dirty=%t\n", version, dirty)
|
||||
return nil
|
||||
}
|
||||
25
cmd/go-omni/migrate_test.go
Normal file
25
cmd/go-omni/migrate_test.go
Normal file
@@ -0,0 +1,25 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMigrationPreviewDoesNotConnect(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "0001_createItems.up.sql"), []byte("CREATE TABLE testItems(id BIGINT);"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "not a DSN")
|
||||
if err := runMigrate([]string{"--dir", dir}); err != nil {
|
||||
t.Fatal("preview tried to connect", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "")
|
||||
if err := runMigrate([]string{"--dir", dir, "--apply"}); err == nil {
|
||||
t.Fatal("missing DSN accepted")
|
||||
}
|
||||
if err := runMigrate([]string{"--dir", dir, "--force"}); err == nil {
|
||||
t.Fatal("unsafe unknown flag accepted")
|
||||
}
|
||||
}
|
||||
109
cmd/go-omni/scaffold/addModule.go
Normal file
109
cmd/go-omni/scaffold/addModule.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"text/template"
|
||||
|
||||
"golang.org/x/mod/modfile"
|
||||
)
|
||||
|
||||
var moduleName = regexp.MustCompile(`^[a-z][a-zA-Z0-9]{0,47}$`)
|
||||
|
||||
// AddModule creates a complete, injectable memory-backed CRUD module. It never
|
||||
// edits existing routes, go.mod, migrations or files in another module.
|
||||
func AddModule(directory, name string) (err error) {
|
||||
if strings.TrimSpace(directory) == "" || !moduleName.MatchString(name) || token.Lookup(name).IsKeyword() || name == "main" || name == "init" || name == "internal" {
|
||||
return errors.New("module name must be a non-keyword lowerCamelCase Go identifier (1..48 characters)")
|
||||
}
|
||||
root, err := filepath.Abs(directory)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
root, err = filepath.EvalSymlinks(root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return fmt.Errorf("read project go.mod: %w", err)
|
||||
}
|
||||
manifest, err := modfile.Parse("go.mod", data, nil)
|
||||
if err != nil || manifest.Module == nil {
|
||||
return errors.New("project go.mod must declare a valid module")
|
||||
}
|
||||
found := false
|
||||
for _, dependency := range manifest.Require {
|
||||
if dependency.Mod.Path == frameworkModule {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return errors.New("project must already require go-omni; create it with go-omni new")
|
||||
}
|
||||
module := manifest.Module.Mod.Path
|
||||
if strings.ContainsAny(module, "\"\\\r\n") {
|
||||
return errors.New("invalid project module path")
|
||||
}
|
||||
paths := []string{"module.go", "model/model.go", "repository/repository.go", "service/service.go", "handler/handler.go", "README.md"}
|
||||
rendered := make(map[string][]byte, len(paths))
|
||||
for _, path := range paths {
|
||||
source, readErr := templateFiles.ReadFile("moduleTemplates/" + strings.ReplaceAll(path, "/", "-") + ".tmpl")
|
||||
if readErr != nil {
|
||||
return readErr
|
||||
}
|
||||
tmpl, parseErr := template.New(path).Parse(string(source))
|
||||
if parseErr != nil {
|
||||
return parseErr
|
||||
}
|
||||
var output bytes.Buffer
|
||||
if err := tmpl.Execute(&output, struct{ Module, Name string }{module, name}); err != nil {
|
||||
return err
|
||||
}
|
||||
content := output.Bytes()
|
||||
if strings.HasSuffix(path, ".go") {
|
||||
content, err = format.Source(content)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
rendered[path] = content
|
||||
}
|
||||
parent := filepath.Join(root, "internal")
|
||||
if info, statErr := os.Lstat(parent); statErr == nil {
|
||||
if !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||
return errors.New("internal must be a real directory, not a symlink")
|
||||
}
|
||||
} else if !errors.Is(statErr, os.ErrNotExist) {
|
||||
return statErr
|
||||
} else if err := os.Mkdir(parent, 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
target := filepath.Join(parent, name)
|
||||
// Exclusive creation protects existing files, including empty directories.
|
||||
if err := os.Mkdir(target, 0755); err != nil {
|
||||
return fmt.Errorf("create module (existing modules are never overwritten): %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if err != nil {
|
||||
_ = os.RemoveAll(target)
|
||||
}
|
||||
}()
|
||||
for _, path := range paths {
|
||||
file := filepath.Join(target, filepath.FromSlash(path))
|
||||
if err = os.MkdirAll(filepath.Dir(file), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
if err = os.WriteFile(file, rendered[path], 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
@@ -0,0 +1,137 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func moduleProject(t *testing.T) string {
|
||||
t.Helper()
|
||||
root := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: root, Module: "example.com/billing", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return root
|
||||
}
|
||||
|
||||
func TestAddModulePreservesProjectAndRejectsOverwrite(t *testing.T) {
|
||||
root := moduleProject(t)
|
||||
manifest, _ := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
main, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go"))
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, path := range []string{"module.go", "model/model.go", "handler/handler.go", "service/service.go", "repository/repository.go", "README.md"} {
|
||||
if _, err := os.Stat(filepath.Join(root, "internal/orderItems", path)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
marker := filepath.Join(root, "internal/orderItems/keep.txt")
|
||||
if err := os.WriteFile(marker, []byte("keep"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orderItems"); err == nil {
|
||||
t.Fatal("overwrote existing module")
|
||||
}
|
||||
if got, _ := os.ReadFile(marker); string(got) != "keep" {
|
||||
t.Fatal("existing file changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "go.mod")); string(got) != string(manifest) {
|
||||
t.Fatal("go.mod changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go")); string(got) != string(main) {
|
||||
t.Fatal("main changed")
|
||||
}
|
||||
for _, name := range []string{"../escape", "Order", "order_items", "func", "main", "init", "internal", "", strings.Repeat("a", 49)} {
|
||||
if err := AddModule(root, name); err == nil {
|
||||
t.Fatalf("accepted %q", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleRejectsForeignProjectAndSymlink(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "go.mod"), []byte("module example.com/foreign\ngo 1.22\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted foreign project")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(root, "internal")); !os.IsNotExist(err) {
|
||||
t.Fatal("created files before validation")
|
||||
}
|
||||
root = moduleProject(t)
|
||||
if err := os.Rename(filepath.Join(root, "internal"), filepath.Join(root, "original")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(filepath.Join(root, "original"), filepath.Join(root, "internal")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted symlink parent")
|
||||
}
|
||||
}
|
||||
|
||||
// Opt-in because this compiles a separate consuming module and may need module
|
||||
// downloads. The Linux feature acceptance runner enables it explicitly.
|
||||
func TestAddModuleIntegration(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-project CRUD")
|
||||
}
|
||||
root := moduleProject(t)
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "internal/orderItems/module_test.go"), []byte(generatedModuleTest), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "./...")
|
||||
cmd.Dir = root
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("generated project: %v\n%s", err, output)
|
||||
}
|
||||
}
|
||||
|
||||
const generatedModuleTest = `package orderItems_test
|
||||
import (
|
||||
"context"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"example.com/billing/internal/orderItems"
|
||||
)
|
||||
func TestGeneratedCRUD(t *testing.T) {
|
||||
app:=omni.New()
|
||||
repo:=orderItems.NewMemory()
|
||||
orderItems.Register(app,repo,func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
if c.Header("Authorization")!="Bearer demo" {return omni.Unauthorized("DENIED","denied")};return next(c)
|
||||
}})
|
||||
request:=func(method,path,body string,authorized bool)*httptest.ResponseRecorder{
|
||||
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type","application/json")
|
||||
if authorized {r.Header.Set("Authorization","Bearer demo")}
|
||||
w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,r);return w
|
||||
}
|
||||
if w:=request("POST","/orderItems","{\"name\":\"first\"}",false);w.Code!=401 {t.Fatal(w.Code)}
|
||||
cases:=[]struct{method,path,body string;status int;contains string}{
|
||||
{"POST","/orderItems","{\"name\":\" first \"}",201,"createdAt"},
|
||||
{"GET","/orderItems/1","",200,"first"},
|
||||
{"PUT","/orderItems/1","{\"name\":\"changed\"}",200,"changed"},
|
||||
{"GET","/orderItems","",200,"changed"},
|
||||
{"POST","/orderItems","{\"name\":\" \"}",400,"INVALID_NAME"},
|
||||
{"DELETE","/orderItems/1","",204,""},
|
||||
{"GET","/orderItems/1","",404,"NOT_FOUND"},
|
||||
}
|
||||
for _,tc:=range cases{w:=request(tc.method,tc.path,tc.body,true);if w.Code!=tc.status||!strings.Contains(w.Body.String(),tc.contains){t.Fatalf("%s %s: %d %s",tc.method,tc.path,w.Code,w.Body.String())}}
|
||||
ctx,cancel:=context.WithCancel(context.Background());cancel()
|
||||
if _,err:=repo.List(ctx);err!=context.Canceled {t.Fatal(err)}
|
||||
}
|
||||
`
|
||||
29
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
29
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
@@ -0,0 +1,29 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// APIInfrastructure renders shared infrastructure without inventing business
|
||||
// tables or routes. The API generator applies these as developer-owned seeds.
|
||||
func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
|
||||
data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true}
|
||||
files := map[string]string{
|
||||
"internal/settings/config.go": "templates/modules-settings.go.tmpl",
|
||||
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
|
||||
"internal/servicecontext/context.go": "templates/api-context.go.tmpl",
|
||||
}
|
||||
if storage != "none" {
|
||||
files["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||
}
|
||||
result := make(map[string][]byte, len(files))
|
||||
for name, source := range files {
|
||||
content, err := render(source, data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("render %s: %w", name, err)
|
||||
}
|
||||
result[name] = []byte(strings.TrimRight(content, "\n") + "\n")
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
@@ -0,0 +1,143 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type moduleProfile struct{ database, storage, output, format string }
|
||||
|
||||
var moduleProfiles = []moduleProfile{
|
||||
{"memory", "none", "stdout", "json"},
|
||||
{"memory", "local", "file", "json"},
|
||||
{"memory", "s3", "stdout", "text"},
|
||||
{"memory", "r2", "stderr", "json"},
|
||||
{"memory", "oss", "stderr", "text"},
|
||||
{"mysql", "none", "file", "text"},
|
||||
{"mysql", "local", "stdout", "json"},
|
||||
{"mysql", "s3", "file", "text"},
|
||||
{"mysql", "r2", "stderr", "text"},
|
||||
{"mysql", "oss", "file", "json"},
|
||||
}
|
||||
|
||||
func generateProfile(t *testing.T, p moduleProfile) string {
|
||||
t.Helper()
|
||||
project := filepath.Join(t.TempDir(), "project")
|
||||
if err := Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: p.database, Storage: p.storage, LogOutput: p.output, LogFormat: p.format}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return project
|
||||
}
|
||||
func TestGenerateModuleProfiles(t *testing.T) {
|
||||
for _, p := range moduleProfiles {
|
||||
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||
project := generateProfile(t, p)
|
||||
for _, name := range []string{"etc/config.yaml", "internal/settings/config.go", "MODULES.md", ".gitignore"} {
|
||||
if _, err := os.Stat(filepath.Join(project, name)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
_, err := os.Stat(filepath.Join(project, "internal/handler/files.go"))
|
||||
if p.storage == "none" && !os.IsNotExist(err) {
|
||||
t.Fatal("unselected storage was emitted")
|
||||
}
|
||||
if p.storage != "none" && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
module, err := os.ReadFile(filepath.Join(project, "go.mod"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pieces := strings.Split(string(module), "=> ")
|
||||
if len(pieces) != 2 {
|
||||
t.Fatal("local replace missing")
|
||||
}
|
||||
relative, err := strconv.Unquote(strings.TrimSpace(pieces[1]))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(project, relative, "go.mod")); err != nil {
|
||||
t.Fatal("replacement does not resolve", err)
|
||||
}
|
||||
cfg, _ := os.ReadFile(filepath.Join(project, "etc/config.yaml"))
|
||||
if strings.Contains(string(cfg), "secretKey:") || strings.Contains(string(cfg), "filesToken:") {
|
||||
t.Fatal("secret defaults emitted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestModuleOptionsRejectBeforeWriting(t *testing.T) {
|
||||
for _, options := range []Options{
|
||||
{LogOutput: "invalid"}, {LogFormat: "xml"}, {Storage: "ftp"},
|
||||
{Storage: "local", FrameworkVersion: "v0.6.0-preview"},
|
||||
{Module: "example.com/x\nrequire malicious v1.0.0"},
|
||||
} {
|
||||
options.Dir = filepath.Join(t.TempDir(), "project")
|
||||
if options.Module == "" {
|
||||
options.Module = "example.com/profile"
|
||||
}
|
||||
if err := Generate(options); err == nil {
|
||||
t.Fatalf("invalid options accepted: %+v", options)
|
||||
}
|
||||
if _, err := os.Stat(options.Dir); !os.IsNotExist(err) {
|
||||
t.Fatal("invalid options touched target")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestCanonicalPathBeforeDirectoryExists(t *testing.T) {
|
||||
parent := t.TempDir()
|
||||
alias := filepath.Join(t.TempDir(), "alias")
|
||||
if err := os.Symlink(parent, alias); err != nil {
|
||||
t.Skip(err)
|
||||
}
|
||||
path := filepath.Join(alias, "new", "nested")
|
||||
if got, want := canonicalPath(path), filepath.Join(canonicalPath(parent), "new", "nested"); got != want {
|
||||
t.Fatalf("%s != %s", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// Compile consuming modules, then run their generated tests. No provider account
|
||||
// is used: cloud constructors are checked offline, file workflows use Local.
|
||||
func TestModuleProfilesIntegration(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-profile tests")
|
||||
}
|
||||
for _, p := range moduleProfiles {
|
||||
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||
project := generateProfile(t, p)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-p=2", "-count=1", "-timeout=60s", "./...")
|
||||
cmd.Dir = project
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOMAXPROCS=2", "GO_OMNI_TEST_MYSQL_DSN=")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("generated %s/%s: %v\n%s", p.database, p.storage, err, output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestForceRejectsRemovedDependenciesWithoutTouchingCode(t *testing.T) {
|
||||
project := generateProfile(t, moduleProfile{"mysql", "local", "file", "json"})
|
||||
path := filepath.Join(project, "cmd/server/main.go")
|
||||
before, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Storage: "none"}); err == nil {
|
||||
t.Fatal("incompatible force overwrite accepted")
|
||||
}
|
||||
after, _ := os.ReadFile(path)
|
||||
if string(after) != string(before) {
|
||||
t.Fatal("failed profile switch changed code")
|
||||
}
|
||||
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Database: "mysql", Storage: "local", LogOutput: "stdout"}); err != nil {
|
||||
t.Fatal("compatible force overwrite rejected", err)
|
||||
}
|
||||
}
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
# {{.Name}}
|
||||
|
||||
在业务启动文件导入并注册:
|
||||
|
||||
```go
|
||||
import "{{.Module}}/internal/{{.Name}}"
|
||||
|
||||
{{.Name}}.Register(application, {{.Name}}.NewMemory())
|
||||
```
|
||||
|
||||
提供 `POST /{{.Name}}`、`GET /{{.Name}}`、`GET /{{.Name}}/:id`、`PUT /{{.Name}}/:id`、
|
||||
`DELETE /{{.Name}}/:id`。写入 JSON 为 `{"name":"example"}`;输出字段为 id、name、createdAt。
|
||||
代码按 Handler / Service / Repository / Model 分层,`Register` 可注入自己的 Repository 和认证中间件。
|
||||
内存示例不持久化、不提供分页;列表适合小规模演示。接实际业务时定义领域字段、分页、鉴权和数据库实现。
|
||||
生成器不会自动修改主入口、已有模块或数据库结构,也不会覆盖已有文件。
|
||||
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
@@ -0,0 +1,52 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"errors"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Handler struct { service *service.Service }
|
||||
type WriteRequest struct { Name string `json:"name" binding:"required,max=200"` }
|
||||
func New(service *service.Service) *Handler {return &Handler{service:service}}
|
||||
func (h *Handler) Register(routes *omni.RouterGroup) {
|
||||
routes.POST("",h.Create)
|
||||
routes.GET("",h.List)
|
||||
routes.GET("/:id",h.Get)
|
||||
routes.PUT("/:id",h.Update)
|
||||
routes.DELETE("/:id",h.Delete)
|
||||
}
|
||||
func (h *Handler) Create(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Create(c.Context(),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.Created(item)
|
||||
}
|
||||
func (h *Handler) Get(c *omni.Context) error {
|
||||
item,err:=h.service.Get(c.Context(),c.Param("id"))
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) List(c *omni.Context) error {
|
||||
items,err:=h.service.List(c.Context())
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(items)
|
||||
}
|
||||
func (h *Handler) Update(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Update(c.Context(),c.Param("id"),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) Delete(c *omni.Context) error {
|
||||
if err:=h.service.Delete(c.Context(),c.Param("id")); err!=nil {return publicError(err)}
|
||||
return c.NoContent()
|
||||
}
|
||||
func publicError(err error) error {
|
||||
if errors.Is(err,model.ErrNotFound) {return omni.NotFound("NOT_FOUND","record not found").WithCause(err)}
|
||||
if errors.Is(err,model.ErrInvalidName) {return omni.BadRequest("INVALID_NAME","name must contain 1..200 bytes").WithCause(err)}
|
||||
return err
|
||||
}
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"time"
|
||||
)
|
||||
|
||||
var ErrNotFound = errors.New("{{.Name}}: record not found")
|
||||
var ErrInvalidName = errors.New("{{.Name}}: name must contain 1..200 bytes")
|
||||
|
||||
type Item struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
@@ -0,0 +1,19 @@
|
||||
package {{.Name}}
|
||||
|
||||
import (
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/handler"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Repository = repository.Repository
|
||||
|
||||
func NewMemory() Repository { return repository.NewMemory() }
|
||||
|
||||
// Register wires this module with caller-owned storage. Pass auth middleware
|
||||
// here to apply it to every route before module-specific middleware.
|
||||
func Register(app *omni.App, repo Repository, middleware ...omni.Middleware) {
|
||||
if repo == nil { panic("{{.Name}}: repository is required") }
|
||||
handler.New(service.New(repo)).Register(app.Group("/{{.Name}}", middleware...))
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"strconv"
|
||||
"sync"
|
||||
"time"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
)
|
||||
|
||||
type Repository interface {
|
||||
Create(context.Context,string) (model.Item,error)
|
||||
Get(context.Context,string) (model.Item,error)
|
||||
List(context.Context) ([]model.Item,error)
|
||||
Update(context.Context,string,string) (model.Item,error)
|
||||
Delete(context.Context,string) error
|
||||
}
|
||||
|
||||
// Memory is demonstration storage; data and IDs reset with the process.
|
||||
type Memory struct { mu sync.Mutex; next uint64; items map[string]model.Item }
|
||||
func NewMemory() *Memory { return &Memory{items:make(map[string]model.Item)} }
|
||||
func (m *Memory) Create(ctx context.Context, name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
m.next++
|
||||
item:=model.Item{ID:strconv.FormatUint(m.next,10),Name:name,CreatedAt:time.Now().UTC()}
|
||||
m.items[item.ID]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Get(ctx context.Context,id string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) List(ctx context.Context) ([]model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return nil,err}
|
||||
items:=make([]model.Item,0,len(m.items))
|
||||
for _,item:=range m.items {items=append(items,item)}
|
||||
sort.Slice(items,func(i,j int)bool{return items[i].ID<items[j].ID})
|
||||
return items,nil
|
||||
}
|
||||
func (m *Memory) Update(ctx context.Context,id,name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
item.Name=name; m.items[id]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Delete(ctx context.Context,id string) error {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return err}
|
||||
if _,ok:=m.items[id]; !ok {return model.ErrNotFound}
|
||||
delete(m.items,id)
|
||||
return nil
|
||||
}
|
||||
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
@@ -0,0 +1,27 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
)
|
||||
|
||||
type Service struct { repo repository.Repository }
|
||||
func New(repo repository.Repository) *Service { return &Service{repo:repo} }
|
||||
func name(value string) (string,error) {
|
||||
value=strings.TrimSpace(value)
|
||||
if len(value)==0 || len(value)>200 {return "",model.ErrInvalidName}
|
||||
return value,nil
|
||||
}
|
||||
func (s *Service) Create(ctx context.Context,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Create(ctx,value)
|
||||
}
|
||||
func (s *Service) Get(ctx context.Context,id string) (model.Item,error) { return s.repo.Get(ctx,id) }
|
||||
func (s *Service) List(ctx context.Context) ([]model.Item,error) { return s.repo.List(ctx) }
|
||||
func (s *Service) Update(ctx context.Context,id,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Update(ctx,id,value)
|
||||
}
|
||||
func (s *Service) Delete(ctx context.Context,id string) error {return s.repo.Delete(ctx,id)}
|
||||
@@ -4,23 +4,33 @@ import (
|
||||
"embed"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"io/fs"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"text/template"
|
||||
|
||||
"golang.org/x/mod/module"
|
||||
"golang.org/x/mod/semver"
|
||||
)
|
||||
|
||||
//go:embed templates
|
||||
//go:embed templates moduleTemplates
|
||||
var templateFiles embed.FS
|
||||
|
||||
// Options controls project generation.
|
||||
type Options struct {
|
||||
Dir string
|
||||
Module string
|
||||
FrameworkDir string
|
||||
Force bool
|
||||
Dir string
|
||||
Module string
|
||||
FrameworkDir string
|
||||
FrameworkVersion string
|
||||
Database string
|
||||
LogOutput string
|
||||
LogFormat string
|
||||
Storage string
|
||||
Force bool
|
||||
}
|
||||
|
||||
var files = map[string]string{
|
||||
@@ -28,6 +38,8 @@ var files = map[string]string{
|
||||
"README.md": "templates/README.md.tmpl",
|
||||
"cmd/server/main.go": "templates/cmd-server-main.go.tmpl",
|
||||
"internal/model/user.go": "templates/internal-model-user.go.tmpl",
|
||||
"internal/model/errors.go": "templates/internal-model-errors.go.tmpl",
|
||||
"internal/repository/users.go": "templates/internal-repository-users.go.tmpl",
|
||||
"internal/repository/memory.go": "templates/internal-repository-memory.go.tmpl",
|
||||
"internal/servicecontext/context.go": "templates/internal-servicecontext-context.go.tmpl",
|
||||
"internal/service/user.go": "templates/internal-service-user.go.tmpl",
|
||||
@@ -42,6 +54,48 @@ func Generate(options Options) error {
|
||||
if strings.TrimSpace(options.Module) == "" {
|
||||
return errors.New("module path is required")
|
||||
}
|
||||
if err := module.CheckImportPath(options.Module); err != nil {
|
||||
return fmt.Errorf("invalid module path: %w", err)
|
||||
}
|
||||
modules := options.LogOutput != "" || options.LogFormat != "" || options.Storage != ""
|
||||
logOutput, logFormat, storage := options.LogOutput, options.LogFormat, options.Storage
|
||||
if logOutput == "" {
|
||||
logOutput = "stderr"
|
||||
}
|
||||
if logFormat == "" {
|
||||
logFormat = "json"
|
||||
}
|
||||
if storage == "" {
|
||||
storage = "none"
|
||||
}
|
||||
if logOutput != "stdout" && logOutput != "stderr" && logOutput != "file" {
|
||||
return fmt.Errorf("unsupported log output %q: choose stdout, stderr or file", logOutput)
|
||||
}
|
||||
if logFormat != "json" && logFormat != "text" {
|
||||
return fmt.Errorf("unsupported log format %q: choose json or text", logFormat)
|
||||
}
|
||||
switch storage {
|
||||
case "none", "local", "s3", "r2", "oss":
|
||||
default:
|
||||
return fmt.Errorf("unsupported object storage %q: choose none, local, s3, r2 or oss", storage)
|
||||
}
|
||||
database := strings.ToLower(strings.TrimSpace(options.Database))
|
||||
if database == "" {
|
||||
database = "memory"
|
||||
}
|
||||
if database != "memory" && database != "mysql" {
|
||||
return fmt.Errorf("unsupported database %q: choose memory or mysql", database)
|
||||
}
|
||||
version := strings.TrimSpace(options.FrameworkVersion)
|
||||
if version == "" {
|
||||
version = currentFrameworkVersion()
|
||||
if modules && version == defaultFrameworkVersion {
|
||||
version = modulesFrameworkVersion
|
||||
}
|
||||
}
|
||||
if !validFrameworkVersion(version) {
|
||||
return fmt.Errorf("framework version %q must be a complete v0 or v1 semantic version", version)
|
||||
}
|
||||
|
||||
directory, err := filepath.Abs(options.Dir)
|
||||
if err != nil {
|
||||
@@ -50,24 +104,81 @@ func Generate(options Options) error {
|
||||
if err := checkTarget(directory, options.Force); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
||||
return fmt.Errorf("create project directory: %w", err)
|
||||
// An explicit remote version opts out of automatic local development mode.
|
||||
replace := ""
|
||||
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
|
||||
replace = frameworkReplace(directory, options.FrameworkDir)
|
||||
}
|
||||
if modules && replace == "" && semver.Compare(version, modulesFrameworkVersion) < 0 && version != installedFrameworkVersion() {
|
||||
return fmt.Errorf("module templates require %s or newer; use --framework-dir for unpublished source", modulesFrameworkVersion)
|
||||
}
|
||||
|
||||
data := templateData{
|
||||
Module: options.Module,
|
||||
FrameworkReplace: frameworkReplace(directory, options.FrameworkDir),
|
||||
FrameworkVersion: version,
|
||||
FrameworkReplace: replace,
|
||||
Database: database,
|
||||
Modules: modules, LogOutput: logOutput, LogFormat: logFormat, Storage: storage,
|
||||
}
|
||||
paths := make([]string, 0, len(files))
|
||||
for path := range files {
|
||||
projectFiles := make(map[string]string, len(files)+3)
|
||||
for path, source := range files {
|
||||
projectFiles[path] = source
|
||||
}
|
||||
if database == "mysql" {
|
||||
projectFiles["internal/repository/mysql.go"] = "templates/internal-repository-mysql.go.tmpl"
|
||||
projectFiles["schema.sql"] = "templates/schema.sql.tmpl"
|
||||
projectFiles["internal/servicecontext/databaseIntegration_test.go"] = "templates/databaseIntegration_test.go.tmpl"
|
||||
}
|
||||
if modules {
|
||||
projectFiles["cmd/server/main.go"] = "templates/modules-main.go.tmpl"
|
||||
projectFiles["internal/settings/config.go"] = "templates/modules-settings.go.tmpl"
|
||||
projectFiles["internal/settings/config_test.go"] = "templates/modules-settings_test.go.tmpl"
|
||||
projectFiles["etc/config.yaml"] = "templates/modules-config.yaml.tmpl"
|
||||
projectFiles[".gitignore"] = "templates/modules-gitignore.tmpl"
|
||||
projectFiles["MODULES.md"] = "templates/modules-README.md.tmpl"
|
||||
if storage != "none" {
|
||||
projectFiles["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||
projectFiles["internal/servicecontext/storage_test.go"] = "templates/modules-storage_test.go.tmpl"
|
||||
projectFiles["internal/service/files.go"] = "templates/modules-filesService.go.tmpl"
|
||||
projectFiles["internal/handler/files.go"] = "templates/modules-filesHandler.go.tmpl"
|
||||
projectFiles["internal/handler/files_test.go"] = "templates/modules-filesHandler_test.go.tmpl"
|
||||
}
|
||||
}
|
||||
if options.Force {
|
||||
// Do not leave old generated Go files that reference dependencies omitted
|
||||
// by this profile, and do not silently delete potentially edited code.
|
||||
for _, path := range []string{"internal/repository/mysql.go", "internal/servicecontext/databaseIntegration_test.go", "schema.sql", "internal/settings/config.go", "internal/settings/config_test.go", "etc/config.yaml", "MODULES.md", "internal/servicecontext/storage.go", "internal/servicecontext/storage_test.go", "internal/service/files.go", "internal/handler/files.go", "internal/handler/files_test.go"} {
|
||||
if _, emitted := projectFiles[path]; emitted {
|
||||
continue
|
||||
}
|
||||
if _, err := os.Lstat(filepath.Join(directory, path)); err == nil {
|
||||
return fmt.Errorf("existing %s conflicts with the selected profile; generate into a new directory instead of removing existing code", path)
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
paths := make([]string, 0, len(projectFiles))
|
||||
for path := range projectFiles {
|
||||
paths = append(paths, path)
|
||||
}
|
||||
sort.Strings(paths)
|
||||
// Render and format the whole project before touching the target directory.
|
||||
contents := make(map[string]string, len(paths))
|
||||
for _, path := range paths {
|
||||
content, err := render(files[path], data)
|
||||
content, err := render(projectFiles[path], data)
|
||||
if err != nil {
|
||||
return fmt.Errorf("render %s: %w", path, err)
|
||||
}
|
||||
if strings.HasSuffix(path, ".go") {
|
||||
formatted, err := format.Source([]byte(content))
|
||||
if err != nil {
|
||||
return fmt.Errorf("format %s: %w", path, err)
|
||||
}
|
||||
content = string(formatted)
|
||||
}
|
||||
contents[path] = content
|
||||
}
|
||||
for _, path := range paths {
|
||||
target := filepath.Join(directory, filepath.FromSlash(path))
|
||||
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
||||
return fmt.Errorf("create %s: %w", filepath.Dir(path), err)
|
||||
@@ -79,7 +190,7 @@ func Generate(options Options) error {
|
||||
return fmt.Errorf("check %s: %w", target, err)
|
||||
}
|
||||
}
|
||||
if err := os.WriteFile(target, []byte(content), 0o644); err != nil {
|
||||
if err := os.WriteFile(target, []byte(contents[path]), 0o644); err != nil {
|
||||
return fmt.Errorf("write %s: %w", target, err)
|
||||
}
|
||||
}
|
||||
@@ -87,8 +198,15 @@ func Generate(options Options) error {
|
||||
}
|
||||
|
||||
type templateData struct {
|
||||
API bool
|
||||
Module string
|
||||
FrameworkVersion string
|
||||
FrameworkReplace string
|
||||
Database string
|
||||
Modules bool
|
||||
LogOutput string
|
||||
LogFormat string
|
||||
Storage string
|
||||
}
|
||||
|
||||
func render(name string, data templateData) (string, error) {
|
||||
@@ -149,7 +267,7 @@ func frameworkReplace(projectDir, configured string) string {
|
||||
if err != nil {
|
||||
return ""
|
||||
}
|
||||
return filepath.ToSlash(relative)
|
||||
return strconv.Quote(filepath.ToSlash(relative))
|
||||
}
|
||||
|
||||
func canonicalPath(path string) string {
|
||||
@@ -157,11 +275,28 @@ func canonicalPath(path string) string {
|
||||
if err != nil {
|
||||
return path
|
||||
}
|
||||
resolved, err := filepath.EvalSymlinks(abs)
|
||||
if err == nil {
|
||||
return resolved
|
||||
// Resolve the nearest existing ancestor too: generation now renders before
|
||||
// creating directories, and macOS /var commonly resolves through /private.
|
||||
candidate := abs
|
||||
var tail []string
|
||||
for {
|
||||
resolved, err := filepath.EvalSymlinks(candidate)
|
||||
if err == nil {
|
||||
for i := len(tail) - 1; i >= 0; i-- {
|
||||
resolved = filepath.Join(resolved, tail[i])
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
if !errors.Is(err, os.ErrNotExist) {
|
||||
return abs
|
||||
}
|
||||
parent := filepath.Dir(candidate)
|
||||
if parent == candidate {
|
||||
return abs
|
||||
}
|
||||
tail = append(tail, filepath.Base(candidate))
|
||||
candidate = parent
|
||||
}
|
||||
return abs
|
||||
}
|
||||
|
||||
func findFrameworkRoot() string {
|
||||
@@ -172,8 +307,14 @@ func findFrameworkRoot() string {
|
||||
for {
|
||||
moduleFile := filepath.Join(directory, "go.mod")
|
||||
content, readErr := os.ReadFile(moduleFile)
|
||||
if readErr == nil && strings.Contains(string(content), "module git.yuntuo.hk/k2safe/go-omni") {
|
||||
return directory
|
||||
if readErr == nil {
|
||||
for _, line := range strings.Split(string(content), "\n") {
|
||||
fields := strings.Fields(line)
|
||||
if len(fields) >= 2 && fields[0] == "module" && strings.Trim(fields[1], "\"") == frameworkModule {
|
||||
return directory
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
parent := filepath.Dir(directory)
|
||||
if parent == directory {
|
||||
|
||||
@@ -3,6 +3,8 @@ package scaffold
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime/debug"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
@@ -37,7 +39,7 @@ func TestGenerateCreatesServiceLayout(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatalf("read generated server: %v", err)
|
||||
}
|
||||
for _, expected := range []string{"config.LoadFromEnv(\"\")", "logx.SetDefault(logger)", "omni.WithConfig(settings)", "servicecontext.New(logger, users)", "health.New()", "application.Run(settings.Address)"} {
|
||||
for _, expected := range []string{"config.LoadFromEnv(\"\")", "logx.SetDefault(logger)", "omni.WithConfig(settings)", "servicecontext.New(ctx, logger)", "services.Health.ReadinessHandler()", "application.RunContext(ctx, settings.Address)"} {
|
||||
if !strings.Contains(string(mainContent), expected) {
|
||||
t.Fatalf("generated server missing %q: %s", expected, mainContent)
|
||||
}
|
||||
@@ -54,6 +56,35 @@ func TestGenerateRejectsNonEmptyDirectory(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateMySQLLayout(t *testing.T) {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: project, Module: "example.com/billing", Database: "mysql", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, path := range []string{"schema.sql", "internal/repository/mysql.go", "internal/repository/users.go", "internal/model/errors.go", "internal/servicecontext/databaseIntegration_test.go"} {
|
||||
if _, err := os.Stat(filepath.Join(project, path)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
content, err := os.ReadFile(filepath.Join(project, "go.mod"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(content), "github.com/go-sql-driver/mysql v1.9.3") {
|
||||
t.Fatal("mysql driver version is not pinned")
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateRejectsUnknownDatabaseBeforeWriting(t *testing.T) {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: project, Module: "example.com/billing", Database: "unknown"}); err == nil {
|
||||
t.Fatal("accepted unknown backend")
|
||||
}
|
||||
if _, err := os.Stat(project); !os.IsNotExist(err) {
|
||||
t.Fatalf("invalid backend created files: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrameworkReplaceUsesCanonicalPaths(t *testing.T) {
|
||||
root := frameworkRoot(t)
|
||||
project := filepath.Join(t.TempDir(), "generated")
|
||||
@@ -73,11 +104,59 @@ func TestFrameworkReplaceUsesCanonicalPaths(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if replace != filepath.ToSlash(expected) {
|
||||
if replace != strconv.Quote(filepath.ToSlash(expected)) {
|
||||
t.Fatalf("expected canonical replace %q, got %q", filepath.ToSlash(expected), replace)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateRemoteProjectPinsVersion(t *testing.T) {
|
||||
for _, version := range []string{"v0.1.0", "v1.2.3-rc.1", "v0.0.0-20260920140624-8b19aa79eea0"} {
|
||||
t.Run(version, func(t *testing.T) {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: project, Module: "example.com/billing", FrameworkVersion: version}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
content, err := os.ReadFile(filepath.Join(project, "go.mod"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(content), "require "+frameworkModule+" "+version+"\n") || strings.Contains(string(content), "replace ") {
|
||||
t.Fatalf("expected pinned remote dependency without replace: %s", content)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestGenerateRejectsInvalidVersionBeforeWriting(t *testing.T) {
|
||||
for _, version := range []string{"main", "latest", "v1.2", "v01.2.3", "v2.0.0", "v0.1.0-01", "v0.1.0\nreplace bad => ./bad"} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: project, Module: "example.com/billing", FrameworkVersion: version}); err == nil {
|
||||
t.Fatalf("accepted invalid version %q", version)
|
||||
}
|
||||
if _, err := os.Stat(project); !os.IsNotExist(err) {
|
||||
t.Fatalf("invalid version %q created a project: %v", version, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFrameworkVersionUsesInstalledModuleMetadata(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
info *debug.BuildInfo
|
||||
want string
|
||||
}{
|
||||
{nil, defaultFrameworkVersion},
|
||||
{&debug.BuildInfo{Main: debug.Module{Path: frameworkModule, Version: "(devel)"}}, defaultFrameworkVersion},
|
||||
{&debug.BuildInfo{Main: debug.Module{Path: frameworkModule, Version: "v0.1.0+dirty"}}, defaultFrameworkVersion},
|
||||
{&debug.BuildInfo{Main: debug.Module{Path: "example.com/unrelated", Version: "v1.2.3"}}, defaultFrameworkVersion},
|
||||
{&debug.BuildInfo{Main: debug.Module{Path: frameworkModule, Version: "v0.9.2", Sum: "h1:test"}}, "v0.9.2"},
|
||||
{&debug.BuildInfo{Main: debug.Module{Path: frameworkModule, Version: "v0.0.0-20260920140624-8b19aa79eea0", Sum: "h1:test"}}, "v0.0.0-20260920140624-8b19aa79eea0"},
|
||||
} {
|
||||
if got := frameworkVersion(test.info); got != test.want {
|
||||
t.Fatalf("version = %q, want %q", got, test.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func frameworkRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
directory, err := os.Getwd()
|
||||
|
||||
@@ -1,6 +1,31 @@
|
||||
# GoOmni service
|
||||
|
||||
This service was generated with `go-omni new`.
|
||||
{{if .Modules}}
|
||||
**本项目启用了模块化配置,启动步骤和环境变量请先看 [MODULES.md](MODULES.md)。** 配置来自 `etc/config.yaml`,文件模块需要配置环境密钥后启动。
|
||||
{{end}}
|
||||
|
||||
Framework dependency: `git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}`.
|
||||
|
||||
Database: `{{.Database}}`.
|
||||
|
||||
{{if eq .Database "mysql"}}
|
||||
先创建业务数据库,再执行项目的建表脚本(替换示例数据库名、账号和地址):
|
||||
|
||||
```sh
|
||||
mysql -h 127.0.0.1 -u app -p app < schema.sql
|
||||
```
|
||||
|
||||
配置数据库连接后启动;初始化失败或缺少表结构时,服务会退出并关闭已创建的连接池:
|
||||
|
||||
```sh
|
||||
export GO_OMNI_DB_DSN='app:password@tcp(127.0.0.1:3306)/app?parseTime=true'
|
||||
```
|
||||
|
||||
应用只检查表结构,不会自动建库、建表或迁移。MySQL 驱动由本项目注册,核心框架仍依赖标准库 `database/sql`。
|
||||
{{else}}
|
||||
默认使用进程内存存储,服务重启后数据清空。需要 MySQL 时,生成新项目时选择 `--database mysql`。
|
||||
{{end}}
|
||||
|
||||
```sh
|
||||
go mod tidy
|
||||
@@ -16,7 +41,72 @@ GO_OMNI_ENABLE_LOGGER=true GO_OMNI_LOG_LEVEL=debug go run ./cmd/server
|
||||
`GO_OMNI_HTTP_ADDR` 可以修改监听地址。生成项目默认提供 `/health/live` 和
|
||||
`/health/ready`,日志、请求 ID 和优雅退出等基础能力由 GoOmni 统一初始化;业务依赖集中放在项目自己的 `internal/servicecontext`。
|
||||
|
||||
The generated layout keeps HTTP handlers, business services, repositories,
|
||||
models and a project-owned ServiceContext separate. Replace the memory
|
||||
repository with a database implementation when the service is ready for
|
||||
persistence, then add it to `internal/servicecontext`.
|
||||
服务使用 `signal.NotifyContext` 处理 `SIGINT` / `SIGTERM`。部署时构建并运行服务二进制,
|
||||
使信号直接到达服务进程:
|
||||
|
||||
```sh
|
||||
go build -o server ./cmd/server
|
||||
./server
|
||||
```
|
||||
|
||||
`GO_OMNI_SHUTDOWN_TIMEOUT` 控制等待请求完成的时间,默认 `10s`。
|
||||
|
||||
使用支持统一网络配置的框架版本时,设置 `GO_OMNI_TRANSPORT=native` 即可切换为自研 HTTP/1.1;
|
||||
默认 `nethttp`。`GO_OMNI_TLS_CERT_FILE` 和 `GO_OMNI_TLS_KEY_FILE` 配对启用 HTTPS。
|
||||
实验版 HTTP/2 使用 `GO_OMNI_TRANSPORT=http2`,必须同时配置 TLS 证书和私钥。
|
||||
业务 Handler / Service / Repository 不需要修改;三种适配器共用上述退出等待时间。
|
||||
{{if eq .Database "mysql"}}
|
||||
`/health/ready` 已注册数据库探针:连接失败返回 `503`,数据库恢复后重新返回 `200`。
|
||||
`/health/live` 始终只检查服务存活。`GO_OMNI_DB_PING_TIMEOUT` 限制初始化检查和数据库探针的等待时间。
|
||||
退出时先排空 HTTP 请求,再关闭数据库连接池,最后关闭日志。
|
||||
|
||||
数据库连接池参数:
|
||||
|
||||
```text
|
||||
GO_OMNI_DB_MAX_OPEN_CONNS=10
|
||||
GO_OMNI_DB_MAX_IDLE_CONNS=5
|
||||
GO_OMNI_DB_CONN_MAX_LIFETIME=30m
|
||||
GO_OMNI_DB_CONN_MAX_IDLE_TIME=5m
|
||||
GO_OMNI_DB_PING_TIMEOUT=5s
|
||||
```
|
||||
|
||||
`MAX_IDLE_CONNS=0` 禁用空闲连接,`MAX_OPEN_CONNS=0` 表示不限制连接数。
|
||||
项目会启用 MySQL `parseTime` 并按 UTC 读取 DATETIME 字段,JSON 字段保持 lowerCamelCase。
|
||||
|
||||
事务内复用 Repository(在业务代码中导入 `database/sql` 和项目的 `internal/repository`):
|
||||
|
||||
```go
|
||||
err := services.DB.InTx(ctx, nil, func(tx *sql.Tx) error {
|
||||
users := repository.NewMySQLUsers(tx)
|
||||
_, err := users.Create(ctx, input)
|
||||
return err
|
||||
})
|
||||
```
|
||||
|
||||
业务函数返回错误、发生 panic 或 context 取消时会回滚。事务内必须使用基于 `tx` 创建的 Repository。
|
||||
|
||||
可选集成测试(只针对已应用 schema.sql 的独立测试数据库):
|
||||
|
||||
```sh
|
||||
GO_OMNI_TEST_MYSQL_DSN='app:password@tcp(127.0.0.1:3306)/test?parseTime=true' go test -race ./...
|
||||
```
|
||||
|
||||
未设置 `GO_OMNI_TEST_MYSQL_DSN` 时跳过数据库集成测试。框架仓库中的 `scripts/verifyDatabase.py`
|
||||
可以自动创建、验收和清理独立的 MySQL 容器。
|
||||
{{else}}
|
||||
内存模式没有外部依赖探针;后续添加依赖时,在 `internal/servicecontext` 注册检查与资源清理。
|
||||
{{end}}
|
||||
|
||||
示例接口(无认证,接入实际业务时按项目需求添加认证中间件):
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| POST | `/users` | 创建,JSON 包含 name、email |
|
||||
| GET | `/users/:id` | 查询 |
|
||||
| GET | `/users?page=1&pageSize=20` | 分页 |
|
||||
| PUT | `/users/:id` | 更新 name、email |
|
||||
| DELETE | `/users/:id` | 删除,返回 204 |
|
||||
|
||||
接口使用 GoOmni 统一响应,包含 code、message、data 和 requestId。邮箱按小写存储,重复返回 409;
|
||||
无效输入返回 400,找不到记录返回 404。项目按 Handler / Service / Repository / Model 分层,
|
||||
基础依赖集中放在 `internal/servicecontext`。
|
||||
|
||||
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
@@ -0,0 +1,69 @@
|
||||
package servicecontext
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
{{if ne .Storage "none"}}"net/http"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if eq .Database "mysql"}}"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"github.com/go-sql-driver/mysql"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/settings"
|
||||
"{{.Module}}/internal/middleware"
|
||||
)
|
||||
// Context owns infrastructure, while Logger is borrowed from main.
|
||||
// Add business repositories here; HTTP contracts do not define database tables.
|
||||
type Context struct {
|
||||
Logger *logx.Logger
|
||||
Health *health.Checker
|
||||
Middlewares map[string]httpx.Middleware
|
||||
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||
{{if ne .Storage "none"}}
|
||||
Files storage.Store
|
||||
Uploads *storage.Uploader
|
||||
Downloads storage.Presigner
|
||||
UploadLinks storage.UploadPresigner
|
||||
FileHTTP http.Handler
|
||||
storageClose func()error
|
||||
{{end}}
|
||||
closeMu sync.Mutex
|
||||
closed bool
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
}
|
||||
// Pass the settings loaded by main. The optional form supports lightweight
|
||||
// tests without infrastructure; selected adapters still require valid settings.
|
||||
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
|
||||
if ctx==nil{ctx=context.Background()}
|
||||
if err=ctx.Err();err!=nil{return nil,err}
|
||||
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
|
||||
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||
_=cfg
|
||||
checker,err:=health.New();if err!=nil{return nil,err}
|
||||
c:=&Context{Logger:logger,Health:checker,Middlewares:middleware.Defaults()}
|
||||
defer func(){if err!=nil{err=errors.Join(err,c.Close())}}()
|
||||
{{if eq .Database "mysql"}}
|
||||
if err=cfg.Validate();err!=nil{return nil,err}
|
||||
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
|
||||
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
|
||||
dsn.ParseTime=true;dsn.Loc=time.UTC
|
||||
cfg.Database.DSN=dsn.FormatDSN()
|
||||
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
|
||||
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
|
||||
{{end}}
|
||||
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
|
||||
return c,nil
|
||||
}
|
||||
// Close is idempotent, including startup failure and graceful shutdown.
|
||||
func(c *Context)Close()error{
|
||||
if c==nil{return nil}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock();c.closed=true
|
||||
c.closeOnce.Do(func(){
|
||||
{{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}}
|
||||
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||
})
|
||||
return c.closeErr
|
||||
}
|
||||
@@ -1,49 +1,57 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/handler"
|
||||
"{{.Module}}/internal/repository"
|
||||
"{{.Module}}/internal/service"
|
||||
"{{.Module}}/internal/servicecontext"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func run() error {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
|
||||
settings, err := config.LoadFromEnv("")
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
return err
|
||||
}
|
||||
if strings.TrimSpace(settings.Log.Service) == "" {
|
||||
settings.Log.Service = "{{.Module}}"
|
||||
}
|
||||
logger, err := logx.New(settings.Log)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
return err
|
||||
}
|
||||
defer logger.Close()
|
||||
// Access logs and business logs share the same configured sink.
|
||||
logx.SetDefault(logger)
|
||||
|
||||
users := repository.NewMemoryUsers()
|
||||
services := servicecontext.New(logger, users)
|
||||
services, err := servicecontext.New(ctx, logger)
|
||||
if err != nil { return err }
|
||||
defer services.Close()
|
||||
userService := service.NewUsers(services)
|
||||
userHandler := handler.NewUsers(userService)
|
||||
checker, err := health.New()
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
application := omni.New(omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
application.GET("/health/live", checker.LivenessHandler())
|
||||
application.GET("/health/ready", checker.ReadinessHandler())
|
||||
application.OnShutdown(services.Close)
|
||||
application.GET("/health/live", services.Health.LivenessHandler())
|
||||
application.GET("/health/ready", services.Health.ReadinessHandler())
|
||||
userHandler.RegisterRoutes(application)
|
||||
if err := application.Run(settings.Address); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
return application.RunContext(ctx, settings.Address)
|
||||
}
|
||||
|
||||
106
cmd/go-omni/scaffold/templates/databaseIntegration_test.go.tmpl
Normal file
106
cmd/go-omni/scaffold/templates/databaseIntegration_test.go.tmpl
Normal file
@@ -0,0 +1,106 @@
|
||||
package servicecontext_test
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/model"
|
||||
"{{.Module}}/internal/repository"
|
||||
"{{.Module}}/internal/servicecontext"
|
||||
)
|
||||
|
||||
// These tests require a dedicated database with schema.sql applied. Ordinary
|
||||
// go test runs skip them; scripts/verifyDatabase.py provisions an isolated DB.
|
||||
func databaseContext(t *testing.T) *servicecontext.Context {
|
||||
t.Helper()
|
||||
dsn := os.Getenv("GO_OMNI_TEST_MYSQL_DSN")
|
||||
if dsn == "" { t.Skip("set GO_OMNI_TEST_MYSQL_DSN to a dedicated test database") }
|
||||
for _, key := range []string{"MAX_OPEN_CONNS", "MAX_IDLE_CONNS", "CONN_MAX_LIFETIME", "CONN_MAX_IDLE_TIME", "PING_TIMEOUT"} {
|
||||
t.Setenv("GO_OMNI_DB_"+key, "")
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", dsn)
|
||||
t.Setenv("GO_OMNI_DB_DRIVER", "mysql")
|
||||
logger, err := logx.New(logx.Config{Writer: io.Discard})
|
||||
if err != nil { t.Fatal(err) }
|
||||
t.Cleanup(func() { _ = logger.Close() })
|
||||
services, err := servicecontext.New(context.Background(), logger)
|
||||
if err != nil { t.Fatal(err) }
|
||||
t.Cleanup(func() { _ = services.Close() })
|
||||
return services
|
||||
}
|
||||
|
||||
func TestDatabaseTransactions(t *testing.T) {
|
||||
services := databaseContext(t)
|
||||
ctx := context.Background()
|
||||
for _, mode := range []string{"commit", "error", "panic", "cancel"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
email := fmt.Sprintf("transaction-%d-%s@example.com", time.Now().UnixNano(), mode)
|
||||
t.Cleanup(func() { _, _ = services.DB.ExecContext(ctx, "DELETE FROM users WHERE email = ?", email) })
|
||||
txCtx, cancel := context.WithCancel(ctx)
|
||||
defer cancel()
|
||||
abort := errors.New("abort business operation")
|
||||
var err error
|
||||
var recovered any
|
||||
func() {
|
||||
defer func() { recovered = recover() }()
|
||||
err = services.DB.InTx(txCtx, nil, func(tx *sql.Tx) error {
|
||||
users := repository.NewMySQLUsers(tx)
|
||||
if _, err := users.Create(txCtx, model.UserInput{Name: "transaction", Email: email}); err != nil { return err }
|
||||
switch mode {
|
||||
case "error": return abort
|
||||
case "panic": panic(abort)
|
||||
case "cancel": cancel(); return txCtx.Err()
|
||||
}
|
||||
return nil
|
||||
})
|
||||
}()
|
||||
switch mode {
|
||||
case "commit": if err != nil || recovered != nil { t.Fatalf("commit: error=%v panic=%v", err, recovered) }
|
||||
case "error": if !errors.Is(err, abort) { t.Fatalf("lost business error: %v", err) }
|
||||
case "panic": if recovered != abort { t.Fatalf("lost panic: %v", recovered) }
|
||||
case "cancel": if !errors.Is(err, context.Canceled) { t.Fatalf("lost cancellation: %v", err) }
|
||||
}
|
||||
var count int
|
||||
if err := services.DB.QueryRowContext(ctx, "SELECT COUNT(*) FROM users WHERE email = ?", email).Scan(&count); err != nil { t.Fatal(err) }
|
||||
want := 0
|
||||
if mode == "commit" { want = 1 }
|
||||
if count != want { t.Fatalf("%s persisted %d rows, want %d", mode, count, want) }
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestDatabasePoolHealthAndClose(t *testing.T) {
|
||||
services := databaseContext(t)
|
||||
ctx := context.Background()
|
||||
if result := services.Health.Check(ctx); !result.Ready() || len(result.Checks) != 1 { t.Fatalf("missing database readiness: %+v", result) }
|
||||
|
||||
cfg := services.DB.Config()
|
||||
cfg.MaxOpenConns, cfg.MaxIdleConns = 1, 0
|
||||
pool, err := database.Open(ctx, cfg)
|
||||
if err != nil { t.Fatal(err) }
|
||||
if idle := pool.Stats().Idle; idle != 0 { t.Fatalf("zero idle config retained %d connections", idle) }
|
||||
_ = pool.Close()
|
||||
|
||||
cfg.MaxIdleConns, cfg.PingTimeout = 1, 50*time.Millisecond
|
||||
pool, err = database.Open(ctx, cfg)
|
||||
if err != nil { t.Fatal(err) }
|
||||
defer pool.Close()
|
||||
connection, err := pool.Conn(ctx)
|
||||
if err != nil { t.Fatal(err) }
|
||||
defer connection.Close()
|
||||
started := time.Now()
|
||||
err = pool.Health(ctx)
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(started) > time.Second { t.Fatalf("health did not bound pool waiting: %v", err) }
|
||||
|
||||
if err := services.Close(); err != nil { t.Fatal(err) }
|
||||
if err := services.Close(); err != nil { t.Fatal(err) }
|
||||
if result := services.Health.Check(ctx); result.Ready() { t.Fatal("closed database was still ready") }
|
||||
}
|
||||
@@ -2,7 +2,9 @@ module {{.Module}}
|
||||
|
||||
go 1.22
|
||||
|
||||
require git.yuntuo.hk/k2safe/go-omni v0.0.0
|
||||
require git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}
|
||||
{{if eq .Database "mysql"}}require github.com/go-sql-driver/mysql v1.9.3
|
||||
{{end}}
|
||||
{{- if .FrameworkReplace}}
|
||||
replace git.yuntuo.hk/k2safe/go-omni => {{.FrameworkReplace}}
|
||||
{{- end}}
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/model"
|
||||
"{{.Module}}/internal/service"
|
||||
)
|
||||
@@ -12,20 +13,109 @@ type Users struct {
|
||||
service *service.Users
|
||||
}
|
||||
|
||||
func NewUsers(service *service.Users) *Users { return &Users{service: service} }
|
||||
func NewUsers(service *service.Users) *Users {
|
||||
return &Users{service: service}
|
||||
}
|
||||
|
||||
func (h *Users) RegisterRoutes(application *omni.App) {
|
||||
application.POST("/users", h.Create)
|
||||
users := application.Group("/users")
|
||||
users.POST("", h.Create)
|
||||
users.GET("", h.List)
|
||||
users.GET("/:id", h.Get)
|
||||
users.PUT("/:id", h.Update)
|
||||
users.DELETE("/:id", h.Delete)
|
||||
}
|
||||
|
||||
// HTTP-specific validation tags stay here, outside the business model.
|
||||
type userBody struct {
|
||||
Name string `json:"name" binding:"required,max=100"`
|
||||
Email string `json:"email" binding:"required,email,max=254"`
|
||||
}
|
||||
|
||||
type userPath struct {
|
||||
ID int64 `uri:"id" binding:"gt=0"`
|
||||
}
|
||||
|
||||
type listQuery struct {
|
||||
Page int `form:"page,default=1" binding:"gte=1"`
|
||||
PageSize int `form:"pageSize,default=20" binding:"gte=1,lte=100"`
|
||||
}
|
||||
|
||||
func (h *Users) Create(ctx *omni.Context) error {
|
||||
var input model.UserInput
|
||||
if err := ctx.BindJSON(&input); err != nil {
|
||||
var body userBody
|
||||
if err := ctx.BindJSON(&body); err != nil {
|
||||
return err
|
||||
}
|
||||
user, err := h.service.Create(ctx.Context(), input)
|
||||
user, err := h.service.Create(ctx.Request().Context(), model.UserInput(body))
|
||||
if err != nil {
|
||||
return userError(err)
|
||||
}
|
||||
return ctx.Created(user)
|
||||
}
|
||||
|
||||
func (h *Users) Get(ctx *omni.Context) error {
|
||||
var path userPath
|
||||
if err := ctx.BindURI(&path); err != nil {
|
||||
return err
|
||||
}
|
||||
user, err := h.service.Get(ctx.Request().Context(), path.ID)
|
||||
if err != nil {
|
||||
return userError(err)
|
||||
}
|
||||
return ctx.OK(user)
|
||||
}
|
||||
|
||||
func (h *Users) List(ctx *omni.Context) error {
|
||||
var query listQuery
|
||||
if err := ctx.BindQuery(&query); err != nil {
|
||||
return err
|
||||
}
|
||||
page, err := h.service.List(ctx.Request().Context(), query.Page, query.PageSize)
|
||||
if err != nil {
|
||||
return userError(err)
|
||||
}
|
||||
return ctx.OK(page)
|
||||
}
|
||||
|
||||
func (h *Users) Update(ctx *omni.Context) error {
|
||||
var path userPath
|
||||
if err := ctx.BindURI(&path); err != nil {
|
||||
return err
|
||||
}
|
||||
var body userBody
|
||||
if err := ctx.BindJSON(&body); err != nil {
|
||||
return err
|
||||
}
|
||||
user, err := h.service.Update(ctx.Request().Context(), path.ID, model.UserInput(body))
|
||||
if err != nil {
|
||||
return userError(err)
|
||||
}
|
||||
return ctx.OK(user)
|
||||
}
|
||||
|
||||
func (h *Users) Delete(ctx *omni.Context) error {
|
||||
var path userPath
|
||||
if err := ctx.BindURI(&path); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := h.service.Delete(ctx.Request().Context(), path.ID); err != nil {
|
||||
return userError(err)
|
||||
}
|
||||
return ctx.NoContent()
|
||||
}
|
||||
|
||||
func userError(err error) error {
|
||||
switch {
|
||||
case errors.Is(err, model.ErrUserNotFound):
|
||||
return omni.NotFound("USER_NOT_FOUND", "user was not found").WithCause(err)
|
||||
case errors.Is(err, model.ErrEmailTaken):
|
||||
return omni.NewError(http.StatusConflict, "EMAIL_TAKEN", "email is already in use").WithCause(err)
|
||||
case errors.Is(err, model.ErrInvalidUser):
|
||||
return omni.BadRequest("INVALID_USER", "name or email is invalid").WithCause(err)
|
||||
case errors.Is(err, model.ErrInvalidPage):
|
||||
return omni.BadRequest("INVALID_QUERY", "pagination is invalid").WithCause(err)
|
||||
default:
|
||||
// GoOmni masks unexpected errors; internal details stay in server logs.
|
||||
return err
|
||||
}
|
||||
return ctx.JSON(http.StatusCreated, user)
|
||||
}
|
||||
|
||||
11
cmd/go-omni/scaffold/templates/internal-model-errors.go.tmpl
Normal file
11
cmd/go-omni/scaffold/templates/internal-model-errors.go.tmpl
Normal file
@@ -0,0 +1,11 @@
|
||||
package model
|
||||
|
||||
import "errors"
|
||||
|
||||
// Domain errors are translated into HTTP responses only by the handler layer.
|
||||
var (
|
||||
ErrUserNotFound = errors.New("user not found")
|
||||
ErrEmailTaken = errors.New("email already in use")
|
||||
ErrInvalidUser = errors.New("invalid user")
|
||||
ErrInvalidPage = errors.New("invalid pagination")
|
||||
)
|
||||
@@ -2,6 +2,7 @@ package model
|
||||
|
||||
import "time"
|
||||
|
||||
// User is the business entity; it has no dependency on HTTP or a database driver.
|
||||
type User struct {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
@@ -10,7 +11,15 @@ type User struct {
|
||||
UpdatedAt time.Time `json:"updatedAt"`
|
||||
}
|
||||
|
||||
// UserInput is shared by creation and full replacement (PUT).
|
||||
type UserInput struct {
|
||||
Name string `json:"name"`
|
||||
Email string `json:"email"`
|
||||
Name string
|
||||
Email string
|
||||
}
|
||||
|
||||
type UserPage struct {
|
||||
Items []User `json:"items"`
|
||||
Total int `json:"total"`
|
||||
Page int `json:"page"`
|
||||
PageSize int `json:"pageSize"`
|
||||
}
|
||||
|
||||
@@ -2,30 +2,125 @@ package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"{{.Module}}/internal/model"
|
||||
)
|
||||
|
||||
type UserRepository interface {
|
||||
Create(context.Context, model.UserInput) (model.User, error)
|
||||
}
|
||||
|
||||
// MemoryUsers stores data only for this process. Replace it with a database
|
||||
// implementation of UserRepository when persistence is required.
|
||||
type MemoryUsers struct {
|
||||
mu sync.Mutex
|
||||
mu sync.RWMutex
|
||||
users map[int64]model.User
|
||||
emails map[string]int64
|
||||
nextID int64
|
||||
users []model.User
|
||||
}
|
||||
|
||||
func NewMemoryUsers() *MemoryUsers { return &MemoryUsers{nextID: 1} }
|
||||
func NewMemoryUsers() *MemoryUsers {
|
||||
return &MemoryUsers{
|
||||
users: make(map[int64]model.User),
|
||||
emails: make(map[string]int64),
|
||||
nextID: 1,
|
||||
}
|
||||
}
|
||||
|
||||
func (r *MemoryUsers) Create(_ context.Context, input model.UserInput) (model.User, error) {
|
||||
func (r *MemoryUsers) Create(ctx context.Context, input model.UserInput) (model.User, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
if _, exists := r.emails[input.Email]; exists {
|
||||
return model.User{}, model.ErrEmailTaken
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
user := model.User{ID: r.nextID, Name: input.Name, Email: input.Email, CreatedAt: now, UpdatedAt: now}
|
||||
user := model.User{
|
||||
ID: r.nextID, Name: input.Name, Email: input.Email,
|
||||
CreatedAt: now, UpdatedAt: now,
|
||||
}
|
||||
r.nextID++
|
||||
r.users = append(r.users, user)
|
||||
r.users[user.ID] = user
|
||||
r.emails[user.Email] = user.ID
|
||||
return user, nil
|
||||
}
|
||||
|
||||
func (r *MemoryUsers) Get(ctx context.Context, id int64) (model.User, error) {
|
||||
r.mu.RLock()
|
||||
defer r.mu.RUnlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
user, exists := r.users[id]
|
||||
if !exists {
|
||||
return model.User{}, model.ErrUserNotFound
|
||||
}
|
||||
return user, nil
|
||||
}
|
||||
|
||||
func (r *MemoryUsers) List(ctx context.Context, page, pageSize int) (model.UserPage, error) {
|
||||
if page < 1 || pageSize < 1 || pageSize > 100 {
|
||||
return model.UserPage{}, model.ErrInvalidPage
|
||||
}
|
||||
r.mu.RLock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
r.mu.RUnlock()
|
||||
return model.UserPage{}, err
|
||||
}
|
||||
users := make([]model.User, 0, len(r.users))
|
||||
for _, user := range r.users {
|
||||
users = append(users, user)
|
||||
}
|
||||
r.mu.RUnlock()
|
||||
|
||||
// A stable order is essential: Go map iteration order changes between calls.
|
||||
sort.Slice(users, func(i, j int) bool { return users[i].ID < users[j].ID })
|
||||
result := model.UserPage{
|
||||
Items: []model.User{}, Total: len(users), Page: page, PageSize: pageSize,
|
||||
}
|
||||
// Check the page before multiplying so even a very large page cannot overflow.
|
||||
if len(users) == 0 || page-1 > (len(users)-1)/pageSize {
|
||||
return result, nil
|
||||
}
|
||||
start := (page - 1) * pageSize
|
||||
count := min(pageSize, len(users)-start)
|
||||
result.Items = users[start : start+count]
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (r *MemoryUsers) Update(ctx context.Context, id int64, input model.UserInput) (model.User, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
user, exists := r.users[id]
|
||||
if !exists {
|
||||
return model.User{}, model.ErrUserNotFound
|
||||
}
|
||||
if owner, exists := r.emails[input.Email]; exists && owner != id {
|
||||
return model.User{}, model.ErrEmailTaken
|
||||
}
|
||||
delete(r.emails, user.Email)
|
||||
user.Name, user.Email = input.Name, input.Email
|
||||
user.UpdatedAt = time.Now().UTC()
|
||||
r.users[id] = user
|
||||
r.emails[user.Email] = id
|
||||
return user, nil
|
||||
}
|
||||
|
||||
func (r *MemoryUsers) Delete(ctx context.Context, id int64) error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
user, exists := r.users[id]
|
||||
if !exists {
|
||||
return model.ErrUserNotFound
|
||||
}
|
||||
delete(r.users, id)
|
||||
delete(r.emails, user.Email)
|
||||
return nil
|
||||
}
|
||||
|
||||
139
cmd/go-omni/scaffold/templates/internal-repository-mysql.go.tmpl
Normal file
139
cmd/go-omni/scaffold/templates/internal-repository-mysql.go.tmpl
Normal file
@@ -0,0 +1,139 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"errors"
|
||||
|
||||
"{{.Module}}/internal/model"
|
||||
"github.com/go-sql-driver/mysql"
|
||||
)
|
||||
|
||||
// SQLExecutor is implemented by both *sql.DB and *sql.Tx. A repository created
|
||||
// with a transaction keeps all of its queries on that same transaction.
|
||||
type SQLExecutor interface {
|
||||
ExecContext(context.Context, string, ...any) (sql.Result, error)
|
||||
QueryContext(context.Context, string, ...any) (*sql.Rows, error)
|
||||
QueryRowContext(context.Context, string, ...any) *sql.Row
|
||||
}
|
||||
|
||||
// MySQLUsers is the persistent implementation of UserRepository.
|
||||
// The schema is documented in schema.sql; migrations stay outside the framework.
|
||||
type MySQLUsers struct {
|
||||
db SQLExecutor
|
||||
}
|
||||
|
||||
func NewMySQLUsers(db SQLExecutor) *MySQLUsers {
|
||||
return &MySQLUsers{db: db}
|
||||
}
|
||||
|
||||
// CheckSchema validates that the application's migration has been applied.
|
||||
// Runtime startup never creates or alters tables.
|
||||
func (r *MySQLUsers) CheckSchema(ctx context.Context) error {
|
||||
rows, err := r.db.QueryContext(ctx, `SELECT id, name, email, created_at, updated_at FROM users LIMIT 0`)
|
||||
if err != nil { return err }
|
||||
return rows.Close()
|
||||
}
|
||||
|
||||
func (r *MySQLUsers) Create(ctx context.Context, input model.UserInput) (model.User, error) {
|
||||
result, err := r.db.ExecContext(ctx, `
|
||||
INSERT INTO users (name, email, created_at, updated_at)
|
||||
VALUES (?, ?, UTC_TIMESTAMP(6), UTC_TIMESTAMP(6))`, input.Name, input.Email)
|
||||
if err != nil {
|
||||
return model.User{}, mapMySQLError(err)
|
||||
}
|
||||
id, err := result.LastInsertId()
|
||||
if err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
return r.Get(ctx, id)
|
||||
}
|
||||
|
||||
func (r *MySQLUsers) Get(ctx context.Context, id int64) (model.User, error) {
|
||||
var user model.User
|
||||
err := r.db.QueryRowContext(ctx, `
|
||||
SELECT id, name, email, created_at, updated_at
|
||||
FROM users WHERE id = ?`, id).Scan(
|
||||
&user.ID, &user.Name, &user.Email, &user.CreatedAt, &user.UpdatedAt,
|
||||
)
|
||||
if errors.Is(err, sql.ErrNoRows) {
|
||||
return model.User{}, model.ErrUserNotFound
|
||||
}
|
||||
if err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
return user, nil
|
||||
}
|
||||
|
||||
func (r *MySQLUsers) List(ctx context.Context, page, pageSize int) (model.UserPage, error) {
|
||||
if page < 1 || pageSize < 1 || pageSize > 100 { return model.UserPage{}, model.ErrInvalidPage }
|
||||
var total int
|
||||
if err := r.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM users`).Scan(&total); err != nil {
|
||||
return model.UserPage{}, err
|
||||
}
|
||||
result := model.UserPage{Items: []model.User{}, Total: total, Page: page, PageSize: pageSize}
|
||||
if total == 0 || page-1 > (total-1)/pageSize {
|
||||
return result, nil
|
||||
}
|
||||
offset := (page - 1) * pageSize
|
||||
rows, err := r.db.QueryContext(ctx, `
|
||||
SELECT id, name, email, created_at, updated_at
|
||||
FROM users ORDER BY id LIMIT ? OFFSET ?`, pageSize, offset)
|
||||
if err != nil {
|
||||
return model.UserPage{}, err
|
||||
}
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var user model.User
|
||||
if err := rows.Scan(&user.ID, &user.Name, &user.Email, &user.CreatedAt, &user.UpdatedAt); err != nil {
|
||||
return model.UserPage{}, err
|
||||
}
|
||||
result.Items = append(result.Items, user)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return model.UserPage{}, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (r *MySQLUsers) Update(ctx context.Context, id int64, input model.UserInput) (model.User, error) {
|
||||
result, err := r.db.ExecContext(ctx, `
|
||||
UPDATE users SET name = ?, email = ?, updated_at = UTC_TIMESTAMP(6)
|
||||
WHERE id = ?`, input.Name, input.Email, id)
|
||||
if err != nil {
|
||||
return model.User{}, mapMySQLError(err)
|
||||
}
|
||||
changed, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
if changed == 0 {
|
||||
// MySQL may report zero affected rows when the replacement is identical;
|
||||
// distinguish that from an ID that does not exist.
|
||||
return r.Get(ctx, id)
|
||||
}
|
||||
return r.Get(ctx, id)
|
||||
}
|
||||
|
||||
func (r *MySQLUsers) Delete(ctx context.Context, id int64) error {
|
||||
result, err := r.db.ExecContext(ctx, `DELETE FROM users WHERE id = ?`, id)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
deleted, err := result.RowsAffected()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if deleted == 0 {
|
||||
return model.ErrUserNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func mapMySQLError(err error) error {
|
||||
var mysqlErr *mysql.MySQLError
|
||||
if errors.As(err, &mysqlErr) && mysqlErr.Number == 1062 {
|
||||
return model.ErrEmailTaken
|
||||
}
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,16 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"{{.Module}}/internal/model"
|
||||
)
|
||||
|
||||
// UserRepository describes the storage operations the business needs.
|
||||
// Implementations must atomically enforce unique, normalized email addresses.
|
||||
type UserRepository interface {
|
||||
Create(context.Context, model.UserInput) (model.User, error)
|
||||
Get(context.Context, int64) (model.User, error)
|
||||
List(context.Context, int, int) (model.UserPage, error)
|
||||
Update(context.Context, int64, model.UserInput) (model.User, error)
|
||||
Delete(context.Context, int64) error
|
||||
}
|
||||
@@ -2,20 +2,64 @@ package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/mail"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"{{.Module}}/internal/model"
|
||||
"{{.Module}}/internal/servicecontext"
|
||||
)
|
||||
|
||||
type Users struct {
|
||||
context *servicecontext.Context
|
||||
services *servicecontext.Context
|
||||
}
|
||||
|
||||
func NewUsers(context *servicecontext.Context) *Users {
|
||||
return &Users{context: context}
|
||||
func NewUsers(services *servicecontext.Context) *Users {
|
||||
return &Users{services: services}
|
||||
}
|
||||
|
||||
func (s *Users) Create(ctx context.Context, input model.UserInput) (model.User, error) {
|
||||
// Put business normalization and rules here.
|
||||
return s.context.Users.Create(ctx, input)
|
||||
input, err := normalize(input)
|
||||
if err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
user, err := s.services.Users.Create(ctx, input)
|
||||
if err != nil { s.services.Logger.ErrorContext(ctx,"user.create.failed","error",err); return model.User{},err }
|
||||
s.services.Logger.InfoContext(ctx,"user.created","userId",user.ID)
|
||||
return user,nil
|
||||
}
|
||||
|
||||
func (s *Users) Get(ctx context.Context, id int64) (model.User, error) {
|
||||
return s.services.Users.Get(ctx, id)
|
||||
}
|
||||
|
||||
func (s *Users) List(ctx context.Context, page, pageSize int) (model.UserPage, error) {
|
||||
if page < 1 || pageSize < 1 || pageSize > 100 {
|
||||
return model.UserPage{}, model.ErrInvalidPage
|
||||
}
|
||||
return s.services.Users.List(ctx, page, pageSize)
|
||||
}
|
||||
|
||||
func (s *Users) Update(ctx context.Context, id int64, input model.UserInput) (model.User, error) {
|
||||
input, err := normalize(input)
|
||||
if err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
return s.services.Users.Update(ctx, id, input)
|
||||
}
|
||||
|
||||
func (s *Users) Delete(ctx context.Context, id int64) error {
|
||||
return s.services.Users.Delete(ctx, id)
|
||||
}
|
||||
|
||||
func normalize(input model.UserInput) (model.UserInput, error) {
|
||||
input.Name = strings.TrimSpace(input.Name)
|
||||
// This example treats the entire email address as case-insensitive.
|
||||
input.Email = strings.ToLower(strings.TrimSpace(input.Email))
|
||||
address, err := mail.ParseAddress(input.Email)
|
||||
if input.Name == "" || utf8.RuneCountInString(input.Name) > 100 ||
|
||||
len(input.Email) > 254 || err != nil || address.Address != input.Email {
|
||||
return model.UserInput{}, model.ErrInvalidUser
|
||||
}
|
||||
return input, nil
|
||||
}
|
||||
|
||||
@@ -1,6 +1,16 @@
|
||||
package servicecontext
|
||||
|
||||
import (
|
||||
"context"
|
||||
{{if and .Modules (ne .Storage "none")}}"errors"
|
||||
"net/http"
|
||||
"sync"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if eq .Database "mysql"}}"fmt"
|
||||
"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"github.com/go-sql-driver/mysql"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/repository"
|
||||
)
|
||||
@@ -10,8 +20,72 @@ import (
|
||||
type Context struct {
|
||||
Logger *logx.Logger
|
||||
Users repository.UserRepository
|
||||
Health *health.Checker
|
||||
{{if and .Modules (ne .Storage "none")}}
|
||||
Files storage.Store
|
||||
Uploads *storage.Uploader
|
||||
Downloads storage.Presigner
|
||||
UploadLinks storage.UploadPresigner
|
||||
FileHTTP http.Handler
|
||||
storageClose func() error
|
||||
closeMu sync.Mutex
|
||||
closed bool
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
{{end}}
|
||||
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||
}
|
||||
|
||||
func New(logger *logx.Logger, users repository.UserRepository) *Context {
|
||||
return &Context{Logger: logger, Users: users}
|
||||
// New initializes the project's dependencies before the HTTP listener opens.
|
||||
// The caller owns Logger; Close releases only resources created here.
|
||||
func New(ctx context.Context, logger *logx.Logger) (*Context, error) {
|
||||
if ctx == nil { ctx = context.Background() }
|
||||
if err := ctx.Err(); err != nil { return nil, err }
|
||||
checker, err := health.New()
|
||||
if err != nil { return nil, err }
|
||||
services := &Context{Logger: logger, Health: checker}
|
||||
{{if eq .Database "mysql"}}
|
||||
cfg, err := database.ConfigFromEnv("")
|
||||
if err != nil { return nil, err }
|
||||
if cfg.Driver != "mysql" { return nil, fmt.Errorf("this project requires GO_OMNI_DB_DRIVER=mysql") }
|
||||
dsn, err := mysql.ParseDSN(cfg.DSN)
|
||||
if err != nil { return nil, fmt.Errorf("parse mysql connection settings: %w", err) }
|
||||
if dsn.DBName == "" { return nil, fmt.Errorf("mysql connection settings require a database name") }
|
||||
// DATETIME columns are returned as time.Time in UTC in the business model.
|
||||
dsn.ParseTime = true
|
||||
dsn.Loc = time.UTC
|
||||
cfg.DSN = dsn.FormatDSN()
|
||||
db, err := database.Open(ctx, cfg)
|
||||
if err != nil { return nil, err }
|
||||
users := repository.NewMySQLUsers(db.DB)
|
||||
checkCtx, cancel := context.WithTimeout(ctx, cfg.PingTimeout)
|
||||
defer cancel()
|
||||
if err := users.CheckSchema(checkCtx); err != nil {
|
||||
_ = db.Close()
|
||||
return nil, fmt.Errorf("check users schema (apply schema.sql before starting): %w", err)
|
||||
}
|
||||
if err := checker.Add(health.Probe{Name: "database", Timeout: cfg.PingTimeout, Check: db.Health}); err != nil {
|
||||
_ = db.Close()
|
||||
return nil, err
|
||||
}
|
||||
services.DB, services.Users = db, users
|
||||
{{else}}
|
||||
services.Users = repository.NewMemoryUsers()
|
||||
{{end}}
|
||||
return services, nil
|
||||
}
|
||||
|
||||
// Close is safe to call on startup failure and again during application exit.
|
||||
func (c *Context) Close() error {
|
||||
{{if and .Modules (ne .Storage "none")}}
|
||||
if c==nil{return nil}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
c.closed=true
|
||||
c.closeOnce.Do(func(){
|
||||
if c.storageClose!=nil {c.closeErr=errors.Join(c.closeErr,c.storageClose())}
|
||||
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||
})
|
||||
return c.closeErr
|
||||
{{else}}{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}{{end}}
|
||||
{{if not (and .Modules (ne .Storage "none"))}}return nil{{end}}
|
||||
}
|
||||
|
||||
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
@@ -0,0 +1,97 @@
|
||||
# 按需装配的日志与文件模块
|
||||
|
||||
本项目选用日志输出 `{{.LogOutput}}`、格式 `{{.LogFormat}}`、对象存储 `{{.Storage}}`。数据库选择 `{{.Database}}` 独立于对象存储。
|
||||
|
||||
配置入口为 `etc/config.yaml`,通过 `go run ./cmd/server -config /path/to/config.yaml` 指定其他 JSON/YAML 文件;`-config ''` 只用代码默认值和环境变量。优先级:默认值 → 文件 → 非空环境变量。未知字段、无效参数会在监听前拒绝。
|
||||
|
||||
```sh
|
||||
go mod tidy
|
||||
{{if ne .Storage "none"}}# 示例文件接口使用单一共享 Token,不是用户/租户权限系统。
|
||||
export GO_OMNI_FILES_TOKEN="$(openssl rand -hex 32)"
|
||||
{{if eq .Storage "local"}}# 本地签名密钥需要稳定保存;换密钥后旧链接失效。
|
||||
export GO_OMNI_STORAGE_SIGNING_KEY="$(openssl rand -hex 32)"
|
||||
{{else}}# 配置当前存储平台已有的桶和凭据,不会自动创建桶。
|
||||
export GO_OMNI_STORAGE_BUCKET='your-bucket'
|
||||
export GO_OMNI_STORAGE_ACCESS_KEY='your-access-key'
|
||||
export GO_OMNI_STORAGE_SECRET_KEY='your-secret-key'
|
||||
{{if eq .Storage "s3"}}export GO_OMNI_STORAGE_ENDPOINT='s3.us-east-1.amazonaws.com'
|
||||
export GO_OMNI_STORAGE_REGION='us-east-1'
|
||||
{{else if eq .Storage "r2"}}export GO_OMNI_STORAGE_ACCOUNT_ID='your-32-character-account-id'
|
||||
{{else}}export GO_OMNI_STORAGE_REGION='cn-hangzhou'
|
||||
{{end}}{{end}}{{end}}
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
{{if eq .Database "mysql"}}还需按 README 配置独立 MySQL 数据库并应用 schema.sql;应用不自动迁移。{{end}}
|
||||
|
||||
默认仅监听 `127.0.0.1:8080`。`GO_OMNI_HTTP_ADDR` 覆盖地址;本地签名 URL 的外部地址通过 `GO_OMNI_STORAGE_PUBLIC_URL` 单独配置,始终以 `/storage/object` 结尾,不使用请求 Host 自动拼接可信链接。
|
||||
|
||||
## 日志
|
||||
|
||||
业务示例在创建用户/上传文件成功时通过 logx 写结构化日志。Context 原样传递 requestId,框架不自动增加用户、租户等业务字段。文件模式已配置轮转:100 MiB、10 份、7 天、压缩。调整 `server.log` 即可更改;`redactKeys` 可增加业务敏感键。
|
||||
|
||||
```sh
|
||||
curl -H 'Content-Type: application/json' -H 'X-Request-ID: example-request' \
|
||||
-d '{"name":"Example","email":"example@example.com"}' http://127.0.0.1:8080/users
|
||||
```
|
||||
|
||||
`GO_OMNI_LOG_LEVEL`、`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_FILE` 覆盖对应项;显式设置 `GO_OMNI_LOG_OUTPUT=stdout` 且未设置 LOG_FILE 时,切换到控制台并关闭文件轮转。慢请求阈值为 500ms,可通过 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 调整。字段脱敏不扫描 msg/error 中的自由文本;不要把凭据或签名 URL 拼到日志中。
|
||||
|
||||
{{if ne .Storage "none"}}## 文件示例
|
||||
|
||||
Handler → Service → storage 接口;底层构造与连接池集中在 `internal/servicecontext/storage.go`。示例接口要求 `Authorization: Bearer $GO_OMNI_FILES_TOKEN`,它授权这个示例的全部文件,实际业务应替换认证和对象归属检查。存储适配器不承担业务授权。
|
||||
|
||||
```sh
|
||||
printf 'hello go-omni' > /tmp/goOmniExample.txt
|
||||
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" \
|
||||
-H 'Content-Type: text/plain' -H 'X-Request-ID: file-example' \
|
||||
--data-binary @/tmp/goOmniExample.txt http://127.0.0.1:8080/files
|
||||
```
|
||||
|
||||
保存响应中 `data.key`,再请求下载链接(把示例 key 替换为返回值):
|
||||
|
||||
```sh
|
||||
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||
-d '{"key":"uploads/returned-key"}' http://127.0.0.1:8080/files/downloadLink
|
||||
```
|
||||
|
||||
返回 `data.url`,在有效期内 GET 即可下载,无需再携带 Token。签发前会检查对象存在;URL 默认 5 分钟有效,不写日志,响应设置 no-store。
|
||||
|
||||
需要直传时先申请链接:
|
||||
|
||||
```sh
|
||||
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||
-d '{"size":13,"contentType":"text/plain"}' http://127.0.0.1:8080/files/uploadLink
|
||||
```
|
||||
|
||||
响应包括随机 `data.key` 和 `data.request`(method/url/headers/expiresAt)。按返回的 method 和 headers 用原始文件字节上传到 URL;不能额外包 FormData。浏览器自动填写 Content-Length,其余签名 Header 原样发送;云桶需配置允许业务域名的 CORS。云端直传只绑定声明类型,实际内容需上传后验证;本地和 API 中转上传会检查允许类型及前缀嗅探结果。未完成内容验证前不要把暂存文件当作可信文件公开。
|
||||
|
||||
| 参数 | 默认值 / 环境变量 |
|
||||
| --- | --- |
|
||||
| 大小 | 32 MiB / GO_OMNI_STORAGE_MAX_BYTES |
|
||||
| 类型 | text/plain、image/png、image/jpeg、application/pdf / GO_OMNI_STORAGE_CONTENT_TYPES(逗号分隔) |
|
||||
| 链接有效期 | 5m / GO_OMNI_STORAGE_LINK_TTL(1s..1h) |
|
||||
| 请求超时 | 30s / GO_OMNI_STORAGE_REQUEST_TIMEOUT(>0 且 <=5m) |
|
||||
| 本地目录 | var/files / GO_OMNI_STORAGE_ROOT |
|
||||
| 本地外部 URL | http://127.0.0.1:8080/storage/object / GO_OMNI_STORAGE_PUBLIC_URL |
|
||||
| 云配置 | GO_OMNI_STORAGE_ENDPOINT、REGION、BUCKET、ACCOUNT_ID、SECURE |
|
||||
| 环境密钥 | GO_OMNI_FILES_TOKEN、GO_OMNI_STORAGE_SIGNING_KEY、GO_OMNI_STORAGE_ACCESS_KEY、SECRET_KEY、SESSION_TOKEN |
|
||||
|
||||
全部 JSON/YAML 字段使用 lowerCamelCase。密钥字段仅允许环境变量,配置文件中写入会报未知字段;不要把运行环境文件提交到仓库。S3 的 endpoint 为 host[:port],OSS 的可选 endpoint 为 HTTPS origin;R2 从 accountId 派生官方 endpoint,region 为 auto。仅 S3/MinIO 的本地测试可设置 SECURE=false。
|
||||
|
||||
错误响应:未授权 401、无 Content-Length 411、超限 413、类型不允许 415、长度不符/非法 key 400、对象不存在 404、存储超时 504,其余内部错误统一隐藏细节。底层签名入口遵循存储协议返回 201/403 等状态,流式传输且访问日志保留实际状态。
|
||||
|
||||
云客户端只在组合处创建,使用专用有界连接池;配置正确不代表桶和网络已经可用,首次真实调用仍可能失败。不创建桶、不修改桶权限、不初始化任何业务数据库。更换平台时替换组合处适配器,业务 Service 接口保持不变。
|
||||
{{end}}
|
||||
|
||||
## 生命周期与验证
|
||||
|
||||
启动失败会清理已创建资源;SIGINT/SIGTERM 后先等待 HTTP 请求退出,再关闭服务依赖,最后关闭日志。只关闭项目自己创建的资源,不关闭全局默认 HTTP Transport。
|
||||
|
||||
```sh
|
||||
go test -race ./...
|
||||
go build -o server ./cmd/server
|
||||
./server
|
||||
```
|
||||
|
||||
生成的测试覆盖配置优先级/校验{{if ne .Storage "none"}}、资源关闭幂等性和借用日志所有权、上传与临时链接、签名篡改、限额/类型错误。业务测试使用临时 Local,不访问云账号;云端配置需另做实网验收{{end}}。基础用户 CRUD 示例仍未启用认证,部署前按业务需要替换。
|
||||
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
@@ -0,0 +1,46 @@
|
||||
# Non-secret settings only. Environment variables override these values.
|
||||
server:
|
||||
address: '127.0.0.1:8080'
|
||||
enableLogger: true
|
||||
slowRequestThreshold: 500ms
|
||||
shutdownTimeout: 10s
|
||||
log:
|
||||
service: {{printf "%q" .Module}}
|
||||
level: info
|
||||
format: {{.LogFormat}}
|
||||
{{if eq .LogOutput "file"}} file: './var/log/server.log'
|
||||
rotation:
|
||||
enabled: true
|
||||
maxSizeMb: 100
|
||||
maxBackups: 10
|
||||
maxAgeDays: 7
|
||||
compress: true
|
||||
{{else}} output: {{.LogOutput}}
|
||||
{{end}} redactKeys: []
|
||||
{{if ne .Storage "none"}}storage:
|
||||
provider: {{.Storage}}
|
||||
maxBytes: 33554432
|
||||
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||
linkTtl: 5m
|
||||
requestTimeout: 30s
|
||||
{{if eq .Storage "local"}} root: './var/files'
|
||||
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
|
||||
secure: true
|
||||
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
|
||||
region: auto
|
||||
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
|
||||
region: us-east-1
|
||||
{{else}} region: cn-hangzhou
|
||||
endpoint: '' # empty uses the official regional endpoint
|
||||
{{end}}{{end}}{{end}}
|
||||
|
||||
{{if and .API (eq .Database "mysql")}}database:
|
||||
driver: mysql
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||
{{end}}
|
||||
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
@@ -0,0 +1,86 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"errors"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"{{.Module}}/internal/service"
|
||||
)
|
||||
|
||||
type Files struct {service *service.Files; maxBytes int64; timeout time.Duration}
|
||||
func NewFiles(service *service.Files,maxBytes int64,timeout time.Duration)*Files{return &Files{service:service,maxBytes:maxBytes,timeout:timeout}}
|
||||
|
||||
// RegisterRoutes protects example control endpoints with one shared demo token.
|
||||
// Replace this gate with project authentication and object ownership checks.
|
||||
// Signed object URLs carry their own short-lived authorization.
|
||||
func (h *Files) RegisterRoutes(application *omni.App,token string,signed http.Handler)error {
|
||||
if len(token)<32{return errors.New("file example requires a token of at least 32 bytes")}
|
||||
bounded:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
ctx,cancel:=context.WithTimeout(c.Context(),h.timeout);defer cancel();c.WithContext(ctx);return next(c)
|
||||
}}
|
||||
authorized:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
if len(c.Request().Header.Values("Authorization"))!=1 || subtle.ConstantTimeCompare([]byte(c.Header("Authorization")),[]byte("Bearer "+token))!=1{return omni.Unauthorized("UNAUTHORIZED","invalid file access token")}
|
||||
return next(c)
|
||||
}}
|
||||
group:=application.Group("/files",authorized,bounded)
|
||||
group.POST("",h.Upload)
|
||||
group.POST("/uploadLink",h.UploadLink)
|
||||
group.POST("/downloadLink",h.DownloadLink)
|
||||
if signed!=nil {application.Any("/storage/object",func(c *omni.Context)error{
|
||||
signed.ServeHTTP(contextWriter{c},c.Request());return nil
|
||||
},bounded)}
|
||||
return nil
|
||||
}
|
||||
func (h *Files) Upload(c *omni.Context)error {
|
||||
size:=c.Request().ContentLength
|
||||
if size<0{return omni.NewError(http.StatusLengthRequired,"LENGTH_REQUIRED","Content-Length is required")}
|
||||
if size>h.maxBytes{return omni.NewError(http.StatusRequestEntityTooLarge,"FILE_TOO_LARGE","file exceeds upload limit")}
|
||||
body:=http.MaxBytesReader(c.Raw(),c.Request().Body,h.maxBytes)
|
||||
defer body.Close()
|
||||
info,err:=h.service.Upload(c.Context(),body,size,c.Header("Content-Type"))
|
||||
if err!=nil{return fileError(err)}
|
||||
return c.Created(info)
|
||||
}
|
||||
func (h *Files) UploadLink(c *omni.Context)error {
|
||||
var input struct{Size int64 `json:"size" binding:"gte=0"`;ContentType string `json:"contentType" binding:"required,max=256"`}
|
||||
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||
link,err:=h.service.UploadLink(c.Context(),input.Size,input.ContentType)
|
||||
if err!=nil{return fileError(err)}
|
||||
c.SetResponseHeader("Cache-Control","no-store")
|
||||
return c.OK(link)
|
||||
}
|
||||
func (h *Files) DownloadLink(c *omni.Context)error {
|
||||
var input struct{Key string `json:"key" binding:"required,max=1024"`}
|
||||
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||
if err:=storage.Check(c.Context(),input.Key);err!=nil{return omni.BadRequest("INVALID_KEY","invalid object key")}
|
||||
link,err:=h.service.DownloadLink(c.Context(),input.Key)
|
||||
if err!=nil{return fileError(err)}
|
||||
c.SetResponseHeader("Cache-Control","no-store")
|
||||
return c.OK(map[string]string{"url":link})
|
||||
}
|
||||
func fileError(err error)error {
|
||||
var limit *http.MaxBytesError
|
||||
switch {
|
||||
case errors.Is(err,storage.ErrTooLarge),errors.As(err,&limit):return omni.NewError(413,"FILE_TOO_LARGE","file exceeds upload limit").WithCause(err)
|
||||
case errors.Is(err,storage.ErrContentType):return omni.NewError(415,"CONTENT_TYPE_NOT_ALLOWED","file content type is not allowed").WithCause(err)
|
||||
case errors.Is(err,storage.ErrSizeMismatch):return omni.BadRequest("SIZE_MISMATCH","file size does not match Content-Length").WithCause(err)
|
||||
case errors.Is(err,storage.ErrNotFound):return omni.NotFound("FILE_NOT_FOUND","file was not found").WithCause(err)
|
||||
case errors.Is(err,context.DeadlineExceeded):return omni.NewError(504,"STORAGE_TIMEOUT","storage operation timed out").WithCause(err)
|
||||
default:return err
|
||||
}
|
||||
}
|
||||
|
||||
// Track raw streaming handler responses through Omni so access logs retain the
|
||||
// actual status (including failed signatures), without buffering object bodies.
|
||||
type contextWriter struct{ctx *omni.Context}
|
||||
func (w contextWriter) Header()http.Header{return w.ctx.Raw().Header()}
|
||||
func (w contextWriter) WriteHeader(status int){_ = w.ctx.WriteResponse(status,nil,nil)}
|
||||
func (w contextWriter) Write(data []byte)(int,error){
|
||||
if !w.ctx.Responded(){if err:=w.ctx.WriteResponse(http.StatusOK,nil,nil);err!=nil{return 0,err}}
|
||||
return w.ctx.Raw().Write(data)
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package handler_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"{{.Module}}/internal/handler"
|
||||
"{{.Module}}/internal/service"
|
||||
)
|
||||
|
||||
// Exercise business code using Local even when production selects a cloud
|
||||
// adapter. These tests do not use cloud accounts or network credentials.
|
||||
func TestFileUploadAndLinks(t *testing.T){
|
||||
files,err:=local.New(t.TempDir(),64);if err!=nil{t.Fatal(err)}
|
||||
policy:=storage.UploadPolicy{MaxBytes:64,ContentTypes:[]string{"text/plain"}}
|
||||
signer,err:=storage.NewUploadSigner(files,"http://localhost/storage/object",bytes.Repeat([]byte{1},32),policy);if err!=nil{t.Fatal(err)}
|
||||
uploader,err:=storage.NewUploader(files,policy);if err!=nil{t.Fatal(err)}
|
||||
var logs bytes.Buffer
|
||||
logger,err:=logx.New(logx.Config{Writer:&logs});if err!=nil{t.Fatal(err)}
|
||||
logx.SetDefault(logger);defer logx.SetDefault(nil)
|
||||
app:=omni.New(omni.WithLogger(true))
|
||||
token:=strings.Repeat("t",32)
|
||||
svc:=service.NewFiles(files,uploader,signer,signer,logger,time.Minute)
|
||||
if err=handler.NewFiles(svc,64,time.Second).RegisterRoutes(app,token,signer);err!=nil{t.Fatal(err)}
|
||||
request:=func(method,path,body,contentType string,authorized bool)*httptest.ResponseRecorder{
|
||||
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type",contentType);r.Header.Set("X-Request-ID","upload-test")
|
||||
if authorized{r.Header.Set("Authorization","Bearer "+token)}
|
||||
w:=httptest.NewRecorder();app.ServeHTTP(w,r);return w
|
||||
}
|
||||
if w:=request("POST","/files","hello","text/plain",false);w.Code!=401{t.Fatal(w.Code)}
|
||||
w:=request("POST","/files","hello","text/plain",true)
|
||||
var saved struct{Data storage.Info}
|
||||
if w.Code!=201 || json.Unmarshal(w.Body.Bytes(),&saved)!=nil || saved.Data.Size!=5{t.Fatal(w.Code,w.Body.String())}
|
||||
raw,_:=json.Marshal(map[string]string{"key":saved.Data.Key})
|
||||
w=request("POST","/files/downloadLink",string(raw),"application/json",true)
|
||||
var download struct{Data struct{URL string}}
|
||||
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&download)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||
w=request("GET",download.Data.URL,"","",false)
|
||||
if w.Code!=200 || w.Body.String()!="hello"{t.Fatal(w.Code,w.Body.String())}
|
||||
u,_:=url.Parse(download.Data.URL);q:=u.Query();q.Set("signature","tampered");u.RawQuery=q.Encode()
|
||||
if w=request("GET",u.String(),"","",false);w.Code!=403{t.Fatal(w.Code)}
|
||||
for _,tc:=range []struct{path,body,contentType string;status int}{
|
||||
{"/files",strings.Repeat("x",65),"text/plain",413},
|
||||
{"/files","hello","image/png",415},
|
||||
{"/files","<html>hello</html>","text/plain",415},
|
||||
{"/files/downloadLink",`{"key":"../bad"}`,"application/json",400},
|
||||
{"/files/downloadLink",`{"key":"missing"}`,"application/json",404},
|
||||
{"/files/uploadLink",`{"size":65,"contentType":"text/plain"}`,"application/json",413},
|
||||
{"/files/uploadLink",`{"size":5,"contentType":"image/png"}`,"application/json",415},
|
||||
}{if w=request("POST",tc.path,tc.body,tc.contentType,true);w.Code!=tc.status{t.Fatal(tc.path,w.Code,w.Body.String())}}
|
||||
w=request("POST","/files/uploadLink",`{"size":5,"contentType":"text/plain"}`,"application/json",true)
|
||||
var direct struct{Data service.FileUploadLink}
|
||||
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&direct)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||
if w=request("PUT",direct.Data.Request.URL,"world","text/plain",false);w.Code!=201{t.Fatal(w.Code,w.Body.String())}
|
||||
object,err:=files.Get(context.Background(),direct.Data.Key);if err!=nil{t.Fatal(err)};body,err:=io.ReadAll(object.Body);object.Body.Close();if err!=nil || string(body)!="world"{t.Fatal(err,string(body))}
|
||||
decoder:=json.NewDecoder(&logs);sawBusiness,sawDenied:=false,false
|
||||
for {var entry map[string]any;err:=decoder.Decode(&entry);if err==io.EOF{break};if err!=nil{t.Fatal(err)}
|
||||
if entry["requestId"]!="upload-test"{t.Fatal("lost request correlation")}
|
||||
if entry["msg"]=="file.uploaded"{sawBusiness=true}
|
||||
if entry["msg"]=="http.request" && entry["path"]=="/storage/object" && entry["status"]==float64(http.StatusForbidden){sawDenied=true}
|
||||
}
|
||||
if !sawBusiness || !sawDenied{t.Fatal("business log or signed-handler status missing")}
|
||||
}
|
||||
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
@@ -0,0 +1,61 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"io"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
)
|
||||
|
||||
// Files borrows provider-neutral capabilities; it never creates or closes clients.
|
||||
// Authentication/ownership rules belong to this project, outside storage adapters.
|
||||
type Files struct {
|
||||
store storage.Store
|
||||
uploader *storage.Uploader
|
||||
downloads storage.Presigner
|
||||
uploadLinks storage.UploadPresigner
|
||||
logger *logx.Logger
|
||||
ttl time.Duration
|
||||
}
|
||||
func NewFiles(store storage.Store,uploader *storage.Uploader,downloads storage.Presigner,uploadLinks storage.UploadPresigner,logger *logx.Logger,ttl time.Duration)*Files{
|
||||
return &Files{store:store,uploader:uploader,downloads:downloads,uploadLinks:uploadLinks,logger:logger.With("component","files"),ttl:ttl}
|
||||
}
|
||||
func newFileKey()(string,error){
|
||||
var id [16]byte
|
||||
if _,err:=rand.Read(id[:]);err!=nil{return "",err}
|
||||
return "uploads/"+hex.EncodeToString(id[:]),nil
|
||||
}
|
||||
func (s *Files) Upload(ctx context.Context,body io.Reader,size int64,contentType string)(storage.Info,error){
|
||||
key,err:=newFileKey();if err!=nil{return storage.Info{},err}
|
||||
info,err:=s.uploader.Put(ctx,key,body,size,contentType)
|
||||
if err!=nil{s.logger.WarnContext(ctx,"file.upload.failed","error",err);return storage.Info{},err}
|
||||
s.logger.InfoContext(ctx,"file.uploaded","key",info.Key,"size",info.Size,"contentType",info.ContentType)
|
||||
return info,nil
|
||||
}
|
||||
type FileUploadLink struct {
|
||||
Key string `json:"key"`
|
||||
Request storage.UploadRequest `json:"request"`
|
||||
}
|
||||
func (s *Files) UploadLink(ctx context.Context,size int64,contentType string)(FileUploadLink,error){
|
||||
key,err:=newFileKey();if err!=nil{return FileUploadLink{},err}
|
||||
// Check the same policy before issuing a local or cloud capability.
|
||||
// Cloud byte-level checks must happen in a post-upload validation workflow.
|
||||
request,err:=s.uploadLinks.PresignPut(ctx,key,size,contentType,s.ttl)
|
||||
if err!=nil{return FileUploadLink{},err}
|
||||
s.logger.InfoContext(ctx,"file.uploadLink.issued","key",key)
|
||||
return FileUploadLink{Key:key,Request:request},nil
|
||||
}
|
||||
func (s *Files) DownloadLink(ctx context.Context,key string)(string,error){
|
||||
if err:=storage.Check(ctx,key);err!=nil{return "",err}
|
||||
// Confirm the object exists without buffering its contents.
|
||||
object,err:=s.store.Get(ctx,key);if err!=nil{return "",err}
|
||||
if err=object.Body.Close();err!=nil{return "",err}
|
||||
link,err:=s.downloads.PresignGet(ctx,key,s.ttl)
|
||||
if err!=nil{return "",err}
|
||||
s.logger.InfoContext(ctx,"file.downloadLink.issued","key",key)
|
||||
return link.String(),nil
|
||||
}
|
||||
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
@@ -0,0 +1,6 @@
|
||||
/var/
|
||||
/server
|
||||
.env
|
||||
.env.*
|
||||
*.local.yaml
|
||||
*.local.json
|
||||
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
@@ -0,0 +1,50 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/handler"
|
||||
"{{.Module}}/internal/service"
|
||||
"{{.Module}}/internal/servicecontext"
|
||||
"{{.Module}}/internal/settings"
|
||||
)
|
||||
|
||||
func main(){
|
||||
path:=flag.String("config","etc/config.yaml","JSON/YAML settings; empty uses defaults and environment")
|
||||
flag.Parse()
|
||||
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||
}
|
||||
func run(path string)(runErr error){
|
||||
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||
defer stop()
|
||||
cfg,err:=settings.Load(path)
|
||||
if err!=nil{return err}
|
||||
logger,err:=logx.New(cfg.Server.Log)
|
||||
if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||
logx.SetDefault(logger)
|
||||
services,err:=servicecontext.New(ctx,logger)
|
||||
if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,services.Close())}()
|
||||
{{if ne .Storage "none"}}if err=services.InitStorage(ctx,cfg.Storage);err!=nil{return err}{{end}}
|
||||
application:=omni.New(omni.WithConfig(cfg.Server))
|
||||
application.OnShutdown(logger.Close)
|
||||
application.OnShutdown(services.Close)
|
||||
application.GET("/health/live",services.Health.LivenessHandler())
|
||||
application.GET("/health/ready",services.Health.ReadinessHandler())
|
||||
handler.NewUsers(service.NewUsers(services)).RegisterRoutes(application)
|
||||
{{if ne .Storage "none"}}
|
||||
files:=service.NewFiles(services.Files,services.Uploads,services.Downloads,services.UploadLinks,logger,cfg.Storage.LinkTTL)
|
||||
if err=handler.NewFiles(files,cfg.Storage.MaxBytes,cfg.Storage.RequestTimeout).RegisterRoutes(application,cfg.Storage.FilesToken,services.FileHTTP);err!=nil{return err}
|
||||
{{end}}
|
||||
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||
return application.RunContext(ctx,cfg.Server.Address)
|
||||
}
|
||||
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
@@ -0,0 +1,140 @@
|
||||
// Package settings owns this project's configuration; it adds no business identity model.
|
||||
package settings
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
{{if ne .Storage "none"}}"fmt"
|
||||
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||
"strconv"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Server config.Config `json:"server"`
|
||||
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
|
||||
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
server:=config.Default()
|
||||
server.Address="127.0.0.1:8080"
|
||||
server.EnableLogger=true
|
||||
server.SlowRequestThreshold=500*time.Millisecond
|
||||
server.Log.Service={{printf "%q" .Module}}
|
||||
server.Log.Format={{printf "%q" .LogFormat}}
|
||||
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
|
||||
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
|
||||
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
|
||||
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
|
||||
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
|
||||
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
|
||||
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
|
||||
{{if eq .Storage "r2"}}Region:"auto",{{end}}
|
||||
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
|
||||
}{{end}}}
|
||||
}
|
||||
|
||||
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||
// Pass an empty path to use only defaults and environment.
|
||||
func Load(path string)(Config,error) {
|
||||
cfg:=Default()
|
||||
var err error
|
||||
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
|
||||
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
|
||||
if err!=nil{return Config{},err}
|
||||
// A stream explicitly selected by environment overrides a file profile.
|
||||
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
|
||||
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
|
||||
{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
|
||||
if err=cfg.Validate();err!=nil{return Config{},err}
|
||||
return cfg,nil
|
||||
}
|
||||
func (c Config) Validate()error {
|
||||
{{if and .API (eq .Database "mysql")}}
|
||||
if err:=c.Database.Validate();err!=nil{return err}
|
||||
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
|
||||
{{end}}
|
||||
if err:=c.Server.Validate();err!=nil{return err}
|
||||
if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")}
|
||||
switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")}
|
||||
{{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}}
|
||||
}
|
||||
|
||||
{{if ne .Storage "none"}}
|
||||
type StorageConfig struct {
|
||||
Provider string `json:"provider"`
|
||||
Root string `json:"root"`
|
||||
PublicURL string `json:"publicUrl"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
Region string `json:"region"`
|
||||
Bucket string `json:"bucket"`
|
||||
AccountID string `json:"accountId"`
|
||||
Secure bool `json:"secure"`
|
||||
MaxBytes int64 `json:"maxBytes"`
|
||||
ContentTypes []string `json:"contentTypes"`
|
||||
LinkTTL time.Duration `json:"linkTtl"`
|
||||
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||
// Secrets are environment-only and cannot be populated by config files.
|
||||
FilesToken string `json:"-"`
|
||||
SigningKey string `json:"-"`
|
||||
AccessKey string `json:"-"`
|
||||
SecretKey string `json:"-"`
|
||||
SessionToken string `json:"-"`
|
||||
}
|
||||
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||
func (s *StorageConfig) applyEnv()error {
|
||||
values:=map[string]*string{
|
||||
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
|
||||
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
|
||||
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
|
||||
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
|
||||
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
|
||||
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
|
||||
}
|
||||
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
|
||||
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
|
||||
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
|
||||
}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
|
||||
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
|
||||
return nil
|
||||
}
|
||||
func (s StorageConfig) Validate()error {
|
||||
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
|
||||
if err:=s.Policy().Validate();err!=nil{return err}
|
||||
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||
{{if eq .Storage "local"}}
|
||||
if s.Root=="" {return errors.New("storage.root is required")}
|
||||
u,err:=url.Parse(s.PublicURL)
|
||||
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||
{{else}}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||
{{if eq .Storage "r2"}}
|
||||
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||
{{else if eq .Storage "s3"}}
|
||||
u,err:=url.Parse("https://"+s.Endpoint)
|
||||
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
|
||||
{{else if eq .Storage "oss"}}
|
||||
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
|
||||
if !s.Secure{return errors.New("OSS requires secure=true")}
|
||||
{{end}}
|
||||
{{end}}
|
||||
return nil
|
||||
}
|
||||
{{end}}
|
||||
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
func environment(t *testing.T){
|
||||
t.Helper()
|
||||
for _,entry:=range os.Environ(){key,_,_:=strings.Cut(entry,"=");if strings.HasPrefix(key,"GO_OMNI_"){t.Setenv(key,"")}}
|
||||
{{if ne .Storage "none"}}
|
||||
t.Setenv("GO_OMNI_FILES_TOKEN",strings.Repeat("t",32))
|
||||
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY",strings.Repeat("s",32))
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","fixtureAccess")
|
||||
t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","fixtureSecret")
|
||||
t.Setenv("GO_OMNI_STORAGE_BUCKET","testbucket")
|
||||
{{if eq .Storage "s3"}}t.Setenv("GO_OMNI_STORAGE_ENDPOINT","127.0.0.1:9000"){{end}}
|
||||
{{if eq .Storage "r2"}}t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID",strings.Repeat("a",32)){{end}}
|
||||
{{end}}
|
||||
}
|
||||
func TestSettingsPrecedenceAndStrictness(t *testing.T){
|
||||
environment(t)
|
||||
path:=filepath.Join(t.TempDir(),"config.yaml")
|
||||
if err:=os.WriteFile(path,[]byte("server:\n address: '127.0.0.1:8123'\n log:\n level: debug\n"),0600);err!=nil{t.Fatal(err)}
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR","127.0.0.1:9123")
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout")
|
||||
cfg,err:=Load(path);if err!=nil{t.Fatal(err)}
|
||||
if cfg.Server.Address!="127.0.0.1:9123" || cfg.Server.Log.Level!="debug" || cfg.Server.Log.Output!="stdout" || cfg.Server.Log.File!="" || cfg.Server.Log.Rotation.Enabled {t.Fatal("configuration precedence failed")}
|
||||
for _,body:=range []string{"unexpected: true","server:\n slowRequestThreshold: -1s","server:\n log:\n format: invalid"{{if ne .Storage "none"}},"storage:\n filesToken: forbidden","storage:\n maxBytes: -1","storage:\n provider: unsupported"{{end}}}{
|
||||
if err:=os.WriteFile(path,[]byte(body),0600);err!=nil{t.Fatal(err)}
|
||||
if _,err:=Load(path);err==nil{t.Fatal("invalid configuration accepted")}
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
t.Setenv("GO_OMNI_FILES_TOKEN","")
|
||||
if _,err:=Load("");err==nil{t.Fatal("missing file token accepted")}
|
||||
{{end}}
|
||||
}
|
||||
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
@@ -0,0 +1,79 @@
|
||||
package servicecontext
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
{{if eq .Storage "local"}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
{{else}}
|
||||
"net"
|
||||
"net/http"
|
||||
"time"
|
||||
{{if eq .Storage "oss"}}
|
||||
ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss"
|
||||
ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
|
||||
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
|
||||
{{else}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/s3"
|
||||
"github.com/minio/minio-go/v7/pkg/credentials"
|
||||
{{end}}{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"{{.Module}}/internal/settings"
|
||||
)
|
||||
|
||||
// InitStorage is called once, before serving requests. The composition root
|
||||
// owns provider clients; business services receive only capability interfaces.
|
||||
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
|
||||
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
if c.closed{return errors.New("service dependencies are closed")}
|
||||
if err:=ctx.Err();err!=nil{return err}
|
||||
if c.Files!=nil{return errors.New("storage already initialized")}
|
||||
if err:=cfg.Validate();err!=nil{return err}
|
||||
{{if eq .Storage "local"}}
|
||||
files,err:=local.New(cfg.Root,cfg.MaxBytes)
|
||||
if err!=nil{return err}
|
||||
signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer
|
||||
{{else}}
|
||||
// Dedicated bounded pool, never the process-wide default transport.
|
||||
transport:=&http.Transport{
|
||||
Proxy:http.ProxyFromEnvironment,
|
||||
DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext,
|
||||
ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32,
|
||||
IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second,
|
||||
ResponseHeaderTimeout:cfg.RequestTimeout,
|
||||
}
|
||||
initialized:=false
|
||||
defer func(){if !initialized{transport.CloseIdleConnections()}}()
|
||||
{{if eq .Storage "oss"}}
|
||||
sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region).
|
||||
WithSignatureVersion(ossSDK.SignatureVersionV4).
|
||||
WithAdditionalHeaders([]string{"content-length"}).
|
||||
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)).
|
||||
WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout})
|
||||
if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)}
|
||||
files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes)
|
||||
{{else}}
|
||||
creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)
|
||||
{{if eq .Storage "r2"}}
|
||||
options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds)
|
||||
if err!=nil{return err}
|
||||
options.MaxBytes,options.Transport=cfg.MaxBytes,transport
|
||||
{{else}}
|
||||
options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport}
|
||||
{{end}}
|
||||
files,err:=s3.Open(options)
|
||||
{{end}}
|
||||
if err!=nil{return err}
|
||||
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
|
||||
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
|
||||
initialized=true
|
||||
{{end}}
|
||||
return nil
|
||||
}
|
||||
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
||||
package servicecontext
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/settings"
|
||||
)
|
||||
func TestStorageInitializationAndOwnership(t *testing.T){
|
||||
cfg:=settings.Default().Storage
|
||||
cfg.FilesToken=strings.Repeat("t",32)
|
||||
cfg.SigningKey=strings.Repeat("s",32)
|
||||
cfg.Root=t.TempDir()
|
||||
cfg.AccessKey,cfg.SecretKey,cfg.Bucket="fixtureAccess","fixtureSecret","testbucket"
|
||||
{{if eq .Storage "s3"}}cfg.Endpoint="127.0.0.1:9000"{{end}}
|
||||
{{if eq .Storage "r2"}}cfg.AccountID=strings.Repeat("a",32){{end}}
|
||||
var output bytes.Buffer
|
||||
logger,err:=logx.New(logx.Config{Writer:&output});if err!=nil{t.Fatal(err)};defer logger.Close()
|
||||
c:=&Context{Logger:logger}
|
||||
canceled,cancel:=context.WithCancel(context.Background());cancel()
|
||||
if err=c.InitStorage(canceled,cfg);err==nil || c.Files!=nil{t.Fatal("canceled startup initialized storage")}
|
||||
invalid:=cfg;invalid.FilesToken=""
|
||||
if err=c.InitStorage(context.Background(),invalid);err==nil || c.Files!=nil{t.Fatal("invalid settings initialized storage")}
|
||||
if err=c.InitStorage(context.Background(),cfg);err!=nil{t.Fatal(err)}
|
||||
if c.Files==nil || c.Uploads==nil || c.Downloads==nil || c.UploadLinks==nil{t.Fatal("dependencies not assembled")}
|
||||
if err=c.InitStorage(context.Background(),cfg);err==nil{t.Fatal("duplicate initialization accepted")}
|
||||
cleanup:=c.storageClose;count:=0
|
||||
c.storageClose=func()error{count++;if cleanup!=nil{return cleanup()};return nil}
|
||||
if err=c.Close();err!=nil{t.Fatal(err)};if err=c.Close();err!=nil{t.Fatal(err)}
|
||||
logger.Info("still usable")
|
||||
if count!=1 || !strings.Contains(output.String(),"still usable"){t.Fatal("resource ownership violated")}
|
||||
closed:=&Context{Logger:logger}
|
||||
if err=closed.Close();err!=nil{t.Fatal(err)}
|
||||
if err=closed.InitStorage(context.Background(),cfg);err==nil || closed.Files!=nil{t.Fatal("closed context reopened resources")}
|
||||
}
|
||||
9
cmd/go-omni/scaffold/templates/schema.sql.tmpl
Normal file
9
cmd/go-omni/scaffold/templates/schema.sql.tmpl
Normal file
@@ -0,0 +1,9 @@
|
||||
CREATE TABLE users (
|
||||
id BIGINT NOT NULL AUTO_INCREMENT,
|
||||
name VARCHAR(100) NOT NULL,
|
||||
email VARCHAR(254) NOT NULL,
|
||||
created_at DATETIME(6) NOT NULL,
|
||||
updated_at DATETIME(6) NOT NULL,
|
||||
PRIMARY KEY (id),
|
||||
UNIQUE KEY uq_users_email (email)
|
||||
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;
|
||||
43
cmd/go-omni/scaffold/version.go
Normal file
43
cmd/go-omni/scaffold/version.go
Normal file
@@ -0,0 +1,43 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"regexp"
|
||||
"runtime/debug"
|
||||
)
|
||||
|
||||
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
|
||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||
// downloaded module version, which also has a module checksum.
|
||||
const defaultFrameworkVersion = "v0.8.0-preview"
|
||||
|
||||
// This target is not a claim that the release already exists. Source users
|
||||
// select a local framework checkout until the module templates are published.
|
||||
const modulesFrameworkVersion = "v0.7.0-preview"
|
||||
|
||||
func installedFrameworkVersion() string {
|
||||
info, _ := debug.ReadBuildInfo()
|
||||
if info != nil && info.Main.Path == frameworkModule && info.Main.Sum != "" && validFrameworkVersion(info.Main.Version) {
|
||||
return info.Main.Version
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
||||
|
||||
func validFrameworkVersion(version string) bool {
|
||||
return frameworkVersionPattern.MatchString(version)
|
||||
}
|
||||
|
||||
func currentFrameworkVersion() string {
|
||||
info, _ := debug.ReadBuildInfo()
|
||||
return frameworkVersion(info)
|
||||
}
|
||||
|
||||
func frameworkVersion(info *debug.BuildInfo) string {
|
||||
if info != nil && info.Main.Path == frameworkModule && info.Main.Sum != "" && validFrameworkVersion(info.Main.Version) {
|
||||
return info.Main.Version
|
||||
}
|
||||
return defaultFrameworkVersion
|
||||
}
|
||||
111
config/config.go
111
config/config.go
@@ -4,24 +4,44 @@ import (
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/response"
|
||||
)
|
||||
|
||||
const defaultRequestIDHeader = "X-Request-ID"
|
||||
|
||||
const (
|
||||
TransportNetHTTP = "nethttp"
|
||||
TransportNative = "native"
|
||||
TransportHTTP2 = "http2"
|
||||
)
|
||||
|
||||
// TLSConfig enables HTTPS when both certificate paths are provided.
|
||||
type TLSConfig struct {
|
||||
CertFile string `json:"certFile"`
|
||||
KeyFile string `json:"keyFile"`
|
||||
}
|
||||
|
||||
// Config controls the default runtime behavior of an App.
|
||||
type Config struct {
|
||||
Mode string
|
||||
Address string
|
||||
RequestIDHeader string
|
||||
EnableLogger bool
|
||||
ShutdownTimeout time.Duration
|
||||
Log logx.Config
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Log logx.Config `json:"log"`
|
||||
Response response.Config `json:"response"`
|
||||
ResponseEncoder response.Encoder `json:"-"`
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
return Config{
|
||||
Address: ":8080",
|
||||
Transport: TransportNetHTTP,
|
||||
RequestIDHeader: defaultRequestIDHeader,
|
||||
ShutdownTimeout: 10 * time.Second,
|
||||
Log: logx.Default(),
|
||||
@@ -30,6 +50,12 @@ func Default() Config {
|
||||
|
||||
// Validate checks configuration values before an application starts.
|
||||
func (c Config) Validate() error {
|
||||
if err := c.Response.Validate(); err != nil {
|
||||
return err
|
||||
}
|
||||
if c.SlowRequestThreshold < 0 {
|
||||
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
|
||||
}
|
||||
if strings.TrimSpace(c.Address) == "" {
|
||||
return &ConfigError{Field: "address", Message: "is required"}
|
||||
}
|
||||
@@ -39,6 +65,27 @@ func (c Config) Validate() error {
|
||||
if c.ShutdownTimeout <= 0 {
|
||||
return &ConfigError{Field: "shutdown_timeout", Message: "must be positive"}
|
||||
}
|
||||
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
|
||||
return &ConfigError{Field: "trusted_proxies", Message: err.Error()}
|
||||
}
|
||||
return c.ValidateTransport()
|
||||
}
|
||||
|
||||
// ValidateTransport checks built-in transport selection without opening files
|
||||
// or sockets. Empty Transport preserves the standard net/http default.
|
||||
func (c Config) ValidateTransport() error {
|
||||
switch c.Transport {
|
||||
case "", TransportNetHTTP, TransportNative, TransportHTTP2:
|
||||
default:
|
||||
return &ConfigError{Field: "transport", Message: "must be nethttp, native or http2"}
|
||||
}
|
||||
cert, key := strings.TrimSpace(c.TLS.CertFile), strings.TrimSpace(c.TLS.KeyFile)
|
||||
if (cert == "") != (key == "") {
|
||||
return &ConfigError{Field: "tls", Message: "requires both certFile and keyFile"}
|
||||
}
|
||||
if c.Transport == TransportHTTP2 && cert == "" {
|
||||
return &ConfigError{Field: "tls", Message: "is required for the experimental http2 transport"}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -53,9 +100,14 @@ func (e *ConfigError) Error() string { return "config " + e.Field + " " + e.Mess
|
||||
// Option changes an App's configuration.
|
||||
type Option func(*Config)
|
||||
|
||||
// WithMode sets the default transport's running mode.
|
||||
func WithMode(mode string) Option {
|
||||
return func(cfg *Config) { cfg.Mode = mode }
|
||||
// WithTransport selects a built-in server. HTTP/2 is experimental and requires TLS.
|
||||
func WithTransport(name string) Option {
|
||||
return func(cfg *Config) { cfg.Transport = name }
|
||||
}
|
||||
|
||||
// WithTLS enables HTTPS. Certificates are loaded before Run opens its listener.
|
||||
func WithTLS(certFile, keyFile string) Option {
|
||||
return func(cfg *Config) { cfg.TLS = TLSConfig{CertFile: certFile, KeyFile: keyFile} }
|
||||
}
|
||||
|
||||
// WithRequestIDHeader changes the header used to read and return request IDs.
|
||||
@@ -72,15 +124,52 @@ func WithLogger(enabled bool) Option {
|
||||
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
||||
}
|
||||
|
||||
// WithSlowRequestThreshold enables access logs and warns after this duration.
|
||||
func WithSlowRequestThreshold(threshold time.Duration) Option {
|
||||
return func(cfg *Config) { cfg.SlowRequestThreshold = threshold; cfg.EnableLogger = true }
|
||||
}
|
||||
|
||||
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
||||
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
||||
// by Config.Validate and panic when passed directly to App.New.
|
||||
func WithTrustedProxies(proxies ...string) Option {
|
||||
proxies = append([]string(nil), proxies...)
|
||||
return func(cfg *Config) { cfg.TrustedProxies = append([]string(nil), proxies...) }
|
||||
}
|
||||
|
||||
// WithConfig applies a complete configuration loaded from a file or
|
||||
// environment. Later options can still override individual fields.
|
||||
func WithConfig(settings Config) Option {
|
||||
return func(cfg *Config) { *cfg = settings }
|
||||
settings.Response = settings.Response.Clone()
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
return func(cfg *Config) {
|
||||
*cfg = settings
|
||||
cfg.Response = settings.Response.Clone()
|
||||
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
cfg.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
}
|
||||
}
|
||||
|
||||
// WithResponse configures this application's default JSON envelope.
|
||||
func WithResponse(settings response.Config) Option {
|
||||
settings = settings.Clone()
|
||||
return func(cfg *Config) { cfg.Response = settings.Clone(); cfg.ResponseEncoder = nil }
|
||||
}
|
||||
|
||||
// WithResponseEncoder replaces the default encoder with application code.
|
||||
// OpenAPI for custom encoders must describe their actual wire contract explicitly.
|
||||
func WithResponseEncoder(encoder response.Encoder) Option {
|
||||
return func(cfg *Config) { cfg.ResponseEncoder = encoder }
|
||||
}
|
||||
|
||||
// WithLogConfig configures the logger used by the application runtime.
|
||||
func WithLogConfig(settings logx.Config) Option {
|
||||
return func(cfg *Config) { cfg.Log = settings }
|
||||
settings.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||
return func(cfg *Config) {
|
||||
cfg.Log = settings
|
||||
cfg.Log.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||
}
|
||||
}
|
||||
|
||||
// WithShutdownTimeout controls how long RunContext waits for active requests.
|
||||
|
||||
@@ -8,10 +8,13 @@ import (
|
||||
|
||||
func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
for _, key := range []string{
|
||||
"GO_OMNI_MODE", "GO_OMNI_HTTP_ADDR", "GO_OMNI_REQUEST_ID_HEADER", "GO_OMNI_ENABLE_LOGGER",
|
||||
"GO_OMNI_HTTP_ADDR", "GO_OMNI_REQUEST_ID_HEADER", "GO_OMNI_ENABLE_LOGGER",
|
||||
"GO_OMNI_TRUSTED_PROXIES",
|
||||
"GO_OMNI_TRANSPORT", "GO_OMNI_TLS_CERT_FILE", "GO_OMNI_TLS_KEY_FILE",
|
||||
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
||||
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
||||
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
||||
"GO_OMNI_SLOW_REQUEST_THRESHOLD",
|
||||
} {
|
||||
t.Setenv(key, "")
|
||||
}
|
||||
@@ -20,7 +23,7 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Address != ":8080" || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
if cfg.Address != ":8080" || cfg.Transport != TransportNetHTTP || cfg.TLS != (TLSConfig{}) || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
t.Fatalf("unexpected defaults: %+v", cfg)
|
||||
}
|
||||
if cfg.EnableLogger || cfg.Log.Level != "info" || cfg.Log.Format != "json" || cfg.Log.Output != "stderr" {
|
||||
@@ -28,10 +31,48 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTransportEnvironment(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport, cert, key, errorField string
|
||||
}{
|
||||
{"default", "", "", "", ""},
|
||||
{"native", "native", "", "", ""},
|
||||
{"nativeTLS", "native", "server.crt", "server.key", ""},
|
||||
{"http2", "http2", "server.crt", "server.key", ""},
|
||||
{"unknown", "typo", "", "", "transport"},
|
||||
{"certOnly", "nethttp", "server.crt", "", "tls"},
|
||||
{"keyOnly", "native", "", "server.key", "tls"},
|
||||
{"http2WithoutTLS", "http2", "", "", "tls"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
t.Setenv("TRANSPORT_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_CERT_FILE", test.cert)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_KEY_FILE", test.key)
|
||||
cfg, err := LoadFromEnv("TRANSPORT_TEST_")
|
||||
if test.errorField != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorField) {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := test.transport
|
||||
if want == "" {
|
||||
want = TransportNetHTTP
|
||||
}
|
||||
if cfg.Transport != want || cfg.TLS.CertFile != test.cert || cfg.TLS.KeyFile != test.key {
|
||||
t.Fatalf("settings: %+v", cfg)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_MODE", "release")
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:9090")
|
||||
t.Setenv("GO_OMNI_REQUEST_ID_HEADER", "X-Correlation-ID")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "127.0.0.1, 10.0.0.0/8")
|
||||
t.Setenv("GO_OMNI_ENABLE_LOGGER", "true")
|
||||
t.Setenv("GO_OMNI_LOG_LEVEL", "debug")
|
||||
t.Setenv("GO_OMNI_LOG_FORMAT", "text")
|
||||
@@ -40,17 +81,21 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
|
||||
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
|
||||
t.Setenv("GO_OMNI_SLOW_REQUEST_THRESHOLD", "500ms")
|
||||
|
||||
cfg, err := LoadFromEnv("")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Mode != "release" || cfg.Address != "127.0.0.1:9090" || cfg.RequestIDHeader != "X-Correlation-ID" || !cfg.EnableLogger {
|
||||
if cfg.Address != "127.0.0.1:9090" || cfg.RequestIDHeader != "X-Correlation-ID" || !cfg.EnableLogger || len(cfg.TrustedProxies) != 2 || cfg.TrustedProxies[1] != "10.0.0.0/8" {
|
||||
t.Fatalf("unexpected application settings: %+v", cfg)
|
||||
}
|
||||
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
||||
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
|
||||
}
|
||||
if cfg.SlowRequestThreshold != 500*time.Millisecond {
|
||||
t.Fatal(cfg.SlowRequestThreshold)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
@@ -66,6 +111,11 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
}
|
||||
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "1s")
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "proxy.example.com")
|
||||
if _, err := LoadFromEnv(""); err == nil || !strings.Contains(err.Error(), "trusted_proxies") {
|
||||
t.Fatalf("expected invalid trusted proxy error, got %v", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_TRUSTED_PROXIES", "")
|
||||
cfg := Default()
|
||||
cfg.Address = ""
|
||||
if err := cfg.Validate(); err == nil || !strings.Contains(err.Error(), "address") {
|
||||
@@ -80,16 +130,40 @@ func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
|
||||
func TestWithConfigAndWithLogConfig(t *testing.T) {
|
||||
settings := Default()
|
||||
settings.Mode = "release"
|
||||
settings.Log.Service = "orders"
|
||||
settings.TrustedProxies = []string{"127.0.0.1"}
|
||||
cfg := Default()
|
||||
WithConfig(settings)(&cfg)
|
||||
if cfg.Mode != "release" || cfg.Log.Service != "orders" {
|
||||
if cfg.Log.Service != "orders" || len(cfg.TrustedProxies) != 1 {
|
||||
t.Fatalf("complete config was not applied: %+v", cfg)
|
||||
}
|
||||
settings.TrustedProxies[0] = "10.0.0.0/8"
|
||||
if cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("WithConfig should copy trusted proxy settings")
|
||||
}
|
||||
|
||||
WithLogConfig(Default().Log)(&cfg)
|
||||
if cfg.Log.Service != "" {
|
||||
t.Fatalf("log config was not replaced: %+v", cfg.Log)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyEnvPreservesProjectDefaults(t *testing.T) {
|
||||
cfg := Default()
|
||||
cfg.Address = "127.0.0.1:8123"
|
||||
cfg.Log.RedactKeys = []string{"bankAccount"}
|
||||
cfg.TrustedProxies = []string{"127.0.0.1"}
|
||||
t.Setenv("OVERLAY_TEST_LOG_LEVEL", "debug")
|
||||
got, err := ApplyEnv(cfg, "OVERLAY_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Address != cfg.Address || got.Log.Level != "debug" {
|
||||
t.Fatal(got)
|
||||
}
|
||||
got.Log.RedactKeys[0] = "changed"
|
||||
got.TrustedProxies[0] = "192.0.2.1"
|
||||
if cfg.Log.RedactKeys[0] != "bankAccount" || cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("caller-owned defaults mutated")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,14 +13,33 @@ const defaultEnvPrefix = "GO_OMNI_"
|
||||
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
|
||||
// database settings remain owned by database.ConfigFromEnv.
|
||||
func LoadFromEnv(prefix string) (Config, error) {
|
||||
return applyEnv(Default(), prefix)
|
||||
}
|
||||
|
||||
// ApplyEnv overlays nonempty environment values on a caller's defaults or file
|
||||
// settings, then validates the result. Caller-owned slices are not retained.
|
||||
func ApplyEnv(defaults Config, prefix string) (Config, error) {
|
||||
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
|
||||
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
|
||||
return applyEnv(defaults, prefix)
|
||||
}
|
||||
|
||||
func applyEnv(cfg Config, prefix string) (Config, error) {
|
||||
prefix = strings.TrimSpace(prefix)
|
||||
if prefix == "" {
|
||||
prefix = defaultEnvPrefix
|
||||
}
|
||||
cfg := Default()
|
||||
cfg.Mode = envString(prefix+"MODE", cfg.Mode)
|
||||
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
|
||||
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
|
||||
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
|
||||
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
|
||||
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
|
||||
cfg.TrustedProxies = strings.Split(value, ",")
|
||||
for i := range cfg.TrustedProxies {
|
||||
cfg.TrustedProxies[i] = strings.TrimSpace(cfg.TrustedProxies[i])
|
||||
}
|
||||
}
|
||||
cfg.Log.Level = envString(prefix+"LOG_LEVEL", cfg.Log.Level)
|
||||
cfg.Log.Format = envString(prefix+"LOG_FORMAT", cfg.Log.Format)
|
||||
cfg.Log.Output = envString(prefix+"LOG_OUTPUT", cfg.Log.Output)
|
||||
@@ -35,6 +54,12 @@ func LoadFromEnv(prefix string) (Config, error) {
|
||||
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "SLOW_REQUEST_THRESHOLD")); value != "" {
|
||||
cfg.SlowRequestThreshold, err = time.ParseDuration(value)
|
||||
if err != nil {
|
||||
return Config{}, fmt.Errorf("%sSLOW_REQUEST_THRESHOLD must be a duration: %w", prefix, err)
|
||||
}
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
|
||||
cfg.ShutdownTimeout, err = time.ParseDuration(value)
|
||||
if err != nil {
|
||||
|
||||
205
config/file.go
Normal file
205
config/file.go
Normal file
@@ -0,0 +1,205 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gopkg.in/yaml.v3"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
|
||||
// field names for both formats, rejects unknown fields and accepts durations
|
||||
// such as "3s". Application-specific validation remains with the caller.
|
||||
func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||
var zero T
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if len(data) > 1<<20 {
|
||||
return zero, errors.New("config: file exceeds 1 MiB")
|
||||
}
|
||||
var value any
|
||||
switch strings.ToLower(filepath.Ext(path)) {
|
||||
case ".yaml", ".yml":
|
||||
dec := yaml.NewDecoder(bytes.NewReader(data))
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one YAML document required")
|
||||
}
|
||||
case ".json":
|
||||
dec := json.NewDecoder(bytes.NewReader(data))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one JSON document required")
|
||||
}
|
||||
default:
|
||||
return zero, errors.New("config: expected .json, .yaml or .yml")
|
||||
}
|
||||
if _, ok := value.(map[string]any); !ok {
|
||||
return zero, errors.New("config: root must be an object")
|
||||
}
|
||||
value, err = durations(value, reflect.TypeOf(defaults))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
// Deep copy defaults so errors cannot partially mutate caller-owned maps.
|
||||
original, err := json.Marshal(defaults)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var result T
|
||||
if err = json.Unmarshal(original, &result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(encoded))
|
||||
dec.DisallowUnknownFields()
|
||||
if err = dec.Decode(&result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func durations(v any, t reflect.Type) (any, error) {
|
||||
if t == nil {
|
||||
return v, nil
|
||||
}
|
||||
for t.Kind() == reflect.Pointer {
|
||||
t = t.Elem()
|
||||
}
|
||||
if t == reflect.TypeOf(time.Duration(0)) {
|
||||
if s, ok := v.(string); ok {
|
||||
d, err := time.ParseDuration(s)
|
||||
return int64(d), err
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
if t.Kind() == reflect.Struct {
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
for n := 0; n < t.NumField(); n++ {
|
||||
field := t.Field(n)
|
||||
name := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||
if name == "-" {
|
||||
continue
|
||||
}
|
||||
if name == "" {
|
||||
name = field.Name
|
||||
}
|
||||
if item, exists := m[name]; exists {
|
||||
converted, err := durations(item, field.Type)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m[name] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Slice {
|
||||
if items, ok := v.([]any); ok {
|
||||
for n, item := range items {
|
||||
converted, err := durations(item, t.Elem())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items[n] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
func LoadFile(path, prefix string) (Config, error) {
|
||||
cfg, err := DecodeFile(path, Default())
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return applyEnv(cfg, prefix)
|
||||
}
|
||||
|
||||
// Reloader only accepts log.level changes. Static changes and invalid files
|
||||
// leave the current snapshot and logger untouched. Watch does not own a goroutine.
|
||||
type Reloader struct {
|
||||
mu sync.Mutex
|
||||
path, prefix string
|
||||
current Config
|
||||
logger *logx.Logger
|
||||
}
|
||||
|
||||
func NewReloader(path, prefix string, logger *logx.Logger) (*Reloader, error) {
|
||||
cfg, err := LoadFile(path, prefix)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = logger.SetLevel(cfg.Log.Level); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Reloader{path: path, prefix: prefix, current: cfg, logger: logger}, nil
|
||||
}
|
||||
func (r *Reloader) Reload() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
next, err := LoadFile(r.path, r.prefix)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
comparable := next
|
||||
comparable.Log.Level = r.current.Log.Level
|
||||
if !reflect.DeepEqual(comparable, r.current) {
|
||||
return errors.New("config: static fields changed; restart required")
|
||||
}
|
||||
if err = r.logger.SetLevel(next.Log.Level); err != nil {
|
||||
return err
|
||||
}
|
||||
r.current = next
|
||||
return nil
|
||||
}
|
||||
func (r *Reloader) Snapshot() Config {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
cfg := r.current
|
||||
cfg.Response = r.current.Response.Clone()
|
||||
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
|
||||
cfg.Log.RedactKeys = append([]string(nil), cfg.Log.RedactKeys...)
|
||||
return cfg
|
||||
}
|
||||
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
|
||||
if ctx == nil || interval < 100*time.Millisecond {
|
||||
return errors.New("config: context and interval >=100ms required")
|
||||
}
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-ticker.C:
|
||||
if err := r.Reload(); err != nil && onError != nil {
|
||||
onError(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
59
config/file_test.go
Normal file
59
config/file_test.go
Normal file
@@ -0,0 +1,59 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestFileMergeAndSafeReload(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "service.yaml")
|
||||
write := func(raw string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: info\n")
|
||||
t.Setenv("TEST_MODULE_HTTP_ADDR", ":9123")
|
||||
cfg, err := LoadFile(path, "TEST_MODULE_")
|
||||
if err != nil || cfg.Address != ":9123" || cfg.ShutdownTimeout != 7*time.Second {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
var output bytes.Buffer
|
||||
lc := cfg.Log
|
||||
lc.Writer = &output
|
||||
l, err := logx.New(lc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer l.Close()
|
||||
child := l.With("scope", "test")
|
||||
r, err := NewReloader(path, "TEST_MODULE_", l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("hidden")
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: debug\n")
|
||||
if err = r.Reload(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("visible")
|
||||
if strings.Contains(output.String(), "hidden") || !strings.Contains(output.String(), "visible") {
|
||||
t.Fatal(output.String())
|
||||
}
|
||||
write("shutdownTimeout: 8s\nlog:\n level: error\n")
|
||||
if err = r.Reload(); err == nil || r.Snapshot().Log.Level != "debug" {
|
||||
t.Fatal("static reload changed live config", err)
|
||||
}
|
||||
for _, raw := range []string{"unknownOption: true", "log:\n level: bad", "shutdownTimeout: invalid", "address: x\naddress: y", "---\n{}\n---\n{}"} {
|
||||
write(raw)
|
||||
if err = r.Reload(); err == nil {
|
||||
t.Fatal(raw)
|
||||
}
|
||||
}
|
||||
}
|
||||
30
contract/README.md
Normal file
30
contract/README.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# contract
|
||||
|
||||
Generated API projects use this package to compile Go field tags once and bind
|
||||
each request without depending on a specific HTTP server. `New[T]` validates a
|
||||
struct and returns an immutable Binder. `ForType` provides the same compiler to
|
||||
the CLI. `BodySchema`, `Parameters`, and `ResponseSchema` return detached OpenAPI
|
||||
snapshots. `MustNew[T]` is intended for generated, already checked declarations.
|
||||
|
||||
```go
|
||||
type Request struct {
|
||||
ID int64 `path:"id,range=[1:100]"`
|
||||
Name string `json:"name" binding:"required,max=50"`
|
||||
}
|
||||
|
||||
var request = contract.MustNew[Request]()
|
||||
|
||||
func Handle(c *httpx.Context) error {
|
||||
var req Request
|
||||
if err := request.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||
return err
|
||||
}
|
||||
return c.OK(req.Name)
|
||||
}
|
||||
```
|
||||
|
||||
The default body limit is 1 MiB. An optional positive final argument to Bind
|
||||
overrides it. Application transport read timeouts still control slow clients.
|
||||
See [API.md](../API.md) for source tags, validation rules, null/presence behavior,
|
||||
strict decoding and supported type limits. This API leaves the existing
|
||||
`Context.BindJSON/BindQuery/BindURI` behavior unchanged.
|
||||
310
contract/bind.go
Normal file
310
contract/bind.go
Normal file
@@ -0,0 +1,310 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"mime"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"reflect"
|
||||
"strconv"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
)
|
||||
|
||||
// Bind fills a fresh DTO transactionally. A JSON body, if declared, is required,
|
||||
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
|
||||
// parameters never overwrite path/body/header values. form means URL query only.
|
||||
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
|
||||
if b == nil || r == nil {
|
||||
return apierrors.Internal(errors.New("contract: nil binder or request"))
|
||||
}
|
||||
v := reflect.ValueOf(target)
|
||||
if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ {
|
||||
return apierrors.Internal(errors.New("contract: target type mismatch"))
|
||||
}
|
||||
limit := DefaultMaxBodyBytes
|
||||
if len(maxBytes) > 1 || (len(maxBytes) == 1 && (maxBytes[0] <= 0 || maxBytes[0] == math.MaxInt64)) {
|
||||
return apierrors.Internal(errors.New("contract: invalid body limit"))
|
||||
}
|
||||
if len(maxBytes) == 1 {
|
||||
limit = maxBytes[0]
|
||||
}
|
||||
fresh := reflect.New(b.typ).Elem()
|
||||
body := map[string]any{}
|
||||
if len(b.root.schema.Properties) > 0 {
|
||||
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
|
||||
if err != nil || ct != "application/json" {
|
||||
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json")
|
||||
}
|
||||
if r.Body == nil {
|
||||
return invalid(errors.New("body required"))
|
||||
}
|
||||
if r.ContentLength > limit {
|
||||
return tooLarge()
|
||||
}
|
||||
reader := &io.LimitedReader{R: r.Body, N: limit + 1}
|
||||
dec := json.NewDecoder(reader)
|
||||
dec.UseNumber()
|
||||
value, err := decodeValue(dec, 0)
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != nil {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) {
|
||||
return tooLarge()
|
||||
}
|
||||
return invalid(err)
|
||||
}
|
||||
_, err = dec.Token()
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != io.EOF {
|
||||
return invalid(errors.New("exactly one JSON value required"))
|
||||
}
|
||||
var ok bool
|
||||
body, ok = value.(map[string]any)
|
||||
if !ok {
|
||||
return invalid(errors.New("body must be an object"))
|
||||
}
|
||||
for key := range body {
|
||||
if _, ok = b.root.schema.Properties[key]; !ok {
|
||||
return invalid(fmt.Errorf("unknown body field %s", key))
|
||||
}
|
||||
}
|
||||
}
|
||||
query := url.Values{}
|
||||
if r.URL != nil {
|
||||
var err error
|
||||
query, err = url.ParseQuery(r.URL.RawQuery)
|
||||
if err != nil {
|
||||
return invalid(errors.New("malformed query"))
|
||||
}
|
||||
}
|
||||
for _, f := range b.root.fields {
|
||||
var value any
|
||||
var present bool
|
||||
if f.source == "json" {
|
||||
value, present = body[f.name]
|
||||
} else {
|
||||
var raw []string
|
||||
switch f.source {
|
||||
case "path":
|
||||
if s, ok := params[f.name]; ok {
|
||||
raw = []string{s}
|
||||
}
|
||||
case "form":
|
||||
raw = query[f.name]
|
||||
case "header":
|
||||
raw = r.Header.Values(f.name)
|
||||
}
|
||||
present = len(raw) > 0
|
||||
if present {
|
||||
var err error
|
||||
value, err = fromStrings(f.value, raw)
|
||||
if err != nil {
|
||||
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := assignField(f, fresh.Field(f.index), value, present); err != nil {
|
||||
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||
}
|
||||
}
|
||||
v.Elem().Set(fresh)
|
||||
return nil
|
||||
}
|
||||
func invalid(err error) error {
|
||||
return apierrors.BadRequest("INVALID_REQUEST", "request parameters are invalid").WithCause(err)
|
||||
}
|
||||
func tooLarge() error { return apierrors.New(413, "BODY_TOO_LARGE", "request body exceeds limit") }
|
||||
|
||||
func fromStrings(n *node, values []string) (any, error) {
|
||||
n = base(n)
|
||||
if n.typ.Kind() == reflect.Slice {
|
||||
out := make([]any, 0, len(values))
|
||||
for _, s := range values {
|
||||
v, err := parseScalar(s, base(n.elem).typ)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
if len(values) != 1 {
|
||||
return nil, errors.New("duplicate scalar parameter")
|
||||
}
|
||||
return parseScalar(values[0], n.typ)
|
||||
}
|
||||
func assignField(f field, dst reflect.Value, value any, present bool) error {
|
||||
if !present {
|
||||
if f.hasDefault {
|
||||
value = f.fallback
|
||||
} else if f.optional {
|
||||
return nil
|
||||
} else {
|
||||
return errors.New("required field missing")
|
||||
}
|
||||
}
|
||||
if err := setValue(f.value, dst, value); err != nil {
|
||||
return err
|
||||
}
|
||||
return checkField(f, dst)
|
||||
}
|
||||
func setValue(n *node, dst reflect.Value, value any) error {
|
||||
if value == nil {
|
||||
if n.typ.Kind() == reflect.Pointer {
|
||||
dst.SetZero()
|
||||
return nil
|
||||
}
|
||||
return errors.New("null requires a pointer type")
|
||||
}
|
||||
if n.typ.Kind() == reflect.Pointer {
|
||||
dst.Set(reflect.New(n.typ.Elem()))
|
||||
return setValue(n.elem, dst.Elem(), value)
|
||||
}
|
||||
switch n.typ.Kind() {
|
||||
case reflect.Struct:
|
||||
object, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return errors.New("expected object")
|
||||
}
|
||||
for key := range object {
|
||||
if _, ok = n.schema.Properties[key]; !ok {
|
||||
return fmt.Errorf("unknown field %s", key)
|
||||
}
|
||||
}
|
||||
for _, f := range n.fields {
|
||||
item, exists := object[f.name]
|
||||
if err := assignField(f, dst.Field(f.index), item, exists); err != nil {
|
||||
return fmt.Errorf("%s: %w", f.name, err)
|
||||
}
|
||||
}
|
||||
case reflect.Slice:
|
||||
items, ok := value.([]any)
|
||||
if !ok {
|
||||
return errors.New("expected array")
|
||||
}
|
||||
dst.Set(reflect.MakeSlice(n.typ, len(items), len(items)))
|
||||
for i, item := range items {
|
||||
if err := setValue(n.elem, dst.Index(i), item); err != nil {
|
||||
return fmt.Errorf("item %d: %w", i, err)
|
||||
}
|
||||
}
|
||||
case reflect.Map:
|
||||
items, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return errors.New("expected object")
|
||||
}
|
||||
dst.Set(reflect.MakeMapWithSize(n.typ, len(items)))
|
||||
for key, item := range items {
|
||||
v := reflect.New(n.typ.Elem()).Elem()
|
||||
if err := setValue(n.elem, v, item); err != nil {
|
||||
return fmt.Errorf("map value: %w", err)
|
||||
}
|
||||
dst.SetMapIndex(reflect.ValueOf(key).Convert(n.typ.Key()), v)
|
||||
}
|
||||
case reflect.String:
|
||||
s, ok := value.(string)
|
||||
if !ok {
|
||||
return errors.New("expected string")
|
||||
}
|
||||
dst.SetString(s)
|
||||
case reflect.Bool:
|
||||
v, ok := value.(bool)
|
||||
if !ok {
|
||||
return errors.New("expected boolean")
|
||||
}
|
||||
dst.SetBool(v)
|
||||
default:
|
||||
num, ok := value.(json.Number)
|
||||
if !ok {
|
||||
return errors.New("expected number")
|
||||
}
|
||||
switch n.typ.Kind() {
|
||||
case reflect.Float32, reflect.Float64:
|
||||
v, err := strconv.ParseFloat(string(num), n.typ.Bits())
|
||||
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||
return errors.New("number out of range")
|
||||
}
|
||||
dst.SetFloat(v)
|
||||
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
v, err := strconv.ParseUint(string(num), 10, n.typ.Bits())
|
||||
if err != nil {
|
||||
return errors.New("invalid unsigned integer")
|
||||
}
|
||||
dst.SetUint(v)
|
||||
default:
|
||||
v, err := strconv.ParseInt(string(num), 10, n.typ.Bits())
|
||||
if err != nil {
|
||||
return errors.New("invalid integer")
|
||||
}
|
||||
dst.SetInt(v)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Token decoding rejects duplicate keys at every level (encoding/json normally
|
||||
// keeps the last value). Depth and body limits bound client-controlled work.
|
||||
func decodeValue(dec *json.Decoder, depth int) (any, error) {
|
||||
if depth > 64 {
|
||||
return nil, errors.New("JSON nesting exceeds 64")
|
||||
}
|
||||
token, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch token {
|
||||
case json.Delim('{'):
|
||||
object := map[string]any{}
|
||||
for dec.More() {
|
||||
key, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
name, ok := key.(string)
|
||||
if !ok {
|
||||
return nil, errors.New("invalid object key")
|
||||
}
|
||||
if _, exists := object[name]; exists {
|
||||
return nil, errors.New("duplicate JSON field")
|
||||
}
|
||||
value, err := decodeValue(dec, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
object[name] = value
|
||||
}
|
||||
end, err := dec.Token()
|
||||
if err != nil || end != json.Delim('}') {
|
||||
return nil, errors.New("unterminated object")
|
||||
}
|
||||
return object, nil
|
||||
case json.Delim('['):
|
||||
items := []any{}
|
||||
for dec.More() {
|
||||
v, err := decodeValue(dec, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, v)
|
||||
}
|
||||
end, err := dec.Token()
|
||||
if err != nil || end != json.Delim(']') {
|
||||
return nil, errors.New("unterminated array")
|
||||
}
|
||||
return items, nil
|
||||
default:
|
||||
if _, bad := token.(json.Delim); bad {
|
||||
return nil, errors.New("unexpected JSON delimiter")
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
}
|
||||
394
contract/contract.go
Normal file
394
contract/contract.go
Normal file
@@ -0,0 +1,394 @@
|
||||
// Package contract compiles API field tags into a reusable request binder and
|
||||
// OpenAPI schemas. Generated services and the CLI use the same compiler.
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"github.com/go-playground/validator/v10"
|
||||
)
|
||||
|
||||
const DefaultMaxBodyBytes int64 = 1 << 20
|
||||
|
||||
var wireName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||
var headerName = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9-]*$`)
|
||||
var valuesValidator = validator.New()
|
||||
|
||||
// Binder is immutable after compilation and safe for concurrent requests.
|
||||
type Binder struct {
|
||||
root *node
|
||||
typ reflect.Type
|
||||
}
|
||||
type node struct {
|
||||
typ reflect.Type
|
||||
schema *openapi.Schema
|
||||
fields []field
|
||||
elem *node
|
||||
}
|
||||
type field struct {
|
||||
index int
|
||||
name, source string
|
||||
value *node
|
||||
optional bool
|
||||
fallback any
|
||||
hasDefault bool
|
||||
binding string
|
||||
}
|
||||
|
||||
func New[T any]() (*Binder, error) { return ForType(reflect.TypeOf((*T)(nil)).Elem()) }
|
||||
func MustNew[T any]() *Binder {
|
||||
b, err := New[T]()
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// ForType also lets generators check the exact runtime contract before writing files.
|
||||
// The root must be a struct; recursive and custom-marshaling types are rejected.
|
||||
func ForType(t reflect.Type) (*Binder, error) {
|
||||
if t == nil || t.Kind() != reflect.Struct {
|
||||
return nil, errors.New("contract: root must be a struct")
|
||||
}
|
||||
budget := 10000
|
||||
n, err := compile(t, true, map[reflect.Type]bool{}, 0, &budget)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Binder{root: n, typ: t}, nil
|
||||
}
|
||||
|
||||
func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int, budget *int) (*node, error) {
|
||||
*budget--
|
||||
if *budget < 0 {
|
||||
return nil, errors.New("contract: schema expansion exceeds 10000 nodes")
|
||||
}
|
||||
if depth > 32 {
|
||||
return nil, errors.New("contract: type nesting exceeds 32")
|
||||
}
|
||||
if stack[t] {
|
||||
return nil, errors.New("contract: recursive types are not supported")
|
||||
}
|
||||
stack[t] = true
|
||||
defer delete(stack, t)
|
||||
n := &node{typ: t, schema: &openapi.Schema{}}
|
||||
// Wire behavior must be described completely by these tags.
|
||||
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||
unmarshaler := reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
|
||||
if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler) {
|
||||
return nil, errors.New("contract: custom JSON marshalers are unsupported")
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Pointer:
|
||||
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n.elem = e
|
||||
*n.schema = *e.schema
|
||||
n.schema.Nullable = true
|
||||
case reflect.String:
|
||||
n.schema.Type = "string"
|
||||
case reflect.Bool:
|
||||
n.schema.Type = "boolean"
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
n.schema.Type = "integer"
|
||||
if t.Bits() <= 32 {
|
||||
n.schema.Format = "int32"
|
||||
} else {
|
||||
n.schema.Format = "int64"
|
||||
}
|
||||
if t.Kind() >= reflect.Uint && t.Kind() <= reflect.Uint64 {
|
||||
v := float64(0)
|
||||
n.schema.Minimum = &v
|
||||
if t.Bits() < 64 {
|
||||
max := float64((uint64(1) << uint(t.Bits())) - 1)
|
||||
n.schema.Maximum = &max
|
||||
if t.Bits() == 32 {
|
||||
n.schema.Format = "int64"
|
||||
}
|
||||
} else {
|
||||
n.schema.Format = ""
|
||||
}
|
||||
} else if t.Bits() < 32 {
|
||||
min, max := -float64(int64(1)<<uint(t.Bits()-1)), float64((int64(1)<<uint(t.Bits()-1))-1)
|
||||
n.schema.Minimum, n.schema.Maximum = &min, &max
|
||||
}
|
||||
case reflect.Float32, reflect.Float64:
|
||||
n.schema.Type = "number"
|
||||
if t.Kind() == reflect.Float32 {
|
||||
n.schema.Format = "float"
|
||||
} else {
|
||||
n.schema.Format = "double"
|
||||
}
|
||||
case reflect.Slice, reflect.Map:
|
||||
if t.Kind() == reflect.Map && t.Key().Kind() != reflect.String {
|
||||
return nil, errors.New("contract: map keys must be strings")
|
||||
}
|
||||
if t.Kind() == reflect.Slice && t.Elem().Kind() == reflect.Uint8 {
|
||||
return nil, errors.New("contract: byte slices use base64 in Go; use string for binary data")
|
||||
}
|
||||
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n.elem = e
|
||||
if t.Kind() == reflect.Slice {
|
||||
n.schema.Type = "array"
|
||||
n.schema.Items = e.schema
|
||||
} else {
|
||||
n.schema.Type = "object"
|
||||
n.schema.AdditionalProperties = e.schema
|
||||
}
|
||||
case reflect.Struct:
|
||||
n.schema.Type = "object"
|
||||
allowed := false
|
||||
n.schema.AdditionalPropertiesAllowed = &allowed
|
||||
n.schema.Properties = map[string]*openapi.Schema{}
|
||||
seen := map[string]bool{}
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
sf := t.Field(i)
|
||||
if sf.PkgPath != "" || sf.Anonymous {
|
||||
return nil, fmt.Errorf("contract: %s must be an exported named field", sf.Name)
|
||||
}
|
||||
f, err := compileField(sf, root, stack, depth+1, budget)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("contract: %s: %w", sf.Name, err)
|
||||
}
|
||||
f.index = i
|
||||
key := f.source + ":" + strings.ToLower(f.name)
|
||||
if seen[key] {
|
||||
return nil, fmt.Errorf("contract: duplicate field %s", f.name)
|
||||
}
|
||||
seen[key] = true
|
||||
n.fields = append(n.fields, f)
|
||||
if f.source == "json" {
|
||||
n.schema.Properties[f.name] = f.value.schema
|
||||
if !f.optional && !f.hasDefault {
|
||||
n.schema.Required = append(n.schema.Required, f.name)
|
||||
}
|
||||
}
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("contract: unsupported type %s", t)
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool, depth int, budget *int) (field, error) {
|
||||
f := field{}
|
||||
// Reject typos and unsupported tags instead of silently losing validation.
|
||||
tags, err := ParseTags(string(sf.Tag))
|
||||
if err != nil {
|
||||
return f, err
|
||||
}
|
||||
var sourceTag string
|
||||
for key, value := range tags {
|
||||
switch key {
|
||||
case "json", "path", "form", "header":
|
||||
if value == "-" {
|
||||
return f, errors.New("ignored fields are not part of an API contract")
|
||||
}
|
||||
if f.source != "" {
|
||||
return f, errors.New("exactly one source tag is required")
|
||||
}
|
||||
f.source = key
|
||||
sourceTag = value
|
||||
case "binding":
|
||||
f.binding = value
|
||||
default:
|
||||
return f, fmt.Errorf("unsupported tag %q", key)
|
||||
}
|
||||
}
|
||||
if f.source == "" {
|
||||
return f, errors.New("json, path, form or header tag required")
|
||||
}
|
||||
if !root && f.source != "json" {
|
||||
return f, errors.New("nested fields must use json tags")
|
||||
}
|
||||
parts := strings.Split(sourceTag, ",")
|
||||
f.name = parts[0]
|
||||
if (f.source == "header" && !headerName.MatchString(f.name)) || (f.source != "header" && !wireName.MatchString(f.name)) {
|
||||
return f, errors.New("wire names must be lowerCamelCase (headers use HTTP names)")
|
||||
}
|
||||
n, err := compile(sf.Type, false, stack, depth, budget)
|
||||
if err != nil {
|
||||
return f, err
|
||||
}
|
||||
f.value = n
|
||||
scalar := base(n).typ.Kind()
|
||||
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
|
||||
return f, errors.New("path/query/header values must be scalars or scalar slices")
|
||||
}
|
||||
if f.source == "path" && !isScalar(scalar) {
|
||||
return f, errors.New("path values must be scalars")
|
||||
}
|
||||
if f.source == "header" && !isScalar(scalar) {
|
||||
return f, errors.New("header values must be scalars")
|
||||
}
|
||||
options := map[string]bool{}
|
||||
for _, option := range parts[1:] {
|
||||
key, value, _ := strings.Cut(option, "=")
|
||||
if options[key] {
|
||||
return f, fmt.Errorf("duplicate option %q", key)
|
||||
}
|
||||
options[key] = true
|
||||
switch key {
|
||||
case "optional", "omitempty":
|
||||
if option != key {
|
||||
return f, errors.New("optional takes no value")
|
||||
}
|
||||
f.optional = true
|
||||
case "default":
|
||||
if !isScalar(scalar) {
|
||||
return f, errors.New("default requires a scalar")
|
||||
}
|
||||
v, e := parseScalar(value, base(n).typ)
|
||||
if e != nil {
|
||||
return f, fmt.Errorf("invalid default: %w", e)
|
||||
}
|
||||
f.fallback = v
|
||||
f.hasDefault = true
|
||||
n.schema.Default = v
|
||||
case "options":
|
||||
if !isScalar(scalar) || value == "" {
|
||||
return f, errors.New("options requires scalar values")
|
||||
}
|
||||
for _, s := range strings.Split(value, "|") {
|
||||
v, e := parseScalar(s, base(n).typ)
|
||||
if e != nil {
|
||||
return f, e
|
||||
}
|
||||
n.schema.Enum = append(n.schema.Enum, v)
|
||||
}
|
||||
case "range":
|
||||
if !isNumber(scalar) {
|
||||
return f, errors.New("range requires numeric type")
|
||||
}
|
||||
if err = applyRange(n.schema, value); err != nil {
|
||||
return f, err
|
||||
}
|
||||
default:
|
||||
return f, fmt.Errorf("unsupported field option %q", option)
|
||||
}
|
||||
}
|
||||
if err = applyBinding(&f); err != nil {
|
||||
return f, err
|
||||
}
|
||||
if n.schema.Nullable && len(n.schema.Enum) > 0 {
|
||||
n.schema.Enum = append(n.schema.Enum, nil)
|
||||
}
|
||||
if f.source == "path" && (f.optional || f.hasDefault || sf.Type.Kind() == reflect.Pointer) {
|
||||
return f, errors.New("path fields must be required non-pointer values")
|
||||
}
|
||||
if f.hasDefault {
|
||||
v := reflect.New(sf.Type).Elem()
|
||||
if err = setValue(n, v, f.fallback); err != nil {
|
||||
return f, err
|
||||
}
|
||||
if err = checkField(f, v); err != nil {
|
||||
return f, fmt.Errorf("default violates validation: %w", err)
|
||||
}
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// ParseTags strictly parses Go struct tags, including duplicate/unknown key checks
|
||||
// performed by the compiler. It is also used to normalize generated Go JSON tags.
|
||||
func ParseTags(raw string) (map[string]string, error) {
|
||||
out := map[string]string{}
|
||||
for strings.TrimSpace(raw) != "" {
|
||||
raw = strings.TrimLeft(raw, " ")
|
||||
i := strings.Index(raw, ":")
|
||||
if i < 1 {
|
||||
return nil, errors.New("invalid struct tag")
|
||||
}
|
||||
key := raw[:i]
|
||||
raw = raw[i+1:]
|
||||
if strings.ContainsAny(key, " \t\r\n\"`") || len(raw) == 0 || raw[0] != '"' {
|
||||
return nil, errors.New("invalid struct tag")
|
||||
}
|
||||
end := 1
|
||||
for end < len(raw) {
|
||||
if raw[end] == '\\' {
|
||||
end += 2
|
||||
continue
|
||||
}
|
||||
if raw[end] == '"' {
|
||||
break
|
||||
}
|
||||
end++
|
||||
}
|
||||
if end >= len(raw) {
|
||||
return nil, errors.New("unterminated struct tag")
|
||||
}
|
||||
v, err := strconv.Unquote(raw[:end+1])
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := out[key]; ok {
|
||||
return nil, fmt.Errorf("duplicate tag %q", key)
|
||||
}
|
||||
out[key] = v
|
||||
raw = raw[end+1:]
|
||||
if raw != "" && raw[0] != ' ' {
|
||||
return nil, errors.New("struct tags require spaces")
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func base(n *node) *node {
|
||||
for n.typ.Kind() == reflect.Pointer {
|
||||
n = n.elem
|
||||
}
|
||||
return n
|
||||
}
|
||||
func isNumber(k reflect.Kind) bool {
|
||||
return k >= reflect.Int && k <= reflect.Float64 && k != reflect.Uintptr
|
||||
}
|
||||
func isScalar(k reflect.Kind) bool { return k == reflect.String || k == reflect.Bool || isNumber(k) }
|
||||
|
||||
func (b *Binder) BodySchema() *openapi.Schema {
|
||||
if len(b.root.schema.Properties) == 0 {
|
||||
return nil
|
||||
}
|
||||
return copySchema(b.root.schema)
|
||||
}
|
||||
|
||||
// ResponseSchema rejects request-only fields to keep response docs honest.
|
||||
func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
|
||||
for _, f := range b.root.fields {
|
||||
if f.source != "json" {
|
||||
return nil, errors.New("contract: response fields must use json tags")
|
||||
}
|
||||
}
|
||||
return copySchema(b.root.schema), nil
|
||||
}
|
||||
func (b *Binder) Parameters() []openapi.Parameter {
|
||||
var out []openapi.Parameter
|
||||
for _, f := range b.root.fields {
|
||||
if f.source == "json" {
|
||||
continue
|
||||
}
|
||||
in := f.source
|
||||
if in == "form" {
|
||||
in = "query"
|
||||
}
|
||||
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: copySchema(f.value.schema)})
|
||||
}
|
||||
return out
|
||||
}
|
||||
func copySchema(s *openapi.Schema) *openapi.Schema {
|
||||
data, _ := json.Marshal(s)
|
||||
var out openapi.Schema
|
||||
_ = json.Unmarshal(data, &out)
|
||||
return &out
|
||||
}
|
||||
244
contract/contract_test.go
Normal file
244
contract/contract_test.go
Normal file
@@ -0,0 +1,244 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http/httptest"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
)
|
||||
|
||||
type address struct {
|
||||
City string `json:"city" binding:"required,max=30"`
|
||||
}
|
||||
type mixed struct {
|
||||
ID int64 `path:"userId,range=[1:100]"`
|
||||
Page int `form:"page,default=1,range=[1:50]"`
|
||||
Tags []string `form:"tag,optional" binding:"max=3"`
|
||||
Client string `header:"X-Client"`
|
||||
Name string `json:"name" binding:"required,max=20"`
|
||||
Active bool `json:"active"`
|
||||
Count *int `json:"count,optional"`
|
||||
Contact address `json:"contact"`
|
||||
Items []address `json:"items,optional"`
|
||||
Labels map[string]address `json:"labels,optional"`
|
||||
}
|
||||
|
||||
const validBody = `{"name":"小明","active":false,"count":0,"contact":{"city":"杭州"}}`
|
||||
|
||||
func bindMixed(body, query string) (mixed, error) {
|
||||
b := MustNew[mixed]()
|
||||
r := httptest.NewRequest("POST", "/users/7"+query, strings.NewReader(body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("X-Client", "web")
|
||||
var v mixed
|
||||
return v, b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||
}
|
||||
func TestMixedSourcesAndPresence(t *testing.T) {
|
||||
b := MustNew[mixed]()
|
||||
r := httptest.NewRequest("POST", "/users/7?tag=a&tag=b&userId=99&name=query", strings.NewReader(validBody))
|
||||
r.Header.Set("Content-Type", "application/json; charset=utf-8")
|
||||
r.Header.Set("X-Client", "web")
|
||||
v := mixed{Name: "previous"}
|
||||
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if v.ID != 7 || v.Page != 1 || v.Name != "小明" || v.Active || v.Count == nil || *v.Count != 0 || len(v.Tags) != 2 || v.Client != "web" || v.Contact.City != "杭州" {
|
||||
t.Fatalf("bad binding: %+v", v)
|
||||
}
|
||||
r = httptest.NewRequest("POST", "/", strings.NewReader(strings.Replace(validBody, `,"count":0`, "", 1)))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("X-Client", "web")
|
||||
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil || v.Count != nil {
|
||||
t.Fatalf("missing != zero: %+v %v", v, err)
|
||||
}
|
||||
}
|
||||
func TestRejectInvalidRequests(t *testing.T) {
|
||||
cases := []struct{ name, body, query string }{
|
||||
{"missing", `{"name":"a","contact":{"city":"杭州"}}`, ""},
|
||||
{"empty", strings.Replace(validBody, `"小明"`, `""`, 1), ""},
|
||||
{"caseSensitive", strings.Replace(validBody, `"name"`, `"Name"`, 1), ""},
|
||||
{"unknown", strings.Replace(validBody, `"active":false`, `"active":false,"extra":1`, 1), ""},
|
||||
{"duplicate", strings.Replace(validBody, `"active":false`, `"active":false,"active":true`, 1), ""},
|
||||
{"nestedDuplicate", strings.Replace(validBody, `"city":"杭州"`, `"city":"杭州","city":"北京"`, 1), ""},
|
||||
{"nestedRequired", strings.Replace(validBody, `{"city":"杭州"}`, `{}`, 1), ""},
|
||||
{"nestedUnknown", strings.Replace(validBody, `{"city":"杭州"}`, `{"city":"杭州","other":0}`, 1), ""},
|
||||
{"trailing", validBody + ` {}`, ""},
|
||||
{"null", strings.Replace(validBody, `false`, `null`, 1), ""},
|
||||
{"numberOverflow", strings.Replace(validBody, `"count":0`, `"count":99999999999999999999`, 1), ""},
|
||||
{"arrayValidation", strings.TrimSuffix(validBody, "}") + `,"items":[{"city":""}]}`, ""},
|
||||
{"mapValidation", strings.TrimSuffix(validBody, "}") + `,"labels":{"home":{"city":""}}}`, ""},
|
||||
{"queryDuplicate", validBody, "?page=1&page=2"},
|
||||
{"queryDefaultNotOnEmpty", validBody, "?page="},
|
||||
{"range", validBody, "?page=51"},
|
||||
{"malformedQuery", validBody, "?page=%zz"},
|
||||
{"arrayLimit", validBody, "?tag=a&tag=b&tag=c&tag=d"},
|
||||
}
|
||||
b := MustNew[mixed]()
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
r := httptest.NewRequest("POST", "/"+tc.query, strings.NewReader(tc.body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("X-Client", "web")
|
||||
v := mixed{Name: "unchanged"}
|
||||
err := b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||
if err == nil || apierrors.From(err).Status != 400 {
|
||||
t.Fatalf("wanted 400: %v", err)
|
||||
}
|
||||
if v.Name != "unchanged" || v.Page != 0 {
|
||||
t.Fatal("failed binding modified target")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestBodyLimitsAndMediaType(t *testing.T) {
|
||||
type request struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
b := MustNew[request]()
|
||||
for _, tc := range []struct {
|
||||
body, ct string
|
||||
limit int64
|
||||
status int
|
||||
}{
|
||||
{`{"name":"x"}`, "text/plain", 100, 415},
|
||||
{`{"name":"long"}`, "application/json", 10, 413},
|
||||
{`{"name":"x"}` + strings.Repeat(" ", 2048), "application/json", 1024, 413},
|
||||
{`{"name":"x"}`, "application/json", 12, 0},
|
||||
} {
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||
r.ContentLength = -1
|
||||
r.Header.Set("Content-Type", tc.ct)
|
||||
var req request
|
||||
err := b.Bind(r, nil, &req, tc.limit)
|
||||
if tc.status == 0 {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
} else if err == nil || apierrors.From(err).Status != tc.status {
|
||||
t.Fatalf("status %d wanted: %v", tc.status, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestSchemaUsesSameRules(t *testing.T) {
|
||||
b := MustNew[mixed]()
|
||||
s := b.BodySchema()
|
||||
if len(s.Required) != 3 || s.Properties["count"].Nullable != true || s.Properties["name"].MaxLength == nil || *s.Properties["name"].MaxLength != 20 {
|
||||
t.Fatalf("wrong schema: %+v", s)
|
||||
}
|
||||
encoded, err := json.Marshal(s)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(encoded), `"additionalProperties":false`) {
|
||||
t.Fatal(string(encoded))
|
||||
}
|
||||
params := b.Parameters()
|
||||
if len(params) != 4 || params[1].Required || params[1].Schema.Default != json.Number("1") {
|
||||
t.Fatalf("wrong parameters: %+v", params)
|
||||
}
|
||||
// Returned documentation snapshots do not mutate concurrent binders.
|
||||
s.Properties["name"].MaxLength = nil
|
||||
if b.BodySchema().Properties["name"].MaxLength == nil {
|
||||
t.Fatal("shared schema")
|
||||
}
|
||||
if _, err = b.ResponseSchema(); err == nil {
|
||||
t.Fatal("request sources in response accepted")
|
||||
}
|
||||
}
|
||||
func TestRulesAndConcurrentReuse(t *testing.T) {
|
||||
type req struct {
|
||||
Score float64 `json:"score,options=1.0|2,range=(0:2]"`
|
||||
Email string `json:"email" binding:"email"`
|
||||
Value *int `json:"value" binding:"required"`
|
||||
}
|
||||
b := MustNew[req]()
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 24; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"score":1,"email":"a@example.com","value":3}`))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
var v req
|
||||
if err := b.Bind(r, nil, &v); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
for _, body := range []string{`{"score":0,"email":"a@example.com","value":1}`, `{"score":2,"email":"bad","value":1}`, `{"score":2,"email":"a@example.com","value":null}`, `{"score":2,"email":"a@example.com","value":0}`} {
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
var v req
|
||||
if err := b.Bind(r, nil, &v); err == nil {
|
||||
t.Fatal(body)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestInvalidContracts(t *testing.T) {
|
||||
for _, tag := range []string{`json:"bad_name"`, `json:"a" form:"a"`, `json:"a,optionl"`, `json:"a,range=[5:2]"`, `json:"a,default=20,range=[1:10]"`, `json:"a,optional" binding:"required"`, `json:"a" binding:"unknown"`, `path:"a,optional"`, `json:"a" json:"b"`, `jsoon:"a"`, `json:"a,options=x"`} {
|
||||
typ := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: reflect.TypeOf(0), Tag: reflect.StructTag(tag)}})
|
||||
if _, err := ForType(typ); err == nil {
|
||||
t.Fatalf("accepted %s", tag)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExactLargeIntegerConstraints(t *testing.T) {
|
||||
type req struct {
|
||||
Value int64 `json:"value,range=[:9007199254740992]"`
|
||||
}
|
||||
b := MustNew[req]()
|
||||
for _, tc := range []struct {
|
||||
body string
|
||||
valid bool
|
||||
}{{`{"value":9007199254740992}`, true}, {`{"value":9007199254740993}`, false}} {
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
var v req
|
||||
if err := b.Bind(r, nil, &v); (err == nil) != tc.valid {
|
||||
t.Fatalf("%s: %v", tc.body, err)
|
||||
}
|
||||
}
|
||||
type defaults struct {
|
||||
Value uint64 `form:"value,default=18446744073709551615,options=18446744073709551615"`
|
||||
}
|
||||
d := MustNew[defaults]()
|
||||
data, err := json.Marshal(d.Parameters())
|
||||
if err != nil || !strings.Contains(string(data), "18446744073709551615") {
|
||||
t.Fatalf("lost precision: %s %v", data, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntrinsicIntegerBounds(t *testing.T) {
|
||||
type req struct {
|
||||
Small uint8 `json:"small,range=[-1:500]"`
|
||||
Wide uint64 `json:"wide"`
|
||||
}
|
||||
b := MustNew[req]()
|
||||
s := b.BodySchema()
|
||||
if *s.Properties["small"].Minimum != 0 || *s.Properties["small"].Maximum != 255 || s.Properties["wide"].Format != "" {
|
||||
t.Fatalf("wrong integer bounds: %+v", s.Properties)
|
||||
}
|
||||
}
|
||||
|
||||
func FuzzStrictBody(f *testing.F) {
|
||||
for _, seed := range []string{validBody, `{}`, `{"name":"a","name":"b"}`, `[null]`, `{"x":[]}`} {
|
||||
f.Add(seed)
|
||||
}
|
||||
binder := MustNew[mixed]()
|
||||
f.Fuzz(func(t *testing.T, body string) {
|
||||
if len(body) > 16384 {
|
||||
t.Skip()
|
||||
}
|
||||
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("X-Client", "fuzz")
|
||||
var value mixed
|
||||
_ = binder.Bind(req, map[string]string{"userId": "1"}, &value, 16384)
|
||||
})
|
||||
}
|
||||
259
contract/rules.go
Normal file
259
contract/rules.go
Normal file
@@ -0,0 +1,259 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
)
|
||||
|
||||
func applyRange(s *openapi.Schema, value string) error {
|
||||
if len(value) < 3 || (value[0] != '[' && value[0] != '(') || (value[len(value)-1] != ']' && value[len(value)-1] != ')') {
|
||||
return errors.New("invalid range: use [min:max], (min:max), or mixed bounds")
|
||||
}
|
||||
lo, hi, ok := strings.Cut(value[1:len(value)-1], ":")
|
||||
if !ok || (lo == "" && hi == "") {
|
||||
return errors.New("invalid numeric range")
|
||||
}
|
||||
if lo != "" {
|
||||
v, err := bound(lo)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if s.Minimum == nil || v > *s.Minimum || (v == *s.Minimum && value[0] == '(') {
|
||||
s.Minimum = &v
|
||||
s.ExclusiveMinimum = value[0] == '('
|
||||
}
|
||||
}
|
||||
if hi != "" {
|
||||
v, err := bound(hi)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if s.Maximum == nil || v < *s.Maximum || (v == *s.Maximum && value[len(value)-1] == ')') {
|
||||
s.Maximum = &v
|
||||
s.ExclusiveMaximum = value[len(value)-1] == ')'
|
||||
}
|
||||
}
|
||||
return checkBounds(s)
|
||||
}
|
||||
func bound(s string) (float64, error) {
|
||||
v, err := strconv.ParseFloat(s, 64)
|
||||
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) || math.Abs(v) > 1<<53 {
|
||||
return 0, errors.New("constraint bound must be finite and within ±2^53")
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
func checkBounds(s *openapi.Schema) error {
|
||||
if s.Minimum != nil && s.Maximum != nil && (*s.Minimum > *s.Maximum || (*s.Minimum == *s.Maximum && (s.ExclusiveMinimum || s.ExclusiveMaximum))) {
|
||||
return errors.New("empty numeric range")
|
||||
}
|
||||
if s.MinLength != nil && s.MaxLength != nil && *s.MinLength > *s.MaxLength {
|
||||
return errors.New("empty string length range")
|
||||
}
|
||||
if s.MinItems != nil && s.MaxItems != nil && *s.MinItems > *s.MaxItems {
|
||||
return errors.New("empty array length range")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func applyBinding(f *field) error {
|
||||
if f.binding == "" {
|
||||
return nil
|
||||
}
|
||||
n := base(f.value)
|
||||
k := n.typ.Kind()
|
||||
s := f.value.schema
|
||||
seen := map[string]bool{}
|
||||
for _, rule := range strings.Split(f.binding, ",") {
|
||||
name, value, _ := strings.Cut(rule, "=")
|
||||
if seen[name] {
|
||||
return fmt.Errorf("duplicate binding rule %s", name)
|
||||
}
|
||||
seen[name] = true
|
||||
switch name {
|
||||
case "required":
|
||||
if rule != name || f.optional || f.hasDefault {
|
||||
return errors.New("required conflicts with optional/default")
|
||||
}
|
||||
if !isScalar(k) {
|
||||
return errors.New("binding required supports scalars; other fields are already required by presence")
|
||||
}
|
||||
s.Nullable = false
|
||||
switch k {
|
||||
case reflect.String:
|
||||
one := 1
|
||||
s.MinLength = &one
|
||||
case reflect.Bool:
|
||||
s.Not = &openapi.Schema{Enum: []any{false}}
|
||||
default:
|
||||
s.Not = &openapi.Schema{Enum: []any{0}}
|
||||
}
|
||||
case "email":
|
||||
if rule != name || k != reflect.String {
|
||||
return errors.New("email requires a string")
|
||||
}
|
||||
s.Format = "email"
|
||||
case "min", "max", "len":
|
||||
if k == reflect.String || k == reflect.Slice {
|
||||
v, err := strconv.Atoi(value)
|
||||
if err != nil || v < 0 {
|
||||
return errors.New("length constraints must be nonnegative integers")
|
||||
}
|
||||
if k == reflect.String {
|
||||
if name == "min" || name == "len" {
|
||||
if s.MinLength == nil || v > *s.MinLength {
|
||||
s.MinLength = &v
|
||||
}
|
||||
}
|
||||
if name == "max" || name == "len" {
|
||||
s.MaxLength = &v
|
||||
}
|
||||
} else {
|
||||
if name == "min" || name == "len" {
|
||||
s.MinItems = &v
|
||||
}
|
||||
if name == "max" || name == "len" {
|
||||
s.MaxItems = &v
|
||||
}
|
||||
}
|
||||
} else if isNumber(k) && name != "len" {
|
||||
v, err := bound(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if name == "min" {
|
||||
if s.Minimum == nil || v > *s.Minimum {
|
||||
s.Minimum = &v
|
||||
s.ExclusiveMinimum = false
|
||||
}
|
||||
} else {
|
||||
if s.Maximum == nil || v < *s.Maximum {
|
||||
s.Maximum = &v
|
||||
s.ExclusiveMaximum = false
|
||||
}
|
||||
}
|
||||
} else {
|
||||
return fmt.Errorf("%s requires a number, string or slice", name)
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("unsupported binding rule %q; use required,min,max,len,email or source-tag options/range", name)
|
||||
}
|
||||
}
|
||||
return checkBounds(s)
|
||||
}
|
||||
|
||||
func parseScalar(raw string, t reflect.Type) (any, error) {
|
||||
switch t.Kind() {
|
||||
case reflect.String:
|
||||
return raw, nil
|
||||
case reflect.Bool:
|
||||
if raw == "true" {
|
||||
return true, nil
|
||||
}
|
||||
if raw == "false" {
|
||||
return false, nil
|
||||
}
|
||||
return nil, errors.New("expected true or false")
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
|
||||
v, err := strconv.ParseInt(raw, 10, t.Bits())
|
||||
if err != nil {
|
||||
return nil, errors.New("invalid integer")
|
||||
}
|
||||
return json.Number(strconv.FormatInt(v, 10)), nil
|
||||
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
v, err := strconv.ParseUint(raw, 10, t.Bits())
|
||||
if err != nil {
|
||||
return nil, errors.New("invalid unsigned integer")
|
||||
}
|
||||
return json.Number(strconv.FormatUint(v, 10)), nil
|
||||
case reflect.Float32, reflect.Float64:
|
||||
v, err := strconv.ParseFloat(raw, t.Bits())
|
||||
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||
return nil, errors.New("invalid finite number")
|
||||
}
|
||||
return json.Number(strconv.FormatFloat(v, 'g', -1, t.Bits())), nil
|
||||
}
|
||||
return nil, errors.New("expected scalar")
|
||||
}
|
||||
|
||||
func checkField(f field, v reflect.Value) error {
|
||||
if v.Kind() == reflect.Pointer {
|
||||
if v.IsNil() {
|
||||
if strings.Contains(","+f.binding+",", ",required,") {
|
||||
return errors.New("must not be null")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
v = v.Elem()
|
||||
}
|
||||
if strings.Contains(","+f.binding+",", ",required,") && v.IsZero() {
|
||||
return errors.New("must not be zero or empty")
|
||||
}
|
||||
s := f.value.schema
|
||||
if s.Format == "email" && valuesValidator.Var(v.Interface(), "email") != nil {
|
||||
return errors.New("invalid email")
|
||||
}
|
||||
if len(s.Enum) > 0 {
|
||||
found := false
|
||||
for _, item := range s.Enum {
|
||||
option := reflect.New(v.Type()).Elem()
|
||||
if setValue(base(f.value), option, item) == nil && reflect.DeepEqual(option.Interface(), v.Interface()) {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return errors.New("not an allowed option")
|
||||
}
|
||||
}
|
||||
if isNumber(v.Kind()) {
|
||||
var number float64
|
||||
switch v.Kind() {
|
||||
case reflect.Float32, reflect.Float64:
|
||||
number = v.Float()
|
||||
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
number = float64(v.Uint())
|
||||
if v.Uint() > 1<<53 {
|
||||
number = math.Inf(1)
|
||||
}
|
||||
default:
|
||||
number = float64(v.Int())
|
||||
if v.Int() > 1<<53 {
|
||||
number = math.Inf(1)
|
||||
} else if v.Int() < -(1 << 53) {
|
||||
number = math.Inf(-1)
|
||||
}
|
||||
}
|
||||
if s.Minimum != nil && (number < *s.Minimum || (s.ExclusiveMinimum && number == *s.Minimum)) {
|
||||
return errors.New("below minimum")
|
||||
}
|
||||
if s.Maximum != nil && (number > *s.Maximum || (s.ExclusiveMaximum && number == *s.Maximum)) {
|
||||
return errors.New("above maximum")
|
||||
}
|
||||
}
|
||||
if v.Kind() == reflect.String {
|
||||
size := utf8.RuneCountInString(v.String())
|
||||
if s.MinLength != nil && size < *s.MinLength {
|
||||
return errors.New("string too short")
|
||||
}
|
||||
if s.MaxLength != nil && size > *s.MaxLength {
|
||||
return errors.New("string too long")
|
||||
}
|
||||
}
|
||||
if v.Kind() == reflect.Slice {
|
||||
size := v.Len()
|
||||
if s.MinItems != nil && size < *s.MinItems {
|
||||
return errors.New("too few items")
|
||||
}
|
||||
if s.MaxItems != nil && size > *s.MaxItems {
|
||||
return errors.New("too many items")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
- `Config` / `ConfigFromEnv`:连接池参数和 `GO_OMNI_DB_*` 环境变量。
|
||||
- `Open`:创建连接池、设置连接上限并在返回前执行 `PingContext`。
|
||||
- `Health`:用于就绪检查。
|
||||
- `Health`:用于就绪检查,等待连接和 Ping 均受 `PingTimeout` 限制。
|
||||
- `InTx`:统一事务提交和回滚;工作函数返回错误或 panic 时回滚。
|
||||
- `Close`:由嵌入的 `*sql.DB` 提供,应用退出时显式调用。
|
||||
|
||||
@@ -18,7 +18,13 @@ import (
|
||||
)
|
||||
|
||||
cfg, err := database.ConfigFromEnv("GO_OMNI_DB_")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
db, err := database.Open(context.Background(), cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer db.Close()
|
||||
```
|
||||
|
||||
@@ -35,3 +41,26 @@ GO_OMNI_DB_PING_TIMEOUT=5s
|
||||
```
|
||||
|
||||
`DSN` 必填,驱动默认是 `mysql`。数据库建库、迁移、表结构和具体 Repository 属于业务项目或可选模块,不会被核心包自动执行。
|
||||
|
||||
`MaxIdleConns=0` 会禁用空闲连接。初始化 Ping 失败会关闭已创建的连接池。
|
||||
|
||||
`go-omni new --database mysql` 会生成项目自己的 MySQL Repository、ServiceContext、
|
||||
数据库就绪探针和 `schema.sql`。核心包不需要知道业务表结构。
|
||||
|
||||
真实数据库验收(需要本地 Docker):
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyDatabase.py --version v0.2.0
|
||||
```
|
||||
|
||||
脚本仅操作本次创建的独立容器与数据库,验证 HTTP CRUD、服务重启持久化、事务提交与
|
||||
错误/panic/取消回滚、连接池关闭、缺少表结构时启动失败,以及停库后的 readiness 503 和恢复后的 200。
|
||||
结束时清理容器、匿名数据卷和临时项目。常规 `go test ./...` 不依赖 Docker。
|
||||
|
||||
|
||||
## 项目配置叠加
|
||||
|
||||
`database.ApplyEnv(cfg, prefix)` 在已有文件/默认配置上覆盖非空环境变量,保留未覆盖的池参数。
|
||||
`Config` 的 JSON/YAML 字段使用 lowerCamelCase,`DSN` 使用 `json:"-"`,连接密钥从环境变量加载。
|
||||
API 脚手架 `--database mysql` 使用此流程,并要求正的 `maxOpenConns`,提供数据库 readiness 与幂等清理。
|
||||
具体表结构、迁移和 Repository 留在业务项目,参考 `examples/userService`。
|
||||
|
||||
35
database/configOverlay_test.go
Normal file
35
database/configOverlay_test.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package database
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestApplyEnvPreservesFilePoolAndHidesDSN(t *testing.T) {
|
||||
cfg := DefaultConfig()
|
||||
cfg.MaxOpenConns = 3
|
||||
cfg.MaxIdleConns = 1
|
||||
cfg.PingTimeout = time.Second
|
||||
t.Setenv("OMNI_TEST_DSN", "user:secret@tcp(localhost:3306)/test")
|
||||
t.Setenv("OMNI_TEST_MAX_IDLE_CONNS", "0")
|
||||
got, err := ApplyEnv(cfg, "OMNI_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.MaxOpenConns != 3 || got.MaxIdleConns != 0 || got.PingTimeout != time.Second {
|
||||
t.Fatalf("lost file defaults: %+v", got)
|
||||
}
|
||||
data, err := json.Marshal(got)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(data), "secret") || !strings.Contains(string(data), "maxOpenConns") {
|
||||
t.Fatalf("unsafe config encoding: %s", data)
|
||||
}
|
||||
t.Setenv("OMNI_TEST_MAX_OPEN_CONNS", "bad")
|
||||
if _, err = ApplyEnv(cfg, "OMNI_TEST_"); err == nil {
|
||||
t.Fatal("invalid override accepted")
|
||||
}
|
||||
}
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user