Compare commits
4 Commits
f91034477b
...
v0.7.0-pre
| Author | SHA1 | Date | |
|---|---|---|---|
| 0ce3efb6fc | |||
| 2c74dc0859 | |||
| 04a1f7999d | |||
| 208a9c3fbd |
151
ADVANCED.md
Normal file
151
ADVANCED.md
Normal file
@@ -0,0 +1,151 @@
|
|||||||
|
# 扩展业务模块
|
||||||
|
|
||||||
|
本批提供八类扩展,以及分页、审计和显式数据库迁移。模块按需导入;`omni.New` 与业务 Handler 不绑定存储、队列或网络实现。JSON、配置和新示例使用 lowerCamelCase;Go 导出标识符和 `_test.go` 保持语言约定。
|
||||||
|
|
||||||
|
按本次范围决定,**不集成 Orbit,不引入 Orbit SDK,也不实现工作流编排**。通用消息队列与 Orbit 的持久流程、执行授权、结果订阅和人工恢复不是等价能力。业务项目以后需要 Orbit,可独立依赖它的 SDK;当前没有承诺只换队列适配器就能迁移其全部语义。
|
||||||
|
|
||||||
|
## 能力与边界
|
||||||
|
|
||||||
|
| 模块 | 本批实现 | 明确边界 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
|
||||||
|
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
|
||||||
|
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
|
||||||
|
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别、轮转/脱敏/堆栈/慢请求 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
|
||||||
|
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
|
||||||
|
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
|
||||||
|
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
|
||||||
|
| `storage` | Local、S3/R2、OSS、临时下载、受限直传、云端分片 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
|
||||||
|
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
|
||||||
|
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
|
||||||
|
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
|
||||||
|
|
||||||
|
日志的配置、脱敏边界、轮转和慢请求记录见 [logx 使用说明](logx/README.md)。框架保留通用 Context/结构化字段扩展,不新增固定业务身份上下文。
|
||||||
|
|
||||||
|
## 对象存储
|
||||||
|
|
||||||
|
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
|
||||||
|
|
||||||
|
```go
|
||||||
|
type FileService struct { Files storage.Store }
|
||||||
|
|
||||||
|
// Local:私有目录,流式临时文件 + 原子替换。
|
||||||
|
files, err := local.New("/srv/app/objects", 32<<20)
|
||||||
|
|
||||||
|
// S3:AWS/MinIO 或其他经过契约测试的 S3 兼容服务。
|
||||||
|
files, err := s3.Open(s3.Config{
|
||||||
|
Endpoint: "s3.us-east-1.amazonaws.com", Region: "us-east-1",
|
||||||
|
Bucket: "example-bucket", Secure: true, MaxBytes: 32<<20,
|
||||||
|
Credentials: credentialProvider, Transport: transport,
|
||||||
|
})
|
||||||
|
|
||||||
|
// Cloudflare R2:复用 S3 适配器,签名区域为 auto。
|
||||||
|
cfg, err := s3.R2Config(accountID, bucket, credentialProvider)
|
||||||
|
cfg.Transport = transport
|
||||||
|
files, err := s3.Open(cfg)
|
||||||
|
|
||||||
|
// OSS:单独适配官方 Go SDK v2,保留平台签名与凭据提供器。
|
||||||
|
client := ossSdk.NewClient(ossSdk.LoadDefaultConfig().
|
||||||
|
WithRegion("cn-hangzhou").
|
||||||
|
WithCredentialsProvider(ossCredentials.NewEnvironmentVariableCredentialsProvider()))
|
||||||
|
files, err := oss.New(client, bucket, 32<<20)
|
||||||
|
```
|
||||||
|
|
||||||
|
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
|
||||||
|
|
||||||
|
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,并检查实际读取长度。默认单对象 32 MiB,流式读写;OSS 简单 Put 上限 5 GiB,更大使用独立分片能力。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
|
||||||
|
|
||||||
|
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。受限直传和云端分片现已提供可选接口,见 [上传与续传说明](storage/README.md)。生命周期、ACL、CDN 自定义域名和版本管理仍由平台/部署管理。
|
||||||
|
|
||||||
|
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。
|
||||||
|
|
||||||
|
验证区分:Local 契约、OSS SDK HTTP fixture、S3 SDK fixture 和 R2 签名参数属于本地测试;Linux runner 再跑真实 MinIO 上传/读取/删除与签名拒绝。**MinIO 通过不能代替 OSS/R2 实网验证。**迁移存储平台还需要复制已有对象和调整部署权限,不只是切配置。
|
||||||
|
|
||||||
|
## 可靠队列
|
||||||
|
|
||||||
|
```go
|
||||||
|
backend, err := queueRedis.New(redisClient, queueRedis.Config{
|
||||||
|
Namespace: "serviceA:jobs:v1", MaxRecords: 1000,
|
||||||
|
MaxPayloadBytes: 64 << 10, DedupTTL: 24*time.Hour,
|
||||||
|
})
|
||||||
|
consumer, err := queueRedis.NewConsumer(backend, queueRedis.ConsumerConfig{
|
||||||
|
Workers: 4, MaxAttempts: 3,
|
||||||
|
TaskTimeout: 30*time.Second, VisibilityTimeout: time.Minute,
|
||||||
|
})
|
||||||
|
|
||||||
|
// ID 与内容在业务端持久化;超时后复用原 ID 和原内容。
|
||||||
|
receipt, err := backend.Publish(ctx, queue.Message{
|
||||||
|
ID: operationID, Name: "generateReport", Payload: payload,
|
||||||
|
})
|
||||||
|
|
||||||
|
// 在服务生命周期中运行;返回错误须让监控/进程管理感知。
|
||||||
|
err = consumer.Run(serviceContext, func(ctx context.Context, item queue.Delivery) error {
|
||||||
|
return generateReportIdempotently(ctx, item.ID, item.Payload)
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
业务依赖 `queue.Publisher` / `queue.Consumer`,Redis 类型留在组合处。receipt 表示受理,不代表业务成功。普通 error/panic 按配置重试,`queue.Reject(code)` 保留为失败消息;超过次数同样保留。`DeadLetters(after, limit)` 是有界分页,`DiscardDead` 仅在人工核对后删除匹配的失败消息,并继续保留一段提交去重窗口。错误正文不写入失败记录。需要再次处理时由业务决定新操作身份及去重策略,框架不自动无限重投。
|
||||||
|
|
||||||
|
一个 namespace 使用一个受管 consumer group;多个 consumer 可竞争消费同一队列。配置一致性检查拒绝不同的容量/去重策略和执行超时/重试策略混用。确认检查当前 pending owner,旧执行不能确认新执行;但不能阻止旧处理器已发出的外部副作用,因此仍必须业务幂等。处理器需响应 context;Go 无法强杀不配合的函数,超过 shutdownTimeout 返回错误时,最多 Workers 个处理器可能尚未退出。
|
||||||
|
|
||||||
|
容量包含待处理、失败及未过期的已完成身份,满额拒绝新 ID,保留原 ID 重报回执。完成/人工丢弃后的身份按 DedupTTL 过期;未完成/失败消息不自动清除,不用 MAXLEN 静默裁掉未消费消息。Redis 需启用合适的 AOF/复制策略与 noeviction;本批验证的是投递/接管协议,未证明 Redis 主从故障切换后零丢失。异常进程留下的无 pending consumer 元数据可在运维核实后清理,不能用 DELCONSUMER 丢掉尚未接管的 pending。
|
||||||
|
|
||||||
|
`task.Runner` 仍适用于本进程短任务和定时轮询,重启会丢失;需要持久交付时使用 `queue`。这里没有分布式 Cron、业务工作流、事务 Outbox 或 exactly-once 承诺。
|
||||||
|
|
||||||
|
## 认证与授权
|
||||||
|
|
||||||
|
`auth.NewJWT` 要求固定 issuer/audience 和可信 kid→公钥列表;只开放 RS256 / EdDSA,必须有 exp/sub,检查有效期、nbf 与 iat。`ReplaceKeys` 原子替换列表,轮换期可同时保留新旧 kid。JWT 签发、私钥托管、登录口令验证留给身份服务。
|
||||||
|
|
||||||
|
`auth.NewSessions(cacheStore, "sessions:", time.Hour)` 签发随机 opaque bearer token;存储仅使用 token 的 SHA-256 键,可 `Revoke`。Redis 缓存可共享 Session,内存缓存只在单实例生效且可能因容量淘汰提前登出。不要把 Session token 当普通缓存值写日志。
|
||||||
|
|
||||||
|
`RequireAuth(authenticator)` 在前,`RequirePermission(policy, action, resource)` 在后。Policy 接收已验证 Principal,业务自行校验租户、角色、资源归属。示例 `examples/platform` 验证了无身份 401、角色不足 403 和授权写入。
|
||||||
|
|
||||||
|
## 配置与日志热更新
|
||||||
|
|
||||||
|
`config.LoadFile(path, prefix)` 按默认值→文件→非空环境变量加载并验证。两种文件格式都使用 JSON tag 的 lowerCamelCase,duration 可以写 `"5s"`;文件上限 1 MiB,拒绝未知字段、多文档和非法值。自定义 prefix 要包含期望的尾部下划线,例如 `BILLING_`。空环境变量沿用文件值,不清空已配置字段。
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
address: '127.0.0.1:8080'
|
||||||
|
transport: native
|
||||||
|
shutdownTimeout: 10s
|
||||||
|
log:
|
||||||
|
level: info
|
||||||
|
format: json
|
||||||
|
```
|
||||||
|
|
||||||
|
先按配置创建 `logx.New(cfg.Log)`,再 `config.NewReloader(path, prefix, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`;后者阻塞,生命周期由应用负责。只有 `log.level` 允许变化,校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
|
||||||
|
|
||||||
|
## 熔断与观测
|
||||||
|
|
||||||
|
为每个上游创建 `resilience.New(Config{MaxConcurrent: 16, FailureThreshold: 5})`;通用调用用 `Do`,HTTP 用 `resilience.Transport{Base: transport, Breaker: breaker}`。HTTP 槽位覆盖响应体读取,必须关闭 body;网络错误/5xx 计失败,冷却后只放行一个探针。不要对效果未知的写操作自行换身份重试。
|
||||||
|
|
||||||
|
`telemetry.New` 接受 OTel `sdktrace.SpanExporter` 和 `sdkmetric.Reader`;可注入官方 OTLP exporter、PeriodicReader 或测试用 exporter,基础包不强制选择 collector 协议。`app.Use(runtime.Middleware())` 采集入站,`telemetry.Transport` 采集出站并传播 W3C trace context;Redis 使用 `client.AddHook(telemetry.RedisHook{Runtime: runtime})`,SQL 使用 `telemetry.SQL{DB: db, Runtime: runtime}`。业务操作/事务可用 `runtime.Do(ctx, "fixedOperationName", fn)` 包裹。
|
||||||
|
|
||||||
|
创建者负责在停止请求/Worker 后调用 `runtime.Shutdown(ctx)` 刷新并关闭 exporter/reader。指标按注册路由模板和固定方法分类;自动采集不记录 URL query、SQL 正文、Redis key、token 或业务载荷。业务自定义 span 名称同样应固定,避免用用户 ID 或对象 key。
|
||||||
|
|
||||||
|
## 分布式限流与幂等
|
||||||
|
|
||||||
|
`middleware/redis.NewRateLimiter(client, namespace, cfg)` 跨副本使用 Redis 时间与原子令牌桶;满额/存储失败返回 503,额度不足返回 429 和 Retry-After。`NewIdempotencyStore` 注入 `middleware.IdempotencyConfig.Store`;Scope 必须来自认证后的租户/用户,先认证再幂等。不同请求内容复用同一身份返回 409,已完成请求回放原响应和 requestId。
|
||||||
|
|
||||||
|
claim 在开始业务前持久保存,完成按 owner 原子检查。进程崩溃留下的 running claim 不自动按租约过期;返回 409 等待核查,`Resolve` 仅由运维核对后显式调用。已保存的完成/失败记录按配置 TTL 清理,TTL 过后不再提供业务去重保证。失败结果、含 Set-Cookie/no-store 的响应不用于成功回放。共享 Redis 需要 noeviction 和独立 namespace,数据丢失时不存在框架层面的重复执行保证。
|
||||||
|
|
||||||
|
## 迁移、审计与分页
|
||||||
|
|
||||||
|
迁移命名:`0001_createUsers.up.sql`,可附 `.down.sql`,但第一版 CLI 只执行 up。`go-omni migrate --dir migrations` 默认打印版本/方向/SHA-256 清单,不连接数据库;部署步骤显式传 `--apply` 才读取 `GO_OMNI_DB_DSN` 执行。迁移使用独立连接与数据库锁;MySQL DDL 可能部分提交,失败保留 dirty,禁止自动 force 或回滚。当前基于 golang-migrate,数据库版本表不保存每个历史文件校验和,部署系统应保留并校对预览清单。取消在当前有界 SQL 操作结束后停止,重试需新建 Runner。
|
||||||
|
|
||||||
|
审计:`audit.New(sink, "approvedField")` 建白名单,再 `Record(ctx, Event{ActorID, Action, Resource, Outcome, Fields})`;补入服务端时间、requestId/traceId,只保留明确允许的属性,Sink 失败原样返回。JSONSink 不保证与业务提交原子一致,需要这种语义时由业务在同一数据库事务内提供 Sink/Outbox。
|
||||||
|
|
||||||
|
分页:`pagination.Parse(request.URL.Query(), pagination.Config{})` 默认 page=1/pageSize=20,上限 100,最大 offset=1,000,000;拒绝重复参数、溢出和越界。`pagination.NewResult(request, items, total)` 输出 items/page/pageSize/total/totalPages,空列表为 `[]`。查询与租户过滤仍由 Repository 负责。
|
||||||
|
|
||||||
|
## 验证入口
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test -race ./...
|
||||||
|
go vet ./...
|
||||||
|
go build ./...
|
||||||
|
python3 scripts/verifyModules.py --advanced --goproxy https://goproxy.cn,direct --output performance/advancedLinux.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Linux runner 顺序运行编译、Redis、MinIO、MySQL 四个容器,各限 2 CPU、2 GiB、256 pids;数据阶段无外网、无发布端口,不连接现有数据库。原始结果和源码摘要以生成的 JSON/TXT 为准,未提供云凭据的 OSS/R2 不标为实网通过。
|
||||||
|
|
||||||
|
本批已通过上述 Linux 验证,源代码提交、98 个文件摘要、服务版本和边界见 [验收记录](performance/advancedAcceptance.md)。
|
||||||
169
API.md
Normal file
169
API.md
Normal file
@@ -0,0 +1,169 @@
|
|||||||
|
# API 契约与项目生成
|
||||||
|
|
||||||
|
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
|
||||||
|
|
||||||
|
## 与 go-zero 的对应关系
|
||||||
|
|
||||||
|
调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式:统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写,不引入 go-zero 或 Gin 运行时依赖。
|
||||||
|
|
||||||
|
| 使用体验 | go-omni 实现 |
|
||||||
|
| --- | --- |
|
||||||
|
| 新建接口项目 | `go-omni api new <directory>` |
|
||||||
|
| 按契约生成、更新 | `go-omni api generate --api user.api --dir .` |
|
||||||
|
| 熟悉的命令别名 | `api go -api user.api -dir .` |
|
||||||
|
| 校验契约 | `api validate --api user.api` |
|
||||||
|
| CI 检查生成产物是否过期 | `api generate --api user.api --dir . --check` |
|
||||||
|
| 业务逻辑目录 | `internal/service/<group>`,沿用 Omni 的 Service 分层 |
|
||||||
|
| 依赖装配 | `internal/servicecontext` |
|
||||||
|
| 接口文档 | 自动生成 `docs/openapi.json`,同时提供 `/openapi.json` |
|
||||||
|
|
||||||
|
这是明确限定的 go-zero 风格方言,**不是完整 goctl 兼容实现**。不支持的声明和校验标签会在生成前报错。
|
||||||
|
|
||||||
|
## 立即试用
|
||||||
|
|
||||||
|
源码开发时,从 go-omni 仓库运行,显式引用本地框架:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni api new \
|
||||||
|
--module example.com/omniDemo --framework-dir . ../omniDemo
|
||||||
|
|
||||||
|
cd ../omniDemo
|
||||||
|
go mod tidy
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
访问 `http://127.0.0.1:8080/api/greet/Alice` 可得到带 `data.message` 的统一响应。`api new` 的问候示例包含可运行逻辑;自定义契约新生成的 Service 会明确返回 501,等待业务实现。
|
||||||
|
|
||||||
|
用现成的多分组定义生成项目,在 go-omni 仓库执行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni new --api examples/contracts/user.api \
|
||||||
|
--module example.com/userService --framework-dir . \
|
||||||
|
--log-output file --log-format json ../userService
|
||||||
|
```
|
||||||
|
|
||||||
|
`api generate` 在新目录中也会生成完整项目。默认只监听 localhost;`GO_OMNI_HTTP_ADDR` 修改地址,`GO_OMNI_TRANSPORT=native` 切换网络层。配置顺序为默认值 → `etc/config.yaml` → 环境变量。文件日志支持现有轮转配置。远程试用使用与 CLI 一致的固定版本,见 [发布说明](RELEASE.md)。
|
||||||
|
|
||||||
|
API 项目现在支持组合数据库、存储和日志:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni new --api user.api --module example.com/userService \
|
||||||
|
--database mysql --storage local --log-output file ./userService
|
||||||
|
```
|
||||||
|
|
||||||
|
- `--database none|memory|mysql`:API 默认 `none`,`memory` 是不创建数据库连接的别名;MySQL 生成连接池、配置、事务入口和 readiness 探针。
|
||||||
|
- `--storage none|local|s3|r2|oss`:默认 none,生成统一 Files / Uploads / Downloads / UploadLinks 依赖;本地签名对象端点为 `/storage/object`。
|
||||||
|
- `--log-output stdout|stderr|file` 与 `--log-format json|text`:配置和无配置文件启动使用一致默认值。
|
||||||
|
|
||||||
|
初次选项记录在 `.goOmni/apiManifest.json`,以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错,因为初始化代码属于业务方;可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。旧式 CRUD 脚手架仍保留。
|
||||||
|
|
||||||
|
MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
|
||||||
|
|
||||||
|
HTTP 请求类型不能推导数据库表结构;迁移和 Repository 由业务自己维护。完整案例见 [用户管理示例](examples/userService/README.md),覆盖真实 MySQL、事务审计、鉴权、文件导出和日志。
|
||||||
|
|
||||||
|
## 接口定义
|
||||||
|
|
||||||
|
```go
|
||||||
|
syntax = "v1"
|
||||||
|
|
||||||
|
type UpdateUserReq {
|
||||||
|
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||||
|
Notify bool `form:"notify,default=false"`
|
||||||
|
Name string `json:"name" binding:"required,max=50"`
|
||||||
|
Age *int `json:"age,optional,range=[0:150]"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type UserResp {
|
||||||
|
UserID int64 `json:"userId"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
|
||||||
|
@server (
|
||||||
|
group: user
|
||||||
|
prefix: /api/v1
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@doc "Update a user"
|
||||||
|
@handler updateUser
|
||||||
|
put /users/:userId (UpdateUserReq) returns (UserResp)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
类型和 Go 字段使用导出命名;JSON、query、path 名称使用 lowerCamelCase;Header 使用 HTTP 标准名称。响应类型只定义 `data`,统一响应的 `code`、`message`、`requestId` 由框架添加。
|
||||||
|
|
||||||
|
支持:
|
||||||
|
|
||||||
|
- 可省略的 `syntax = "v1"`;`info` 中的 title、version 和描述性元数据。
|
||||||
|
- 单独的 `type X { ... }` 与 `type (...)`;基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。
|
||||||
|
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
|
||||||
|
- 同一 service 名称的多个块;`@server` 的 group、prefix、middleware,group 可以写 `admin/user`。
|
||||||
|
- 每个路由必须有 `@handler`;可选 `@doc "..."`;GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。
|
||||||
|
- 可省略请求或响应;有响应默认 200,无响应默认 204。Omni 扩展 `@status 201`、`@status 202`;204 不允许响应类型,HEAD 不允许响应体,GET/HEAD 不允许 JSON 请求体。
|
||||||
|
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query,不解析 multipart 或表单 body。query 切片用重复键;header/path 使用标量。
|
||||||
|
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
|
||||||
|
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
|
||||||
|
|
||||||
|
暂不支持匿名嵌入、别名、递归类型、外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证请用命名中间件接入。`@doc` 块等未列出的语法也会明确报错。
|
||||||
|
|
||||||
|
## 参数语义
|
||||||
|
|
||||||
|
无 optional/default 的字段按“是否出现”判断必填,`false`、`0`、`""` 是已提供的值。额外声明 `binding:"required"` 才会拒绝标量零值。默认值只在字段缺失时使用,不替换显式的空值。
|
||||||
|
|
||||||
|
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;非指针拒绝 null;`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
|
||||||
|
|
||||||
|
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
|
||||||
|
|
||||||
|
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;非 nullable 集合请返回空集合而不是 nil。
|
||||||
|
|
||||||
|
## 生成目录和业务边界
|
||||||
|
|
||||||
|
```text
|
||||||
|
api/ # 开发者维护的契约及导入
|
||||||
|
cmd/server/main.go # 启动、日志、优雅退出
|
||||||
|
etc/config.yaml
|
||||||
|
internal/settings/
|
||||||
|
internal/servicecontext/ # 依赖初始化和释放
|
||||||
|
internal/types/typesGen.go
|
||||||
|
internal/handler/routesGen.go
|
||||||
|
internal/handler/openapiGen.go
|
||||||
|
internal/handler/<group>/handlersGen.go
|
||||||
|
internal/service/<group>/service.go
|
||||||
|
internal/service/<group>/<operation>.go
|
||||||
|
internal/repository/<group>/repository.go
|
||||||
|
internal/middleware/registryGen.go
|
||||||
|
internal/middleware/<name>.go
|
||||||
|
docs/openapi.json
|
||||||
|
.goOmni/apiManifest.json
|
||||||
|
```
|
||||||
|
|
||||||
|
Service 方法接收标准 `context.Context` 与生成的请求 DTO;Handler 调用 Service 并完成响应包装。网络适配器仍经 Omni/httpx 接入。Repository 是业务所有的扩展位置,不自动生成虚构 CRUD 查询。
|
||||||
|
|
||||||
|
命名中间件按声明顺序组合,第一个在最外层。新 hook 初始返回 501,必须实现后才放行;registry 自动维护,hook 文件只创建一次。缺失 hook 在注册任何路由前报错。文档通过 `x-omni-middleware` 标出命名 hook,不把名称误当成已实现的认证协议。
|
||||||
|
|
||||||
|
## 安全更新与 CI
|
||||||
|
|
||||||
|
在生成项目中安装/使用与框架版本相符的 go-omni CLI:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni api validate --api api/user.api
|
||||||
|
go-omni api generate --api api/user.api --dir .
|
||||||
|
go-omni api generate --api api/user.api --dir . --check
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
`Gen.go` 和 OpenAPI 由生成器管理,manifest 保存其 SHA-256。手动修改后更新会报冲突。Service、Repository、main、配置、依赖装配和中间件实现按字节保留;只补缺失文件。删除接口/分组时只移除旧的生成文件,保留业务代码;已删除类型仍被旧业务引用时,需要明确迁移该业务文件,编译器会提示。
|
||||||
|
|
||||||
|
`--check` 只读,检查产物是否过期,以及所需骨架是否缺失。契约复制到项目后,请编辑项目内的 API;外部定义与已存在副本不一致时会拒绝覆盖。一个项目由一个 API 入口管理,可以通过导入拆分文件。
|
||||||
|
|
||||||
|
生成器拒绝输出路径穿越、受管理路径上的符号链接、大小写路径冲突和无所有权的覆盖。生成前统一校验/格式化,锁文件防止两个生成器同时修改;文件以原子替换写入,普通写入失败会回滚。本版不是整个目录的崩溃原子事务:进程被强杀或机器断电后,应按 Git 恢复生成产物和 manifest,再重试。请将 manifest 与生成产物一并提交,也不要在生成过程中编辑这些文件。
|
||||||
|
|
||||||
|
生成器不自动安装依赖、不访问数据库、不推送仓库、不替业务实现权限规则。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test -race -p=2 ./contract ./openapi ./cmd/go-omni/...
|
||||||
|
python3 scripts/verifyAPI.py
|
||||||
|
```
|
||||||
|
|
||||||
|
Go 测试覆盖契约拒绝规则、多源绑定、零值/缺失、默认/范围/嵌套集合、文档约束、幂等更新、只读检查、业务保留、删除路由、路径边界及生成项目编译。Python 验收会在临时目录生成项目,分别启动 nethttp/native,验证真实 HTTP 请求、中间件、日志 requestId 和优雅退出;结束后清理自己创建的进程和临时文件。结果记录在 `performance/apiGenerationLocal.json`。
|
||||||
12
CHANGELOG.md
12
CHANGELOG.md
@@ -1,6 +1,16 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
## Unreleased
|
## v0.7.0-preview
|
||||||
|
|
||||||
|
- 新增 `.api` 契约驱动生成器:DTO、严格多来源绑定与校验、路由、分组 Handler / Service / Repository、命名中间件及 OpenAPI;生成文件哈希管理、业务文件保护和只读 `--check`。
|
||||||
|
- `new --api` / `api new` / `api generate` 可以组合 MySQL、日志以及 Local / S3 / R2 / OSS。保存初始基础设施选项,重生成自动沿用;显式切换已有项目选项会拒绝,避免静默丢失初始化代码。
|
||||||
|
- 新增 `examples/userService` 独立业务示例与 `verifyAPIWorkflow.py`:用户 CRUD / 分页、管理员 Token 鉴权、MySQL 事务审计、故障回滚、文件导出、requestId 日志、数据库 readiness 和优雅退出;支持从远程版本验收。
|
||||||
|
- 日志新增文件轮转、结构化敏感字段脱敏、异常堆栈及慢请求配置;存储新增统一上传策略、直传签名和云端分片能力。
|
||||||
|
- 新增缓存、HTTP 客户端、限流 / 幂等 / 超时、后台任务、Redis 队列、事务 outbox、分布式锁、审计、迁移与 telemetry 等可选模块;业务身份与 Orbit 保持由应用自行组合。
|
||||||
|
- `database.ApplyEnv` 支持文件/默认配置叠加环境变量;数据库配置使用 lowerCamelCase,DSN 不参与 JSON/YAML 序列化。
|
||||||
|
- 修复 native 在业务层提前拒绝请求体为 413 时等待继续读取 body 的问题,拒绝后关闭未读连接。
|
||||||
|
|
||||||
|
以下是自上一版本以来的网络层变化与历史验收记录:
|
||||||
|
|
||||||
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
||||||
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
||||||
|
|||||||
@@ -1,6 +1,8 @@
|
|||||||
# 业务基础模块
|
# 业务基础模块
|
||||||
|
|
||||||
这批能力位于 main,尚未包含在 `v0.6.0-preview` 标签中。统一示例见
|
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
|
||||||
|
|
||||||
|
这批能力从 `v0.7.0-preview` 提供。统一示例见
|
||||||
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
||||||
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
||||||
|
|
||||||
|
|||||||
30
README.md
30
README.md
@@ -5,7 +5,7 @@ GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用
|
|||||||
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
go get git.yuntuo.hk/k2safe/go-omni@v0.7.0-preview
|
||||||
```
|
```
|
||||||
|
|
||||||
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
||||||
@@ -13,7 +13,7 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
|||||||
创建新业务项目:
|
创建新业务项目:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||||
go-omni new --module example.com/billing ./billing
|
go-omni new --module example.com/billing ./billing
|
||||||
cd billing
|
cd billing
|
||||||
go mod tidy
|
go mod tidy
|
||||||
@@ -27,6 +27,17 @@ go run ./cmd/server
|
|||||||
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
|
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
|
||||||
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
|
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
|
||||||
|
|
||||||
|
## API 定义与项目生成
|
||||||
|
|
||||||
|
当前源码新增 **API 定义驱动项目生成**:写 `.api` 定义请求、响应和路由,自动生成项目目录、DTO、Handler、Service 骨架及 OpenAPI。重复生成保留业务代码,支持只读 CI 检查,完整语法和示例见 [API 开发指南](API.md)。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni api new --framework-dir . \
|
||||||
|
--module example.com/omniDemo ../omniDemo
|
||||||
|
```
|
||||||
|
|
||||||
|
API 生成支持 `--database mysql --storage local --log-output file` 组合初始化。上面的命令使用本地源码;远程安装见 [发布说明](RELEASE.md),完整业务案例见 [userService](examples/userService/README.md)。
|
||||||
|
|
||||||
## 目录结构
|
## 目录结构
|
||||||
|
|
||||||
```text
|
```text
|
||||||
@@ -41,11 +52,19 @@ go-omni/
|
|||||||
├── metrics/ # 请求指标与状态统计
|
├── metrics/ # 请求指标与状态统计
|
||||||
├── config/ # 配置管理
|
├── config/ # 配置管理
|
||||||
├── validate/ # 参数校验
|
├── validate/ # 参数校验
|
||||||
|
├── contract/ # API 字段契约、严格多源绑定与文档约束
|
||||||
├── auth/ # 认证和权限接口
|
├── auth/ # 认证和权限接口
|
||||||
├── database/ # database/sql 生命周期和事务
|
├── database/ # database/sql 生命周期和事务
|
||||||
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
||||||
├── httpc/ # 出站 HTTP 连接池、超时和重试
|
├── httpc/ # 出站 HTTP 连接池、超时和重试
|
||||||
├── task/ # 有界后台任务与间隔调度
|
├── task/ # 有界后台任务与间隔调度
|
||||||
|
├── queue/ # 持久队列接口与 Redis Streams 适配
|
||||||
|
├── storage/ # Local、S3/R2、OSS 与临时下载链接
|
||||||
|
├── openapi/ # 路由和 OpenAPI 3.0.3 契约
|
||||||
|
├── telemetry/ # 可注入 OpenTelemetry 观测
|
||||||
|
├── resilience/ # 熔断与上游并发隔离
|
||||||
|
├── audit/ # 显式业务审计与可注入 Sink
|
||||||
|
├── pagination/ # 有界分页和统一结果
|
||||||
├── transport/nethttp/ # 标准库网络适配器
|
├── transport/nethttp/ # 标准库网络适配器
|
||||||
├── transport/native/ # 自研 HTTP/1.1 服务端预览
|
├── transport/native/ # 自研 HTTP/1.1 服务端预览
|
||||||
├── transport/http2/ # opt-in HTTP/2 实验适配器
|
├── transport/http2/ # opt-in HTTP/2 实验适配器
|
||||||
@@ -55,7 +74,10 @@ go-omni/
|
|||||||
|
|
||||||
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
||||||
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
||||||
可以直接运行 [组合示例](examples/modules/README.md)。这些新增能力尚未包含在上面的历史版本标签中。
|
可以直接运行 [组合示例](examples/modules/README.md)。这些能力从 `v0.7.0-preview` 提供。
|
||||||
|
|
||||||
|
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
|
||||||
|
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
|
||||||
|
|
||||||
## 快速开始
|
## 快速开始
|
||||||
|
|
||||||
@@ -157,7 +179,7 @@ app.GET("/metrics", requestMetrics.Handler())
|
|||||||
|
|
||||||
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
||||||
|
|
||||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。
|
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。文件轮转、敏感字段脱敏、异常堆栈与慢请求配置见 [logx 文档](logx/README.md);文件直传与云端分片上传见 [storage 文档](storage/README.md)。 创建新业务项目时可通过 `go-omni new --log-output file --storage local` 按需生成配置、装配代码与可运行示例,当前源码使用方式见 [CLI 文档](cmd/go-omni/README.md)。
|
||||||
|
|
||||||
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
||||||
|
|
||||||
|
|||||||
63
RELEASE.md
Normal file
63
RELEASE.md
Normal file
@@ -0,0 +1,63 @@
|
|||||||
|
# v0.7.0-preview
|
||||||
|
|
||||||
|
这一版把 API 定义、项目生成、基础模块初始化和完整业务示例接通。
|
||||||
|
预览版用于固定版本试用;API 方言有明确范围,HTTP/2 仍是独立实验适配器。
|
||||||
|
|
||||||
|
## 安装与开始
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GOPRIVATE=git.yuntuo.hk go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||||
|
|
||||||
|
go-omni api new --module example.com/demo ./demo
|
||||||
|
cd demo
|
||||||
|
GOPRIVATE=git.yuntuo.hk go mod tidy
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
需要该 Git 仓库的读取权限时,通过自己的 Git 凭据管理器配置;不要将凭据写入 module path 或 go.mod。
|
||||||
|
CLI 会将自身下载版本固定到生成项目的 go.mod。业务项目正常使用远程模块,不需要本地 replace。
|
||||||
|
|
||||||
|
使用已有定义,并一次选择基础模块:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni new --api user.api --module example.com/userService \
|
||||||
|
--database mysql --storage local --log-output file ./userService
|
||||||
|
```
|
||||||
|
|
||||||
|
MySQL 与存储需要按生成 README 设置环境变量。任意契约生成的业务方法先返回 501,
|
||||||
|
由业务方实现;可直接参考 [完整用户示例](examples/userService/README.md)。
|
||||||
|
|
||||||
|
## 主要变化
|
||||||
|
|
||||||
|
- `.api` 生成项目、绑定/校验、路由、业务骨架和 OpenAPI;支持分组、导入、命名中间件。
|
||||||
|
- 同一入口初始化 MySQL、日志、Local / S3 / R2 / OSS;标准 context 与 httpx 保持业务/网络分离。
|
||||||
|
- 生成器保护手写业务文件,校验自有文件哈希,支持只读 --check;基础设施选项持久化到 manifest。
|
||||||
|
- 完整用户示例包含数据库迁移、CRUD / 分页、事务审计、鉴权、文件导出与 requestId 日志。
|
||||||
|
- 日志文件轮转/结构化脱敏、上传策略/直传/分片,以及此前 main 分支的通用业务模块一并提供。
|
||||||
|
|
||||||
|
所有变化及网络层验收记录见 [CHANGELOG](CHANGELOG.md)。
|
||||||
|
|
||||||
|
## 升级注意
|
||||||
|
|
||||||
|
1. 将 CLI 和项目框架依赖固定为相同版本。旧版 CRUD 项目继续使用现有目录;不能直接用 API 生成器接管已有 Go 目录。
|
||||||
|
2. Gen.go 由生成器维护,业务实现/配置/迁移由项目维护。新增 API 后执行 generate 和 go test;修改或删除类型需要同步处理自己的业务代码。
|
||||||
|
3. `.goOmni/apiManifest.json` 需随代码提交。切换数据库或存储选项时,在新目录生成对照后迁移;重新生成不会改写现有基础设施文件。
|
||||||
|
4. 配置使用 lowerCamelCase。DSN、Token、存储密钥由环境变量提供;`database.Config` 的 DSN 不会被 JSON/YAML 序列化。
|
||||||
|
5. API 的 none/memory 数据库选项仅表示不打开连接,不会从 HTTP DTO 自动推导内存仓库或数据表。
|
||||||
|
|
||||||
|
## 验证入口
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=2 -count=1 -timeout=300s ./...
|
||||||
|
go vet ./...
|
||||||
|
go build ./...
|
||||||
|
python3 scripts/verifyAPIWorkflow.py
|
||||||
|
python3 scripts/verifyAPIWorkflow.py --version v0.7.0-preview \
|
||||||
|
--report performance/apiWorkflowRemote.json
|
||||||
|
```
|
||||||
|
|
||||||
|
最后一项在全新 module cache 安装远程 CLI,并验证无本地 replace 的项目生成、
|
||||||
|
真实 MySQL、nethttp/native 启动与业务流程。容器只使用本机临时资源,结束时清理。
|
||||||
|
本轮验收见 [API 业务闭环报告](performance/apiWorkflowAcceptance.md)。
|
||||||
|
|
||||||
|
OSS/R2 云端仍需项目自己的账号做上线环境验收;当前云端适配器检查不等同于这些账号的实网测试。
|
||||||
@@ -81,4 +81,11 @@
|
|||||||
- 最终代码已通过 Linux amd64 全仓库 race/vet/build、独立生成项目 CRUD、真实 Redis 7.4.11 与三种网络层 TLS 组合测试;测试容器清理完成、原有 7 个业务容器仍运行。提交、源码摘要与完整输出见 [验收记录](performance/modulesAcceptance.md)。
|
- 最终代码已通过 Linux amd64 全仓库 race/vet/build、独立生成项目 CRUD、真实 Redis 7.4.11 与三种网络层 TLS 组合测试;测试容器清理完成、原有 7 个业务容器仍运行。提交、源码摘要与完整输出见 [验收记录](performance/modulesAcceptance.md)。
|
||||||
- 当前幂等、限流、任务调度和合并回源均为单进程能力;Redis 仅提供缓存存储,不代表已实现分布式去重、任务持久化或跨副本选主。eTax 指定业务模块接入依用户要求暂缓。
|
- 当前幂等、限流、任务调度和合并回源均为单进程能力;Redis 仅提供缓存存储,不代表已实现分布式去重、任务持久化或跨副本选主。eTax 指定业务模块接入依用户要求暂缓。
|
||||||
|
|
||||||
|
## 7. 扩展业务模块(本批)
|
||||||
|
|
||||||
|
- 新增 OpenAPI、JWT/Session/Policy、OpenTelemetry、文件配置与日志级别热更新、熔断/并发隔离、Redis 共享限流/幂等、Redis Streams 持久队列、Local/S3/R2/OSS 存储;补充迁移、审计和分页。
|
||||||
|
- 业务依赖独立接口,云平台 SDK 和 Redis 留在适配器中;按本次范围决定不集成 Orbit,不实现工作流编排。
|
||||||
|
- 已完成本地模块测试及 race 检查,Linux 全仓库 race/vet/build、生成项目 CRUD、三个 Runner 的 TLS 示例和独立 Redis/MinIO/MySQL 均通过。共享状态、重复提交、失败消息、未确认消息接管、旧确认拒绝、对象签名和迁移 dirty 阻断有对应证据,见 [扩展模块验收](performance/advancedAcceptance.md)。OSS/R2 尚无云账号实网验收。
|
||||||
|
- 契约、使用示例与限制见 [扩展模块](ADVANCED.md);验证入口为 `scripts/verifyModules.py --advanced`,继续限制单个测试容器 2 CPU / 2 GiB / 256 pids。
|
||||||
|
|
||||||
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,自研网络层已有可运行预览,稳定性与生产规模验证尚未完成。
|
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,自研网络层已有可运行预览,稳定性与生产规模验证尚未完成。
|
||||||
|
|||||||
@@ -65,13 +65,14 @@ func configure(options ...config.Option) config.Config {
|
|||||||
|
|
||||||
func newApp(router httpx.Router, settings config.Config) *App {
|
func newApp(router httpx.Router, settings config.Config) *App {
|
||||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
panic("go-omni: " + err.Error())
|
panic("go-omni: " + err.Error())
|
||||||
}
|
}
|
||||||
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
||||||
if settings.EnableLogger {
|
if settings.EnableLogger {
|
||||||
middlewares = append(middlewares, middleware.AccessLog())
|
middlewares = append(middlewares, middleware.AccessLog(middleware.WithSlowRequestThreshold(settings.SlowRequestThreshold)))
|
||||||
}
|
}
|
||||||
middlewares = append(middlewares, middleware.Recovery())
|
middlewares = append(middlewares, middleware.Recovery())
|
||||||
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
|
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
|
||||||
@@ -93,6 +94,7 @@ func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
|
|||||||
func (a *App) Config() config.Config {
|
func (a *App) Config() config.Config {
|
||||||
settings := a.config
|
settings := a.config
|
||||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
return settings
|
return settings
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
106
audit/audit.go
Normal file
106
audit/audit.go
Normal file
@@ -0,0 +1,106 @@
|
|||||||
|
// Package audit records explicitly selected business events without request bodies.
|
||||||
|
package audit
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"io"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Event struct {
|
||||||
|
Time time.Time `json:"time"`
|
||||||
|
ActorID string `json:"actorId"`
|
||||||
|
TenantID string `json:"tenantId,omitempty"`
|
||||||
|
Action string `json:"action"`
|
||||||
|
Resource string `json:"resource"`
|
||||||
|
Outcome string `json:"outcome"`
|
||||||
|
RequestID string `json:"requestId,omitempty"`
|
||||||
|
TraceID string `json:"traceId,omitempty"`
|
||||||
|
Fields map[string]string `json:"fields,omitempty"`
|
||||||
|
}
|
||||||
|
type Sink interface {
|
||||||
|
Write(context.Context, Event) error
|
||||||
|
}
|
||||||
|
type Recorder struct {
|
||||||
|
sink Sink
|
||||||
|
allowed map[string]bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func New(sink Sink, allowedFields ...string) (*Recorder, error) {
|
||||||
|
if sink == nil || len(allowedFields) > 32 {
|
||||||
|
return nil, errors.New("audit: sink and at most 32 fields required")
|
||||||
|
}
|
||||||
|
r := &Recorder{sink: sink, allowed: map[string]bool{}}
|
||||||
|
for _, name := range allowedFields {
|
||||||
|
if name == "" || len(name) > 64 {
|
||||||
|
return nil, errors.New("audit: invalid field")
|
||||||
|
}
|
||||||
|
r.allowed[name] = true
|
||||||
|
}
|
||||||
|
return r, nil
|
||||||
|
}
|
||||||
|
func (r *Recorder) Record(ctx context.Context, event Event) error {
|
||||||
|
if ctx == nil {
|
||||||
|
return errors.New("audit: nil context")
|
||||||
|
}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if event.ActorID == "" || event.Action == "" || event.Resource == "" || (event.Outcome != "success" && event.Outcome != "failure" && event.Outcome != "denied") {
|
||||||
|
return errors.New("audit: actor/action/resource and valid outcome required")
|
||||||
|
}
|
||||||
|
for _, value := range []string{event.ActorID, event.TenantID, event.Action, event.Resource} {
|
||||||
|
if len(value) > 512 {
|
||||||
|
return errors.New("audit: event field too large")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
event.Time = time.Now().UTC()
|
||||||
|
event.RequestID = logx.RequestID(ctx)
|
||||||
|
event.TraceID = logx.TraceID(ctx)
|
||||||
|
filtered := map[string]string{}
|
||||||
|
for name, value := range event.Fields {
|
||||||
|
if r.allowed[name] {
|
||||||
|
if len(value) > 1024 {
|
||||||
|
return errors.New("audit: attribute too large")
|
||||||
|
}
|
||||||
|
filtered[name] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
event.Fields = filtered
|
||||||
|
return r.sink.Write(ctx, event)
|
||||||
|
}
|
||||||
|
|
||||||
|
// JSONSink borrows its writer and serializes complete JSON lines. Write errors
|
||||||
|
// are returned; callers choose whether a failed audit must reject an operation.
|
||||||
|
type JSONSink struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
writer io.Writer
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewJSONSink(writer io.Writer) (*JSONSink, error) {
|
||||||
|
if writer == nil {
|
||||||
|
return nil, errors.New("audit: writer required")
|
||||||
|
}
|
||||||
|
return &JSONSink{writer: writer}, nil
|
||||||
|
}
|
||||||
|
func (s *JSONSink) Write(ctx context.Context, event Event) error {
|
||||||
|
data, err := json.Marshal(event)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
data = append(data, '\n')
|
||||||
|
s.mu.Lock()
|
||||||
|
defer s.mu.Unlock()
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
n, err := s.writer.Write(data)
|
||||||
|
if err == nil && n != len(data) {
|
||||||
|
return io.ErrShortWrite
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
50
audit/audit_test.go
Normal file
50
audit/audit_test.go
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
package audit
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"io"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
type failedWriter struct{}
|
||||||
|
|
||||||
|
func (failedWriter) Write([]byte) (int, error) { return 0, io.ErrClosedPipe }
|
||||||
|
func TestAllowlistCorrelationAndWriteFailure(t *testing.T) {
|
||||||
|
var buf bytes.Buffer
|
||||||
|
sink, _ := NewJSONSink(&buf)
|
||||||
|
recorder, _ := New(sink, "status")
|
||||||
|
ctx := logx.WithRequestID(context.Background(), "r1")
|
||||||
|
ctx = logx.WithTraceID(ctx, "t1")
|
||||||
|
event := Event{ActorID: "u1", Action: "update", Resource: "object:1", Outcome: "success", Fields: map[string]string{"status": "ok", "password": "secret"}}
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 20; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
if err := recorder.Record(ctx, event); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
decoder := json.NewDecoder(&buf)
|
||||||
|
for i := 0; i < 20; i++ {
|
||||||
|
var got Event
|
||||||
|
if err := decoder.Decode(&got); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.Fields["password"] != "" || got.RequestID != "r1" || got.TraceID != "t1" || got.Time.IsZero() {
|
||||||
|
t.Fatal(got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
bad, _ := NewJSONSink(failedWriter{})
|
||||||
|
recorder, _ = New(bad)
|
||||||
|
if err := recorder.Record(ctx, event); !errors.Is(err, io.ErrClosedPipe) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
105
auth/adapters_test.go
Normal file
105
auth/adapters_test.go
Normal file
@@ -0,0 +1,105 @@
|
|||||||
|
package auth
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/ed25519"
|
||||||
|
"crypto/rand"
|
||||||
|
"errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||||
|
"github.com/golang-jwt/jwt/v5"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestJWTValidationAndRotation(t *testing.T) {
|
||||||
|
pub, private, err := ed25519.GenerateKey(rand.Reader)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
j, err := NewJWT(JWTConfig{Issuer: "issuer", Audience: "api", Keys: map[string]JWTKey{"first": {Algorithm: "EdDSA", PublicKey: pub}}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
claims := JWTClaims{RegisteredClaims: jwt.RegisteredClaims{Issuer: "issuer", Subject: "user1", Audience: jwt.ClaimStrings{"api"}, ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour))}, TenantID: "tenant1", Roles: []string{"reader"}}
|
||||||
|
sign := func(c JWTClaims, kid string, method jwt.SigningMethod, key any) string {
|
||||||
|
t.Helper()
|
||||||
|
token := jwt.NewWithClaims(method, c)
|
||||||
|
token.Header["kid"] = kid
|
||||||
|
raw, err := token.SignedString(key)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return raw
|
||||||
|
}
|
||||||
|
raw := sign(claims, "first", jwt.SigningMethodEdDSA, private)
|
||||||
|
// Constructor clones caller-owned key bytes.
|
||||||
|
pub[0] ^= 1
|
||||||
|
p, err := j.Authenticate(context.Background(), raw)
|
||||||
|
if err != nil || p.TenantID != "tenant1" || !p.HasRole("reader") {
|
||||||
|
t.Fatalf("%+v %v", p, err)
|
||||||
|
}
|
||||||
|
for name, change := range map[string]func(*JWTClaims){"issuer": func(c *JWTClaims) { c.Issuer = "bad" }, "audience": func(c *JWTClaims) { c.Audience = jwt.ClaimStrings{"other"} }, "expired": func(c *JWTClaims) { c.ExpiresAt = jwt.NewNumericDate(time.Now().Add(-time.Hour)) }, "noExpiry": func(c *JWTClaims) { c.ExpiresAt = nil }, "future": func(c *JWTClaims) { c.NotBefore = jwt.NewNumericDate(time.Now().Add(time.Hour)) }, "subject": func(c *JWTClaims) { c.Subject = "" }} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
bad := claims
|
||||||
|
change(&bad)
|
||||||
|
if _, err = j.Authenticate(context.Background(), sign(bad, "first", jwt.SigningMethodEdDSA, private)); !errors.Is(err, ErrInvalidToken) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
for _, invalid := range []string{sign(claims, "missing", jwt.SigningMethodEdDSA, private), sign(claims, "first", jwt.SigningMethodHS256, []byte("invalid secret")), strings.Repeat("x", 8193)} {
|
||||||
|
if _, err = j.Authenticate(context.Background(), invalid); !errors.Is(err, ErrInvalidToken) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err = j.ReplaceKeys(map[string]JWTKey{"second": {Algorithm: "EdDSA", PublicKey: private.Public()}}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = j.Authenticate(context.Background(), raw); !errors.Is(err, ErrInvalidToken) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = j.Authenticate(context.Background(), sign(claims, "second", jwt.SigningMethodEdDSA, private)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
cancel()
|
||||||
|
if _, err = j.Authenticate(ctx, raw); !errors.Is(err, context.Canceled) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestSessionsRevokeAndIsolation(t *testing.T) {
|
||||||
|
mem, _ := cache.NewMemory(cache.MemoryConfig{})
|
||||||
|
defer mem.Close()
|
||||||
|
s, err := NewSessions(mem, "session:", time.Minute)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx := context.Background()
|
||||||
|
token, err := s.Issue(ctx, Principal{UserID: "u1", TenantID: "t1", Roles: []string{"reader"}})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
key, _ := s.key(token)
|
||||||
|
if strings.Contains(key, token) {
|
||||||
|
t.Fatal("plaintext session secret stored")
|
||||||
|
}
|
||||||
|
p, err := s.Authenticate(ctx, token)
|
||||||
|
if err != nil || p.UserID != "u1" {
|
||||||
|
t.Fatal(p, err)
|
||||||
|
}
|
||||||
|
p.Roles[0] = "admin"
|
||||||
|
p, _ = s.Authenticate(ctx, token)
|
||||||
|
if p.HasRole("admin") {
|
||||||
|
t.Fatal("principal alias")
|
||||||
|
}
|
||||||
|
if err = s.Revoke(ctx, token); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = s.Authenticate(ctx, token); !errors.Is(err, ErrInvalidToken) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = s.Authenticate(ctx, "bad"); !errors.Is(err, ErrInvalidToken) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -21,9 +21,10 @@ var (
|
|||||||
// Principal is the authenticated identity made available to business handlers.
|
// Principal is the authenticated identity made available to business handlers.
|
||||||
// Claims are optional provider-specific values and should not be required by core business code.
|
// Claims are optional provider-specific values and should not be required by core business code.
|
||||||
type Principal struct {
|
type Principal struct {
|
||||||
UserID string
|
UserID string `json:"userId"`
|
||||||
Roles []string
|
TenantID string `json:"tenantId,omitempty"`
|
||||||
Claims map[string]any
|
Roles []string `json:"roles,omitempty"`
|
||||||
|
Claims map[string]any `json:"claims,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p Principal) HasRole(role string) bool {
|
func (p Principal) HasRole(role string) bool {
|
||||||
|
|||||||
127
auth/jwt.go
Normal file
127
auth/jwt.go
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
package auth
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/ed25519"
|
||||||
|
"crypto/rsa"
|
||||||
|
"crypto/x509"
|
||||||
|
"errors"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"github.com/golang-jwt/jwt/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
// JWTKey binds a trusted key ID to exactly one asymmetric signing algorithm.
|
||||||
|
// Keys come from server configuration, never a token's jku/x5u URL.
|
||||||
|
type JWTKey struct {
|
||||||
|
Algorithm string
|
||||||
|
PublicKey any
|
||||||
|
}
|
||||||
|
type JWTConfig struct {
|
||||||
|
Issuer string `json:"issuer"`
|
||||||
|
Audience string `json:"audience"`
|
||||||
|
Leeway time.Duration `json:"leeway"`
|
||||||
|
Keys map[string]JWTKey `json:"-"`
|
||||||
|
}
|
||||||
|
type JWT struct {
|
||||||
|
config JWTConfig
|
||||||
|
mu sync.RWMutex
|
||||||
|
keys map[string]JWTKey
|
||||||
|
}
|
||||||
|
type JWTClaims struct {
|
||||||
|
jwt.RegisteredClaims
|
||||||
|
TenantID string `json:"tenantId,omitempty"`
|
||||||
|
Roles []string `json:"roles,omitempty"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewJWT(cfg JWTConfig) (*JWT, error) {
|
||||||
|
if cfg.Issuer == "" || cfg.Audience == "" || cfg.Leeway < 0 || cfg.Leeway > 2*time.Minute {
|
||||||
|
return nil, errors.New("auth: issuer/audience required, leeway must be 0..2m")
|
||||||
|
}
|
||||||
|
j := &JWT{config: cfg}
|
||||||
|
j.config.Keys = nil
|
||||||
|
if err := j.ReplaceKeys(cfg.Keys); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return j, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ReplaceKeys atomically rotates the trusted key set. Retain old keys until
|
||||||
|
// their tokens expire if a rotation should preserve existing sessions.
|
||||||
|
func (j *JWT) ReplaceKeys(keys map[string]JWTKey) error {
|
||||||
|
if len(keys) == 0 || len(keys) > 64 {
|
||||||
|
return errors.New("auth: require 1..64 JWT keys")
|
||||||
|
}
|
||||||
|
copyKeys := make(map[string]JWTKey, len(keys))
|
||||||
|
for id, key := range keys {
|
||||||
|
if id == "" || len(id) > 128 {
|
||||||
|
return errors.New("auth: invalid key ID")
|
||||||
|
}
|
||||||
|
switch key.Algorithm {
|
||||||
|
case "RS256":
|
||||||
|
k, ok := key.PublicKey.(*rsa.PublicKey)
|
||||||
|
if !ok || k == nil || k.N == nil || k.N.BitLen() < 2048 || k.E < 3 {
|
||||||
|
return errors.New("auth: RS256 requires a >=2048-bit RSA public key")
|
||||||
|
}
|
||||||
|
case "EdDSA":
|
||||||
|
k, ok := key.PublicKey.(ed25519.PublicKey)
|
||||||
|
if !ok || len(k) != ed25519.PublicKeySize {
|
||||||
|
return errors.New("auth: EdDSA requires an Ed25519 public key")
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return errors.New("auth: supported algorithms are RS256 and EdDSA")
|
||||||
|
}
|
||||||
|
encoded, err := x509.MarshalPKIXPublicKey(key.PublicKey)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
key.PublicKey, err = x509.ParsePKIXPublicKey(encoded)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
copyKeys[id] = key
|
||||||
|
}
|
||||||
|
j.mu.Lock()
|
||||||
|
j.keys = copyKeys
|
||||||
|
j.mu.Unlock()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (j *JWT) Authenticate(ctx context.Context, raw string) (Principal, error) {
|
||||||
|
if ctx == nil {
|
||||||
|
return Principal{}, ErrInvalidToken
|
||||||
|
}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return Principal{}, err
|
||||||
|
}
|
||||||
|
if len(raw) == 0 || len(raw) > 8192 {
|
||||||
|
return Principal{}, ErrInvalidToken
|
||||||
|
}
|
||||||
|
claims := new(JWTClaims)
|
||||||
|
token, err := jwt.ParseWithClaims(raw, claims, func(t *jwt.Token) (any, error) {
|
||||||
|
id, ok := t.Header["kid"].(string)
|
||||||
|
if !ok {
|
||||||
|
return nil, ErrInvalidToken
|
||||||
|
}
|
||||||
|
j.mu.RLock()
|
||||||
|
key, exists := j.keys[id]
|
||||||
|
j.mu.RUnlock()
|
||||||
|
if !exists || t.Method.Alg() != key.Algorithm {
|
||||||
|
return nil, ErrInvalidToken
|
||||||
|
}
|
||||||
|
return key.PublicKey, nil
|
||||||
|
}, jwt.WithValidMethods([]string{"RS256", "EdDSA"}), jwt.WithIssuer(j.config.Issuer), jwt.WithAudience(j.config.Audience), jwt.WithExpirationRequired(), jwt.WithIssuedAt(), jwt.WithLeeway(j.config.Leeway))
|
||||||
|
if err != nil || !token.Valid || claims.Subject == "" || len(claims.Subject) > 256 || len(claims.TenantID) > 256 || len(claims.Roles) > 64 {
|
||||||
|
return Principal{}, ErrInvalidToken
|
||||||
|
}
|
||||||
|
for _, role := range claims.Roles {
|
||||||
|
if role == "" || len(role) > 128 {
|
||||||
|
return Principal{}, ErrInvalidToken
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return Principal{}, err
|
||||||
|
}
|
||||||
|
return Principal{UserID: claims.Subject, TenantID: claims.TenantID, Roles: append([]string(nil), claims.Roles...)}, nil
|
||||||
|
}
|
||||||
34
auth/policy.go
Normal file
34
auth/policy.go
Normal file
@@ -0,0 +1,34 @@
|
|||||||
|
package auth
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Policy interface {
|
||||||
|
Authorize(context.Context, Principal, string, string) error
|
||||||
|
}
|
||||||
|
type PolicyFunc func(context.Context, Principal, string, string) error
|
||||||
|
|
||||||
|
func (f PolicyFunc) Authorize(ctx context.Context, p Principal, action, resource string) error {
|
||||||
|
return f(ctx, p, action, resource)
|
||||||
|
}
|
||||||
|
func RequirePermission(policy Policy, action string, resource func(*httpx.Context) string) httpx.Middleware {
|
||||||
|
return func(next httpx.Handler) httpx.Handler {
|
||||||
|
return func(c *httpx.Context) error {
|
||||||
|
p, ok := FromContext(c)
|
||||||
|
if !ok {
|
||||||
|
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||||
|
}
|
||||||
|
if policy == nil || action == "" || resource == nil {
|
||||||
|
return apierrors.Internal(errors.New("auth: incomplete permission policy"))
|
||||||
|
}
|
||||||
|
if err := policy.Authorize(c.Context(), p, action, resource(c)); err != nil {
|
||||||
|
return apierrors.Forbidden("FORBIDDEN", "permission denied").WithCause(err)
|
||||||
|
}
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
87
auth/session.go
Normal file
87
auth/session.go
Normal file
@@ -0,0 +1,87 @@
|
|||||||
|
package auth
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Sessions stores server-issued opaque bearer credentials as SHA-256 keys.
|
||||||
|
// The caller verifies login credentials before Issue; Store owns persistence.
|
||||||
|
type Sessions struct {
|
||||||
|
store cache.Store
|
||||||
|
prefix string
|
||||||
|
ttl time.Duration
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewSessions(store cache.Store, prefix string, ttl time.Duration) (*Sessions, error) {
|
||||||
|
if store == nil || len(prefix) > 512 || ttl < time.Second || ttl > 30*24*time.Hour {
|
||||||
|
return nil, errors.New("auth: invalid session configuration")
|
||||||
|
}
|
||||||
|
return &Sessions{store: store, prefix: prefix, ttl: ttl}, nil
|
||||||
|
}
|
||||||
|
func (s *Sessions) Issue(ctx context.Context, p Principal) (string, error) {
|
||||||
|
if p.UserID == "" || len(p.UserID) > 256 || len(p.TenantID) > 256 {
|
||||||
|
return "", ErrInvalidToken
|
||||||
|
}
|
||||||
|
data, err := json.Marshal(p)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if len(data) > 16384 {
|
||||||
|
return "", errors.New("auth: session principal too large")
|
||||||
|
}
|
||||||
|
secret := make([]byte, 32)
|
||||||
|
if _, err := rand.Read(secret); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
token := base64.RawURLEncoding.EncodeToString(secret)
|
||||||
|
key, _ := s.key(token)
|
||||||
|
if err := s.store.Set(ctx, key, data, s.ttl); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return token, nil
|
||||||
|
}
|
||||||
|
func (s *Sessions) key(token string) (string, error) {
|
||||||
|
if len(token) != 43 {
|
||||||
|
return "", ErrInvalidToken
|
||||||
|
}
|
||||||
|
b, err := base64.RawURLEncoding.DecodeString(token)
|
||||||
|
if err != nil || len(b) != 32 {
|
||||||
|
return "", ErrInvalidToken
|
||||||
|
}
|
||||||
|
sum := sha256.Sum256(b)
|
||||||
|
return s.prefix + hex.EncodeToString(sum[:]), nil
|
||||||
|
}
|
||||||
|
func (s *Sessions) Authenticate(ctx context.Context, token string) (Principal, error) {
|
||||||
|
key, err := s.key(token)
|
||||||
|
if err != nil {
|
||||||
|
return Principal{}, err
|
||||||
|
}
|
||||||
|
data, err := s.store.Get(ctx, key)
|
||||||
|
if errors.Is(err, cache.ErrMiss) {
|
||||||
|
return Principal{}, ErrInvalidToken
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return Principal{}, err
|
||||||
|
}
|
||||||
|
var p Principal
|
||||||
|
if len(data) > 16384 || json.Unmarshal(data, &p) != nil || p.UserID == "" {
|
||||||
|
return Principal{}, ErrInvalidToken
|
||||||
|
}
|
||||||
|
return p, nil
|
||||||
|
}
|
||||||
|
func (s *Sessions) Revoke(ctx context.Context, token string) error {
|
||||||
|
key, err := s.key(token)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return s.store.Delete(ctx, key)
|
||||||
|
}
|
||||||
@@ -1,16 +1,79 @@
|
|||||||
# go-omni CLI
|
# go-omni CLI
|
||||||
|
|
||||||
|
## API 契约驱动项目
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# 创建包含可运行问候接口的新项目
|
||||||
|
go run ./cmd/go-omni api new --module example.com/demo --framework-dir . ../demo
|
||||||
|
|
||||||
|
# 从已有定义生成对应目录、类型、路由和业务骨架
|
||||||
|
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
|
||||||
|
--dir ../userService --module example.com/userService --framework-dir .
|
||||||
|
|
||||||
|
# 校验定义;也接受 goctl 风格的 api go -api ... -dir ...
|
||||||
|
go run ./cmd/go-omni api validate --api examples/contracts/user.api
|
||||||
|
```
|
||||||
|
|
||||||
|
`new --api <file>` 是契约项目的另一个入口。可以组合 `--database mysql --storage local --log-output file`,生成配置、依赖初始化与资源关闭。API 默认无数据库、无对象存储;`--database memory` 等同于不打开连接。`--force` 不允许覆盖 API 项目。初始配置选择写入 manifest,重新生成自动沿用。
|
||||||
|
|
||||||
|
后续在生成项目内执行 `go-omni api generate --api api/user.api --dir .`。
|
||||||
|
`--check` 只读检查生成产物是否过期。生成器记录所有权,保留 Service / Repository / 配置 / main 的手写内容,遇到生成文件被手动修改则报冲突。新 Service 与中间件占位实现返回 501。
|
||||||
|
|
||||||
|
支持范围、参数语义、目录及安全更新策略见 [API 开发指南](../../API.md)。本版借鉴 go-zero 的工作流,不宣称兼容全部 goctl 语法;仍通过 Omni/httpx 保留网络层替换能力。
|
||||||
|
|
||||||
|
## 日志与对象存储按需装配
|
||||||
|
|
||||||
|
新增 `--log-output stdout|stderr|file`、`--log-format json|text`、`--storage none|local|s3|r2|oss`。
|
||||||
|
指定任意一个新选项后,生成严格的配置文件、初始化代码和项目自身的测试;不指定则保留原有轻量骨架。
|
||||||
|
对象存储独立于 `--database memory|mysql`。只有选择文件存储才会生成文件 Handler / Service 和对应适配器。
|
||||||
|
|
||||||
|
开发阶段可以使用本地源码;固定远程版本的安装和验收见 [发布说明](../../RELEASE.md):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# 在 go-omni 仓库中运行;目标目录须为空。
|
||||||
|
go run ./cmd/go-omni new --framework-dir . \
|
||||||
|
--module example.com/fileService --log-output file --log-format json \
|
||||||
|
--storage local ../fileService
|
||||||
|
```
|
||||||
|
|
||||||
|
生成项目包含 `etc/config.yaml`、`internal/settings`、`internal/servicecontext/storage.go`、
|
||||||
|
文件 Service/Handler、测试和 `MODULES.md`。按生成的 MODULES.md 配置示例 Token 和本地签名密钥后,
|
||||||
|
`go mod tidy && go run ./cmd/server` 即可体验。云存储分别选择 `--storage s3`、`r2`、`oss`,
|
||||||
|
再设置已有桶/凭据等环境变量;不自动创建桶或修改云资源。
|
||||||
|
|
||||||
|
配置优先级为默认值 → YAML/JSON → 环境变量;密钥仅允许环境变量。Service 依赖统一存储接口,
|
||||||
|
组合处负责创建客户端与有界连接池,退出时关闭自己创建的资源。文件接口提供服务端上传、直传签名、
|
||||||
|
临时下载链接及统一错误处理;日志示例携带 requestId,文件输出默认带轮转。没有固定业务身份上下文。
|
||||||
|
|
||||||
|
新模板要求 `v0.7.0-preview` 或更新版本;明确指定旧版本并启用新模块会被拒绝,
|
||||||
|
避免模板引用旧框架不存在的接口。本地 `--framework-dir` 验证不受此限制。`--force` 可重新生成兼容配置,
|
||||||
|
但如果切换配置会遗留引用已移除依赖的旧文件,则拒绝操作,要求使用新目录;不会自动删除修改过的代码。
|
||||||
|
|
||||||
|
验证当前生成器:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./cmd/go-omni/...
|
||||||
|
python3 scripts/verifyScaffoldModules.py
|
||||||
|
```
|
||||||
|
|
||||||
|
前者覆盖两种数据库与五种文件存储选择、日志方式/格式,以及生成项目的配置、资源关闭和接口测试。
|
||||||
|
后者实际编译启动服务,验证日志、上传/下载/直传链接、错误状态、nethttp/native 下的 SIGINT/SIGTERM 和监听端口释放。
|
||||||
|
云端选项的测试使用本地替身,不代表 OSS/R2 实网验收;本批不访问业务服务器。报告见
|
||||||
|
[脚手架模块验收](../../performance/scaffoldModulesLocal.json)。
|
||||||
|
|
||||||
|
## 其他命令与远程安装
|
||||||
|
|
||||||
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
||||||
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
||||||
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
||||||
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
||||||
该命令尚未包含在下方历史标签中;当前源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
该命令从 `v0.7.0-preview` 提供;源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
||||||
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
||||||
|
|
||||||
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||||
go-omni new --module example.com/billing ./billing
|
go-omni new --module example.com/billing ./billing
|
||||||
cd billing
|
cd billing
|
||||||
go mod tidy
|
go mod tidy
|
||||||
@@ -72,3 +135,20 @@ python3 scripts/verifyDatabase.py --cli /tmp/go-omni --framework-dir .
|
|||||||
```
|
```
|
||||||
|
|
||||||
数据库验收需要本地 Docker,使用独立 MySQL 8.4 容器;不会连接已有业务数据库,结束后清理测试容器与数据卷。
|
数据库验收需要本地 Docker,使用独立 MySQL 8.4 容器;不会连接已有业务数据库,结束后清理测试容器与数据卷。
|
||||||
|
|
||||||
|
## 显式迁移(main)
|
||||||
|
|
||||||
|
SQL 命名为 `0001_createUsers.up.sql`。先预览版本和 SHA-256 清单:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni migrate --dir migrations
|
||||||
|
```
|
||||||
|
|
||||||
|
部署步骤配置 `GO_OMNI_DB_DSN`,显式执行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni migrate --dir migrations --apply
|
||||||
|
```
|
||||||
|
|
||||||
|
预览不连接数据库;应用启动不自动迁移。失败保留 dirty 状态,不提供自动 down/force。
|
||||||
|
脚手架原有 `schema.sql` 不会自动变成有版本的迁移,已有库需要先确定基线。详见 [扩展模块](../../ADVANCED.md)。
|
||||||
|
|||||||
127
cmd/go-omni/api.go
Normal file
127
cmd/go-omni/api.go
Normal file
@@ -0,0 +1,127 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||||
|
)
|
||||||
|
|
||||||
|
func runAPI(args []string) error {
|
||||||
|
if len(args) == 0 || args[0] == "help" || args[0] == "--help" || args[0] == "-h" {
|
||||||
|
usage()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
command := args[0]
|
||||||
|
if command == "go" {
|
||||||
|
command = "generate"
|
||||||
|
}
|
||||||
|
if command == "check" {
|
||||||
|
command = "validate"
|
||||||
|
}
|
||||||
|
if command != "new" && command != "generate" && command != "validate" {
|
||||||
|
return fmt.Errorf("unknown api command %q", command)
|
||||||
|
}
|
||||||
|
o := apigen.Options{}
|
||||||
|
for i := 1; i < len(args); i++ {
|
||||||
|
arg := args[i]
|
||||||
|
if arg == "--help" || arg == "-h" {
|
||||||
|
usage()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if arg == "--check" {
|
||||||
|
if command != "generate" {
|
||||||
|
return errors.New("--check belongs to api generate")
|
||||||
|
}
|
||||||
|
o.Check = true
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(arg, "-") {
|
||||||
|
if command == "new" && o.Dir == "" {
|
||||||
|
o.Dir = arg
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
return fmt.Errorf("unexpected argument %q", arg)
|
||||||
|
}
|
||||||
|
name, value, inline := strings.Cut(arg, "=")
|
||||||
|
// goctl-style -api/-dir aliases ease migration; long forms are canonical.
|
||||||
|
if name == "-api" {
|
||||||
|
name = "--api"
|
||||||
|
}
|
||||||
|
if name == "-dir" {
|
||||||
|
name = "--dir"
|
||||||
|
}
|
||||||
|
switch name {
|
||||||
|
case "--api", "--dir", "--module", "--framework-dir", "--framework-version", "--log-output", "--log-format", "--database", "--storage":
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("unknown api option %q", name)
|
||||||
|
}
|
||||||
|
if !inline {
|
||||||
|
if i+1 >= len(args) || strings.HasPrefix(args[i+1], "-") {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
i++
|
||||||
|
value = args[i]
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(value) == "" {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
if command == "validate" && name != "--api" {
|
||||||
|
return fmt.Errorf("api validate does not accept %s", name)
|
||||||
|
}
|
||||||
|
switch name {
|
||||||
|
case "--api":
|
||||||
|
o.API = value
|
||||||
|
case "--dir":
|
||||||
|
o.Dir = value
|
||||||
|
case "--module":
|
||||||
|
o.Module = value
|
||||||
|
case "--framework-dir":
|
||||||
|
o.FrameworkDir = value
|
||||||
|
case "--framework-version":
|
||||||
|
o.FrameworkVersion = value
|
||||||
|
case "--database":
|
||||||
|
o.Database = value
|
||||||
|
case "--storage":
|
||||||
|
o.Storage = value
|
||||||
|
case "--log-output":
|
||||||
|
o.LogOutput = value
|
||||||
|
case "--log-format":
|
||||||
|
o.LogFormat = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if command == "validate" {
|
||||||
|
if o.API == "" {
|
||||||
|
return errors.New("--api is required")
|
||||||
|
}
|
||||||
|
spec, err := apigen.Parse(o.API)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if _, err = apigen.Compile(spec); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Printf("valid API: %d types, %d routes\n", len(spec.Types), len(spec.Routes))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var out apigen.Output
|
||||||
|
var err error
|
||||||
|
if command == "new" {
|
||||||
|
out, err = apigen.New(o)
|
||||||
|
} else {
|
||||||
|
if o.Dir == "" {
|
||||||
|
o.Dir = "."
|
||||||
|
}
|
||||||
|
out, err = apigen.Generate(o)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if o.Check {
|
||||||
|
fmt.Println("API generated files are current")
|
||||||
|
} else {
|
||||||
|
fmt.Printf("API project ready: %d generated, %d created, %d preserved, %d removed\n", len(out.Generated), len(out.Created), len(out.Preserved), len(out.Removed))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
44
cmd/go-omni/api_test.go
Normal file
44
cmd/go-omni/api_test.go
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAPICommands(t *testing.T) {
|
||||||
|
root, err := filepath.Abs("../..")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
project := filepath.Join(t.TempDir(), "demo")
|
||||||
|
if err = runAPI([]string{"new", project, "--module=example.com/demo", "--framework-dir", root}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
api := filepath.Join(project, "api/service.api")
|
||||||
|
if err = runAPI([]string{"validate", "-api", api}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = runAPI([]string{"go", "-api", api, "-dir", project, "--check"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = runNew([]string{"--force", project}); err == nil {
|
||||||
|
t.Fatal("CRUD starter overwrote API project")
|
||||||
|
}
|
||||||
|
target := filepath.Join(t.TempDir(), "other")
|
||||||
|
if err = runNew([]string{"--api", api, "--module", "example.com/other", "--framework-dir", root, target}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = os.Stat(filepath.Join(target, "internal/service/greet/greet.go")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, args := range [][]string{{"validate"}, {"generate", "--api"}, {"new", "--api=x", "somewhere"}, {"generate", "--dir="}, {"generate", "--force"}, {"validate", "--api", api, "--dir", project}, {"validate", "--check"}} {
|
||||||
|
if err = runAPI(args); err == nil {
|
||||||
|
t.Fatalf("invalid args accepted: %v", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err = runNew([]string{"--api", api, "--force", target}); err == nil || !strings.Contains(err.Error(), "does not accept --force") {
|
||||||
|
t.Fatalf("incompatible profile: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
360
cmd/go-omni/apigen/apigen_test.go
Normal file
360
cmd/go-omni/apigen/apigen_test.go
Normal file
@@ -0,0 +1,360 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func frameworkRoot(t *testing.T) string {
|
||||||
|
t.Helper()
|
||||||
|
p, err := filepath.Abs("../../..")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return p
|
||||||
|
}
|
||||||
|
func writeFile(t *testing.T, name, body string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.MkdirAll(filepath.Dir(name), 0755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(name, []byte(body), 0644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func readFile(t *testing.T, name string) string {
|
||||||
|
t.Helper()
|
||||||
|
b, err := os.ReadFile(name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return string(b)
|
||||||
|
}
|
||||||
|
func parseText(t *testing.T, source string) (*Spec, error) {
|
||||||
|
t.Helper()
|
||||||
|
file := filepath.Join(t.TempDir(), "service.api")
|
||||||
|
writeFile(t, file, source)
|
||||||
|
return Parse(file)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCompileExample(t *testing.T) {
|
||||||
|
s, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/user.api"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
c, err := Compile(s)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(c.Groups) != 2 || len(s.Routes) != 3 || len(s.Sources) != 2 {
|
||||||
|
t.Fatalf("wrong spec: %+v", s)
|
||||||
|
}
|
||||||
|
var doc map[string]any
|
||||||
|
if err = json.Unmarshal(c.Document, &doc); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
route := doc["paths"].(map[string]any)["/api/v1/users/{userId}"].(map[string]any)["put"].(map[string]any)
|
||||||
|
if len(route["parameters"].([]any)) != 3 {
|
||||||
|
t.Fatal("missing mixed parameters")
|
||||||
|
}
|
||||||
|
schema := route["requestBody"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
|
||||||
|
if schema["additionalProperties"] != false || len(schema["required"].([]any)) != 2 {
|
||||||
|
t.Fatalf("wrong body schema: %+v", schema)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(c.Document), `"user_id"`) {
|
||||||
|
t.Fatal("snake case wire name")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestInvalidDefinitions(t *testing.T) {
|
||||||
|
cases := []string{
|
||||||
|
`syntax="v2"`,
|
||||||
|
`@server ( jwt: Auth ) service user { @handler go get /go }`,
|
||||||
|
"type Request { Name string `json:\"name\" binding:\"typo\"` }\nservice user { @handler run\n post /go (Request) }",
|
||||||
|
"type A { Other *A `json:\"other,optional\"` }\nservice user { @handler run\n post /go (A) }",
|
||||||
|
"type A { Other Missing `json:\"other\"` }\nservice user { @handler run\n post /go (A) }",
|
||||||
|
"type A { UserID int `json:\"user_id\"` }\nservice user { @handler run\n post /go (A) }",
|
||||||
|
"service user { @handler go\n get /go\n @handler other\n get /go\n}",
|
||||||
|
"service user { @handler go\n get /:id\n}",
|
||||||
|
"service user { @handler service\n get /go\n}",
|
||||||
|
"type A { ID int `path:\"id\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||||
|
"type A { Name string `json:\"name\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||||
|
"service user { @handler go\n @status 201\n post /go\n}",
|
||||||
|
"type A { Name string `header:\"Name\"` }\nservice user { @handler go\n get /go returns (A)\n}",
|
||||||
|
"@server (group: ../../bad)\nservice user { @handler go\n get /go\n}",
|
||||||
|
"service user { @handler go\n get /health/ready\n}",
|
||||||
|
"type A { Name string `json:\"name\"`\n Name int `json:\"other\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||||
|
"service user { @handler go\n get /go\n @handler Go\n get /other\n}",
|
||||||
|
"type A { Value []byte `json:\"value\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||||
|
}
|
||||||
|
for i, source := range cases {
|
||||||
|
s, err := parseText(t, source)
|
||||||
|
if err == nil {
|
||||||
|
_, err = Compile(s)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
t.Fatalf("case %d accepted: %s", i, source)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestImports(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
entry := filepath.Join(root, "api", "entry.api")
|
||||||
|
writeFile(t, entry, "import \"types.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||||
|
writeFile(t, filepath.Join(root, "api/types.api"), `import "entry.api"`)
|
||||||
|
if _, err := Parse(entry); err == nil || !strings.Contains(err.Error(), "circular") {
|
||||||
|
t.Fatalf("cycle: %v", err)
|
||||||
|
}
|
||||||
|
writeFile(t, entry, "import \"../outside.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||||
|
writeFile(t, filepath.Join(root, "outside.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||||
|
if _, err := Parse(entry); err == nil {
|
||||||
|
t.Fatal("escaped import root")
|
||||||
|
}
|
||||||
|
writeFile(t, entry, "import (\n \"shared.api\"\n \"./shared.api\"\n)\nservice user { @handler ping\n get /ping\n}")
|
||||||
|
writeFile(t, filepath.Join(root, "api/shared.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||||
|
s, err := Parse(entry)
|
||||||
|
if err != nil || len(s.Types) != 1 {
|
||||||
|
t.Fatalf("duplicate import: %+v %v", s, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSharedPrefixConflict(t *testing.T) {
|
||||||
|
source := "type A { ID int `path:\"id\"` }\ntype B { Name string `path:\"name\"` }\nservice user {\n @handler first\n get /users/:id (A)\n @handler second\n get /users/:name/items (B)\n}"
|
||||||
|
s, err := parseText(t, source)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = Compile(s); err == nil {
|
||||||
|
t.Fatal("shared prefix conflict was accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func FuzzParser(f *testing.F) {
|
||||||
|
for _, s := range []string{starterAPI, "syntax = \"v1\"", "type (", "@server (prefix: /api)"} {
|
||||||
|
f.Add(s)
|
||||||
|
}
|
||||||
|
f.Fuzz(func(t *testing.T, source string) {
|
||||||
|
if len(source) > 16384 {
|
||||||
|
t.Skip()
|
||||||
|
}
|
||||||
|
tokens, err := lex("fuzz.api", source)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
spec := &Spec{Title: "Fuzz", Version: "1"}
|
||||||
|
p := parser{tokens: tokens, spec: spec}
|
||||||
|
if p.parse() == nil {
|
||||||
|
_, _ = Compile(spec)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
func TestRegenerationAndConflicts(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "app")
|
||||||
|
if _, err := New(Options{Dir: dir, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
manifestBefore := readFile(t, filepath.Join(dir, manifestPath))
|
||||||
|
typesBefore := readFile(t, filepath.Join(dir, "internal/types/typesGen.go"))
|
||||||
|
o.Check = true
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
servicePath := filepath.Join(dir, "internal/service/greet/greet.go")
|
||||||
|
service := readFile(t, servicePath) + "\n// user edit must survive\n"
|
||||||
|
writeFile(t, servicePath, service)
|
||||||
|
for _, file := range []string{"cmd/server/main.go", "etc/config.yaml", "internal/servicecontext/context.go", "internal/repository/greet/repository.go"} {
|
||||||
|
body := readFile(t, filepath.Join(dir, file))
|
||||||
|
if strings.HasSuffix(file, ".go") {
|
||||||
|
body += "\n// handwritten extension\n"
|
||||||
|
} else {
|
||||||
|
body += "\n# handwritten extension\n"
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, file), body)
|
||||||
|
}
|
||||||
|
api := readFile(t, o.API)
|
||||||
|
api += "\n@server (group: extra)\nservice greet-api {\n @handler ping\n get /ping\n}\n"
|
||||||
|
writeFile(t, o.API, api)
|
||||||
|
if _, err := Generate(o); err == nil {
|
||||||
|
t.Fatal("check accepted stale output")
|
||||||
|
}
|
||||||
|
if readFile(t, filepath.Join(dir, manifestPath)) != manifestBefore {
|
||||||
|
t.Fatal("--check wrote manifest")
|
||||||
|
}
|
||||||
|
o.Check = false
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if readFile(t, servicePath) != service {
|
||||||
|
t.Fatal("business code overwritten")
|
||||||
|
}
|
||||||
|
// Retiring a group removes only owned handlers, keeping its business files.
|
||||||
|
writeFile(t, o.API, strings.Split(api, "\n@server (group: extra)")[0])
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "internal/handler/extra/handlersGen.go")); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("stale handler remains")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(dir, "internal/service/extra/ping.go")); err != nil {
|
||||||
|
t.Fatal("business stub removed")
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), typesBefore+"\n// edited generated file\n")
|
||||||
|
before := snapshot(t, dir)
|
||||||
|
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "was edited") {
|
||||||
|
t.Fatalf("did not protect edited output: %v", err)
|
||||||
|
}
|
||||||
|
after := snapshot(t, dir)
|
||||||
|
if !sameFiles(before, after) {
|
||||||
|
t.Fatal("failed regeneration changed files")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestOutputBoundaries(t *testing.T) {
|
||||||
|
for _, kind := range []string{"symlink", "unowned", "lock", "caseCollision"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
root := t.TempDir()
|
||||||
|
dir := filepath.Join(root, "app")
|
||||||
|
api := filepath.Join(root, "input/service.api")
|
||||||
|
writeFile(t, api, starterAPI)
|
||||||
|
switch kind {
|
||||||
|
case "symlink":
|
||||||
|
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
outside := filepath.Join(root, "outside")
|
||||||
|
os.MkdirAll(outside, 0755)
|
||||||
|
if err := os.Symlink(outside, filepath.Join(dir, "internal")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "unowned":
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), "package types // mine\n")
|
||||||
|
case "lock":
|
||||||
|
writeFile(t, filepath.Join(dir, ".goOmni/api.lock"), "someone else")
|
||||||
|
case "caseCollision":
|
||||||
|
writeFile(t, api, starterAPI+"\n@server (group: gReet)\nservice greet-api {\n @handler extra\n get /extra\n}\n")
|
||||||
|
}
|
||||||
|
before := snapshot(t, root)
|
||||||
|
if _, err := Generate(Options{Dir: dir, API: api, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err == nil {
|
||||||
|
t.Fatal("unsafe output accepted")
|
||||||
|
}
|
||||||
|
if !sameFiles(before, snapshot(t, root)) {
|
||||||
|
t.Fatal("conflict changed files")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func snapshot(t *testing.T, root string) map[string]string {
|
||||||
|
t.Helper()
|
||||||
|
out := map[string]string{}
|
||||||
|
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if d.Type().IsRegular() {
|
||||||
|
data, e := os.ReadFile(path)
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
rel, _ := filepath.Rel(root, path)
|
||||||
|
out[rel] = string(data)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
func sameFiles(a, b map[string]string) bool {
|
||||||
|
if len(a) != len(b) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for k, v := range a {
|
||||||
|
if b[k] != v {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestGeneratedProjectsBuildAndRun(t *testing.T) {
|
||||||
|
if testing.Short() {
|
||||||
|
t.Skip("consumer module integration")
|
||||||
|
}
|
||||||
|
root := frameworkRoot(t)
|
||||||
|
for _, starter := range []bool{true, false} {
|
||||||
|
name := "contract"
|
||||||
|
if starter {
|
||||||
|
name = "starter"
|
||||||
|
}
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "project")
|
||||||
|
o := Options{Dir: dir, Module: "example.com/consumer", FrameworkDir: root, LogOutput: "file", LogFormat: "text"}
|
||||||
|
var err error
|
||||||
|
if starter {
|
||||||
|
_, err = New(o)
|
||||||
|
} else {
|
||||||
|
o.API = filepath.Join(root, "examples/contracts/user.api")
|
||||||
|
_, err = Generate(o)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
source := consumerContractTest
|
||||||
|
if starter {
|
||||||
|
source = consumerStarterTest
|
||||||
|
} else {
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/service/user/updateUser.go"), `package user
|
||||||
|
import("context";"example.com/consumer/internal/types")
|
||||||
|
func(s *Service) UpdateUser(ctx context.Context,req *types.UpdateUserReq)(*types.UserResp,error){return &types.UserResp{UserID:req.UserID,Name:req.Name,Active:req.Active},nil}
|
||||||
|
`)
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, "contract_test.go"), source)
|
||||||
|
o.API = filepath.Join(dir, "api/service.api")
|
||||||
|
if !starter {
|
||||||
|
o.API = filepath.Join(dir, "api/user.api")
|
||||||
|
}
|
||||||
|
if _, err = Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||||
|
cmd.Dir = dir
|
||||||
|
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("consumer: %v\n%s", err, output)
|
||||||
|
}
|
||||||
|
t.Log(string(output))
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const consumerStarterTest = `package consumer_test
|
||||||
|
import("context";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext";"example.com/consumer/internal/settings")
|
||||||
|
func TestGreeting(t *testing.T){cfg,err:=settings.Load("etc/config.yaml");if err!=nil{t.Fatal(err)};cfg.Server.Log.Writer=io.Discard;cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{};logger,err:=logx.New(cfg.Server.Log);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)};w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,httptest.NewRequest("GET","/api/greet/Alice",nil));if w.Code!=200||!strings.Contains(w.Body.String(),"Hello Alice")||w.Header().Get("X-Request-ID")==""{t.Fatalf("%d %s",w.Code,w.Body.String())}}
|
||||||
|
`
|
||||||
|
const consumerContractTest = `package consumer_test
|
||||||
|
import("context";"encoding/json";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext")
|
||||||
|
func TestContract(t *testing.T){cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
|
||||||
|
for _,tc:=range []struct{method,path,body string;status int}{
|
||||||
|
{"PUT","/api/v1/users/7?notify=true",` + "`" + `{"name":"Alice","active":false,"age":0}` + "`" + `,200},
|
||||||
|
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice"}` + "`" + `,400},
|
||||||
|
{"PUT","/api/v1/users/7?notify=true¬ify=false",` + "`" + `{"name":"Alice","active":false}` + "`" + `,400},
|
||||||
|
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice","active":false,"age":151}` + "`" + `,400},
|
||||||
|
{"GET","/api/v1/users?page=0","",400},
|
||||||
|
{"GET","/api/v1/users?page=1","",501},
|
||||||
|
{"DELETE","/admin/v1/users/7","",501},
|
||||||
|
}{req:=httptest.NewRequest(tc.method,tc.path,strings.NewReader(tc.body));req.Header.Set("Content-Type","application/json");w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,req);if w.Code!=tc.status{t.Fatalf("%s: %d %s",tc.path,w.Code,w.Body.String())};if tc.status==200{var v struct{Data struct{UserID int64 ` + "`json:\"userId\"`" + `;Name string ` + "`json:\"name\"`" + `}};if err:=json.Unmarshal(w.Body.Bytes(),&v);err!=nil||v.Data.UserID!=7||v.Data.Name!="Alice"{t.Fatal(w.Body.String())}}}
|
||||||
|
delete(deps.Middlewares,"Auth");if err:=handler.RegisterRoutes(omni.New(),deps);err==nil{t.Fatal("missing auth hook accepted")}
|
||||||
|
}
|
||||||
|
`
|
||||||
332
cmd/go-omni/apigen/compile.go
Normal file
332
cmd/go-omni/apigen/compile.go
Normal file
@@ -0,0 +1,332 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"go/ast"
|
||||||
|
goparser "go/parser"
|
||||||
|
gotoken "go/token"
|
||||||
|
"reflect"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Compiled struct {
|
||||||
|
Spec *Spec
|
||||||
|
Bindings map[string]*contract.Binder
|
||||||
|
Document []byte
|
||||||
|
Groups []string
|
||||||
|
}
|
||||||
|
|
||||||
|
var exportedName = regexp.MustCompile(`^[A-Z][A-Za-z0-9]*$`)
|
||||||
|
var goName = regexp.MustCompile(`^[a-zA-Z][A-Za-z0-9]*$`)
|
||||||
|
var lowerName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||||
|
var serviceName = regexp.MustCompile(`^[a-zA-Z][a-zA-Z0-9-]*$`)
|
||||||
|
var staticSegment = regexp.MustCompile(`^[A-Za-z0-9._~-]+$`)
|
||||||
|
|
||||||
|
func Compile(spec *Spec) (*Compiled, error) {
|
||||||
|
if spec == nil || len(spec.Routes) == 0 || !serviceName.MatchString(spec.Service) {
|
||||||
|
return nil, errors.New("api: valid service and routes required")
|
||||||
|
}
|
||||||
|
definitions := map[string]Type{}
|
||||||
|
for _, t := range spec.Types {
|
||||||
|
if !exportedName.MatchString(t.Name) {
|
||||||
|
return nil, fmt.Errorf("%s: type names must be UpperCamelCase", t.Position)
|
||||||
|
}
|
||||||
|
if _, ok := definitions[t.Name]; ok {
|
||||||
|
return nil, fmt.Errorf("%s: duplicate type %s", t.Position, t.Name)
|
||||||
|
}
|
||||||
|
definitions[t.Name] = t
|
||||||
|
}
|
||||||
|
resolved := map[string]reflect.Type{}
|
||||||
|
resolving := map[string]bool{}
|
||||||
|
var resolve func(string, int) (reflect.Type, error)
|
||||||
|
var expression func(ast.Expr, int) (reflect.Type, error)
|
||||||
|
primitives := map[string]reflect.Type{"string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||||
|
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
|
||||||
|
if depth > 32 {
|
||||||
|
return nil, errors.New("type nesting exceeds 32")
|
||||||
|
}
|
||||||
|
switch e := e.(type) {
|
||||||
|
case *ast.Ident:
|
||||||
|
if t, ok := primitives[e.Name]; ok {
|
||||||
|
return t, nil
|
||||||
|
}
|
||||||
|
return resolve(e.Name, depth+1)
|
||||||
|
case *ast.StarExpr:
|
||||||
|
if _, double := e.X.(*ast.StarExpr); double {
|
||||||
|
return nil, errors.New("multiple pointer levels unsupported")
|
||||||
|
}
|
||||||
|
t, err := expression(e.X, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return reflect.PointerTo(t), nil
|
||||||
|
case *ast.ArrayType:
|
||||||
|
if e.Len != nil {
|
||||||
|
return nil, errors.New("use slices, not fixed arrays")
|
||||||
|
}
|
||||||
|
t, err := expression(e.Elt, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return reflect.SliceOf(t), nil
|
||||||
|
case *ast.MapType:
|
||||||
|
key, ok := e.Key.(*ast.Ident)
|
||||||
|
if !ok || key.Name != "string" {
|
||||||
|
return nil, errors.New("map keys must be string")
|
||||||
|
}
|
||||||
|
t, err := expression(e.Value, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return reflect.MapOf(reflect.TypeOf(""), t), nil
|
||||||
|
default:
|
||||||
|
return nil, errors.New("unsupported type expression")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resolve = func(name string, depth int) (reflect.Type, error) {
|
||||||
|
if depth > 32 {
|
||||||
|
return nil, errors.New("type nesting exceeds 32")
|
||||||
|
}
|
||||||
|
if t, ok := resolved[name]; ok {
|
||||||
|
return t, nil
|
||||||
|
}
|
||||||
|
if resolving[name] {
|
||||||
|
return nil, fmt.Errorf("recursive type %s is unsupported", name)
|
||||||
|
}
|
||||||
|
typ, ok := definitions[name]
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("unknown type %s", name)
|
||||||
|
}
|
||||||
|
resolving[name] = true
|
||||||
|
defer delete(resolving, name)
|
||||||
|
var fields []reflect.StructField
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, f := range typ.Fields {
|
||||||
|
if !exportedName.MatchString(f.Name) || seen[f.Name] {
|
||||||
|
return nil, fmt.Errorf("%s: invalid or duplicate field %s", name, f.Name)
|
||||||
|
}
|
||||||
|
seen[f.Name] = true
|
||||||
|
expr, err := goparser.ParseExpr(f.Type)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
t, err := expression(expr, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err)
|
||||||
|
}
|
||||||
|
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)})
|
||||||
|
}
|
||||||
|
t := reflect.StructOf(fields)
|
||||||
|
resolved[name] = t
|
||||||
|
return t, nil
|
||||||
|
}
|
||||||
|
c := &Compiled{Spec: spec, Bindings: map[string]*contract.Binder{}}
|
||||||
|
for _, t := range spec.Types {
|
||||||
|
rt, err := resolve(t.Name, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
b, err := contract.ForType(rt)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", t.Position, err)
|
||||||
|
}
|
||||||
|
c.Bindings[t.Name] = b
|
||||||
|
}
|
||||||
|
api, err := openapi.New(spec.Title, spec.Version)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{}
|
||||||
|
middlewareSymbols := map[string]string{}
|
||||||
|
for _, r := range spec.Routes {
|
||||||
|
if !goName.MatchString(r.Handler) {
|
||||||
|
return nil, fmt.Errorf("%s: invalid handler name", r.Position)
|
||||||
|
}
|
||||||
|
if strings.EqualFold(r.Handler, "service") {
|
||||||
|
return nil, errors.New("handler name service conflicts with the service constructor file")
|
||||||
|
}
|
||||||
|
if r.Group == "" {
|
||||||
|
return nil, errors.New("group required")
|
||||||
|
}
|
||||||
|
for _, part := range strings.Split(r.Group, "/") {
|
||||||
|
if !lowerName.MatchString(part) || gotoken.Lookup(part).IsKeyword() {
|
||||||
|
return nil, fmt.Errorf("invalid lowerCamelCase group %q", r.Group)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
h := r.Group + "/" + strings.ToLower(r.Handler)
|
||||||
|
if handlers[h] {
|
||||||
|
return nil, fmt.Errorf("duplicate handler %s in group %s", r.Handler, r.Group)
|
||||||
|
}
|
||||||
|
handlers[h] = true
|
||||||
|
groups[r.Group] = true
|
||||||
|
mseen := map[string]bool{}
|
||||||
|
for _, m := range r.Middleware {
|
||||||
|
if !goName.MatchString(m) || mseen[m] {
|
||||||
|
return nil, fmt.Errorf("invalid or duplicate middleware %q", m)
|
||||||
|
}
|
||||||
|
mseen[m] = true
|
||||||
|
if strings.EqualFold(m, "defaults") || strings.EqualFold(m, "registryGen") {
|
||||||
|
return nil, fmt.Errorf("reserved middleware name %q", m)
|
||||||
|
}
|
||||||
|
if prior, ok := middlewareSymbols[strings.ToLower(m)]; ok && prior != m {
|
||||||
|
return nil, fmt.Errorf("middleware names %s and %s collide", prior, m)
|
||||||
|
}
|
||||||
|
middlewareSymbols[strings.ToLower(m)] = m
|
||||||
|
}
|
||||||
|
shape, err := routeShape(r.Path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||||
|
}
|
||||||
|
for _, prior := range spec.Routes {
|
||||||
|
if prior.Method != r.Method || prior.Path == r.Path {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if conflictPrefix(prior.Path, r.Path) {
|
||||||
|
return nil, fmt.Errorf("conflicting parameter names at a shared route prefix: %s and %s", prior.Path, r.Path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if prior, exists := shapes[shape]; exists && prior != r.Path {
|
||||||
|
return nil, fmt.Errorf("ambiguous route parameter names: %s and %s", prior, r.Path)
|
||||||
|
}
|
||||||
|
shapes[shape] = r.Path
|
||||||
|
pathKey := r.Method + " " + shape
|
||||||
|
if paths[pathKey] {
|
||||||
|
return nil, fmt.Errorf("duplicate route %s", pathKey)
|
||||||
|
}
|
||||||
|
paths[pathKey] = true
|
||||||
|
if r.Path == "/health/live" || r.Path == "/health/ready" || r.Path == "/openapi.json" {
|
||||||
|
return nil, fmt.Errorf("route %s is reserved by the generated server", r.Path)
|
||||||
|
}
|
||||||
|
if r.Method != strings.ToLower(r.Method) {
|
||||||
|
return nil, errors.New("HTTP methods must be lowercase")
|
||||||
|
}
|
||||||
|
if r.Status != 200 && r.Status != 201 && r.Status != 202 && r.Status != 204 {
|
||||||
|
return nil, errors.New("@status supports 200, 201, 202 or 204")
|
||||||
|
}
|
||||||
|
if (r.Status == 204) != (r.Response == "") {
|
||||||
|
return nil, errors.New("204 requires no response; other success statuses require a response type")
|
||||||
|
}
|
||||||
|
if r.Method == "head" && r.Response != "" {
|
||||||
|
return nil, errors.New("HEAD must not declare a response body")
|
||||||
|
}
|
||||||
|
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
|
||||||
|
if r.Request != "" {
|
||||||
|
b, ok := c.Bindings[r.Request]
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("unknown request %s", r.Request)
|
||||||
|
}
|
||||||
|
op.Parameters = b.Parameters()
|
||||||
|
op.Request = b.BodySchema()
|
||||||
|
if op.Request != nil && (r.Method == "get" || r.Method == "head") {
|
||||||
|
return nil, errors.New("GET/HEAD requests cannot declare JSON bodies; use form/path/header tags")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var response *openapi.Schema
|
||||||
|
if r.Response != "" {
|
||||||
|
b, ok := c.Bindings[r.Response]
|
||||||
|
if !ok {
|
||||||
|
return nil, fmt.Errorf("unknown response %s", r.Response)
|
||||||
|
}
|
||||||
|
s, err := b.ResponseSchema()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
response = openapi.Envelope(s)
|
||||||
|
response.Required = append(response.Required, "data")
|
||||||
|
}
|
||||||
|
op.Responses[r.Status] = openapi.Response{Description: "Success", Schema: response}
|
||||||
|
errorSchema := &openapi.Schema{Type: "object", Required: []string{"code", "message"}, Properties: map[string]*openapi.Schema{"code": {Type: "string"}, "message": {Type: "string"}, "requestId": {Type: "string"}}}
|
||||||
|
for _, code := range []int{400, 413, 415, 500, 501} {
|
||||||
|
op.Responses[code] = openapi.Response{Description: "Error", Schema: errorSchema}
|
||||||
|
}
|
||||||
|
if err = api.Handle(discardRoutes{}, op, func(*httpx.Context) error { return nil }); err != nil {
|
||||||
|
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for group := range groups {
|
||||||
|
c.Groups = append(c.Groups, group)
|
||||||
|
}
|
||||||
|
sort.Strings(c.Groups)
|
||||||
|
c.Document, err = api.JSON()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
// Show named hooks in documentation without claiming they implement a
|
||||||
|
// particular authentication scheme. Applications own that policy.
|
||||||
|
var doc map[string]any
|
||||||
|
decoder := json.NewDecoder(bytes.NewReader(c.Document))
|
||||||
|
decoder.UseNumber()
|
||||||
|
if err = decoder.Decode(&doc); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, r := range spec.Routes {
|
||||||
|
if len(r.Middleware) > 0 {
|
||||||
|
path := r.Path
|
||||||
|
for _, part := range strings.Split(r.Path, "/") {
|
||||||
|
if strings.HasPrefix(part, ":") {
|
||||||
|
path = strings.ReplaceAll(path, part, "{"+part[1:]+"}")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
doc["paths"].(map[string]any)[path].(map[string]any)[r.Method].(map[string]any)["x-omni-middleware"] = r.Middleware
|
||||||
|
}
|
||||||
|
}
|
||||||
|
c.Document, err = json.MarshalIndent(doc, "", " ")
|
||||||
|
return c, err
|
||||||
|
}
|
||||||
|
|
||||||
|
type discardRoutes struct{}
|
||||||
|
|
||||||
|
func (discardRoutes) Handle(string, string, httpx.Handler, ...httpx.Middleware) {}
|
||||||
|
func operationID(r Route) string {
|
||||||
|
return strings.ReplaceAll(r.Group, "/", "_") + "_" + upper(r.Handler)
|
||||||
|
}
|
||||||
|
func upper(s string) string { return strings.ToUpper(s[:1]) + s[1:] }
|
||||||
|
func lower(s string) string { return strings.ToLower(s[:1]) + s[1:] }
|
||||||
|
func routeShape(path string) (string, error) {
|
||||||
|
if path == "/" {
|
||||||
|
return path, nil
|
||||||
|
}
|
||||||
|
if !strings.HasPrefix(path, "/") {
|
||||||
|
return "", errors.New("route must start with /")
|
||||||
|
}
|
||||||
|
parts := strings.Split(path[1:], "/")
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for i, part := range parts {
|
||||||
|
if strings.HasPrefix(part, ":") {
|
||||||
|
name := part[1:]
|
||||||
|
if !lowerName.MatchString(name) || seen[name] {
|
||||||
|
return "", errors.New("invalid/duplicate path parameter")
|
||||||
|
}
|
||||||
|
seen[name] = true
|
||||||
|
parts[i] = ":"
|
||||||
|
} else if !staticSegment.MatchString(part) || part == "." || part == ".." {
|
||||||
|
return "", errors.New("invalid route segment")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "/" + strings.Join(parts, "/"), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func conflictPrefix(left, right string) bool {
|
||||||
|
a, b := strings.Split(left, "/"), strings.Split(right, "/")
|
||||||
|
for i := 0; i < len(a) && i < len(b); i++ {
|
||||||
|
if strings.HasPrefix(a[i], ":") && strings.HasPrefix(b[i], ":") {
|
||||||
|
if a[i] != b[i] {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if a[i] != b[i] {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
458
cmd/go-omni/apigen/generate.go
Normal file
458
cmd/go-omni/apigen/generate.go
Normal file
@@ -0,0 +1,458 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"go/format"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"runtime/debug"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||||
|
"golang.org/x/mod/modfile"
|
||||||
|
"golang.org/x/mod/module"
|
||||||
|
"golang.org/x/mod/semver"
|
||||||
|
)
|
||||||
|
|
||||||
|
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
const minimumVersion = "v0.7.0-preview"
|
||||||
|
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||||
|
|
||||||
|
type Options struct {
|
||||||
|
Database, Storage string
|
||||||
|
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
|
||||||
|
Check bool
|
||||||
|
}
|
||||||
|
type Output struct{ Generated, Created, Preserved, Removed []string }
|
||||||
|
|
||||||
|
// Generate creates or safely updates one contract-driven project. It renders
|
||||||
|
// and validates every output before applying an ownership-checked transaction.
|
||||||
|
func Generate(o Options) (Output, error) {
|
||||||
|
if o.API == "" || o.Dir == "" {
|
||||||
|
return Output{}, errors.New("--api and --dir are required")
|
||||||
|
}
|
||||||
|
spec, err := Parse(o.API)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
compiled, err := Compile(spec)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
return generate(o, compiled, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
const starterAPI = "syntax = \"v1\"\n\ninfo (\n title: \"Greeting API\"\n version: \"1.0.0\"\n)\n\ntype GreetReq {\n Name string `path:\"name\" binding:\"required,max=80\"`\n}\ntype GreetResp {\n Message string `json:\"message\"`\n}\n\n@server (\n group: greet\n prefix: /api\n)\nservice greet-api {\n @doc \"Greet a visitor\"\n @handler greet\n get /greet/:name (GreetReq) returns (GreetResp)\n}\n"
|
||||||
|
|
||||||
|
// New creates a runnable greeting project, including its editable API file.
|
||||||
|
func New(o Options) (Output, error) {
|
||||||
|
if o.API != "" || o.Check {
|
||||||
|
return Output{}, errors.New("api new does not accept --api or --check")
|
||||||
|
}
|
||||||
|
if o.Dir == "" {
|
||||||
|
return Output{}, errors.New("project directory required")
|
||||||
|
}
|
||||||
|
if entries, err := os.ReadDir(o.Dir); err == nil && len(entries) > 0 {
|
||||||
|
return Output{}, errors.New("api new requires an empty directory")
|
||||||
|
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
tokens, err := lex("service.api", starterAPI)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
spec := &Spec{Version: "1.0.0", Entry: "service.api", Sources: map[string][]byte{"service.api": []byte(starterAPI)}}
|
||||||
|
p := parser{tokens: tokens, spec: spec}
|
||||||
|
if err = p.parse(); err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
c, err := Compile(spec)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
return generate(o, c, true)
|
||||||
|
}
|
||||||
|
|
||||||
|
func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||||
|
directory, err := filepath.Abs(o.Dir)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
o.Dir = directory
|
||||||
|
if err := resolveProfile(&o); err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
seeds := map[string][]byte{}
|
||||||
|
moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod"))
|
||||||
|
if readErr == nil {
|
||||||
|
mf, err := modfile.Parse("go.mod", moduleData, nil)
|
||||||
|
if err != nil || mf.Module == nil {
|
||||||
|
return Output{}, errors.New("invalid project go.mod")
|
||||||
|
}
|
||||||
|
if o.Module != "" && o.Module != mf.Module.Mod.Path {
|
||||||
|
return Output{}, errors.New("--module differs from existing go.mod")
|
||||||
|
}
|
||||||
|
o.Module = mf.Module.Mod.Path
|
||||||
|
found := false
|
||||||
|
requiredVersion := ""
|
||||||
|
for _, r := range mf.Require {
|
||||||
|
if r.Mod.Path == frameworkModule {
|
||||||
|
found = true
|
||||||
|
requiredVersion = r.Mod.Version
|
||||||
|
if o.FrameworkVersion != "" && o.FrameworkVersion != r.Mod.Version {
|
||||||
|
return Output{}, errors.New("existing framework version differs; update go.mod explicitly")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
return Output{}, errors.New("existing go.mod must require go-omni")
|
||||||
|
}
|
||||||
|
if semver.Compare(requiredVersion, minimumVersion) < 0 {
|
||||||
|
localCurrent := false
|
||||||
|
for _, r := range mf.Replace {
|
||||||
|
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||||
|
path := r.New.Path
|
||||||
|
if !filepath.IsAbs(path) {
|
||||||
|
path = filepath.Join(directory, path)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(path, "contract", "contract.go")); err == nil {
|
||||||
|
localCurrent = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !localCurrent {
|
||||||
|
return Output{}, fmt.Errorf("existing framework dependency predates API support; use %s or a current local replacement", minimumVersion)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if o.FrameworkDir != "" {
|
||||||
|
matched := false
|
||||||
|
for _, r := range mf.Replace {
|
||||||
|
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||||
|
replacement := r.New.Path
|
||||||
|
if !filepath.IsAbs(replacement) {
|
||||||
|
replacement = filepath.Join(directory, replacement)
|
||||||
|
}
|
||||||
|
matched = canonical(replacement) == canonical(o.FrameworkDir)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !matched {
|
||||||
|
return Output{}, errors.New("existing framework replacement differs; update go.mod explicitly")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if !errors.Is(readErr, os.ErrNotExist) {
|
||||||
|
return Output{}, readErr
|
||||||
|
} else {
|
||||||
|
if o.Module == "" {
|
||||||
|
o.Module = filepath.Base(directory)
|
||||||
|
}
|
||||||
|
data, err := moduleFile(o)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
seeds["go.mod"] = data
|
||||||
|
}
|
||||||
|
if err = module.CheckImportPath(o.Module); err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
generated, base, err := renderProject(o, c, starter)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, err
|
||||||
|
}
|
||||||
|
for name, data := range base {
|
||||||
|
seeds[name] = data
|
||||||
|
}
|
||||||
|
// The API source remains developer-owned. A relocated project regenerates
|
||||||
|
// from its own api/ tree, including all imports.
|
||||||
|
entry := "api/" + c.Spec.Entry
|
||||||
|
for name, data := range c.Spec.Sources {
|
||||||
|
seeds["api/"+name] = data
|
||||||
|
}
|
||||||
|
if o.API != "" {
|
||||||
|
source, _ := filepath.Abs(o.API)
|
||||||
|
sourceReal, _ := filepath.EvalSymlinks(source)
|
||||||
|
targetReal := canonical(directory)
|
||||||
|
rel, e := filepath.Rel(targetReal, sourceReal)
|
||||||
|
if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
|
entry = filepath.ToSlash(rel)
|
||||||
|
for name := range c.Spec.Sources {
|
||||||
|
delete(seeds, "api/"+name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Source collisions must never leave a copied contract different from the
|
||||||
|
// generated code. On updates, use the developer-owned API in the project.
|
||||||
|
for name, data := range seeds {
|
||||||
|
if strings.HasSuffix(name, ".api") {
|
||||||
|
current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name)))
|
||||||
|
if e == nil && string(current) != string(data) {
|
||||||
|
return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
seeds["README.md"] = []byte(projectREADME(o.Module, entry) + infrastructureREADME(o))
|
||||||
|
for _, files := range []map[string][]byte{generated, seeds} {
|
||||||
|
for name, data := range files {
|
||||||
|
if strings.HasSuffix(name, ".go") {
|
||||||
|
data, err = format.Source(data)
|
||||||
|
if err != nil {
|
||||||
|
return Output{}, fmt.Errorf("format %s: %w", name, err)
|
||||||
|
}
|
||||||
|
files[name] = data
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return apply(directory, o.Module, entry, generated, seeds, o.Check, o.profile())
|
||||||
|
}
|
||||||
|
|
||||||
|
func moduleFile(o Options) ([]byte, error) {
|
||||||
|
if err := module.CheckImportPath(o.Module); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
root := o.FrameworkDir
|
||||||
|
version := o.FrameworkVersion
|
||||||
|
if root == "" && version == "" {
|
||||||
|
wd, _ := os.Getwd()
|
||||||
|
for wd != "" {
|
||||||
|
data, err := os.ReadFile(filepath.Join(wd, "go.mod"))
|
||||||
|
if err == nil {
|
||||||
|
mf, e := modfile.Parse("go.mod", data, nil)
|
||||||
|
if e == nil && mf.Module != nil && mf.Module.Mod.Path == frameworkModule {
|
||||||
|
root = wd
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
parent := filepath.Dir(wd)
|
||||||
|
if parent == wd {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
wd = parent
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if version == "" {
|
||||||
|
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
|
||||||
|
version = info.Main.Version
|
||||||
|
} else {
|
||||||
|
version = minimumVersion
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {
|
||||||
|
return nil, errors.New("invalid framework version")
|
||||||
|
}
|
||||||
|
if root == "" && semver.Compare(version, minimumVersion) < 0 {
|
||||||
|
return nil, fmt.Errorf("API generation needs %s or newer; use --framework-dir for unpublished source", minimumVersion)
|
||||||
|
}
|
||||||
|
text := "module " + o.Module + "\n\ngo 1.22.0\n\nrequire " + frameworkModule + " " + version + "\n"
|
||||||
|
if root != "" {
|
||||||
|
root = canonical(root)
|
||||||
|
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
mf, err := modfile.Parse("go.mod", data, nil)
|
||||||
|
if err != nil || mf.Module == nil || mf.Module.Mod.Path != frameworkModule {
|
||||||
|
return nil, errors.New("--framework-dir must contain go-omni")
|
||||||
|
}
|
||||||
|
if _, err = os.Stat(filepath.Join(root, "contract", "contract.go")); err != nil {
|
||||||
|
return nil, errors.New("local framework lacks API contract support")
|
||||||
|
}
|
||||||
|
relative, err := filepath.Rel(canonical(o.Dir), root)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
text += "\nreplace " + frameworkModule + " => " + strconv.Quote(filepath.ToSlash(relative)) + "\n"
|
||||||
|
}
|
||||||
|
return []byte(text), nil
|
||||||
|
}
|
||||||
|
func canonical(path string) string {
|
||||||
|
path, _ = filepath.Abs(path)
|
||||||
|
candidate := path
|
||||||
|
var tail []string
|
||||||
|
for {
|
||||||
|
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
|
||||||
|
for i := len(tail) - 1; i >= 0; i-- {
|
||||||
|
resolved = filepath.Join(resolved, tail[i])
|
||||||
|
}
|
||||||
|
return resolved
|
||||||
|
}
|
||||||
|
parent := filepath.Dir(candidate)
|
||||||
|
if parent == candidate {
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
tail = append(tail, filepath.Base(candidate))
|
||||||
|
candidate = parent
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map[string][]byte, error) {
|
||||||
|
generated, seeds := map[string][]byte{}, map[string][]byte{}
|
||||||
|
put := func(files map[string][]byte, name, content string) { files[name] = []byte(content) }
|
||||||
|
types := "package types\n\n"
|
||||||
|
for _, t := range c.Spec.Types {
|
||||||
|
types += "type " + t.Name + " struct {\n"
|
||||||
|
for _, f := range t.Fields {
|
||||||
|
types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag)
|
||||||
|
}
|
||||||
|
types += "}\n"
|
||||||
|
}
|
||||||
|
put(generated, "internal/types/typesGen.go", generatedHeader+types)
|
||||||
|
put(generated, "docs/openapi.json", string(c.Document)+"\n")
|
||||||
|
put(generated, "internal/handler/openapiGen.go", generatedHeader+fmt.Sprintf("package handler\nimport (\"net/http\"; %q)\nconst OpenAPIDocument = %q\nfunc OpenAPI(c *httpx.Context) error { return c.WriteResponse(200,http.Header{\"Content-Type\": {\"application/json; charset=utf-8\"}},[]byte(OpenAPIDocument)) }\n", frameworkModule+"/httpx", string(c.Document)+"\n"))
|
||||||
|
imports := fmt.Sprintf("import (\"fmt\"; %q; %q; %q;", frameworkModule+"/httpx", frameworkModule+"/openapi", o.Module+"/internal/servicecontext")
|
||||||
|
routes := "func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {\n if router==nil || deps==nil {return fmt.Errorf(\"router and dependencies required\")}\n"
|
||||||
|
middlewareSet := map[string]bool{}
|
||||||
|
for _, r := range c.Spec.Routes {
|
||||||
|
for _, m := range r.Middleware {
|
||||||
|
middlewareSet[m] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var middlewareNames []string
|
||||||
|
for m := range middlewareSet {
|
||||||
|
middlewareNames = append(middlewareNames, m)
|
||||||
|
}
|
||||||
|
sort.Strings(middlewareNames)
|
||||||
|
for _, m := range middlewareNames {
|
||||||
|
routes += fmt.Sprintf("if deps.Middlewares[%q]==nil {return fmt.Errorf(%q)}\n", m, "missing middleware "+m)
|
||||||
|
}
|
||||||
|
for i, group := range c.Groups {
|
||||||
|
imports += fmt.Sprintf("h%d %q; s%d %q;", i, o.Module+"/internal/handler/"+group, i, o.Module+"/internal/service/"+group)
|
||||||
|
routes += fmt.Sprintf("h%d := h%d.New(s%d.New(deps))\n", i, i, i)
|
||||||
|
groupRoutes := []Route{}
|
||||||
|
for _, r := range c.Spec.Routes {
|
||||||
|
if r.Group == group {
|
||||||
|
groupRoutes = append(groupRoutes, r)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
packageName := filepath.Base(group)
|
||||||
|
text := "package " + packageName + "\nimport(\"context\"; \"fmt\";"
|
||||||
|
needsTypes, needsBinder := false, false
|
||||||
|
for _, r := range groupRoutes {
|
||||||
|
needsTypes = needsTypes || r.Request != "" || r.Response != ""
|
||||||
|
needsBinder = needsBinder || r.Request != ""
|
||||||
|
}
|
||||||
|
text += fmt.Sprintf("%q;", frameworkModule+"/httpx")
|
||||||
|
if needsTypes {
|
||||||
|
text += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||||
|
}
|
||||||
|
if needsBinder {
|
||||||
|
text += fmt.Sprintf("%q;", frameworkModule+"/contract")
|
||||||
|
}
|
||||||
|
text += ")\n"
|
||||||
|
text += "type Service interface {\n"
|
||||||
|
for _, r := range groupRoutes {
|
||||||
|
text += signature(r) + "\n"
|
||||||
|
}
|
||||||
|
text += "}\ntype Handler struct {service Service}\nfunc New(s Service)*Handler {return &Handler{service:s}}\n"
|
||||||
|
for _, r := range groupRoutes {
|
||||||
|
name := upper(r.Handler)
|
||||||
|
if r.Request != "" {
|
||||||
|
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s]()\n", lower(r.Handler), r.Request)
|
||||||
|
}
|
||||||
|
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
|
||||||
|
call := "c.Context()"
|
||||||
|
if r.Request != "" {
|
||||||
|
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req);err!=nil{return err}\n", r.Request, lower(r.Handler))
|
||||||
|
call += ", &req"
|
||||||
|
}
|
||||||
|
if r.Response != "" {
|
||||||
|
text += fmt.Sprintf("resp,err:=h.service.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name)
|
||||||
|
method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status]
|
||||||
|
text += "return c." + method + "(resp)\n}\n"
|
||||||
|
} else {
|
||||||
|
text += fmt.Sprintf("if err:=h.service.%s(%s);err!=nil{return err};return c.NoContent()\n}\n", name, call)
|
||||||
|
}
|
||||||
|
routes += fmt.Sprintf("router.Handle(%q,%q,h%d.%s", strings.ToUpper(r.Method), r.Path, i, name)
|
||||||
|
for _, m := range r.Middleware {
|
||||||
|
routes += fmt.Sprintf(",deps.Middlewares[%q]", m)
|
||||||
|
}
|
||||||
|
routes += ")\n"
|
||||||
|
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
|
||||||
|
if r.Request != "" || r.Response != "" {
|
||||||
|
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||||
|
}
|
||||||
|
service += ")\n"
|
||||||
|
service += "// " + name + " implements the business operation. This file is never overwritten.\nfunc(s *Service) " + signature(r) + " {\n"
|
||||||
|
if starter && r.Handler == "greet" {
|
||||||
|
service = strings.Replace(service, "apierrors "+strconv.Quote(frameworkModule+"/errors")+";", "\"fmt\";", 1)
|
||||||
|
service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}"
|
||||||
|
} else {
|
||||||
|
service += "// TODO: implement business behavior and use s.deps for dependencies.\n"
|
||||||
|
if r.Response != "" {
|
||||||
|
service += "return nil, "
|
||||||
|
} else {
|
||||||
|
service += "return "
|
||||||
|
}
|
||||||
|
service += "apierrors.New(501,\"NOT_IMPLEMENTED\",\"operation is not implemented\")\n}"
|
||||||
|
}
|
||||||
|
put(seeds, "internal/service/"+group+"/"+lower(r.Handler)+".go", service)
|
||||||
|
}
|
||||||
|
if !hasResponse(groupRoutes) {
|
||||||
|
text = strings.Replace(text, "\"fmt\";", "", 1)
|
||||||
|
}
|
||||||
|
put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text)
|
||||||
|
put(seeds, "internal/service/"+group+"/service.go", fmt.Sprintf("package %s\nimport %q\ntype Service struct {deps *servicecontext.Context}\nfunc New(deps *servicecontext.Context)*Service{return &Service{deps:deps}}\n", packageName, o.Module+"/internal/servicecontext"))
|
||||||
|
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
|
||||||
|
}
|
||||||
|
// Avoid unused imports when no route uses a named middleware.
|
||||||
|
imports = strings.Replace(imports, strconv.Quote(frameworkModule+"/httpx")+";", "", 1)
|
||||||
|
put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n")
|
||||||
|
registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n"
|
||||||
|
for _, m := range middlewareNames {
|
||||||
|
registry += fmt.Sprintf("%q:%s,\n", m, upper(m))
|
||||||
|
put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m)))
|
||||||
|
}
|
||||||
|
put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n")
|
||||||
|
for name, source := range baseTemplates {
|
||||||
|
content := strings.ReplaceAll(source, "MODULE", o.Module)
|
||||||
|
content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat)
|
||||||
|
content = strings.ReplaceAll(content, "LOGOUTPUT", o.LogOutput)
|
||||||
|
if name == "etc/config.yaml" && o.LogOutput == "file" {
|
||||||
|
content = strings.Replace(content, "output: file", "output: stderr\n file: ./var/log/server.log\n rotation:\n enabled: true\n maxSizeMb: 100\n maxBackups: 10\n maxAgeDays: 7\n compress: true", 1)
|
||||||
|
}
|
||||||
|
put(seeds, name, content)
|
||||||
|
}
|
||||||
|
|
||||||
|
infra, err := scaffold.APIInfrastructure(o.Module, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
for name, data := range infra {
|
||||||
|
seeds[name] = data
|
||||||
|
}
|
||||||
|
if o.Storage == "local" {
|
||||||
|
for _, r := range c.Spec.Routes {
|
||||||
|
if r.Path == "/storage/object" {
|
||||||
|
return nil, nil, errors.New("/storage/object is reserved by local storage")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
main := string(seeds["cmd/server/main.go"])
|
||||||
|
main = strings.Replace(main, " if err=handler.RegisterRoutes", ` for _,method:=range []string{"GET","HEAD","PUT"}{app.Handle(method,"/storage/object",func(c *httpx.Context)error{deps.FileHTTP.ServeHTTP(c.Raw(),c.Request());return nil})}
|
||||||
|
if err=handler.RegisterRoutes`, 1)
|
||||||
|
main = strings.Replace(main, `"git.yuntuo.hk/k2safe/go-omni/logx"`, `"git.yuntuo.hk/k2safe/go-omni/logx"; "git.yuntuo.hk/k2safe/go-omni/httpx"`, 1)
|
||||||
|
seeds["cmd/server/main.go"] = []byte(main)
|
||||||
|
}
|
||||||
|
return generated, seeds, nil
|
||||||
|
}
|
||||||
|
func hasResponse(routes []Route) bool {
|
||||||
|
for _, r := range routes {
|
||||||
|
if r.Response != "" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
func signature(r Route) string {
|
||||||
|
s := upper(r.Handler) + "(ctx context.Context"
|
||||||
|
if r.Request != "" {
|
||||||
|
s += ",req *types." + r.Request
|
||||||
|
}
|
||||||
|
s += ") "
|
||||||
|
if r.Response != "" {
|
||||||
|
s += "(*types." + r.Response + ",error)"
|
||||||
|
} else {
|
||||||
|
s += "error"
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
460
cmd/go-omni/apigen/parse.go
Normal file
460
cmd/go-omni/apigen/parse.go
Normal file
@@ -0,0 +1,460 @@
|
|||||||
|
// Package apigen implements go-omni's deliberately bounded go-zero-style API
|
||||||
|
// dialect. It generates Omni handlers, never go-zero or transport dependencies.
|
||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"text/scanner"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Spec struct {
|
||||||
|
Title, Version, Service string
|
||||||
|
Types []Type
|
||||||
|
Routes []Route
|
||||||
|
Sources map[string][]byte // relative to the entry API's directory
|
||||||
|
Entry string
|
||||||
|
}
|
||||||
|
type Type struct {
|
||||||
|
Name string
|
||||||
|
Fields []Field
|
||||||
|
Position string
|
||||||
|
}
|
||||||
|
type Field struct{ Name, Type, Tag string }
|
||||||
|
type Route struct {
|
||||||
|
Method, Path, Request, Response, Handler, Group, Summary string
|
||||||
|
Middleware []string
|
||||||
|
Status int
|
||||||
|
Position string
|
||||||
|
}
|
||||||
|
type token struct {
|
||||||
|
text, position string
|
||||||
|
kind rune
|
||||||
|
}
|
||||||
|
type parser struct {
|
||||||
|
tokens []token
|
||||||
|
at int
|
||||||
|
spec *Spec
|
||||||
|
imports []string
|
||||||
|
}
|
||||||
|
|
||||||
|
func Parse(path string) (*Spec, error) {
|
||||||
|
entry, err := filepath.Abs(path)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
entry, err = filepath.EvalSymlinks(entry)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
root := filepath.Dir(entry)
|
||||||
|
spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||||
|
visiting, visited := map[string]bool{}, map[string]bool{}
|
||||||
|
total := 0
|
||||||
|
var visit func(string, int) error
|
||||||
|
visit = func(name string, depth int) error {
|
||||||
|
if depth > 32 || len(visited)+len(visiting) >= 64 {
|
||||||
|
return errors.New("api: import limit exceeded")
|
||||||
|
}
|
||||||
|
real, err := filepath.EvalSymlinks(name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
rel, err := filepath.Rel(root, real)
|
||||||
|
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||||
|
return errors.New("api: imports must stay within the entry API directory")
|
||||||
|
}
|
||||||
|
if filepath.Ext(real) != ".api" {
|
||||||
|
return errors.New("api: expected .api file")
|
||||||
|
}
|
||||||
|
if visiting[real] {
|
||||||
|
return fmt.Errorf("api: circular import %s", rel)
|
||||||
|
}
|
||||||
|
if visited[real] {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
visiting[real] = true
|
||||||
|
defer delete(visiting, real)
|
||||||
|
f, err := os.Open(real)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||||
|
_ = f.Close()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
total += len(data)
|
||||||
|
if len(data) > 1<<20 || total > 8<<20 {
|
||||||
|
return errors.New("api: source size limit exceeded")
|
||||||
|
}
|
||||||
|
tokens, err := lex(rel, string(data))
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p := &parser{tokens: tokens, spec: spec}
|
||||||
|
if err = p.parse(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
spec.Sources[filepath.ToSlash(rel)] = data
|
||||||
|
for _, imp := range p.imports {
|
||||||
|
if filepath.IsAbs(imp) || strings.Contains(imp, "\\") {
|
||||||
|
return errors.New("api: imports must use relative slash paths")
|
||||||
|
}
|
||||||
|
if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
visited[real] = true
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err = visit(entry, 0); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if spec.Service == "" || len(spec.Routes) == 0 {
|
||||||
|
return nil, errors.New("api: at least one service route required")
|
||||||
|
}
|
||||||
|
if spec.Title == "" {
|
||||||
|
spec.Title = spec.Service
|
||||||
|
}
|
||||||
|
return spec, nil
|
||||||
|
}
|
||||||
|
func lex(name, source string) ([]token, error) {
|
||||||
|
var s scanner.Scanner
|
||||||
|
s.Init(strings.NewReader(source))
|
||||||
|
s.Filename = name
|
||||||
|
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments
|
||||||
|
s.Whitespace ^= 1 << '\n'
|
||||||
|
var scanErr error
|
||||||
|
s.Error = func(s *scanner.Scanner, message string) {
|
||||||
|
if scanErr == nil {
|
||||||
|
scanErr = fmt.Errorf("%s: %s", s.Position, message)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var out []token
|
||||||
|
for {
|
||||||
|
kind := s.Scan()
|
||||||
|
out = append(out, token{s.TokenText(), s.Position.String(), kind})
|
||||||
|
if kind == scanner.EOF {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, scanErr
|
||||||
|
}
|
||||||
|
func (p *parser) peek() string { return p.tokens[p.at].text }
|
||||||
|
func (p *parser) next() token {
|
||||||
|
t := p.tokens[p.at]
|
||||||
|
if t.kind != scanner.EOF {
|
||||||
|
p.at++
|
||||||
|
}
|
||||||
|
return t
|
||||||
|
}
|
||||||
|
func (p *parser) skip() {
|
||||||
|
for p.peek() == "\n" || p.peek() == ";" {
|
||||||
|
p.next()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (p *parser) err(message string) error {
|
||||||
|
return fmt.Errorf("%s: %s (got %q)", p.tokens[p.at].position, message, p.peek())
|
||||||
|
}
|
||||||
|
func (p *parser) expect(s string) error {
|
||||||
|
if p.peek() != s {
|
||||||
|
return p.err("expected " + strconv.Quote(s))
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) ident() (string, error) {
|
||||||
|
if p.tokens[p.at].kind != scanner.Ident {
|
||||||
|
return "", p.err("expected identifier")
|
||||||
|
}
|
||||||
|
return p.next().text, nil
|
||||||
|
}
|
||||||
|
func (p *parser) quoted() (string, error) {
|
||||||
|
if p.tokens[p.at].kind != scanner.String {
|
||||||
|
return "", p.err("expected quoted string")
|
||||||
|
}
|
||||||
|
return strconv.Unquote(p.next().text)
|
||||||
|
}
|
||||||
|
func (p *parser) parse() error {
|
||||||
|
server := map[string]string{}
|
||||||
|
syntaxSeen := false
|
||||||
|
for p.skip(); p.tokens[p.at].kind != scanner.EOF; p.skip() {
|
||||||
|
switch p.peek() {
|
||||||
|
case "syntax":
|
||||||
|
if syntaxSeen {
|
||||||
|
return p.err("duplicate syntax")
|
||||||
|
}
|
||||||
|
syntaxSeen = true
|
||||||
|
p.next()
|
||||||
|
if err := p.expect("="); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
v, err := p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if v != "v1" {
|
||||||
|
return p.err("only syntax v1 supported")
|
||||||
|
}
|
||||||
|
case "info":
|
||||||
|
p.next()
|
||||||
|
m, err := p.metadata()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for k, v := range m {
|
||||||
|
switch k {
|
||||||
|
case "title":
|
||||||
|
p.spec.Title = v
|
||||||
|
case "version":
|
||||||
|
p.spec.Version = v
|
||||||
|
case "desc", "description", "author", "email":
|
||||||
|
default:
|
||||||
|
return p.err("unsupported info property " + k)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "import":
|
||||||
|
p.next()
|
||||||
|
if p.peek() == "(" {
|
||||||
|
p.next()
|
||||||
|
for p.skip(); p.peek() != ")"; p.skip() {
|
||||||
|
v, err := p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.imports = append(p.imports, v)
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
} else {
|
||||||
|
v, err := p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.imports = append(p.imports, v)
|
||||||
|
}
|
||||||
|
case "type":
|
||||||
|
p.next()
|
||||||
|
if p.peek() == "(" {
|
||||||
|
p.next()
|
||||||
|
for p.skip(); p.peek() != ")"; p.skip() {
|
||||||
|
if err := p.parseType(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
} else {
|
||||||
|
if err := p.parseType(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "@":
|
||||||
|
p.next()
|
||||||
|
if err := p.expect("server"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if len(server) != 0 {
|
||||||
|
return p.err("duplicate server block")
|
||||||
|
}
|
||||||
|
m, err := p.metadata()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
server = m
|
||||||
|
for k := range m {
|
||||||
|
if k != "prefix" && k != "group" && k != "middleware" {
|
||||||
|
return p.err("unsupported @server option " + k + "; wire authentication through named middleware")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "service":
|
||||||
|
if err := p.parseService(server); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
server = map[string]string{}
|
||||||
|
default:
|
||||||
|
return p.err("unsupported declaration")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(server) > 0 {
|
||||||
|
return p.err("@server must precede a service")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) metadata() (map[string]string, error) {
|
||||||
|
if err := p.expect("("); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out := map[string]string{}
|
||||||
|
for p.skip(); p.peek() != ")"; p.skip() {
|
||||||
|
key, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err = p.expect(":"); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
value := ""
|
||||||
|
if p.tokens[p.at].kind == scanner.String {
|
||||||
|
value, err = p.quoted()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
for p.peek() != "\n" && p.peek() != ";" && p.peek() != ")" && p.tokens[p.at].kind != scanner.EOF {
|
||||||
|
value += p.next().text
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if value == "" {
|
||||||
|
return nil, p.err("metadata value required")
|
||||||
|
}
|
||||||
|
if _, exists := out[key]; exists {
|
||||||
|
return nil, p.err("duplicate metadata " + key)
|
||||||
|
}
|
||||||
|
out[key] = value
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
func (p *parser) parseType() error {
|
||||||
|
pos := p.tokens[p.at].position
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p.skip()
|
||||||
|
if err = p.expect("{"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
typ := Type{Name: name, Position: pos}
|
||||||
|
for p.skip(); p.peek() != "}"; p.skip() {
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
expression := ""
|
||||||
|
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" && p.tokens[p.at].kind != scanner.RawString && p.tokens[p.at].kind != scanner.EOF {
|
||||||
|
expression += p.next().text
|
||||||
|
}
|
||||||
|
if expression == "" {
|
||||||
|
return p.err("field type required; embedded types are not supported")
|
||||||
|
}
|
||||||
|
if p.tokens[p.at].kind != scanner.RawString {
|
||||||
|
return p.err("field source tag required")
|
||||||
|
}
|
||||||
|
tag, err := strconv.Unquote(p.next().text)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag})
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
p.spec.Types = append(p.spec.Types, typ)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) parseService(server map[string]string) error {
|
||||||
|
p.next()
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for p.peek() == "-" {
|
||||||
|
p.next()
|
||||||
|
part, e := p.ident()
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
name += "-" + part
|
||||||
|
}
|
||||||
|
if p.spec.Service != "" && p.spec.Service != name {
|
||||||
|
return p.err("one service name per entry API is supported")
|
||||||
|
}
|
||||||
|
p.spec.Service = name
|
||||||
|
p.skip()
|
||||||
|
if err = p.expect("{"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for p.skip(); p.peek() != "}"; p.skip() {
|
||||||
|
route := Route{Group: server["group"], Status: 200, Position: p.tokens[p.at].position}
|
||||||
|
if route.Group == "" {
|
||||||
|
route.Group = "api"
|
||||||
|
}
|
||||||
|
if server["middleware"] != "" {
|
||||||
|
route.Middleware = strings.Split(server["middleware"], ",")
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for p.peek() == "@" {
|
||||||
|
p.next()
|
||||||
|
annotation, e := p.ident()
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
if seen[annotation] {
|
||||||
|
return p.err("duplicate route annotation")
|
||||||
|
}
|
||||||
|
seen[annotation] = true
|
||||||
|
switch annotation {
|
||||||
|
case "handler":
|
||||||
|
route.Handler, e = p.ident()
|
||||||
|
case "doc":
|
||||||
|
route.Summary, e = p.quoted()
|
||||||
|
case "status":
|
||||||
|
route.Status, e = strconv.Atoi(p.next().text)
|
||||||
|
default:
|
||||||
|
return p.err("unsupported route annotation " + annotation)
|
||||||
|
}
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
p.skip()
|
||||||
|
}
|
||||||
|
if route.Handler == "" {
|
||||||
|
return p.err("@handler required")
|
||||||
|
}
|
||||||
|
route.Method, err = p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
path := ""
|
||||||
|
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "(" && p.peek() != "returns" && p.peek() != "}" && p.tokens[p.at].kind != scanner.EOF {
|
||||||
|
path += p.next().text
|
||||||
|
}
|
||||||
|
route.Path = server["prefix"] + path
|
||||||
|
if p.peek() == "(" {
|
||||||
|
route.Request, err = p.routeType()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if p.peek() == "returns" {
|
||||||
|
p.next()
|
||||||
|
route.Response, err = p.routeType()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" {
|
||||||
|
return p.err("unexpected route suffix")
|
||||||
|
}
|
||||||
|
if route.Response == "" && !seen["status"] {
|
||||||
|
route.Status = 204
|
||||||
|
}
|
||||||
|
p.spec.Routes = append(p.spec.Routes, route)
|
||||||
|
}
|
||||||
|
p.next()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (p *parser) routeType() (string, error) {
|
||||||
|
if err := p.expect("("); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
name, err := p.ident()
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return name, p.expect(")")
|
||||||
|
}
|
||||||
73
cmd/go-omni/apigen/profile.go
Normal file
73
cmd/go-omni/apigen/profile.go
Normal file
@@ -0,0 +1,73 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
)
|
||||||
|
|
||||||
|
type projectProfile struct {
|
||||||
|
Database string `json:"database"`
|
||||||
|
Storage string `json:"storage"`
|
||||||
|
LogOutput string `json:"logOutput"`
|
||||||
|
LogFormat string `json:"logFormat"`
|
||||||
|
}
|
||||||
|
|
||||||
|
func (o Options) profile() projectProfile {
|
||||||
|
return projectProfile{o.Database, o.Storage, o.LogOutput, o.LogFormat}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Infrastructure files belong to the developer. Persist the original selection
|
||||||
|
// so a plain regeneration cannot silently change adapters or repair a missing
|
||||||
|
// file with settings from another profile.
|
||||||
|
func resolveProfile(o *Options) error {
|
||||||
|
defaults := projectProfile{"none", "none", "stderr", "json"}
|
||||||
|
data, err := os.ReadFile(filepath.Join(o.Dir, manifestPath))
|
||||||
|
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
existing := err == nil
|
||||||
|
if existing {
|
||||||
|
var previous manifest
|
||||||
|
if err = json.Unmarshal(data, &previous); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if previous.Profile != nil {
|
||||||
|
defaults = *previous.Profile
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if o.Database == "memory" {
|
||||||
|
o.Database = "none"
|
||||||
|
}
|
||||||
|
for _, item := range []struct {
|
||||||
|
name string
|
||||||
|
value *string
|
||||||
|
fallback string
|
||||||
|
}{
|
||||||
|
{"database", &o.Database, defaults.Database}, {"storage", &o.Storage, defaults.Storage},
|
||||||
|
{"log-output", &o.LogOutput, defaults.LogOutput}, {"log-format", &o.LogFormat, defaults.LogFormat},
|
||||||
|
} {
|
||||||
|
if *item.value == "" {
|
||||||
|
*item.value = item.fallback
|
||||||
|
} else if existing && *item.value != item.fallback {
|
||||||
|
return fmt.Errorf("--%s differs from the original project profile; infrastructure is developer-owned, generate a new project to compare and migrate it explicitly", item.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if o.Database != "none" && o.Database != "mysql" {
|
||||||
|
return errors.New("--database must be none, memory or mysql")
|
||||||
|
}
|
||||||
|
switch o.Storage {
|
||||||
|
case "none", "local", "s3", "r2", "oss":
|
||||||
|
default:
|
||||||
|
return errors.New("--storage must be none, local, s3, r2 or oss")
|
||||||
|
}
|
||||||
|
if o.LogOutput != "stderr" && o.LogOutput != "stdout" && o.LogOutput != "file" {
|
||||||
|
return errors.New("--log-output must be stdout, stderr or file")
|
||||||
|
}
|
||||||
|
if o.LogFormat != "json" && o.LogFormat != "text" {
|
||||||
|
return errors.New("--log-format must be json or text")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
98
cmd/go-omni/apigen/profile_test.go
Normal file
98
cmd/go-omni/apigen/profile_test.go
Normal file
@@ -0,0 +1,98 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestInfrastructureProfileOwnership(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "project")
|
||||||
|
o := Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: "mysql", Storage: "local", LogOutput: "file"}
|
||||||
|
if _, err := New(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
o = Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before := snapshot(t, dir)
|
||||||
|
for _, changed := range []Options{
|
||||||
|
{Database: "none"}, {Storage: "s3"}, {LogOutput: "stdout"}, {LogFormat: "text"},
|
||||||
|
} {
|
||||||
|
changed.Dir, changed.API = dir, o.API
|
||||||
|
if _, err := Generate(changed); err == nil {
|
||||||
|
t.Fatal("profile switch accepted")
|
||||||
|
}
|
||||||
|
if !sameFiles(before, snapshot(t, dir)) {
|
||||||
|
t.Fatal("profile rejection changed project")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Restoring a missing seed must use the saved profile, not current defaults.
|
||||||
|
if err := os.Remove(filepath.Join(dir, "internal/servicecontext/storage.go")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := Generate(o); err == nil {
|
||||||
|
t.Fatal("check accepted missing infrastructure")
|
||||||
|
}
|
||||||
|
o.Check = false
|
||||||
|
if _, err := Generate(o); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !sameFiles(before, snapshot(t, dir)) {
|
||||||
|
t.Fatal("profile reconstruction changed project")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestAPIInfrastructureConsumers(t *testing.T) {
|
||||||
|
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
|
||||||
|
t.Skip("consumer module integration")
|
||||||
|
}
|
||||||
|
for _, db := range []string{"none", "mysql"} {
|
||||||
|
for _, storage := range []string{"none", "local", "s3", "r2", "oss"} {
|
||||||
|
t.Run(db+"/"+storage, func(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "project")
|
||||||
|
if _, err := New(Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: db, Storage: storage, LogOutput: "file"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
test := `package settings
|
||||||
|
import("testing";"os")
|
||||||
|
func TestConfiguration(t *testing.T){
|
||||||
|
t.Setenv("GO_OMNI_DB_DSN","test:test@tcp(127.0.0.1:1)/test")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY","01234567890123456789012345678901")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","test");t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","test")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_BUCKET","test");t.Setenv("GO_OMNI_STORAGE_ENDPOINT","ENDPOINT")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID","01234567890123456789012345678901")
|
||||||
|
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||||
|
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
|
||||||
|
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
|
||||||
|
_=os.Getenv("unused")
|
||||||
|
}`
|
||||||
|
endpoint := ""
|
||||||
|
if storage == "s3" {
|
||||||
|
endpoint = "127.0.0.1:9000"
|
||||||
|
}
|
||||||
|
if storage == "oss" {
|
||||||
|
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
|
||||||
|
}
|
||||||
|
writeFile(t, filepath.Join(dir, "internal/settings/profile_test.go"), strings.ReplaceAll(test, `"ENDPOINT"`, `"`+endpoint+`"`))
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||||
|
cmd.Dir = dir
|
||||||
|
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||||
|
out, err := cmd.CombinedOutput()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("%v\n%s", err, out)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
160
cmd/go-omni/apigen/templates.go
Normal file
160
cmd/go-omni/apigen/templates.go
Normal file
@@ -0,0 +1,160 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import "fmt"
|
||||||
|
|
||||||
|
var baseTemplates = map[string]string{
|
||||||
|
"cmd/server/main.go": `package main
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"MODULE/internal/handler"
|
||||||
|
"MODULE/internal/servicecontext"
|
||||||
|
"MODULE/internal/settings"
|
||||||
|
)
|
||||||
|
func main(){
|
||||||
|
path:=flag.String("config","etc/config.yaml","configuration file; empty uses defaults and environment")
|
||||||
|
flag.Parse()
|
||||||
|
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||||
|
}
|
||||||
|
func run(path string)(runErr error){
|
||||||
|
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
cfg,err:=settings.Load(path);if err!=nil{return err}
|
||||||
|
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||||
|
logx.SetDefault(logger)
|
||||||
|
deps,err:=servicecontext.New(ctx,logger,cfg);if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,deps.Close())}()
|
||||||
|
app:=omni.New(omni.WithConfig(cfg.Server))
|
||||||
|
app.OnShutdown(logger.Close)
|
||||||
|
app.OnShutdown(deps.Close)
|
||||||
|
if err=handler.RegisterRoutes(app,deps);err!=nil{return err}
|
||||||
|
app.GET("/openapi.json",handler.OpenAPI)
|
||||||
|
app.GET("/health/live",deps.Health.LivenessHandler())
|
||||||
|
app.GET("/health/ready",deps.Health.ReadinessHandler())
|
||||||
|
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||||
|
return app.RunContext(ctx,cfg.Server.Address)
|
||||||
|
}
|
||||||
|
`,
|
||||||
|
".gitignore": "/server\n/var/\n.env\n.env.*\n!.env.example\n/.goOmni/api.lock\n",
|
||||||
|
}
|
||||||
|
|
||||||
|
func projectREADME(module, entry string) string {
|
||||||
|
return fmt.Sprintf(`# %s
|
||||||
|
|
||||||
|
This project is generated from an editable API contract. Business code uses
|
||||||
|
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
|
||||||
|
|
||||||
|
## Run
|
||||||
|
|
||||||
|
`+"```sh"+`
|
||||||
|
go mod tidy
|
||||||
|
go run ./cmd/server
|
||||||
|
`+"```"+`
|
||||||
|
|
||||||
|
GET /health/live and /health/ready check the running service. GET /openapi.json
|
||||||
|
serves the same document as docs/openapi.json. Bind address defaults to localhost.
|
||||||
|
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
|
||||||
|
The greeting starter works immediately; other new operation stubs return 501 until
|
||||||
|
their business logic is implemented. Authentication hooks also fail closed with 501.
|
||||||
|
|
||||||
|
## Develop
|
||||||
|
|
||||||
|
1. Edit %s and its relative imports.
|
||||||
|
2. Run the same go-omni CLI version as the framework dependency:
|
||||||
|
|
||||||
|
`+"```sh"+`
|
||||||
|
go-omni api validate --api %s
|
||||||
|
go-omni api generate --api %s --dir .
|
||||||
|
go-omni api generate --api %s --dir . --check
|
||||||
|
go test ./...
|
||||||
|
`+"```"+`
|
||||||
|
|
||||||
|
The --check command is read-only and suitable for CI. It fails on stale or missing
|
||||||
|
generated files and missing business scaffolds. go.mod, configuration, main,
|
||||||
|
Service, Repository, servicecontext, middleware hook bodies and API files are
|
||||||
|
created once and then preserved byte for byte. Gen.go files and OpenAPI are owned
|
||||||
|
by the generator; manual edits cause a conflict instead of being overwritten.
|
||||||
|
Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it.
|
||||||
|
Avoid editing files while generation is running; api.lock serializes generators.
|
||||||
|
|
||||||
|
Routes and types that disappear are removed from generated output. Handwritten
|
||||||
|
business files remain: remove/adapt obsolete implementations explicitly if an
|
||||||
|
API change makes their old types or signatures fail to compile. Every regeneration
|
||||||
|
must be followed by go test ./... . Do not delete service files to change contracts.
|
||||||
|
|
||||||
|
## Layout
|
||||||
|
|
||||||
|
`+"```text"+`
|
||||||
|
api/ # editable contract and imported types
|
||||||
|
cmd/server/main.go # startup, lifecycle and route registration
|
||||||
|
etc/config.yaml # lowerCamelCase configuration
|
||||||
|
internal/settings/ # defaults -> file -> environment
|
||||||
|
internal/servicecontext/ # dependency initialization and cleanup
|
||||||
|
internal/types/typesGen.go # generated request/response types
|
||||||
|
internal/handler/<group>/ # generated binding and service adapters
|
||||||
|
internal/service/<group>/ # developer-owned business operations
|
||||||
|
internal/repository/<group>/ # developer-owned persistence contracts
|
||||||
|
internal/middleware/ # named hooks and generated registry
|
||||||
|
docs/openapi.json # generated OpenAPI 3.0.3
|
||||||
|
.goOmni/apiManifest.json # ownership and content hashes
|
||||||
|
`+"```"+`
|
||||||
|
|
||||||
|
The API response describes data only. Omni adds code, message and requestId.
|
||||||
|
Do not define the envelope again in each response. Handlers reject nil successful
|
||||||
|
responses. Body DTOs use application/json and have a 1 MiB limit by default.
|
||||||
|
Use contract.Binder.Bind's optional limit in a custom adapter for larger payloads.
|
||||||
|
|
||||||
|
Fields require exactly one source tag: json, path, form (URL query only), header.
|
||||||
|
All are required by presence unless optional, omitempty or default is specified.
|
||||||
|
Zero and false are valid present values; binding:"required" rejects scalar zero.
|
||||||
|
Use *T with optional when business logic needs to distinguish missing from zero.
|
||||||
|
Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON
|
||||||
|
fields and duplicate scalar query/header values are rejected. Arrays in the query
|
||||||
|
use repeated keys. Enum/range/default and nested fields share the documentation
|
||||||
|
and binding compiler. Responses are expected to honor their declared constraints;
|
||||||
|
they are not validated on every response. Do not return nil slices/maps for a
|
||||||
|
non-nullable collection (initialize empty collections instead).
|
||||||
|
|
||||||
|
Named middleware is composed in declaration order (first is outermost). Implement
|
||||||
|
the generated hooks or replace entries in servicecontext; never bypass an auth
|
||||||
|
hook merely to make the server start. Middleware names do not imply a JWT policy.
|
||||||
|
|
||||||
|
This CLI supports a documented go-zero-style subset, not all goctl features.
|
||||||
|
Database tables and repositories cannot be inferred from an HTTP contract. Add
|
||||||
|
business repositories through servicecontext. Database and storage options are
|
||||||
|
composed at creation; the original profile is retained on regeneration.
|
||||||
|
`, module, entry, entry, entry, entry)
|
||||||
|
}
|
||||||
|
|
||||||
|
func infrastructureREADME(o Options) string {
|
||||||
|
return fmt.Sprintf(`
|
||||||
|
## Infrastructure
|
||||||
|
|
||||||
|
Selected database: %s; storage: %s; logging: %s/%s.
|
||||||
|
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
|
||||||
|
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||||
|
No tables are invented or migrated automatically from request/response DTOs.
|
||||||
|
The none/memory selection opens no database; implement repositories per business.
|
||||||
|
|
||||||
|
With storage enabled, services receive deps.Files, Uploads, Downloads and
|
||||||
|
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
|
||||||
|
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
|
||||||
|
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
|
||||||
|
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
|
||||||
|
and authorize before issuing them. No public upload-link route is added.
|
||||||
|
Storage failures during startup close an already-open database; Close is idempotent.
|
||||||
|
|
||||||
|
Database/storage/logging selection is recorded in .goOmni/apiManifest.json.
|
||||||
|
Changing selection through generation flags is rejected on an existing project;
|
||||||
|
edit your owned configuration and initialization explicitly, or generate a fresh
|
||||||
|
project for comparison. New routes still regenerate without repeating these flags.
|
||||||
|
`, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||||
|
}
|
||||||
295
cmd/go-omni/apigen/write.go
Normal file
295
cmd/go-omni/apigen/write.go
Normal file
@@ -0,0 +1,295 @@
|
|||||||
|
package apigen
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
const manifestPath = ".goOmni/apiManifest.json"
|
||||||
|
|
||||||
|
type manifest struct {
|
||||||
|
Profile *projectProfile `json:"profile,omitempty"`
|
||||||
|
Version int `json:"version"`
|
||||||
|
Module string `json:"module"`
|
||||||
|
Entry string `json:"entry"`
|
||||||
|
Generated map[string]string `json:"generated"`
|
||||||
|
}
|
||||||
|
type change struct {
|
||||||
|
name string
|
||||||
|
before, after []byte
|
||||||
|
existed, remove bool
|
||||||
|
mode os.FileMode
|
||||||
|
}
|
||||||
|
|
||||||
|
func apply(root, module, entry string, generated, seeds map[string][]byte, check bool, profile projectProfile) (Output, error) {
|
||||||
|
var result Output
|
||||||
|
if info, err := os.Lstat(root); err == nil {
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() {
|
||||||
|
return result, errors.New("api: output root must be a real directory")
|
||||||
|
}
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
root = canonical(root)
|
||||||
|
names := map[string]bool{}
|
||||||
|
caseNames := map[string]string{}
|
||||||
|
for _, files := range []map[string][]byte{generated, seeds} {
|
||||||
|
for name := range files {
|
||||||
|
if names[name] {
|
||||||
|
return result, fmt.Errorf("duplicate output %s", name)
|
||||||
|
}
|
||||||
|
names[name] = true
|
||||||
|
folded := strings.ToLower(name)
|
||||||
|
if old, ok := caseNames[folded]; ok {
|
||||||
|
return result, fmt.Errorf("case-insensitive path collision: %s and %s", old, name)
|
||||||
|
}
|
||||||
|
caseNames[folded] = name
|
||||||
|
if err := safePath(root, name); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := safePath(root, manifestPath); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
if !check {
|
||||||
|
if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
lock := filepath.Join(root, ".goOmni", "api.lock")
|
||||||
|
f, err := os.OpenFile(lock, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600)
|
||||||
|
if err != nil {
|
||||||
|
return result, fmt.Errorf("api: acquire generation lock (remove stale api.lock only after checking no generator is running): %w", err)
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(f, "pid=%d\n", os.Getpid())
|
||||||
|
_ = f.Close()
|
||||||
|
defer os.Remove(lock)
|
||||||
|
}
|
||||||
|
old := manifest{Generated: map[string]string{}}
|
||||||
|
oldData, err := os.ReadFile(filepath.Join(root, manifestPath))
|
||||||
|
if err == nil {
|
||||||
|
if err = json.Unmarshal(oldData, &old); err != nil {
|
||||||
|
return result, fmt.Errorf("invalid ownership manifest: %w", err)
|
||||||
|
}
|
||||||
|
if old.Profile != nil && *old.Profile != profile {
|
||||||
|
return result, errors.New("api: profile changed during generation; retry")
|
||||||
|
}
|
||||||
|
if old.Version != 1 || old.Module != module || old.Entry != entry {
|
||||||
|
return result, errors.New("api: manifest version/module/entry differs; one API entry owns this project")
|
||||||
|
}
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
if oldData == nil {
|
||||||
|
if err = filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
|
||||||
|
if errors.Is(walkErr, os.ErrNotExist) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if walkErr != nil {
|
||||||
|
return walkErr
|
||||||
|
}
|
||||||
|
if d.IsDir() && d.Name() == ".git" {
|
||||||
|
return filepath.SkipDir
|
||||||
|
}
|
||||||
|
if !d.IsDir() && strings.HasSuffix(d.Name(), ".go") {
|
||||||
|
return fmt.Errorf("api: %s belongs to an existing Go layout; generate into a new directory", path)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
next := manifest{Profile: &profile, Version: 1, Module: module, Entry: entry, Generated: map[string]string{}}
|
||||||
|
for name, data := range generated {
|
||||||
|
next.Generated[name] = digest(data)
|
||||||
|
}
|
||||||
|
var changes []change
|
||||||
|
// Check ALL previous outputs, even stale ones, before writing any new file.
|
||||||
|
for name, hash := range old.Generated {
|
||||||
|
if !ownedPath(name) {
|
||||||
|
return result, fmt.Errorf("api: invalid owned path %q", name)
|
||||||
|
}
|
||||||
|
if err = safePath(root, name); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
data, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(name)))
|
||||||
|
if errors.Is(e, os.ErrNotExist) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if e != nil {
|
||||||
|
return result, e
|
||||||
|
}
|
||||||
|
if digest(data) != hash {
|
||||||
|
return result, fmt.Errorf("api: generated file %s was edited; move custom code into developer-owned files before regenerating", name)
|
||||||
|
}
|
||||||
|
if _, exists := generated[name]; !exists {
|
||||||
|
info, _ := os.Stat(filepath.Join(root, filepath.FromSlash(name)))
|
||||||
|
changes = append(changes, change{name: name, before: data, existed: true, remove: true, mode: info.Mode().Perm()})
|
||||||
|
result.Removed = append(result.Removed, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
paths := make([]string, 0, len(names))
|
||||||
|
for name := range names {
|
||||||
|
paths = append(paths, name)
|
||||||
|
}
|
||||||
|
sort.Strings(paths)
|
||||||
|
for _, name := range paths {
|
||||||
|
target := filepath.Join(root, filepath.FromSlash(name))
|
||||||
|
current, err := os.ReadFile(target)
|
||||||
|
exists := err == nil
|
||||||
|
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
content, isGenerated := generated[name]
|
||||||
|
if !isGenerated {
|
||||||
|
content = seeds[name]
|
||||||
|
}
|
||||||
|
if exists && !isGenerated {
|
||||||
|
result.Preserved = append(result.Preserved, name)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if exists && isGenerated {
|
||||||
|
if _, owned := old.Generated[name]; !owned {
|
||||||
|
return result, fmt.Errorf("api: refusing to adopt existing unowned file %s", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if isGenerated {
|
||||||
|
result.Generated = append(result.Generated, name)
|
||||||
|
} else {
|
||||||
|
result.Created = append(result.Created, name)
|
||||||
|
}
|
||||||
|
if exists && bytes.Equal(current, content) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
mode := os.FileMode(0644)
|
||||||
|
if exists {
|
||||||
|
info, e := os.Stat(target)
|
||||||
|
if e != nil {
|
||||||
|
return result, e
|
||||||
|
}
|
||||||
|
mode = info.Mode().Perm()
|
||||||
|
}
|
||||||
|
changes = append(changes, change{name: name, before: current, after: content, existed: exists, mode: mode})
|
||||||
|
}
|
||||||
|
data, err := json.MarshalIndent(next, "", " ")
|
||||||
|
if err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
data = append(data, '\n')
|
||||||
|
if !bytes.Equal(oldData, data) {
|
||||||
|
changes = append(changes, change{name: manifestPath, before: oldData, after: data, existed: oldData != nil, mode: 0644})
|
||||||
|
}
|
||||||
|
sort.Strings(result.Removed)
|
||||||
|
if check {
|
||||||
|
if len(changes) > 0 {
|
||||||
|
var stale []string
|
||||||
|
for _, c := range changes {
|
||||||
|
stale = append(stale, c.name)
|
||||||
|
}
|
||||||
|
sort.Strings(stale)
|
||||||
|
return result, fmt.Errorf("api: generated project is stale: %s", strings.Join(stale, ", "))
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
// A rollback restores bytes/modes if an ordinary filesystem write fails.
|
||||||
|
// File replacement is atomic; the whole directory is not crash-atomic.
|
||||||
|
var applied []change
|
||||||
|
for _, c := range changes {
|
||||||
|
if err = safePath(root, c.name); err == nil {
|
||||||
|
current, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||||
|
if c.existed {
|
||||||
|
if e != nil || !bytes.Equal(current, c.before) {
|
||||||
|
err = fmt.Errorf("file changed during generation: %s", c.name)
|
||||||
|
}
|
||||||
|
} else if !errors.Is(e, os.ErrNotExist) {
|
||||||
|
err = fmt.Errorf("file appeared during generation: %s", c.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
if c.remove {
|
||||||
|
err = os.Remove(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||||
|
} else {
|
||||||
|
err = atomicWrite(root, c.name, c.after, c.mode)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
for i := len(applied) - 1; i >= 0; i-- {
|
||||||
|
previous := applied[i]
|
||||||
|
var rollback error
|
||||||
|
if previous.existed {
|
||||||
|
rollback = atomicWrite(root, previous.name, previous.before, previous.mode)
|
||||||
|
} else {
|
||||||
|
rollback = os.Remove(filepath.Join(root, filepath.FromSlash(previous.name)))
|
||||||
|
}
|
||||||
|
err = errors.Join(err, rollback)
|
||||||
|
}
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
applied = append(applied, c)
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
|
||||||
|
func ownedPath(name string) bool {
|
||||||
|
if name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return strings.HasPrefix(name, "internal/handler/") && strings.HasSuffix(name, "/handlersGen.go")
|
||||||
|
}
|
||||||
|
func safePath(root, name string) error {
|
||||||
|
if filepath.IsAbs(name) || strings.Contains(name, "\\") || filepath.ToSlash(filepath.Clean(name)) != name || name == "." || strings.HasPrefix(name, "../") {
|
||||||
|
return fmt.Errorf("api: unsafe output path %q", name)
|
||||||
|
}
|
||||||
|
path := root
|
||||||
|
parts := strings.Split(name, "/")
|
||||||
|
for i, part := range parts {
|
||||||
|
path = filepath.Join(path, part)
|
||||||
|
info, err := os.Lstat(path)
|
||||||
|
if errors.Is(err, os.ErrNotExist) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if info.Mode()&os.ModeSymlink != 0 {
|
||||||
|
return fmt.Errorf("api: refusing symlink %s", path)
|
||||||
|
}
|
||||||
|
if i < len(parts)-1 && !info.IsDir() {
|
||||||
|
return fmt.Errorf("api: expected directory %s", path)
|
||||||
|
}
|
||||||
|
if i == len(parts)-1 && !info.Mode().IsRegular() {
|
||||||
|
return fmt.Errorf("api: expected regular file %s", path)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func atomicWrite(root, name string, data []byte, mode os.FileMode) error {
|
||||||
|
target := filepath.Join(root, filepath.FromSlash(name))
|
||||||
|
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
f, err := os.CreateTemp(filepath.Dir(target), ".apiGen-*")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer os.Remove(f.Name())
|
||||||
|
if err = f.Chmod(mode); err == nil {
|
||||||
|
_, err = f.Write(data)
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
err = f.Sync()
|
||||||
|
}
|
||||||
|
err = errors.Join(err, f.Close())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.Rename(f.Name(), target)
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"path/filepath"
|
"path/filepath"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -17,6 +18,16 @@ func main() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
switch os.Args[1] {
|
switch os.Args[1] {
|
||||||
|
case "api":
|
||||||
|
if err := runAPI(os.Args[2:]); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, "go-omni api:", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
|
case "migrate":
|
||||||
|
if err := runMigrate(os.Args[2:]); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
|
||||||
|
os.Exit(1)
|
||||||
|
}
|
||||||
case "add":
|
case "add":
|
||||||
if err := runAdd(os.Args[2:]); err != nil {
|
if err := runAdd(os.Args[2:]); err != nil {
|
||||||
fmt.Fprintln(os.Stderr, "go-omni add:", err)
|
fmt.Fprintln(os.Stderr, "go-omni add:", err)
|
||||||
@@ -39,9 +50,43 @@ func main() {
|
|||||||
func runNew(args []string) error {
|
func runNew(args []string) error {
|
||||||
options := scaffold.Options{}
|
options := scaffold.Options{}
|
||||||
var target string
|
var target string
|
||||||
|
var apiFile string
|
||||||
for index := 0; index < len(args); index++ {
|
for index := 0; index < len(args); index++ {
|
||||||
arg := args[index]
|
arg := args[index]
|
||||||
switch {
|
switch {
|
||||||
|
case arg == "--api" || strings.HasPrefix(arg, "--api="):
|
||||||
|
_, value, inline := strings.Cut(arg, "=")
|
||||||
|
if !inline {
|
||||||
|
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "-") {
|
||||||
|
return errors.New("--api requires a file")
|
||||||
|
}
|
||||||
|
index++
|
||||||
|
value = args[index]
|
||||||
|
}
|
||||||
|
if value == "" {
|
||||||
|
return errors.New("--api requires a file")
|
||||||
|
}
|
||||||
|
apiFile = value
|
||||||
|
case arg == "--log-output" || arg == "--log-format" || arg == "--storage" || strings.HasPrefix(arg, "--log-output=") || strings.HasPrefix(arg, "--log-format=") || strings.HasPrefix(arg, "--storage="):
|
||||||
|
name, value, hasValue := strings.Cut(arg, "=")
|
||||||
|
if !hasValue {
|
||||||
|
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "--") {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
index++
|
||||||
|
value = args[index]
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(value) == "" {
|
||||||
|
return fmt.Errorf("%s requires a value", name)
|
||||||
|
}
|
||||||
|
switch name {
|
||||||
|
case "--log-output":
|
||||||
|
options.LogOutput = value
|
||||||
|
case "--log-format":
|
||||||
|
options.LogFormat = value
|
||||||
|
case "--storage":
|
||||||
|
options.Storage = value
|
||||||
|
}
|
||||||
case arg == "--force":
|
case arg == "--force":
|
||||||
options.Force = true
|
options.Force = true
|
||||||
case arg == "--database":
|
case arg == "--database":
|
||||||
@@ -93,10 +138,26 @@ func runNew(args []string) error {
|
|||||||
if target == "" {
|
if target == "" {
|
||||||
return errors.New("project directory is required")
|
return errors.New("project directory is required")
|
||||||
}
|
}
|
||||||
if options.Module == "" {
|
if options.Module == "" && apiFile == "" {
|
||||||
options.Module = filepath.Base(filepath.Clean(target))
|
options.Module = filepath.Base(filepath.Clean(target))
|
||||||
}
|
}
|
||||||
options.Dir = target
|
options.Dir = target
|
||||||
|
if apiFile != "" {
|
||||||
|
if options.Force {
|
||||||
|
return errors.New("--api protects existing business code and does not accept --force")
|
||||||
|
}
|
||||||
|
_, err := apigen.Generate(apigen.Options{API: apiFile, Dir: target, Module: options.Module, FrameworkDir: options.FrameworkDir, FrameworkVersion: options.FrameworkVersion, LogOutput: options.LogOutput, LogFormat: options.LogFormat, Database: options.Database, Storage: options.Storage})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Printf("created API project %s; implement internal/service operations\n", filepath.Clean(target))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(filepath.Join(target, ".goOmni", "apiManifest.json")); err == nil {
|
||||||
|
return errors.New("this is an API project; use api generate instead of the CRUD/module starter")
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
if err := scaffold.Generate(options); err != nil {
|
if err := scaffold.Generate(options); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -108,11 +169,19 @@ func usage() {
|
|||||||
fmt.Fprintln(os.Stderr, "GoOmni project tool")
|
fmt.Fprintln(os.Stderr, "GoOmni project tool")
|
||||||
fmt.Fprintln(os.Stderr, "")
|
fmt.Fprintln(os.Stderr, "")
|
||||||
fmt.Fprintln(os.Stderr, "Usage:")
|
fmt.Fprintln(os.Stderr, "Usage:")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
||||||
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni api new [--module <module>] <directory> Create a runnable API project")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni api generate --api <file.api> --dir <project> [--check]")
|
||||||
|
fmt.Fprintln(os.Stderr, " go-omni api validate --api <file.api> Validate definitions and OpenAPI")
|
||||||
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
||||||
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
||||||
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
||||||
fmt.Fprintln(os.Stderr, " --database <memory|mysql> Select generated storage (default memory)")
|
fmt.Fprintln(os.Stderr, " --database <none|memory|mysql> API defaults to none; CRUD defaults to memory")
|
||||||
|
fmt.Fprintln(os.Stderr, " --log-output <stdout|stderr|file> Generate configured logging (default stderr)")
|
||||||
|
fmt.Fprintln(os.Stderr, " --log-format <json|text> Select structured log format (default json)")
|
||||||
|
fmt.Fprintln(os.Stderr, " --storage <none|local|s3|r2|oss> Select optional storage adapter (default none)")
|
||||||
}
|
}
|
||||||
|
|
||||||
func runAdd(args []string) error {
|
func runAdd(args []string) error {
|
||||||
|
|||||||
@@ -73,3 +73,28 @@ func TestNewMySQLDatabaseOptions(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestModuleOptionCLI(t *testing.T) {
|
||||||
|
for _, flags := range [][]string{
|
||||||
|
{"--log-output", "file", "--log-format", "json", "--storage", "local"},
|
||||||
|
{"--log-output=stdout", "--log-format=text", "--storage=none"},
|
||||||
|
} {
|
||||||
|
project := filepath.Join(t.TempDir(), "demo")
|
||||||
|
args := append([]string{"--module", "example.com/demo"}, flags...)
|
||||||
|
if err := runNew(append(args, project)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(filepath.Join(project, "etc/config.yaml")); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, args := range [][]string{
|
||||||
|
{"--storage"}, {"--storage="}, {"--storage", "--force"},
|
||||||
|
{"--log-output"}, {"--log-output="}, {"--log-format="},
|
||||||
|
{"--storage", "unknown", "demo"}, {"--log-format", "xml", "demo"},
|
||||||
|
} {
|
||||||
|
if err := runNew(args); err == nil {
|
||||||
|
t.Fatal("invalid module option accepted", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
60
cmd/go-omni/migrate.go
Normal file
60
cmd/go-omni/migrate.go
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
migration "git.yuntuo.hk/k2safe/go-omni/database/migrate"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
func runMigrate(args []string) error {
|
||||||
|
flags := flag.NewFlagSet("migrate", flag.ContinueOnError)
|
||||||
|
flags.SetOutput(io.Discard)
|
||||||
|
directory := flags.String("dir", "migrations", "SQL directory")
|
||||||
|
dsnEnv := flags.String("dsn-env", "GO_OMNI_DB_DSN", "DSN environment variable")
|
||||||
|
apply := flags.Bool("apply", false, "apply pending up migrations")
|
||||||
|
if err := flags.Parse(args); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if flags.NArg() != 0 {
|
||||||
|
return errors.New("migrate: unexpected argument")
|
||||||
|
}
|
||||||
|
_, manifest, err := migration.Snapshot(os.DirFS(*directory), ".")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !*apply {
|
||||||
|
data, err := json.MarshalIndent(manifest, "", " ")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Println(string(data))
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
dsn := os.Getenv(*dsnEnv)
|
||||||
|
if dsn == "" {
|
||||||
|
return errors.New("migrate: DSN environment variable is empty")
|
||||||
|
}
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
runner, err := migration.NewMySQL(ctx, dsn, os.DirFS(*directory), ".")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer runner.Close()
|
||||||
|
if err = runner.Up(ctx); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
version, dirty, err := runner.Version()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
fmt.Printf("migration version=%d dirty=%t\n", version, dirty)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
25
cmd/go-omni/migrate_test.go
Normal file
25
cmd/go-omni/migrate_test.go
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestMigrationPreviewDoesNotConnect(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, "0001_createItems.up.sql"), []byte("CREATE TABLE testItems(id BIGINT);"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Setenv("GO_OMNI_DB_DSN", "not a DSN")
|
||||||
|
if err := runMigrate([]string{"--dir", dir}); err != nil {
|
||||||
|
t.Fatal("preview tried to connect", err)
|
||||||
|
}
|
||||||
|
t.Setenv("GO_OMNI_DB_DSN", "")
|
||||||
|
if err := runMigrate([]string{"--dir", dir, "--apply"}); err == nil {
|
||||||
|
t.Fatal("missing DSN accepted")
|
||||||
|
}
|
||||||
|
if err := runMigrate([]string{"--dir", dir, "--force"}); err == nil {
|
||||||
|
t.Fatal("unsafe unknown flag accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
29
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
29
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
@@ -0,0 +1,29 @@
|
|||||||
|
package scaffold
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// APIInfrastructure renders shared infrastructure without inventing business
|
||||||
|
// tables or routes. The API generator applies these as developer-owned seeds.
|
||||||
|
func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
|
||||||
|
data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true}
|
||||||
|
files := map[string]string{
|
||||||
|
"internal/settings/config.go": "templates/modules-settings.go.tmpl",
|
||||||
|
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
|
||||||
|
"internal/servicecontext/context.go": "templates/api-context.go.tmpl",
|
||||||
|
}
|
||||||
|
if storage != "none" {
|
||||||
|
files["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||||
|
}
|
||||||
|
result := make(map[string][]byte, len(files))
|
||||||
|
for name, source := range files {
|
||||||
|
content, err := render(source, data)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("render %s: %w", name, err)
|
||||||
|
}
|
||||||
|
result[name] = []byte(strings.TrimRight(content, "\n") + "\n")
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
package scaffold
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type moduleProfile struct{ database, storage, output, format string }
|
||||||
|
|
||||||
|
var moduleProfiles = []moduleProfile{
|
||||||
|
{"memory", "none", "stdout", "json"},
|
||||||
|
{"memory", "local", "file", "json"},
|
||||||
|
{"memory", "s3", "stdout", "text"},
|
||||||
|
{"memory", "r2", "stderr", "json"},
|
||||||
|
{"memory", "oss", "stderr", "text"},
|
||||||
|
{"mysql", "none", "file", "text"},
|
||||||
|
{"mysql", "local", "stdout", "json"},
|
||||||
|
{"mysql", "s3", "file", "text"},
|
||||||
|
{"mysql", "r2", "stderr", "text"},
|
||||||
|
{"mysql", "oss", "file", "json"},
|
||||||
|
}
|
||||||
|
|
||||||
|
func generateProfile(t *testing.T, p moduleProfile) string {
|
||||||
|
t.Helper()
|
||||||
|
project := filepath.Join(t.TempDir(), "project")
|
||||||
|
if err := Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: p.database, Storage: p.storage, LogOutput: p.output, LogFormat: p.format}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return project
|
||||||
|
}
|
||||||
|
func TestGenerateModuleProfiles(t *testing.T) {
|
||||||
|
for _, p := range moduleProfiles {
|
||||||
|
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||||
|
project := generateProfile(t, p)
|
||||||
|
for _, name := range []string{"etc/config.yaml", "internal/settings/config.go", "MODULES.md", ".gitignore"} {
|
||||||
|
if _, err := os.Stat(filepath.Join(project, name)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, err := os.Stat(filepath.Join(project, "internal/handler/files.go"))
|
||||||
|
if p.storage == "none" && !os.IsNotExist(err) {
|
||||||
|
t.Fatal("unselected storage was emitted")
|
||||||
|
}
|
||||||
|
if p.storage != "none" && err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
module, err := os.ReadFile(filepath.Join(project, "go.mod"))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
pieces := strings.Split(string(module), "=> ")
|
||||||
|
if len(pieces) != 2 {
|
||||||
|
t.Fatal("local replace missing")
|
||||||
|
}
|
||||||
|
relative, err := strconv.Unquote(strings.TrimSpace(pieces[1]))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err = os.Stat(filepath.Join(project, relative, "go.mod")); err != nil {
|
||||||
|
t.Fatal("replacement does not resolve", err)
|
||||||
|
}
|
||||||
|
cfg, _ := os.ReadFile(filepath.Join(project, "etc/config.yaml"))
|
||||||
|
if strings.Contains(string(cfg), "secretKey:") || strings.Contains(string(cfg), "filesToken:") {
|
||||||
|
t.Fatal("secret defaults emitted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestModuleOptionsRejectBeforeWriting(t *testing.T) {
|
||||||
|
for _, options := range []Options{
|
||||||
|
{LogOutput: "invalid"}, {LogFormat: "xml"}, {Storage: "ftp"},
|
||||||
|
{Storage: "local", FrameworkVersion: "v0.6.0-preview"},
|
||||||
|
{Module: "example.com/x\nrequire malicious v1.0.0"},
|
||||||
|
} {
|
||||||
|
options.Dir = filepath.Join(t.TempDir(), "project")
|
||||||
|
if options.Module == "" {
|
||||||
|
options.Module = "example.com/profile"
|
||||||
|
}
|
||||||
|
if err := Generate(options); err == nil {
|
||||||
|
t.Fatalf("invalid options accepted: %+v", options)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(options.Dir); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("invalid options touched target")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestCanonicalPathBeforeDirectoryExists(t *testing.T) {
|
||||||
|
parent := t.TempDir()
|
||||||
|
alias := filepath.Join(t.TempDir(), "alias")
|
||||||
|
if err := os.Symlink(parent, alias); err != nil {
|
||||||
|
t.Skip(err)
|
||||||
|
}
|
||||||
|
path := filepath.Join(alias, "new", "nested")
|
||||||
|
if got, want := canonicalPath(path), filepath.Join(canonicalPath(parent), "new", "nested"); got != want {
|
||||||
|
t.Fatalf("%s != %s", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Compile consuming modules, then run their generated tests. No provider account
|
||||||
|
// is used: cloud constructors are checked offline, file workflows use Local.
|
||||||
|
func TestModuleProfilesIntegration(t *testing.T) {
|
||||||
|
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||||
|
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-profile tests")
|
||||||
|
}
|
||||||
|
for _, p := range moduleProfiles {
|
||||||
|
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||||
|
project := generateProfile(t, p)
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-p=2", "-count=1", "-timeout=60s", "./...")
|
||||||
|
cmd.Dir = project
|
||||||
|
cmd.Env = append(os.Environ(), "GOWORK=off", "GOMAXPROCS=2", "GO_OMNI_TEST_MYSQL_DSN=")
|
||||||
|
if output, err := cmd.CombinedOutput(); err != nil {
|
||||||
|
t.Fatalf("generated %s/%s: %v\n%s", p.database, p.storage, err, output)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestForceRejectsRemovedDependenciesWithoutTouchingCode(t *testing.T) {
|
||||||
|
project := generateProfile(t, moduleProfile{"mysql", "local", "file", "json"})
|
||||||
|
path := filepath.Join(project, "cmd/server/main.go")
|
||||||
|
before, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Storage: "none"}); err == nil {
|
||||||
|
t.Fatal("incompatible force overwrite accepted")
|
||||||
|
}
|
||||||
|
after, _ := os.ReadFile(path)
|
||||||
|
if string(after) != string(before) {
|
||||||
|
t.Fatal("failed profile switch changed code")
|
||||||
|
}
|
||||||
|
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Database: "mysql", Storage: "local", LogOutput: "stdout"}); err != nil {
|
||||||
|
t.Fatal("compatible force overwrite rejected", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -12,6 +12,9 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"text/template"
|
"text/template"
|
||||||
|
|
||||||
|
"golang.org/x/mod/module"
|
||||||
|
"golang.org/x/mod/semver"
|
||||||
)
|
)
|
||||||
|
|
||||||
//go:embed templates moduleTemplates
|
//go:embed templates moduleTemplates
|
||||||
@@ -24,6 +27,9 @@ type Options struct {
|
|||||||
FrameworkDir string
|
FrameworkDir string
|
||||||
FrameworkVersion string
|
FrameworkVersion string
|
||||||
Database string
|
Database string
|
||||||
|
LogOutput string
|
||||||
|
LogFormat string
|
||||||
|
Storage string
|
||||||
Force bool
|
Force bool
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -48,6 +54,31 @@ func Generate(options Options) error {
|
|||||||
if strings.TrimSpace(options.Module) == "" {
|
if strings.TrimSpace(options.Module) == "" {
|
||||||
return errors.New("module path is required")
|
return errors.New("module path is required")
|
||||||
}
|
}
|
||||||
|
if err := module.CheckImportPath(options.Module); err != nil {
|
||||||
|
return fmt.Errorf("invalid module path: %w", err)
|
||||||
|
}
|
||||||
|
modules := options.LogOutput != "" || options.LogFormat != "" || options.Storage != ""
|
||||||
|
logOutput, logFormat, storage := options.LogOutput, options.LogFormat, options.Storage
|
||||||
|
if logOutput == "" {
|
||||||
|
logOutput = "stderr"
|
||||||
|
}
|
||||||
|
if logFormat == "" {
|
||||||
|
logFormat = "json"
|
||||||
|
}
|
||||||
|
if storage == "" {
|
||||||
|
storage = "none"
|
||||||
|
}
|
||||||
|
if logOutput != "stdout" && logOutput != "stderr" && logOutput != "file" {
|
||||||
|
return fmt.Errorf("unsupported log output %q: choose stdout, stderr or file", logOutput)
|
||||||
|
}
|
||||||
|
if logFormat != "json" && logFormat != "text" {
|
||||||
|
return fmt.Errorf("unsupported log format %q: choose json or text", logFormat)
|
||||||
|
}
|
||||||
|
switch storage {
|
||||||
|
case "none", "local", "s3", "r2", "oss":
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("unsupported object storage %q: choose none, local, s3, r2 or oss", storage)
|
||||||
|
}
|
||||||
database := strings.ToLower(strings.TrimSpace(options.Database))
|
database := strings.ToLower(strings.TrimSpace(options.Database))
|
||||||
if database == "" {
|
if database == "" {
|
||||||
database = "memory"
|
database = "memory"
|
||||||
@@ -58,6 +89,9 @@ func Generate(options Options) error {
|
|||||||
version := strings.TrimSpace(options.FrameworkVersion)
|
version := strings.TrimSpace(options.FrameworkVersion)
|
||||||
if version == "" {
|
if version == "" {
|
||||||
version = currentFrameworkVersion()
|
version = currentFrameworkVersion()
|
||||||
|
if modules && version == defaultFrameworkVersion {
|
||||||
|
version = modulesFrameworkVersion
|
||||||
|
}
|
||||||
}
|
}
|
||||||
if !validFrameworkVersion(version) {
|
if !validFrameworkVersion(version) {
|
||||||
return fmt.Errorf("framework version %q must be a complete v0 or v1 semantic version", version)
|
return fmt.Errorf("framework version %q must be a complete v0 or v1 semantic version", version)
|
||||||
@@ -70,20 +104,20 @@ func Generate(options Options) error {
|
|||||||
if err := checkTarget(directory, options.Force); err != nil {
|
if err := checkTarget(directory, options.Force); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
|
||||||
return fmt.Errorf("create project directory: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// An explicit remote version opts out of automatic local development mode.
|
// An explicit remote version opts out of automatic local development mode.
|
||||||
replace := ""
|
replace := ""
|
||||||
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
|
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
|
||||||
replace = frameworkReplace(directory, options.FrameworkDir)
|
replace = frameworkReplace(directory, options.FrameworkDir)
|
||||||
}
|
}
|
||||||
|
if modules && replace == "" && semver.Compare(version, modulesFrameworkVersion) < 0 && version != installedFrameworkVersion() {
|
||||||
|
return fmt.Errorf("module templates require %s or newer; use --framework-dir for unpublished source", modulesFrameworkVersion)
|
||||||
|
}
|
||||||
data := templateData{
|
data := templateData{
|
||||||
Module: options.Module,
|
Module: options.Module,
|
||||||
FrameworkVersion: version,
|
FrameworkVersion: version,
|
||||||
FrameworkReplace: replace,
|
FrameworkReplace: replace,
|
||||||
Database: database,
|
Database: database,
|
||||||
|
Modules: modules, LogOutput: logOutput, LogFormat: logFormat, Storage: storage,
|
||||||
}
|
}
|
||||||
projectFiles := make(map[string]string, len(files)+3)
|
projectFiles := make(map[string]string, len(files)+3)
|
||||||
for path, source := range files {
|
for path, source := range files {
|
||||||
@@ -94,11 +128,42 @@ func Generate(options Options) error {
|
|||||||
projectFiles["schema.sql"] = "templates/schema.sql.tmpl"
|
projectFiles["schema.sql"] = "templates/schema.sql.tmpl"
|
||||||
projectFiles["internal/servicecontext/databaseIntegration_test.go"] = "templates/databaseIntegration_test.go.tmpl"
|
projectFiles["internal/servicecontext/databaseIntegration_test.go"] = "templates/databaseIntegration_test.go.tmpl"
|
||||||
}
|
}
|
||||||
|
if modules {
|
||||||
|
projectFiles["cmd/server/main.go"] = "templates/modules-main.go.tmpl"
|
||||||
|
projectFiles["internal/settings/config.go"] = "templates/modules-settings.go.tmpl"
|
||||||
|
projectFiles["internal/settings/config_test.go"] = "templates/modules-settings_test.go.tmpl"
|
||||||
|
projectFiles["etc/config.yaml"] = "templates/modules-config.yaml.tmpl"
|
||||||
|
projectFiles[".gitignore"] = "templates/modules-gitignore.tmpl"
|
||||||
|
projectFiles["MODULES.md"] = "templates/modules-README.md.tmpl"
|
||||||
|
if storage != "none" {
|
||||||
|
projectFiles["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||||
|
projectFiles["internal/servicecontext/storage_test.go"] = "templates/modules-storage_test.go.tmpl"
|
||||||
|
projectFiles["internal/service/files.go"] = "templates/modules-filesService.go.tmpl"
|
||||||
|
projectFiles["internal/handler/files.go"] = "templates/modules-filesHandler.go.tmpl"
|
||||||
|
projectFiles["internal/handler/files_test.go"] = "templates/modules-filesHandler_test.go.tmpl"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if options.Force {
|
||||||
|
// Do not leave old generated Go files that reference dependencies omitted
|
||||||
|
// by this profile, and do not silently delete potentially edited code.
|
||||||
|
for _, path := range []string{"internal/repository/mysql.go", "internal/servicecontext/databaseIntegration_test.go", "schema.sql", "internal/settings/config.go", "internal/settings/config_test.go", "etc/config.yaml", "MODULES.md", "internal/servicecontext/storage.go", "internal/servicecontext/storage_test.go", "internal/service/files.go", "internal/handler/files.go", "internal/handler/files_test.go"} {
|
||||||
|
if _, emitted := projectFiles[path]; emitted {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(filepath.Join(directory, path)); err == nil {
|
||||||
|
return fmt.Errorf("existing %s conflicts with the selected profile; generate into a new directory instead of removing existing code", path)
|
||||||
|
} else if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
paths := make([]string, 0, len(projectFiles))
|
paths := make([]string, 0, len(projectFiles))
|
||||||
for path := range projectFiles {
|
for path := range projectFiles {
|
||||||
paths = append(paths, path)
|
paths = append(paths, path)
|
||||||
}
|
}
|
||||||
sort.Strings(paths)
|
sort.Strings(paths)
|
||||||
|
// Render and format the whole project before touching the target directory.
|
||||||
|
contents := make(map[string]string, len(paths))
|
||||||
for _, path := range paths {
|
for _, path := range paths {
|
||||||
content, err := render(projectFiles[path], data)
|
content, err := render(projectFiles[path], data)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -111,6 +176,9 @@ func Generate(options Options) error {
|
|||||||
}
|
}
|
||||||
content = string(formatted)
|
content = string(formatted)
|
||||||
}
|
}
|
||||||
|
contents[path] = content
|
||||||
|
}
|
||||||
|
for _, path := range paths {
|
||||||
target := filepath.Join(directory, filepath.FromSlash(path))
|
target := filepath.Join(directory, filepath.FromSlash(path))
|
||||||
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
||||||
return fmt.Errorf("create %s: %w", filepath.Dir(path), err)
|
return fmt.Errorf("create %s: %w", filepath.Dir(path), err)
|
||||||
@@ -122,7 +190,7 @@ func Generate(options Options) error {
|
|||||||
return fmt.Errorf("check %s: %w", target, err)
|
return fmt.Errorf("check %s: %w", target, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if err := os.WriteFile(target, []byte(content), 0o644); err != nil {
|
if err := os.WriteFile(target, []byte(contents[path]), 0o644); err != nil {
|
||||||
return fmt.Errorf("write %s: %w", target, err)
|
return fmt.Errorf("write %s: %w", target, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -130,10 +198,15 @@ func Generate(options Options) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type templateData struct {
|
type templateData struct {
|
||||||
|
API bool
|
||||||
Module string
|
Module string
|
||||||
FrameworkVersion string
|
FrameworkVersion string
|
||||||
FrameworkReplace string
|
FrameworkReplace string
|
||||||
Database string
|
Database string
|
||||||
|
Modules bool
|
||||||
|
LogOutput string
|
||||||
|
LogFormat string
|
||||||
|
Storage string
|
||||||
}
|
}
|
||||||
|
|
||||||
func render(name string, data templateData) (string, error) {
|
func render(name string, data templateData) (string, error) {
|
||||||
@@ -202,11 +275,28 @@ func canonicalPath(path string) string {
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return path
|
return path
|
||||||
}
|
}
|
||||||
resolved, err := filepath.EvalSymlinks(abs)
|
// Resolve the nearest existing ancestor too: generation now renders before
|
||||||
if err == nil {
|
// creating directories, and macOS /var commonly resolves through /private.
|
||||||
return resolved
|
candidate := abs
|
||||||
|
var tail []string
|
||||||
|
for {
|
||||||
|
resolved, err := filepath.EvalSymlinks(candidate)
|
||||||
|
if err == nil {
|
||||||
|
for i := len(tail) - 1; i >= 0; i-- {
|
||||||
|
resolved = filepath.Join(resolved, tail[i])
|
||||||
|
}
|
||||||
|
return resolved
|
||||||
|
}
|
||||||
|
if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return abs
|
||||||
|
}
|
||||||
|
parent := filepath.Dir(candidate)
|
||||||
|
if parent == candidate {
|
||||||
|
return abs
|
||||||
|
}
|
||||||
|
tail = append(tail, filepath.Base(candidate))
|
||||||
|
candidate = parent
|
||||||
}
|
}
|
||||||
return abs
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func findFrameworkRoot() string {
|
func findFrameworkRoot() string {
|
||||||
|
|||||||
@@ -1,10 +1,13 @@
|
|||||||
# GoOmni service
|
# GoOmni service
|
||||||
|
|
||||||
This service was generated with `go-omni new`.
|
This service was generated with `go-omni new`.
|
||||||
|
{{if .Modules}}
|
||||||
|
**本项目启用了模块化配置,启动步骤和环境变量请先看 [MODULES.md](MODULES.md)。** 配置来自 `etc/config.yaml`,文件模块需要配置环境密钥后启动。
|
||||||
|
{{end}}
|
||||||
|
|
||||||
Framework dependency: `git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}`.
|
Framework dependency: `git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}`.
|
||||||
|
|
||||||
Storage: `{{.Database}}`.
|
Database: `{{.Database}}`.
|
||||||
|
|
||||||
{{if eq .Database "mysql"}}
|
{{if eq .Database "mysql"}}
|
||||||
先创建业务数据库,再执行项目的建表脚本(替换示例数据库名、账号和地址):
|
先创建业务数据库,再执行项目的建表脚本(替换示例数据库名、账号和地址):
|
||||||
|
|||||||
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
@@ -0,0 +1,69 @@
|
|||||||
|
package servicecontext
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"sync"
|
||||||
|
{{if ne .Storage "none"}}"net/http"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||||
|
{{if eq .Database "mysql"}}"time"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"github.com/go-sql-driver/mysql"{{end}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
"{{.Module}}/internal/middleware"
|
||||||
|
)
|
||||||
|
// Context owns infrastructure, while Logger is borrowed from main.
|
||||||
|
// Add business repositories here; HTTP contracts do not define database tables.
|
||||||
|
type Context struct {
|
||||||
|
Logger *logx.Logger
|
||||||
|
Health *health.Checker
|
||||||
|
Middlewares map[string]httpx.Middleware
|
||||||
|
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
Files storage.Store
|
||||||
|
Uploads *storage.Uploader
|
||||||
|
Downloads storage.Presigner
|
||||||
|
UploadLinks storage.UploadPresigner
|
||||||
|
FileHTTP http.Handler
|
||||||
|
storageClose func()error
|
||||||
|
{{end}}
|
||||||
|
closeMu sync.Mutex
|
||||||
|
closed bool
|
||||||
|
closeOnce sync.Once
|
||||||
|
closeErr error
|
||||||
|
}
|
||||||
|
// Pass the settings loaded by main. The optional form supports lightweight
|
||||||
|
// tests without infrastructure; selected adapters still require valid settings.
|
||||||
|
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
|
||||||
|
if ctx==nil{ctx=context.Background()}
|
||||||
|
if err=ctx.Err();err!=nil{return nil,err}
|
||||||
|
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
|
||||||
|
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||||
|
_=cfg
|
||||||
|
checker,err:=health.New();if err!=nil{return nil,err}
|
||||||
|
c:=&Context{Logger:logger,Health:checker,Middlewares:middleware.Defaults()}
|
||||||
|
defer func(){if err!=nil{err=errors.Join(err,c.Close())}}()
|
||||||
|
{{if eq .Database "mysql"}}
|
||||||
|
if err=cfg.Validate();err!=nil{return nil,err}
|
||||||
|
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
|
||||||
|
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
|
||||||
|
dsn.ParseTime=true;dsn.Loc=time.UTC
|
||||||
|
cfg.Database.DSN=dsn.FormatDSN()
|
||||||
|
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
|
||||||
|
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
|
||||||
|
{{end}}
|
||||||
|
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
|
||||||
|
return c,nil
|
||||||
|
}
|
||||||
|
// Close is idempotent, including startup failure and graceful shutdown.
|
||||||
|
func(c *Context)Close()error{
|
||||||
|
if c==nil{return nil}
|
||||||
|
c.closeMu.Lock();defer c.closeMu.Unlock();c.closed=true
|
||||||
|
c.closeOnce.Do(func(){
|
||||||
|
{{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}}
|
||||||
|
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||||
|
})
|
||||||
|
return c.closeErr
|
||||||
|
}
|
||||||
@@ -23,7 +23,10 @@ func (s *Users) Create(ctx context.Context, input model.UserInput) (model.User,
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return model.User{}, err
|
return model.User{}, err
|
||||||
}
|
}
|
||||||
return s.services.Users.Create(ctx, input)
|
user, err := s.services.Users.Create(ctx, input)
|
||||||
|
if err != nil { s.services.Logger.ErrorContext(ctx,"user.create.failed","error",err); return model.User{},err }
|
||||||
|
s.services.Logger.InfoContext(ctx,"user.created","userId",user.ID)
|
||||||
|
return user,nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Users) Get(ctx context.Context, id int64) (model.User, error) {
|
func (s *Users) Get(ctx context.Context, id int64) (model.User, error) {
|
||||||
|
|||||||
@@ -2,6 +2,10 @@ package servicecontext
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
{{if and .Modules (ne .Storage "none")}}"errors"
|
||||||
|
"net/http"
|
||||||
|
"sync"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||||
{{if eq .Database "mysql"}}"fmt"
|
{{if eq .Database "mysql"}}"fmt"
|
||||||
"time"
|
"time"
|
||||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
@@ -17,6 +21,18 @@ type Context struct {
|
|||||||
Logger *logx.Logger
|
Logger *logx.Logger
|
||||||
Users repository.UserRepository
|
Users repository.UserRepository
|
||||||
Health *health.Checker
|
Health *health.Checker
|
||||||
|
{{if and .Modules (ne .Storage "none")}}
|
||||||
|
Files storage.Store
|
||||||
|
Uploads *storage.Uploader
|
||||||
|
Downloads storage.Presigner
|
||||||
|
UploadLinks storage.UploadPresigner
|
||||||
|
FileHTTP http.Handler
|
||||||
|
storageClose func() error
|
||||||
|
closeMu sync.Mutex
|
||||||
|
closed bool
|
||||||
|
closeOnce sync.Once
|
||||||
|
closeErr error
|
||||||
|
{{end}}
|
||||||
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -61,6 +77,15 @@ func New(ctx context.Context, logger *logx.Logger) (*Context, error) {
|
|||||||
|
|
||||||
// Close is safe to call on startup failure and again during application exit.
|
// Close is safe to call on startup failure and again during application exit.
|
||||||
func (c *Context) Close() error {
|
func (c *Context) Close() error {
|
||||||
{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}
|
{{if and .Modules (ne .Storage "none")}}
|
||||||
return nil
|
if c==nil{return nil}
|
||||||
|
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||||
|
c.closed=true
|
||||||
|
c.closeOnce.Do(func(){
|
||||||
|
if c.storageClose!=nil {c.closeErr=errors.Join(c.closeErr,c.storageClose())}
|
||||||
|
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||||
|
})
|
||||||
|
return c.closeErr
|
||||||
|
{{else}}{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}{{end}}
|
||||||
|
{{if not (and .Modules (ne .Storage "none"))}}return nil{{end}}
|
||||||
}
|
}
|
||||||
|
|||||||
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
# 按需装配的日志与文件模块
|
||||||
|
|
||||||
|
本项目选用日志输出 `{{.LogOutput}}`、格式 `{{.LogFormat}}`、对象存储 `{{.Storage}}`。数据库选择 `{{.Database}}` 独立于对象存储。
|
||||||
|
|
||||||
|
配置入口为 `etc/config.yaml`,通过 `go run ./cmd/server -config /path/to/config.yaml` 指定其他 JSON/YAML 文件;`-config ''` 只用代码默认值和环境变量。优先级:默认值 → 文件 → 非空环境变量。未知字段、无效参数会在监听前拒绝。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go mod tidy
|
||||||
|
{{if ne .Storage "none"}}# 示例文件接口使用单一共享 Token,不是用户/租户权限系统。
|
||||||
|
export GO_OMNI_FILES_TOKEN="$(openssl rand -hex 32)"
|
||||||
|
{{if eq .Storage "local"}}# 本地签名密钥需要稳定保存;换密钥后旧链接失效。
|
||||||
|
export GO_OMNI_STORAGE_SIGNING_KEY="$(openssl rand -hex 32)"
|
||||||
|
{{else}}# 配置当前存储平台已有的桶和凭据,不会自动创建桶。
|
||||||
|
export GO_OMNI_STORAGE_BUCKET='your-bucket'
|
||||||
|
export GO_OMNI_STORAGE_ACCESS_KEY='your-access-key'
|
||||||
|
export GO_OMNI_STORAGE_SECRET_KEY='your-secret-key'
|
||||||
|
{{if eq .Storage "s3"}}export GO_OMNI_STORAGE_ENDPOINT='s3.us-east-1.amazonaws.com'
|
||||||
|
export GO_OMNI_STORAGE_REGION='us-east-1'
|
||||||
|
{{else if eq .Storage "r2"}}export GO_OMNI_STORAGE_ACCOUNT_ID='your-32-character-account-id'
|
||||||
|
{{else}}export GO_OMNI_STORAGE_REGION='cn-hangzhou'
|
||||||
|
{{end}}{{end}}{{end}}
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
{{if eq .Database "mysql"}}还需按 README 配置独立 MySQL 数据库并应用 schema.sql;应用不自动迁移。{{end}}
|
||||||
|
|
||||||
|
默认仅监听 `127.0.0.1:8080`。`GO_OMNI_HTTP_ADDR` 覆盖地址;本地签名 URL 的外部地址通过 `GO_OMNI_STORAGE_PUBLIC_URL` 单独配置,始终以 `/storage/object` 结尾,不使用请求 Host 自动拼接可信链接。
|
||||||
|
|
||||||
|
## 日志
|
||||||
|
|
||||||
|
业务示例在创建用户/上传文件成功时通过 logx 写结构化日志。Context 原样传递 requestId,框架不自动增加用户、租户等业务字段。文件模式已配置轮转:100 MiB、10 份、7 天、压缩。调整 `server.log` 即可更改;`redactKeys` 可增加业务敏感键。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -H 'Content-Type: application/json' -H 'X-Request-ID: example-request' \
|
||||||
|
-d '{"name":"Example","email":"example@example.com"}' http://127.0.0.1:8080/users
|
||||||
|
```
|
||||||
|
|
||||||
|
`GO_OMNI_LOG_LEVEL`、`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_FILE` 覆盖对应项;显式设置 `GO_OMNI_LOG_OUTPUT=stdout` 且未设置 LOG_FILE 时,切换到控制台并关闭文件轮转。慢请求阈值为 500ms,可通过 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 调整。字段脱敏不扫描 msg/error 中的自由文本;不要把凭据或签名 URL 拼到日志中。
|
||||||
|
|
||||||
|
{{if ne .Storage "none"}}## 文件示例
|
||||||
|
|
||||||
|
Handler → Service → storage 接口;底层构造与连接池集中在 `internal/servicecontext/storage.go`。示例接口要求 `Authorization: Bearer $GO_OMNI_FILES_TOKEN`,它授权这个示例的全部文件,实际业务应替换认证和对象归属检查。存储适配器不承担业务授权。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
printf 'hello go-omni' > /tmp/goOmniExample.txt
|
||||||
|
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" \
|
||||||
|
-H 'Content-Type: text/plain' -H 'X-Request-ID: file-example' \
|
||||||
|
--data-binary @/tmp/goOmniExample.txt http://127.0.0.1:8080/files
|
||||||
|
```
|
||||||
|
|
||||||
|
保存响应中 `data.key`,再请求下载链接(把示例 key 替换为返回值):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||||
|
-d '{"key":"uploads/returned-key"}' http://127.0.0.1:8080/files/downloadLink
|
||||||
|
```
|
||||||
|
|
||||||
|
返回 `data.url`,在有效期内 GET 即可下载,无需再携带 Token。签发前会检查对象存在;URL 默认 5 分钟有效,不写日志,响应设置 no-store。
|
||||||
|
|
||||||
|
需要直传时先申请链接:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||||
|
-d '{"size":13,"contentType":"text/plain"}' http://127.0.0.1:8080/files/uploadLink
|
||||||
|
```
|
||||||
|
|
||||||
|
响应包括随机 `data.key` 和 `data.request`(method/url/headers/expiresAt)。按返回的 method 和 headers 用原始文件字节上传到 URL;不能额外包 FormData。浏览器自动填写 Content-Length,其余签名 Header 原样发送;云桶需配置允许业务域名的 CORS。云端直传只绑定声明类型,实际内容需上传后验证;本地和 API 中转上传会检查允许类型及前缀嗅探结果。未完成内容验证前不要把暂存文件当作可信文件公开。
|
||||||
|
|
||||||
|
| 参数 | 默认值 / 环境变量 |
|
||||||
|
| --- | --- |
|
||||||
|
| 大小 | 32 MiB / GO_OMNI_STORAGE_MAX_BYTES |
|
||||||
|
| 类型 | text/plain、image/png、image/jpeg、application/pdf / GO_OMNI_STORAGE_CONTENT_TYPES(逗号分隔) |
|
||||||
|
| 链接有效期 | 5m / GO_OMNI_STORAGE_LINK_TTL(1s..1h) |
|
||||||
|
| 请求超时 | 30s / GO_OMNI_STORAGE_REQUEST_TIMEOUT(>0 且 <=5m) |
|
||||||
|
| 本地目录 | var/files / GO_OMNI_STORAGE_ROOT |
|
||||||
|
| 本地外部 URL | http://127.0.0.1:8080/storage/object / GO_OMNI_STORAGE_PUBLIC_URL |
|
||||||
|
| 云配置 | GO_OMNI_STORAGE_ENDPOINT、REGION、BUCKET、ACCOUNT_ID、SECURE |
|
||||||
|
| 环境密钥 | GO_OMNI_FILES_TOKEN、GO_OMNI_STORAGE_SIGNING_KEY、GO_OMNI_STORAGE_ACCESS_KEY、SECRET_KEY、SESSION_TOKEN |
|
||||||
|
|
||||||
|
全部 JSON/YAML 字段使用 lowerCamelCase。密钥字段仅允许环境变量,配置文件中写入会报未知字段;不要把运行环境文件提交到仓库。S3 的 endpoint 为 host[:port],OSS 的可选 endpoint 为 HTTPS origin;R2 从 accountId 派生官方 endpoint,region 为 auto。仅 S3/MinIO 的本地测试可设置 SECURE=false。
|
||||||
|
|
||||||
|
错误响应:未授权 401、无 Content-Length 411、超限 413、类型不允许 415、长度不符/非法 key 400、对象不存在 404、存储超时 504,其余内部错误统一隐藏细节。底层签名入口遵循存储协议返回 201/403 等状态,流式传输且访问日志保留实际状态。
|
||||||
|
|
||||||
|
云客户端只在组合处创建,使用专用有界连接池;配置正确不代表桶和网络已经可用,首次真实调用仍可能失败。不创建桶、不修改桶权限、不初始化任何业务数据库。更换平台时替换组合处适配器,业务 Service 接口保持不变。
|
||||||
|
{{end}}
|
||||||
|
|
||||||
|
## 生命周期与验证
|
||||||
|
|
||||||
|
启动失败会清理已创建资源;SIGINT/SIGTERM 后先等待 HTTP 请求退出,再关闭服务依赖,最后关闭日志。只关闭项目自己创建的资源,不关闭全局默认 HTTP Transport。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go test -race ./...
|
||||||
|
go build -o server ./cmd/server
|
||||||
|
./server
|
||||||
|
```
|
||||||
|
|
||||||
|
生成的测试覆盖配置优先级/校验{{if ne .Storage "none"}}、资源关闭幂等性和借用日志所有权、上传与临时链接、签名篡改、限额/类型错误。业务测试使用临时 Local,不访问云账号;云端配置需另做实网验收{{end}}。基础用户 CRUD 示例仍未启用认证,部署前按业务需要替换。
|
||||||
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
@@ -0,0 +1,46 @@
|
|||||||
|
# Non-secret settings only. Environment variables override these values.
|
||||||
|
server:
|
||||||
|
address: '127.0.0.1:8080'
|
||||||
|
enableLogger: true
|
||||||
|
slowRequestThreshold: 500ms
|
||||||
|
shutdownTimeout: 10s
|
||||||
|
log:
|
||||||
|
service: {{printf "%q" .Module}}
|
||||||
|
level: info
|
||||||
|
format: {{.LogFormat}}
|
||||||
|
{{if eq .LogOutput "file"}} file: './var/log/server.log'
|
||||||
|
rotation:
|
||||||
|
enabled: true
|
||||||
|
maxSizeMb: 100
|
||||||
|
maxBackups: 10
|
||||||
|
maxAgeDays: 7
|
||||||
|
compress: true
|
||||||
|
{{else}} output: {{.LogOutput}}
|
||||||
|
{{end}} redactKeys: []
|
||||||
|
{{if ne .Storage "none"}}storage:
|
||||||
|
provider: {{.Storage}}
|
||||||
|
maxBytes: 33554432
|
||||||
|
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||||
|
linkTtl: 5m
|
||||||
|
requestTimeout: 30s
|
||||||
|
{{if eq .Storage "local"}} root: './var/files'
|
||||||
|
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||||
|
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
|
||||||
|
secure: true
|
||||||
|
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
|
||||||
|
region: auto
|
||||||
|
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
|
||||||
|
region: us-east-1
|
||||||
|
{{else}} region: cn-hangzhou
|
||||||
|
endpoint: '' # empty uses the official regional endpoint
|
||||||
|
{{end}}{{end}}{{end}}
|
||||||
|
|
||||||
|
{{if and .API (eq .Database "mysql")}}database:
|
||||||
|
driver: mysql
|
||||||
|
maxOpenConns: 10
|
||||||
|
maxIdleConns: 5
|
||||||
|
connMaxLifetime: 30m
|
||||||
|
connMaxIdleTime: 5m
|
||||||
|
pingTimeout: 5s
|
||||||
|
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||||
|
{{end}}
|
||||||
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
@@ -0,0 +1,86 @@
|
|||||||
|
package handler
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/subtle"
|
||||||
|
"errors"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"{{.Module}}/internal/service"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Files struct {service *service.Files; maxBytes int64; timeout time.Duration}
|
||||||
|
func NewFiles(service *service.Files,maxBytes int64,timeout time.Duration)*Files{return &Files{service:service,maxBytes:maxBytes,timeout:timeout}}
|
||||||
|
|
||||||
|
// RegisterRoutes protects example control endpoints with one shared demo token.
|
||||||
|
// Replace this gate with project authentication and object ownership checks.
|
||||||
|
// Signed object URLs carry their own short-lived authorization.
|
||||||
|
func (h *Files) RegisterRoutes(application *omni.App,token string,signed http.Handler)error {
|
||||||
|
if len(token)<32{return errors.New("file example requires a token of at least 32 bytes")}
|
||||||
|
bounded:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||||
|
ctx,cancel:=context.WithTimeout(c.Context(),h.timeout);defer cancel();c.WithContext(ctx);return next(c)
|
||||||
|
}}
|
||||||
|
authorized:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||||
|
if len(c.Request().Header.Values("Authorization"))!=1 || subtle.ConstantTimeCompare([]byte(c.Header("Authorization")),[]byte("Bearer "+token))!=1{return omni.Unauthorized("UNAUTHORIZED","invalid file access token")}
|
||||||
|
return next(c)
|
||||||
|
}}
|
||||||
|
group:=application.Group("/files",authorized,bounded)
|
||||||
|
group.POST("",h.Upload)
|
||||||
|
group.POST("/uploadLink",h.UploadLink)
|
||||||
|
group.POST("/downloadLink",h.DownloadLink)
|
||||||
|
if signed!=nil {application.Any("/storage/object",func(c *omni.Context)error{
|
||||||
|
signed.ServeHTTP(contextWriter{c},c.Request());return nil
|
||||||
|
},bounded)}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (h *Files) Upload(c *omni.Context)error {
|
||||||
|
size:=c.Request().ContentLength
|
||||||
|
if size<0{return omni.NewError(http.StatusLengthRequired,"LENGTH_REQUIRED","Content-Length is required")}
|
||||||
|
if size>h.maxBytes{return omni.NewError(http.StatusRequestEntityTooLarge,"FILE_TOO_LARGE","file exceeds upload limit")}
|
||||||
|
body:=http.MaxBytesReader(c.Raw(),c.Request().Body,h.maxBytes)
|
||||||
|
defer body.Close()
|
||||||
|
info,err:=h.service.Upload(c.Context(),body,size,c.Header("Content-Type"))
|
||||||
|
if err!=nil{return fileError(err)}
|
||||||
|
return c.Created(info)
|
||||||
|
}
|
||||||
|
func (h *Files) UploadLink(c *omni.Context)error {
|
||||||
|
var input struct{Size int64 `json:"size" binding:"gte=0"`;ContentType string `json:"contentType" binding:"required,max=256"`}
|
||||||
|
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||||
|
link,err:=h.service.UploadLink(c.Context(),input.Size,input.ContentType)
|
||||||
|
if err!=nil{return fileError(err)}
|
||||||
|
c.SetResponseHeader("Cache-Control","no-store")
|
||||||
|
return c.OK(link)
|
||||||
|
}
|
||||||
|
func (h *Files) DownloadLink(c *omni.Context)error {
|
||||||
|
var input struct{Key string `json:"key" binding:"required,max=1024"`}
|
||||||
|
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||||
|
if err:=storage.Check(c.Context(),input.Key);err!=nil{return omni.BadRequest("INVALID_KEY","invalid object key")}
|
||||||
|
link,err:=h.service.DownloadLink(c.Context(),input.Key)
|
||||||
|
if err!=nil{return fileError(err)}
|
||||||
|
c.SetResponseHeader("Cache-Control","no-store")
|
||||||
|
return c.OK(map[string]string{"url":link})
|
||||||
|
}
|
||||||
|
func fileError(err error)error {
|
||||||
|
var limit *http.MaxBytesError
|
||||||
|
switch {
|
||||||
|
case errors.Is(err,storage.ErrTooLarge),errors.As(err,&limit):return omni.NewError(413,"FILE_TOO_LARGE","file exceeds upload limit").WithCause(err)
|
||||||
|
case errors.Is(err,storage.ErrContentType):return omni.NewError(415,"CONTENT_TYPE_NOT_ALLOWED","file content type is not allowed").WithCause(err)
|
||||||
|
case errors.Is(err,storage.ErrSizeMismatch):return omni.BadRequest("SIZE_MISMATCH","file size does not match Content-Length").WithCause(err)
|
||||||
|
case errors.Is(err,storage.ErrNotFound):return omni.NotFound("FILE_NOT_FOUND","file was not found").WithCause(err)
|
||||||
|
case errors.Is(err,context.DeadlineExceeded):return omni.NewError(504,"STORAGE_TIMEOUT","storage operation timed out").WithCause(err)
|
||||||
|
default:return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Track raw streaming handler responses through Omni so access logs retain the
|
||||||
|
// actual status (including failed signatures), without buffering object bodies.
|
||||||
|
type contextWriter struct{ctx *omni.Context}
|
||||||
|
func (w contextWriter) Header()http.Header{return w.ctx.Raw().Header()}
|
||||||
|
func (w contextWriter) WriteHeader(status int){_ = w.ctx.WriteResponse(status,nil,nil)}
|
||||||
|
func (w contextWriter) Write(data []byte)(int,error){
|
||||||
|
if !w.ctx.Responded(){if err:=w.ctx.WriteResponse(http.StatusOK,nil,nil);err!=nil{return 0,err}}
|
||||||
|
return w.ctx.Raw().Write(data)
|
||||||
|
}
|
||||||
@@ -0,0 +1,75 @@
|
|||||||
|
package handler_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"net/url"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
"{{.Module}}/internal/handler"
|
||||||
|
"{{.Module}}/internal/service"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Exercise business code using Local even when production selects a cloud
|
||||||
|
// adapter. These tests do not use cloud accounts or network credentials.
|
||||||
|
func TestFileUploadAndLinks(t *testing.T){
|
||||||
|
files,err:=local.New(t.TempDir(),64);if err!=nil{t.Fatal(err)}
|
||||||
|
policy:=storage.UploadPolicy{MaxBytes:64,ContentTypes:[]string{"text/plain"}}
|
||||||
|
signer,err:=storage.NewUploadSigner(files,"http://localhost/storage/object",bytes.Repeat([]byte{1},32),policy);if err!=nil{t.Fatal(err)}
|
||||||
|
uploader,err:=storage.NewUploader(files,policy);if err!=nil{t.Fatal(err)}
|
||||||
|
var logs bytes.Buffer
|
||||||
|
logger,err:=logx.New(logx.Config{Writer:&logs});if err!=nil{t.Fatal(err)}
|
||||||
|
logx.SetDefault(logger);defer logx.SetDefault(nil)
|
||||||
|
app:=omni.New(omni.WithLogger(true))
|
||||||
|
token:=strings.Repeat("t",32)
|
||||||
|
svc:=service.NewFiles(files,uploader,signer,signer,logger,time.Minute)
|
||||||
|
if err=handler.NewFiles(svc,64,time.Second).RegisterRoutes(app,token,signer);err!=nil{t.Fatal(err)}
|
||||||
|
request:=func(method,path,body,contentType string,authorized bool)*httptest.ResponseRecorder{
|
||||||
|
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type",contentType);r.Header.Set("X-Request-ID","upload-test")
|
||||||
|
if authorized{r.Header.Set("Authorization","Bearer "+token)}
|
||||||
|
w:=httptest.NewRecorder();app.ServeHTTP(w,r);return w
|
||||||
|
}
|
||||||
|
if w:=request("POST","/files","hello","text/plain",false);w.Code!=401{t.Fatal(w.Code)}
|
||||||
|
w:=request("POST","/files","hello","text/plain",true)
|
||||||
|
var saved struct{Data storage.Info}
|
||||||
|
if w.Code!=201 || json.Unmarshal(w.Body.Bytes(),&saved)!=nil || saved.Data.Size!=5{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
raw,_:=json.Marshal(map[string]string{"key":saved.Data.Key})
|
||||||
|
w=request("POST","/files/downloadLink",string(raw),"application/json",true)
|
||||||
|
var download struct{Data struct{URL string}}
|
||||||
|
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&download)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
w=request("GET",download.Data.URL,"","",false)
|
||||||
|
if w.Code!=200 || w.Body.String()!="hello"{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
u,_:=url.Parse(download.Data.URL);q:=u.Query();q.Set("signature","tampered");u.RawQuery=q.Encode()
|
||||||
|
if w=request("GET",u.String(),"","",false);w.Code!=403{t.Fatal(w.Code)}
|
||||||
|
for _,tc:=range []struct{path,body,contentType string;status int}{
|
||||||
|
{"/files",strings.Repeat("x",65),"text/plain",413},
|
||||||
|
{"/files","hello","image/png",415},
|
||||||
|
{"/files","<html>hello</html>","text/plain",415},
|
||||||
|
{"/files/downloadLink",`{"key":"../bad"}`,"application/json",400},
|
||||||
|
{"/files/downloadLink",`{"key":"missing"}`,"application/json",404},
|
||||||
|
{"/files/uploadLink",`{"size":65,"contentType":"text/plain"}`,"application/json",413},
|
||||||
|
{"/files/uploadLink",`{"size":5,"contentType":"image/png"}`,"application/json",415},
|
||||||
|
}{if w=request("POST",tc.path,tc.body,tc.contentType,true);w.Code!=tc.status{t.Fatal(tc.path,w.Code,w.Body.String())}}
|
||||||
|
w=request("POST","/files/uploadLink",`{"size":5,"contentType":"text/plain"}`,"application/json",true)
|
||||||
|
var direct struct{Data service.FileUploadLink}
|
||||||
|
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&direct)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
if w=request("PUT",direct.Data.Request.URL,"world","text/plain",false);w.Code!=201{t.Fatal(w.Code,w.Body.String())}
|
||||||
|
object,err:=files.Get(context.Background(),direct.Data.Key);if err!=nil{t.Fatal(err)};body,err:=io.ReadAll(object.Body);object.Body.Close();if err!=nil || string(body)!="world"{t.Fatal(err,string(body))}
|
||||||
|
decoder:=json.NewDecoder(&logs);sawBusiness,sawDenied:=false,false
|
||||||
|
for {var entry map[string]any;err:=decoder.Decode(&entry);if err==io.EOF{break};if err!=nil{t.Fatal(err)}
|
||||||
|
if entry["requestId"]!="upload-test"{t.Fatal("lost request correlation")}
|
||||||
|
if entry["msg"]=="file.uploaded"{sawBusiness=true}
|
||||||
|
if entry["msg"]=="http.request" && entry["path"]=="/storage/object" && entry["status"]==float64(http.StatusForbidden){sawDenied=true}
|
||||||
|
}
|
||||||
|
if !sawBusiness || !sawDenied{t.Fatal("business log or signed-handler status missing")}
|
||||||
|
}
|
||||||
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
@@ -0,0 +1,61 @@
|
|||||||
|
package service
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"io"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Files borrows provider-neutral capabilities; it never creates or closes clients.
|
||||||
|
// Authentication/ownership rules belong to this project, outside storage adapters.
|
||||||
|
type Files struct {
|
||||||
|
store storage.Store
|
||||||
|
uploader *storage.Uploader
|
||||||
|
downloads storage.Presigner
|
||||||
|
uploadLinks storage.UploadPresigner
|
||||||
|
logger *logx.Logger
|
||||||
|
ttl time.Duration
|
||||||
|
}
|
||||||
|
func NewFiles(store storage.Store,uploader *storage.Uploader,downloads storage.Presigner,uploadLinks storage.UploadPresigner,logger *logx.Logger,ttl time.Duration)*Files{
|
||||||
|
return &Files{store:store,uploader:uploader,downloads:downloads,uploadLinks:uploadLinks,logger:logger.With("component","files"),ttl:ttl}
|
||||||
|
}
|
||||||
|
func newFileKey()(string,error){
|
||||||
|
var id [16]byte
|
||||||
|
if _,err:=rand.Read(id[:]);err!=nil{return "",err}
|
||||||
|
return "uploads/"+hex.EncodeToString(id[:]),nil
|
||||||
|
}
|
||||||
|
func (s *Files) Upload(ctx context.Context,body io.Reader,size int64,contentType string)(storage.Info,error){
|
||||||
|
key,err:=newFileKey();if err!=nil{return storage.Info{},err}
|
||||||
|
info,err:=s.uploader.Put(ctx,key,body,size,contentType)
|
||||||
|
if err!=nil{s.logger.WarnContext(ctx,"file.upload.failed","error",err);return storage.Info{},err}
|
||||||
|
s.logger.InfoContext(ctx,"file.uploaded","key",info.Key,"size",info.Size,"contentType",info.ContentType)
|
||||||
|
return info,nil
|
||||||
|
}
|
||||||
|
type FileUploadLink struct {
|
||||||
|
Key string `json:"key"`
|
||||||
|
Request storage.UploadRequest `json:"request"`
|
||||||
|
}
|
||||||
|
func (s *Files) UploadLink(ctx context.Context,size int64,contentType string)(FileUploadLink,error){
|
||||||
|
key,err:=newFileKey();if err!=nil{return FileUploadLink{},err}
|
||||||
|
// Check the same policy before issuing a local or cloud capability.
|
||||||
|
// Cloud byte-level checks must happen in a post-upload validation workflow.
|
||||||
|
request,err:=s.uploadLinks.PresignPut(ctx,key,size,contentType,s.ttl)
|
||||||
|
if err!=nil{return FileUploadLink{},err}
|
||||||
|
s.logger.InfoContext(ctx,"file.uploadLink.issued","key",key)
|
||||||
|
return FileUploadLink{Key:key,Request:request},nil
|
||||||
|
}
|
||||||
|
func (s *Files) DownloadLink(ctx context.Context,key string)(string,error){
|
||||||
|
if err:=storage.Check(ctx,key);err!=nil{return "",err}
|
||||||
|
// Confirm the object exists without buffering its contents.
|
||||||
|
object,err:=s.store.Get(ctx,key);if err!=nil{return "",err}
|
||||||
|
if err=object.Body.Close();err!=nil{return "",err}
|
||||||
|
link,err:=s.downloads.PresignGet(ctx,key,s.ttl)
|
||||||
|
if err!=nil{return "",err}
|
||||||
|
s.logger.InfoContext(ctx,"file.downloadLink.issued","key",key)
|
||||||
|
return link.String(),nil
|
||||||
|
}
|
||||||
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
/var/
|
||||||
|
/server
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
*.local.yaml
|
||||||
|
*.local.json
|
||||||
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
@@ -0,0 +1,50 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"{{.Module}}/internal/handler"
|
||||||
|
"{{.Module}}/internal/service"
|
||||||
|
"{{.Module}}/internal/servicecontext"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main(){
|
||||||
|
path:=flag.String("config","etc/config.yaml","JSON/YAML settings; empty uses defaults and environment")
|
||||||
|
flag.Parse()
|
||||||
|
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||||
|
}
|
||||||
|
func run(path string)(runErr error){
|
||||||
|
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
cfg,err:=settings.Load(path)
|
||||||
|
if err!=nil{return err}
|
||||||
|
logger,err:=logx.New(cfg.Server.Log)
|
||||||
|
if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||||
|
logx.SetDefault(logger)
|
||||||
|
services,err:=servicecontext.New(ctx,logger)
|
||||||
|
if err!=nil{return err}
|
||||||
|
defer func(){runErr=errors.Join(runErr,services.Close())}()
|
||||||
|
{{if ne .Storage "none"}}if err=services.InitStorage(ctx,cfg.Storage);err!=nil{return err}{{end}}
|
||||||
|
application:=omni.New(omni.WithConfig(cfg.Server))
|
||||||
|
application.OnShutdown(logger.Close)
|
||||||
|
application.OnShutdown(services.Close)
|
||||||
|
application.GET("/health/live",services.Health.LivenessHandler())
|
||||||
|
application.GET("/health/ready",services.Health.ReadinessHandler())
|
||||||
|
handler.NewUsers(service.NewUsers(services)).RegisterRoutes(application)
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
files:=service.NewFiles(services.Files,services.Uploads,services.Downloads,services.UploadLinks,logger,cfg.Storage.LinkTTL)
|
||||||
|
if err=handler.NewFiles(files,cfg.Storage.MaxBytes,cfg.Storage.RequestTimeout).RegisterRoutes(application,cfg.Storage.FilesToken,services.FileHTTP);err!=nil{return err}
|
||||||
|
{{end}}
|
||||||
|
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||||
|
return application.RunContext(ctx,cfg.Server.Address)
|
||||||
|
}
|
||||||
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
@@ -0,0 +1,140 @@
|
|||||||
|
// Package settings owns this project's configuration; it adds no business identity model.
|
||||||
|
package settings
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
{{if ne .Storage "none"}}"fmt"
|
||||||
|
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||||
|
"strconv"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||||
|
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Config struct {
|
||||||
|
Server config.Config `json:"server"`
|
||||||
|
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
|
||||||
|
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
|
||||||
|
}
|
||||||
|
|
||||||
|
func Default() Config {
|
||||||
|
server:=config.Default()
|
||||||
|
server.Address="127.0.0.1:8080"
|
||||||
|
server.EnableLogger=true
|
||||||
|
server.SlowRequestThreshold=500*time.Millisecond
|
||||||
|
server.Log.Service={{printf "%q" .Module}}
|
||||||
|
server.Log.Format={{printf "%q" .LogFormat}}
|
||||||
|
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
|
||||||
|
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
|
||||||
|
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
|
||||||
|
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||||
|
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
|
||||||
|
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
|
||||||
|
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
|
||||||
|
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
|
||||||
|
{{if eq .Storage "r2"}}Region:"auto",{{end}}
|
||||||
|
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
|
||||||
|
}{{end}}}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||||
|
// Pass an empty path to use only defaults and environment.
|
||||||
|
func Load(path string)(Config,error) {
|
||||||
|
cfg:=Default()
|
||||||
|
var err error
|
||||||
|
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
|
||||||
|
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
|
||||||
|
if err!=nil{return Config{},err}
|
||||||
|
// A stream explicitly selected by environment overrides a file profile.
|
||||||
|
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
|
||||||
|
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
|
||||||
|
}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
|
||||||
|
{{end}}
|
||||||
|
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
|
||||||
|
if err=cfg.Validate();err!=nil{return Config{},err}
|
||||||
|
return cfg,nil
|
||||||
|
}
|
||||||
|
func (c Config) Validate()error {
|
||||||
|
{{if and .API (eq .Database "mysql")}}
|
||||||
|
if err:=c.Database.Validate();err!=nil{return err}
|
||||||
|
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
|
||||||
|
{{end}}
|
||||||
|
if err:=c.Server.Validate();err!=nil{return err}
|
||||||
|
if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")}
|
||||||
|
switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")}
|
||||||
|
{{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}}
|
||||||
|
}
|
||||||
|
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
type StorageConfig struct {
|
||||||
|
Provider string `json:"provider"`
|
||||||
|
Root string `json:"root"`
|
||||||
|
PublicURL string `json:"publicUrl"`
|
||||||
|
Endpoint string `json:"endpoint"`
|
||||||
|
Region string `json:"region"`
|
||||||
|
Bucket string `json:"bucket"`
|
||||||
|
AccountID string `json:"accountId"`
|
||||||
|
Secure bool `json:"secure"`
|
||||||
|
MaxBytes int64 `json:"maxBytes"`
|
||||||
|
ContentTypes []string `json:"contentTypes"`
|
||||||
|
LinkTTL time.Duration `json:"linkTtl"`
|
||||||
|
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||||
|
// Secrets are environment-only and cannot be populated by config files.
|
||||||
|
FilesToken string `json:"-"`
|
||||||
|
SigningKey string `json:"-"`
|
||||||
|
AccessKey string `json:"-"`
|
||||||
|
SecretKey string `json:"-"`
|
||||||
|
SessionToken string `json:"-"`
|
||||||
|
}
|
||||||
|
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||||
|
func (s *StorageConfig) applyEnv()error {
|
||||||
|
values:=map[string]*string{
|
||||||
|
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
|
||||||
|
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
|
||||||
|
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
|
||||||
|
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
|
||||||
|
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
|
||||||
|
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
|
||||||
|
}
|
||||||
|
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
|
||||||
|
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
|
||||||
|
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
|
||||||
|
}
|
||||||
|
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
|
||||||
|
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
|
||||||
|
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (s StorageConfig) Validate()error {
|
||||||
|
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
|
||||||
|
if err:=s.Policy().Validate();err!=nil{return err}
|
||||||
|
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||||
|
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||||
|
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||||
|
{{if eq .Storage "local"}}
|
||||||
|
if s.Root=="" {return errors.New("storage.root is required")}
|
||||||
|
u,err:=url.Parse(s.PublicURL)
|
||||||
|
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||||
|
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||||
|
{{else}}
|
||||||
|
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||||
|
{{if eq .Storage "r2"}}
|
||||||
|
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||||
|
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||||
|
{{else if eq .Storage "s3"}}
|
||||||
|
u,err:=url.Parse("https://"+s.Endpoint)
|
||||||
|
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
|
||||||
|
{{else if eq .Storage "oss"}}
|
||||||
|
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
|
||||||
|
if !s.Secure{return errors.New("OSS requires secure=true")}
|
||||||
|
{{end}}
|
||||||
|
{{end}}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
{{end}}
|
||||||
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
package settings
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
func environment(t *testing.T){
|
||||||
|
t.Helper()
|
||||||
|
for _,entry:=range os.Environ(){key,_,_:=strings.Cut(entry,"=");if strings.HasPrefix(key,"GO_OMNI_"){t.Setenv(key,"")}}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
t.Setenv("GO_OMNI_FILES_TOKEN",strings.Repeat("t",32))
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY",strings.Repeat("s",32))
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","fixtureAccess")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","fixtureSecret")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_BUCKET","testbucket")
|
||||||
|
{{if eq .Storage "s3"}}t.Setenv("GO_OMNI_STORAGE_ENDPOINT","127.0.0.1:9000"){{end}}
|
||||||
|
{{if eq .Storage "r2"}}t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID",strings.Repeat("a",32)){{end}}
|
||||||
|
{{end}}
|
||||||
|
}
|
||||||
|
func TestSettingsPrecedenceAndStrictness(t *testing.T){
|
||||||
|
environment(t)
|
||||||
|
path:=filepath.Join(t.TempDir(),"config.yaml")
|
||||||
|
if err:=os.WriteFile(path,[]byte("server:\n address: '127.0.0.1:8123'\n log:\n level: debug\n"),0600);err!=nil{t.Fatal(err)}
|
||||||
|
t.Setenv("GO_OMNI_HTTP_ADDR","127.0.0.1:9123")
|
||||||
|
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout")
|
||||||
|
cfg,err:=Load(path);if err!=nil{t.Fatal(err)}
|
||||||
|
if cfg.Server.Address!="127.0.0.1:9123" || cfg.Server.Log.Level!="debug" || cfg.Server.Log.Output!="stdout" || cfg.Server.Log.File!="" || cfg.Server.Log.Rotation.Enabled {t.Fatal("configuration precedence failed")}
|
||||||
|
for _,body:=range []string{"unexpected: true","server:\n slowRequestThreshold: -1s","server:\n log:\n format: invalid"{{if ne .Storage "none"}},"storage:\n filesToken: forbidden","storage:\n maxBytes: -1","storage:\n provider: unsupported"{{end}}}{
|
||||||
|
if err:=os.WriteFile(path,[]byte(body),0600);err!=nil{t.Fatal(err)}
|
||||||
|
if _,err:=Load(path);err==nil{t.Fatal("invalid configuration accepted")}
|
||||||
|
}
|
||||||
|
{{if ne .Storage "none"}}
|
||||||
|
t.Setenv("GO_OMNI_FILES_TOKEN","")
|
||||||
|
if _,err:=Load("");err==nil{t.Fatal("missing file token accepted")}
|
||||||
|
{{end}}
|
||||||
|
}
|
||||||
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
{{if eq .Storage "local"}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
{{else}}
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
{{if eq .Storage "oss"}}
|
||||||
|
ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss"
|
||||||
|
ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
|
||||||
|
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
|
||||||
|
{{else}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/s3"
|
||||||
|
"github.com/minio/minio-go/v7/pkg/credentials"
|
||||||
|
{{end}}{{end}}
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InitStorage is called once, before serving requests. The composition root
|
||||||
|
// owns provider clients; business services receive only capability interfaces.
|
||||||
|
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
|
||||||
|
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
|
||||||
|
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||||
|
if c.closed{return errors.New("service dependencies are closed")}
|
||||||
|
if err:=ctx.Err();err!=nil{return err}
|
||||||
|
if c.Files!=nil{return errors.New("storage already initialized")}
|
||||||
|
if err:=cfg.Validate();err!=nil{return err}
|
||||||
|
{{if eq .Storage "local"}}
|
||||||
|
files,err:=local.New(cfg.Root,cfg.MaxBytes)
|
||||||
|
if err!=nil{return err}
|
||||||
|
signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy())
|
||||||
|
if err!=nil{return err}
|
||||||
|
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||||
|
if err!=nil{return err}
|
||||||
|
c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer
|
||||||
|
{{else}}
|
||||||
|
// Dedicated bounded pool, never the process-wide default transport.
|
||||||
|
transport:=&http.Transport{
|
||||||
|
Proxy:http.ProxyFromEnvironment,
|
||||||
|
DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext,
|
||||||
|
ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32,
|
||||||
|
IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second,
|
||||||
|
ResponseHeaderTimeout:cfg.RequestTimeout,
|
||||||
|
}
|
||||||
|
initialized:=false
|
||||||
|
defer func(){if !initialized{transport.CloseIdleConnections()}}()
|
||||||
|
{{if eq .Storage "oss"}}
|
||||||
|
sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region).
|
||||||
|
WithSignatureVersion(ossSDK.SignatureVersionV4).
|
||||||
|
WithAdditionalHeaders([]string{"content-length"}).
|
||||||
|
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)).
|
||||||
|
WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout})
|
||||||
|
if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)}
|
||||||
|
files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes)
|
||||||
|
{{else}}
|
||||||
|
creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)
|
||||||
|
{{if eq .Storage "r2"}}
|
||||||
|
options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds)
|
||||||
|
if err!=nil{return err}
|
||||||
|
options.MaxBytes,options.Transport=cfg.MaxBytes,transport
|
||||||
|
{{else}}
|
||||||
|
options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport}
|
||||||
|
{{end}}
|
||||||
|
files,err:=s3.Open(options)
|
||||||
|
{{end}}
|
||||||
|
if err!=nil{return err}
|
||||||
|
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||||
|
if err!=nil{return err}
|
||||||
|
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
|
||||||
|
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
|
||||||
|
initialized=true
|
||||||
|
{{end}}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"{{.Module}}/internal/settings"
|
||||||
|
)
|
||||||
|
func TestStorageInitializationAndOwnership(t *testing.T){
|
||||||
|
cfg:=settings.Default().Storage
|
||||||
|
cfg.FilesToken=strings.Repeat("t",32)
|
||||||
|
cfg.SigningKey=strings.Repeat("s",32)
|
||||||
|
cfg.Root=t.TempDir()
|
||||||
|
cfg.AccessKey,cfg.SecretKey,cfg.Bucket="fixtureAccess","fixtureSecret","testbucket"
|
||||||
|
{{if eq .Storage "s3"}}cfg.Endpoint="127.0.0.1:9000"{{end}}
|
||||||
|
{{if eq .Storage "r2"}}cfg.AccountID=strings.Repeat("a",32){{end}}
|
||||||
|
var output bytes.Buffer
|
||||||
|
logger,err:=logx.New(logx.Config{Writer:&output});if err!=nil{t.Fatal(err)};defer logger.Close()
|
||||||
|
c:=&Context{Logger:logger}
|
||||||
|
canceled,cancel:=context.WithCancel(context.Background());cancel()
|
||||||
|
if err=c.InitStorage(canceled,cfg);err==nil || c.Files!=nil{t.Fatal("canceled startup initialized storage")}
|
||||||
|
invalid:=cfg;invalid.FilesToken=""
|
||||||
|
if err=c.InitStorage(context.Background(),invalid);err==nil || c.Files!=nil{t.Fatal("invalid settings initialized storage")}
|
||||||
|
if err=c.InitStorage(context.Background(),cfg);err!=nil{t.Fatal(err)}
|
||||||
|
if c.Files==nil || c.Uploads==nil || c.Downloads==nil || c.UploadLinks==nil{t.Fatal("dependencies not assembled")}
|
||||||
|
if err=c.InitStorage(context.Background(),cfg);err==nil{t.Fatal("duplicate initialization accepted")}
|
||||||
|
cleanup:=c.storageClose;count:=0
|
||||||
|
c.storageClose=func()error{count++;if cleanup!=nil{return cleanup()};return nil}
|
||||||
|
if err=c.Close();err!=nil{t.Fatal(err)};if err=c.Close();err!=nil{t.Fatal(err)}
|
||||||
|
logger.Info("still usable")
|
||||||
|
if count!=1 || !strings.Contains(output.String(),"still usable"){t.Fatal("resource ownership violated")}
|
||||||
|
closed:=&Context{Logger:logger}
|
||||||
|
if err=closed.Close();err!=nil{t.Fatal(err)}
|
||||||
|
if err=closed.InitStorage(context.Background(),cfg);err==nil || closed.Files!=nil{t.Fatal("closed context reopened resources")}
|
||||||
|
}
|
||||||
@@ -10,7 +10,19 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
|||||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||||
// downloaded module version, which also has a module checksum.
|
// downloaded module version, which also has a module checksum.
|
||||||
const defaultFrameworkVersion = "v0.6.0-preview"
|
const defaultFrameworkVersion = "v0.7.0-preview"
|
||||||
|
|
||||||
|
// This target is not a claim that the release already exists. Source users
|
||||||
|
// select a local framework checkout until the module templates are published.
|
||||||
|
const modulesFrameworkVersion = "v0.7.0-preview"
|
||||||
|
|
||||||
|
func installedFrameworkVersion() string {
|
||||||
|
info, _ := debug.ReadBuildInfo()
|
||||||
|
if info != nil && info.Main.Path == frameworkModule && info.Main.Sum != "" && validFrameworkVersion(info.Main.Version) {
|
||||||
|
return info.Main.Version
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
||||||
|
|
||||||
|
|||||||
@@ -24,14 +24,15 @@ type TLSConfig struct {
|
|||||||
|
|
||||||
// Config controls the default runtime behavior of an App.
|
// Config controls the default runtime behavior of an App.
|
||||||
type Config struct {
|
type Config struct {
|
||||||
Address string `json:"address"`
|
Address string `json:"address"`
|
||||||
Transport string `json:"transport"`
|
Transport string `json:"transport"`
|
||||||
TLS TLSConfig `json:"tls"`
|
TLS TLSConfig `json:"tls"`
|
||||||
RequestIDHeader string `json:"requestIdHeader"`
|
RequestIDHeader string `json:"requestIdHeader"`
|
||||||
EnableLogger bool `json:"enableLogger"`
|
EnableLogger bool `json:"enableLogger"`
|
||||||
TrustedProxies []string `json:"trustedProxies"`
|
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
TrustedProxies []string `json:"trustedProxies"`
|
||||||
Log logx.Config `json:"log"`
|
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||||
|
Log logx.Config `json:"log"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func Default() Config {
|
func Default() Config {
|
||||||
@@ -46,6 +47,9 @@ func Default() Config {
|
|||||||
|
|
||||||
// Validate checks configuration values before an application starts.
|
// Validate checks configuration values before an application starts.
|
||||||
func (c Config) Validate() error {
|
func (c Config) Validate() error {
|
||||||
|
if c.SlowRequestThreshold < 0 {
|
||||||
|
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
|
||||||
|
}
|
||||||
if strings.TrimSpace(c.Address) == "" {
|
if strings.TrimSpace(c.Address) == "" {
|
||||||
return &ConfigError{Field: "address", Message: "is required"}
|
return &ConfigError{Field: "address", Message: "is required"}
|
||||||
}
|
}
|
||||||
@@ -114,6 +118,11 @@ func WithLogger(enabled bool) Option {
|
|||||||
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// WithSlowRequestThreshold enables access logs and warns after this duration.
|
||||||
|
func WithSlowRequestThreshold(threshold time.Duration) Option {
|
||||||
|
return func(cfg *Config) { cfg.SlowRequestThreshold = threshold; cfg.EnableLogger = true }
|
||||||
|
}
|
||||||
|
|
||||||
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
||||||
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
||||||
// by Config.Validate and panic when passed directly to App.New.
|
// by Config.Validate and panic when passed directly to App.New.
|
||||||
@@ -126,15 +135,21 @@ func WithTrustedProxies(proxies ...string) Option {
|
|||||||
// environment. Later options can still override individual fields.
|
// environment. Later options can still override individual fields.
|
||||||
func WithConfig(settings Config) Option {
|
func WithConfig(settings Config) Option {
|
||||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
return func(cfg *Config) {
|
return func(cfg *Config) {
|
||||||
*cfg = settings
|
*cfg = settings
|
||||||
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||||
|
cfg.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// WithLogConfig configures the logger used by the application runtime.
|
// WithLogConfig configures the logger used by the application runtime.
|
||||||
func WithLogConfig(settings logx.Config) Option {
|
func WithLogConfig(settings logx.Config) Option {
|
||||||
return func(cfg *Config) { cfg.Log = settings }
|
settings.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||||
|
return func(cfg *Config) {
|
||||||
|
cfg.Log = settings
|
||||||
|
cfg.Log.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// WithShutdownTimeout controls how long RunContext waits for active requests.
|
// WithShutdownTimeout controls how long RunContext waits for active requests.
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
|||||||
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
||||||
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
||||||
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
||||||
|
"GO_OMNI_SLOW_REQUEST_THRESHOLD",
|
||||||
} {
|
} {
|
||||||
t.Setenv(key, "")
|
t.Setenv(key, "")
|
||||||
}
|
}
|
||||||
@@ -80,6 +81,7 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
|||||||
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
|
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
|
||||||
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
|
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
|
||||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
|
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
|
||||||
|
t.Setenv("GO_OMNI_SLOW_REQUEST_THRESHOLD", "500ms")
|
||||||
|
|
||||||
cfg, err := LoadFromEnv("")
|
cfg, err := LoadFromEnv("")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -91,6 +93,9 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
|||||||
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
||||||
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
|
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
|
||||||
}
|
}
|
||||||
|
if cfg.SlowRequestThreshold != 500*time.Millisecond {
|
||||||
|
t.Fatal(cfg.SlowRequestThreshold)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||||
@@ -142,3 +147,23 @@ func TestWithConfigAndWithLogConfig(t *testing.T) {
|
|||||||
t.Fatalf("log config was not replaced: %+v", cfg.Log)
|
t.Fatalf("log config was not replaced: %+v", cfg.Log)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestApplyEnvPreservesProjectDefaults(t *testing.T) {
|
||||||
|
cfg := Default()
|
||||||
|
cfg.Address = "127.0.0.1:8123"
|
||||||
|
cfg.Log.RedactKeys = []string{"bankAccount"}
|
||||||
|
cfg.TrustedProxies = []string{"127.0.0.1"}
|
||||||
|
t.Setenv("OVERLAY_TEST_LOG_LEVEL", "debug")
|
||||||
|
got, err := ApplyEnv(cfg, "OVERLAY_TEST_")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.Address != cfg.Address || got.Log.Level != "debug" {
|
||||||
|
t.Fatal(got)
|
||||||
|
}
|
||||||
|
got.Log.RedactKeys[0] = "changed"
|
||||||
|
got.TrustedProxies[0] = "192.0.2.1"
|
||||||
|
if cfg.Log.RedactKeys[0] != "bankAccount" || cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||||
|
t.Fatal("caller-owned defaults mutated")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -13,15 +13,26 @@ const defaultEnvPrefix = "GO_OMNI_"
|
|||||||
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
|
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
|
||||||
// database settings remain owned by database.ConfigFromEnv.
|
// database settings remain owned by database.ConfigFromEnv.
|
||||||
func LoadFromEnv(prefix string) (Config, error) {
|
func LoadFromEnv(prefix string) (Config, error) {
|
||||||
|
return applyEnv(Default(), prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ApplyEnv overlays nonempty environment values on a caller's defaults or file
|
||||||
|
// settings, then validates the result. Caller-owned slices are not retained.
|
||||||
|
func ApplyEnv(defaults Config, prefix string) (Config, error) {
|
||||||
|
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
|
||||||
|
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
|
||||||
|
return applyEnv(defaults, prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
func applyEnv(cfg Config, prefix string) (Config, error) {
|
||||||
prefix = strings.TrimSpace(prefix)
|
prefix = strings.TrimSpace(prefix)
|
||||||
if prefix == "" {
|
if prefix == "" {
|
||||||
prefix = defaultEnvPrefix
|
prefix = defaultEnvPrefix
|
||||||
}
|
}
|
||||||
cfg := Default()
|
|
||||||
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
|
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
|
||||||
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
|
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
|
||||||
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", "")
|
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
|
||||||
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", "")
|
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
|
||||||
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
|
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
|
||||||
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
|
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
|
||||||
cfg.TrustedProxies = strings.Split(value, ",")
|
cfg.TrustedProxies = strings.Split(value, ",")
|
||||||
@@ -43,6 +54,12 @@ func LoadFromEnv(prefix string) (Config, error) {
|
|||||||
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
|
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
|
||||||
return Config{}, err
|
return Config{}, err
|
||||||
}
|
}
|
||||||
|
if value := strings.TrimSpace(os.Getenv(prefix + "SLOW_REQUEST_THRESHOLD")); value != "" {
|
||||||
|
cfg.SlowRequestThreshold, err = time.ParseDuration(value)
|
||||||
|
if err != nil {
|
||||||
|
return Config{}, fmt.Errorf("%sSLOW_REQUEST_THRESHOLD must be a duration: %w", prefix, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
|
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
|
||||||
cfg.ShutdownTimeout, err = time.ParseDuration(value)
|
cfg.ShutdownTimeout, err = time.ParseDuration(value)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
204
config/file.go
Normal file
204
config/file.go
Normal file
@@ -0,0 +1,204 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"gopkg.in/yaml.v3"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
|
||||||
|
// field names for both formats, rejects unknown fields and accepts durations
|
||||||
|
// such as "3s". Application-specific validation remains with the caller.
|
||||||
|
func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||||
|
var zero T
|
||||||
|
f, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||||
|
if err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
if len(data) > 1<<20 {
|
||||||
|
return zero, errors.New("config: file exceeds 1 MiB")
|
||||||
|
}
|
||||||
|
var value any
|
||||||
|
switch strings.ToLower(filepath.Ext(path)) {
|
||||||
|
case ".yaml", ".yml":
|
||||||
|
dec := yaml.NewDecoder(bytes.NewReader(data))
|
||||||
|
if err := dec.Decode(&value); err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
var extra any
|
||||||
|
if err := dec.Decode(&extra); err != io.EOF {
|
||||||
|
return zero, errors.New("config: one YAML document required")
|
||||||
|
}
|
||||||
|
case ".json":
|
||||||
|
dec := json.NewDecoder(bytes.NewReader(data))
|
||||||
|
dec.UseNumber()
|
||||||
|
if err := dec.Decode(&value); err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
var extra any
|
||||||
|
if err := dec.Decode(&extra); err != io.EOF {
|
||||||
|
return zero, errors.New("config: one JSON document required")
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return zero, errors.New("config: expected .json, .yaml or .yml")
|
||||||
|
}
|
||||||
|
if _, ok := value.(map[string]any); !ok {
|
||||||
|
return zero, errors.New("config: root must be an object")
|
||||||
|
}
|
||||||
|
value, err = durations(value, reflect.TypeOf(defaults))
|
||||||
|
if err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(value)
|
||||||
|
if err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
// Deep copy defaults so errors cannot partially mutate caller-owned maps.
|
||||||
|
original, err := json.Marshal(defaults)
|
||||||
|
if err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
var result T
|
||||||
|
if err = json.Unmarshal(original, &result); err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
dec := json.NewDecoder(bytes.NewReader(encoded))
|
||||||
|
dec.DisallowUnknownFields()
|
||||||
|
if err = dec.Decode(&result); err != nil {
|
||||||
|
return zero, err
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
func durations(v any, t reflect.Type) (any, error) {
|
||||||
|
if t == nil {
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
for t.Kind() == reflect.Pointer {
|
||||||
|
t = t.Elem()
|
||||||
|
}
|
||||||
|
if t == reflect.TypeOf(time.Duration(0)) {
|
||||||
|
if s, ok := v.(string); ok {
|
||||||
|
d, err := time.ParseDuration(s)
|
||||||
|
return int64(d), err
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
if t.Kind() == reflect.Struct {
|
||||||
|
if m, ok := v.(map[string]any); ok {
|
||||||
|
for n := 0; n < t.NumField(); n++ {
|
||||||
|
field := t.Field(n)
|
||||||
|
name := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||||
|
if name == "-" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if name == "" {
|
||||||
|
name = field.Name
|
||||||
|
}
|
||||||
|
if item, exists := m[name]; exists {
|
||||||
|
converted, err := durations(item, field.Type)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
m[name] = converted
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if t.Kind() == reflect.Slice {
|
||||||
|
if items, ok := v.([]any); ok {
|
||||||
|
for n, item := range items {
|
||||||
|
converted, err := durations(item, t.Elem())
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
items[n] = converted
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
func LoadFile(path, prefix string) (Config, error) {
|
||||||
|
cfg, err := DecodeFile(path, Default())
|
||||||
|
if err != nil {
|
||||||
|
return Config{}, err
|
||||||
|
}
|
||||||
|
return applyEnv(cfg, prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Reloader only accepts log.level changes. Static changes and invalid files
|
||||||
|
// leave the current snapshot and logger untouched. Watch does not own a goroutine.
|
||||||
|
type Reloader struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
path, prefix string
|
||||||
|
current Config
|
||||||
|
logger *logx.Logger
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewReloader(path, prefix string, logger *logx.Logger) (*Reloader, error) {
|
||||||
|
cfg, err := LoadFile(path, prefix)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err = logger.SetLevel(cfg.Log.Level); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &Reloader{path: path, prefix: prefix, current: cfg, logger: logger}, nil
|
||||||
|
}
|
||||||
|
func (r *Reloader) Reload() error {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
next, err := LoadFile(r.path, r.prefix)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
comparable := next
|
||||||
|
comparable.Log.Level = r.current.Log.Level
|
||||||
|
if !reflect.DeepEqual(comparable, r.current) {
|
||||||
|
return errors.New("config: static fields changed; restart required")
|
||||||
|
}
|
||||||
|
if err = r.logger.SetLevel(next.Log.Level); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
r.current = next
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (r *Reloader) Snapshot() Config {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
cfg := r.current
|
||||||
|
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
|
||||||
|
cfg.Log.RedactKeys = append([]string(nil), cfg.Log.RedactKeys...)
|
||||||
|
return cfg
|
||||||
|
}
|
||||||
|
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
|
||||||
|
if ctx == nil || interval < 100*time.Millisecond {
|
||||||
|
return errors.New("config: context and interval >=100ms required")
|
||||||
|
}
|
||||||
|
ticker := time.NewTicker(interval)
|
||||||
|
defer ticker.Stop()
|
||||||
|
for {
|
||||||
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return ctx.Err()
|
||||||
|
case <-ticker.C:
|
||||||
|
if err := r.Reload(); err != nil && onError != nil {
|
||||||
|
onError(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
59
config/file_test.go
Normal file
59
config/file_test.go
Normal file
@@ -0,0 +1,59 @@
|
|||||||
|
package config
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestFileMergeAndSafeReload(t *testing.T) {
|
||||||
|
path := filepath.Join(t.TempDir(), "service.yaml")
|
||||||
|
write := func(raw string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: info\n")
|
||||||
|
t.Setenv("TEST_MODULE_HTTP_ADDR", ":9123")
|
||||||
|
cfg, err := LoadFile(path, "TEST_MODULE_")
|
||||||
|
if err != nil || cfg.Address != ":9123" || cfg.ShutdownTimeout != 7*time.Second {
|
||||||
|
t.Fatal(cfg, err)
|
||||||
|
}
|
||||||
|
var output bytes.Buffer
|
||||||
|
lc := cfg.Log
|
||||||
|
lc.Writer = &output
|
||||||
|
l, err := logx.New(lc)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer l.Close()
|
||||||
|
child := l.With("scope", "test")
|
||||||
|
r, err := NewReloader(path, "TEST_MODULE_", l)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
child.Debug("hidden")
|
||||||
|
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: debug\n")
|
||||||
|
if err = r.Reload(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
child.Debug("visible")
|
||||||
|
if strings.Contains(output.String(), "hidden") || !strings.Contains(output.String(), "visible") {
|
||||||
|
t.Fatal(output.String())
|
||||||
|
}
|
||||||
|
write("shutdownTimeout: 8s\nlog:\n level: error\n")
|
||||||
|
if err = r.Reload(); err == nil || r.Snapshot().Log.Level != "debug" {
|
||||||
|
t.Fatal("static reload changed live config", err)
|
||||||
|
}
|
||||||
|
for _, raw := range []string{"unknownOption: true", "log:\n level: bad", "shutdownTimeout: invalid", "address: x\naddress: y", "---\n{}\n---\n{}"} {
|
||||||
|
write(raw)
|
||||||
|
if err = r.Reload(); err == nil {
|
||||||
|
t.Fatal(raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
30
contract/README.md
Normal file
30
contract/README.md
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
# contract
|
||||||
|
|
||||||
|
Generated API projects use this package to compile Go field tags once and bind
|
||||||
|
each request without depending on a specific HTTP server. `New[T]` validates a
|
||||||
|
struct and returns an immutable Binder. `ForType` provides the same compiler to
|
||||||
|
the CLI. `BodySchema`, `Parameters`, and `ResponseSchema` return detached OpenAPI
|
||||||
|
snapshots. `MustNew[T]` is intended for generated, already checked declarations.
|
||||||
|
|
||||||
|
```go
|
||||||
|
type Request struct {
|
||||||
|
ID int64 `path:"id,range=[1:100]"`
|
||||||
|
Name string `json:"name" binding:"required,max=50"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var request = contract.MustNew[Request]()
|
||||||
|
|
||||||
|
func Handle(c *httpx.Context) error {
|
||||||
|
var req Request
|
||||||
|
if err := request.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return c.OK(req.Name)
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The default body limit is 1 MiB. An optional positive final argument to Bind
|
||||||
|
overrides it. Application transport read timeouts still control slow clients.
|
||||||
|
See [API.md](../API.md) for source tags, validation rules, null/presence behavior,
|
||||||
|
strict decoding and supported type limits. This API leaves the existing
|
||||||
|
`Context.BindJSON/BindQuery/BindURI` behavior unchanged.
|
||||||
310
contract/bind.go
Normal file
310
contract/bind.go
Normal file
@@ -0,0 +1,310 @@
|
|||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"math"
|
||||||
|
"mime"
|
||||||
|
"net/http"
|
||||||
|
"net/url"
|
||||||
|
"reflect"
|
||||||
|
"strconv"
|
||||||
|
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Bind fills a fresh DTO transactionally. A JSON body, if declared, is required,
|
||||||
|
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
|
||||||
|
// parameters never overwrite path/body/header values. form means URL query only.
|
||||||
|
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
|
||||||
|
if b == nil || r == nil {
|
||||||
|
return apierrors.Internal(errors.New("contract: nil binder or request"))
|
||||||
|
}
|
||||||
|
v := reflect.ValueOf(target)
|
||||||
|
if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ {
|
||||||
|
return apierrors.Internal(errors.New("contract: target type mismatch"))
|
||||||
|
}
|
||||||
|
limit := DefaultMaxBodyBytes
|
||||||
|
if len(maxBytes) > 1 || (len(maxBytes) == 1 && (maxBytes[0] <= 0 || maxBytes[0] == math.MaxInt64)) {
|
||||||
|
return apierrors.Internal(errors.New("contract: invalid body limit"))
|
||||||
|
}
|
||||||
|
if len(maxBytes) == 1 {
|
||||||
|
limit = maxBytes[0]
|
||||||
|
}
|
||||||
|
fresh := reflect.New(b.typ).Elem()
|
||||||
|
body := map[string]any{}
|
||||||
|
if len(b.root.schema.Properties) > 0 {
|
||||||
|
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
|
||||||
|
if err != nil || ct != "application/json" {
|
||||||
|
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json")
|
||||||
|
}
|
||||||
|
if r.Body == nil {
|
||||||
|
return invalid(errors.New("body required"))
|
||||||
|
}
|
||||||
|
if r.ContentLength > limit {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
reader := &io.LimitedReader{R: r.Body, N: limit + 1}
|
||||||
|
dec := json.NewDecoder(reader)
|
||||||
|
dec.UseNumber()
|
||||||
|
value, err := decodeValue(dec, 0)
|
||||||
|
if reader.N == 0 {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
var maxErr *http.MaxBytesError
|
||||||
|
if errors.As(err, &maxErr) {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
return invalid(err)
|
||||||
|
}
|
||||||
|
_, err = dec.Token()
|
||||||
|
if reader.N == 0 {
|
||||||
|
return tooLarge()
|
||||||
|
}
|
||||||
|
if err != io.EOF {
|
||||||
|
return invalid(errors.New("exactly one JSON value required"))
|
||||||
|
}
|
||||||
|
var ok bool
|
||||||
|
body, ok = value.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return invalid(errors.New("body must be an object"))
|
||||||
|
}
|
||||||
|
for key := range body {
|
||||||
|
if _, ok = b.root.schema.Properties[key]; !ok {
|
||||||
|
return invalid(fmt.Errorf("unknown body field %s", key))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
query := url.Values{}
|
||||||
|
if r.URL != nil {
|
||||||
|
var err error
|
||||||
|
query, err = url.ParseQuery(r.URL.RawQuery)
|
||||||
|
if err != nil {
|
||||||
|
return invalid(errors.New("malformed query"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, f := range b.root.fields {
|
||||||
|
var value any
|
||||||
|
var present bool
|
||||||
|
if f.source == "json" {
|
||||||
|
value, present = body[f.name]
|
||||||
|
} else {
|
||||||
|
var raw []string
|
||||||
|
switch f.source {
|
||||||
|
case "path":
|
||||||
|
if s, ok := params[f.name]; ok {
|
||||||
|
raw = []string{s}
|
||||||
|
}
|
||||||
|
case "form":
|
||||||
|
raw = query[f.name]
|
||||||
|
case "header":
|
||||||
|
raw = r.Header.Values(f.name)
|
||||||
|
}
|
||||||
|
present = len(raw) > 0
|
||||||
|
if present {
|
||||||
|
var err error
|
||||||
|
value, err = fromStrings(f.value, raw)
|
||||||
|
if err != nil {
|
||||||
|
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := assignField(f, fresh.Field(f.index), value, present); err != nil {
|
||||||
|
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
v.Elem().Set(fresh)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func invalid(err error) error {
|
||||||
|
return apierrors.BadRequest("INVALID_REQUEST", "request parameters are invalid").WithCause(err)
|
||||||
|
}
|
||||||
|
func tooLarge() error { return apierrors.New(413, "BODY_TOO_LARGE", "request body exceeds limit") }
|
||||||
|
|
||||||
|
func fromStrings(n *node, values []string) (any, error) {
|
||||||
|
n = base(n)
|
||||||
|
if n.typ.Kind() == reflect.Slice {
|
||||||
|
out := make([]any, 0, len(values))
|
||||||
|
for _, s := range values {
|
||||||
|
v, err := parseScalar(s, base(n.elem).typ)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, v)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
if len(values) != 1 {
|
||||||
|
return nil, errors.New("duplicate scalar parameter")
|
||||||
|
}
|
||||||
|
return parseScalar(values[0], n.typ)
|
||||||
|
}
|
||||||
|
func assignField(f field, dst reflect.Value, value any, present bool) error {
|
||||||
|
if !present {
|
||||||
|
if f.hasDefault {
|
||||||
|
value = f.fallback
|
||||||
|
} else if f.optional {
|
||||||
|
return nil
|
||||||
|
} else {
|
||||||
|
return errors.New("required field missing")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := setValue(f.value, dst, value); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return checkField(f, dst)
|
||||||
|
}
|
||||||
|
func setValue(n *node, dst reflect.Value, value any) error {
|
||||||
|
if value == nil {
|
||||||
|
if n.typ.Kind() == reflect.Pointer {
|
||||||
|
dst.SetZero()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return errors.New("null requires a pointer type")
|
||||||
|
}
|
||||||
|
if n.typ.Kind() == reflect.Pointer {
|
||||||
|
dst.Set(reflect.New(n.typ.Elem()))
|
||||||
|
return setValue(n.elem, dst.Elem(), value)
|
||||||
|
}
|
||||||
|
switch n.typ.Kind() {
|
||||||
|
case reflect.Struct:
|
||||||
|
object, ok := value.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected object")
|
||||||
|
}
|
||||||
|
for key := range object {
|
||||||
|
if _, ok = n.schema.Properties[key]; !ok {
|
||||||
|
return fmt.Errorf("unknown field %s", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, f := range n.fields {
|
||||||
|
item, exists := object[f.name]
|
||||||
|
if err := assignField(f, dst.Field(f.index), item, exists); err != nil {
|
||||||
|
return fmt.Errorf("%s: %w", f.name, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case reflect.Slice:
|
||||||
|
items, ok := value.([]any)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected array")
|
||||||
|
}
|
||||||
|
dst.Set(reflect.MakeSlice(n.typ, len(items), len(items)))
|
||||||
|
for i, item := range items {
|
||||||
|
if err := setValue(n.elem, dst.Index(i), item); err != nil {
|
||||||
|
return fmt.Errorf("item %d: %w", i, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case reflect.Map:
|
||||||
|
items, ok := value.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected object")
|
||||||
|
}
|
||||||
|
dst.Set(reflect.MakeMapWithSize(n.typ, len(items)))
|
||||||
|
for key, item := range items {
|
||||||
|
v := reflect.New(n.typ.Elem()).Elem()
|
||||||
|
if err := setValue(n.elem, v, item); err != nil {
|
||||||
|
return fmt.Errorf("map value: %w", err)
|
||||||
|
}
|
||||||
|
dst.SetMapIndex(reflect.ValueOf(key).Convert(n.typ.Key()), v)
|
||||||
|
}
|
||||||
|
case reflect.String:
|
||||||
|
s, ok := value.(string)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected string")
|
||||||
|
}
|
||||||
|
dst.SetString(s)
|
||||||
|
case reflect.Bool:
|
||||||
|
v, ok := value.(bool)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected boolean")
|
||||||
|
}
|
||||||
|
dst.SetBool(v)
|
||||||
|
default:
|
||||||
|
num, ok := value.(json.Number)
|
||||||
|
if !ok {
|
||||||
|
return errors.New("expected number")
|
||||||
|
}
|
||||||
|
switch n.typ.Kind() {
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
v, err := strconv.ParseFloat(string(num), n.typ.Bits())
|
||||||
|
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||||
|
return errors.New("number out of range")
|
||||||
|
}
|
||||||
|
dst.SetFloat(v)
|
||||||
|
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
v, err := strconv.ParseUint(string(num), 10, n.typ.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("invalid unsigned integer")
|
||||||
|
}
|
||||||
|
dst.SetUint(v)
|
||||||
|
default:
|
||||||
|
v, err := strconv.ParseInt(string(num), 10, n.typ.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return errors.New("invalid integer")
|
||||||
|
}
|
||||||
|
dst.SetInt(v)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Token decoding rejects duplicate keys at every level (encoding/json normally
|
||||||
|
// keeps the last value). Depth and body limits bound client-controlled work.
|
||||||
|
func decodeValue(dec *json.Decoder, depth int) (any, error) {
|
||||||
|
if depth > 64 {
|
||||||
|
return nil, errors.New("JSON nesting exceeds 64")
|
||||||
|
}
|
||||||
|
token, err := dec.Token()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
switch token {
|
||||||
|
case json.Delim('{'):
|
||||||
|
object := map[string]any{}
|
||||||
|
for dec.More() {
|
||||||
|
key, err := dec.Token()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
name, ok := key.(string)
|
||||||
|
if !ok {
|
||||||
|
return nil, errors.New("invalid object key")
|
||||||
|
}
|
||||||
|
if _, exists := object[name]; exists {
|
||||||
|
return nil, errors.New("duplicate JSON field")
|
||||||
|
}
|
||||||
|
value, err := decodeValue(dec, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
object[name] = value
|
||||||
|
}
|
||||||
|
end, err := dec.Token()
|
||||||
|
if err != nil || end != json.Delim('}') {
|
||||||
|
return nil, errors.New("unterminated object")
|
||||||
|
}
|
||||||
|
return object, nil
|
||||||
|
case json.Delim('['):
|
||||||
|
items := []any{}
|
||||||
|
for dec.More() {
|
||||||
|
v, err := decodeValue(dec, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
items = append(items, v)
|
||||||
|
}
|
||||||
|
end, err := dec.Token()
|
||||||
|
if err != nil || end != json.Delim(']') {
|
||||||
|
return nil, errors.New("unterminated array")
|
||||||
|
}
|
||||||
|
return items, nil
|
||||||
|
default:
|
||||||
|
if _, bad := token.(json.Delim); bad {
|
||||||
|
return nil, errors.New("unexpected JSON delimiter")
|
||||||
|
}
|
||||||
|
return token, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
394
contract/contract.go
Normal file
394
contract/contract.go
Normal file
@@ -0,0 +1,394 @@
|
|||||||
|
// Package contract compiles API field tags into a reusable request binder and
|
||||||
|
// OpenAPI schemas. Generated services and the CLI use the same compiler.
|
||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"reflect"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
"github.com/go-playground/validator/v10"
|
||||||
|
)
|
||||||
|
|
||||||
|
const DefaultMaxBodyBytes int64 = 1 << 20
|
||||||
|
|
||||||
|
var wireName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||||
|
var headerName = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9-]*$`)
|
||||||
|
var valuesValidator = validator.New()
|
||||||
|
|
||||||
|
// Binder is immutable after compilation and safe for concurrent requests.
|
||||||
|
type Binder struct {
|
||||||
|
root *node
|
||||||
|
typ reflect.Type
|
||||||
|
}
|
||||||
|
type node struct {
|
||||||
|
typ reflect.Type
|
||||||
|
schema *openapi.Schema
|
||||||
|
fields []field
|
||||||
|
elem *node
|
||||||
|
}
|
||||||
|
type field struct {
|
||||||
|
index int
|
||||||
|
name, source string
|
||||||
|
value *node
|
||||||
|
optional bool
|
||||||
|
fallback any
|
||||||
|
hasDefault bool
|
||||||
|
binding string
|
||||||
|
}
|
||||||
|
|
||||||
|
func New[T any]() (*Binder, error) { return ForType(reflect.TypeOf((*T)(nil)).Elem()) }
|
||||||
|
func MustNew[T any]() *Binder {
|
||||||
|
b, err := New[T]()
|
||||||
|
if err != nil {
|
||||||
|
panic(err)
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
// ForType also lets generators check the exact runtime contract before writing files.
|
||||||
|
// The root must be a struct; recursive and custom-marshaling types are rejected.
|
||||||
|
func ForType(t reflect.Type) (*Binder, error) {
|
||||||
|
if t == nil || t.Kind() != reflect.Struct {
|
||||||
|
return nil, errors.New("contract: root must be a struct")
|
||||||
|
}
|
||||||
|
budget := 10000
|
||||||
|
n, err := compile(t, true, map[reflect.Type]bool{}, 0, &budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return &Binder{root: n, typ: t}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int, budget *int) (*node, error) {
|
||||||
|
*budget--
|
||||||
|
if *budget < 0 {
|
||||||
|
return nil, errors.New("contract: schema expansion exceeds 10000 nodes")
|
||||||
|
}
|
||||||
|
if depth > 32 {
|
||||||
|
return nil, errors.New("contract: type nesting exceeds 32")
|
||||||
|
}
|
||||||
|
if stack[t] {
|
||||||
|
return nil, errors.New("contract: recursive types are not supported")
|
||||||
|
}
|
||||||
|
stack[t] = true
|
||||||
|
defer delete(stack, t)
|
||||||
|
n := &node{typ: t, schema: &openapi.Schema{}}
|
||||||
|
// Wire behavior must be described completely by these tags.
|
||||||
|
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||||
|
unmarshaler := reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
|
||||||
|
if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler) {
|
||||||
|
return nil, errors.New("contract: custom JSON marshalers are unsupported")
|
||||||
|
}
|
||||||
|
switch t.Kind() {
|
||||||
|
case reflect.Pointer:
|
||||||
|
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
n.elem = e
|
||||||
|
*n.schema = *e.schema
|
||||||
|
n.schema.Nullable = true
|
||||||
|
case reflect.String:
|
||||||
|
n.schema.Type = "string"
|
||||||
|
case reflect.Bool:
|
||||||
|
n.schema.Type = "boolean"
|
||||||
|
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
n.schema.Type = "integer"
|
||||||
|
if t.Bits() <= 32 {
|
||||||
|
n.schema.Format = "int32"
|
||||||
|
} else {
|
||||||
|
n.schema.Format = "int64"
|
||||||
|
}
|
||||||
|
if t.Kind() >= reflect.Uint && t.Kind() <= reflect.Uint64 {
|
||||||
|
v := float64(0)
|
||||||
|
n.schema.Minimum = &v
|
||||||
|
if t.Bits() < 64 {
|
||||||
|
max := float64((uint64(1) << uint(t.Bits())) - 1)
|
||||||
|
n.schema.Maximum = &max
|
||||||
|
if t.Bits() == 32 {
|
||||||
|
n.schema.Format = "int64"
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
n.schema.Format = ""
|
||||||
|
}
|
||||||
|
} else if t.Bits() < 32 {
|
||||||
|
min, max := -float64(int64(1)<<uint(t.Bits()-1)), float64((int64(1)<<uint(t.Bits()-1))-1)
|
||||||
|
n.schema.Minimum, n.schema.Maximum = &min, &max
|
||||||
|
}
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
n.schema.Type = "number"
|
||||||
|
if t.Kind() == reflect.Float32 {
|
||||||
|
n.schema.Format = "float"
|
||||||
|
} else {
|
||||||
|
n.schema.Format = "double"
|
||||||
|
}
|
||||||
|
case reflect.Slice, reflect.Map:
|
||||||
|
if t.Kind() == reflect.Map && t.Key().Kind() != reflect.String {
|
||||||
|
return nil, errors.New("contract: map keys must be strings")
|
||||||
|
}
|
||||||
|
if t.Kind() == reflect.Slice && t.Elem().Kind() == reflect.Uint8 {
|
||||||
|
return nil, errors.New("contract: byte slices use base64 in Go; use string for binary data")
|
||||||
|
}
|
||||||
|
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
n.elem = e
|
||||||
|
if t.Kind() == reflect.Slice {
|
||||||
|
n.schema.Type = "array"
|
||||||
|
n.schema.Items = e.schema
|
||||||
|
} else {
|
||||||
|
n.schema.Type = "object"
|
||||||
|
n.schema.AdditionalProperties = e.schema
|
||||||
|
}
|
||||||
|
case reflect.Struct:
|
||||||
|
n.schema.Type = "object"
|
||||||
|
allowed := false
|
||||||
|
n.schema.AdditionalPropertiesAllowed = &allowed
|
||||||
|
n.schema.Properties = map[string]*openapi.Schema{}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for i := 0; i < t.NumField(); i++ {
|
||||||
|
sf := t.Field(i)
|
||||||
|
if sf.PkgPath != "" || sf.Anonymous {
|
||||||
|
return nil, fmt.Errorf("contract: %s must be an exported named field", sf.Name)
|
||||||
|
}
|
||||||
|
f, err := compileField(sf, root, stack, depth+1, budget)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("contract: %s: %w", sf.Name, err)
|
||||||
|
}
|
||||||
|
f.index = i
|
||||||
|
key := f.source + ":" + strings.ToLower(f.name)
|
||||||
|
if seen[key] {
|
||||||
|
return nil, fmt.Errorf("contract: duplicate field %s", f.name)
|
||||||
|
}
|
||||||
|
seen[key] = true
|
||||||
|
n.fields = append(n.fields, f)
|
||||||
|
if f.source == "json" {
|
||||||
|
n.schema.Properties[f.name] = f.value.schema
|
||||||
|
if !f.optional && !f.hasDefault {
|
||||||
|
n.schema.Required = append(n.schema.Required, f.name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("contract: unsupported type %s", t)
|
||||||
|
}
|
||||||
|
return n, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool, depth int, budget *int) (field, error) {
|
||||||
|
f := field{}
|
||||||
|
// Reject typos and unsupported tags instead of silently losing validation.
|
||||||
|
tags, err := ParseTags(string(sf.Tag))
|
||||||
|
if err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
var sourceTag string
|
||||||
|
for key, value := range tags {
|
||||||
|
switch key {
|
||||||
|
case "json", "path", "form", "header":
|
||||||
|
if value == "-" {
|
||||||
|
return f, errors.New("ignored fields are not part of an API contract")
|
||||||
|
}
|
||||||
|
if f.source != "" {
|
||||||
|
return f, errors.New("exactly one source tag is required")
|
||||||
|
}
|
||||||
|
f.source = key
|
||||||
|
sourceTag = value
|
||||||
|
case "binding":
|
||||||
|
f.binding = value
|
||||||
|
default:
|
||||||
|
return f, fmt.Errorf("unsupported tag %q", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if f.source == "" {
|
||||||
|
return f, errors.New("json, path, form or header tag required")
|
||||||
|
}
|
||||||
|
if !root && f.source != "json" {
|
||||||
|
return f, errors.New("nested fields must use json tags")
|
||||||
|
}
|
||||||
|
parts := strings.Split(sourceTag, ",")
|
||||||
|
f.name = parts[0]
|
||||||
|
if (f.source == "header" && !headerName.MatchString(f.name)) || (f.source != "header" && !wireName.MatchString(f.name)) {
|
||||||
|
return f, errors.New("wire names must be lowerCamelCase (headers use HTTP names)")
|
||||||
|
}
|
||||||
|
n, err := compile(sf.Type, false, stack, depth, budget)
|
||||||
|
if err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
f.value = n
|
||||||
|
scalar := base(n).typ.Kind()
|
||||||
|
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
|
||||||
|
return f, errors.New("path/query/header values must be scalars or scalar slices")
|
||||||
|
}
|
||||||
|
if f.source == "path" && !isScalar(scalar) {
|
||||||
|
return f, errors.New("path values must be scalars")
|
||||||
|
}
|
||||||
|
if f.source == "header" && !isScalar(scalar) {
|
||||||
|
return f, errors.New("header values must be scalars")
|
||||||
|
}
|
||||||
|
options := map[string]bool{}
|
||||||
|
for _, option := range parts[1:] {
|
||||||
|
key, value, _ := strings.Cut(option, "=")
|
||||||
|
if options[key] {
|
||||||
|
return f, fmt.Errorf("duplicate option %q", key)
|
||||||
|
}
|
||||||
|
options[key] = true
|
||||||
|
switch key {
|
||||||
|
case "optional", "omitempty":
|
||||||
|
if option != key {
|
||||||
|
return f, errors.New("optional takes no value")
|
||||||
|
}
|
||||||
|
f.optional = true
|
||||||
|
case "default":
|
||||||
|
if !isScalar(scalar) {
|
||||||
|
return f, errors.New("default requires a scalar")
|
||||||
|
}
|
||||||
|
v, e := parseScalar(value, base(n).typ)
|
||||||
|
if e != nil {
|
||||||
|
return f, fmt.Errorf("invalid default: %w", e)
|
||||||
|
}
|
||||||
|
f.fallback = v
|
||||||
|
f.hasDefault = true
|
||||||
|
n.schema.Default = v
|
||||||
|
case "options":
|
||||||
|
if !isScalar(scalar) || value == "" {
|
||||||
|
return f, errors.New("options requires scalar values")
|
||||||
|
}
|
||||||
|
for _, s := range strings.Split(value, "|") {
|
||||||
|
v, e := parseScalar(s, base(n).typ)
|
||||||
|
if e != nil {
|
||||||
|
return f, e
|
||||||
|
}
|
||||||
|
n.schema.Enum = append(n.schema.Enum, v)
|
||||||
|
}
|
||||||
|
case "range":
|
||||||
|
if !isNumber(scalar) {
|
||||||
|
return f, errors.New("range requires numeric type")
|
||||||
|
}
|
||||||
|
if err = applyRange(n.schema, value); err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return f, fmt.Errorf("unsupported field option %q", option)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err = applyBinding(&f); err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
if n.schema.Nullable && len(n.schema.Enum) > 0 {
|
||||||
|
n.schema.Enum = append(n.schema.Enum, nil)
|
||||||
|
}
|
||||||
|
if f.source == "path" && (f.optional || f.hasDefault || sf.Type.Kind() == reflect.Pointer) {
|
||||||
|
return f, errors.New("path fields must be required non-pointer values")
|
||||||
|
}
|
||||||
|
if f.hasDefault {
|
||||||
|
v := reflect.New(sf.Type).Elem()
|
||||||
|
if err = setValue(n, v, f.fallback); err != nil {
|
||||||
|
return f, err
|
||||||
|
}
|
||||||
|
if err = checkField(f, v); err != nil {
|
||||||
|
return f, fmt.Errorf("default violates validation: %w", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return f, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ParseTags strictly parses Go struct tags, including duplicate/unknown key checks
|
||||||
|
// performed by the compiler. It is also used to normalize generated Go JSON tags.
|
||||||
|
func ParseTags(raw string) (map[string]string, error) {
|
||||||
|
out := map[string]string{}
|
||||||
|
for strings.TrimSpace(raw) != "" {
|
||||||
|
raw = strings.TrimLeft(raw, " ")
|
||||||
|
i := strings.Index(raw, ":")
|
||||||
|
if i < 1 {
|
||||||
|
return nil, errors.New("invalid struct tag")
|
||||||
|
}
|
||||||
|
key := raw[:i]
|
||||||
|
raw = raw[i+1:]
|
||||||
|
if strings.ContainsAny(key, " \t\r\n\"`") || len(raw) == 0 || raw[0] != '"' {
|
||||||
|
return nil, errors.New("invalid struct tag")
|
||||||
|
}
|
||||||
|
end := 1
|
||||||
|
for end < len(raw) {
|
||||||
|
if raw[end] == '\\' {
|
||||||
|
end += 2
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if raw[end] == '"' {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
end++
|
||||||
|
}
|
||||||
|
if end >= len(raw) {
|
||||||
|
return nil, errors.New("unterminated struct tag")
|
||||||
|
}
|
||||||
|
v, err := strconv.Unquote(raw[:end+1])
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if _, ok := out[key]; ok {
|
||||||
|
return nil, fmt.Errorf("duplicate tag %q", key)
|
||||||
|
}
|
||||||
|
out[key] = v
|
||||||
|
raw = raw[end+1:]
|
||||||
|
if raw != "" && raw[0] != ' ' {
|
||||||
|
return nil, errors.New("struct tags require spaces")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func base(n *node) *node {
|
||||||
|
for n.typ.Kind() == reflect.Pointer {
|
||||||
|
n = n.elem
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
func isNumber(k reflect.Kind) bool {
|
||||||
|
return k >= reflect.Int && k <= reflect.Float64 && k != reflect.Uintptr
|
||||||
|
}
|
||||||
|
func isScalar(k reflect.Kind) bool { return k == reflect.String || k == reflect.Bool || isNumber(k) }
|
||||||
|
|
||||||
|
func (b *Binder) BodySchema() *openapi.Schema {
|
||||||
|
if len(b.root.schema.Properties) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return copySchema(b.root.schema)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ResponseSchema rejects request-only fields to keep response docs honest.
|
||||||
|
func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
|
||||||
|
for _, f := range b.root.fields {
|
||||||
|
if f.source != "json" {
|
||||||
|
return nil, errors.New("contract: response fields must use json tags")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return copySchema(b.root.schema), nil
|
||||||
|
}
|
||||||
|
func (b *Binder) Parameters() []openapi.Parameter {
|
||||||
|
var out []openapi.Parameter
|
||||||
|
for _, f := range b.root.fields {
|
||||||
|
if f.source == "json" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
in := f.source
|
||||||
|
if in == "form" {
|
||||||
|
in = "query"
|
||||||
|
}
|
||||||
|
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: copySchema(f.value.schema)})
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
func copySchema(s *openapi.Schema) *openapi.Schema {
|
||||||
|
data, _ := json.Marshal(s)
|
||||||
|
var out openapi.Schema
|
||||||
|
_ = json.Unmarshal(data, &out)
|
||||||
|
return &out
|
||||||
|
}
|
||||||
244
contract/contract_test.go
Normal file
244
contract/contract_test.go
Normal file
@@ -0,0 +1,244 @@
|
|||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"net/http/httptest"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
)
|
||||||
|
|
||||||
|
type address struct {
|
||||||
|
City string `json:"city" binding:"required,max=30"`
|
||||||
|
}
|
||||||
|
type mixed struct {
|
||||||
|
ID int64 `path:"userId,range=[1:100]"`
|
||||||
|
Page int `form:"page,default=1,range=[1:50]"`
|
||||||
|
Tags []string `form:"tag,optional" binding:"max=3"`
|
||||||
|
Client string `header:"X-Client"`
|
||||||
|
Name string `json:"name" binding:"required,max=20"`
|
||||||
|
Active bool `json:"active"`
|
||||||
|
Count *int `json:"count,optional"`
|
||||||
|
Contact address `json:"contact"`
|
||||||
|
Items []address `json:"items,optional"`
|
||||||
|
Labels map[string]address `json:"labels,optional"`
|
||||||
|
}
|
||||||
|
|
||||||
|
const validBody = `{"name":"小明","active":false,"count":0,"contact":{"city":"杭州"}}`
|
||||||
|
|
||||||
|
func bindMixed(body, query string) (mixed, error) {
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
r := httptest.NewRequest("POST", "/users/7"+query, strings.NewReader(body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
var v mixed
|
||||||
|
return v, b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||||
|
}
|
||||||
|
func TestMixedSourcesAndPresence(t *testing.T) {
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
r := httptest.NewRequest("POST", "/users/7?tag=a&tag=b&userId=99&name=query", strings.NewReader(validBody))
|
||||||
|
r.Header.Set("Content-Type", "application/json; charset=utf-8")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
v := mixed{Name: "previous"}
|
||||||
|
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if v.ID != 7 || v.Page != 1 || v.Name != "小明" || v.Active || v.Count == nil || *v.Count != 0 || len(v.Tags) != 2 || v.Client != "web" || v.Contact.City != "杭州" {
|
||||||
|
t.Fatalf("bad binding: %+v", v)
|
||||||
|
}
|
||||||
|
r = httptest.NewRequest("POST", "/", strings.NewReader(strings.Replace(validBody, `,"count":0`, "", 1)))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil || v.Count != nil {
|
||||||
|
t.Fatalf("missing != zero: %+v %v", v, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRejectInvalidRequests(t *testing.T) {
|
||||||
|
cases := []struct{ name, body, query string }{
|
||||||
|
{"missing", `{"name":"a","contact":{"city":"杭州"}}`, ""},
|
||||||
|
{"empty", strings.Replace(validBody, `"小明"`, `""`, 1), ""},
|
||||||
|
{"caseSensitive", strings.Replace(validBody, `"name"`, `"Name"`, 1), ""},
|
||||||
|
{"unknown", strings.Replace(validBody, `"active":false`, `"active":false,"extra":1`, 1), ""},
|
||||||
|
{"duplicate", strings.Replace(validBody, `"active":false`, `"active":false,"active":true`, 1), ""},
|
||||||
|
{"nestedDuplicate", strings.Replace(validBody, `"city":"杭州"`, `"city":"杭州","city":"北京"`, 1), ""},
|
||||||
|
{"nestedRequired", strings.Replace(validBody, `{"city":"杭州"}`, `{}`, 1), ""},
|
||||||
|
{"nestedUnknown", strings.Replace(validBody, `{"city":"杭州"}`, `{"city":"杭州","other":0}`, 1), ""},
|
||||||
|
{"trailing", validBody + ` {}`, ""},
|
||||||
|
{"null", strings.Replace(validBody, `false`, `null`, 1), ""},
|
||||||
|
{"numberOverflow", strings.Replace(validBody, `"count":0`, `"count":99999999999999999999`, 1), ""},
|
||||||
|
{"arrayValidation", strings.TrimSuffix(validBody, "}") + `,"items":[{"city":""}]}`, ""},
|
||||||
|
{"mapValidation", strings.TrimSuffix(validBody, "}") + `,"labels":{"home":{"city":""}}}`, ""},
|
||||||
|
{"queryDuplicate", validBody, "?page=1&page=2"},
|
||||||
|
{"queryDefaultNotOnEmpty", validBody, "?page="},
|
||||||
|
{"range", validBody, "?page=51"},
|
||||||
|
{"malformedQuery", validBody, "?page=%zz"},
|
||||||
|
{"arrayLimit", validBody, "?tag=a&tag=b&tag=c&tag=d"},
|
||||||
|
}
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
for _, tc := range cases {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
r := httptest.NewRequest("POST", "/"+tc.query, strings.NewReader(tc.body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
r.Header.Set("X-Client", "web")
|
||||||
|
v := mixed{Name: "unchanged"}
|
||||||
|
err := b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||||
|
if err == nil || apierrors.From(err).Status != 400 {
|
||||||
|
t.Fatalf("wanted 400: %v", err)
|
||||||
|
}
|
||||||
|
if v.Name != "unchanged" || v.Page != 0 {
|
||||||
|
t.Fatal("failed binding modified target")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestBodyLimitsAndMediaType(t *testing.T) {
|
||||||
|
type request struct {
|
||||||
|
Name string `json:"name"`
|
||||||
|
}
|
||||||
|
b := MustNew[request]()
|
||||||
|
for _, tc := range []struct {
|
||||||
|
body, ct string
|
||||||
|
limit int64
|
||||||
|
status int
|
||||||
|
}{
|
||||||
|
{`{"name":"x"}`, "text/plain", 100, 415},
|
||||||
|
{`{"name":"long"}`, "application/json", 10, 413},
|
||||||
|
{`{"name":"x"}` + strings.Repeat(" ", 2048), "application/json", 1024, 413},
|
||||||
|
{`{"name":"x"}`, "application/json", 12, 0},
|
||||||
|
} {
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||||
|
r.ContentLength = -1
|
||||||
|
r.Header.Set("Content-Type", tc.ct)
|
||||||
|
var req request
|
||||||
|
err := b.Bind(r, nil, &req, tc.limit)
|
||||||
|
if tc.status == 0 {
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
} else if err == nil || apierrors.From(err).Status != tc.status {
|
||||||
|
t.Fatalf("status %d wanted: %v", tc.status, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestSchemaUsesSameRules(t *testing.T) {
|
||||||
|
b := MustNew[mixed]()
|
||||||
|
s := b.BodySchema()
|
||||||
|
if len(s.Required) != 3 || s.Properties["count"].Nullable != true || s.Properties["name"].MaxLength == nil || *s.Properties["name"].MaxLength != 20 {
|
||||||
|
t.Fatalf("wrong schema: %+v", s)
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(s)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !strings.Contains(string(encoded), `"additionalProperties":false`) {
|
||||||
|
t.Fatal(string(encoded))
|
||||||
|
}
|
||||||
|
params := b.Parameters()
|
||||||
|
if len(params) != 4 || params[1].Required || params[1].Schema.Default != json.Number("1") {
|
||||||
|
t.Fatalf("wrong parameters: %+v", params)
|
||||||
|
}
|
||||||
|
// Returned documentation snapshots do not mutate concurrent binders.
|
||||||
|
s.Properties["name"].MaxLength = nil
|
||||||
|
if b.BodySchema().Properties["name"].MaxLength == nil {
|
||||||
|
t.Fatal("shared schema")
|
||||||
|
}
|
||||||
|
if _, err = b.ResponseSchema(); err == nil {
|
||||||
|
t.Fatal("request sources in response accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestRulesAndConcurrentReuse(t *testing.T) {
|
||||||
|
type req struct {
|
||||||
|
Score float64 `json:"score,options=1.0|2,range=(0:2]"`
|
||||||
|
Email string `json:"email" binding:"email"`
|
||||||
|
Value *int `json:"value" binding:"required"`
|
||||||
|
}
|
||||||
|
b := MustNew[req]()
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i := 0; i < 24; i++ {
|
||||||
|
wg.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer wg.Done()
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"score":1,"email":"a@example.com","value":3}`))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
var v req
|
||||||
|
if err := b.Bind(r, nil, &v); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
for _, body := range []string{`{"score":0,"email":"a@example.com","value":1}`, `{"score":2,"email":"bad","value":1}`, `{"score":2,"email":"a@example.com","value":null}`, `{"score":2,"email":"a@example.com","value":0}`} {
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
var v req
|
||||||
|
if err := b.Bind(r, nil, &v); err == nil {
|
||||||
|
t.Fatal(body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestInvalidContracts(t *testing.T) {
|
||||||
|
for _, tag := range []string{`json:"bad_name"`, `json:"a" form:"a"`, `json:"a,optionl"`, `json:"a,range=[5:2]"`, `json:"a,default=20,range=[1:10]"`, `json:"a,optional" binding:"required"`, `json:"a" binding:"unknown"`, `path:"a,optional"`, `json:"a" json:"b"`, `jsoon:"a"`, `json:"a,options=x"`} {
|
||||||
|
typ := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: reflect.TypeOf(0), Tag: reflect.StructTag(tag)}})
|
||||||
|
if _, err := ForType(typ); err == nil {
|
||||||
|
t.Fatalf("accepted %s", tag)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExactLargeIntegerConstraints(t *testing.T) {
|
||||||
|
type req struct {
|
||||||
|
Value int64 `json:"value,range=[:9007199254740992]"`
|
||||||
|
}
|
||||||
|
b := MustNew[req]()
|
||||||
|
for _, tc := range []struct {
|
||||||
|
body string
|
||||||
|
valid bool
|
||||||
|
}{{`{"value":9007199254740992}`, true}, {`{"value":9007199254740993}`, false}} {
|
||||||
|
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
var v req
|
||||||
|
if err := b.Bind(r, nil, &v); (err == nil) != tc.valid {
|
||||||
|
t.Fatalf("%s: %v", tc.body, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
type defaults struct {
|
||||||
|
Value uint64 `form:"value,default=18446744073709551615,options=18446744073709551615"`
|
||||||
|
}
|
||||||
|
d := MustNew[defaults]()
|
||||||
|
data, err := json.Marshal(d.Parameters())
|
||||||
|
if err != nil || !strings.Contains(string(data), "18446744073709551615") {
|
||||||
|
t.Fatalf("lost precision: %s %v", data, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestIntrinsicIntegerBounds(t *testing.T) {
|
||||||
|
type req struct {
|
||||||
|
Small uint8 `json:"small,range=[-1:500]"`
|
||||||
|
Wide uint64 `json:"wide"`
|
||||||
|
}
|
||||||
|
b := MustNew[req]()
|
||||||
|
s := b.BodySchema()
|
||||||
|
if *s.Properties["small"].Minimum != 0 || *s.Properties["small"].Maximum != 255 || s.Properties["wide"].Format != "" {
|
||||||
|
t.Fatalf("wrong integer bounds: %+v", s.Properties)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func FuzzStrictBody(f *testing.F) {
|
||||||
|
for _, seed := range []string{validBody, `{}`, `{"name":"a","name":"b"}`, `[null]`, `{"x":[]}`} {
|
||||||
|
f.Add(seed)
|
||||||
|
}
|
||||||
|
binder := MustNew[mixed]()
|
||||||
|
f.Fuzz(func(t *testing.T, body string) {
|
||||||
|
if len(body) > 16384 {
|
||||||
|
t.Skip()
|
||||||
|
}
|
||||||
|
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
|
req.Header.Set("X-Client", "fuzz")
|
||||||
|
var value mixed
|
||||||
|
_ = binder.Bind(req, map[string]string{"userId": "1"}, &value, 16384)
|
||||||
|
})
|
||||||
|
}
|
||||||
259
contract/rules.go
Normal file
259
contract/rules.go
Normal file
@@ -0,0 +1,259 @@
|
|||||||
|
package contract
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"math"
|
||||||
|
"reflect"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"unicode/utf8"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
)
|
||||||
|
|
||||||
|
func applyRange(s *openapi.Schema, value string) error {
|
||||||
|
if len(value) < 3 || (value[0] != '[' && value[0] != '(') || (value[len(value)-1] != ']' && value[len(value)-1] != ')') {
|
||||||
|
return errors.New("invalid range: use [min:max], (min:max), or mixed bounds")
|
||||||
|
}
|
||||||
|
lo, hi, ok := strings.Cut(value[1:len(value)-1], ":")
|
||||||
|
if !ok || (lo == "" && hi == "") {
|
||||||
|
return errors.New("invalid numeric range")
|
||||||
|
}
|
||||||
|
if lo != "" {
|
||||||
|
v, err := bound(lo)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if s.Minimum == nil || v > *s.Minimum || (v == *s.Minimum && value[0] == '(') {
|
||||||
|
s.Minimum = &v
|
||||||
|
s.ExclusiveMinimum = value[0] == '('
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if hi != "" {
|
||||||
|
v, err := bound(hi)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if s.Maximum == nil || v < *s.Maximum || (v == *s.Maximum && value[len(value)-1] == ')') {
|
||||||
|
s.Maximum = &v
|
||||||
|
s.ExclusiveMaximum = value[len(value)-1] == ')'
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return checkBounds(s)
|
||||||
|
}
|
||||||
|
func bound(s string) (float64, error) {
|
||||||
|
v, err := strconv.ParseFloat(s, 64)
|
||||||
|
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) || math.Abs(v) > 1<<53 {
|
||||||
|
return 0, errors.New("constraint bound must be finite and within ±2^53")
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
func checkBounds(s *openapi.Schema) error {
|
||||||
|
if s.Minimum != nil && s.Maximum != nil && (*s.Minimum > *s.Maximum || (*s.Minimum == *s.Maximum && (s.ExclusiveMinimum || s.ExclusiveMaximum))) {
|
||||||
|
return errors.New("empty numeric range")
|
||||||
|
}
|
||||||
|
if s.MinLength != nil && s.MaxLength != nil && *s.MinLength > *s.MaxLength {
|
||||||
|
return errors.New("empty string length range")
|
||||||
|
}
|
||||||
|
if s.MinItems != nil && s.MaxItems != nil && *s.MinItems > *s.MaxItems {
|
||||||
|
return errors.New("empty array length range")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func applyBinding(f *field) error {
|
||||||
|
if f.binding == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
n := base(f.value)
|
||||||
|
k := n.typ.Kind()
|
||||||
|
s := f.value.schema
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, rule := range strings.Split(f.binding, ",") {
|
||||||
|
name, value, _ := strings.Cut(rule, "=")
|
||||||
|
if seen[name] {
|
||||||
|
return fmt.Errorf("duplicate binding rule %s", name)
|
||||||
|
}
|
||||||
|
seen[name] = true
|
||||||
|
switch name {
|
||||||
|
case "required":
|
||||||
|
if rule != name || f.optional || f.hasDefault {
|
||||||
|
return errors.New("required conflicts with optional/default")
|
||||||
|
}
|
||||||
|
if !isScalar(k) {
|
||||||
|
return errors.New("binding required supports scalars; other fields are already required by presence")
|
||||||
|
}
|
||||||
|
s.Nullable = false
|
||||||
|
switch k {
|
||||||
|
case reflect.String:
|
||||||
|
one := 1
|
||||||
|
s.MinLength = &one
|
||||||
|
case reflect.Bool:
|
||||||
|
s.Not = &openapi.Schema{Enum: []any{false}}
|
||||||
|
default:
|
||||||
|
s.Not = &openapi.Schema{Enum: []any{0}}
|
||||||
|
}
|
||||||
|
case "email":
|
||||||
|
if rule != name || k != reflect.String {
|
||||||
|
return errors.New("email requires a string")
|
||||||
|
}
|
||||||
|
s.Format = "email"
|
||||||
|
case "min", "max", "len":
|
||||||
|
if k == reflect.String || k == reflect.Slice {
|
||||||
|
v, err := strconv.Atoi(value)
|
||||||
|
if err != nil || v < 0 {
|
||||||
|
return errors.New("length constraints must be nonnegative integers")
|
||||||
|
}
|
||||||
|
if k == reflect.String {
|
||||||
|
if name == "min" || name == "len" {
|
||||||
|
if s.MinLength == nil || v > *s.MinLength {
|
||||||
|
s.MinLength = &v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if name == "max" || name == "len" {
|
||||||
|
s.MaxLength = &v
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if name == "min" || name == "len" {
|
||||||
|
s.MinItems = &v
|
||||||
|
}
|
||||||
|
if name == "max" || name == "len" {
|
||||||
|
s.MaxItems = &v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else if isNumber(k) && name != "len" {
|
||||||
|
v, err := bound(value)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if name == "min" {
|
||||||
|
if s.Minimum == nil || v > *s.Minimum {
|
||||||
|
s.Minimum = &v
|
||||||
|
s.ExclusiveMinimum = false
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if s.Maximum == nil || v < *s.Maximum {
|
||||||
|
s.Maximum = &v
|
||||||
|
s.ExclusiveMaximum = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
return fmt.Errorf("%s requires a number, string or slice", name)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return fmt.Errorf("unsupported binding rule %q; use required,min,max,len,email or source-tag options/range", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return checkBounds(s)
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseScalar(raw string, t reflect.Type) (any, error) {
|
||||||
|
switch t.Kind() {
|
||||||
|
case reflect.String:
|
||||||
|
return raw, nil
|
||||||
|
case reflect.Bool:
|
||||||
|
if raw == "true" {
|
||||||
|
return true, nil
|
||||||
|
}
|
||||||
|
if raw == "false" {
|
||||||
|
return false, nil
|
||||||
|
}
|
||||||
|
return nil, errors.New("expected true or false")
|
||||||
|
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
|
||||||
|
v, err := strconv.ParseInt(raw, 10, t.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New("invalid integer")
|
||||||
|
}
|
||||||
|
return json.Number(strconv.FormatInt(v, 10)), nil
|
||||||
|
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
v, err := strconv.ParseUint(raw, 10, t.Bits())
|
||||||
|
if err != nil {
|
||||||
|
return nil, errors.New("invalid unsigned integer")
|
||||||
|
}
|
||||||
|
return json.Number(strconv.FormatUint(v, 10)), nil
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
v, err := strconv.ParseFloat(raw, t.Bits())
|
||||||
|
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||||
|
return nil, errors.New("invalid finite number")
|
||||||
|
}
|
||||||
|
return json.Number(strconv.FormatFloat(v, 'g', -1, t.Bits())), nil
|
||||||
|
}
|
||||||
|
return nil, errors.New("expected scalar")
|
||||||
|
}
|
||||||
|
|
||||||
|
func checkField(f field, v reflect.Value) error {
|
||||||
|
if v.Kind() == reflect.Pointer {
|
||||||
|
if v.IsNil() {
|
||||||
|
if strings.Contains(","+f.binding+",", ",required,") {
|
||||||
|
return errors.New("must not be null")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
v = v.Elem()
|
||||||
|
}
|
||||||
|
if strings.Contains(","+f.binding+",", ",required,") && v.IsZero() {
|
||||||
|
return errors.New("must not be zero or empty")
|
||||||
|
}
|
||||||
|
s := f.value.schema
|
||||||
|
if s.Format == "email" && valuesValidator.Var(v.Interface(), "email") != nil {
|
||||||
|
return errors.New("invalid email")
|
||||||
|
}
|
||||||
|
if len(s.Enum) > 0 {
|
||||||
|
found := false
|
||||||
|
for _, item := range s.Enum {
|
||||||
|
option := reflect.New(v.Type()).Elem()
|
||||||
|
if setValue(base(f.value), option, item) == nil && reflect.DeepEqual(option.Interface(), v.Interface()) {
|
||||||
|
found = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
return errors.New("not an allowed option")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if isNumber(v.Kind()) {
|
||||||
|
var number float64
|
||||||
|
switch v.Kind() {
|
||||||
|
case reflect.Float32, reflect.Float64:
|
||||||
|
number = v.Float()
|
||||||
|
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||||
|
number = float64(v.Uint())
|
||||||
|
if v.Uint() > 1<<53 {
|
||||||
|
number = math.Inf(1)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
number = float64(v.Int())
|
||||||
|
if v.Int() > 1<<53 {
|
||||||
|
number = math.Inf(1)
|
||||||
|
} else if v.Int() < -(1 << 53) {
|
||||||
|
number = math.Inf(-1)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s.Minimum != nil && (number < *s.Minimum || (s.ExclusiveMinimum && number == *s.Minimum)) {
|
||||||
|
return errors.New("below minimum")
|
||||||
|
}
|
||||||
|
if s.Maximum != nil && (number > *s.Maximum || (s.ExclusiveMaximum && number == *s.Maximum)) {
|
||||||
|
return errors.New("above maximum")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v.Kind() == reflect.String {
|
||||||
|
size := utf8.RuneCountInString(v.String())
|
||||||
|
if s.MinLength != nil && size < *s.MinLength {
|
||||||
|
return errors.New("string too short")
|
||||||
|
}
|
||||||
|
if s.MaxLength != nil && size > *s.MaxLength {
|
||||||
|
return errors.New("string too long")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v.Kind() == reflect.Slice {
|
||||||
|
size := v.Len()
|
||||||
|
if s.MinItems != nil && size < *s.MinItems {
|
||||||
|
return errors.New("too few items")
|
||||||
|
}
|
||||||
|
if s.MaxItems != nil && size > *s.MaxItems {
|
||||||
|
return errors.New("too many items")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
35
database/configOverlay_test.go
Normal file
35
database/configOverlay_test.go
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
package database
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestApplyEnvPreservesFilePoolAndHidesDSN(t *testing.T) {
|
||||||
|
cfg := DefaultConfig()
|
||||||
|
cfg.MaxOpenConns = 3
|
||||||
|
cfg.MaxIdleConns = 1
|
||||||
|
cfg.PingTimeout = time.Second
|
||||||
|
t.Setenv("OMNI_TEST_DSN", "user:secret@tcp(localhost:3306)/test")
|
||||||
|
t.Setenv("OMNI_TEST_MAX_IDLE_CONNS", "0")
|
||||||
|
got, err := ApplyEnv(cfg, "OMNI_TEST_")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if got.MaxOpenConns != 3 || got.MaxIdleConns != 0 || got.PingTimeout != time.Second {
|
||||||
|
t.Fatalf("lost file defaults: %+v", got)
|
||||||
|
}
|
||||||
|
data, err := json.Marshal(got)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if strings.Contains(string(data), "secret") || !strings.Contains(string(data), "maxOpenConns") {
|
||||||
|
t.Fatalf("unsafe config encoding: %s", data)
|
||||||
|
}
|
||||||
|
t.Setenv("OMNI_TEST_MAX_OPEN_CONNS", "bad")
|
||||||
|
if _, err = ApplyEnv(cfg, "OMNI_TEST_"); err == nil {
|
||||||
|
t.Fatal("invalid override accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -14,13 +14,13 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
type Config struct {
|
type Config struct {
|
||||||
Driver string
|
Driver string `json:"driver"`
|
||||||
DSN string
|
DSN string `json:"-"`
|
||||||
MaxOpenConns int
|
MaxOpenConns int `json:"maxOpenConns"`
|
||||||
MaxIdleConns int
|
MaxIdleConns int `json:"maxIdleConns"`
|
||||||
ConnMaxLifetime time.Duration
|
ConnMaxLifetime time.Duration `json:"connMaxLifetime"`
|
||||||
ConnMaxIdleTime time.Duration
|
ConnMaxIdleTime time.Duration `json:"connMaxIdleTime"`
|
||||||
PingTimeout time.Duration
|
PingTimeout time.Duration `json:"pingTimeout"`
|
||||||
}
|
}
|
||||||
|
|
||||||
func DefaultConfig() Config {
|
func DefaultConfig() Config {
|
||||||
@@ -35,16 +35,25 @@ func DefaultConfig() Config {
|
|||||||
|
|
||||||
// ConfigFromEnv reads GO_OMNI_DB_* style settings. DSN is required.
|
// ConfigFromEnv reads GO_OMNI_DB_* style settings. DSN is required.
|
||||||
func ConfigFromEnv(prefix string) (Config, error) {
|
func ConfigFromEnv(prefix string) (Config, error) {
|
||||||
|
return ApplyEnv(DefaultConfig(), prefix)
|
||||||
|
}
|
||||||
|
|
||||||
|
// ApplyEnv overlays nonempty environment settings on file/default configuration.
|
||||||
|
// DSN is environment-only when Config is decoded from JSON or YAML.
|
||||||
|
func ApplyEnv(cfg Config, prefix string) (Config, error) {
|
||||||
prefix = strings.TrimSpace(prefix)
|
prefix = strings.TrimSpace(prefix)
|
||||||
if prefix == "" {
|
if prefix == "" {
|
||||||
prefix = "GO_OMNI_DB_"
|
prefix = "GO_OMNI_DB_"
|
||||||
}
|
}
|
||||||
cfg := DefaultConfig()
|
if value := strings.TrimSpace(os.Getenv(prefix + "DRIVER")); value != "" {
|
||||||
cfg.Driver = strings.TrimSpace(os.Getenv(prefix + "DRIVER"))
|
cfg.Driver = value
|
||||||
|
}
|
||||||
if cfg.Driver == "" {
|
if cfg.Driver == "" {
|
||||||
cfg.Driver = "mysql"
|
cfg.Driver = "mysql"
|
||||||
}
|
}
|
||||||
cfg.DSN = strings.TrimSpace(os.Getenv(prefix + "DSN"))
|
if value := strings.TrimSpace(os.Getenv(prefix + "DSN")); value != "" {
|
||||||
|
cfg.DSN = value
|
||||||
|
}
|
||||||
if cfg.DSN == "" {
|
if cfg.DSN == "" {
|
||||||
return Config{}, fmt.Errorf("%sDSN is required", prefix)
|
return Config{}, fmt.Errorf("%sDSN is required", prefix)
|
||||||
}
|
}
|
||||||
|
|||||||
245
database/migrate/migrate.go
Normal file
245
database/migrate/migrate.go
Normal file
@@ -0,0 +1,245 @@
|
|||||||
|
// Package migrate provides explicit MySQL schema migration jobs. Application
|
||||||
|
// startup never runs migrations automatically; use a separate deployment step.
|
||||||
|
package migrate
|
||||||
|
|
||||||
|
import (
|
||||||
|
"archive/zip"
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"database/sql"
|
||||||
|
"encoding/hex"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
mysqlDriver "github.com/go-sql-driver/mysql"
|
||||||
|
migration "github.com/golang-migrate/migrate/v4"
|
||||||
|
migrateMySQL "github.com/golang-migrate/migrate/v4/database/mysql"
|
||||||
|
"github.com/golang-migrate/migrate/v4/source/iofs"
|
||||||
|
"io"
|
||||||
|
"io/fs"
|
||||||
|
"path"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type File struct {
|
||||||
|
Version uint64 `json:"version"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Direction string `json:"direction"`
|
||||||
|
SHA256 string `json:"sha256"`
|
||||||
|
}
|
||||||
|
|
||||||
|
var filename = regexp.MustCompile(`^([0-9]+)_([A-Za-z][A-Za-z0-9]*)\.(up|down)\.sql$`)
|
||||||
|
|
||||||
|
// Snapshot returns immutable migration bytes and a reviewable digest manifest.
|
||||||
|
// Follow lowerCamelCase names, e.g. 0001_createUsers.up.sql.
|
||||||
|
func Snapshot(source fs.FS, directory string) (fs.FS, []File, error) {
|
||||||
|
if source == nil || !fs.ValidPath(directory) {
|
||||||
|
return nil, nil, errors.New("migrate: invalid source directory")
|
||||||
|
}
|
||||||
|
entries, err := fs.ReadDir(source, directory)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
var archive bytes.Buffer
|
||||||
|
writer := zip.NewWriter(&archive)
|
||||||
|
manifest := []File{}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
total := 0
|
||||||
|
for _, entry := range entries {
|
||||||
|
if entry.IsDir() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
parts := filename.FindStringSubmatch(entry.Name())
|
||||||
|
if parts == nil {
|
||||||
|
if strings.HasSuffix(entry.Name(), ".sql") {
|
||||||
|
return nil, nil, fmt.Errorf("migrate: invalid migration filename %s", entry.Name())
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(manifest) >= 1000 {
|
||||||
|
return nil, nil, errors.New("migrate: too many files")
|
||||||
|
}
|
||||||
|
version, err := strconv.ParseUint(parts[1], 10, 32)
|
||||||
|
if err != nil || version == 0 {
|
||||||
|
return nil, nil, errors.New("migrate: version must be 1..4294967295")
|
||||||
|
}
|
||||||
|
identity := fmt.Sprint(version) + parts[3]
|
||||||
|
if seen[identity] {
|
||||||
|
return nil, nil, errors.New("migrate: duplicate migration version/direction")
|
||||||
|
}
|
||||||
|
seen[identity] = true
|
||||||
|
file, err := source.Open(path.Join(directory, entry.Name()))
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
|
||||||
|
closeErr := file.Close()
|
||||||
|
if err == nil {
|
||||||
|
err = closeErr
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
total += len(data)
|
||||||
|
if len(data) == 0 || len(data) > 1<<20 || total > 16<<20 {
|
||||||
|
return nil, nil, errors.New("migrate: files must be 1..1MiB and <=16MiB total")
|
||||||
|
}
|
||||||
|
sum := sha256.Sum256(data)
|
||||||
|
manifest = append(manifest, File{Version: version, Name: parts[2], Direction: parts[3], SHA256: hex.EncodeToString(sum[:])})
|
||||||
|
fileWriter, err := writer.CreateHeader(&zip.FileHeader{Name: entry.Name(), Method: zip.Store})
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
if _, err = fileWriter.Write(data); err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(manifest) == 0 {
|
||||||
|
return nil, nil, errors.New("migrate: no SQL migrations")
|
||||||
|
}
|
||||||
|
for _, file := range manifest {
|
||||||
|
if file.Direction == "down" && !seen[fmt.Sprint(file.Version)+"up"] {
|
||||||
|
return nil, nil, errors.New("migrate: down file lacks up migration")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Slice(manifest, func(i, j int) bool {
|
||||||
|
if manifest[i].Version == manifest[j].Version {
|
||||||
|
return manifest[i].Direction < manifest[j].Direction
|
||||||
|
}
|
||||||
|
return manifest[i].Version < manifest[j].Version
|
||||||
|
})
|
||||||
|
if err = writer.Close(); err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
files, err := zip.NewReader(bytes.NewReader(archive.Bytes()), int64(archive.Len()))
|
||||||
|
return files, manifest, err
|
||||||
|
}
|
||||||
|
|
||||||
|
type Runner struct {
|
||||||
|
mu sync.Mutex
|
||||||
|
engine *migration.Migrate
|
||||||
|
db *sql.DB
|
||||||
|
closed bool
|
||||||
|
stopped bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewMySQL(ctx context.Context, dsn string, source fs.FS, directory string) (runner *Runner, err error) {
|
||||||
|
if ctx == nil {
|
||||||
|
return nil, errors.New("migrate: context required")
|
||||||
|
}
|
||||||
|
snapshot, _, err := Snapshot(source, directory)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
cfg, err := mysqlDriver.ParseDSN(dsn)
|
||||||
|
if err != nil || cfg.DBName == "" {
|
||||||
|
return nil, errors.New("migrate: valid MySQL DSN with database required")
|
||||||
|
}
|
||||||
|
cfg.MultiStatements = true
|
||||||
|
if cfg.Timeout == 0 || cfg.Timeout > 5*time.Second {
|
||||||
|
cfg.Timeout = 5 * time.Second
|
||||||
|
}
|
||||||
|
if cfg.ReadTimeout == 0 || cfg.ReadTimeout > 30*time.Second {
|
||||||
|
cfg.ReadTimeout = 30 * time.Second
|
||||||
|
}
|
||||||
|
if cfg.WriteTimeout == 0 || cfg.WriteTimeout > 30*time.Second {
|
||||||
|
cfg.WriteTimeout = 30 * time.Second
|
||||||
|
}
|
||||||
|
connector, err := mysqlDriver.NewConnector(cfg)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
db := sql.OpenDB(connector)
|
||||||
|
db.SetMaxOpenConns(1)
|
||||||
|
db.SetMaxIdleConns(1)
|
||||||
|
defer func() {
|
||||||
|
if err != nil {
|
||||||
|
db.Close()
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
conn, err := db.Conn(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
driver, err := migrateMySQL.WithConnection(ctx, conn, &migrateMySQL.Config{DatabaseName: cfg.DBName, StatementTimeout: 30 * time.Second})
|
||||||
|
if err != nil {
|
||||||
|
conn.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
src, err := iofs.New(snapshot, ".")
|
||||||
|
if err != nil {
|
||||||
|
driver.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
engine, err := migration.NewWithInstance("iofs", src, cfg.DBName, driver)
|
||||||
|
if err != nil {
|
||||||
|
src.Close()
|
||||||
|
driver.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
engine.LockTimeout = 10 * time.Second
|
||||||
|
return &Runner{engine: engine, db: db}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Up applies pending up scripts. MySQL DDL can partially commit; a failed
|
||||||
|
// migration remains dirty and requires operator investigation. No automatic
|
||||||
|
// down/force/dirty reset is provided. Cancellation stops after the current SQL
|
||||||
|
// operation; statements/read I/O are bounded to 30 seconds.
|
||||||
|
func (r *Runner) Up(ctx context.Context) error {
|
||||||
|
if ctx == nil {
|
||||||
|
return errors.New("migrate: context required")
|
||||||
|
}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
if r.closed || r.stopped {
|
||||||
|
return errors.New("migrate: closed")
|
||||||
|
}
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() { done <- r.engine.Up() }()
|
||||||
|
var err error
|
||||||
|
select {
|
||||||
|
case err = <-done:
|
||||||
|
case <-ctx.Done():
|
||||||
|
r.stopped = true // golang-migrate retains its graceful-stop state; create a fresh runner to retry.
|
||||||
|
select {
|
||||||
|
case r.engine.GracefulStop <- true:
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
err = <-done
|
||||||
|
return errors.Join(ctx.Err(), err)
|
||||||
|
}
|
||||||
|
if errors.Is(err, migration.ErrNoChange) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
func (r *Runner) Version() (uint, bool, error) {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
if r.closed {
|
||||||
|
return 0, false, errors.New("migrate: closed")
|
||||||
|
}
|
||||||
|
version, dirty, err := r.engine.Version()
|
||||||
|
if errors.Is(err, migration.ErrNilVersion) {
|
||||||
|
return 0, false, nil
|
||||||
|
}
|
||||||
|
return version, dirty, err
|
||||||
|
}
|
||||||
|
func (r *Runner) Close() error {
|
||||||
|
r.mu.Lock()
|
||||||
|
defer r.mu.Unlock()
|
||||||
|
if r.closed {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
r.closed = true
|
||||||
|
a, b := r.engine.Close()
|
||||||
|
return errors.Join(a, b, r.db.Close())
|
||||||
|
}
|
||||||
99
database/migrate/migrate_test.go
Normal file
99
database/migrate/migrate_test.go
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
package migrate
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||||
|
"go.opentelemetry.io/otel/sdk/trace/tracetest"
|
||||||
|
"io/fs"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
"testing/fstest"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestSnapshotBoundsAndIdentity(t *testing.T) {
|
||||||
|
source := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("SELECT 1;")}}
|
||||||
|
f, manifest, err := Snapshot(source, ".")
|
||||||
|
if err != nil || len(manifest) != 1 {
|
||||||
|
t.Fatal(manifest, err)
|
||||||
|
}
|
||||||
|
source["0001_createItems.up.sql"].Data[0] = 'x'
|
||||||
|
raw, err := fs.ReadFile(f, "0001_createItems.up.sql")
|
||||||
|
if err != nil || string(raw) != "SELECT 1;" {
|
||||||
|
t.Fatal(string(raw), err)
|
||||||
|
}
|
||||||
|
for _, files := range []fstest.MapFS{{"0001_bad_name.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: []byte("x")}, "1_b.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.down.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: make([]byte, (1<<20)+1)}}} {
|
||||||
|
if _, _, err = Snapshot(files, "."); err == nil {
|
||||||
|
t.Fatal("invalid migration accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func TestMySQLIntegration(t *testing.T) {
|
||||||
|
dsn := os.Getenv("GO_OMNI_MIGRATE_TEST_DSN")
|
||||||
|
if dsn == "" {
|
||||||
|
t.Skip("requires disposable MySQL database")
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
files := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("CREATE TABLE testItems (id BIGINT NOT NULL PRIMARY KEY, value VARCHAR(64) NOT NULL);")}, "0002_seedItems.up.sql": &fstest.MapFile{Data: []byte("INSERT INTO testItems (id,value) VALUES (1,'original');")}}
|
||||||
|
runner, err := NewMySQL(ctx, dsn, files, ".")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = runner.Up(ctx); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = runner.Up(ctx); err != nil {
|
||||||
|
t.Fatal("repeat", err)
|
||||||
|
}
|
||||||
|
v, dirty, err := runner.Version()
|
||||||
|
if err != nil || v != 2 || dirty {
|
||||||
|
t.Fatal(v, dirty, err)
|
||||||
|
}
|
||||||
|
runner.Close()
|
||||||
|
db, err := sql.Open("mysql", dsn)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer db.Close()
|
||||||
|
exporter := tracetest.NewInMemoryExporter()
|
||||||
|
tr, _ := telemetry.New(telemetry.Config{Service: "dbTest", SampleRatio: 1, Exporter: exporter})
|
||||||
|
defer tr.Close()
|
||||||
|
instrumented := telemetry.SQL{DB: db, Runtime: tr}
|
||||||
|
if _, err = instrumented.ExecContext(ctx, "UPDATE testItems SET value=? WHERE id=1", "secret parameter"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rows, err := instrumented.QueryContext(ctx, "SELECT value FROM testItems WHERE id=1")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !rows.Next() {
|
||||||
|
t.Fatal("missing row")
|
||||||
|
}
|
||||||
|
var value string
|
||||||
|
if err = rows.Scan(&value); err != nil || value != "secret parameter" {
|
||||||
|
t.Fatal(value, err)
|
||||||
|
}
|
||||||
|
rows.Close()
|
||||||
|
tr.Flush(ctx)
|
||||||
|
if len(exporter.GetSpans()) != 2 {
|
||||||
|
t.Fatal("SQL spans missing")
|
||||||
|
}
|
||||||
|
files["0003_failMigration.up.sql"] = &fstest.MapFile{Data: []byte("INSERT INTO missingTable VALUES (1);")}
|
||||||
|
runner, err = NewMySQL(ctx, dsn, files, ".")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer runner.Close()
|
||||||
|
if err = runner.Up(ctx); err == nil {
|
||||||
|
t.Fatal("bad migration passed")
|
||||||
|
}
|
||||||
|
v, dirty, err = runner.Version()
|
||||||
|
if err != nil || v != 3 || !dirty {
|
||||||
|
t.Fatal(v, dirty, err)
|
||||||
|
}
|
||||||
|
if err = runner.Up(ctx); err == nil {
|
||||||
|
t.Fatal("dirty migration auto-cleared")
|
||||||
|
}
|
||||||
|
}
|
||||||
15
examples/contracts/README.md
Normal file
15
examples/contracts/README.md
Normal file
@@ -0,0 +1,15 @@
|
|||||||
|
# 接口定义示例
|
||||||
|
|
||||||
|
`user.api` 是入口,导入 `shared.api`。包含 JSON/path/query/header 混合请求、分页与重复 query 参数、嵌套对象、optional 指针、user 与 admin/user 分组以及 Auth/Audit 命名中间件。
|
||||||
|
|
||||||
|
在 go-omni 根目录执行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go run ./cmd/go-omni api validate --api examples/contracts/user.api
|
||||||
|
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
|
||||||
|
--dir ../userService --module example.com/userService --framework-dir .
|
||||||
|
```
|
||||||
|
|
||||||
|
项目中补充 `internal/service` 的业务实现和 `internal/middleware` 的 Auth/Audit hook,再启动服务。初始占位实现返回 501。
|
||||||
|
|
||||||
|
需要立即能返回成功结果的最小项目,用 `go-omni api new` 创建问候示例。完整语法、目录、更新策略见 [API.md](../../API.md)。
|
||||||
11
examples/contracts/shared.api
Normal file
11
examples/contracts/shared.api
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
syntax = "v1"
|
||||||
|
|
||||||
|
type Address {
|
||||||
|
City string `json:"city" binding:"required,max=50"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type UserResp {
|
||||||
|
UserID int64 `json:"userId"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Active bool `json:"active"`
|
||||||
|
}
|
||||||
58
examples/contracts/user.api
Normal file
58
examples/contracts/user.api
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
syntax = "v1"
|
||||||
|
|
||||||
|
info (
|
||||||
|
title: "User API"
|
||||||
|
version: "1.0.0"
|
||||||
|
)
|
||||||
|
|
||||||
|
import "shared.api"
|
||||||
|
|
||||||
|
type (
|
||||||
|
UpdateUserReq {
|
||||||
|
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||||
|
Notify bool `form:"notify,default=false"`
|
||||||
|
Client string `header:"X-Client,optional"`
|
||||||
|
Name string `json:"name" binding:"required,max=50"`
|
||||||
|
Active bool `json:"active"`
|
||||||
|
Age *int `json:"age,optional,range=[0:150]"`
|
||||||
|
Address *Address `json:"address,optional"`
|
||||||
|
}
|
||||||
|
ListUsersReq {
|
||||||
|
Page int `form:"page,default=1,range=[1:1000]"`
|
||||||
|
PageSize int `form:"pageSize,default=20,range=[1:100]"`
|
||||||
|
Tags []string `form:"tag,optional" binding:"max=10"`
|
||||||
|
}
|
||||||
|
ListUsersResp {
|
||||||
|
Items []UserResp `json:"items"`
|
||||||
|
Page int `json:"page"`
|
||||||
|
Total int64 `json:"total"`
|
||||||
|
}
|
||||||
|
DeleteUserReq {
|
||||||
|
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
@server (
|
||||||
|
group: user
|
||||||
|
prefix: /api/v1
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@doc "Update a user"
|
||||||
|
@handler updateUser
|
||||||
|
put /users/:userId (UpdateUserReq) returns (UserResp)
|
||||||
|
|
||||||
|
@doc "List users"
|
||||||
|
@handler listUsers
|
||||||
|
get /users (ListUsersReq) returns (ListUsersResp)
|
||||||
|
}
|
||||||
|
|
||||||
|
@server (
|
||||||
|
group: admin/user
|
||||||
|
prefix: /admin/v1
|
||||||
|
middleware: Auth,Audit
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@doc "Delete a user; business-owned authentication and auditing required"
|
||||||
|
@handler deleteUser
|
||||||
|
delete /users/:userId (DeleteUserReq)
|
||||||
|
}
|
||||||
14
examples/platform/README.md
Normal file
14
examples/platform/README.md
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
# 扩展模块组合示例
|
||||||
|
|
||||||
|
业务通过 `storage.Store` 写文件,身份接口和 Policy 控制权限,OpenAPI 与路由一起注册,链路追踪关联 HTTP 与文件操作。`newApp` 不依赖 Local/S3/OSS 类型;启动处本例选择 Local。可替换成 S3/R2/OSS 构造结果,业务路由不变。
|
||||||
|
|
||||||
|
```sh
|
||||||
|
export GO_OMNI_DEMO_TOKEN='use-a-private-development-token'
|
||||||
|
go run ./examples/platform --root /tmp/goOmniDemoFiles
|
||||||
|
```
|
||||||
|
|
||||||
|
接口:`POST /api/files`,JSON 为 `{"name":"hello.txt","content":"hello"}`,请求加 `Authorization: Bearer <GO_OMNI_DEMO_TOKEN>`;返回 201 和存储信息。`GET /openapi.json` 同样要求身份。
|
||||||
|
|
||||||
|
也可传 `--config service.yaml`,配置规则见 [扩展模块](../../ADVANCED.md)。本例的静态身份只用于演示,实际应用注入 `auth.JWT` 或 `auth.Sessions`;trace provider 默认不外发,接入 collector 时注入 exporter/reader。
|
||||||
|
|
||||||
|
`TestPlatformTLSAdapters` 通过三个 Runner 的真实 TLS 请求验证 401/403/201、文件内容、非法 key、OpenAPI 和关闭。文件根目录必须是应用私有目录。示例不执行自动数据迁移,也不启动 Orbit。
|
||||||
132
examples/platform/main.go
Normal file
132
examples/platform/main.go
Normal file
@@ -0,0 +1,132 @@
|
|||||||
|
// platform composes storage, authentication, permission policy, tracing and
|
||||||
|
// OpenAPI using provider-neutral business dependencies.
|
||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"flag"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||||
|
)
|
||||||
|
|
||||||
|
type dependencies struct {
|
||||||
|
Files storage.Store
|
||||||
|
Auth auth.Authenticator
|
||||||
|
Tracing *telemetry.Runtime
|
||||||
|
}
|
||||||
|
type upload struct {
|
||||||
|
Name string `json:"name" binding:"required,max=128"`
|
||||||
|
Content string `json:"content" binding:"max=4096"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Dependencies are borrowed; the composition root owns their shutdown.
|
||||||
|
func newApp(cfg omni.Config, d dependencies) (*omni.App, error) {
|
||||||
|
if d.Files == nil || d.Auth == nil || d.Tracing == nil {
|
||||||
|
return nil, errors.New("platform: dependencies required")
|
||||||
|
}
|
||||||
|
app := omni.New(omni.WithConfig(cfg))
|
||||||
|
app.Use(d.Tracing.Middleware())
|
||||||
|
api, err := openapi.New("File service", "1")
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
input, err := openapi.SchemaOf[upload]()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
output, err := openapi.SchemaOf[storage.Info]()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
policy := auth.PolicyFunc(func(_ context.Context, p auth.Principal, action, resource string) error {
|
||||||
|
if p.TenantID != "" && p.HasRole("writer") && action == "create" && resource == "files" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return errors.New("denied")
|
||||||
|
})
|
||||||
|
err = api.Handle(app, openapi.Operation{Method: "POST", Path: "/api/files", OperationID: "createFile", Summary: "Store a small text file", Request: input, BearerAuth: true, Responses: map[int]openapi.Response{201: {Description: "Created", Schema: openapi.Envelope(output)}}}, func(c *omni.Context) error {
|
||||||
|
var in upload
|
||||||
|
if err := c.BindJSON(&in); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
p, _ := auth.FromContext(c)
|
||||||
|
if strings.ContainsAny(in.Name, "/\\") {
|
||||||
|
return omni.BadRequest("INVALID_NAME", "name must be one path segment")
|
||||||
|
}
|
||||||
|
key := p.TenantID + "/" + in.Name
|
||||||
|
if err := storage.Check(c.Context(), key); err != nil {
|
||||||
|
return omni.BadRequest("INVALID_KEY", "invalid storage key")
|
||||||
|
}
|
||||||
|
var info storage.Info
|
||||||
|
err := d.Tracing.Do(c.Context(), "files.put", func(ctx context.Context) error {
|
||||||
|
var err error
|
||||||
|
info, err = d.Files.Put(ctx, key, strings.NewReader(in.Content), int64(len(in.Content)), "text/plain; charset=utf-8")
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return c.Created(info)
|
||||||
|
}, auth.RequireAuth(d.Auth), auth.RequirePermission(policy, "create", func(*omni.Context) string { return "files" }))
|
||||||
|
if err != nil {
|
||||||
|
app.Close()
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
app.GET("/openapi.json", api.Handler(), auth.RequireAuth(d.Auth))
|
||||||
|
return app, nil
|
||||||
|
}
|
||||||
|
func run(ctx context.Context, configFile, root string) error {
|
||||||
|
var cfg config.Config
|
||||||
|
var err error
|
||||||
|
if configFile != "" {
|
||||||
|
cfg, err = config.LoadFile(configFile, "")
|
||||||
|
} else {
|
||||||
|
cfg, err = config.LoadFromEnv("")
|
||||||
|
if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
|
||||||
|
cfg.Address = "127.0.0.1:8080"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
token := os.Getenv("GO_OMNI_DEMO_TOKEN")
|
||||||
|
if token == "" {
|
||||||
|
return errors.New("set GO_OMNI_DEMO_TOKEN for the demonstration credential")
|
||||||
|
}
|
||||||
|
files, err := local.New(root, 64<<10)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
tracing, err := telemetry.New(telemetry.Config{Service: "fileService", SampleRatio: 1})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer tracing.Close()
|
||||||
|
app, err := newApp(cfg, dependencies{Files: files, Auth: auth.StaticAuthenticator{token: {UserID: "demoUser", TenantID: "demoTenant", Roles: []string{"writer"}}}, Tracing: tracing})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return app.RunContext(ctx)
|
||||||
|
}
|
||||||
|
func main() {
|
||||||
|
file := flag.String("config", "", "optional JSON/YAML config")
|
||||||
|
root := flag.String("root", "./fileData", "private local object directory")
|
||||||
|
flag.Parse()
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
if err := run(ctx, *file, *root); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
102
examples/platform/main_test.go
Normal file
102
examples/platform/main_test.go
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestPlatformTLSAdapters(t *testing.T) {
|
||||||
|
for _, name := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
cfg := config.Default()
|
||||||
|
cfg.Transport = name
|
||||||
|
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||||
|
cfg.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||||
|
store, _ := local.New(t.TempDir(), 64<<10)
|
||||||
|
tr, _ := telemetry.New(telemetry.Config{Service: "platformTest", SampleRatio: 1})
|
||||||
|
defer tr.Close()
|
||||||
|
app, err := newApp(cfg, dependencies{Files: store, Auth: auth.StaticAuthenticator{"writer": {UserID: "u", TenantID: "t", Roles: []string{"writer"}}, "reader": {UserID: "r", TenantID: "t"}}, Tracing: tr})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() { done <- app.ServeContext(ctx, listener) }()
|
||||||
|
defer func() {
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case err := <-done:
|
||||||
|
if err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
}
|
||||||
|
case <-time.After(5 * time.Second):
|
||||||
|
listener.Close()
|
||||||
|
t.Error("shutdown timeout")
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||||
|
defer client.CloseIdleConnections()
|
||||||
|
request := func(method, path, body, token string) (int, string) {
|
||||||
|
t.Helper()
|
||||||
|
r, _ := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
|
||||||
|
r.Header.Set("Authorization", "Bearer "+token)
|
||||||
|
r.Header.Set("Content-Type", "application/json")
|
||||||
|
response, err := client.Do(r)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
raw, err := io.ReadAll(response.Body)
|
||||||
|
response.Body.Close()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
|
||||||
|
t.Fatal("TLS unverified")
|
||||||
|
}
|
||||||
|
if name == omni.TransportHTTP2 && response.ProtoMajor != 2 {
|
||||||
|
t.Fatal("protocol")
|
||||||
|
}
|
||||||
|
return response.StatusCode, string(raw)
|
||||||
|
}
|
||||||
|
for _, test := range []struct {
|
||||||
|
token string
|
||||||
|
status int
|
||||||
|
}{{"", 401}, {"reader", 403}, {"writer", 201}} {
|
||||||
|
status, body := request("POST", "/api/files", `{"name":"hello.txt","content":"hello"}`, test.token)
|
||||||
|
if status != test.status {
|
||||||
|
t.Fatal(status, body)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
object, err := store.Get(context.Background(), "t/hello.txt")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
raw, _ := io.ReadAll(object.Body)
|
||||||
|
object.Body.Close()
|
||||||
|
if string(raw) != "hello" {
|
||||||
|
t.Fatal(string(raw))
|
||||||
|
}
|
||||||
|
if status, _ := request("POST", "/api/files", `{"name":"../escape","content":"x"}`, "writer"); status != 400 {
|
||||||
|
t.Fatal(status)
|
||||||
|
}
|
||||||
|
if status, body := request("GET", "/openapi.json", "", "writer"); status != 200 || !strings.Contains(body, `"/api/files"`) {
|
||||||
|
t.Fatal(status, body)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
6
examples/userService/.gitignore
vendored
Normal file
6
examples/userService/.gitignore
vendored
Normal file
@@ -0,0 +1,6 @@
|
|||||||
|
/server
|
||||||
|
/var/
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
|
/.goOmni/api.lock
|
||||||
19
examples/userService/.goOmni/apiManifest.json
Normal file
19
examples/userService/.goOmni/apiManifest.json
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
{
|
||||||
|
"profile": {
|
||||||
|
"database": "mysql",
|
||||||
|
"storage": "local",
|
||||||
|
"logOutput": "file",
|
||||||
|
"logFormat": "json"
|
||||||
|
},
|
||||||
|
"version": 1,
|
||||||
|
"module": "example.com/userService",
|
||||||
|
"entry": "api/user.api",
|
||||||
|
"generated": {
|
||||||
|
"docs/openapi.json": "0675e4eb1278b36828aa31b12587b0d33d2f384112f67a43179eb72b52c3a0ea",
|
||||||
|
"internal/handler/openapiGen.go": "8ddc60c338630db915b9f074ade776f526eabf60fc99b33e425359fd415c4926",
|
||||||
|
"internal/handler/routesGen.go": "7f45030e7eb9f055c4b4d2ff12ad855d7c726aad4aee63c5b3c3f97559a64257",
|
||||||
|
"internal/handler/user/handlersGen.go": "06f3c2798ff323812ff3eb2d8b933a569af3ce1b86a67b66036374add98c0d7e",
|
||||||
|
"internal/middleware/registryGen.go": "a6122a5453dedc6a4415860108814c50e155128cfe5412b39dd33ce2d01b4b00",
|
||||||
|
"internal/types/typesGen.go": "3ab2ce9ffe467c3ea53ac3d1fec283ecebc87cef57fd592d08dca991fd964876"
|
||||||
|
}
|
||||||
|
}
|
||||||
79
examples/userService/README.md
Normal file
79
examples/userService/README.md
Normal file
@@ -0,0 +1,79 @@
|
|||||||
|
# API 驱动的用户管理示例
|
||||||
|
|
||||||
|
这个独立 Go module 从 `api/user.api` 生成,随后在开发者拥有的文件中实现业务。
|
||||||
|
使用 MySQL 持久化、事务审计、环境变量 Token 鉴权、文件日志和 local 存储;业务层只接收标准 context 和基础模块接口。
|
||||||
|
|
||||||
|
## 启动
|
||||||
|
|
||||||
|
先准备空的业务数据库,再设置环境变量(替换占位值,密钥不写进配置或 Git):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
export GO_OMNI_DB_DSN='user:password@tcp(127.0.0.1:3306)/user_service?parseTime=true'
|
||||||
|
export USER_SERVICE_API_TOKEN='<至少32字节的随机值>'
|
||||||
|
export GO_OMNI_STORAGE_SIGNING_KEY='<另一个至少32字节的随机值>'
|
||||||
|
export GO_OMNI_STORAGE_PUBLIC_URL='http://127.0.0.1:8080/storage/object'
|
||||||
|
go mod tidy
|
||||||
|
# 在本目录运行与框架版本一致的 CLI;迁移必须显式执行。
|
||||||
|
go-omni migrate --dir migrations --apply
|
||||||
|
go run ./cmd/server
|
||||||
|
```
|
||||||
|
|
||||||
|
源码示例的 go.mod 使用 `replace ... => ../..`。复制到独立仓库时,删除这条 replace,
|
||||||
|
保留已发布的固定依赖版本,再执行 go mod tidy。远程版本验证脚本会自动检查没有本地替换。
|
||||||
|
`GO_OMNI_TRANSPORT=native` 可切换为 native,业务代码无需调整。
|
||||||
|
|
||||||
|
配置是默认值 → etc/config.yaml → 环境变量。日志默认写入 var/log/server.log 并轮转,
|
||||||
|
`GO_OMNI_LOG_OUTPUT=stderr` 可以切换为标准错误输出。MySQL 默认最多 10 条连接。
|
||||||
|
应用检查 users 和 user_audit 表,缺失时拒绝启动,不自动改表。
|
||||||
|
|
||||||
|
## 接口
|
||||||
|
|
||||||
|
除健康检查、OpenAPI 和持有效签名的对象下载外,业务接口需要 `Authorization: Bearer <Token>`。
|
||||||
|
这个 Token 代表示例的管理员;可通过同一个命名中间件槽位替换为 JWT/OIDC 和业务授权策略。
|
||||||
|
框架不会推断租户模型或自动给接口放行。
|
||||||
|
|
||||||
|
| 接口 | 行为 |
|
||||||
|
| --- | --- |
|
||||||
|
| POST /api/v1/users | 创建,返回 201;邮箱规范化并唯一 |
|
||||||
|
| GET /api/v1/users/:id | 查询,缺失返回 404 |
|
||||||
|
| GET /api/v1/users?page=1&pageSize=20 | 分页,pageSize 最大 100,空结果返回 [] |
|
||||||
|
| PUT /api/v1/users/:id | 完整更新;邮箱冲突返回 409 |
|
||||||
|
| DELETE /api/v1/users/:id | 删除,返回 204 |
|
||||||
|
| POST /api/v1/users/:id/export | 私有文本导出,返回 5 分钟下载链接 |
|
||||||
|
| GET /health/ready | 数据库不可用时返回 503,恢复后重新就绪 |
|
||||||
|
| GET /openapi.json | 与 docs/openapi.json 一致 |
|
||||||
|
|
||||||
|
参数不合法返回 400,认证失败 401,邮箱重复 409,内部故障只返回通用 500。
|
||||||
|
OpenAPI 的业务数据类型来自契约;401/404/409 的业务错误码见上表。
|
||||||
|
|
||||||
|
创建、更新、删除都会在**同一个 MySQL 事务**写入 user_audit;审计写入失败时业务变更一起回滚。
|
||||||
|
事务提交后才写成功业务日志,requestId 同时出现在 HTTP 响应、业务日志和审计记录。
|
||||||
|
数据库异常日志不记录 SQL、DSN 或用户输入。
|
||||||
|
|
||||||
|
文件导出演示 `deps.Uploads` 和 `deps.Downloads`。对象使用随机 key,签名 URL 不进入日志。
|
||||||
|
签名过期只限制下载,不删除文件;实际项目应给 exports/ 配置保留期限或清理任务。
|
||||||
|
只签发下载链接,不对外开放任意对象上传或签名接口。
|
||||||
|
|
||||||
|
## 开发与验证
|
||||||
|
|
||||||
|
```sh
|
||||||
|
go-omni api generate --api api/user.api --dir .
|
||||||
|
go-omni api generate --api api/user.api --dir . --check
|
||||||
|
go test ./...
|
||||||
|
```
|
||||||
|
|
||||||
|
Gen.go 和 docs/openapi.json 由生成器维护;Service、Repository、迁移、鉴权、配置和 main 保留。
|
||||||
|
修改 .api 的类型/方法签名后,需要同步修改自己的业务实现并运行测试。
|
||||||
|
|
||||||
|
在框架仓库运行:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
python3 scripts/verifyAPIWorkflow.py
|
||||||
|
# 已发布版本:全新 module cache 安装 CLI,无本地 replace 消费框架。
|
||||||
|
python3 scripts/verifyAPIWorkflow.py --version v0.7.0-preview \
|
||||||
|
--report performance/apiWorkflowRemote.json
|
||||||
|
```
|
||||||
|
|
||||||
|
验收创建只绑定 localhost 的临时 MySQL 容器(1 CPU / 768 MiB),使用随机密钥,结束时删除。
|
||||||
|
覆盖 nethttp/native 的 CRUD、校验、鉴权、事务错误/异常/取消回滚、数据库停机恢复、
|
||||||
|
重启持久化、签名文件下载、文件日志与优雅退出。不会连接业务服务器。
|
||||||
60
examples/userService/api/user.api
Normal file
60
examples/userService/api/user.api
Normal file
@@ -0,0 +1,60 @@
|
|||||||
|
syntax = "v1"
|
||||||
|
info (
|
||||||
|
title: "User service reference"
|
||||||
|
version: "1.0.0"
|
||||||
|
)
|
||||||
|
type (
|
||||||
|
UserResp {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Email string `json:"email"`
|
||||||
|
CreatedAt string `json:"createdAt"`
|
||||||
|
UpdatedAt string `json:"updatedAt"`
|
||||||
|
}
|
||||||
|
CreateUserReq {
|
||||||
|
Name string `json:"name" binding:"required,max=100"`
|
||||||
|
Email string `json:"email" binding:"required,email,max=254"`
|
||||||
|
}
|
||||||
|
UserIDReq {
|
||||||
|
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||||
|
}
|
||||||
|
UpdateUserReq {
|
||||||
|
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||||
|
Name string `json:"name" binding:"required,max=100"`
|
||||||
|
Email string `json:"email" binding:"required,email,max=254"`
|
||||||
|
}
|
||||||
|
ListUsersReq {
|
||||||
|
Page int `form:"page,default=1,range=[1:1000000]"`
|
||||||
|
PageSize int `form:"pageSize,default=20,range=[1:100]"`
|
||||||
|
}
|
||||||
|
UserPageResp {
|
||||||
|
Items []UserResp `json:"items"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
Page int `json:"page"`
|
||||||
|
PageSize int `json:"pageSize"`
|
||||||
|
}
|
||||||
|
ExportResp {
|
||||||
|
DownloadURL string `json:"downloadUrl"`
|
||||||
|
}
|
||||||
|
)
|
||||||
|
@server (
|
||||||
|
group: user
|
||||||
|
prefix: /api/v1
|
||||||
|
middleware: Auth
|
||||||
|
)
|
||||||
|
service user-api {
|
||||||
|
@handler createUser
|
||||||
|
@status 201
|
||||||
|
post /users (CreateUserReq) returns (UserResp)
|
||||||
|
@handler getUser
|
||||||
|
get /users/:id (UserIDReq) returns (UserResp)
|
||||||
|
@handler listUsers
|
||||||
|
get /users (ListUsersReq) returns (UserPageResp)
|
||||||
|
@handler updateUser
|
||||||
|
put /users/:id (UpdateUserReq) returns (UserResp)
|
||||||
|
@handler deleteUser
|
||||||
|
delete /users/:id (UserIDReq)
|
||||||
|
@doc "Export one user as a private text object with a short-lived download link"
|
||||||
|
@handler exportUser
|
||||||
|
post /users/:id/export (UserIDReq) returns (ExportResp)
|
||||||
|
}
|
||||||
58
examples/userService/cmd/server/main.go
Normal file
58
examples/userService/cmd/server/main.go
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"example.com/userService/internal/handler"
|
||||||
|
"example.com/userService/internal/servicecontext"
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"flag"
|
||||||
|
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"log"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
func main() {
|
||||||
|
path := flag.String("config", "etc/config.yaml", "configuration file; empty uses defaults and environment")
|
||||||
|
flag.Parse()
|
||||||
|
if err := run(*path); err != nil {
|
||||||
|
log.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func run(path string) (runErr error) {
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
cfg, err := settings.Load(path)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
logger, err := logx.New(cfg.Server.Log)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer func() { runErr = errors.Join(runErr, logger.Close()) }()
|
||||||
|
logx.SetDefault(logger)
|
||||||
|
deps, err := servicecontext.New(ctx, logger, cfg)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer func() { runErr = errors.Join(runErr, deps.Close()) }()
|
||||||
|
app := omni.New(omni.WithConfig(cfg.Server))
|
||||||
|
app.OnShutdown(logger.Close)
|
||||||
|
app.OnShutdown(deps.Close)
|
||||||
|
for _, method := range []string{"GET", "HEAD", "PUT"} {
|
||||||
|
app.Handle(method, "/storage/object", func(c *httpx.Context) error { deps.FileHTTP.ServeHTTP(c.Raw(), c.Request()); return nil })
|
||||||
|
}
|
||||||
|
if err = handler.RegisterRoutes(app, deps); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
app.GET("/openapi.json", handler.OpenAPI)
|
||||||
|
app.GET("/health/live", deps.Health.LivenessHandler())
|
||||||
|
app.GET("/health/ready", deps.Health.ReadinessHandler())
|
||||||
|
logger.InfoContext(ctx, "service.starting", "address", cfg.Server.Address)
|
||||||
|
return app.RunContext(ctx, cfg.Server.Address)
|
||||||
|
}
|
||||||
1249
examples/userService/docs/openapi.json
Normal file
1249
examples/userService/docs/openapi.json
Normal file
File diff suppressed because it is too large
Load Diff
36
examples/userService/etc/config.yaml
Normal file
36
examples/userService/etc/config.yaml
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
# Non-secret settings only. Environment variables override these values.
|
||||||
|
server:
|
||||||
|
address: '127.0.0.1:8080'
|
||||||
|
enableLogger: true
|
||||||
|
slowRequestThreshold: 500ms
|
||||||
|
shutdownTimeout: 10s
|
||||||
|
log:
|
||||||
|
service: "example.com/userService"
|
||||||
|
level: info
|
||||||
|
format: json
|
||||||
|
file: './var/log/server.log'
|
||||||
|
rotation:
|
||||||
|
enabled: true
|
||||||
|
maxSizeMb: 100
|
||||||
|
maxBackups: 10
|
||||||
|
maxAgeDays: 7
|
||||||
|
compress: true
|
||||||
|
redactKeys: []
|
||||||
|
storage:
|
||||||
|
provider: local
|
||||||
|
maxBytes: 33554432
|
||||||
|
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||||
|
linkTtl: 5m
|
||||||
|
requestTimeout: 30s
|
||||||
|
root: './var/files'
|
||||||
|
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||||
|
|
||||||
|
|
||||||
|
database:
|
||||||
|
driver: mysql
|
||||||
|
maxOpenConns: 10
|
||||||
|
maxIdleConns: 5
|
||||||
|
connMaxLifetime: 30m
|
||||||
|
connMaxIdleTime: 5m
|
||||||
|
pingTimeout: 5s
|
||||||
|
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||||
26
examples/userService/go.mod
Normal file
26
examples/userService/go.mod
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
module example.com/userService
|
||||||
|
|
||||||
|
go 1.22.0
|
||||||
|
|
||||||
|
require (
|
||||||
|
git.yuntuo.hk/k2safe/go-omni v0.7.0-preview
|
||||||
|
github.com/go-sql-driver/mysql v1.9.3
|
||||||
|
)
|
||||||
|
|
||||||
|
require (
|
||||||
|
filippo.io/edwards25519 v1.1.0 // indirect
|
||||||
|
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||||
|
github.com/go-playground/locales v0.14.1 // indirect
|
||||||
|
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||||
|
github.com/go-playground/validator/v10 v10.20.0 // indirect
|
||||||
|
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
|
||||||
|
github.com/leodido/go-urn v1.4.0 // indirect
|
||||||
|
golang.org/x/crypto v0.31.0 // indirect
|
||||||
|
golang.org/x/net v0.33.0 // indirect
|
||||||
|
golang.org/x/sys v0.28.0 // indirect
|
||||||
|
golang.org/x/text v0.21.0 // indirect
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||||
|
)
|
||||||
|
|
||||||
|
replace git.yuntuo.hk/k2safe/go-omni => ../..
|
||||||
38
examples/userService/go.sum
Normal file
38
examples/userService/go.sum
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
|
||||||
|
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
|
||||||
|
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||||
|
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||||
|
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
||||||
|
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
||||||
|
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
||||||
|
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
||||||
|
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||||
|
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
|
||||||
|
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
|
||||||
|
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
||||||
|
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
|
||||||
|
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
||||||
|
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
|
||||||
|
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
|
||||||
|
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||||
|
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||||
|
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||||
|
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||||
|
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||||
|
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
|
||||||
|
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||||
|
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
|
||||||
|
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
|
||||||
|
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
|
||||||
|
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
|
||||||
|
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
|
||||||
|
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||||
|
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
|
||||||
|
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||||
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
|
||||||
|
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||||
|
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||||
13
examples/userService/internal/handler/openapiGen.go
Normal file
13
examples/userService/internal/handler/openapiGen.go
Normal file
File diff suppressed because one or more lines are too long
27
examples/userService/internal/handler/routesGen.go
Normal file
27
examples/userService/internal/handler/routesGen.go
Normal file
@@ -0,0 +1,27 @@
|
|||||||
|
// Code generated by go-omni api. DO NOT EDIT.
|
||||||
|
package handler
|
||||||
|
|
||||||
|
import (
|
||||||
|
h0 "example.com/userService/internal/handler/user"
|
||||||
|
s0 "example.com/userService/internal/service/user"
|
||||||
|
"example.com/userService/internal/servicecontext"
|
||||||
|
"fmt"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||||
|
)
|
||||||
|
|
||||||
|
func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {
|
||||||
|
if router == nil || deps == nil {
|
||||||
|
return fmt.Errorf("router and dependencies required")
|
||||||
|
}
|
||||||
|
if deps.Middlewares["Auth"] == nil {
|
||||||
|
return fmt.Errorf("missing middleware Auth")
|
||||||
|
}
|
||||||
|
h0 := h0.New(s0.New(deps))
|
||||||
|
router.Handle("POST", "/api/v1/users", h0.CreateUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("GET", "/api/v1/users/:id", h0.GetUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("GET", "/api/v1/users", h0.ListUsers, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("PUT", "/api/v1/users/:id", h0.UpdateUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("DELETE", "/api/v1/users/:id", h0.DeleteUser, deps.Middlewares["Auth"])
|
||||||
|
router.Handle("POST", "/api/v1/users/:id/export", h0.ExportUser, deps.Middlewares["Auth"])
|
||||||
|
return nil
|
||||||
|
}
|
||||||
120
examples/userService/internal/handler/user/handlersGen.go
Normal file
120
examples/userService/internal/handler/user/handlersGen.go
Normal file
@@ -0,0 +1,120 @@
|
|||||||
|
// Code generated by go-omni api. DO NOT EDIT.
|
||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
"fmt"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Service interface {
|
||||||
|
CreateUser(ctx context.Context, req *types.CreateUserReq) (*types.UserResp, error)
|
||||||
|
GetUser(ctx context.Context, req *types.UserIDReq) (*types.UserResp, error)
|
||||||
|
ListUsers(ctx context.Context, req *types.ListUsersReq) (*types.UserPageResp, error)
|
||||||
|
UpdateUser(ctx context.Context, req *types.UpdateUserReq) (*types.UserResp, error)
|
||||||
|
DeleteUser(ctx context.Context, req *types.UserIDReq) error
|
||||||
|
ExportUser(ctx context.Context, req *types.UserIDReq) (*types.ExportResp, error)
|
||||||
|
}
|
||||||
|
type Handler struct{ service Service }
|
||||||
|
|
||||||
|
func New(s Service) *Handler { return &Handler{service: s} }
|
||||||
|
|
||||||
|
var createUserRequest = contract.MustNew[types.CreateUserReq]()
|
||||||
|
|
||||||
|
func (h *Handler) CreateUser(c *httpx.Context) error {
|
||||||
|
var req types.CreateUserReq
|
||||||
|
if err := createUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.CreateUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("CreateUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.Created(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var getUserRequest = contract.MustNew[types.UserIDReq]()
|
||||||
|
|
||||||
|
func (h *Handler) GetUser(c *httpx.Context) error {
|
||||||
|
var req types.UserIDReq
|
||||||
|
if err := getUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.GetUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("GetUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var listUsersRequest = contract.MustNew[types.ListUsersReq]()
|
||||||
|
|
||||||
|
func (h *Handler) ListUsers(c *httpx.Context) error {
|
||||||
|
var req types.ListUsersReq
|
||||||
|
if err := listUsersRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.ListUsers(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("ListUsers returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var updateUserRequest = contract.MustNew[types.UpdateUserReq]()
|
||||||
|
|
||||||
|
func (h *Handler) UpdateUser(c *httpx.Context) error {
|
||||||
|
var req types.UpdateUserReq
|
||||||
|
if err := updateUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.UpdateUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("UpdateUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
|
|
||||||
|
var deleteUserRequest = contract.MustNew[types.UserIDReq]()
|
||||||
|
|
||||||
|
func (h *Handler) DeleteUser(c *httpx.Context) error {
|
||||||
|
var req types.UserIDReq
|
||||||
|
if err := deleteUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := h.service.DeleteUser(c.Context(), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return c.NoContent()
|
||||||
|
}
|
||||||
|
|
||||||
|
var exportUserRequest = contract.MustNew[types.UserIDReq]()
|
||||||
|
|
||||||
|
func (h *Handler) ExportUser(c *httpx.Context) error {
|
||||||
|
var req types.UserIDReq
|
||||||
|
if err := exportUserRequest.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
resp, err := h.service.ExportUser(c.Context(), &req)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if resp == nil {
|
||||||
|
return fmt.Errorf("ExportUser returned a nil response")
|
||||||
|
}
|
||||||
|
return c.OK(resp)
|
||||||
|
}
|
||||||
40
examples/userService/internal/middleware/auth.go
Normal file
40
examples/userService/internal/middleware/auth.go
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
package middleware
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"crypto/subtle"
|
||||||
|
"errors"
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Auth is the fail-closed registry default. Startup replaces it with configured
|
||||||
|
// authentication. This administrative example uses one environment-only key;
|
||||||
|
// production applications can inject JWT/OIDC through the same middleware slot.
|
||||||
|
func Auth(next httpx.Handler) httpx.Handler {
|
||||||
|
return func(c *httpx.Context) error {
|
||||||
|
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func NewAuthFromEnv() (httpx.Middleware, error) {
|
||||||
|
token := os.Getenv("USER_SERVICE_API_TOKEN")
|
||||||
|
if len(token) < 32 || strings.ContainsAny(token, " \t\r\n") {
|
||||||
|
return nil, errors.New("USER_SERVICE_API_TOKEN requires at least 32 bytes without whitespace")
|
||||||
|
}
|
||||||
|
expected := sha256.Sum256([]byte(token))
|
||||||
|
return func(next httpx.Handler) httpx.Handler {
|
||||||
|
return func(c *httpx.Context) error {
|
||||||
|
parts := strings.Fields(c.Header("Authorization"))
|
||||||
|
if len(parts) != 2 || !strings.EqualFold(parts[0], "Bearer") {
|
||||||
|
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||||
|
}
|
||||||
|
actual := sha256.Sum256([]byte(parts[1]))
|
||||||
|
if subtle.ConstantTimeCompare(expected[:], actual[:]) != 1 {
|
||||||
|
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||||
|
}
|
||||||
|
return next(c)
|
||||||
|
}
|
||||||
|
}, nil
|
||||||
|
}
|
||||||
35
examples/userService/internal/middleware/auth_test.go
Normal file
35
examples/userService/internal/middleware/auth_test.go
Normal file
@@ -0,0 +1,35 @@
|
|||||||
|
package middleware
|
||||||
|
|
||||||
|
import (
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAuthFailsClosedAndDoesNotLeakKey(t *testing.T) {
|
||||||
|
t.Setenv("USER_SERVICE_API_TOKEN", "")
|
||||||
|
if _, err := NewAuthFromEnv(); err == nil {
|
||||||
|
t.Fatal("missing credential accepted")
|
||||||
|
}
|
||||||
|
token := "test-key-012345678901234567890123456789"
|
||||||
|
t.Setenv("USER_SERVICE_API_TOKEN", token)
|
||||||
|
middleware, err := NewAuthFromEnv()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, header := range []string{"", "Basic " + token, "Bearer wrong", "Bearer " + token} {
|
||||||
|
called := false
|
||||||
|
r := httptest.NewRequest("GET", "/", nil)
|
||||||
|
r.Header.Set("Authorization", header)
|
||||||
|
c := httpx.NewContext(r, httptest.NewRecorder(), nil)
|
||||||
|
err = middleware(func(*httpx.Context) error { called = true; return nil })(c)
|
||||||
|
if header == "Bearer "+token {
|
||||||
|
if err != nil || !called {
|
||||||
|
t.Fatal("valid token rejected")
|
||||||
|
}
|
||||||
|
} else if err == nil || called || apierrors.From(err).Status != 401 {
|
||||||
|
t.Fatal("unauthenticated request reached service")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
10
examples/userService/internal/middleware/registryGen.go
Normal file
10
examples/userService/internal/middleware/registryGen.go
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
// Code generated by go-omni api. DO NOT EDIT.
|
||||||
|
package middleware
|
||||||
|
|
||||||
|
import "git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
|
||||||
|
func Defaults() map[string]httpx.Middleware {
|
||||||
|
return map[string]httpx.Middleware{
|
||||||
|
"Auth": Auth,
|
||||||
|
}
|
||||||
|
}
|
||||||
11
examples/userService/internal/model/errors.go
Normal file
11
examples/userService/internal/model/errors.go
Normal file
@@ -0,0 +1,11 @@
|
|||||||
|
package model
|
||||||
|
|
||||||
|
import "errors"
|
||||||
|
|
||||||
|
// Domain errors are translated into HTTP responses only by the handler layer.
|
||||||
|
var (
|
||||||
|
ErrUserNotFound = errors.New("user not found")
|
||||||
|
ErrEmailTaken = errors.New("email already in use")
|
||||||
|
ErrInvalidUser = errors.New("invalid user")
|
||||||
|
ErrInvalidPage = errors.New("invalid pagination")
|
||||||
|
)
|
||||||
25
examples/userService/internal/model/user.go
Normal file
25
examples/userService/internal/model/user.go
Normal file
@@ -0,0 +1,25 @@
|
|||||||
|
package model
|
||||||
|
|
||||||
|
import "time"
|
||||||
|
|
||||||
|
// User is the business entity; it has no dependency on HTTP or a database driver.
|
||||||
|
type User struct {
|
||||||
|
ID int64 `json:"id"`
|
||||||
|
Name string `json:"name"`
|
||||||
|
Email string `json:"email"`
|
||||||
|
CreatedAt time.Time `json:"createdAt"`
|
||||||
|
UpdatedAt time.Time `json:"updatedAt"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// UserInput is shared by creation and full replacement (PUT).
|
||||||
|
type UserInput struct {
|
||||||
|
Name string
|
||||||
|
Email string
|
||||||
|
}
|
||||||
|
|
||||||
|
type UserPage struct {
|
||||||
|
Items []User `json:"items"`
|
||||||
|
Total int `json:"total"`
|
||||||
|
Page int `json:"page"`
|
||||||
|
PageSize int `json:"pageSize"`
|
||||||
|
}
|
||||||
143
examples/userService/internal/repository/user/mysql.go
Normal file
143
examples/userService/internal/repository/user/mysql.go
Normal file
@@ -0,0 +1,143 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
"github.com/go-sql-driver/mysql"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SQLExecutor is implemented by both *sql.DB and *sql.Tx. A repository created
|
||||||
|
// with a transaction keeps all of its queries on that same transaction.
|
||||||
|
type SQLExecutor interface {
|
||||||
|
ExecContext(context.Context, string, ...any) (sql.Result, error)
|
||||||
|
QueryContext(context.Context, string, ...any) (*sql.Rows, error)
|
||||||
|
QueryRowContext(context.Context, string, ...any) *sql.Row
|
||||||
|
}
|
||||||
|
|
||||||
|
// MySQLUsers is the persistent implementation of UserRepository.
|
||||||
|
// The schema is documented in schema.sql; migrations stay outside the framework.
|
||||||
|
type MySQLUsers struct {
|
||||||
|
db SQLExecutor
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewMySQLUsers(db SQLExecutor) *MySQLUsers {
|
||||||
|
return &MySQLUsers{db: db}
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckSchema validates that the application's migration has been applied.
|
||||||
|
// Runtime startup never creates or alters tables.
|
||||||
|
func (r *MySQLUsers) CheckSchema(ctx context.Context) error {
|
||||||
|
rows, err := r.db.QueryContext(ctx, `SELECT id, name, email, created_at, updated_at FROM users LIMIT 0`)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return rows.Close()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Create(ctx context.Context, input model.UserInput) (model.User, error) {
|
||||||
|
result, err := r.db.ExecContext(ctx, `
|
||||||
|
INSERT INTO users (name, email, created_at, updated_at)
|
||||||
|
VALUES (?, ?, UTC_TIMESTAMP(6), UTC_TIMESTAMP(6))`, input.Name, input.Email)
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, mapMySQLError(err)
|
||||||
|
}
|
||||||
|
id, err := result.LastInsertId()
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, err
|
||||||
|
}
|
||||||
|
return r.Get(ctx, id)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Get(ctx context.Context, id int64) (model.User, error) {
|
||||||
|
var user model.User
|
||||||
|
err := r.db.QueryRowContext(ctx, `
|
||||||
|
SELECT id, name, email, created_at, updated_at
|
||||||
|
FROM users WHERE id = ?`, id).Scan(
|
||||||
|
&user.ID, &user.Name, &user.Email, &user.CreatedAt, &user.UpdatedAt,
|
||||||
|
)
|
||||||
|
if errors.Is(err, sql.ErrNoRows) {
|
||||||
|
return model.User{}, model.ErrUserNotFound
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, err
|
||||||
|
}
|
||||||
|
return user, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) List(ctx context.Context, page, pageSize int) (model.UserPage, error) {
|
||||||
|
if page < 1 || pageSize < 1 || pageSize > 100 {
|
||||||
|
return model.UserPage{}, model.ErrInvalidPage
|
||||||
|
}
|
||||||
|
var total int
|
||||||
|
if err := r.db.QueryRowContext(ctx, `SELECT COUNT(*) FROM users`).Scan(&total); err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
result := model.UserPage{Items: []model.User{}, Total: total, Page: page, PageSize: pageSize}
|
||||||
|
if total == 0 || page-1 > (total-1)/pageSize {
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
offset := (page - 1) * pageSize
|
||||||
|
rows, err := r.db.QueryContext(ctx, `
|
||||||
|
SELECT id, name, email, created_at, updated_at
|
||||||
|
FROM users ORDER BY id LIMIT ? OFFSET ?`, pageSize, offset)
|
||||||
|
if err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
for rows.Next() {
|
||||||
|
var user model.User
|
||||||
|
if err := rows.Scan(&user.ID, &user.Name, &user.Email, &user.CreatedAt, &user.UpdatedAt); err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
result.Items = append(result.Items, user)
|
||||||
|
}
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return model.UserPage{}, err
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Update(ctx context.Context, id int64, input model.UserInput) (model.User, error) {
|
||||||
|
result, err := r.db.ExecContext(ctx, `
|
||||||
|
UPDATE users SET name = ?, email = ?, updated_at = UTC_TIMESTAMP(6)
|
||||||
|
WHERE id = ?`, input.Name, input.Email, id)
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, mapMySQLError(err)
|
||||||
|
}
|
||||||
|
changed, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return model.User{}, err
|
||||||
|
}
|
||||||
|
if changed == 0 {
|
||||||
|
// MySQL may report zero affected rows when the replacement is identical;
|
||||||
|
// distinguish that from an ID that does not exist.
|
||||||
|
return r.Get(ctx, id)
|
||||||
|
}
|
||||||
|
return r.Get(ctx, id)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r *MySQLUsers) Delete(ctx context.Context, id int64) error {
|
||||||
|
result, err := r.db.ExecContext(ctx, `DELETE FROM users WHERE id = ?`, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
deleted, err := result.RowsAffected()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if deleted == 0 {
|
||||||
|
return model.ErrUserNotFound
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func mapMySQLError(err error) error {
|
||||||
|
var mysqlErr *mysql.MySQLError
|
||||||
|
if errors.As(err, &mysqlErr) && mysqlErr.Number == 1062 {
|
||||||
|
return model.ErrEmailTaken
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
16
examples/userService/internal/repository/user/repository.go
Normal file
16
examples/userService/internal/repository/user/repository.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
)
|
||||||
|
|
||||||
|
// UserRepository describes the storage operations the business needs.
|
||||||
|
// Implementations must atomically enforce unique, normalized email addresses.
|
||||||
|
type UserRepository interface {
|
||||||
|
Create(context.Context, model.UserInput) (model.User, error)
|
||||||
|
Get(context.Context, int64) (model.User, error)
|
||||||
|
List(context.Context, int, int) (model.UserPage, error)
|
||||||
|
Update(context.Context, int64, model.UserInput) (model.User, error)
|
||||||
|
Delete(context.Context, int64) error
|
||||||
|
}
|
||||||
41
examples/userService/internal/repository/user/store.go
Normal file
41
examples/userService/internal/repository/user/store.go
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type UnitOfWork interface {
|
||||||
|
UserRepository
|
||||||
|
WithTx(context.Context, func(UserRepository, func(string, int64, string) error) error) error
|
||||||
|
}
|
||||||
|
type Store struct {
|
||||||
|
*MySQLUsers
|
||||||
|
db *database.DB
|
||||||
|
}
|
||||||
|
|
||||||
|
func NewStore(db *database.DB) *Store { return &Store{MySQLUsers: NewMySQLUsers(db.DB), db: db} }
|
||||||
|
func (s *Store) WithTx(ctx context.Context, work func(UserRepository, func(string, int64, string) error) error) error {
|
||||||
|
return s.db.InTx(ctx, nil, func(tx *sql.Tx) error {
|
||||||
|
audit := func(operation string, id int64, requestID string) error {
|
||||||
|
_, err := tx.ExecContext(ctx, "INSERT INTO user_audit (user_id,operation,request_id,created_at) VALUES (?,?,?,?)", id, operation, requestID, time.Now().UTC())
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return work(NewMySQLUsers(tx), audit)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
// CheckSchema validates columns; it never mutates database schema.
|
||||||
|
func (s *Store) CheckSchema(ctx context.Context) error {
|
||||||
|
if err := s.MySQLUsers.CheckSchema(ctx); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
rows, err := s.db.QueryContext(ctx, "SELECT id,user_id,operation,request_id,created_at FROM user_audit LIMIT 0")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return errors.Join(rows.Err(), rows.Close())
|
||||||
|
}
|
||||||
@@ -0,0 +1,84 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
)
|
||||||
|
|
||||||
|
// This suite runs only against the disposable database created by verifyAPIWorkflow.
|
||||||
|
func TestTransactions(t *testing.T) {
|
||||||
|
if os.Getenv("USER_SERVICE_TEST_DATABASE") != "1" {
|
||||||
|
t.Skip("isolated MySQL required")
|
||||||
|
}
|
||||||
|
cfg, err := database.ConfigFromEnv("")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
db, err := database.Open(context.Background(), cfg)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer db.Close()
|
||||||
|
store := NewStore(db)
|
||||||
|
for _, kind := range []string{"error", "panic", "cancel", "commit"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
email := kind + "@transaction.example"
|
||||||
|
var recovered any
|
||||||
|
func() {
|
||||||
|
defer func() { recovered = recover() }()
|
||||||
|
err = store.WithTx(ctx, func(r UserRepository, audit func(string, int64, string) error) error {
|
||||||
|
u, e := r.Create(ctx, model.UserInput{Name: "Transaction", Email: email})
|
||||||
|
if e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
if e = audit("created", u.ID, "transactionCheck"); e != nil {
|
||||||
|
return e
|
||||||
|
}
|
||||||
|
switch kind {
|
||||||
|
case "error":
|
||||||
|
return errors.New("rollback sentinel")
|
||||||
|
case "panic":
|
||||||
|
panic("rollback sentinel")
|
||||||
|
case "cancel":
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
}()
|
||||||
|
if kind == "panic" && recovered == nil {
|
||||||
|
t.Fatal("panic lost")
|
||||||
|
}
|
||||||
|
if kind == "commit" && err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if kind != "commit" && kind != "panic" && err == nil {
|
||||||
|
t.Fatal("failed transaction committed")
|
||||||
|
}
|
||||||
|
var count int
|
||||||
|
if e := db.QueryRow("SELECT COUNT(*) FROM users WHERE email=?", email).Scan(&count); e != nil {
|
||||||
|
t.Fatal(e)
|
||||||
|
}
|
||||||
|
want := 0
|
||||||
|
if kind == "commit" {
|
||||||
|
want = 1
|
||||||
|
}
|
||||||
|
if count != want {
|
||||||
|
t.Fatalf("rows=%d want=%d", count, want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
var audits int
|
||||||
|
if err = db.QueryRow("SELECT COUNT(*) FROM user_audit WHERE request_id='transactionCheck'").Scan(&audits); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if audits != 1 {
|
||||||
|
t.Fatalf("audit transaction mismatch: %d", audits)
|
||||||
|
}
|
||||||
|
}
|
||||||
16
examples/userService/internal/service/user/createUser.go
Normal file
16
examples/userService/internal/service/user/createUser.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) CreateUser(ctx context.Context, req *types.CreateUserReq) (*types.UserResp, error) {
|
||||||
|
input, err := normalize(req.Name, req.Email)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
return s.mutate(ctx, "created", func(r repo.UserRepository) (model.User, error) { return r.Create(ctx, input) })
|
||||||
|
}
|
||||||
16
examples/userService/internal/service/user/deleteUser.go
Normal file
16
examples/userService/internal/service/user/deleteUser.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) DeleteUser(ctx context.Context, req *types.UserIDReq) error {
|
||||||
|
_, err := s.mutate(ctx, "deleted", func(r repo.UserRepository) (model.User, error) {
|
||||||
|
err := r.Delete(ctx, req.ID)
|
||||||
|
return model.User{ID: req.ID}, err
|
||||||
|
})
|
||||||
|
return err
|
||||||
|
}
|
||||||
40
examples/userService/internal/service/user/exportUser.go
Normal file
40
examples/userService/internal/service/user/exportUser.go
Normal file
@@ -0,0 +1,40 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) ExportUser(ctx context.Context, req *types.UserIDReq) (*types.ExportResp, error) {
|
||||||
|
u, err := s.GetUser(ctx, req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
payload, err := json.Marshal(u)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
var nonce [16]byte
|
||||||
|
if _, err = rand.Read(nonce[:]); err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
key := fmt.Sprintf("exports/users/%d/%s.txt", u.ID, hex.EncodeToString(nonce[:]))
|
||||||
|
if _, err = s.deps.Uploads.Put(ctx, key, strings.NewReader(string(payload)), int64(len(payload)), "text/plain"); err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
url, err := s.deps.Downloads.PresignGet(ctx, key, 5*time.Minute)
|
||||||
|
if err != nil {
|
||||||
|
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_ = s.deps.Files.Delete(cleanup, key)
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
s.deps.Logger.InfoContext(ctx, "user.exported", "userId", u.ID)
|
||||||
|
return &types.ExportResp{DownloadURL: url.String()}, nil
|
||||||
|
}
|
||||||
14
examples/userService/internal/service/user/getUser.go
Normal file
14
examples/userService/internal/service/user/getUser.go
Normal file
@@ -0,0 +1,14 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) GetUser(ctx context.Context, req *types.UserIDReq) (*types.UserResp, error) {
|
||||||
|
u, err := s.deps.Users.Get(ctx, req.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
return response(u), nil
|
||||||
|
}
|
||||||
18
examples/userService/internal/service/user/listUsers.go
Normal file
18
examples/userService/internal/service/user/listUsers.go
Normal file
@@ -0,0 +1,18 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) ListUsers(ctx context.Context, req *types.ListUsersReq) (*types.UserPageResp, error) {
|
||||||
|
page, err := s.deps.Users.List(ctx, req.Page, req.PageSize)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
result := &types.UserPageResp{Items: []types.UserResp{}, Total: page.Total, Page: page.Page, PageSize: page.PageSize}
|
||||||
|
for _, item := range page.Items {
|
||||||
|
result.Items = append(result.Items, *response(item))
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
65
examples/userService/internal/service/user/service.go
Normal file
65
examples/userService/internal/service/user/service.go
Normal file
@@ -0,0 +1,65 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/servicecontext"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"net/mail"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
"unicode/utf8"
|
||||||
|
)
|
||||||
|
|
||||||
|
type Service struct{ deps *servicecontext.Context }
|
||||||
|
|
||||||
|
func New(deps *servicecontext.Context) *Service { return &Service{deps: deps} }
|
||||||
|
func normalize(name, email string) (model.UserInput, error) {
|
||||||
|
input := model.UserInput{Name: strings.TrimSpace(name), Email: strings.ToLower(strings.TrimSpace(email))}
|
||||||
|
address, err := mail.ParseAddress(input.Email)
|
||||||
|
if input.Name == "" || utf8.RuneCountInString(input.Name) > 100 || len(input.Email) > 254 || err != nil || address.Address != input.Email {
|
||||||
|
return model.UserInput{}, model.ErrInvalidUser
|
||||||
|
}
|
||||||
|
return input, nil
|
||||||
|
}
|
||||||
|
func response(u model.User) *types.UserResp {
|
||||||
|
return &types.UserResp{ID: u.ID, Name: u.Name, Email: u.Email, CreatedAt: u.CreatedAt.UTC().Format(time.RFC3339Nano), UpdatedAt: u.UpdatedAt.UTC().Format(time.RFC3339Nano)}
|
||||||
|
}
|
||||||
|
func (s *Service) failure(ctx context.Context, err error) error {
|
||||||
|
switch {
|
||||||
|
case err == nil:
|
||||||
|
return nil
|
||||||
|
case errors.Is(err, model.ErrUserNotFound):
|
||||||
|
return apierrors.NotFound("USER_NOT_FOUND", "user not found")
|
||||||
|
case errors.Is(err, model.ErrEmailTaken):
|
||||||
|
return apierrors.New(409, "EMAIL_TAKEN", "email already in use")
|
||||||
|
case errors.Is(err, model.ErrInvalidUser):
|
||||||
|
return apierrors.BadRequest("INVALID_USER", "invalid user")
|
||||||
|
case errors.Is(err, model.ErrInvalidPage):
|
||||||
|
return apierrors.BadRequest("INVALID_PAGE", "invalid pagination")
|
||||||
|
default:
|
||||||
|
// Do not log SQL/DSN or user input; requestId correlates the failed operation.
|
||||||
|
s.deps.Logger.ErrorContext(ctx, "user.operationFailed", "category", "persistence")
|
||||||
|
return apierrors.Internal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (s *Service) mutate(ctx context.Context, operation string, fn func(repo.UserRepository) (model.User, error)) (*types.UserResp, error) {
|
||||||
|
var u model.User
|
||||||
|
err := s.deps.Users.WithTx(ctx, func(r repo.UserRepository, audit func(string, int64, string) error) error {
|
||||||
|
var err error
|
||||||
|
u, err = fn(r)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return audit(operation, u.ID, logx.RequestID(ctx))
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
s.deps.Logger.InfoContext(ctx, "user."+operation, "userId", u.ID)
|
||||||
|
return response(u), nil
|
||||||
|
}
|
||||||
16
examples/userService/internal/service/user/updateUser.go
Normal file
16
examples/userService/internal/service/user/updateUser.go
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
package user
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"example.com/userService/internal/model"
|
||||||
|
repo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/types"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (s *Service) UpdateUser(ctx context.Context, req *types.UpdateUserReq) (*types.UserResp, error) {
|
||||||
|
input, err := normalize(req.Name, req.Email)
|
||||||
|
if err != nil {
|
||||||
|
return nil, s.failure(ctx, err)
|
||||||
|
}
|
||||||
|
return s.mutate(ctx, "updated", func(r repo.UserRepository) (model.User, error) { return r.Update(ctx, req.ID, input) })
|
||||||
|
}
|
||||||
122
examples/userService/internal/servicecontext/context.go
Normal file
122
examples/userService/internal/servicecontext/context.go
Normal file
@@ -0,0 +1,122 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"example.com/userService/internal/middleware"
|
||||||
|
userrepo "example.com/userService/internal/repository/user"
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
"github.com/go-sql-driver/mysql"
|
||||||
|
"net/http"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Context owns infrastructure, while Logger is borrowed from main.
|
||||||
|
// Add business repositories here; HTTP contracts do not define database tables.
|
||||||
|
type Context struct {
|
||||||
|
Users userrepo.UnitOfWork
|
||||||
|
Logger *logx.Logger
|
||||||
|
Health *health.Checker
|
||||||
|
Middlewares map[string]httpx.Middleware
|
||||||
|
DB *database.DB
|
||||||
|
|
||||||
|
Files storage.Store
|
||||||
|
Uploads *storage.Uploader
|
||||||
|
Downloads storage.Presigner
|
||||||
|
UploadLinks storage.UploadPresigner
|
||||||
|
FileHTTP http.Handler
|
||||||
|
storageClose func() error
|
||||||
|
|
||||||
|
closeMu sync.Mutex
|
||||||
|
closed bool
|
||||||
|
closeOnce sync.Once
|
||||||
|
closeErr error
|
||||||
|
}
|
||||||
|
|
||||||
|
// Pass the settings loaded by main. The optional form supports lightweight
|
||||||
|
// tests without infrastructure; selected adapters still require valid settings.
|
||||||
|
func New(ctx context.Context, logger *logx.Logger, configs ...settings.Config) (result *Context, err error) {
|
||||||
|
if ctx == nil {
|
||||||
|
ctx = context.Background()
|
||||||
|
}
|
||||||
|
if err = ctx.Err(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if len(configs) > 1 {
|
||||||
|
return nil, errors.New("expected at most one configuration")
|
||||||
|
}
|
||||||
|
cfg := settings.Default()
|
||||||
|
if len(configs) == 1 {
|
||||||
|
cfg = configs[0]
|
||||||
|
}
|
||||||
|
_ = cfg
|
||||||
|
checker, err := health.New()
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
c := &Context{Logger: logger, Health: checker, Middlewares: middleware.Defaults()}
|
||||||
|
defer func() {
|
||||||
|
if err != nil {
|
||||||
|
err = errors.Join(err, c.Close())
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
|
||||||
|
if err = cfg.Validate(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
dsn, parseErr := mysql.ParseDSN(cfg.Database.DSN)
|
||||||
|
if parseErr != nil || dsn.DBName == "" {
|
||||||
|
return nil, errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")
|
||||||
|
}
|
||||||
|
dsn.ParseTime = true
|
||||||
|
dsn.Loc = time.UTC
|
||||||
|
cfg.Database.DSN = dsn.FormatDSN()
|
||||||
|
c.DB, err = database.Open(ctx, cfg.Database)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err = checker.Add(health.Probe{Name: "database", Timeout: cfg.Database.PingTimeout, Check: c.DB.Health}); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
if err = c.InitStorage(ctx, cfg.Storage); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
c.Users = userrepo.NewStore(c.DB)
|
||||||
|
checkCtx, cancel := context.WithTimeout(ctx, cfg.Database.PingTimeout)
|
||||||
|
defer cancel()
|
||||||
|
if err = userrepo.NewStore(c.DB).CheckSchema(checkCtx); err != nil {
|
||||||
|
return nil, errors.New("user schema is missing or incompatible; apply migrations first")
|
||||||
|
}
|
||||||
|
authHook, authErr := middleware.NewAuthFromEnv()
|
||||||
|
if authErr != nil {
|
||||||
|
return nil, authErr
|
||||||
|
}
|
||||||
|
c.Middlewares["Auth"] = authHook
|
||||||
|
return c, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Close is idempotent, including startup failure and graceful shutdown.
|
||||||
|
func (c *Context) Close() error {
|
||||||
|
if c == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
c.closeMu.Lock()
|
||||||
|
defer c.closeMu.Unlock()
|
||||||
|
c.closed = true
|
||||||
|
c.closeOnce.Do(func() {
|
||||||
|
if c.storageClose != nil {
|
||||||
|
c.closeErr = errors.Join(c.closeErr, c.storageClose())
|
||||||
|
}
|
||||||
|
if c.DB != nil {
|
||||||
|
c.closeErr = errors.Join(c.closeErr, c.DB.Close())
|
||||||
|
}
|
||||||
|
})
|
||||||
|
return c.closeErr
|
||||||
|
}
|
||||||
81
examples/userService/internal/servicecontext/context_test.go
Normal file
81
examples/userService/internal/servicecontext/context_test.go
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestStartupFailureAndIdempotentCleanup(t *testing.T) {
|
||||||
|
if os.Getenv("USER_SERVICE_TEST_DATABASE") != "1" {
|
||||||
|
t.Skip("isolated MySQL required")
|
||||||
|
}
|
||||||
|
t.Setenv("USER_SERVICE_API_TOKEN", "test-012345678901234567890123456789")
|
||||||
|
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY", "test-012345678901234567890123456789")
|
||||||
|
cfg, err := settings.Load("../../etc/config.yaml")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
observer, err := database.Open(context.Background(), cfg.Database)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer observer.Close()
|
||||||
|
count := func() int {
|
||||||
|
var n int
|
||||||
|
if err := observer.QueryRow("SELECT COUNT(*) FROM information_schema.processlist WHERE USER=SUBSTRING_INDEX(CURRENT_USER(),'@',1) AND DB=DATABASE()").Scan(&n); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
baseline := count()
|
||||||
|
cfg.Server.Log.File = ""
|
||||||
|
cfg.Server.Log.Writer = io.Discard
|
||||||
|
cfg.Server.Log.Rotation = logx.RotationConfig{}
|
||||||
|
logger, err := logx.New(cfg.Server.Log)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer logger.Close()
|
||||||
|
root := t.TempDir()
|
||||||
|
file := filepath.Join(root, "notADirectory")
|
||||||
|
if err = os.WriteFile(file, []byte("x"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cfg.Storage.Root = file
|
||||||
|
if c, err := New(context.Background(), logger, cfg); err == nil {
|
||||||
|
c.Close()
|
||||||
|
t.Fatal("invalid storage root accepted")
|
||||||
|
}
|
||||||
|
deadline := time.Now().Add(3 * time.Second)
|
||||||
|
for count() != baseline {
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
t.Fatal("failed startup leaked MySQL connection")
|
||||||
|
}
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
cfg.Storage.Root = filepath.Join(root, "files")
|
||||||
|
c, err := New(context.Background(), logger, cfg)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = c.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = c.Close(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err = c.DB.Ping(); err == nil {
|
||||||
|
t.Fatal("closed application retained database")
|
||||||
|
}
|
||||||
|
if err = c.InitStorage(context.Background(), cfg.Storage); err == nil {
|
||||||
|
t.Fatal("closed dependencies reopened storage")
|
||||||
|
}
|
||||||
|
}
|
||||||
49
examples/userService/internal/servicecontext/storage.go
Normal file
49
examples/userService/internal/servicecontext/storage.go
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
package servicecontext
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||||
|
|
||||||
|
"example.com/userService/internal/settings"
|
||||||
|
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||||
|
)
|
||||||
|
|
||||||
|
// InitStorage is called once, before serving requests. The composition root
|
||||||
|
// owns provider clients; business services receive only capability interfaces.
|
||||||
|
func (c *Context) InitStorage(ctx context.Context, cfg settings.StorageConfig) error {
|
||||||
|
if c == nil || ctx == nil {
|
||||||
|
return errors.New("storage initialization requires context")
|
||||||
|
}
|
||||||
|
c.closeMu.Lock()
|
||||||
|
defer c.closeMu.Unlock()
|
||||||
|
if c.closed {
|
||||||
|
return errors.New("service dependencies are closed")
|
||||||
|
}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if c.Files != nil {
|
||||||
|
return errors.New("storage already initialized")
|
||||||
|
}
|
||||||
|
if err := cfg.Validate(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
files, err := local.New(cfg.Root, cfg.MaxBytes)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
signer, err := storage.NewUploadSigner(files, cfg.PublicURL, []byte(cfg.SigningKey), cfg.Policy())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
uploader, err := storage.NewUploader(files, cfg.Policy())
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
c.Files, c.Uploads, c.Downloads, c.UploadLinks, c.FileHTTP = files, uploader, signer, signer, signer
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user