Compare commits
10 Commits
59bf04a461
...
v0.7.0-pre
| Author | SHA1 | Date | |
|---|---|---|---|
| 0ce3efb6fc | |||
| 2c74dc0859 | |||
| 04a1f7999d | |||
| 208a9c3fbd | |||
| f91034477b | |||
| b9259e628e | |||
| fe66221a9c | |||
| 467c890363 | |||
| be620d2243 | |||
| 8e18528fc2 |
151
ADVANCED.md
Normal file
151
ADVANCED.md
Normal file
@@ -0,0 +1,151 @@
|
||||
# 扩展业务模块
|
||||
|
||||
本批提供八类扩展,以及分页、审计和显式数据库迁移。模块按需导入;`omni.New` 与业务 Handler 不绑定存储、队列或网络实现。JSON、配置和新示例使用 lowerCamelCase;Go 导出标识符和 `_test.go` 保持语言约定。
|
||||
|
||||
按本次范围决定,**不集成 Orbit,不引入 Orbit SDK,也不实现工作流编排**。通用消息队列与 Orbit 的持久流程、执行授权、结果订阅和人工恢复不是等价能力。业务项目以后需要 Orbit,可独立依赖它的 SDK;当前没有承诺只换队列适配器就能迁移其全部语义。
|
||||
|
||||
## 能力与边界
|
||||
|
||||
| 模块 | 本批实现 | 明确边界 |
|
||||
| --- | --- | --- |
|
||||
| `openapi` | 路由与 OpenAPI 3.0.3 同时注册、DTO schema、响应信封 | 显式声明请求/响应;不自动推断任意 Handler 行为,不附带 Swagger UI |
|
||||
| `auth` | RS256/EdDSA JWT 验证与公钥轮换、可撤销 Session、Policy 接口 | 登录验证和业务授权规则由接入方提供;不自动获取 token 指定的密钥 |
|
||||
| `telemetry` | HTTP 入站/出站、Redis、SQL spans,HTTP 指标和 traceId 日志关联 | 注入 OTel exporter/reader 后才外发;SQL Query span 不覆盖后续遍历;不接管全局 provider |
|
||||
| `config` / `logx` | JSON/YAML 文件、环境变量覆盖、动态日志级别、轮转/脱敏/堆栈/慢请求 | 热更新只开放日志级别;连接池、监听、认证等静态变更要求重启 |
|
||||
| `resilience` | 失败阈值、熔断冷却、单探针恢复、上游并发隔离 | 一个实例对应一个依赖;默认内存状态,不做跨进程全局熔断 |
|
||||
| `middleware/redis` | 跨副本限流、幂等 claim/回放、容量限制 | 专用 Redis namespace;TTL 窗口内保护不等于永久业务去重 |
|
||||
| `queue` / `queue/redis` | 持久消息接口、Streams 接管、有限重试、失败消息管理 | 至少一次交付,处理器必须幂等;持久性取决于 Redis 部署 |
|
||||
| `storage` | Local、S3/R2、OSS、临时下载、受限直传、云端分片 | 云平台专属功能单独扩展;本批没有云账号实网验收 |
|
||||
| `database/migrate` | MySQL 显式部署迁移、摘要预览、失败 dirty 阻断 | 不随应用启动自动改表;无自动 down/force;已应用文件摘要需部署方留存校对 |
|
||||
| `audit` | 可注入 Sink、字段白名单、JSON 行串行输出、错误返回 | 不是防篡改存储;与业务同事务的审计要实现事务型 Sink |
|
||||
| `pagination` | page/pageSize、最大偏移与页长、统一结果 | offset 分页,不承诺跨页快照,不实现游标分页 |
|
||||
|
||||
日志的配置、脱敏边界、轮转和慢请求记录见 [logx 使用说明](logx/README.md)。框架保留通用 Context/结构化字段扩展,不新增固定业务身份上下文。
|
||||
|
||||
## 对象存储
|
||||
|
||||
业务只持有 `storage.Store`,调用 `Put`、`Get`、`Delete`。构造函数属于启动组合处:
|
||||
|
||||
```go
|
||||
type FileService struct { Files storage.Store }
|
||||
|
||||
// Local:私有目录,流式临时文件 + 原子替换。
|
||||
files, err := local.New("/srv/app/objects", 32<<20)
|
||||
|
||||
// S3:AWS/MinIO 或其他经过契约测试的 S3 兼容服务。
|
||||
files, err := s3.Open(s3.Config{
|
||||
Endpoint: "s3.us-east-1.amazonaws.com", Region: "us-east-1",
|
||||
Bucket: "example-bucket", Secure: true, MaxBytes: 32<<20,
|
||||
Credentials: credentialProvider, Transport: transport,
|
||||
})
|
||||
|
||||
// Cloudflare R2:复用 S3 适配器,签名区域为 auto。
|
||||
cfg, err := s3.R2Config(accountID, bucket, credentialProvider)
|
||||
cfg.Transport = transport
|
||||
files, err := s3.Open(cfg)
|
||||
|
||||
// OSS:单独适配官方 Go SDK v2,保留平台签名与凭据提供器。
|
||||
client := ossSdk.NewClient(ossSdk.LoadDefaultConfig().
|
||||
WithRegion("cn-hangzhou").
|
||||
WithCredentialsProvider(ossCredentials.NewEnvironmentVariableCredentialsProvider()))
|
||||
files, err := oss.New(client, bucket, 32<<20)
|
||||
```
|
||||
|
||||
上方是互斥初始化片段,实际代码需逐次处理错误。import 路径分别为 `go-omni/storage/local`、`go-omni/storage/s3`、`go-omni/storage/oss`(完整 module 前缀 `git.yuntuo.hk/k2safe/`);`ossSdk` 与 `ossCredentials` 来自 `github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss` 及其 `credentials` 子包。S3 凭据来自 `github.com/minio/minio-go/v7/pkg/credentials`。
|
||||
|
||||
统一约定:key 是对象标识,不是磁盘路径;拒绝绝对路径、`..`、空路径段和反斜杠。Put 必须传已知长度,并检查实际读取长度。默认单对象 32 MiB,流式读写;OSS 简单 Put 上限 5 GiB,更大使用独立分片能力。Get 返回的 `Body` 由调用方关闭,读取过程也应有 context 截止。Delete 缺失对象成功;Get 缺失对象返回 `storage.ErrNotFound`。底层客户端/传输连接池由应用管理,不会在单次操作后关闭。
|
||||
|
||||
业务需要临时链接时显式依赖额外接口 `storage.Presigner`。S3/OSS 实现 `PresignGet`;Local 用 `storage.NewSigner(files, downloadBaseURL, secret)` 提供 HTTP 下载 Handler。链接 TTL 为 1 秒到 1 小时,私钥/临时凭据可能让云链接提前失效。签发链接前必须完成业务授权;对象 key 及 bucket 本身不是租户权限。签名链接不要写日志。受限直传和云端分片现已提供可选接口,见 [上传与续传说明](storage/README.md)。生命周期、ACL、CDN 自定义域名和版本管理仍由平台/部署管理。
|
||||
|
||||
R2 实现 S3 的子集,配置入口使用标准账户 endpoint;法域专属 endpoint 可显式配置。兼容性依据:[R2 S3 API](https://developers.cloudflare.com/r2/api/s3/api/)、[R2 签名链接](https://developers.cloudflare.com/r2/api/s3/presigned-urls/)。OSS 使用其[官方 Go SDK v2](https://www.alibabacloud.com/help/en/oss/developer-reference/manual-for-go-sdk-v2/),没有假定全部 S3 行为与 OSS 等价。
|
||||
|
||||
验证区分:Local 契约、OSS SDK HTTP fixture、S3 SDK fixture 和 R2 签名参数属于本地测试;Linux runner 再跑真实 MinIO 上传/读取/删除与签名拒绝。**MinIO 通过不能代替 OSS/R2 实网验证。**迁移存储平台还需要复制已有对象和调整部署权限,不只是切配置。
|
||||
|
||||
## 可靠队列
|
||||
|
||||
```go
|
||||
backend, err := queueRedis.New(redisClient, queueRedis.Config{
|
||||
Namespace: "serviceA:jobs:v1", MaxRecords: 1000,
|
||||
MaxPayloadBytes: 64 << 10, DedupTTL: 24*time.Hour,
|
||||
})
|
||||
consumer, err := queueRedis.NewConsumer(backend, queueRedis.ConsumerConfig{
|
||||
Workers: 4, MaxAttempts: 3,
|
||||
TaskTimeout: 30*time.Second, VisibilityTimeout: time.Minute,
|
||||
})
|
||||
|
||||
// ID 与内容在业务端持久化;超时后复用原 ID 和原内容。
|
||||
receipt, err := backend.Publish(ctx, queue.Message{
|
||||
ID: operationID, Name: "generateReport", Payload: payload,
|
||||
})
|
||||
|
||||
// 在服务生命周期中运行;返回错误须让监控/进程管理感知。
|
||||
err = consumer.Run(serviceContext, func(ctx context.Context, item queue.Delivery) error {
|
||||
return generateReportIdempotently(ctx, item.ID, item.Payload)
|
||||
})
|
||||
```
|
||||
|
||||
业务依赖 `queue.Publisher` / `queue.Consumer`,Redis 类型留在组合处。receipt 表示受理,不代表业务成功。普通 error/panic 按配置重试,`queue.Reject(code)` 保留为失败消息;超过次数同样保留。`DeadLetters(after, limit)` 是有界分页,`DiscardDead` 仅在人工核对后删除匹配的失败消息,并继续保留一段提交去重窗口。错误正文不写入失败记录。需要再次处理时由业务决定新操作身份及去重策略,框架不自动无限重投。
|
||||
|
||||
一个 namespace 使用一个受管 consumer group;多个 consumer 可竞争消费同一队列。配置一致性检查拒绝不同的容量/去重策略和执行超时/重试策略混用。确认检查当前 pending owner,旧执行不能确认新执行;但不能阻止旧处理器已发出的外部副作用,因此仍必须业务幂等。处理器需响应 context;Go 无法强杀不配合的函数,超过 shutdownTimeout 返回错误时,最多 Workers 个处理器可能尚未退出。
|
||||
|
||||
容量包含待处理、失败及未过期的已完成身份,满额拒绝新 ID,保留原 ID 重报回执。完成/人工丢弃后的身份按 DedupTTL 过期;未完成/失败消息不自动清除,不用 MAXLEN 静默裁掉未消费消息。Redis 需启用合适的 AOF/复制策略与 noeviction;本批验证的是投递/接管协议,未证明 Redis 主从故障切换后零丢失。异常进程留下的无 pending consumer 元数据可在运维核实后清理,不能用 DELCONSUMER 丢掉尚未接管的 pending。
|
||||
|
||||
`task.Runner` 仍适用于本进程短任务和定时轮询,重启会丢失;需要持久交付时使用 `queue`。这里没有分布式 Cron、业务工作流、事务 Outbox 或 exactly-once 承诺。
|
||||
|
||||
## 认证与授权
|
||||
|
||||
`auth.NewJWT` 要求固定 issuer/audience 和可信 kid→公钥列表;只开放 RS256 / EdDSA,必须有 exp/sub,检查有效期、nbf 与 iat。`ReplaceKeys` 原子替换列表,轮换期可同时保留新旧 kid。JWT 签发、私钥托管、登录口令验证留给身份服务。
|
||||
|
||||
`auth.NewSessions(cacheStore, "sessions:", time.Hour)` 签发随机 opaque bearer token;存储仅使用 token 的 SHA-256 键,可 `Revoke`。Redis 缓存可共享 Session,内存缓存只在单实例生效且可能因容量淘汰提前登出。不要把 Session token 当普通缓存值写日志。
|
||||
|
||||
`RequireAuth(authenticator)` 在前,`RequirePermission(policy, action, resource)` 在后。Policy 接收已验证 Principal,业务自行校验租户、角色、资源归属。示例 `examples/platform` 验证了无身份 401、角色不足 403 和授权写入。
|
||||
|
||||
## 配置与日志热更新
|
||||
|
||||
`config.LoadFile(path, prefix)` 按默认值→文件→非空环境变量加载并验证。两种文件格式都使用 JSON tag 的 lowerCamelCase,duration 可以写 `"5s"`;文件上限 1 MiB,拒绝未知字段、多文档和非法值。自定义 prefix 要包含期望的尾部下划线,例如 `BILLING_`。空环境变量沿用文件值,不清空已配置字段。
|
||||
|
||||
```yaml
|
||||
address: '127.0.0.1:8080'
|
||||
transport: native
|
||||
shutdownTimeout: 10s
|
||||
log:
|
||||
level: info
|
||||
format: json
|
||||
```
|
||||
|
||||
先按配置创建 `logx.New(cfg.Log)`,再 `config.NewReloader(path, prefix, logger)`。显式调用 `Reload()` 或运行 `Watch(ctx, time.Second, onError)`;后者阻塞,生命周期由应用负责。只有 `log.level` 允许变化,校验失败或静态字段变化保留原快照与日志级别。子 logger 共享动态级别。替换文件时建议使用临时文件 + rename。
|
||||
|
||||
## 熔断与观测
|
||||
|
||||
为每个上游创建 `resilience.New(Config{MaxConcurrent: 16, FailureThreshold: 5})`;通用调用用 `Do`,HTTP 用 `resilience.Transport{Base: transport, Breaker: breaker}`。HTTP 槽位覆盖响应体读取,必须关闭 body;网络错误/5xx 计失败,冷却后只放行一个探针。不要对效果未知的写操作自行换身份重试。
|
||||
|
||||
`telemetry.New` 接受 OTel `sdktrace.SpanExporter` 和 `sdkmetric.Reader`;可注入官方 OTLP exporter、PeriodicReader 或测试用 exporter,基础包不强制选择 collector 协议。`app.Use(runtime.Middleware())` 采集入站,`telemetry.Transport` 采集出站并传播 W3C trace context;Redis 使用 `client.AddHook(telemetry.RedisHook{Runtime: runtime})`,SQL 使用 `telemetry.SQL{DB: db, Runtime: runtime}`。业务操作/事务可用 `runtime.Do(ctx, "fixedOperationName", fn)` 包裹。
|
||||
|
||||
创建者负责在停止请求/Worker 后调用 `runtime.Shutdown(ctx)` 刷新并关闭 exporter/reader。指标按注册路由模板和固定方法分类;自动采集不记录 URL query、SQL 正文、Redis key、token 或业务载荷。业务自定义 span 名称同样应固定,避免用用户 ID 或对象 key。
|
||||
|
||||
## 分布式限流与幂等
|
||||
|
||||
`middleware/redis.NewRateLimiter(client, namespace, cfg)` 跨副本使用 Redis 时间与原子令牌桶;满额/存储失败返回 503,额度不足返回 429 和 Retry-After。`NewIdempotencyStore` 注入 `middleware.IdempotencyConfig.Store`;Scope 必须来自认证后的租户/用户,先认证再幂等。不同请求内容复用同一身份返回 409,已完成请求回放原响应和 requestId。
|
||||
|
||||
claim 在开始业务前持久保存,完成按 owner 原子检查。进程崩溃留下的 running claim 不自动按租约过期;返回 409 等待核查,`Resolve` 仅由运维核对后显式调用。已保存的完成/失败记录按配置 TTL 清理,TTL 过后不再提供业务去重保证。失败结果、含 Set-Cookie/no-store 的响应不用于成功回放。共享 Redis 需要 noeviction 和独立 namespace,数据丢失时不存在框架层面的重复执行保证。
|
||||
|
||||
## 迁移、审计与分页
|
||||
|
||||
迁移命名:`0001_createUsers.up.sql`,可附 `.down.sql`,但第一版 CLI 只执行 up。`go-omni migrate --dir migrations` 默认打印版本/方向/SHA-256 清单,不连接数据库;部署步骤显式传 `--apply` 才读取 `GO_OMNI_DB_DSN` 执行。迁移使用独立连接与数据库锁;MySQL DDL 可能部分提交,失败保留 dirty,禁止自动 force 或回滚。当前基于 golang-migrate,数据库版本表不保存每个历史文件校验和,部署系统应保留并校对预览清单。取消在当前有界 SQL 操作结束后停止,重试需新建 Runner。
|
||||
|
||||
审计:`audit.New(sink, "approvedField")` 建白名单,再 `Record(ctx, Event{ActorID, Action, Resource, Outcome, Fields})`;补入服务端时间、requestId/traceId,只保留明确允许的属性,Sink 失败原样返回。JSONSink 不保证与业务提交原子一致,需要这种语义时由业务在同一数据库事务内提供 Sink/Outbox。
|
||||
|
||||
分页:`pagination.Parse(request.URL.Query(), pagination.Config{})` 默认 page=1/pageSize=20,上限 100,最大 offset=1,000,000;拒绝重复参数、溢出和越界。`pagination.NewResult(request, items, total)` 输出 items/page/pageSize/total/totalPages,空列表为 `[]`。查询与租户过滤仍由 Repository 负责。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --advanced --goproxy https://goproxy.cn,direct --output performance/advancedLinux.json
|
||||
```
|
||||
|
||||
Linux runner 顺序运行编译、Redis、MinIO、MySQL 四个容器,各限 2 CPU、2 GiB、256 pids;数据阶段无外网、无发布端口,不连接现有数据库。原始结果和源码摘要以生成的 JSON/TXT 为准,未提供云凭据的 OSS/R2 不标为实网通过。
|
||||
|
||||
本批已通过上述 Linux 验证,源代码提交、98 个文件摘要、服务版本和边界见 [验收记录](performance/advancedAcceptance.md)。
|
||||
169
API.md
Normal file
169
API.md
Normal file
@@ -0,0 +1,169 @@
|
||||
# API 契约与项目生成
|
||||
|
||||
本阶段目标:从 `.api` 定义生成可运行的 Go 项目;定义、绑定、校验与 OpenAPI 保持一致;重复生成不覆盖业务实现;业务代码继续独立于网络层。实现入口是 `cmd/go-omni/apigen`,运行时契约编译器是 `contract`。
|
||||
|
||||
## 与 go-zero 的对应关系
|
||||
|
||||
调研参考了 go-zero 官方的 [API DSL](https://go-zero.dev/reference/api-dsl/)、[参数规则](https://go-zero.dev/reference/api-dsl/parameter/)、[goctl API](https://go-zero.dev/reference/cli-guide/api/) 和 [项目结构](https://go-zero.dev/concepts/project-structure/)。借鉴其声明接口后生成项目的工作方式:统一的类型定义、路由声明、分组、前缀和命名中间件。实现独立编写,不引入 go-zero 或 Gin 运行时依赖。
|
||||
|
||||
| 使用体验 | go-omni 实现 |
|
||||
| --- | --- |
|
||||
| 新建接口项目 | `go-omni api new <directory>` |
|
||||
| 按契约生成、更新 | `go-omni api generate --api user.api --dir .` |
|
||||
| 熟悉的命令别名 | `api go -api user.api -dir .` |
|
||||
| 校验契约 | `api validate --api user.api` |
|
||||
| CI 检查生成产物是否过期 | `api generate --api user.api --dir . --check` |
|
||||
| 业务逻辑目录 | `internal/service/<group>`,沿用 Omni 的 Service 分层 |
|
||||
| 依赖装配 | `internal/servicecontext` |
|
||||
| 接口文档 | 自动生成 `docs/openapi.json`,同时提供 `/openapi.json` |
|
||||
|
||||
这是明确限定的 go-zero 风格方言,**不是完整 goctl 兼容实现**。不支持的声明和校验标签会在生成前报错。
|
||||
|
||||
## 立即试用
|
||||
|
||||
源码开发时,从 go-omni 仓库运行,显式引用本地框架:
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni api new \
|
||||
--module example.com/omniDemo --framework-dir . ../omniDemo
|
||||
|
||||
cd ../omniDemo
|
||||
go mod tidy
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
访问 `http://127.0.0.1:8080/api/greet/Alice` 可得到带 `data.message` 的统一响应。`api new` 的问候示例包含可运行逻辑;自定义契约新生成的 Service 会明确返回 501,等待业务实现。
|
||||
|
||||
用现成的多分组定义生成项目,在 go-omni 仓库执行:
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni new --api examples/contracts/user.api \
|
||||
--module example.com/userService --framework-dir . \
|
||||
--log-output file --log-format json ../userService
|
||||
```
|
||||
|
||||
`api generate` 在新目录中也会生成完整项目。默认只监听 localhost;`GO_OMNI_HTTP_ADDR` 修改地址,`GO_OMNI_TRANSPORT=native` 切换网络层。配置顺序为默认值 → `etc/config.yaml` → 环境变量。文件日志支持现有轮转配置。远程试用使用与 CLI 一致的固定版本,见 [发布说明](RELEASE.md)。
|
||||
|
||||
API 项目现在支持组合数据库、存储和日志:
|
||||
|
||||
```sh
|
||||
go-omni new --api user.api --module example.com/userService \
|
||||
--database mysql --storage local --log-output file ./userService
|
||||
```
|
||||
|
||||
- `--database none|memory|mysql`:API 默认 `none`,`memory` 是不创建数据库连接的别名;MySQL 生成连接池、配置、事务入口和 readiness 探针。
|
||||
- `--storage none|local|s3|r2|oss`:默认 none,生成统一 Files / Uploads / Downloads / UploadLinks 依赖;本地签名对象端点为 `/storage/object`。
|
||||
- `--log-output stdout|stderr|file` 与 `--log-format json|text`:配置和无配置文件启动使用一致默认值。
|
||||
|
||||
初次选项记录在 `.goOmni/apiManifest.json`,以后直接 `api generate` 会继承。已有项目通过参数切换选项会报错,因为初始化代码属于业务方;可在新目录生成对照后手动迁移。`--force` 不允许覆盖 API 项目。旧式 CRUD 脚手架仍保留。
|
||||
|
||||
MySQL 密钥只从 `GO_OMNI_DB_DSN` 获取,非秘密池配置位于 YAML,环境变量优先。存储使用原有 `GO_OMNI_STORAGE_*`;API 项目不会额外生成公开的签名发放接口,鉴权和对象权限由业务实现。初始化失败会清理已打开资源,退出清理可重复调用。
|
||||
|
||||
HTTP 请求类型不能推导数据库表结构;迁移和 Repository 由业务自己维护。完整案例见 [用户管理示例](examples/userService/README.md),覆盖真实 MySQL、事务审计、鉴权、文件导出和日志。
|
||||
|
||||
## 接口定义
|
||||
|
||||
```go
|
||||
syntax = "v1"
|
||||
|
||||
type UpdateUserReq {
|
||||
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||
Notify bool `form:"notify,default=false"`
|
||||
Name string `json:"name" binding:"required,max=50"`
|
||||
Age *int `json:"age,optional,range=[0:150]"`
|
||||
}
|
||||
|
||||
type UserResp {
|
||||
UserID int64 `json:"userId"`
|
||||
Name string `json:"name"`
|
||||
}
|
||||
|
||||
@server (
|
||||
group: user
|
||||
prefix: /api/v1
|
||||
)
|
||||
service user-api {
|
||||
@doc "Update a user"
|
||||
@handler updateUser
|
||||
put /users/:userId (UpdateUserReq) returns (UserResp)
|
||||
}
|
||||
```
|
||||
|
||||
类型和 Go 字段使用导出命名;JSON、query、path 名称使用 lowerCamelCase;Header 使用 HTTP 标准名称。响应类型只定义 `data`,统一响应的 `code`、`message`、`requestId` 由框架添加。
|
||||
|
||||
支持:
|
||||
|
||||
- 可省略的 `syntax = "v1"`;`info` 中的 title、version 和描述性元数据。
|
||||
- 单独的 `type X { ... }` 与 `type (...)`;基础标量、指针、嵌套命名类型、切片、字符串键 map。嵌套对象只使用 JSON 字段。
|
||||
- 相对 `.api` 导入及 `import (...)`,重复导入去重;拒绝循环和越过入口目录的导入。单文件 1 MiB、总计 8 MiB、最多 64 个文件,限制导入/类型深度及 schema 展开规模。
|
||||
- 同一 service 名称的多个块;`@server` 的 group、prefix、middleware,group 可以写 `admin/user`。
|
||||
- 每个路由必须有 `@handler`;可选 `@doc "..."`;GET/POST/PUT/PATCH/DELETE/HEAD/OPTIONS。
|
||||
- 可省略请求或响应;有响应默认 200,无响应默认 204。Omni 扩展 `@status 201`、`@status 202`;204 不允许响应类型,HEAD 不允许响应体,GET/HEAD 不允许 JSON 请求体。
|
||||
- 每个请求字段只使用一个来源:`json`、`path`、`form`、`header`。本版 `form` 明确表示 URL query,不解析 multipart 或表单 body。query 切片用重复键;header/path 使用标量。
|
||||
- 来源标签中的 `optional` / `omitempty`、标量 `default`、标量 `options=a|b`、数值 `range=[1:100]`(支持开区间和单边界)。
|
||||
- `binding` 的 `required`、`min`、`max`、`len`、`email`。长度按 Unicode 字符或切片元素数;范围边界必须有限且位于 ±2^53 内。
|
||||
|
||||
暂不支持匿名嵌入、别名、递归类型、外部 Go 类型、自定义 JSON 编解码、`[]byte`、泛型、`json:",string"`、任意 validator 标签、go-zero 的 JWT/签名/SSE 声明,以及客户端 SDK 生成。认证请用命名中间件接入。`@doc` 块等未列出的语法也会明确报错。
|
||||
|
||||
## 参数语义
|
||||
|
||||
无 optional/default 的字段按“是否出现”判断必填,`false`、`0`、`""` 是已提供的值。额外声明 `binding:"required"` 才会拒绝标量零值。默认值只在字段缺失时使用,不替换显式的空值。
|
||||
|
||||
业务需要区分“没传”和“传了零值”时使用 `*T` 加 optional。指针允许 JSON null;非指针拒绝 null;`binding:"required"` 拒绝指针 null。缺失与 null 都会得到 nil,本版没有三态 Nullable 类型。响应端的 `optional` 不等于 Go 编码器的 `omitempty`;需要省略零值时显式使用 `omitempty`。
|
||||
|
||||
JSON 必须是一个 object,Content-Type 为 application/json。默认最多 1 MiB,包含 chunked/未知长度和尾部空白;超限返回 413。重复 JSON key、未知字段、多个 JSON 值、标量 query/header 重复值、格式和约束错误返回 400;错误 Content-Type 返回 415。失败不会部分修改目标 DTO;错误详情只保留在内部 cause,公共响应不回显原始参数。
|
||||
|
||||
`contract.Binder` 在服务启动时编译,随后并发复用,不会每次请求重新分析标签。CLI 用相同编译器产生文档,因此来源、必填、默认、nullable、枚举、范围和长度具有同一来源。响应类型和约束由业务实现遵守,不在每次响应上再次校验;非 nullable 集合请返回空集合而不是 nil。
|
||||
|
||||
## 生成目录和业务边界
|
||||
|
||||
```text
|
||||
api/ # 开发者维护的契约及导入
|
||||
cmd/server/main.go # 启动、日志、优雅退出
|
||||
etc/config.yaml
|
||||
internal/settings/
|
||||
internal/servicecontext/ # 依赖初始化和释放
|
||||
internal/types/typesGen.go
|
||||
internal/handler/routesGen.go
|
||||
internal/handler/openapiGen.go
|
||||
internal/handler/<group>/handlersGen.go
|
||||
internal/service/<group>/service.go
|
||||
internal/service/<group>/<operation>.go
|
||||
internal/repository/<group>/repository.go
|
||||
internal/middleware/registryGen.go
|
||||
internal/middleware/<name>.go
|
||||
docs/openapi.json
|
||||
.goOmni/apiManifest.json
|
||||
```
|
||||
|
||||
Service 方法接收标准 `context.Context` 与生成的请求 DTO;Handler 调用 Service 并完成响应包装。网络适配器仍经 Omni/httpx 接入。Repository 是业务所有的扩展位置,不自动生成虚构 CRUD 查询。
|
||||
|
||||
命名中间件按声明顺序组合,第一个在最外层。新 hook 初始返回 501,必须实现后才放行;registry 自动维护,hook 文件只创建一次。缺失 hook 在注册任何路由前报错。文档通过 `x-omni-middleware` 标出命名 hook,不把名称误当成已实现的认证协议。
|
||||
|
||||
## 安全更新与 CI
|
||||
|
||||
在生成项目中安装/使用与框架版本相符的 go-omni CLI:
|
||||
|
||||
```sh
|
||||
go-omni api validate --api api/user.api
|
||||
go-omni api generate --api api/user.api --dir .
|
||||
go-omni api generate --api api/user.api --dir . --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
`Gen.go` 和 OpenAPI 由生成器管理,manifest 保存其 SHA-256。手动修改后更新会报冲突。Service、Repository、main、配置、依赖装配和中间件实现按字节保留;只补缺失文件。删除接口/分组时只移除旧的生成文件,保留业务代码;已删除类型仍被旧业务引用时,需要明确迁移该业务文件,编译器会提示。
|
||||
|
||||
`--check` 只读,检查产物是否过期,以及所需骨架是否缺失。契约复制到项目后,请编辑项目内的 API;外部定义与已存在副本不一致时会拒绝覆盖。一个项目由一个 API 入口管理,可以通过导入拆分文件。
|
||||
|
||||
生成器拒绝输出路径穿越、受管理路径上的符号链接、大小写路径冲突和无所有权的覆盖。生成前统一校验/格式化,锁文件防止两个生成器同时修改;文件以原子替换写入,普通写入失败会回滚。本版不是整个目录的崩溃原子事务:进程被强杀或机器断电后,应按 Git 恢复生成产物和 manifest,再重试。请将 manifest 与生成产物一并提交,也不要在生成过程中编辑这些文件。
|
||||
|
||||
生成器不自动安装依赖、不访问数据库、不推送仓库、不替业务实现权限规则。
|
||||
|
||||
## 验证
|
||||
|
||||
```sh
|
||||
go test -race -p=2 ./contract ./openapi ./cmd/go-omni/...
|
||||
python3 scripts/verifyAPI.py
|
||||
```
|
||||
|
||||
Go 测试覆盖契约拒绝规则、多源绑定、零值/缺失、默认/范围/嵌套集合、文档约束、幂等更新、只读检查、业务保留、删除路由、路径边界及生成项目编译。Python 验收会在临时目录生成项目,分别启动 nethttp/native,验证真实 HTTP 请求、中间件、日志 requestId 和优雅退出;结束后清理自己创建的进程和临时文件。结果记录在 `performance/apiGenerationLocal.json`。
|
||||
12
CHANGELOG.md
12
CHANGELOG.md
@@ -1,6 +1,16 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
## v0.7.0-preview
|
||||
|
||||
- 新增 `.api` 契约驱动生成器:DTO、严格多来源绑定与校验、路由、分组 Handler / Service / Repository、命名中间件及 OpenAPI;生成文件哈希管理、业务文件保护和只读 `--check`。
|
||||
- `new --api` / `api new` / `api generate` 可以组合 MySQL、日志以及 Local / S3 / R2 / OSS。保存初始基础设施选项,重生成自动沿用;显式切换已有项目选项会拒绝,避免静默丢失初始化代码。
|
||||
- 新增 `examples/userService` 独立业务示例与 `verifyAPIWorkflow.py`:用户 CRUD / 分页、管理员 Token 鉴权、MySQL 事务审计、故障回滚、文件导出、requestId 日志、数据库 readiness 和优雅退出;支持从远程版本验收。
|
||||
- 日志新增文件轮转、结构化敏感字段脱敏、异常堆栈及慢请求配置;存储新增统一上传策略、直传签名和云端分片能力。
|
||||
- 新增缓存、HTTP 客户端、限流 / 幂等 / 超时、后台任务、Redis 队列、事务 outbox、分布式锁、审计、迁移与 telemetry 等可选模块;业务身份与 Orbit 保持由应用自行组合。
|
||||
- `database.ApplyEnv` 支持文件/默认配置叠加环境变量;数据库配置使用 lowerCamelCase,DSN 不参与 JSON/YAML 序列化。
|
||||
- 修复 native 在业务层提前拒绝请求体为 413 时等待继续读取 body 的问题,拒绝后关闭未读连接。
|
||||
|
||||
以下是自上一版本以来的网络层变化与历史验收记录:
|
||||
|
||||
- 新增 `TestNginxHTTP2Proxy` 与 `scripts/verifyHTTP2Proxy.py`:验证两段 TLS/h2、回源连接复用、证书校验拒绝、超时/客户端取消恢复,以及 GOAWAY 下的新旧实例切换和请求排空。Linux amd64、Nginx 1.30.5 普通与 race 模式各 3 轮通过;保留源码/镜像/日志校验信息和实际执行记录。
|
||||
- HTTP/2 代理验收的编译与测试阶段顺序运行,均限制 2 核、2 GiB、256 个进程/线程;集成测试禁用外部网络并使用只读根文件系统。总时限、SIGTERM、SIGHUP 清理实测通过;native 默认协议保持 HTTP/1.1。
|
||||
|
||||
172
MODULES.md
Normal file
172
MODULES.md
Normal file
@@ -0,0 +1,172 @@
|
||||
# 业务基础模块
|
||||
|
||||
第二批的共享 Redis 中间件、可靠队列、多平台存储及其他模块见 [扩展模块](ADVANCED.md)。下文的单进程约束指本页原有默认实现。
|
||||
|
||||
这批能力从 `v0.7.0-preview` 提供。统一示例见
|
||||
[examples/modules](examples/modules/README.md)。业务入口继续使用 `omni.New`,中间件依赖
|
||||
`httpx`,不依赖具体服务端网络实现。各模块按需创建、注入 Service,再注册到应用关闭流程。
|
||||
|
||||
## 缓存:cache 与 cache/redis
|
||||
|
||||
```go
|
||||
store, err := cache.NewMemory(cache.MemoryConfig{MaxEntries: 4096, MaxBytes: 32 << 20})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(store.Close)
|
||||
loader, err := cache.NewLoader(store, cache.LoaderConfig{MaxLoads: 128, Timeout: 5 * time.Second})
|
||||
if err != nil { return err }
|
||||
value, err := loader.GetOrLoad(ctx, "tenant:42:product:7", time.Minute,
|
||||
func(ctx context.Context) ([]byte, error) { return repository.LoadProduct(ctx, "42", "7") })
|
||||
```
|
||||
|
||||
- `Store` 统一 `Get/Set/Delete`,返回字节切片;业务可用 JSON 编解码。
|
||||
- 内存缓存使用 LRU,同时限制条目数和 key/value 总字节数;字节上限不包含 Go 对象开销。
|
||||
读写复制切片,过期时懒清理,没有每个条目一个 goroutine。
|
||||
- TTL 为零表示不过期,负值拒绝;缺失返回 `cache.ErrMiss`。删除不存在的 key 成功。
|
||||
- Loader 合并**同进程、同 key**的并发回源,最多同时运行 `MaxLoads` 个不同 key 的加载。
|
||||
单个调用者取消只停止等待;共享加载有独立超时,加载函数必须响应 context。
|
||||
超限返回 `ErrBusy`,缓存故障直接返回错误,不自动当成 miss 冲击数据库。
|
||||
- 同 key 的调用必须具有相同数据含义、权限范围和 TTL;加载采用首个调用者的 context 值。
|
||||
多租户 key 必须含租户标识。进程间合并、分布式锁、缓存更新与数据库事务一致性由业务另外设计。
|
||||
|
||||
Redis 适配器借用调用方创建的 go-redis 客户端和连接池:
|
||||
|
||||
```go
|
||||
client := goredis.NewClient(&goredis.Options{
|
||||
Addr: "127.0.0.1:6379", PoolSize: 16, MaxRetries: -1,
|
||||
DialTimeout: time.Second, ReadTimeout: time.Second,
|
||||
WriteTimeout: time.Second, PoolTimeout: time.Second,
|
||||
ContextTimeoutEnabled: true,
|
||||
})
|
||||
// 启动时先使用有截止时间的 context 执行 client.Ping(ctx)。
|
||||
store, err := rediscache.New(client, "billing:") // cache/redis 的导入别名
|
||||
if err != nil { _ = client.Close(); return err }
|
||||
application.OnShutdown(client.Close)
|
||||
// store 可直接传入 cache.NewLoader。
|
||||
```
|
||||
|
||||
连接失败与 Redis 错误保持原样;只有 Redis nil 映射为 `ErrMiss`。Redis TTL 精度为毫秒,
|
||||
正的亚毫秒 TTL 提升到 1ms。适配器不负责关闭借用的客户端,Redis 服务端容量策略由部署配置。
|
||||
连接池和超时配置参考 [go-redis 官方生产使用说明](https://redis.io/docs/latest/develop/clients/go/produsage/)。
|
||||
|
||||
## 出站 HTTP:httpc
|
||||
|
||||
```go
|
||||
client, err := httpc.New(httpc.Config{
|
||||
Timeout: 3*time.Second, MaxRetries: 2, MaxConnsPerHost: 32,
|
||||
MaxResponseBytes: 1 << 20,
|
||||
})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(client.Close)
|
||||
err = client.JSON(ctx, "GET", serviceURL, nil, &result)
|
||||
```
|
||||
|
||||
客户端应在服务启动时创建并复用。默认使用独立的 `net/http.Transport` 连接池,每个目标主机
|
||||
有连接上限;该上限不是整个进程的全局连接数上限。也可以注入自有 `http.RoundTripper`,
|
||||
其连接限制和生命周期由调用方负责。这样出站实现同样可以替换。
|
||||
|
||||
总超时覆盖建连、重试、等待和读取响应体。`Do` 返回后调用方必须读取/关闭 `response.Body`;
|
||||
`JSON` 自动关闭,并限制解码前读取的字节数。`Do` 允许流式读取,本身不限制响应体大小。
|
||||
`Close` 拒绝新请求、关闭自有池的空闲连接;在途请求应先排空。
|
||||
|
||||
`MaxRetries` 默认 0,最多 5。封装只对可重放的 GET/HEAD/OPTIONS 重试网络错误及
|
||||
429/502/503/504;POST/PUT/PATCH/DELETE 不做应用层自动重试。标准库 Transport 和借用的
|
||||
RoundTripper 自身的重放规则仍生效,不能据此承诺“只执行一次”。退避带抖动,尊重
|
||||
`Retry-After`;服务器要求等待超过 `MaxBackoff` 时直接返回该响应,不提前重试。
|
||||
重定向交给业务处理,3xx 不会自动跟随。
|
||||
|
||||
默认从 context 传递 `X-Request-ID`,日志只包含方法、主机、状态、尝试次数和收到响应头的耗时,
|
||||
不记录完整 URL、查询参数、凭证和响应体。非 2xx 的 JSON 调用返回只带状态码的 `StatusError`。
|
||||
|
||||
## 请求保护:middleware
|
||||
|
||||
```go
|
||||
limiter, err := middleware.NewRateLimiter(middleware.RateLimitConfig{Rate: 20, Burst: 40})
|
||||
if err != nil { return err }
|
||||
idempotency, err := middleware.NewIdempotency(middleware.IdempotencyConfig{
|
||||
Scope: func(c *httpx.Context) (string, error) {
|
||||
principal, _ := auth.FromContext(c)
|
||||
return principal.UserID, nil // 多租户应用应组合租户和用户 ID
|
||||
},
|
||||
})
|
||||
if err != nil { return err }
|
||||
api := application.Group("/api", middleware.Timeout(3*time.Second),
|
||||
auth.RequireAuth(authenticator), limiter.Middleware())
|
||||
api.POST("/orders", createOrder, idempotency.Middleware())
|
||||
```
|
||||
|
||||
限流为进程内令牌桶,`Rate` 为每秒额度,`Burst` 为突发容量。默认按已解析的可信客户端 IP,
|
||||
可用 `Key` 按已认证身份分桶。额度不足返回 **429 + Retry-After**;key 数超过 `MaxKeys`
|
||||
返回 **503**。满容量时不会踢掉活跃桶让攻击者绕过限流;空闲桶经过完整补充周期后才允许回收。
|
||||
|
||||
Timeout 设置协作式 deadline,不额外启动 Handler goroutine。业务及下游调用必须响应 context;
|
||||
超时后尚未提交响应时返回 **504**,已经提交的响应保持原状态。它不能强制中断任意 Go 代码,
|
||||
也不能替代网络层读写超时。不要在业务代码中异步持有 `httpx.Context`。
|
||||
|
||||
幂等保护适用于小型、非流式写接口:
|
||||
|
||||
- 必须先认证/鉴权,再执行幂等中间件;`Scope` 必须取自服务端验证过的身份。
|
||||
GET/HEAD/OPTIONS 直接通过,写请求要求 `Idempotency-Key`,长度 1..128 个非空格 ASCII 字符。
|
||||
- 同 Scope + key + 方法/URL/Content-Type/body 重放首次响应;内容改变返回 **409**。
|
||||
正在执行或结果不可重放的相同请求也返回 **409**,不并发执行第二次。
|
||||
- 默认最多 32 个处理中请求、256 条记录、16 MiB 保存的响应头/body,单次请求体 1 MiB、
|
||||
响应 body 64 KiB。请求超限 **413**,容量满 **503**,响应超限 **500**。
|
||||
- panic、5xx、超时、响应超限、Set-Cookie、Cache-Control:no-store 或保存容量不足时保留失败记录,
|
||||
阻止在 TTL 内自动重执行业务。业务可能已经提交,调用方应查询实际结果后处理。
|
||||
- 完成记录默认保留 10 分钟;在途记录不会因 TTL 到期而允许重复执行。
|
||||
重放保留原响应 body、原 `requestId` 与原 `X-Request-ID`,增加 `Idempotency-Replayed:true`;
|
||||
本次访问日志仍使用本次请求 ID。
|
||||
- 记录仅在当前进程有效,重启、TTL 到期和多副本切换后均无法保证去重。支付、库存等关键写入
|
||||
应在数据库事务内配合唯一业务键。该实现不提供跨进程 exactly-once,也不支持 SSE/流式/升级连接。
|
||||
|
||||
## 后台执行:task
|
||||
|
||||
```go
|
||||
jobs, err := task.New(task.Config{Workers: 4, QueueSize: 128, TaskTimeout: 30*time.Second})
|
||||
if err != nil { return err }
|
||||
application.OnShutdown(jobs.Close) // 在注册数据库/缓存关闭之后注册,先排空任务
|
||||
future, err := jobs.Submit(serviceContext, "refreshCatalog", refreshCatalog)
|
||||
if err != nil { return err } // 满队列立即返回 ErrQueueFull
|
||||
err = future.Wait(ctx) // 可选;不等待时,任务失败仍通过 OnError/logx 报告
|
||||
err = jobs.Every("refreshCatalog", time.Minute, refreshCatalog)
|
||||
```
|
||||
|
||||
固定 worker 和有界队列控制资源。任务保留提交 context;需要独立于 HTTP 请求执行时使用服务
|
||||
context,或明确使用 `context.WithoutCancel` 保留日志关联值。任务仍受 Runner 的超时/退出控制。
|
||||
任务函数与 `OnError` 回调应及时返回,尤其要响应取消;panic 被转为失败,worker 继续工作。
|
||||
|
||||
`Every` 使用固定间隔,第一次在一个间隔后执行;同一计划上轮尚未完成、排队或队列满时跳过。
|
||||
默认最多 128 个计划,名称不可重复。`Stats` 提供 active/queued/completed/failed/skipped。
|
||||
它是进程内任务执行器,没有任务持久化、失败自动重试、cron 表达式、跨副本选主或崩溃恢复。
|
||||
|
||||
`Shutdown(ctx)` 停止接收和调度,排空已接受任务;截止时间到后取消任务并及时返回。
|
||||
忽略取消的任务不能被 Go 强杀,可能继续运行。`Close` 默认给排空 10 秒。
|
||||
|
||||
## 增加业务模块:go-omni add
|
||||
|
||||
```sh
|
||||
go build -o /tmp/go-omni ./cmd/go-omni
|
||||
/tmp/go-omni add orderItems --dir /path/to/billing
|
||||
```
|
||||
|
||||
生成 `internal/orderItems/{module.go,model,repository,service,handler,README.md}`,包含 name/createdAt
|
||||
示例模型、内存 CRUD、输入校验、错误映射及可注入 Repository。模块名使用 lowerCamelCase。
|
||||
|
||||
在现有启动代码中导入模块后调用 `orderItems.Register(application, repository, authMiddleware)`。
|
||||
脚手架不自动修改现有路由、main、go.mod 或数据库,已有目录不会覆盖。
|
||||
内存 Repository 仅为可运行示例:重启丢数据,列表未分页;正式业务应实现自己的持久化 Repository。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyModules.py --goproxy https://goproxy.cn,direct
|
||||
```
|
||||
|
||||
Linux runner 顺序运行两个容器:全量 race/vet/build/独立生成项目,随后真实 Redis 适配验证。
|
||||
每次最多一个容器,限制 2 CPU、2 GiB(不额外使用 swap)、256 pids;Redis 测试容器禁外网,
|
||||
只用内部 loopback,不发布端口。Redis 集成二进制为静态非 race,适配器单测在全量 race 中覆盖。
|
||||
结束清理本轮容器,并记录提交、源文件摘要、镜像 ID、日志摘要和原有容器是否仍运行。
|
||||
|
||||
本轮通过结果和原始证据见 [业务模块验收](performance/modulesAcceptance.md)。
|
||||
63
README.md
63
README.md
@@ -5,7 +5,7 @@ GoOmni 是一套面向业务复用的 Go 后端基础框架。当前默认使用
|
||||
远程 module 地址是 `git.yuntuo.hk/k2safe/go-omni`。业务项目可以直接引用指定版本:
|
||||
|
||||
```sh
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
||||
go get git.yuntuo.hk/k2safe/go-omni@v0.7.0-preview
|
||||
```
|
||||
|
||||
`v0.4.0` 移除了 Gin 和 Gin 专属的 `WithMode` 配置;`omni.New()` 直接使用标准库路由,已有业务 Handler / Service 无需修改。
|
||||
@@ -13,7 +13,7 @@ go get git.yuntuo.hk/k2safe/go-omni@v0.6.0-preview
|
||||
创建新业务项目:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
@@ -27,6 +27,17 @@ go run ./cmd/server
|
||||
生成项目包含完整用户 CRUD、数据库 Repository、ServiceContext、建表脚本和就绪探针。
|
||||
先应用生成的 `schema.sql`,再配置 `GO_OMNI_DB_DSN`;启动时会检查连接与表结构,退出时释放连接池。
|
||||
|
||||
## API 定义与项目生成
|
||||
|
||||
当前源码新增 **API 定义驱动项目生成**:写 `.api` 定义请求、响应和路由,自动生成项目目录、DTO、Handler、Service 骨架及 OpenAPI。重复生成保留业务代码,支持只读 CI 检查,完整语法和示例见 [API 开发指南](API.md)。
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni api new --framework-dir . \
|
||||
--module example.com/omniDemo ../omniDemo
|
||||
```
|
||||
|
||||
API 生成支持 `--database mysql --storage local --log-output file` 组合初始化。上面的命令使用本地源码;远程安装见 [发布说明](RELEASE.md),完整业务案例见 [userService](examples/userService/README.md)。
|
||||
|
||||
## 目录结构
|
||||
|
||||
```text
|
||||
@@ -41,8 +52,19 @@ go-omni/
|
||||
├── metrics/ # 请求指标与状态统计
|
||||
├── config/ # 配置管理
|
||||
├── validate/ # 参数校验
|
||||
├── contract/ # API 字段契约、严格多源绑定与文档约束
|
||||
├── auth/ # 认证和权限接口
|
||||
├── database/ # database/sql 生命周期和事务
|
||||
├── cache/ # 有界内存缓存、Redis 适配与合并回源
|
||||
├── httpc/ # 出站 HTTP 连接池、超时和重试
|
||||
├── task/ # 有界后台任务与间隔调度
|
||||
├── queue/ # 持久队列接口与 Redis Streams 适配
|
||||
├── storage/ # Local、S3/R2、OSS 与临时下载链接
|
||||
├── openapi/ # 路由和 OpenAPI 3.0.3 契约
|
||||
├── telemetry/ # 可注入 OpenTelemetry 观测
|
||||
├── resilience/ # 熔断与上游并发隔离
|
||||
├── audit/ # 显式业务审计与可注入 Sink
|
||||
├── pagination/ # 有界分页和统一结果
|
||||
├── transport/nethttp/ # 标准库网络适配器
|
||||
├── transport/native/ # 自研 HTTP/1.1 服务端预览
|
||||
├── transport/http2/ # opt-in HTTP/2 实验适配器
|
||||
@@ -50,6 +72,13 @@ go-omni/
|
||||
└── cmd/ # go-omni 项目脚手架
|
||||
```
|
||||
|
||||
main 分支新增缓存、出站 HTTP、限流/幂等/超时中间件、后台任务和 `go-omni add` 模块脚手架。
|
||||
使用方式、默认容量和进程内能力的适用范围见 [业务基础模块](MODULES.md),
|
||||
可以直接运行 [组合示例](examples/modules/README.md)。这些能力从 `v0.7.0-preview` 提供。
|
||||
|
||||
扩展模块还包括 JWT/Session、Policy、文件配置及安全热更新、Redis 共享中间件、持久队列、多平台对象存储、OpenAPI、OpenTelemetry、熔断、迁移、审计和分页。
|
||||
使用与边界见 [扩展模块](ADVANCED.md),可运行 [文件服务组合示例](examples/platform/README.md)。go-omni 不集成 Orbit;需要流程编排的业务项目独立接入。
|
||||
|
||||
## 快速开始
|
||||
|
||||
```go
|
||||
@@ -150,7 +179,7 @@ app.GET("/metrics", requestMetrics.Handler())
|
||||
|
||||
接口返回请求总数、错误数、当前处理中请求、累计耗时和 HTTP 状态分布。它是进程内快照,生产环境可由业务侧定时采集到监控系统。
|
||||
|
||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。
|
||||
`WithLogger(true)` 会启用 GoOmni 的统一 HTTP 访问日志;日志内容和业务日志共用 `requestId`。应用启动时可以调用 `logx.Setup` 配置 JSON/文本格式、级别、服务名和输出文件。文件轮转、敏感字段脱敏、异常堆栈与慢请求配置见 [logx 文档](logx/README.md);文件直传与云端分片上传见 [storage 文档](storage/README.md)。 创建新业务项目时可通过 `go-omni new --log-output file --storage local` 按需生成配置、装配代码与可运行示例,当前源码使用方式见 [CLI 文档](cmd/go-omni/README.md)。
|
||||
|
||||
应用配置可以统一从 `GO_OMNI_*` 环境变量加载,再交给 `omni.New`:
|
||||
|
||||
@@ -175,6 +204,34 @@ application.OnShutdown(logger.Close)
|
||||
`X-Forwarded-For`,默认只记录 socket 对端地址。数据库连接仍由
|
||||
`database.ConfigFromEnv` 从 `GO_OMNI_DB_*` 加载。
|
||||
|
||||
网络层也由同一份配置选择,Handler / Service / Repository 不需要导入具体适配器:
|
||||
|
||||
| 配置 | 默认值 / 用途 |
|
||||
| --- | --- |
|
||||
| `GO_OMNI_TRANSPORT` | `nethttp`;可选 `native`、`http2` |
|
||||
| `GO_OMNI_TLS_CERT_FILE` | TLS 证书 PEM 文件路径 |
|
||||
| `GO_OMNI_TLS_KEY_FILE` | TLS 私钥 PEM 文件路径,必须与证书一起配置 |
|
||||
| `GO_OMNI_SHUTDOWN_TIMEOUT` | `10s`,三种适配器共用的请求排空时间 |
|
||||
|
||||
例如运行自研 HTTP/1.1,或显式试用独立 HTTP/2 适配器:
|
||||
|
||||
```sh
|
||||
GO_OMNI_TRANSPORT=native go run ./examples/reference
|
||||
GO_OMNI_TRANSPORT=http2 GO_OMNI_HTTP_ADDR=127.0.0.1:8443 \
|
||||
GO_OMNI_TLS_CERT_FILE=./server.crt GO_OMNI_TLS_KEY_FILE=./server.key \
|
||||
go run ./examples/reference
|
||||
```
|
||||
|
||||
`http2` 必须配置 TLS;native TLS 只协商 `http/1.1`,nethttp TLS 支持标准库的 HTTP/1.1 和 HTTP/2。
|
||||
未知适配器、缺少配对证书、无法加载证书时,`Run` / `RunContext` 在监听前返回错误并执行资源清理,不会退回其他协议。
|
||||
HTTP/2 仍为实验适配器,尚未合并到 native 多协议监听器。
|
||||
|
||||
代码配置可写成 `omni.New(omni.WithTransport(omni.TransportNative), omni.WithTLS(certFile, keyFile))`。
|
||||
`ServeContext(ctx, listener)` 使用相同配置并接管已有 listener;失败或退出时也会关闭它。
|
||||
`RunWith(ctx, runner)` 保留自定义网络层入口,由 runner 自己管理 TLS、超时和网络参数;连接/请求上限等适配器专属选项仍通过这个入口设置。
|
||||
`NewServer` 仅创建标准库明文服务器,配置选择应用于 `Run`、`RunContext` 和 `ServeContext`。
|
||||
三种适配器的配置、真实 TLS 业务和关闭行为已通过 Linux 全量 race 及 10 次重复回归,见 [统一启动验收](performance/transportStartup.md)。
|
||||
|
||||
`App` 同时实现标准 `http.Handler`。`Run` 监听 `SIGINT` / `SIGTERM`;`RunContext` 则由调用方的 context 控制退出,生成项目已使用 `signal.NotifyContext` 接好信号。
|
||||
退出时先停止接收新请求,等待活跃请求完成,再执行资源清理。超过 `ShutdownTimeout` 时强制关闭连接、取消请求 context,并返回超时错误。业务代码应响应 context 取消;框架无法强制结束忽略取消的业务 goroutine。数据库连接池由 `database.Open` 创建,在服务退出时调用 `Close`。
|
||||
|
||||
|
||||
63
RELEASE.md
Normal file
63
RELEASE.md
Normal file
@@ -0,0 +1,63 @@
|
||||
# v0.7.0-preview
|
||||
|
||||
这一版把 API 定义、项目生成、基础模块初始化和完整业务示例接通。
|
||||
预览版用于固定版本试用;API 方言有明确范围,HTTP/2 仍是独立实验适配器。
|
||||
|
||||
## 安装与开始
|
||||
|
||||
```sh
|
||||
GOPRIVATE=git.yuntuo.hk go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||
|
||||
go-omni api new --module example.com/demo ./demo
|
||||
cd demo
|
||||
GOPRIVATE=git.yuntuo.hk go mod tidy
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
需要该 Git 仓库的读取权限时,通过自己的 Git 凭据管理器配置;不要将凭据写入 module path 或 go.mod。
|
||||
CLI 会将自身下载版本固定到生成项目的 go.mod。业务项目正常使用远程模块,不需要本地 replace。
|
||||
|
||||
使用已有定义,并一次选择基础模块:
|
||||
|
||||
```sh
|
||||
go-omni new --api user.api --module example.com/userService \
|
||||
--database mysql --storage local --log-output file ./userService
|
||||
```
|
||||
|
||||
MySQL 与存储需要按生成 README 设置环境变量。任意契约生成的业务方法先返回 501,
|
||||
由业务方实现;可直接参考 [完整用户示例](examples/userService/README.md)。
|
||||
|
||||
## 主要变化
|
||||
|
||||
- `.api` 生成项目、绑定/校验、路由、业务骨架和 OpenAPI;支持分组、导入、命名中间件。
|
||||
- 同一入口初始化 MySQL、日志、Local / S3 / R2 / OSS;标准 context 与 httpx 保持业务/网络分离。
|
||||
- 生成器保护手写业务文件,校验自有文件哈希,支持只读 --check;基础设施选项持久化到 manifest。
|
||||
- 完整用户示例包含数据库迁移、CRUD / 分页、事务审计、鉴权、文件导出与 requestId 日志。
|
||||
- 日志文件轮转/结构化脱敏、上传策略/直传/分片,以及此前 main 分支的通用业务模块一并提供。
|
||||
|
||||
所有变化及网络层验收记录见 [CHANGELOG](CHANGELOG.md)。
|
||||
|
||||
## 升级注意
|
||||
|
||||
1. 将 CLI 和项目框架依赖固定为相同版本。旧版 CRUD 项目继续使用现有目录;不能直接用 API 生成器接管已有 Go 目录。
|
||||
2. Gen.go 由生成器维护,业务实现/配置/迁移由项目维护。新增 API 后执行 generate 和 go test;修改或删除类型需要同步处理自己的业务代码。
|
||||
3. `.goOmni/apiManifest.json` 需随代码提交。切换数据库或存储选项时,在新目录生成对照后迁移;重新生成不会改写现有基础设施文件。
|
||||
4. 配置使用 lowerCamelCase。DSN、Token、存储密钥由环境变量提供;`database.Config` 的 DSN 不会被 JSON/YAML 序列化。
|
||||
5. API 的 none/memory 数据库选项仅表示不打开连接,不会从 HTTP DTO 自动推导内存仓库或数据表。
|
||||
|
||||
## 验证入口
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race -p=2 -count=1 -timeout=300s ./...
|
||||
go vet ./...
|
||||
go build ./...
|
||||
python3 scripts/verifyAPIWorkflow.py
|
||||
python3 scripts/verifyAPIWorkflow.py --version v0.7.0-preview \
|
||||
--report performance/apiWorkflowRemote.json
|
||||
```
|
||||
|
||||
最后一项在全新 module cache 安装远程 CLI,并验证无本地 replace 的项目生成、
|
||||
真实 MySQL、nethttp/native 启动与业务流程。容器只使用本机临时资源,结束时清理。
|
||||
本轮验收见 [API 业务闭环报告](performance/apiWorkflowAcceptance.md)。
|
||||
|
||||
OSS/R2 云端仍需项目自己的账号做上线环境验收;当前云端适配器检查不等同于这些账号的实网测试。
|
||||
24
ROADMAP.md
24
ROADMAP.md
@@ -58,7 +58,10 @@
|
||||
- 已补连接级窗口耗尽时的暂停、同连接 204/跨连接隔离、WINDOW_UPDATE 恢复和连接复用;已补零窗口写阻塞在关闭超时后的解除阻塞、请求取消与回收。Linux amd64 的 HTTP/2 全包 race、vet、build 已通过,见 [边界回归报告](performance/http2FlowControlLinux.json)。
|
||||
- 新增 HTTP/2 零 stream 窗口时的慢响应隔离、WINDOW_UPDATE 恢复、RST_STREAM 取消与请求槽位回收测试;补充 TLS 半开握手占满连接上限时的及时拒绝、超时回收及正常请求恢复测试。
|
||||
- 已完成 Nginx 1.30.5 → HTTP/2 的 TLS 回源联调:普通与 race 模式各 3 轮,共 18 个子场景通过;覆盖两段证书校验、32 次 POST 复用一条回源连接、不可信 CA/主机名拒绝、504 超时、客户端取消、GOAWAY 后旧 POST 完成及 20 个新 POST 转入新实例,无重复业务执行。见 [验收说明与报告](transport/http2/PROXY.md)。
|
||||
- `scripts/verifyHTTP2Proxy.py` 顺序运行受限编译容器和禁用外部网络的测试容器;总时限、SIGTERM、SIGHUP 清理已在 Linux 实测,原有 7 个业务容器仍运行。HTTP/2 下一步是统一启动配置、native 多协议接入回归与业务试用;当前结果不覆盖 Nginx master/worker 重载或 Cloudflare 实网部署。
|
||||
- `scripts/verifyHTTP2Proxy.py` 顺序运行受限编译容器和禁用外部网络的测试容器;总时限、SIGTERM、SIGHUP 清理已在 Linux 实测,原有 7 个业务容器仍运行。当前结果不覆盖 Nginx master/worker 重载或 Cloudflare 实网部署。
|
||||
- 已接入 `omni.New` 的统一 transport / TLS / shutdownTimeout 配置:默认 nethttp,可显式选择 native 或实验 HTTP/2;证书在监听前加载,HTTP/2 配置必须启用 TLS。`RunWith` 继续支持自定义网络层,业务不依赖具体适配器。
|
||||
- 参考业务已增加五种真实 TCP/TLS 组合的 CRUD、认证权限、错误响应、requestId 日志与协议协商测试;三种适配器共用排空和超时退出测试。修复 native 退出时过早取消在途请求,并使关闭超时错误返回调用方。下一步仍为业务环境试用、native 多协议监听器设计与接入回归,HTTP/2 保持实验状态。
|
||||
- 统一启动已通过 Linux 全仓库 race/vet/build 及关键场景 10 次重复验证;修复了重复测试发现的 native 活动连接转为空闲时错过关闭的竞态,并保证不响应取消的 Handler 不阻塞关闭超时返回。脚手架独立项目的 native SIGINT/SIGTERM 验证通过;失败复现与最终验收证据见 [统一启动验收](performance/transportStartup.md)。
|
||||
- 已用 16 个并发 worker 完成 3,200 次建连/断连 race 验收;MaxConnections 满载拒绝、槽位释放和 TLS 连接上限均已覆盖。
|
||||
- 新增 `scripts/verifyLinux.py`,固定在 Go 1.25 Linux 容器内运行 race、vet、build 和可选 TLS/非 TLS perfcheck;perfcheck 记录堆、goroutine、连接和 Linux 文件描述符。
|
||||
- Linux/arm64 已完成 30 秒、128 并发、512 空闲连接的 TLS 与非 TLS 持续回归,报告见 `performance/linuxSoak.json` 和 `performance/linuxSoakTLS.json`,均为 0 错误且连接、goroutine、FD 回收正常。
|
||||
@@ -66,4 +69,23 @@
|
||||
- 新增 `scripts/verifyProxy.py`,在 Linux Docker 中用 Nginx 反向代理 native TLS 服务,检查真实 TLS、Keep-Alive、上游超时和 native 优雅关闭。
|
||||
- 自动验收入口:`python3 scripts/verifyPerformance.py`;Linux 固定验收入口:`python3 scripts/verifyLinux.py --load --tls --duration 30m`;代理联调入口:`python3 scripts/verifyProxy.py`。
|
||||
|
||||
## 6. 可复用业务基础模块(第一版)
|
||||
|
||||
- `cache`:有界 LRU 内存缓存、TTL、数据复制、同进程合并回源;`cache/redis` 适配 go-redis 客户端,保留借用连接池的生命周期边界。
|
||||
- `httpc`:复用出站连接、每目标主机连接上限、覆盖响应体的总超时、保守重试、requestId 传递、JSON 响应大小限制与安全日志字段。
|
||||
- `middleware`:有容量上限的令牌桶限流;按已认证身份隔离的幂等保护,覆盖冲突、处理中记录、失败记录、容量与请求/响应大小上限;协作式请求 deadline。
|
||||
- `task`:固定 worker、有界队列、任务超时、panic 隔离、间隔调度、不重叠执行、停止接收与排空/取消。
|
||||
- `go-omni add`:向已有项目生成 Handler / Service / Repository / Model 和模块注册入口;目标存在时拒绝覆盖,默认内存 CRUD 可注入持久化 Repository。
|
||||
- `examples/modules` 将以上运行时能力串联,使用真实 TLS 在 nethttp、native、实验 HTTP/2 下验证缓存、出站 requestId、认证、幂等、后台任务与退出。
|
||||
- 使用文档:[业务基础模块](MODULES.md)。验证入口:`GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./...` 和 `python3 scripts/verifyModules.py`;Linux runner 顺序运行全量检查及独立 Redis 容器,硬限制 2 CPU / 2 GiB / 256 pids。
|
||||
- 最终代码已通过 Linux amd64 全仓库 race/vet/build、独立生成项目 CRUD、真实 Redis 7.4.11 与三种网络层 TLS 组合测试;测试容器清理完成、原有 7 个业务容器仍运行。提交、源码摘要与完整输出见 [验收记录](performance/modulesAcceptance.md)。
|
||||
- 当前幂等、限流、任务调度和合并回源均为单进程能力;Redis 仅提供缓存存储,不代表已实现分布式去重、任务持久化或跨副本选主。eTax 指定业务模块接入依用户要求暂缓。
|
||||
|
||||
## 7. 扩展业务模块(本批)
|
||||
|
||||
- 新增 OpenAPI、JWT/Session/Policy、OpenTelemetry、文件配置与日志级别热更新、熔断/并发隔离、Redis 共享限流/幂等、Redis Streams 持久队列、Local/S3/R2/OSS 存储;补充迁移、审计和分页。
|
||||
- 业务依赖独立接口,云平台 SDK 和 Redis 留在适配器中;按本次范围决定不集成 Orbit,不实现工作流编排。
|
||||
- 已完成本地模块测试及 race 检查,Linux 全仓库 race/vet/build、生成项目 CRUD、三个 Runner 的 TLS 示例和独立 Redis/MinIO/MySQL 均通过。共享状态、重复提交、失败消息、未确认消息接管、旧确认拒绝、对象签名和迁移 dirty 阻断有对应证据,见 [扩展模块验收](performance/advancedAcceptance.md)。OSS/R2 尚无云账号实网验收。
|
||||
- 契约、使用示例与限制见 [扩展模块](ADVANCED.md);验证入口为 `scripts/verifyModules.py --advanced`,继续限制单个测试容器 2 CPU / 2 GiB / 256 pids。
|
||||
|
||||
以上阶段以验收结果判断完成。当前仍是基础框架试用阶段;MySQL、适配层契约和一次业务模块试用已完成,自研网络层已有可运行预览,稳定性与生产规模验证尚未完成。
|
||||
|
||||
46
app/app.go
46
app/app.go
@@ -31,7 +31,7 @@ type App struct {
|
||||
}
|
||||
|
||||
// Runner is the small lifecycle boundary used by an HTTP server
|
||||
// implementation. The default App.RunContext uses net/http; a custom runner
|
||||
// implementation. App.RunContext selects a configured built-in server; a custom runner
|
||||
// can own the listener, protocol parser and connection lifecycle.
|
||||
type Runner interface {
|
||||
Run(context.Context, http.Handler, string) error
|
||||
@@ -65,13 +65,14 @@ func configure(options ...config.Option) config.Config {
|
||||
|
||||
func newApp(router httpx.Router, settings config.Config) *App {
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
resolver, err := httpx.NewClientIPResolver(settings.TrustedProxies...)
|
||||
if err != nil {
|
||||
panic("go-omni: " + err.Error())
|
||||
}
|
||||
middlewares := []httpx.Middleware{middleware.RequestID(settings.RequestIDHeader)}
|
||||
if settings.EnableLogger {
|
||||
middlewares = append(middlewares, middleware.AccessLog())
|
||||
middlewares = append(middlewares, middleware.AccessLog(middleware.WithSlowRequestThreshold(settings.SlowRequestThreshold)))
|
||||
}
|
||||
middlewares = append(middlewares, middleware.Recovery())
|
||||
return &App{router: router, config: settings, middlewares: middlewares, clientIPResolver: resolver}
|
||||
@@ -93,6 +94,7 @@ func (a *App) ServeHTTP(writer http.ResponseWriter, request *http.Request) {
|
||||
func (a *App) Config() config.Config {
|
||||
settings := a.config
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
return settings
|
||||
}
|
||||
|
||||
@@ -133,7 +135,8 @@ func (a *App) Run(address ...string) error {
|
||||
return a.RunContext(ctx, address...)
|
||||
}
|
||||
|
||||
// NewServer creates a standard HTTP server around the application.
|
||||
// NewServer creates a plain standard HTTP server around the application.
|
||||
// Use RunContext or ServeContext to apply the configured transport and TLS.
|
||||
func (a *App) NewServer(address string) *http.Server {
|
||||
if address == "" {
|
||||
address = a.config.Address
|
||||
@@ -152,23 +155,21 @@ func (a *App) RunContext(ctx context.Context, address ...string) error {
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
listenOn := a.config.Address
|
||||
if listenOn == "" {
|
||||
listenOn = ":8080"
|
||||
}
|
||||
if len(address) > 0 && address[0] != "" {
|
||||
listenOn = address[0]
|
||||
}
|
||||
server := a.NewServer(listenOn)
|
||||
listener, err := net.Listen("tcp", server.Addr)
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
return a.serveContext(ctx, server, listener)
|
||||
listener, err := net.Listen("tcp", a.listenAddress(address...))
|
||||
if err != nil {
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
// RunWith starts a custom server runner while keeping App-owned cleanup in the
|
||||
// same lifecycle. The runner receives the transport-neutral App handler.
|
||||
// same lifecycle. The runner receives the transport-neutral App handler and
|
||||
// owns transport/TLS settings; built-in transport selection is bypassed.
|
||||
func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) error {
|
||||
if runner == nil {
|
||||
return errors.Join(errors.New("go-omni: nil runner"), a.Close())
|
||||
@@ -176,6 +177,13 @@ func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) err
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
return a.Close()
|
||||
}
|
||||
return errors.Join(runner.Run(ctx, a, a.listenAddress(address...)), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) listenAddress(address ...string) string {
|
||||
listenOn := a.config.Address
|
||||
if listenOn == "" {
|
||||
listenOn = ":8080"
|
||||
@@ -183,12 +191,18 @@ func (a *App) RunWith(ctx context.Context, runner Runner, address ...string) err
|
||||
if len(address) > 0 && address[0] != "" {
|
||||
listenOn = address[0]
|
||||
}
|
||||
return errors.Join(runner.Run(ctx, a, listenOn), a.Close())
|
||||
return listenOn
|
||||
}
|
||||
|
||||
func (a *App) serveContext(ctx context.Context, server *http.Server, listener net.Listener) error {
|
||||
serveErr := make(chan error, 1)
|
||||
go func() { serveErr <- server.Serve(listener) }()
|
||||
go func() {
|
||||
if server.TLSConfig != nil {
|
||||
serveErr <- server.ServeTLS(listener, "", "")
|
||||
} else {
|
||||
serveErr <- server.Serve(listener)
|
||||
}
|
||||
}()
|
||||
|
||||
select {
|
||||
case err := <-serveErr:
|
||||
|
||||
79
app/transport.go
Normal file
79
app/transport.go
Normal file
@@ -0,0 +1,79 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/tls"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
h2transport "git.yuntuo.hk/k2safe/go-omni/transport/http2"
|
||||
"git.yuntuo.hk/k2safe/go-omni/transport/native"
|
||||
)
|
||||
|
||||
// ServeContext serves a caller-created listener using the configured transport
|
||||
// and TLS. App takes ownership of the listener, including on startup failure,
|
||||
// and runs resource cleanup after the server has stopped.
|
||||
func (a *App) ServeContext(ctx context.Context, listener net.Listener) error {
|
||||
if listener == nil {
|
||||
return errors.Join(errors.New("go-omni: nil listener"), a.Close())
|
||||
}
|
||||
defer listener.Close()
|
||||
if ctx == nil {
|
||||
ctx = context.Background()
|
||||
}
|
||||
if ctx.Err() != nil {
|
||||
_ = listener.Close()
|
||||
return a.Close()
|
||||
}
|
||||
serve, err := a.prepareTransport()
|
||||
if err != nil {
|
||||
_ = listener.Close()
|
||||
return errors.Join(err, a.Close())
|
||||
}
|
||||
return errors.Join(serve(ctx, listener), a.Close())
|
||||
}
|
||||
|
||||
func (a *App) prepareTransport() (func(context.Context, net.Listener) error, error) {
|
||||
if err := a.config.ValidateTransport(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
shutdownTimeout := a.config.ShutdownTimeout
|
||||
if shutdownTimeout <= 0 {
|
||||
shutdownTimeout = 10 * time.Second
|
||||
}
|
||||
var tlsConfig *tls.Config
|
||||
if strings.TrimSpace(a.config.TLS.CertFile) != "" {
|
||||
certificate, err := tls.LoadX509KeyPair(a.config.TLS.CertFile, a.config.TLS.KeyFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("go-omni: load TLS certificate: %w", err)
|
||||
}
|
||||
tlsConfig = &tls.Config{Certificates: []tls.Certificate{certificate}, MinVersion: tls.VersionTLS12}
|
||||
}
|
||||
switch a.config.Transport {
|
||||
case config.TransportNative:
|
||||
server := native.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
if tlsConfig != nil {
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
}
|
||||
return server.Serve, nil
|
||||
case config.TransportHTTP2:
|
||||
server := h2transport.New(a)
|
||||
server.ShutdownTimeout = shutdownTimeout
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return server.ServeTLS(ctx, listener, tlsConfig)
|
||||
}, nil
|
||||
default:
|
||||
server := a.NewServer("")
|
||||
server.TLSConfig = tlsConfig
|
||||
return func(ctx context.Context, listener net.Listener) error {
|
||||
return a.serveContext(ctx, server, listener)
|
||||
}, nil
|
||||
}
|
||||
}
|
||||
322
app/transport_test.go
Normal file
322
app/transport_test.go
Normal file
@@ -0,0 +1,322 @@
|
||||
package app
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
func TestConfiguredTransportLifecycle(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
for _, timeout := range []bool{false, true} {
|
||||
name := transport + "/drain"
|
||||
if timeout {
|
||||
name = transport + "/timeout"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
limit := 3 * time.Second
|
||||
if timeout {
|
||||
limit = 80 * time.Millisecond
|
||||
}
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(limit))
|
||||
started, release, canceled := make(chan struct{}), make(chan struct{}), make(chan struct{})
|
||||
var once sync.Once
|
||||
unblock := func() { once.Do(func() { close(release) }) }
|
||||
defer unblock()
|
||||
var cleaned atomic.Int32
|
||||
application.OnShutdown(func() error { cleaned.Add(1); return nil })
|
||||
application.GET("/slow", func(ctx *httpx.Context) error {
|
||||
close(started)
|
||||
select {
|
||||
case <-ctx.Context().Done():
|
||||
close(canceled)
|
||||
return ctx.Context().Err()
|
||||
case <-release:
|
||||
return ctx.OK("finished")
|
||||
}
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wrapped := &closingListener{Listener: listener, closed: make(chan struct{})}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
defer listener.Close()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, wrapped) }()
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan error, 1)
|
||||
go func() {
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/slow")
|
||||
if err == nil {
|
||||
defer response.Body.Close()
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
err = readErr
|
||||
if response.StatusCode != 200 || !strings.Contains(string(body), "finished") {
|
||||
err = errors.New("drained response was lost")
|
||||
}
|
||||
}
|
||||
requestDone <- err
|
||||
}()
|
||||
await(t, started)
|
||||
began := time.Now()
|
||||
cancel()
|
||||
await(t, wrapped.closed)
|
||||
if !timeout {
|
||||
select {
|
||||
case <-canceled:
|
||||
t.Fatal("active request canceled before drain")
|
||||
default:
|
||||
}
|
||||
if cleaned.Load() != 0 {
|
||||
t.Fatal("cleanup preceded active request")
|
||||
}
|
||||
unblock()
|
||||
}
|
||||
select {
|
||||
case err := <-done:
|
||||
if timeout {
|
||||
if !errors.Is(err, context.DeadlineExceeded) || time.Since(began) < limit {
|
||||
t.Fatalf("timeout ignored: %v after %v", err, time.Since(began))
|
||||
}
|
||||
await(t, canceled)
|
||||
} else if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("server failed to stop")
|
||||
}
|
||||
select {
|
||||
case err := <-requestDone:
|
||||
if !timeout && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Fatal("client failed to finish")
|
||||
}
|
||||
_ = application.Close()
|
||||
if cleaned.Load() != 1 {
|
||||
t.Fatalf("cleanup count %d", cleaned.Load())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredStartupFailureClosesResources(t *testing.T) {
|
||||
occupied, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer occupied.Close()
|
||||
for _, test := range []struct {
|
||||
name, errorText string
|
||||
options []config.Option
|
||||
}{
|
||||
{"unknown", "transport", []config.Option{config.WithTransport("unknown")}},
|
||||
{"http2WithoutTLS", "tls", []config.Option{config.WithTransport("http2")}},
|
||||
{"partialTLS", "tls", []config.Option{config.WithTLS("missing.crt", "")}},
|
||||
{"nethttpCertificate", "load TLS certificate", []config.Option{config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"nativeCertificate", "load TLS certificate", []config.Option{config.WithTransport("native"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
{"http2Certificate", "load TLS certificate", []config.Option{config.WithTransport("http2"), config.WithTLS("missing.crt", "missing.key")}},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
application := New(test.options...)
|
||||
cleanupError := errors.New("cleanup failure")
|
||||
count := 0
|
||||
application.OnShutdown(func() error { count++; return cleanupError })
|
||||
// The occupied address must not mask configuration/certificate errors.
|
||||
err := application.RunContext(context.Background(), occupied.Addr().String())
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorText) || !errors.Is(err, cleanupError) {
|
||||
t.Fatalf("startup: %v", err)
|
||||
}
|
||||
_ = application.Close()
|
||||
if count != 1 {
|
||||
t.Fatalf("cleanup count %d", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("unknown"))
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
err = application.ServeContext(ctx, listener)
|
||||
cancel()
|
||||
if canceled && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !canceled && err == nil {
|
||||
t.Fatal("invalid transport accepted")
|
||||
}
|
||||
if _, err := listener.Accept(); !errors.Is(err, net.ErrClosed) {
|
||||
t.Fatalf("listener leaked: %v", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRunContextConfiguredTLS(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
reserved, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
address := reserved.Addr().String()
|
||||
reserved.Close()
|
||||
settings := config.Default()
|
||||
settings.Address, settings.Transport = address, transport
|
||||
settings.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||
application := New(config.WithConfig(settings))
|
||||
application.GET("/health", func(ctx *httpx.Context) error { return ctx.OK("ok") })
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.RunContext(ctx) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("RunContext failed to stop")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
deadline := time.Now().Add(3 * time.Second)
|
||||
for {
|
||||
response, err := client.Get("https://" + address + "/health")
|
||||
if err == nil {
|
||||
body, readErr := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if readErr != nil || response.StatusCode != 200 || !strings.Contains(string(body), `"data":"ok"`) {
|
||||
t.Fatalf("response %s: %v", body, readErr)
|
||||
}
|
||||
want := "h2"
|
||||
if transport == config.TransportNative {
|
||||
want = "http/1.1"
|
||||
}
|
||||
if response.TLS.NegotiatedProtocol != want || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatalf("TLS negotiation: %+v", response.TLS)
|
||||
}
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type runnerFunc func(context.Context, http.Handler, string) error
|
||||
|
||||
func (f runnerFunc) Run(ctx context.Context, handler http.Handler, address string) error {
|
||||
return f(ctx, handler, address)
|
||||
}
|
||||
|
||||
func TestRunWithRemainsIndependentOfBuiltInTransport(t *testing.T) {
|
||||
for _, canceled := range []bool{false, true} {
|
||||
application := New(config.WithTransport("custom"), config.WithTLS("missing", "missing"))
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
if canceled {
|
||||
cancel()
|
||||
}
|
||||
calls, cleanups := 0, 0
|
||||
application.OnShutdown(func() error { cleanups++; return nil })
|
||||
err := application.RunWith(ctx, runnerFunc(func(_ context.Context, handler http.Handler, address string) error {
|
||||
calls++
|
||||
if handler != application || address != "127.0.0.1:1234" {
|
||||
t.Fatal("custom runner arguments changed")
|
||||
}
|
||||
return nil
|
||||
}), "127.0.0.1:1234")
|
||||
cancel()
|
||||
wantCalls := 1
|
||||
if canceled {
|
||||
wantCalls = 0
|
||||
}
|
||||
if err != nil || calls != wantCalls || cleanups != 1 {
|
||||
t.Fatalf("RunWith: %v calls=%d cleanups=%d", err, calls, cleanups)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConfiguredShutdownDoesNotWaitForUncooperativeHandler(t *testing.T) {
|
||||
for _, transport := range []string{config.TransportNetHTTP, config.TransportNative, config.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
application := New(config.WithTransport(transport), config.WithTLS(cert, key), config.WithShutdownTimeout(80*time.Millisecond))
|
||||
started := make(chan context.Context, 1)
|
||||
release, finished := make(chan struct{}), make(chan struct{})
|
||||
application.GET("/blocked", func(ctx *httpx.Context) error {
|
||||
defer close(finished)
|
||||
started <- ctx.Context()
|
||||
<-release
|
||||
return nil
|
||||
})
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer listener.Close()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
requestDone := make(chan struct{})
|
||||
go func() {
|
||||
defer close(requestDone)
|
||||
response, err := client.Get("https://" + listener.Addr().String() + "/blocked")
|
||||
if err == nil {
|
||||
response.Body.Close()
|
||||
}
|
||||
}()
|
||||
var requestContext context.Context
|
||||
select {
|
||||
case requestContext = <-started:
|
||||
case <-time.After(3 * time.Second):
|
||||
close(release)
|
||||
t.Fatal("handler did not start")
|
||||
}
|
||||
defer func() { close(release); await(t, finished) }()
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatalf("shutdown: %v", err)
|
||||
}
|
||||
case <-time.After(2 * time.Second):
|
||||
t.Fatal("handler defeated shutdown deadline")
|
||||
}
|
||||
// net/http's HTTP/2 streams observe the closed socket asynchronously.
|
||||
await(t, requestContext.Done())
|
||||
await(t, requestDone)
|
||||
})
|
||||
}
|
||||
}
|
||||
106
audit/audit.go
Normal file
106
audit/audit.go
Normal file
@@ -0,0 +1,106 @@
|
||||
// Package audit records explicitly selected business events without request bodies.
|
||||
package audit
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Event struct {
|
||||
Time time.Time `json:"time"`
|
||||
ActorID string `json:"actorId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Action string `json:"action"`
|
||||
Resource string `json:"resource"`
|
||||
Outcome string `json:"outcome"`
|
||||
RequestID string `json:"requestId,omitempty"`
|
||||
TraceID string `json:"traceId,omitempty"`
|
||||
Fields map[string]string `json:"fields,omitempty"`
|
||||
}
|
||||
type Sink interface {
|
||||
Write(context.Context, Event) error
|
||||
}
|
||||
type Recorder struct {
|
||||
sink Sink
|
||||
allowed map[string]bool
|
||||
}
|
||||
|
||||
func New(sink Sink, allowedFields ...string) (*Recorder, error) {
|
||||
if sink == nil || len(allowedFields) > 32 {
|
||||
return nil, errors.New("audit: sink and at most 32 fields required")
|
||||
}
|
||||
r := &Recorder{sink: sink, allowed: map[string]bool{}}
|
||||
for _, name := range allowedFields {
|
||||
if name == "" || len(name) > 64 {
|
||||
return nil, errors.New("audit: invalid field")
|
||||
}
|
||||
r.allowed[name] = true
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
func (r *Recorder) Record(ctx context.Context, event Event) error {
|
||||
if ctx == nil {
|
||||
return errors.New("audit: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if event.ActorID == "" || event.Action == "" || event.Resource == "" || (event.Outcome != "success" && event.Outcome != "failure" && event.Outcome != "denied") {
|
||||
return errors.New("audit: actor/action/resource and valid outcome required")
|
||||
}
|
||||
for _, value := range []string{event.ActorID, event.TenantID, event.Action, event.Resource} {
|
||||
if len(value) > 512 {
|
||||
return errors.New("audit: event field too large")
|
||||
}
|
||||
}
|
||||
event.Time = time.Now().UTC()
|
||||
event.RequestID = logx.RequestID(ctx)
|
||||
event.TraceID = logx.TraceID(ctx)
|
||||
filtered := map[string]string{}
|
||||
for name, value := range event.Fields {
|
||||
if r.allowed[name] {
|
||||
if len(value) > 1024 {
|
||||
return errors.New("audit: attribute too large")
|
||||
}
|
||||
filtered[name] = value
|
||||
}
|
||||
}
|
||||
event.Fields = filtered
|
||||
return r.sink.Write(ctx, event)
|
||||
}
|
||||
|
||||
// JSONSink borrows its writer and serializes complete JSON lines. Write errors
|
||||
// are returned; callers choose whether a failed audit must reject an operation.
|
||||
type JSONSink struct {
|
||||
mu sync.Mutex
|
||||
writer io.Writer
|
||||
}
|
||||
|
||||
func NewJSONSink(writer io.Writer) (*JSONSink, error) {
|
||||
if writer == nil {
|
||||
return nil, errors.New("audit: writer required")
|
||||
}
|
||||
return &JSONSink{writer: writer}, nil
|
||||
}
|
||||
func (s *JSONSink) Write(ctx context.Context, event Event) error {
|
||||
data, err := json.Marshal(event)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data = append(data, '\n')
|
||||
s.mu.Lock()
|
||||
defer s.mu.Unlock()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
n, err := s.writer.Write(data)
|
||||
if err == nil && n != len(data) {
|
||||
return io.ErrShortWrite
|
||||
}
|
||||
return err
|
||||
}
|
||||
50
audit/audit_test.go
Normal file
50
audit/audit_test.go
Normal file
@@ -0,0 +1,50 @@
|
||||
package audit
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"io"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
type failedWriter struct{}
|
||||
|
||||
func (failedWriter) Write([]byte) (int, error) { return 0, io.ErrClosedPipe }
|
||||
func TestAllowlistCorrelationAndWriteFailure(t *testing.T) {
|
||||
var buf bytes.Buffer
|
||||
sink, _ := NewJSONSink(&buf)
|
||||
recorder, _ := New(sink, "status")
|
||||
ctx := logx.WithRequestID(context.Background(), "r1")
|
||||
ctx = logx.WithTraceID(ctx, "t1")
|
||||
event := Event{ActorID: "u1", Action: "update", Resource: "object:1", Outcome: "success", Fields: map[string]string{"status": "ok", "password": "secret"}}
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 20; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
if err := recorder.Record(ctx, event); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
decoder := json.NewDecoder(&buf)
|
||||
for i := 0; i < 20; i++ {
|
||||
var got Event
|
||||
if err := decoder.Decode(&got); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Fields["password"] != "" || got.RequestID != "r1" || got.TraceID != "t1" || got.Time.IsZero() {
|
||||
t.Fatal(got)
|
||||
}
|
||||
}
|
||||
bad, _ := NewJSONSink(failedWriter{})
|
||||
recorder, _ = New(bad)
|
||||
if err := recorder.Record(ctx, event); !errors.Is(err, io.ErrClosedPipe) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
105
auth/adapters_test.go
Normal file
105
auth/adapters_test.go
Normal file
@@ -0,0 +1,105 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rand"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestJWTValidationAndRotation(t *testing.T) {
|
||||
pub, private, err := ed25519.GenerateKey(rand.Reader)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
j, err := NewJWT(JWTConfig{Issuer: "issuer", Audience: "api", Keys: map[string]JWTKey{"first": {Algorithm: "EdDSA", PublicKey: pub}}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
claims := JWTClaims{RegisteredClaims: jwt.RegisteredClaims{Issuer: "issuer", Subject: "user1", Audience: jwt.ClaimStrings{"api"}, ExpiresAt: jwt.NewNumericDate(time.Now().Add(time.Hour))}, TenantID: "tenant1", Roles: []string{"reader"}}
|
||||
sign := func(c JWTClaims, kid string, method jwt.SigningMethod, key any) string {
|
||||
t.Helper()
|
||||
token := jwt.NewWithClaims(method, c)
|
||||
token.Header["kid"] = kid
|
||||
raw, err := token.SignedString(key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return raw
|
||||
}
|
||||
raw := sign(claims, "first", jwt.SigningMethodEdDSA, private)
|
||||
// Constructor clones caller-owned key bytes.
|
||||
pub[0] ^= 1
|
||||
p, err := j.Authenticate(context.Background(), raw)
|
||||
if err != nil || p.TenantID != "tenant1" || !p.HasRole("reader") {
|
||||
t.Fatalf("%+v %v", p, err)
|
||||
}
|
||||
for name, change := range map[string]func(*JWTClaims){"issuer": func(c *JWTClaims) { c.Issuer = "bad" }, "audience": func(c *JWTClaims) { c.Audience = jwt.ClaimStrings{"other"} }, "expired": func(c *JWTClaims) { c.ExpiresAt = jwt.NewNumericDate(time.Now().Add(-time.Hour)) }, "noExpiry": func(c *JWTClaims) { c.ExpiresAt = nil }, "future": func(c *JWTClaims) { c.NotBefore = jwt.NewNumericDate(time.Now().Add(time.Hour)) }, "subject": func(c *JWTClaims) { c.Subject = "" }} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
bad := claims
|
||||
change(&bad)
|
||||
if _, err = j.Authenticate(context.Background(), sign(bad, "first", jwt.SigningMethodEdDSA, private)); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, invalid := range []string{sign(claims, "missing", jwt.SigningMethodEdDSA, private), sign(claims, "first", jwt.SigningMethodHS256, []byte("invalid secret")), strings.Repeat("x", 8193)} {
|
||||
if _, err = j.Authenticate(context.Background(), invalid); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
if err = j.ReplaceKeys(map[string]JWTKey{"second": {Algorithm: "EdDSA", PublicKey: private.Public()}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), raw); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = j.Authenticate(context.Background(), sign(claims, "second", jwt.SigningMethodEdDSA, private)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
cancel()
|
||||
if _, err = j.Authenticate(ctx, raw); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func TestSessionsRevokeAndIsolation(t *testing.T) {
|
||||
mem, _ := cache.NewMemory(cache.MemoryConfig{})
|
||||
defer mem.Close()
|
||||
s, err := NewSessions(mem, "session:", time.Minute)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx := context.Background()
|
||||
token, err := s.Issue(ctx, Principal{UserID: "u1", TenantID: "t1", Roles: []string{"reader"}})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
key, _ := s.key(token)
|
||||
if strings.Contains(key, token) {
|
||||
t.Fatal("plaintext session secret stored")
|
||||
}
|
||||
p, err := s.Authenticate(ctx, token)
|
||||
if err != nil || p.UserID != "u1" {
|
||||
t.Fatal(p, err)
|
||||
}
|
||||
p.Roles[0] = "admin"
|
||||
p, _ = s.Authenticate(ctx, token)
|
||||
if p.HasRole("admin") {
|
||||
t.Fatal("principal alias")
|
||||
}
|
||||
if err = s.Revoke(ctx, token); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, token); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = s.Authenticate(ctx, "bad"); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -21,9 +21,10 @@ var (
|
||||
// Principal is the authenticated identity made available to business handlers.
|
||||
// Claims are optional provider-specific values and should not be required by core business code.
|
||||
type Principal struct {
|
||||
UserID string
|
||||
Roles []string
|
||||
Claims map[string]any
|
||||
UserID string `json:"userId"`
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
Claims map[string]any `json:"claims,omitempty"`
|
||||
}
|
||||
|
||||
func (p Principal) HasRole(role string) bool {
|
||||
|
||||
127
auth/jwt.go
Normal file
127
auth/jwt.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/ed25519"
|
||||
"crypto/rsa"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/golang-jwt/jwt/v5"
|
||||
)
|
||||
|
||||
// JWTKey binds a trusted key ID to exactly one asymmetric signing algorithm.
|
||||
// Keys come from server configuration, never a token's jku/x5u URL.
|
||||
type JWTKey struct {
|
||||
Algorithm string
|
||||
PublicKey any
|
||||
}
|
||||
type JWTConfig struct {
|
||||
Issuer string `json:"issuer"`
|
||||
Audience string `json:"audience"`
|
||||
Leeway time.Duration `json:"leeway"`
|
||||
Keys map[string]JWTKey `json:"-"`
|
||||
}
|
||||
type JWT struct {
|
||||
config JWTConfig
|
||||
mu sync.RWMutex
|
||||
keys map[string]JWTKey
|
||||
}
|
||||
type JWTClaims struct {
|
||||
jwt.RegisteredClaims
|
||||
TenantID string `json:"tenantId,omitempty"`
|
||||
Roles []string `json:"roles,omitempty"`
|
||||
}
|
||||
|
||||
func NewJWT(cfg JWTConfig) (*JWT, error) {
|
||||
if cfg.Issuer == "" || cfg.Audience == "" || cfg.Leeway < 0 || cfg.Leeway > 2*time.Minute {
|
||||
return nil, errors.New("auth: issuer/audience required, leeway must be 0..2m")
|
||||
}
|
||||
j := &JWT{config: cfg}
|
||||
j.config.Keys = nil
|
||||
if err := j.ReplaceKeys(cfg.Keys); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return j, nil
|
||||
}
|
||||
|
||||
// ReplaceKeys atomically rotates the trusted key set. Retain old keys until
|
||||
// their tokens expire if a rotation should preserve existing sessions.
|
||||
func (j *JWT) ReplaceKeys(keys map[string]JWTKey) error {
|
||||
if len(keys) == 0 || len(keys) > 64 {
|
||||
return errors.New("auth: require 1..64 JWT keys")
|
||||
}
|
||||
copyKeys := make(map[string]JWTKey, len(keys))
|
||||
for id, key := range keys {
|
||||
if id == "" || len(id) > 128 {
|
||||
return errors.New("auth: invalid key ID")
|
||||
}
|
||||
switch key.Algorithm {
|
||||
case "RS256":
|
||||
k, ok := key.PublicKey.(*rsa.PublicKey)
|
||||
if !ok || k == nil || k.N == nil || k.N.BitLen() < 2048 || k.E < 3 {
|
||||
return errors.New("auth: RS256 requires a >=2048-bit RSA public key")
|
||||
}
|
||||
case "EdDSA":
|
||||
k, ok := key.PublicKey.(ed25519.PublicKey)
|
||||
if !ok || len(k) != ed25519.PublicKeySize {
|
||||
return errors.New("auth: EdDSA requires an Ed25519 public key")
|
||||
}
|
||||
default:
|
||||
return errors.New("auth: supported algorithms are RS256 and EdDSA")
|
||||
}
|
||||
encoded, err := x509.MarshalPKIXPublicKey(key.PublicKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
key.PublicKey, err = x509.ParsePKIXPublicKey(encoded)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
copyKeys[id] = key
|
||||
}
|
||||
j.mu.Lock()
|
||||
j.keys = copyKeys
|
||||
j.mu.Unlock()
|
||||
return nil
|
||||
}
|
||||
|
||||
func (j *JWT) Authenticate(ctx context.Context, raw string) (Principal, error) {
|
||||
if ctx == nil {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
if len(raw) == 0 || len(raw) > 8192 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
claims := new(JWTClaims)
|
||||
token, err := jwt.ParseWithClaims(raw, claims, func(t *jwt.Token) (any, error) {
|
||||
id, ok := t.Header["kid"].(string)
|
||||
if !ok {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
j.mu.RLock()
|
||||
key, exists := j.keys[id]
|
||||
j.mu.RUnlock()
|
||||
if !exists || t.Method.Alg() != key.Algorithm {
|
||||
return nil, ErrInvalidToken
|
||||
}
|
||||
return key.PublicKey, nil
|
||||
}, jwt.WithValidMethods([]string{"RS256", "EdDSA"}), jwt.WithIssuer(j.config.Issuer), jwt.WithAudience(j.config.Audience), jwt.WithExpirationRequired(), jwt.WithIssuedAt(), jwt.WithLeeway(j.config.Leeway))
|
||||
if err != nil || !token.Valid || claims.Subject == "" || len(claims.Subject) > 256 || len(claims.TenantID) > 256 || len(claims.Roles) > 64 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
for _, role := range claims.Roles {
|
||||
if role == "" || len(role) > 128 {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
return Principal{UserID: claims.Subject, TenantID: claims.TenantID, Roles: append([]string(nil), claims.Roles...)}, nil
|
||||
}
|
||||
34
auth/policy.go
Normal file
34
auth/policy.go
Normal file
@@ -0,0 +1,34 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
)
|
||||
|
||||
type Policy interface {
|
||||
Authorize(context.Context, Principal, string, string) error
|
||||
}
|
||||
type PolicyFunc func(context.Context, Principal, string, string) error
|
||||
|
||||
func (f PolicyFunc) Authorize(ctx context.Context, p Principal, action, resource string) error {
|
||||
return f(ctx, p, action, resource)
|
||||
}
|
||||
func RequirePermission(policy Policy, action string, resource func(*httpx.Context) string) httpx.Middleware {
|
||||
return func(next httpx.Handler) httpx.Handler {
|
||||
return func(c *httpx.Context) error {
|
||||
p, ok := FromContext(c)
|
||||
if !ok {
|
||||
return apierrors.Unauthorized("UNAUTHENTICATED", "authentication required")
|
||||
}
|
||||
if policy == nil || action == "" || resource == nil {
|
||||
return apierrors.Internal(errors.New("auth: incomplete permission policy"))
|
||||
}
|
||||
if err := policy.Authorize(c.Context(), p, action, resource(c)); err != nil {
|
||||
return apierrors.Forbidden("FORBIDDEN", "permission denied").WithCause(err)
|
||||
}
|
||||
return next(c)
|
||||
}
|
||||
}
|
||||
}
|
||||
87
auth/session.go
Normal file
87
auth/session.go
Normal file
@@ -0,0 +1,87 @@
|
||||
package auth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
)
|
||||
|
||||
// Sessions stores server-issued opaque bearer credentials as SHA-256 keys.
|
||||
// The caller verifies login credentials before Issue; Store owns persistence.
|
||||
type Sessions struct {
|
||||
store cache.Store
|
||||
prefix string
|
||||
ttl time.Duration
|
||||
}
|
||||
|
||||
func NewSessions(store cache.Store, prefix string, ttl time.Duration) (*Sessions, error) {
|
||||
if store == nil || len(prefix) > 512 || ttl < time.Second || ttl > 30*24*time.Hour {
|
||||
return nil, errors.New("auth: invalid session configuration")
|
||||
}
|
||||
return &Sessions{store: store, prefix: prefix, ttl: ttl}, nil
|
||||
}
|
||||
func (s *Sessions) Issue(ctx context.Context, p Principal) (string, error) {
|
||||
if p.UserID == "" || len(p.UserID) > 256 || len(p.TenantID) > 256 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
data, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
if len(data) > 16384 {
|
||||
return "", errors.New("auth: session principal too large")
|
||||
}
|
||||
secret := make([]byte, 32)
|
||||
if _, err := rand.Read(secret); err != nil {
|
||||
return "", err
|
||||
}
|
||||
token := base64.RawURLEncoding.EncodeToString(secret)
|
||||
key, _ := s.key(token)
|
||||
if err := s.store.Set(ctx, key, data, s.ttl); err != nil {
|
||||
return "", err
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
func (s *Sessions) key(token string) (string, error) {
|
||||
if len(token) != 43 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
b, err := base64.RawURLEncoding.DecodeString(token)
|
||||
if err != nil || len(b) != 32 {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
sum := sha256.Sum256(b)
|
||||
return s.prefix + hex.EncodeToString(sum[:]), nil
|
||||
}
|
||||
func (s *Sessions) Authenticate(ctx context.Context, token string) (Principal, error) {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
data, err := s.store.Get(ctx, key)
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
if err != nil {
|
||||
return Principal{}, err
|
||||
}
|
||||
var p Principal
|
||||
if len(data) > 16384 || json.Unmarshal(data, &p) != nil || p.UserID == "" {
|
||||
return Principal{}, ErrInvalidToken
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
func (s *Sessions) Revoke(ctx context.Context, token string) error {
|
||||
key, err := s.key(token)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return s.store.Delete(ctx, key)
|
||||
}
|
||||
135
cache/cache.go
vendored
Normal file
135
cache/cache.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
// Package cache provides replaceable byte-oriented caches and bounded local
|
||||
// request coalescing. Storage errors are never silently treated as misses.
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrMiss = errors.New("cache: key not found")
|
||||
ErrClosed = errors.New("cache: closed")
|
||||
ErrTooLarge = errors.New("cache: entry exceeds capacity")
|
||||
ErrBusy = errors.New("cache: too many concurrent loads")
|
||||
)
|
||||
|
||||
// Store implementations copy values and accept ttl=0 for no expiration.
|
||||
// Negative TTLs are invalid. Delete is successful for missing keys.
|
||||
type Store interface {
|
||||
Get(context.Context, string) ([]byte, error)
|
||||
Set(context.Context, string, []byte, time.Duration) error
|
||||
Delete(context.Context, ...string) error
|
||||
}
|
||||
|
||||
func Check(ctx context.Context, key string, ttl time.Duration) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
if key == "" || len(key) > 1024 {
|
||||
return errors.New("cache: key must contain 1..1024 bytes")
|
||||
}
|
||||
if ttl < 0 {
|
||||
return errors.New("cache: negative TTL")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type LoaderConfig struct {
|
||||
MaxLoads int `json:"maxLoads"`
|
||||
Timeout time.Duration `json:"timeout"`
|
||||
}
|
||||
|
||||
type flight struct {
|
||||
done chan struct{}
|
||||
value []byte
|
||||
err error
|
||||
}
|
||||
|
||||
// Loader coalesces concurrent misses for the same key within this process.
|
||||
// Caller cancellation stops waiting without canceling other callers' load.
|
||||
// Load functions must observe their context, which has a bounded deadline.
|
||||
type Loader struct {
|
||||
store Store
|
||||
config LoaderConfig
|
||||
mu sync.Mutex
|
||||
flights map[string]*flight
|
||||
}
|
||||
|
||||
func NewLoader(store Store, cfg LoaderConfig) (*Loader, error) {
|
||||
if store == nil || cfg.MaxLoads < 0 || cfg.Timeout < 0 {
|
||||
return nil, errors.New("cache: invalid loader configuration")
|
||||
}
|
||||
if cfg.MaxLoads == 0 {
|
||||
cfg.MaxLoads = 128
|
||||
}
|
||||
if cfg.Timeout == 0 {
|
||||
cfg.Timeout = 5 * time.Second
|
||||
}
|
||||
return &Loader{store: store, config: cfg, flights: make(map[string]*flight)}, nil
|
||||
}
|
||||
|
||||
func (l *Loader) GetOrLoad(ctx context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error)) ([]byte, error) {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if load == nil {
|
||||
return nil, errors.New("cache: nil loader")
|
||||
}
|
||||
value, err := l.store.Get(ctx, key)
|
||||
if !errors.Is(err, ErrMiss) {
|
||||
return value, err
|
||||
}
|
||||
l.mu.Lock()
|
||||
f := l.flights[key]
|
||||
if f == nil {
|
||||
if len(l.flights) >= l.config.MaxLoads {
|
||||
l.mu.Unlock()
|
||||
return nil, ErrBusy
|
||||
}
|
||||
f = &flight{done: make(chan struct{})}
|
||||
l.flights[key] = f
|
||||
go l.run(context.WithoutCancel(ctx), key, ttl, load, f)
|
||||
}
|
||||
l.mu.Unlock()
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
case <-f.done:
|
||||
return append([]byte(nil), f.value...), f.err
|
||||
}
|
||||
}
|
||||
|
||||
func (l *Loader) run(parent context.Context, key string, ttl time.Duration, load func(context.Context) ([]byte, error), f *flight) {
|
||||
ctx, cancel := context.WithTimeout(parent, l.config.Timeout)
|
||||
defer cancel()
|
||||
defer func() {
|
||||
if value := recover(); value != nil {
|
||||
f.err = fmt.Errorf("cache: loader panic: %v", value)
|
||||
}
|
||||
l.mu.Lock()
|
||||
delete(l.flights, key)
|
||||
close(f.done)
|
||||
l.mu.Unlock()
|
||||
}()
|
||||
// Another request may have populated the key between the first Get and
|
||||
// flight acquisition. Recheck inside the coalesced load.
|
||||
f.value, f.err = l.store.Get(ctx, key)
|
||||
if !errors.Is(f.err, ErrMiss) {
|
||||
return
|
||||
}
|
||||
f.value, f.err = load(ctx)
|
||||
if f.err == nil {
|
||||
f.err = ctx.Err()
|
||||
}
|
||||
if f.err == nil {
|
||||
f.err = l.store.Set(ctx, key, f.value, ttl)
|
||||
}
|
||||
f.value = append([]byte(nil), f.value...)
|
||||
}
|
||||
118
cache/cache_test.go
vendored
Normal file
118
cache/cache_test.go
vendored
Normal file
@@ -0,0 +1,118 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestMemoryOwnershipCapacityExpiryAndClose(t *testing.T) {
|
||||
m, err := NewMemory(MemoryConfig{MaxEntries: 2, MaxBytes: 10})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now()
|
||||
m.now = func() time.Time { return now }
|
||||
ctx := context.Background()
|
||||
value := []byte("aa")
|
||||
if err := m.Set(ctx, "a", value, time.Second); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
value[0] = 'x'
|
||||
got, _ := m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("retained caller memory")
|
||||
}
|
||||
got[0] = 'z'
|
||||
got, _ = m.Get(ctx, "a")
|
||||
if string(got) != "aa" {
|
||||
t.Fatal("returned mutable cache memory")
|
||||
}
|
||||
m.Set(ctx, "b", []byte("bb"), 0)
|
||||
m.Get(ctx, "a")
|
||||
m.Set(ctx, "c", []byte("cc"), 0)
|
||||
if _, err := m.Get(ctx, "b"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("LRU eviction failed")
|
||||
}
|
||||
now = now.Add(time.Second)
|
||||
if _, err := m.Get(ctx, "a"); !errors.Is(err, ErrMiss) {
|
||||
t.Fatal("expiration boundary ignored")
|
||||
}
|
||||
if err := m.Set(ctx, "large", []byte("123456"), 0); !errors.Is(err, ErrTooLarge) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := m.Get(ctx, "c"); err != nil {
|
||||
t.Fatal("oversized write evicted valid data")
|
||||
}
|
||||
canceled, cancel := context.WithCancel(ctx)
|
||||
cancel()
|
||||
if err := m.Delete(canceled, "c"); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
m.Close()
|
||||
if _, err := m.Get(ctx, "c"); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderCoalescesAndSeparatesCallerCancellation(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
loader, _ := NewLoader(m, LoaderConfig{MaxLoads: 1, Timeout: time.Second})
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
started, release := make(chan struct{}), make(chan struct{})
|
||||
var calls atomic.Int32
|
||||
load := func(ctx context.Context) ([]byte, error) {
|
||||
calls.Add(1)
|
||||
close(started)
|
||||
select {
|
||||
case <-release:
|
||||
return []byte("value"), nil
|
||||
case <-ctx.Done():
|
||||
return nil, ctx.Err()
|
||||
}
|
||||
}
|
||||
first := make(chan error, 1)
|
||||
go func() { _, err := loader.GetOrLoad(ctx, "key", time.Minute, load); first <- err }()
|
||||
<-started
|
||||
second := make(chan error, 1)
|
||||
go func() {
|
||||
v, err := loader.GetOrLoad(context.Background(), "key", time.Minute, load)
|
||||
if err == nil && string(v) != "value" {
|
||||
err = errors.New("wrong value")
|
||||
}
|
||||
second <- err
|
||||
}()
|
||||
if _, err := loader.GetOrLoad(context.Background(), "other", 0, load); !errors.Is(err, ErrBusy) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cancel()
|
||||
if err := <-first; !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
close(release)
|
||||
if err := <-second; err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if calls.Load() != 1 {
|
||||
t.Fatalf("loads=%d", calls.Load())
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoaderPanicDeadlineAndStoreFailure(t *testing.T) {
|
||||
m, _ := NewMemory(MemoryConfig{})
|
||||
l, _ := NewLoader(m, LoaderConfig{Timeout: 20 * time.Millisecond})
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(context.Context) ([]byte, error) { panic("failure") }); err == nil {
|
||||
t.Fatal("panic escaped")
|
||||
}
|
||||
if _, err := l.GetOrLoad(context.Background(), "panic", 0, func(ctx context.Context) ([]byte, error) { <-ctx.Done(); return nil, ctx.Err() }); !errors.Is(err, context.DeadlineExceeded) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m.Close()
|
||||
if _, err := l.GetOrLoad(context.Background(), "closed", 0, func(context.Context) ([]byte, error) { t.Fatal("store outage triggered load"); return nil, nil }); !errors.Is(err, ErrClosed) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
135
cache/memory.go
vendored
Normal file
135
cache/memory.go
vendored
Normal file
@@ -0,0 +1,135 @@
|
||||
package cache
|
||||
|
||||
import (
|
||||
"container/list"
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type MemoryConfig struct {
|
||||
MaxEntries int `json:"maxEntries"`
|
||||
MaxBytes int `json:"maxBytes"` // Sum of key and value bytes, excluding Go object overhead.
|
||||
}
|
||||
|
||||
type entry struct {
|
||||
key string
|
||||
value []byte
|
||||
expires time.Time
|
||||
}
|
||||
|
||||
// Memory is an LRU cache with lazy expiration and no background goroutines.
|
||||
type Memory struct {
|
||||
mu sync.Mutex
|
||||
config MemoryConfig
|
||||
items map[string]*list.Element
|
||||
lru *list.List
|
||||
bytes int
|
||||
closed bool
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewMemory(cfg MemoryConfig) (*Memory, error) {
|
||||
if cfg.MaxEntries < 0 || cfg.MaxBytes < 0 {
|
||||
return nil, errors.New("cache: negative capacity")
|
||||
}
|
||||
if cfg.MaxEntries == 0 {
|
||||
cfg.MaxEntries = 4096
|
||||
}
|
||||
if cfg.MaxBytes == 0 {
|
||||
cfg.MaxBytes = 32 << 20
|
||||
}
|
||||
return &Memory{config: cfg, items: make(map[string]*list.Element), lru: list.New(), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (m *Memory) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return nil, ErrClosed
|
||||
}
|
||||
e := m.items[key]
|
||||
if e == nil {
|
||||
return nil, ErrMiss
|
||||
}
|
||||
item := e.Value.(*entry)
|
||||
if !item.expires.IsZero() && !m.now().Before(item.expires) {
|
||||
m.remove(e)
|
||||
return nil, ErrMiss
|
||||
}
|
||||
m.lru.MoveToFront(e)
|
||||
return append([]byte(nil), item.value...), nil
|
||||
}
|
||||
|
||||
func (m *Memory) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(value) > m.config.MaxBytes-len(key) {
|
||||
return ErrTooLarge
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
for len(m.items) >= m.config.MaxEntries || m.bytes+len(key)+len(value) > m.config.MaxBytes {
|
||||
m.remove(m.lru.Back())
|
||||
}
|
||||
item := &entry{key: key, value: append([]byte(nil), value...)}
|
||||
if ttl > 0 {
|
||||
item.expires = m.now().Add(ttl)
|
||||
}
|
||||
m.items[key] = m.lru.PushFront(item)
|
||||
m.bytes += len(key) + len(value)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, key := range keys {
|
||||
if err := Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if m.closed {
|
||||
return ErrClosed
|
||||
}
|
||||
for _, key := range keys {
|
||||
if e := m.items[key]; e != nil {
|
||||
m.remove(e)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Memory) remove(e *list.Element) {
|
||||
item := e.Value.(*entry)
|
||||
m.bytes -= len(item.key) + len(item.value)
|
||||
delete(m.items, item.key)
|
||||
m.lru.Remove(e)
|
||||
}
|
||||
|
||||
func (m *Memory) Close() error {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.closed = true
|
||||
clear(m.items)
|
||||
m.lru.Init()
|
||||
m.bytes = 0
|
||||
return nil
|
||||
}
|
||||
68
cache/redis/redis.go
vendored
Normal file
68
cache/redis/redis.go
vendored
Normal file
@@ -0,0 +1,68 @@
|
||||
// Package redis adapts go-redis's pooled client to cache.Store.
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
// Store borrows the client; its owner must close the client after users stop.
|
||||
type Store struct {
|
||||
client goredis.Cmdable
|
||||
prefix string
|
||||
}
|
||||
|
||||
func New(client goredis.Cmdable, prefix string) (*Store, error) {
|
||||
if client == nil || len(prefix) > 1024 || strings.ContainsAny(prefix, "\r\n") {
|
||||
return nil, errors.New("cache/redis: invalid client or prefix")
|
||||
}
|
||||
return &Store{client: client, prefix: prefix}, nil
|
||||
}
|
||||
|
||||
func (s *Store) Get(ctx context.Context, key string) ([]byte, error) {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
value, err := s.client.Get(ctx, s.prefix+key).Bytes()
|
||||
if errors.Is(err, goredis.Nil) {
|
||||
return nil, cache.ErrMiss
|
||||
}
|
||||
return value, err
|
||||
}
|
||||
|
||||
func (s *Store) Set(ctx context.Context, key string, value []byte, ttl time.Duration) error {
|
||||
if err := cache.Check(ctx, key, ttl); err != nil {
|
||||
return err
|
||||
}
|
||||
// Redis expiration precision is milliseconds; round up, never truncate a
|
||||
// positive duration to an unbounded lifetime.
|
||||
if ttl > 0 && ttl < time.Millisecond {
|
||||
ttl = time.Millisecond
|
||||
}
|
||||
return s.client.Set(ctx, s.prefix+key, value, ttl).Err()
|
||||
}
|
||||
|
||||
func (s *Store) Delete(ctx context.Context, keys ...string) error {
|
||||
if ctx == nil {
|
||||
return errors.New("cache/redis: nil context")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
names := make([]string, len(keys))
|
||||
for i, key := range keys {
|
||||
if err := cache.Check(ctx, key, 0); err != nil {
|
||||
return err
|
||||
}
|
||||
names[i] = s.prefix + key
|
||||
}
|
||||
if len(names) == 0 {
|
||||
return nil
|
||||
}
|
||||
return s.client.Del(ctx, names...).Err()
|
||||
}
|
||||
125
cache/redis/redis_test.go
vendored
Normal file
125
cache/redis/redis_test.go
vendored
Normal file
@@ -0,0 +1,125 @@
|
||||
package redis
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
goredis "github.com/redis/go-redis/v9"
|
||||
)
|
||||
|
||||
type fakeClient struct {
|
||||
goredis.Cmdable
|
||||
key string
|
||||
ttl time.Duration
|
||||
failure error
|
||||
}
|
||||
|
||||
func (f *fakeClient) Get(ctx context.Context, key string) *goredis.StringCmd {
|
||||
f.key = key
|
||||
return goredis.NewStringResult("", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Set(ctx context.Context, key string, value interface{}, ttl time.Duration) *goredis.StatusCmd {
|
||||
f.key, f.ttl = key, ttl
|
||||
return goredis.NewStatusResult("OK", f.failure)
|
||||
}
|
||||
func (f *fakeClient) Del(ctx context.Context, keys ...string) *goredis.IntCmd {
|
||||
f.key = keys[0]
|
||||
return goredis.NewIntResult(1, f.failure)
|
||||
}
|
||||
|
||||
func TestAdapterPreservesErrorsAndExpiration(t *testing.T) {
|
||||
f := &fakeClient{failure: goredis.Nil}
|
||||
store, _ := New(f, "app:")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, cache.ErrMiss) || f.key != "app:key" {
|
||||
t.Fatal(err, f.key)
|
||||
}
|
||||
f.failure = errors.New("unavailable")
|
||||
if _, err := store.Get(context.Background(), "key"); !errors.Is(err, f.failure) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f.failure = nil
|
||||
if err := store.Set(context.Background(), "key", []byte("value"), time.Nanosecond); err != nil || f.ttl != time.Millisecond {
|
||||
t.Fatal(err, f.ttl)
|
||||
}
|
||||
if err := store.Set(context.Background(), "key", nil, -time.Second); err == nil {
|
||||
t.Fatal("negative TTL accepted")
|
||||
}
|
||||
if err := store.Delete(context.Background(), "key"); err != nil || f.key != "app:key" {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisIntegration(t *testing.T) {
|
||||
address := os.Getenv("GO_OMNI_REDIS_TEST_ADDR")
|
||||
if address == "" {
|
||||
t.Skip("set GO_OMNI_REDIS_TEST_ADDR to an isolated test Redis")
|
||||
}
|
||||
client := goredis.NewClient(&goredis.Options{Addr: address, PoolSize: 2, MaxRetries: -1, ContextTimeoutEnabled: true, DialTimeout: time.Second, ReadTimeout: time.Second, WriteTimeout: time.Second})
|
||||
defer client.Close()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
if err := client.Ping(ctx).Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
prefix := fmt.Sprintf("goOmniTest:%d:", time.Now().UnixNano())
|
||||
store, _ := New(client, prefix)
|
||||
defer store.Delete(context.Background(), "key", "forever", "list", "canceled")
|
||||
if err := store.Set(ctx, "key", []byte{0, 1, 255}, 80*time.Millisecond); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
v, err := store.Get(ctx, "key")
|
||||
if err != nil || len(v) != 3 || v[2] != 255 {
|
||||
t.Fatal(v, err)
|
||||
}
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for {
|
||||
_, err := store.Get(ctx, "key")
|
||||
if errors.Is(err, cache.ErrMiss) {
|
||||
break
|
||||
}
|
||||
if err != nil || time.Now().After(deadline) {
|
||||
t.Fatal("TTL failed", err)
|
||||
}
|
||||
time.Sleep(5 * time.Millisecond)
|
||||
}
|
||||
if err := store.Set(ctx, "forever", []byte("ok"), 0); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if ttl := client.PTTL(ctx, prefix+"forever").Val(); ttl != -1 {
|
||||
t.Fatalf("unexpected persistent TTL %v", ttl)
|
||||
}
|
||||
if err := store.Delete(ctx, "forever"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "forever"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.RPush(ctx, prefix+"list", "value").Err(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "list"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("Redis error treated as miss", err)
|
||||
}
|
||||
canceled, stop := context.WithCancel(ctx)
|
||||
stop()
|
||||
if err := store.Set(canceled, "canceled", []byte("value"), 0); !errors.Is(err, context.Canceled) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "canceled"); !errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("canceled write reached Redis", err)
|
||||
}
|
||||
if err := store.Delete(ctx, "list"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.Close(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := store.Get(ctx, "key"); err == nil || errors.Is(err, cache.ErrMiss) {
|
||||
t.Fatal("closed client treated as miss", err)
|
||||
}
|
||||
}
|
||||
@@ -1,9 +1,79 @@
|
||||
# go-omni CLI
|
||||
|
||||
## API 契约驱动项目
|
||||
|
||||
```sh
|
||||
# 创建包含可运行问候接口的新项目
|
||||
go run ./cmd/go-omni api new --module example.com/demo --framework-dir . ../demo
|
||||
|
||||
# 从已有定义生成对应目录、类型、路由和业务骨架
|
||||
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
|
||||
--dir ../userService --module example.com/userService --framework-dir .
|
||||
|
||||
# 校验定义;也接受 goctl 风格的 api go -api ... -dir ...
|
||||
go run ./cmd/go-omni api validate --api examples/contracts/user.api
|
||||
```
|
||||
|
||||
`new --api <file>` 是契约项目的另一个入口。可以组合 `--database mysql --storage local --log-output file`,生成配置、依赖初始化与资源关闭。API 默认无数据库、无对象存储;`--database memory` 等同于不打开连接。`--force` 不允许覆盖 API 项目。初始配置选择写入 manifest,重新生成自动沿用。
|
||||
|
||||
后续在生成项目内执行 `go-omni api generate --api api/user.api --dir .`。
|
||||
`--check` 只读检查生成产物是否过期。生成器记录所有权,保留 Service / Repository / 配置 / main 的手写内容,遇到生成文件被手动修改则报冲突。新 Service 与中间件占位实现返回 501。
|
||||
|
||||
支持范围、参数语义、目录及安全更新策略见 [API 开发指南](../../API.md)。本版借鉴 go-zero 的工作流,不宣称兼容全部 goctl 语法;仍通过 Omni/httpx 保留网络层替换能力。
|
||||
|
||||
## 日志与对象存储按需装配
|
||||
|
||||
新增 `--log-output stdout|stderr|file`、`--log-format json|text`、`--storage none|local|s3|r2|oss`。
|
||||
指定任意一个新选项后,生成严格的配置文件、初始化代码和项目自身的测试;不指定则保留原有轻量骨架。
|
||||
对象存储独立于 `--database memory|mysql`。只有选择文件存储才会生成文件 Handler / Service 和对应适配器。
|
||||
|
||||
开发阶段可以使用本地源码;固定远程版本的安装和验收见 [发布说明](../../RELEASE.md):
|
||||
|
||||
```sh
|
||||
# 在 go-omni 仓库中运行;目标目录须为空。
|
||||
go run ./cmd/go-omni new --framework-dir . \
|
||||
--module example.com/fileService --log-output file --log-format json \
|
||||
--storage local ../fileService
|
||||
```
|
||||
|
||||
生成项目包含 `etc/config.yaml`、`internal/settings`、`internal/servicecontext/storage.go`、
|
||||
文件 Service/Handler、测试和 `MODULES.md`。按生成的 MODULES.md 配置示例 Token 和本地签名密钥后,
|
||||
`go mod tidy && go run ./cmd/server` 即可体验。云存储分别选择 `--storage s3`、`r2`、`oss`,
|
||||
再设置已有桶/凭据等环境变量;不自动创建桶或修改云资源。
|
||||
|
||||
配置优先级为默认值 → YAML/JSON → 环境变量;密钥仅允许环境变量。Service 依赖统一存储接口,
|
||||
组合处负责创建客户端与有界连接池,退出时关闭自己创建的资源。文件接口提供服务端上传、直传签名、
|
||||
临时下载链接及统一错误处理;日志示例携带 requestId,文件输出默认带轮转。没有固定业务身份上下文。
|
||||
|
||||
新模板要求 `v0.7.0-preview` 或更新版本;明确指定旧版本并启用新模块会被拒绝,
|
||||
避免模板引用旧框架不存在的接口。本地 `--framework-dir` 验证不受此限制。`--force` 可重新生成兼容配置,
|
||||
但如果切换配置会遗留引用已移除依赖的旧文件,则拒绝操作,要求使用新目录;不会自动删除修改过的代码。
|
||||
|
||||
验证当前生成器:
|
||||
|
||||
```sh
|
||||
GO_OMNI_SCAFFOLD_INTEGRATION=1 go test -race ./cmd/go-omni/...
|
||||
python3 scripts/verifyScaffoldModules.py
|
||||
```
|
||||
|
||||
前者覆盖两种数据库与五种文件存储选择、日志方式/格式,以及生成项目的配置、资源关闭和接口测试。
|
||||
后者实际编译启动服务,验证日志、上传/下载/直传链接、错误状态、nethttp/native 下的 SIGINT/SIGTERM 和监听端口释放。
|
||||
云端选项的测试使用本地替身,不代表 OSS/R2 实网验收;本批不访问业务服务器。报告见
|
||||
[脚手架模块验收](../../performance/scaffoldModulesLocal.json)。
|
||||
|
||||
## 其他命令与远程安装
|
||||
|
||||
main 分支还提供 `go-omni add orderItems --dir /path/to/project`,向已有 GoOmni 项目增加
|
||||
Handler / Service / Repository / Model 与模块注册入口。模块名为 lowerCamelCase,
|
||||
目标目录存在时拒绝覆盖,已有启动代码保持原样。生成模块带可替换的内存 CRUD Repository,
|
||||
导入后使用 `orderItems.Register(application, repository, authMiddleware)` 注册。
|
||||
该命令从 `v0.7.0-preview` 提供;源码可用 `go build -o /tmp/go-omni ./cmd/go-omni` 构建。
|
||||
边界和验证方式见 [业务基础模块](../../MODULES.md)。
|
||||
|
||||
从远程安装,生成一个带 Handler / Service / Repository / Model 分层的服务骨架:
|
||||
|
||||
```sh
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.6.0-preview
|
||||
go install git.yuntuo.hk/k2safe/go-omni/cmd/go-omni@v0.7.0-preview
|
||||
go-omni new --module example.com/billing ./billing
|
||||
cd billing
|
||||
go mod tidy
|
||||
@@ -65,3 +135,20 @@ python3 scripts/verifyDatabase.py --cli /tmp/go-omni --framework-dir .
|
||||
```
|
||||
|
||||
数据库验收需要本地 Docker,使用独立 MySQL 8.4 容器;不会连接已有业务数据库,结束后清理测试容器与数据卷。
|
||||
|
||||
## 显式迁移(main)
|
||||
|
||||
SQL 命名为 `0001_createUsers.up.sql`。先预览版本和 SHA-256 清单:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations
|
||||
```
|
||||
|
||||
部署步骤配置 `GO_OMNI_DB_DSN`,显式执行:
|
||||
|
||||
```sh
|
||||
go-omni migrate --dir migrations --apply
|
||||
```
|
||||
|
||||
预览不连接数据库;应用启动不自动迁移。失败保留 dirty 状态,不提供自动 down/force。
|
||||
脚手架原有 `schema.sql` 不会自动变成有版本的迁移,已有库需要先确定基线。详见 [扩展模块](../../ADVANCED.md)。
|
||||
|
||||
127
cmd/go-omni/api.go
Normal file
127
cmd/go-omni/api.go
Normal file
@@ -0,0 +1,127 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||
)
|
||||
|
||||
func runAPI(args []string) error {
|
||||
if len(args) == 0 || args[0] == "help" || args[0] == "--help" || args[0] == "-h" {
|
||||
usage()
|
||||
return nil
|
||||
}
|
||||
command := args[0]
|
||||
if command == "go" {
|
||||
command = "generate"
|
||||
}
|
||||
if command == "check" {
|
||||
command = "validate"
|
||||
}
|
||||
if command != "new" && command != "generate" && command != "validate" {
|
||||
return fmt.Errorf("unknown api command %q", command)
|
||||
}
|
||||
o := apigen.Options{}
|
||||
for i := 1; i < len(args); i++ {
|
||||
arg := args[i]
|
||||
if arg == "--help" || arg == "-h" {
|
||||
usage()
|
||||
return nil
|
||||
}
|
||||
if arg == "--check" {
|
||||
if command != "generate" {
|
||||
return errors.New("--check belongs to api generate")
|
||||
}
|
||||
o.Check = true
|
||||
continue
|
||||
}
|
||||
if !strings.HasPrefix(arg, "-") {
|
||||
if command == "new" && o.Dir == "" {
|
||||
o.Dir = arg
|
||||
continue
|
||||
}
|
||||
return fmt.Errorf("unexpected argument %q", arg)
|
||||
}
|
||||
name, value, inline := strings.Cut(arg, "=")
|
||||
// goctl-style -api/-dir aliases ease migration; long forms are canonical.
|
||||
if name == "-api" {
|
||||
name = "--api"
|
||||
}
|
||||
if name == "-dir" {
|
||||
name = "--dir"
|
||||
}
|
||||
switch name {
|
||||
case "--api", "--dir", "--module", "--framework-dir", "--framework-version", "--log-output", "--log-format", "--database", "--storage":
|
||||
default:
|
||||
return fmt.Errorf("unknown api option %q", name)
|
||||
}
|
||||
if !inline {
|
||||
if i+1 >= len(args) || strings.HasPrefix(args[i+1], "-") {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
i++
|
||||
value = args[i]
|
||||
}
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
if command == "validate" && name != "--api" {
|
||||
return fmt.Errorf("api validate does not accept %s", name)
|
||||
}
|
||||
switch name {
|
||||
case "--api":
|
||||
o.API = value
|
||||
case "--dir":
|
||||
o.Dir = value
|
||||
case "--module":
|
||||
o.Module = value
|
||||
case "--framework-dir":
|
||||
o.FrameworkDir = value
|
||||
case "--framework-version":
|
||||
o.FrameworkVersion = value
|
||||
case "--database":
|
||||
o.Database = value
|
||||
case "--storage":
|
||||
o.Storage = value
|
||||
case "--log-output":
|
||||
o.LogOutput = value
|
||||
case "--log-format":
|
||||
o.LogFormat = value
|
||||
}
|
||||
}
|
||||
if command == "validate" {
|
||||
if o.API == "" {
|
||||
return errors.New("--api is required")
|
||||
}
|
||||
spec, err := apigen.Parse(o.API)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err = apigen.Compile(spec); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("valid API: %d types, %d routes\n", len(spec.Types), len(spec.Routes))
|
||||
return nil
|
||||
}
|
||||
var out apigen.Output
|
||||
var err error
|
||||
if command == "new" {
|
||||
out, err = apigen.New(o)
|
||||
} else {
|
||||
if o.Dir == "" {
|
||||
o.Dir = "."
|
||||
}
|
||||
out, err = apigen.Generate(o)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if o.Check {
|
||||
fmt.Println("API generated files are current")
|
||||
} else {
|
||||
fmt.Printf("API project ready: %d generated, %d created, %d preserved, %d removed\n", len(out.Generated), len(out.Created), len(out.Preserved), len(out.Removed))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
44
cmd/go-omni/api_test.go
Normal file
44
cmd/go-omni/api_test.go
Normal file
@@ -0,0 +1,44 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestAPICommands(t *testing.T) {
|
||||
root, err := filepath.Abs("../..")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
project := filepath.Join(t.TempDir(), "demo")
|
||||
if err = runAPI([]string{"new", project, "--module=example.com/demo", "--framework-dir", root}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
api := filepath.Join(project, "api/service.api")
|
||||
if err = runAPI([]string{"validate", "-api", api}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runAPI([]string{"go", "-api", api, "-dir", project, "--check"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runNew([]string{"--force", project}); err == nil {
|
||||
t.Fatal("CRUD starter overwrote API project")
|
||||
}
|
||||
target := filepath.Join(t.TempDir(), "other")
|
||||
if err = runNew([]string{"--api", api, "--module", "example.com/other", "--framework-dir", root, target}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(target, "internal/service/greet/greet.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, args := range [][]string{{"validate"}, {"generate", "--api"}, {"new", "--api=x", "somewhere"}, {"generate", "--dir="}, {"generate", "--force"}, {"validate", "--api", api, "--dir", project}, {"validate", "--check"}} {
|
||||
if err = runAPI(args); err == nil {
|
||||
t.Fatalf("invalid args accepted: %v", args)
|
||||
}
|
||||
}
|
||||
if err = runNew([]string{"--api", api, "--force", target}); err == nil || !strings.Contains(err.Error(), "does not accept --force") {
|
||||
t.Fatalf("incompatible profile: %v", err)
|
||||
}
|
||||
}
|
||||
360
cmd/go-omni/apigen/apigen_test.go
Normal file
360
cmd/go-omni/apigen/apigen_test.go
Normal file
@@ -0,0 +1,360 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func frameworkRoot(t *testing.T) string {
|
||||
t.Helper()
|
||||
p, err := filepath.Abs("../../..")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return p
|
||||
}
|
||||
func writeFile(t *testing.T, name, body string) {
|
||||
t.Helper()
|
||||
if err := os.MkdirAll(filepath.Dir(name), 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(name, []byte(body), 0644); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
func readFile(t *testing.T, name string) string {
|
||||
t.Helper()
|
||||
b, err := os.ReadFile(name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return string(b)
|
||||
}
|
||||
func parseText(t *testing.T, source string) (*Spec, error) {
|
||||
t.Helper()
|
||||
file := filepath.Join(t.TempDir(), "service.api")
|
||||
writeFile(t, file, source)
|
||||
return Parse(file)
|
||||
}
|
||||
|
||||
func TestCompileExample(t *testing.T) {
|
||||
s, err := Parse(filepath.Join(frameworkRoot(t), "examples/contracts/user.api"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
c, err := Compile(s)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(c.Groups) != 2 || len(s.Routes) != 3 || len(s.Sources) != 2 {
|
||||
t.Fatalf("wrong spec: %+v", s)
|
||||
}
|
||||
var doc map[string]any
|
||||
if err = json.Unmarshal(c.Document, &doc); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
route := doc["paths"].(map[string]any)["/api/v1/users/{userId}"].(map[string]any)["put"].(map[string]any)
|
||||
if len(route["parameters"].([]any)) != 3 {
|
||||
t.Fatal("missing mixed parameters")
|
||||
}
|
||||
schema := route["requestBody"].(map[string]any)["content"].(map[string]any)["application/json"].(map[string]any)["schema"].(map[string]any)
|
||||
if schema["additionalProperties"] != false || len(schema["required"].([]any)) != 2 {
|
||||
t.Fatalf("wrong body schema: %+v", schema)
|
||||
}
|
||||
if strings.Contains(string(c.Document), `"user_id"`) {
|
||||
t.Fatal("snake case wire name")
|
||||
}
|
||||
}
|
||||
func TestInvalidDefinitions(t *testing.T) {
|
||||
cases := []string{
|
||||
`syntax="v2"`,
|
||||
`@server ( jwt: Auth ) service user { @handler go get /go }`,
|
||||
"type Request { Name string `json:\"name\" binding:\"typo\"` }\nservice user { @handler run\n post /go (Request) }",
|
||||
"type A { Other *A `json:\"other,optional\"` }\nservice user { @handler run\n post /go (A) }",
|
||||
"type A { Other Missing `json:\"other\"` }\nservice user { @handler run\n post /go (A) }",
|
||||
"type A { UserID int `json:\"user_id\"` }\nservice user { @handler run\n post /go (A) }",
|
||||
"service user { @handler go\n get /go\n @handler other\n get /go\n}",
|
||||
"service user { @handler go\n get /:id\n}",
|
||||
"service user { @handler service\n get /go\n}",
|
||||
"type A { ID int `path:\"id\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||
"type A { Name string `json:\"name\"` }\nservice user { @handler go\n get /go (A)\n}",
|
||||
"service user { @handler go\n @status 201\n post /go\n}",
|
||||
"type A { Name string `header:\"Name\"` }\nservice user { @handler go\n get /go returns (A)\n}",
|
||||
"@server (group: ../../bad)\nservice user { @handler go\n get /go\n}",
|
||||
"service user { @handler go\n get /health/ready\n}",
|
||||
"type A { Name string `json:\"name\"`\n Name int `json:\"other\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||
"service user { @handler go\n get /go\n @handler Go\n get /other\n}",
|
||||
"type A { Value []byte `json:\"value\"` }\nservice user { @handler go\n post /go (A)\n}",
|
||||
}
|
||||
for i, source := range cases {
|
||||
s, err := parseText(t, source)
|
||||
if err == nil {
|
||||
_, err = Compile(s)
|
||||
}
|
||||
if err == nil {
|
||||
t.Fatalf("case %d accepted: %s", i, source)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestImports(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
entry := filepath.Join(root, "api", "entry.api")
|
||||
writeFile(t, entry, "import \"types.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||
writeFile(t, filepath.Join(root, "api/types.api"), `import "entry.api"`)
|
||||
if _, err := Parse(entry); err == nil || !strings.Contains(err.Error(), "circular") {
|
||||
t.Fatalf("cycle: %v", err)
|
||||
}
|
||||
writeFile(t, entry, "import \"../outside.api\"\nservice user { @handler ping\n get /ping\n}")
|
||||
writeFile(t, filepath.Join(root, "outside.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||
if _, err := Parse(entry); err == nil {
|
||||
t.Fatal("escaped import root")
|
||||
}
|
||||
writeFile(t, entry, "import (\n \"shared.api\"\n \"./shared.api\"\n)\nservice user { @handler ping\n get /ping\n}")
|
||||
writeFile(t, filepath.Join(root, "api/shared.api"), `type X { Value string `+"`json:\"value\"`"+` }`)
|
||||
s, err := Parse(entry)
|
||||
if err != nil || len(s.Types) != 1 {
|
||||
t.Fatalf("duplicate import: %+v %v", s, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSharedPrefixConflict(t *testing.T) {
|
||||
source := "type A { ID int `path:\"id\"` }\ntype B { Name string `path:\"name\"` }\nservice user {\n @handler first\n get /users/:id (A)\n @handler second\n get /users/:name/items (B)\n}"
|
||||
s, err := parseText(t, source)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = Compile(s); err == nil {
|
||||
t.Fatal("shared prefix conflict was accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func FuzzParser(f *testing.F) {
|
||||
for _, s := range []string{starterAPI, "syntax = \"v1\"", "type (", "@server (prefix: /api)"} {
|
||||
f.Add(s)
|
||||
}
|
||||
f.Fuzz(func(t *testing.T, source string) {
|
||||
if len(source) > 16384 {
|
||||
t.Skip()
|
||||
}
|
||||
tokens, err := lex("fuzz.api", source)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
spec := &Spec{Title: "Fuzz", Version: "1"}
|
||||
p := parser{tokens: tokens, spec: spec}
|
||||
if p.parse() == nil {
|
||||
_, _ = Compile(spec)
|
||||
}
|
||||
})
|
||||
}
|
||||
func TestRegenerationAndConflicts(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "app")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o := Options{Dir: dir, API: filepath.Join(dir, "api/service.api")}
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
manifestBefore := readFile(t, filepath.Join(dir, manifestPath))
|
||||
typesBefore := readFile(t, filepath.Join(dir, "internal/types/typesGen.go"))
|
||||
o.Check = true
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
servicePath := filepath.Join(dir, "internal/service/greet/greet.go")
|
||||
service := readFile(t, servicePath) + "\n// user edit must survive\n"
|
||||
writeFile(t, servicePath, service)
|
||||
for _, file := range []string{"cmd/server/main.go", "etc/config.yaml", "internal/servicecontext/context.go", "internal/repository/greet/repository.go"} {
|
||||
body := readFile(t, filepath.Join(dir, file))
|
||||
if strings.HasSuffix(file, ".go") {
|
||||
body += "\n// handwritten extension\n"
|
||||
} else {
|
||||
body += "\n# handwritten extension\n"
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, file), body)
|
||||
}
|
||||
api := readFile(t, o.API)
|
||||
api += "\n@server (group: extra)\nservice greet-api {\n @handler ping\n get /ping\n}\n"
|
||||
writeFile(t, o.API, api)
|
||||
if _, err := Generate(o); err == nil {
|
||||
t.Fatal("check accepted stale output")
|
||||
}
|
||||
if readFile(t, filepath.Join(dir, manifestPath)) != manifestBefore {
|
||||
t.Fatal("--check wrote manifest")
|
||||
}
|
||||
o.Check = false
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if readFile(t, servicePath) != service {
|
||||
t.Fatal("business code overwritten")
|
||||
}
|
||||
// Retiring a group removes only owned handlers, keeping its business files.
|
||||
writeFile(t, o.API, strings.Split(api, "\n@server (group: extra)")[0])
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/handler/extra/handlersGen.go")); !os.IsNotExist(err) {
|
||||
t.Fatal("stale handler remains")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(dir, "internal/service/extra/ping.go")); err != nil {
|
||||
t.Fatal("business stub removed")
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), typesBefore+"\n// edited generated file\n")
|
||||
before := snapshot(t, dir)
|
||||
if _, err := Generate(o); err == nil || !strings.Contains(err.Error(), "was edited") {
|
||||
t.Fatalf("did not protect edited output: %v", err)
|
||||
}
|
||||
after := snapshot(t, dir)
|
||||
if !sameFiles(before, after) {
|
||||
t.Fatal("failed regeneration changed files")
|
||||
}
|
||||
}
|
||||
func TestOutputBoundaries(t *testing.T) {
|
||||
for _, kind := range []string{"symlink", "unowned", "lock", "caseCollision"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
dir := filepath.Join(root, "app")
|
||||
api := filepath.Join(root, "input/service.api")
|
||||
writeFile(t, api, starterAPI)
|
||||
switch kind {
|
||||
case "symlink":
|
||||
if err := os.MkdirAll(dir, 0755); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
outside := filepath.Join(root, "outside")
|
||||
os.MkdirAll(outside, 0755)
|
||||
if err := os.Symlink(outside, filepath.Join(dir, "internal")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
case "unowned":
|
||||
writeFile(t, filepath.Join(dir, "internal/types/typesGen.go"), "package types // mine\n")
|
||||
case "lock":
|
||||
writeFile(t, filepath.Join(dir, ".goOmni/api.lock"), "someone else")
|
||||
case "caseCollision":
|
||||
writeFile(t, api, starterAPI+"\n@server (group: gReet)\nservice greet-api {\n @handler extra\n get /extra\n}\n")
|
||||
}
|
||||
before := snapshot(t, root)
|
||||
if _, err := Generate(Options{Dir: dir, API: api, Module: "example.com/test", FrameworkDir: frameworkRoot(t)}); err == nil {
|
||||
t.Fatal("unsafe output accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, root)) {
|
||||
t.Fatal("conflict changed files")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func snapshot(t *testing.T, root string) map[string]string {
|
||||
t.Helper()
|
||||
out := map[string]string{}
|
||||
err := filepath.WalkDir(root, func(path string, d os.DirEntry, err error) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if d.Type().IsRegular() {
|
||||
data, e := os.ReadFile(path)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
rel, _ := filepath.Rel(root, path)
|
||||
out[rel] = string(data)
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return out
|
||||
}
|
||||
func sameFiles(a, b map[string]string) bool {
|
||||
if len(a) != len(b) {
|
||||
return false
|
||||
}
|
||||
for k, v := range a {
|
||||
if b[k] != v {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
func TestGeneratedProjectsBuildAndRun(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("consumer module integration")
|
||||
}
|
||||
root := frameworkRoot(t)
|
||||
for _, starter := range []bool{true, false} {
|
||||
name := "contract"
|
||||
if starter {
|
||||
name = "starter"
|
||||
}
|
||||
t.Run(name, func(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
o := Options{Dir: dir, Module: "example.com/consumer", FrameworkDir: root, LogOutput: "file", LogFormat: "text"}
|
||||
var err error
|
||||
if starter {
|
||||
_, err = New(o)
|
||||
} else {
|
||||
o.API = filepath.Join(root, "examples/contracts/user.api")
|
||||
_, err = Generate(o)
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
source := consumerContractTest
|
||||
if starter {
|
||||
source = consumerStarterTest
|
||||
} else {
|
||||
writeFile(t, filepath.Join(dir, "internal/service/user/updateUser.go"), `package user
|
||||
import("context";"example.com/consumer/internal/types")
|
||||
func(s *Service) UpdateUser(ctx context.Context,req *types.UpdateUserReq)(*types.UserResp,error){return &types.UserResp{UserID:req.UserID,Name:req.Name,Active:req.Active},nil}
|
||||
`)
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "contract_test.go"), source)
|
||||
o.API = filepath.Join(dir, "api/service.api")
|
||||
if !starter {
|
||||
o.API = filepath.Join(dir, "api/user.api")
|
||||
}
|
||||
if _, err = Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||
output, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("consumer: %v\n%s", err, output)
|
||||
}
|
||||
t.Log(string(output))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const consumerStarterTest = `package consumer_test
|
||||
import("context";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext";"example.com/consumer/internal/settings")
|
||||
func TestGreeting(t *testing.T){cfg,err:=settings.Load("etc/config.yaml");if err!=nil{t.Fatal(err)};cfg.Server.Log.Writer=io.Discard;cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{};logger,err:=logx.New(cfg.Server.Log);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)};w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,httptest.NewRequest("GET","/api/greet/Alice",nil));if w.Code!=200||!strings.Contains(w.Body.String(),"Hello Alice")||w.Header().Get("X-Request-ID")==""{t.Fatalf("%d %s",w.Code,w.Body.String())}}
|
||||
`
|
||||
const consumerContractTest = `package consumer_test
|
||||
import("context";"encoding/json";"io";"net/http/httptest";"strings";"testing";omni "git.yuntuo.hk/k2safe/go-omni";"git.yuntuo.hk/k2safe/go-omni/logx";"example.com/consumer/internal/handler";"example.com/consumer/internal/servicecontext")
|
||||
func TestContract(t *testing.T){cfg:=logx.Default();cfg.Writer=io.Discard;logger,err:=logx.New(cfg);if err!=nil{t.Fatal(err)};defer logger.Close();deps,err:=servicecontext.New(context.Background(),logger);if err!=nil{t.Fatal(err)};defer deps.Close();app:=omni.New();defer app.Close();if err=handler.RegisterRoutes(app,deps);err!=nil{t.Fatal(err)}
|
||||
for _,tc:=range []struct{method,path,body string;status int}{
|
||||
{"PUT","/api/v1/users/7?notify=true",` + "`" + `{"name":"Alice","active":false,"age":0}` + "`" + `,200},
|
||||
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice"}` + "`" + `,400},
|
||||
{"PUT","/api/v1/users/7?notify=true¬ify=false",` + "`" + `{"name":"Alice","active":false}` + "`" + `,400},
|
||||
{"PUT","/api/v1/users/7",` + "`" + `{"name":"Alice","active":false,"age":151}` + "`" + `,400},
|
||||
{"GET","/api/v1/users?page=0","",400},
|
||||
{"GET","/api/v1/users?page=1","",501},
|
||||
{"DELETE","/admin/v1/users/7","",501},
|
||||
}{req:=httptest.NewRequest(tc.method,tc.path,strings.NewReader(tc.body));req.Header.Set("Content-Type","application/json");w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,req);if w.Code!=tc.status{t.Fatalf("%s: %d %s",tc.path,w.Code,w.Body.String())};if tc.status==200{var v struct{Data struct{UserID int64 ` + "`json:\"userId\"`" + `;Name string ` + "`json:\"name\"`" + `}};if err:=json.Unmarshal(w.Body.Bytes(),&v);err!=nil||v.Data.UserID!=7||v.Data.Name!="Alice"{t.Fatal(w.Body.String())}}}
|
||||
delete(deps.Middlewares,"Auth");if err:=handler.RegisterRoutes(omni.New(),deps);err==nil{t.Fatal("missing auth hook accepted")}
|
||||
}
|
||||
`
|
||||
332
cmd/go-omni/apigen/compile.go
Normal file
332
cmd/go-omni/apigen/compile.go
Normal file
@@ -0,0 +1,332 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/ast"
|
||||
goparser "go/parser"
|
||||
gotoken "go/token"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/contract"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
)
|
||||
|
||||
type Compiled struct {
|
||||
Spec *Spec
|
||||
Bindings map[string]*contract.Binder
|
||||
Document []byte
|
||||
Groups []string
|
||||
}
|
||||
|
||||
var exportedName = regexp.MustCompile(`^[A-Z][A-Za-z0-9]*$`)
|
||||
var goName = regexp.MustCompile(`^[a-zA-Z][A-Za-z0-9]*$`)
|
||||
var lowerName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||
var serviceName = regexp.MustCompile(`^[a-zA-Z][a-zA-Z0-9-]*$`)
|
||||
var staticSegment = regexp.MustCompile(`^[A-Za-z0-9._~-]+$`)
|
||||
|
||||
func Compile(spec *Spec) (*Compiled, error) {
|
||||
if spec == nil || len(spec.Routes) == 0 || !serviceName.MatchString(spec.Service) {
|
||||
return nil, errors.New("api: valid service and routes required")
|
||||
}
|
||||
definitions := map[string]Type{}
|
||||
for _, t := range spec.Types {
|
||||
if !exportedName.MatchString(t.Name) {
|
||||
return nil, fmt.Errorf("%s: type names must be UpperCamelCase", t.Position)
|
||||
}
|
||||
if _, ok := definitions[t.Name]; ok {
|
||||
return nil, fmt.Errorf("%s: duplicate type %s", t.Position, t.Name)
|
||||
}
|
||||
definitions[t.Name] = t
|
||||
}
|
||||
resolved := map[string]reflect.Type{}
|
||||
resolving := map[string]bool{}
|
||||
var resolve func(string, int) (reflect.Type, error)
|
||||
var expression func(ast.Expr, int) (reflect.Type, error)
|
||||
primitives := map[string]reflect.Type{"string": reflect.TypeOf(""), "bool": reflect.TypeOf(false), "int": reflect.TypeOf(int(0)), "int8": reflect.TypeOf(int8(0)), "int16": reflect.TypeOf(int16(0)), "int32": reflect.TypeOf(int32(0)), "int64": reflect.TypeOf(int64(0)), "uint": reflect.TypeOf(uint(0)), "uint8": reflect.TypeOf(uint8(0)), "uint16": reflect.TypeOf(uint16(0)), "uint32": reflect.TypeOf(uint32(0)), "uint64": reflect.TypeOf(uint64(0)), "float32": reflect.TypeOf(float32(0)), "float64": reflect.TypeOf(float64(0))}
|
||||
expression = func(e ast.Expr, depth int) (reflect.Type, error) {
|
||||
if depth > 32 {
|
||||
return nil, errors.New("type nesting exceeds 32")
|
||||
}
|
||||
switch e := e.(type) {
|
||||
case *ast.Ident:
|
||||
if t, ok := primitives[e.Name]; ok {
|
||||
return t, nil
|
||||
}
|
||||
return resolve(e.Name, depth+1)
|
||||
case *ast.StarExpr:
|
||||
if _, double := e.X.(*ast.StarExpr); double {
|
||||
return nil, errors.New("multiple pointer levels unsupported")
|
||||
}
|
||||
t, err := expression(e.X, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reflect.PointerTo(t), nil
|
||||
case *ast.ArrayType:
|
||||
if e.Len != nil {
|
||||
return nil, errors.New("use slices, not fixed arrays")
|
||||
}
|
||||
t, err := expression(e.Elt, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reflect.SliceOf(t), nil
|
||||
case *ast.MapType:
|
||||
key, ok := e.Key.(*ast.Ident)
|
||||
if !ok || key.Name != "string" {
|
||||
return nil, errors.New("map keys must be string")
|
||||
}
|
||||
t, err := expression(e.Value, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return reflect.MapOf(reflect.TypeOf(""), t), nil
|
||||
default:
|
||||
return nil, errors.New("unsupported type expression")
|
||||
}
|
||||
}
|
||||
resolve = func(name string, depth int) (reflect.Type, error) {
|
||||
if depth > 32 {
|
||||
return nil, errors.New("type nesting exceeds 32")
|
||||
}
|
||||
if t, ok := resolved[name]; ok {
|
||||
return t, nil
|
||||
}
|
||||
if resolving[name] {
|
||||
return nil, fmt.Errorf("recursive type %s is unsupported", name)
|
||||
}
|
||||
typ, ok := definitions[name]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown type %s", name)
|
||||
}
|
||||
resolving[name] = true
|
||||
defer delete(resolving, name)
|
||||
var fields []reflect.StructField
|
||||
seen := map[string]bool{}
|
||||
for _, f := range typ.Fields {
|
||||
if !exportedName.MatchString(f.Name) || seen[f.Name] {
|
||||
return nil, fmt.Errorf("%s: invalid or duplicate field %s", name, f.Name)
|
||||
}
|
||||
seen[f.Name] = true
|
||||
expr, err := goparser.ParseExpr(f.Type)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
t, err := expression(expr, depth+1)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s.%s: %w", name, f.Name, err)
|
||||
}
|
||||
fields = append(fields, reflect.StructField{Name: f.Name, Type: t, Tag: reflect.StructTag(f.Tag)})
|
||||
}
|
||||
t := reflect.StructOf(fields)
|
||||
resolved[name] = t
|
||||
return t, nil
|
||||
}
|
||||
c := &Compiled{Spec: spec, Bindings: map[string]*contract.Binder{}}
|
||||
for _, t := range spec.Types {
|
||||
rt, err := resolve(t.Name, 0)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
b, err := contract.ForType(rt)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", t.Position, err)
|
||||
}
|
||||
c.Bindings[t.Name] = b
|
||||
}
|
||||
api, err := openapi.New(spec.Title, spec.Version)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
handlers, paths, shapes, groups := map[string]bool{}, map[string]bool{}, map[string]string{}, map[string]bool{}
|
||||
middlewareSymbols := map[string]string{}
|
||||
for _, r := range spec.Routes {
|
||||
if !goName.MatchString(r.Handler) {
|
||||
return nil, fmt.Errorf("%s: invalid handler name", r.Position)
|
||||
}
|
||||
if strings.EqualFold(r.Handler, "service") {
|
||||
return nil, errors.New("handler name service conflicts with the service constructor file")
|
||||
}
|
||||
if r.Group == "" {
|
||||
return nil, errors.New("group required")
|
||||
}
|
||||
for _, part := range strings.Split(r.Group, "/") {
|
||||
if !lowerName.MatchString(part) || gotoken.Lookup(part).IsKeyword() {
|
||||
return nil, fmt.Errorf("invalid lowerCamelCase group %q", r.Group)
|
||||
}
|
||||
}
|
||||
h := r.Group + "/" + strings.ToLower(r.Handler)
|
||||
if handlers[h] {
|
||||
return nil, fmt.Errorf("duplicate handler %s in group %s", r.Handler, r.Group)
|
||||
}
|
||||
handlers[h] = true
|
||||
groups[r.Group] = true
|
||||
mseen := map[string]bool{}
|
||||
for _, m := range r.Middleware {
|
||||
if !goName.MatchString(m) || mseen[m] {
|
||||
return nil, fmt.Errorf("invalid or duplicate middleware %q", m)
|
||||
}
|
||||
mseen[m] = true
|
||||
if strings.EqualFold(m, "defaults") || strings.EqualFold(m, "registryGen") {
|
||||
return nil, fmt.Errorf("reserved middleware name %q", m)
|
||||
}
|
||||
if prior, ok := middlewareSymbols[strings.ToLower(m)]; ok && prior != m {
|
||||
return nil, fmt.Errorf("middleware names %s and %s collide", prior, m)
|
||||
}
|
||||
middlewareSymbols[strings.ToLower(m)] = m
|
||||
}
|
||||
shape, err := routeShape(r.Path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||
}
|
||||
for _, prior := range spec.Routes {
|
||||
if prior.Method != r.Method || prior.Path == r.Path {
|
||||
continue
|
||||
}
|
||||
if conflictPrefix(prior.Path, r.Path) {
|
||||
return nil, fmt.Errorf("conflicting parameter names at a shared route prefix: %s and %s", prior.Path, r.Path)
|
||||
}
|
||||
}
|
||||
if prior, exists := shapes[shape]; exists && prior != r.Path {
|
||||
return nil, fmt.Errorf("ambiguous route parameter names: %s and %s", prior, r.Path)
|
||||
}
|
||||
shapes[shape] = r.Path
|
||||
pathKey := r.Method + " " + shape
|
||||
if paths[pathKey] {
|
||||
return nil, fmt.Errorf("duplicate route %s", pathKey)
|
||||
}
|
||||
paths[pathKey] = true
|
||||
if r.Path == "/health/live" || r.Path == "/health/ready" || r.Path == "/openapi.json" {
|
||||
return nil, fmt.Errorf("route %s is reserved by the generated server", r.Path)
|
||||
}
|
||||
if r.Method != strings.ToLower(r.Method) {
|
||||
return nil, errors.New("HTTP methods must be lowercase")
|
||||
}
|
||||
if r.Status != 200 && r.Status != 201 && r.Status != 202 && r.Status != 204 {
|
||||
return nil, errors.New("@status supports 200, 201, 202 or 204")
|
||||
}
|
||||
if (r.Status == 204) != (r.Response == "") {
|
||||
return nil, errors.New("204 requires no response; other success statuses require a response type")
|
||||
}
|
||||
if r.Method == "head" && r.Response != "" {
|
||||
return nil, errors.New("HEAD must not declare a response body")
|
||||
}
|
||||
op := openapi.Operation{Method: r.Method, Path: r.Path, OperationID: operationID(r), Summary: r.Summary, Responses: map[int]openapi.Response{}}
|
||||
if r.Request != "" {
|
||||
b, ok := c.Bindings[r.Request]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown request %s", r.Request)
|
||||
}
|
||||
op.Parameters = b.Parameters()
|
||||
op.Request = b.BodySchema()
|
||||
if op.Request != nil && (r.Method == "get" || r.Method == "head") {
|
||||
return nil, errors.New("GET/HEAD requests cannot declare JSON bodies; use form/path/header tags")
|
||||
}
|
||||
}
|
||||
var response *openapi.Schema
|
||||
if r.Response != "" {
|
||||
b, ok := c.Bindings[r.Response]
|
||||
if !ok {
|
||||
return nil, fmt.Errorf("unknown response %s", r.Response)
|
||||
}
|
||||
s, err := b.ResponseSchema()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
response = openapi.Envelope(s)
|
||||
response.Required = append(response.Required, "data")
|
||||
}
|
||||
op.Responses[r.Status] = openapi.Response{Description: "Success", Schema: response}
|
||||
errorSchema := &openapi.Schema{Type: "object", Required: []string{"code", "message"}, Properties: map[string]*openapi.Schema{"code": {Type: "string"}, "message": {Type: "string"}, "requestId": {Type: "string"}}}
|
||||
for _, code := range []int{400, 413, 415, 500, 501} {
|
||||
op.Responses[code] = openapi.Response{Description: "Error", Schema: errorSchema}
|
||||
}
|
||||
if err = api.Handle(discardRoutes{}, op, func(*httpx.Context) error { return nil }); err != nil {
|
||||
return nil, fmt.Errorf("%s: %w", r.Position, err)
|
||||
}
|
||||
}
|
||||
for group := range groups {
|
||||
c.Groups = append(c.Groups, group)
|
||||
}
|
||||
sort.Strings(c.Groups)
|
||||
c.Document, err = api.JSON()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Show named hooks in documentation without claiming they implement a
|
||||
// particular authentication scheme. Applications own that policy.
|
||||
var doc map[string]any
|
||||
decoder := json.NewDecoder(bytes.NewReader(c.Document))
|
||||
decoder.UseNumber()
|
||||
if err = decoder.Decode(&doc); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
for _, r := range spec.Routes {
|
||||
if len(r.Middleware) > 0 {
|
||||
path := r.Path
|
||||
for _, part := range strings.Split(r.Path, "/") {
|
||||
if strings.HasPrefix(part, ":") {
|
||||
path = strings.ReplaceAll(path, part, "{"+part[1:]+"}")
|
||||
}
|
||||
}
|
||||
doc["paths"].(map[string]any)[path].(map[string]any)[r.Method].(map[string]any)["x-omni-middleware"] = r.Middleware
|
||||
}
|
||||
}
|
||||
c.Document, err = json.MarshalIndent(doc, "", " ")
|
||||
return c, err
|
||||
}
|
||||
|
||||
type discardRoutes struct{}
|
||||
|
||||
func (discardRoutes) Handle(string, string, httpx.Handler, ...httpx.Middleware) {}
|
||||
func operationID(r Route) string {
|
||||
return strings.ReplaceAll(r.Group, "/", "_") + "_" + upper(r.Handler)
|
||||
}
|
||||
func upper(s string) string { return strings.ToUpper(s[:1]) + s[1:] }
|
||||
func lower(s string) string { return strings.ToLower(s[:1]) + s[1:] }
|
||||
func routeShape(path string) (string, error) {
|
||||
if path == "/" {
|
||||
return path, nil
|
||||
}
|
||||
if !strings.HasPrefix(path, "/") {
|
||||
return "", errors.New("route must start with /")
|
||||
}
|
||||
parts := strings.Split(path[1:], "/")
|
||||
seen := map[string]bool{}
|
||||
for i, part := range parts {
|
||||
if strings.HasPrefix(part, ":") {
|
||||
name := part[1:]
|
||||
if !lowerName.MatchString(name) || seen[name] {
|
||||
return "", errors.New("invalid/duplicate path parameter")
|
||||
}
|
||||
seen[name] = true
|
||||
parts[i] = ":"
|
||||
} else if !staticSegment.MatchString(part) || part == "." || part == ".." {
|
||||
return "", errors.New("invalid route segment")
|
||||
}
|
||||
}
|
||||
return "/" + strings.Join(parts, "/"), nil
|
||||
}
|
||||
|
||||
func conflictPrefix(left, right string) bool {
|
||||
a, b := strings.Split(left, "/"), strings.Split(right, "/")
|
||||
for i := 0; i < len(a) && i < len(b); i++ {
|
||||
if strings.HasPrefix(a[i], ":") && strings.HasPrefix(b[i], ":") {
|
||||
if a[i] != b[i] {
|
||||
return true
|
||||
}
|
||||
continue
|
||||
}
|
||||
if a[i] != b[i] {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
458
cmd/go-omni/apigen/generate.go
Normal file
458
cmd/go-omni/apigen/generate.go
Normal file
@@ -0,0 +1,458 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"runtime/debug"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||
"golang.org/x/mod/modfile"
|
||||
"golang.org/x/mod/module"
|
||||
"golang.org/x/mod/semver"
|
||||
)
|
||||
|
||||
const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
const minimumVersion = "v0.7.0-preview"
|
||||
const generatedHeader = "// Code generated by go-omni api. DO NOT EDIT.\n"
|
||||
|
||||
type Options struct {
|
||||
Database, Storage string
|
||||
API, Dir, Module, FrameworkDir, FrameworkVersion, LogOutput, LogFormat string
|
||||
Check bool
|
||||
}
|
||||
type Output struct{ Generated, Created, Preserved, Removed []string }
|
||||
|
||||
// Generate creates or safely updates one contract-driven project. It renders
|
||||
// and validates every output before applying an ownership-checked transaction.
|
||||
func Generate(o Options) (Output, error) {
|
||||
if o.API == "" || o.Dir == "" {
|
||||
return Output{}, errors.New("--api and --dir are required")
|
||||
}
|
||||
spec, err := Parse(o.API)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
compiled, err := Compile(spec)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
return generate(o, compiled, false)
|
||||
}
|
||||
|
||||
const starterAPI = "syntax = \"v1\"\n\ninfo (\n title: \"Greeting API\"\n version: \"1.0.0\"\n)\n\ntype GreetReq {\n Name string `path:\"name\" binding:\"required,max=80\"`\n}\ntype GreetResp {\n Message string `json:\"message\"`\n}\n\n@server (\n group: greet\n prefix: /api\n)\nservice greet-api {\n @doc \"Greet a visitor\"\n @handler greet\n get /greet/:name (GreetReq) returns (GreetResp)\n}\n"
|
||||
|
||||
// New creates a runnable greeting project, including its editable API file.
|
||||
func New(o Options) (Output, error) {
|
||||
if o.API != "" || o.Check {
|
||||
return Output{}, errors.New("api new does not accept --api or --check")
|
||||
}
|
||||
if o.Dir == "" {
|
||||
return Output{}, errors.New("project directory required")
|
||||
}
|
||||
if entries, err := os.ReadDir(o.Dir); err == nil && len(entries) > 0 {
|
||||
return Output{}, errors.New("api new requires an empty directory")
|
||||
} else if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return Output{}, err
|
||||
}
|
||||
tokens, err := lex("service.api", starterAPI)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
spec := &Spec{Version: "1.0.0", Entry: "service.api", Sources: map[string][]byte{"service.api": []byte(starterAPI)}}
|
||||
p := parser{tokens: tokens, spec: spec}
|
||||
if err = p.parse(); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
c, err := Compile(spec)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
return generate(o, c, true)
|
||||
}
|
||||
|
||||
func generate(o Options, c *Compiled, starter bool) (Output, error) {
|
||||
directory, err := filepath.Abs(o.Dir)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
o.Dir = directory
|
||||
if err := resolveProfile(&o); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
seeds := map[string][]byte{}
|
||||
moduleData, readErr := os.ReadFile(filepath.Join(directory, "go.mod"))
|
||||
if readErr == nil {
|
||||
mf, err := modfile.Parse("go.mod", moduleData, nil)
|
||||
if err != nil || mf.Module == nil {
|
||||
return Output{}, errors.New("invalid project go.mod")
|
||||
}
|
||||
if o.Module != "" && o.Module != mf.Module.Mod.Path {
|
||||
return Output{}, errors.New("--module differs from existing go.mod")
|
||||
}
|
||||
o.Module = mf.Module.Mod.Path
|
||||
found := false
|
||||
requiredVersion := ""
|
||||
for _, r := range mf.Require {
|
||||
if r.Mod.Path == frameworkModule {
|
||||
found = true
|
||||
requiredVersion = r.Mod.Version
|
||||
if o.FrameworkVersion != "" && o.FrameworkVersion != r.Mod.Version {
|
||||
return Output{}, errors.New("existing framework version differs; update go.mod explicitly")
|
||||
}
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return Output{}, errors.New("existing go.mod must require go-omni")
|
||||
}
|
||||
if semver.Compare(requiredVersion, minimumVersion) < 0 {
|
||||
localCurrent := false
|
||||
for _, r := range mf.Replace {
|
||||
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||
path := r.New.Path
|
||||
if !filepath.IsAbs(path) {
|
||||
path = filepath.Join(directory, path)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(path, "contract", "contract.go")); err == nil {
|
||||
localCurrent = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !localCurrent {
|
||||
return Output{}, fmt.Errorf("existing framework dependency predates API support; use %s or a current local replacement", minimumVersion)
|
||||
}
|
||||
}
|
||||
if o.FrameworkDir != "" {
|
||||
matched := false
|
||||
for _, r := range mf.Replace {
|
||||
if r.Old.Path == frameworkModule && r.New.Version == "" {
|
||||
replacement := r.New.Path
|
||||
if !filepath.IsAbs(replacement) {
|
||||
replacement = filepath.Join(directory, replacement)
|
||||
}
|
||||
matched = canonical(replacement) == canonical(o.FrameworkDir)
|
||||
}
|
||||
}
|
||||
if !matched {
|
||||
return Output{}, errors.New("existing framework replacement differs; update go.mod explicitly")
|
||||
}
|
||||
}
|
||||
} else if !errors.Is(readErr, os.ErrNotExist) {
|
||||
return Output{}, readErr
|
||||
} else {
|
||||
if o.Module == "" {
|
||||
o.Module = filepath.Base(directory)
|
||||
}
|
||||
data, err := moduleFile(o)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
seeds["go.mod"] = data
|
||||
}
|
||||
if err = module.CheckImportPath(o.Module); err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
generated, base, err := renderProject(o, c, starter)
|
||||
if err != nil {
|
||||
return Output{}, err
|
||||
}
|
||||
for name, data := range base {
|
||||
seeds[name] = data
|
||||
}
|
||||
// The API source remains developer-owned. A relocated project regenerates
|
||||
// from its own api/ tree, including all imports.
|
||||
entry := "api/" + c.Spec.Entry
|
||||
for name, data := range c.Spec.Sources {
|
||||
seeds["api/"+name] = data
|
||||
}
|
||||
if o.API != "" {
|
||||
source, _ := filepath.Abs(o.API)
|
||||
sourceReal, _ := filepath.EvalSymlinks(source)
|
||||
targetReal := canonical(directory)
|
||||
rel, e := filepath.Rel(targetReal, sourceReal)
|
||||
if e == nil && rel != ".." && !strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
entry = filepath.ToSlash(rel)
|
||||
for name := range c.Spec.Sources {
|
||||
delete(seeds, "api/"+name)
|
||||
}
|
||||
}
|
||||
}
|
||||
// Source collisions must never leave a copied contract different from the
|
||||
// generated code. On updates, use the developer-owned API in the project.
|
||||
for name, data := range seeds {
|
||||
if strings.HasSuffix(name, ".api") {
|
||||
current, e := os.ReadFile(filepath.Join(directory, filepath.FromSlash(name)))
|
||||
if e == nil && string(current) != string(data) {
|
||||
return Output{}, fmt.Errorf("%s already differs; regenerate from that project API instead", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
seeds["README.md"] = []byte(projectREADME(o.Module, entry) + infrastructureREADME(o))
|
||||
for _, files := range []map[string][]byte{generated, seeds} {
|
||||
for name, data := range files {
|
||||
if strings.HasSuffix(name, ".go") {
|
||||
data, err = format.Source(data)
|
||||
if err != nil {
|
||||
return Output{}, fmt.Errorf("format %s: %w", name, err)
|
||||
}
|
||||
files[name] = data
|
||||
}
|
||||
}
|
||||
}
|
||||
return apply(directory, o.Module, entry, generated, seeds, o.Check, o.profile())
|
||||
}
|
||||
|
||||
func moduleFile(o Options) ([]byte, error) {
|
||||
if err := module.CheckImportPath(o.Module); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
root := o.FrameworkDir
|
||||
version := o.FrameworkVersion
|
||||
if root == "" && version == "" {
|
||||
wd, _ := os.Getwd()
|
||||
for wd != "" {
|
||||
data, err := os.ReadFile(filepath.Join(wd, "go.mod"))
|
||||
if err == nil {
|
||||
mf, e := modfile.Parse("go.mod", data, nil)
|
||||
if e == nil && mf.Module != nil && mf.Module.Mod.Path == frameworkModule {
|
||||
root = wd
|
||||
}
|
||||
break
|
||||
}
|
||||
parent := filepath.Dir(wd)
|
||||
if parent == wd {
|
||||
break
|
||||
}
|
||||
wd = parent
|
||||
}
|
||||
}
|
||||
if version == "" {
|
||||
if info, ok := debug.ReadBuildInfo(); ok && info.Main.Path == frameworkModule && info.Main.Sum != "" {
|
||||
version = info.Main.Version
|
||||
} else {
|
||||
version = minimumVersion
|
||||
}
|
||||
}
|
||||
if !semver.IsValid(version) || semver.Major(version) != "v0" && semver.Major(version) != "v1" {
|
||||
return nil, errors.New("invalid framework version")
|
||||
}
|
||||
if root == "" && semver.Compare(version, minimumVersion) < 0 {
|
||||
return nil, fmt.Errorf("API generation needs %s or newer; use --framework-dir for unpublished source", minimumVersion)
|
||||
}
|
||||
text := "module " + o.Module + "\n\ngo 1.22.0\n\nrequire " + frameworkModule + " " + version + "\n"
|
||||
if root != "" {
|
||||
root = canonical(root)
|
||||
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
mf, err := modfile.Parse("go.mod", data, nil)
|
||||
if err != nil || mf.Module == nil || mf.Module.Mod.Path != frameworkModule {
|
||||
return nil, errors.New("--framework-dir must contain go-omni")
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(root, "contract", "contract.go")); err != nil {
|
||||
return nil, errors.New("local framework lacks API contract support")
|
||||
}
|
||||
relative, err := filepath.Rel(canonical(o.Dir), root)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
text += "\nreplace " + frameworkModule + " => " + strconv.Quote(filepath.ToSlash(relative)) + "\n"
|
||||
}
|
||||
return []byte(text), nil
|
||||
}
|
||||
func canonical(path string) string {
|
||||
path, _ = filepath.Abs(path)
|
||||
candidate := path
|
||||
var tail []string
|
||||
for {
|
||||
if resolved, err := filepath.EvalSymlinks(candidate); err == nil {
|
||||
for i := len(tail) - 1; i >= 0; i-- {
|
||||
resolved = filepath.Join(resolved, tail[i])
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
parent := filepath.Dir(candidate)
|
||||
if parent == candidate {
|
||||
return path
|
||||
}
|
||||
tail = append(tail, filepath.Base(candidate))
|
||||
candidate = parent
|
||||
}
|
||||
}
|
||||
|
||||
func renderProject(o Options, c *Compiled, starter bool) (map[string][]byte, map[string][]byte, error) {
|
||||
generated, seeds := map[string][]byte{}, map[string][]byte{}
|
||||
put := func(files map[string][]byte, name, content string) { files[name] = []byte(content) }
|
||||
types := "package types\n\n"
|
||||
for _, t := range c.Spec.Types {
|
||||
types += "type " + t.Name + " struct {\n"
|
||||
for _, f := range t.Fields {
|
||||
types += fmt.Sprintf("%s %s `%s`\n", f.Name, f.Type, f.Tag)
|
||||
}
|
||||
types += "}\n"
|
||||
}
|
||||
put(generated, "internal/types/typesGen.go", generatedHeader+types)
|
||||
put(generated, "docs/openapi.json", string(c.Document)+"\n")
|
||||
put(generated, "internal/handler/openapiGen.go", generatedHeader+fmt.Sprintf("package handler\nimport (\"net/http\"; %q)\nconst OpenAPIDocument = %q\nfunc OpenAPI(c *httpx.Context) error { return c.WriteResponse(200,http.Header{\"Content-Type\": {\"application/json; charset=utf-8\"}},[]byte(OpenAPIDocument)) }\n", frameworkModule+"/httpx", string(c.Document)+"\n"))
|
||||
imports := fmt.Sprintf("import (\"fmt\"; %q; %q; %q;", frameworkModule+"/httpx", frameworkModule+"/openapi", o.Module+"/internal/servicecontext")
|
||||
routes := "func RegisterRoutes(router openapi.Registrar, deps *servicecontext.Context) error {\n if router==nil || deps==nil {return fmt.Errorf(\"router and dependencies required\")}\n"
|
||||
middlewareSet := map[string]bool{}
|
||||
for _, r := range c.Spec.Routes {
|
||||
for _, m := range r.Middleware {
|
||||
middlewareSet[m] = true
|
||||
}
|
||||
}
|
||||
var middlewareNames []string
|
||||
for m := range middlewareSet {
|
||||
middlewareNames = append(middlewareNames, m)
|
||||
}
|
||||
sort.Strings(middlewareNames)
|
||||
for _, m := range middlewareNames {
|
||||
routes += fmt.Sprintf("if deps.Middlewares[%q]==nil {return fmt.Errorf(%q)}\n", m, "missing middleware "+m)
|
||||
}
|
||||
for i, group := range c.Groups {
|
||||
imports += fmt.Sprintf("h%d %q; s%d %q;", i, o.Module+"/internal/handler/"+group, i, o.Module+"/internal/service/"+group)
|
||||
routes += fmt.Sprintf("h%d := h%d.New(s%d.New(deps))\n", i, i, i)
|
||||
groupRoutes := []Route{}
|
||||
for _, r := range c.Spec.Routes {
|
||||
if r.Group == group {
|
||||
groupRoutes = append(groupRoutes, r)
|
||||
}
|
||||
}
|
||||
packageName := filepath.Base(group)
|
||||
text := "package " + packageName + "\nimport(\"context\"; \"fmt\";"
|
||||
needsTypes, needsBinder := false, false
|
||||
for _, r := range groupRoutes {
|
||||
needsTypes = needsTypes || r.Request != "" || r.Response != ""
|
||||
needsBinder = needsBinder || r.Request != ""
|
||||
}
|
||||
text += fmt.Sprintf("%q;", frameworkModule+"/httpx")
|
||||
if needsTypes {
|
||||
text += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||
}
|
||||
if needsBinder {
|
||||
text += fmt.Sprintf("%q;", frameworkModule+"/contract")
|
||||
}
|
||||
text += ")\n"
|
||||
text += "type Service interface {\n"
|
||||
for _, r := range groupRoutes {
|
||||
text += signature(r) + "\n"
|
||||
}
|
||||
text += "}\ntype Handler struct {service Service}\nfunc New(s Service)*Handler {return &Handler{service:s}}\n"
|
||||
for _, r := range groupRoutes {
|
||||
name := upper(r.Handler)
|
||||
if r.Request != "" {
|
||||
text += fmt.Sprintf("var %sRequest = contract.MustNew[types.%s]()\n", lower(r.Handler), r.Request)
|
||||
}
|
||||
text += fmt.Sprintf("func(h *Handler) %s(c *httpx.Context)error {\n", name)
|
||||
call := "c.Context()"
|
||||
if r.Request != "" {
|
||||
text += fmt.Sprintf("var req types.%s\nif err:=%sRequest.Bind(c.Request(),httpx.RouteParams(c.Context()),&req);err!=nil{return err}\n", r.Request, lower(r.Handler))
|
||||
call += ", &req"
|
||||
}
|
||||
if r.Response != "" {
|
||||
text += fmt.Sprintf("resp,err:=h.service.%s(%s);if err!=nil{return err};if resp==nil{return fmt.Errorf(\"%s returned a nil response\")}\n", name, call, name)
|
||||
method := map[int]string{200: "OK", 201: "Created", 202: "Accepted"}[r.Status]
|
||||
text += "return c." + method + "(resp)\n}\n"
|
||||
} else {
|
||||
text += fmt.Sprintf("if err:=h.service.%s(%s);err!=nil{return err};return c.NoContent()\n}\n", name, call)
|
||||
}
|
||||
routes += fmt.Sprintf("router.Handle(%q,%q,h%d.%s", strings.ToUpper(r.Method), r.Path, i, name)
|
||||
for _, m := range r.Middleware {
|
||||
routes += fmt.Sprintf(",deps.Middlewares[%q]", m)
|
||||
}
|
||||
routes += ")\n"
|
||||
service := fmt.Sprintf("package %s\nimport(\"context\"; apierrors %q;", packageName, frameworkModule+"/errors")
|
||||
if r.Request != "" || r.Response != "" {
|
||||
service += fmt.Sprintf("%q;", o.Module+"/internal/types")
|
||||
}
|
||||
service += ")\n"
|
||||
service += "// " + name + " implements the business operation. This file is never overwritten.\nfunc(s *Service) " + signature(r) + " {\n"
|
||||
if starter && r.Handler == "greet" {
|
||||
service = strings.Replace(service, "apierrors "+strconv.Quote(frameworkModule+"/errors")+";", "\"fmt\";", 1)
|
||||
service += "s.deps.Logger.InfoContext(ctx,\"greet.called\")\nreturn &types.GreetResp{Message:fmt.Sprintf(\"Hello %s\",req.Name)},nil\n}"
|
||||
} else {
|
||||
service += "// TODO: implement business behavior and use s.deps for dependencies.\n"
|
||||
if r.Response != "" {
|
||||
service += "return nil, "
|
||||
} else {
|
||||
service += "return "
|
||||
}
|
||||
service += "apierrors.New(501,\"NOT_IMPLEMENTED\",\"operation is not implemented\")\n}"
|
||||
}
|
||||
put(seeds, "internal/service/"+group+"/"+lower(r.Handler)+".go", service)
|
||||
}
|
||||
if !hasResponse(groupRoutes) {
|
||||
text = strings.Replace(text, "\"fmt\";", "", 1)
|
||||
}
|
||||
put(generated, "internal/handler/"+group+"/handlersGen.go", generatedHeader+text)
|
||||
put(seeds, "internal/service/"+group+"/service.go", fmt.Sprintf("package %s\nimport %q\ntype Service struct {deps *servicecontext.Context}\nfunc New(deps *servicecontext.Context)*Service{return &Service{deps:deps}}\n", packageName, o.Module+"/internal/servicecontext"))
|
||||
put(seeds, "internal/repository/"+group+"/repository.go", "// Package "+packageName+" owns persistence contracts for this business group.\n// Define repository interfaces and implementations here; HTTP DTOs do not\n// determine a database schema. This file is never overwritten.\npackage "+packageName+"\n")
|
||||
}
|
||||
// Avoid unused imports when no route uses a named middleware.
|
||||
imports = strings.Replace(imports, strconv.Quote(frameworkModule+"/httpx")+";", "", 1)
|
||||
put(generated, "internal/handler/routesGen.go", generatedHeader+"package handler\n"+imports+")\n"+routes+"return nil\n}\n")
|
||||
registry := "package middleware\nimport " + strconv.Quote(frameworkModule+"/httpx") + "\nfunc Defaults()map[string]httpx.Middleware{return map[string]httpx.Middleware{\n"
|
||||
for _, m := range middlewareNames {
|
||||
registry += fmt.Sprintf("%q:%s,\n", m, upper(m))
|
||||
put(seeds, "internal/middleware/"+lower(m)+".go", fmt.Sprintf("package middleware\nimport(%q; apierrors %q)\n// %s is a developer-owned hook. Implement it before enabling this route.\nfunc %s(next httpx.Handler)httpx.Handler{return func(c *httpx.Context)error {return apierrors.New(501,\"MIDDLEWARE_NOT_IMPLEMENTED\",\"middleware is not implemented\")}}\n", frameworkModule+"/httpx", frameworkModule+"/errors", upper(m), upper(m)))
|
||||
}
|
||||
put(generated, "internal/middleware/registryGen.go", generatedHeader+registry+"}}\n")
|
||||
for name, source := range baseTemplates {
|
||||
content := strings.ReplaceAll(source, "MODULE", o.Module)
|
||||
content = strings.ReplaceAll(content, "LOGFORMAT", o.LogFormat)
|
||||
content = strings.ReplaceAll(content, "LOGOUTPUT", o.LogOutput)
|
||||
if name == "etc/config.yaml" && o.LogOutput == "file" {
|
||||
content = strings.Replace(content, "output: file", "output: stderr\n file: ./var/log/server.log\n rotation:\n enabled: true\n maxSizeMb: 100\n maxBackups: 10\n maxAgeDays: 7\n compress: true", 1)
|
||||
}
|
||||
put(seeds, name, content)
|
||||
}
|
||||
|
||||
infra, err := scaffold.APIInfrastructure(o.Module, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
for name, data := range infra {
|
||||
seeds[name] = data
|
||||
}
|
||||
if o.Storage == "local" {
|
||||
for _, r := range c.Spec.Routes {
|
||||
if r.Path == "/storage/object" {
|
||||
return nil, nil, errors.New("/storage/object is reserved by local storage")
|
||||
}
|
||||
}
|
||||
main := string(seeds["cmd/server/main.go"])
|
||||
main = strings.Replace(main, " if err=handler.RegisterRoutes", ` for _,method:=range []string{"GET","HEAD","PUT"}{app.Handle(method,"/storage/object",func(c *httpx.Context)error{deps.FileHTTP.ServeHTTP(c.Raw(),c.Request());return nil})}
|
||||
if err=handler.RegisterRoutes`, 1)
|
||||
main = strings.Replace(main, `"git.yuntuo.hk/k2safe/go-omni/logx"`, `"git.yuntuo.hk/k2safe/go-omni/logx"; "git.yuntuo.hk/k2safe/go-omni/httpx"`, 1)
|
||||
seeds["cmd/server/main.go"] = []byte(main)
|
||||
}
|
||||
return generated, seeds, nil
|
||||
}
|
||||
func hasResponse(routes []Route) bool {
|
||||
for _, r := range routes {
|
||||
if r.Response != "" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
func signature(r Route) string {
|
||||
s := upper(r.Handler) + "(ctx context.Context"
|
||||
if r.Request != "" {
|
||||
s += ",req *types." + r.Request
|
||||
}
|
||||
s += ") "
|
||||
if r.Response != "" {
|
||||
s += "(*types." + r.Response + ",error)"
|
||||
} else {
|
||||
s += "error"
|
||||
}
|
||||
return s
|
||||
}
|
||||
460
cmd/go-omni/apigen/parse.go
Normal file
460
cmd/go-omni/apigen/parse.go
Normal file
@@ -0,0 +1,460 @@
|
||||
// Package apigen implements go-omni's deliberately bounded go-zero-style API
|
||||
// dialect. It generates Omni handlers, never go-zero or transport dependencies.
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"text/scanner"
|
||||
)
|
||||
|
||||
type Spec struct {
|
||||
Title, Version, Service string
|
||||
Types []Type
|
||||
Routes []Route
|
||||
Sources map[string][]byte // relative to the entry API's directory
|
||||
Entry string
|
||||
}
|
||||
type Type struct {
|
||||
Name string
|
||||
Fields []Field
|
||||
Position string
|
||||
}
|
||||
type Field struct{ Name, Type, Tag string }
|
||||
type Route struct {
|
||||
Method, Path, Request, Response, Handler, Group, Summary string
|
||||
Middleware []string
|
||||
Status int
|
||||
Position string
|
||||
}
|
||||
type token struct {
|
||||
text, position string
|
||||
kind rune
|
||||
}
|
||||
type parser struct {
|
||||
tokens []token
|
||||
at int
|
||||
spec *Spec
|
||||
imports []string
|
||||
}
|
||||
|
||||
func Parse(path string) (*Spec, error) {
|
||||
entry, err := filepath.Abs(path)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
entry, err = filepath.EvalSymlinks(entry)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
root := filepath.Dir(entry)
|
||||
spec := &Spec{Version: "1.0.0", Sources: map[string][]byte{}, Entry: filepath.Base(entry)}
|
||||
visiting, visited := map[string]bool{}, map[string]bool{}
|
||||
total := 0
|
||||
var visit func(string, int) error
|
||||
visit = func(name string, depth int) error {
|
||||
if depth > 32 || len(visited)+len(visiting) >= 64 {
|
||||
return errors.New("api: import limit exceeded")
|
||||
}
|
||||
real, err := filepath.EvalSymlinks(name)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
rel, err := filepath.Rel(root, real)
|
||||
if err != nil || rel == ".." || strings.HasPrefix(rel, ".."+string(filepath.Separator)) {
|
||||
return errors.New("api: imports must stay within the entry API directory")
|
||||
}
|
||||
if filepath.Ext(real) != ".api" {
|
||||
return errors.New("api: expected .api file")
|
||||
}
|
||||
if visiting[real] {
|
||||
return fmt.Errorf("api: circular import %s", rel)
|
||||
}
|
||||
if visited[real] {
|
||||
return nil
|
||||
}
|
||||
visiting[real] = true
|
||||
defer delete(visiting, real)
|
||||
f, err := os.Open(real)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
_ = f.Close()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
total += len(data)
|
||||
if len(data) > 1<<20 || total > 8<<20 {
|
||||
return errors.New("api: source size limit exceeded")
|
||||
}
|
||||
tokens, err := lex(rel, string(data))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p := &parser{tokens: tokens, spec: spec}
|
||||
if err = p.parse(); err != nil {
|
||||
return err
|
||||
}
|
||||
spec.Sources[filepath.ToSlash(rel)] = data
|
||||
for _, imp := range p.imports {
|
||||
if filepath.IsAbs(imp) || strings.Contains(imp, "\\") {
|
||||
return errors.New("api: imports must use relative slash paths")
|
||||
}
|
||||
if err = visit(filepath.Join(filepath.Dir(real), filepath.FromSlash(imp)), depth+1); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
visited[real] = true
|
||||
return nil
|
||||
}
|
||||
if err = visit(entry, 0); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if spec.Service == "" || len(spec.Routes) == 0 {
|
||||
return nil, errors.New("api: at least one service route required")
|
||||
}
|
||||
if spec.Title == "" {
|
||||
spec.Title = spec.Service
|
||||
}
|
||||
return spec, nil
|
||||
}
|
||||
func lex(name, source string) ([]token, error) {
|
||||
var s scanner.Scanner
|
||||
s.Init(strings.NewReader(source))
|
||||
s.Filename = name
|
||||
s.Mode = scanner.ScanIdents | scanner.ScanInts | scanner.ScanStrings | scanner.ScanRawStrings | scanner.ScanComments | scanner.SkipComments
|
||||
s.Whitespace ^= 1 << '\n'
|
||||
var scanErr error
|
||||
s.Error = func(s *scanner.Scanner, message string) {
|
||||
if scanErr == nil {
|
||||
scanErr = fmt.Errorf("%s: %s", s.Position, message)
|
||||
}
|
||||
}
|
||||
var out []token
|
||||
for {
|
||||
kind := s.Scan()
|
||||
out = append(out, token{s.TokenText(), s.Position.String(), kind})
|
||||
if kind == scanner.EOF {
|
||||
break
|
||||
}
|
||||
}
|
||||
return out, scanErr
|
||||
}
|
||||
func (p *parser) peek() string { return p.tokens[p.at].text }
|
||||
func (p *parser) next() token {
|
||||
t := p.tokens[p.at]
|
||||
if t.kind != scanner.EOF {
|
||||
p.at++
|
||||
}
|
||||
return t
|
||||
}
|
||||
func (p *parser) skip() {
|
||||
for p.peek() == "\n" || p.peek() == ";" {
|
||||
p.next()
|
||||
}
|
||||
}
|
||||
func (p *parser) err(message string) error {
|
||||
return fmt.Errorf("%s: %s (got %q)", p.tokens[p.at].position, message, p.peek())
|
||||
}
|
||||
func (p *parser) expect(s string) error {
|
||||
if p.peek() != s {
|
||||
return p.err("expected " + strconv.Quote(s))
|
||||
}
|
||||
p.next()
|
||||
return nil
|
||||
}
|
||||
func (p *parser) ident() (string, error) {
|
||||
if p.tokens[p.at].kind != scanner.Ident {
|
||||
return "", p.err("expected identifier")
|
||||
}
|
||||
return p.next().text, nil
|
||||
}
|
||||
func (p *parser) quoted() (string, error) {
|
||||
if p.tokens[p.at].kind != scanner.String {
|
||||
return "", p.err("expected quoted string")
|
||||
}
|
||||
return strconv.Unquote(p.next().text)
|
||||
}
|
||||
func (p *parser) parse() error {
|
||||
server := map[string]string{}
|
||||
syntaxSeen := false
|
||||
for p.skip(); p.tokens[p.at].kind != scanner.EOF; p.skip() {
|
||||
switch p.peek() {
|
||||
case "syntax":
|
||||
if syntaxSeen {
|
||||
return p.err("duplicate syntax")
|
||||
}
|
||||
syntaxSeen = true
|
||||
p.next()
|
||||
if err := p.expect("="); err != nil {
|
||||
return err
|
||||
}
|
||||
v, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if v != "v1" {
|
||||
return p.err("only syntax v1 supported")
|
||||
}
|
||||
case "info":
|
||||
p.next()
|
||||
m, err := p.metadata()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for k, v := range m {
|
||||
switch k {
|
||||
case "title":
|
||||
p.spec.Title = v
|
||||
case "version":
|
||||
p.spec.Version = v
|
||||
case "desc", "description", "author", "email":
|
||||
default:
|
||||
return p.err("unsupported info property " + k)
|
||||
}
|
||||
}
|
||||
case "import":
|
||||
p.next()
|
||||
if p.peek() == "(" {
|
||||
p.next()
|
||||
for p.skip(); p.peek() != ")"; p.skip() {
|
||||
v, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.imports = append(p.imports, v)
|
||||
}
|
||||
p.next()
|
||||
} else {
|
||||
v, err := p.quoted()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.imports = append(p.imports, v)
|
||||
}
|
||||
case "type":
|
||||
p.next()
|
||||
if p.peek() == "(" {
|
||||
p.next()
|
||||
for p.skip(); p.peek() != ")"; p.skip() {
|
||||
if err := p.parseType(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
p.next()
|
||||
} else {
|
||||
if err := p.parseType(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
case "@":
|
||||
p.next()
|
||||
if err := p.expect("server"); err != nil {
|
||||
return err
|
||||
}
|
||||
if len(server) != 0 {
|
||||
return p.err("duplicate server block")
|
||||
}
|
||||
m, err := p.metadata()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
server = m
|
||||
for k := range m {
|
||||
if k != "prefix" && k != "group" && k != "middleware" {
|
||||
return p.err("unsupported @server option " + k + "; wire authentication through named middleware")
|
||||
}
|
||||
}
|
||||
case "service":
|
||||
if err := p.parseService(server); err != nil {
|
||||
return err
|
||||
}
|
||||
server = map[string]string{}
|
||||
default:
|
||||
return p.err("unsupported declaration")
|
||||
}
|
||||
}
|
||||
if len(server) > 0 {
|
||||
return p.err("@server must precede a service")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (p *parser) metadata() (map[string]string, error) {
|
||||
if err := p.expect("("); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out := map[string]string{}
|
||||
for p.skip(); p.peek() != ")"; p.skip() {
|
||||
key, err := p.ident()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = p.expect(":"); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
value := ""
|
||||
if p.tokens[p.at].kind == scanner.String {
|
||||
value, err = p.quoted()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
} else {
|
||||
for p.peek() != "\n" && p.peek() != ";" && p.peek() != ")" && p.tokens[p.at].kind != scanner.EOF {
|
||||
value += p.next().text
|
||||
}
|
||||
}
|
||||
if value == "" {
|
||||
return nil, p.err("metadata value required")
|
||||
}
|
||||
if _, exists := out[key]; exists {
|
||||
return nil, p.err("duplicate metadata " + key)
|
||||
}
|
||||
out[key] = value
|
||||
}
|
||||
p.next()
|
||||
return out, nil
|
||||
}
|
||||
func (p *parser) parseType() error {
|
||||
pos := p.tokens[p.at].position
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.skip()
|
||||
if err = p.expect("{"); err != nil {
|
||||
return err
|
||||
}
|
||||
typ := Type{Name: name, Position: pos}
|
||||
for p.skip(); p.peek() != "}"; p.skip() {
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
expression := ""
|
||||
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" && p.tokens[p.at].kind != scanner.RawString && p.tokens[p.at].kind != scanner.EOF {
|
||||
expression += p.next().text
|
||||
}
|
||||
if expression == "" {
|
||||
return p.err("field type required; embedded types are not supported")
|
||||
}
|
||||
if p.tokens[p.at].kind != scanner.RawString {
|
||||
return p.err("field source tag required")
|
||||
}
|
||||
tag, err := strconv.Unquote(p.next().text)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
typ.Fields = append(typ.Fields, Field{Name: name, Type: expression, Tag: tag})
|
||||
}
|
||||
p.next()
|
||||
p.spec.Types = append(p.spec.Types, typ)
|
||||
return nil
|
||||
}
|
||||
func (p *parser) parseService(server map[string]string) error {
|
||||
p.next()
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for p.peek() == "-" {
|
||||
p.next()
|
||||
part, e := p.ident()
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
name += "-" + part
|
||||
}
|
||||
if p.spec.Service != "" && p.spec.Service != name {
|
||||
return p.err("one service name per entry API is supported")
|
||||
}
|
||||
p.spec.Service = name
|
||||
p.skip()
|
||||
if err = p.expect("{"); err != nil {
|
||||
return err
|
||||
}
|
||||
for p.skip(); p.peek() != "}"; p.skip() {
|
||||
route := Route{Group: server["group"], Status: 200, Position: p.tokens[p.at].position}
|
||||
if route.Group == "" {
|
||||
route.Group = "api"
|
||||
}
|
||||
if server["middleware"] != "" {
|
||||
route.Middleware = strings.Split(server["middleware"], ",")
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for p.peek() == "@" {
|
||||
p.next()
|
||||
annotation, e := p.ident()
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
if seen[annotation] {
|
||||
return p.err("duplicate route annotation")
|
||||
}
|
||||
seen[annotation] = true
|
||||
switch annotation {
|
||||
case "handler":
|
||||
route.Handler, e = p.ident()
|
||||
case "doc":
|
||||
route.Summary, e = p.quoted()
|
||||
case "status":
|
||||
route.Status, e = strconv.Atoi(p.next().text)
|
||||
default:
|
||||
return p.err("unsupported route annotation " + annotation)
|
||||
}
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
p.skip()
|
||||
}
|
||||
if route.Handler == "" {
|
||||
return p.err("@handler required")
|
||||
}
|
||||
route.Method, err = p.ident()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
path := ""
|
||||
for p.peek() != "\n" && p.peek() != ";" && p.peek() != "(" && p.peek() != "returns" && p.peek() != "}" && p.tokens[p.at].kind != scanner.EOF {
|
||||
path += p.next().text
|
||||
}
|
||||
route.Path = server["prefix"] + path
|
||||
if p.peek() == "(" {
|
||||
route.Request, err = p.routeType()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if p.peek() == "returns" {
|
||||
p.next()
|
||||
route.Response, err = p.routeType()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if p.peek() != "\n" && p.peek() != ";" && p.peek() != "}" {
|
||||
return p.err("unexpected route suffix")
|
||||
}
|
||||
if route.Response == "" && !seen["status"] {
|
||||
route.Status = 204
|
||||
}
|
||||
p.spec.Routes = append(p.spec.Routes, route)
|
||||
}
|
||||
p.next()
|
||||
return nil
|
||||
}
|
||||
func (p *parser) routeType() (string, error) {
|
||||
if err := p.expect("("); err != nil {
|
||||
return "", err
|
||||
}
|
||||
name, err := p.ident()
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
return name, p.expect(")")
|
||||
}
|
||||
73
cmd/go-omni/apigen/profile.go
Normal file
73
cmd/go-omni/apigen/profile.go
Normal file
@@ -0,0 +1,73 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
type projectProfile struct {
|
||||
Database string `json:"database"`
|
||||
Storage string `json:"storage"`
|
||||
LogOutput string `json:"logOutput"`
|
||||
LogFormat string `json:"logFormat"`
|
||||
}
|
||||
|
||||
func (o Options) profile() projectProfile {
|
||||
return projectProfile{o.Database, o.Storage, o.LogOutput, o.LogFormat}
|
||||
}
|
||||
|
||||
// Infrastructure files belong to the developer. Persist the original selection
|
||||
// so a plain regeneration cannot silently change adapters or repair a missing
|
||||
// file with settings from another profile.
|
||||
func resolveProfile(o *Options) error {
|
||||
defaults := projectProfile{"none", "none", "stderr", "json"}
|
||||
data, err := os.ReadFile(filepath.Join(o.Dir, manifestPath))
|
||||
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
existing := err == nil
|
||||
if existing {
|
||||
var previous manifest
|
||||
if err = json.Unmarshal(data, &previous); err != nil {
|
||||
return err
|
||||
}
|
||||
if previous.Profile != nil {
|
||||
defaults = *previous.Profile
|
||||
}
|
||||
}
|
||||
if o.Database == "memory" {
|
||||
o.Database = "none"
|
||||
}
|
||||
for _, item := range []struct {
|
||||
name string
|
||||
value *string
|
||||
fallback string
|
||||
}{
|
||||
{"database", &o.Database, defaults.Database}, {"storage", &o.Storage, defaults.Storage},
|
||||
{"log-output", &o.LogOutput, defaults.LogOutput}, {"log-format", &o.LogFormat, defaults.LogFormat},
|
||||
} {
|
||||
if *item.value == "" {
|
||||
*item.value = item.fallback
|
||||
} else if existing && *item.value != item.fallback {
|
||||
return fmt.Errorf("--%s differs from the original project profile; infrastructure is developer-owned, generate a new project to compare and migrate it explicitly", item.name)
|
||||
}
|
||||
}
|
||||
if o.Database != "none" && o.Database != "mysql" {
|
||||
return errors.New("--database must be none, memory or mysql")
|
||||
}
|
||||
switch o.Storage {
|
||||
case "none", "local", "s3", "r2", "oss":
|
||||
default:
|
||||
return errors.New("--storage must be none, local, s3, r2 or oss")
|
||||
}
|
||||
if o.LogOutput != "stderr" && o.LogOutput != "stdout" && o.LogOutput != "file" {
|
||||
return errors.New("--log-output must be stdout, stderr or file")
|
||||
}
|
||||
if o.LogFormat != "json" && o.LogFormat != "text" {
|
||||
return errors.New("--log-format must be json or text")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
98
cmd/go-omni/apigen/profile_test.go
Normal file
98
cmd/go-omni/apigen/profile_test.go
Normal file
@@ -0,0 +1,98 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestInfrastructureProfileOwnership(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
o := Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: "mysql", Storage: "local", LogOutput: "file"}
|
||||
if _, err := New(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
o = Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before := snapshot(t, dir)
|
||||
for _, changed := range []Options{
|
||||
{Database: "none"}, {Storage: "s3"}, {LogOutput: "stdout"}, {LogFormat: "text"},
|
||||
} {
|
||||
changed.Dir, changed.API = dir, o.API
|
||||
if _, err := Generate(changed); err == nil {
|
||||
t.Fatal("profile switch accepted")
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("profile rejection changed project")
|
||||
}
|
||||
}
|
||||
// Restoring a missing seed must use the saved profile, not current defaults.
|
||||
if err := os.Remove(filepath.Join(dir, "internal/servicecontext/storage.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Generate(o); err == nil {
|
||||
t.Fatal("check accepted missing infrastructure")
|
||||
}
|
||||
o.Check = false
|
||||
if _, err := Generate(o); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !sameFiles(before, snapshot(t, dir)) {
|
||||
t.Fatal("profile reconstruction changed project")
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPIInfrastructureConsumers(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") == "" {
|
||||
t.Skip("consumer module integration")
|
||||
}
|
||||
for _, db := range []string{"none", "mysql"} {
|
||||
for _, storage := range []string{"none", "local", "s3", "r2", "oss"} {
|
||||
t.Run(db+"/"+storage, func(t *testing.T) {
|
||||
dir := filepath.Join(t.TempDir(), "project")
|
||||
if _, err := New(Options{Dir: dir, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: db, Storage: storage, LogOutput: "file"}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := Generate(Options{Dir: dir, API: filepath.Join(dir, "api/service.api"), Check: true}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
test := `package settings
|
||||
import("testing";"os")
|
||||
func TestConfiguration(t *testing.T){
|
||||
t.Setenv("GO_OMNI_DB_DSN","test:test@tcp(127.0.0.1:1)/test")
|
||||
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY","01234567890123456789012345678901")
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","test");t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","test")
|
||||
t.Setenv("GO_OMNI_STORAGE_BUCKET","test");t.Setenv("GO_OMNI_STORAGE_ENDPOINT","ENDPOINT")
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID","01234567890123456789012345678901")
|
||||
cfg,err:=Load("../../etc/config.yaml");if err!=nil{t.Fatal(err)}
|
||||
if cfg.Server.Log.File==""{t.Fatal("file logger absent")}
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout");cfg,err=Load("");if err!=nil||cfg.Server.Log.File!=""{t.Fatalf("override: %v",err)}
|
||||
_=os.Getenv("unused")
|
||||
}`
|
||||
endpoint := ""
|
||||
if storage == "s3" {
|
||||
endpoint = "127.0.0.1:9000"
|
||||
}
|
||||
if storage == "oss" {
|
||||
endpoint = "https://oss-cn-hangzhou.aliyuncs.com"
|
||||
}
|
||||
writeFile(t, filepath.Join(dir, "internal/settings/profile_test.go"), strings.ReplaceAll(test, `"ENDPOINT"`, `"`+endpoint+`"`))
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 90*time.Second)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-count=1", "./...")
|
||||
cmd.Dir = dir
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOFLAGS=")
|
||||
out, err := cmd.CombinedOutput()
|
||||
if err != nil {
|
||||
t.Fatalf("%v\n%s", err, out)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
160
cmd/go-omni/apigen/templates.go
Normal file
160
cmd/go-omni/apigen/templates.go
Normal file
@@ -0,0 +1,160 @@
|
||||
package apigen
|
||||
|
||||
import "fmt"
|
||||
|
||||
var baseTemplates = map[string]string{
|
||||
"cmd/server/main.go": `package main
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"MODULE/internal/handler"
|
||||
"MODULE/internal/servicecontext"
|
||||
"MODULE/internal/settings"
|
||||
)
|
||||
func main(){
|
||||
path:=flag.String("config","etc/config.yaml","configuration file; empty uses defaults and environment")
|
||||
flag.Parse()
|
||||
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||
}
|
||||
func run(path string)(runErr error){
|
||||
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||
defer stop()
|
||||
cfg,err:=settings.Load(path);if err!=nil{return err}
|
||||
logger,err:=logx.New(cfg.Server.Log);if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||
logx.SetDefault(logger)
|
||||
deps,err:=servicecontext.New(ctx,logger,cfg);if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,deps.Close())}()
|
||||
app:=omni.New(omni.WithConfig(cfg.Server))
|
||||
app.OnShutdown(logger.Close)
|
||||
app.OnShutdown(deps.Close)
|
||||
if err=handler.RegisterRoutes(app,deps);err!=nil{return err}
|
||||
app.GET("/openapi.json",handler.OpenAPI)
|
||||
app.GET("/health/live",deps.Health.LivenessHandler())
|
||||
app.GET("/health/ready",deps.Health.ReadinessHandler())
|
||||
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||
return app.RunContext(ctx,cfg.Server.Address)
|
||||
}
|
||||
`,
|
||||
".gitignore": "/server\n/var/\n.env\n.env.*\n!.env.example\n/.goOmni/api.lock\n",
|
||||
}
|
||||
|
||||
func projectREADME(module, entry string) string {
|
||||
return fmt.Sprintf(`# %s
|
||||
|
||||
This project is generated from an editable API contract. Business code uses
|
||||
standard context.Context; generated HTTP handlers use Omni's transport-neutral httpx.
|
||||
|
||||
## Run
|
||||
|
||||
`+"```sh"+`
|
||||
go mod tidy
|
||||
go run ./cmd/server
|
||||
`+"```"+`
|
||||
|
||||
GET /health/live and /health/ready check the running service. GET /openapi.json
|
||||
serves the same document as docs/openapi.json. Bind address defaults to localhost.
|
||||
Set GO_OMNI_TRANSPORT=native to select the native transport without changing services.
|
||||
The greeting starter works immediately; other new operation stubs return 501 until
|
||||
their business logic is implemented. Authentication hooks also fail closed with 501.
|
||||
|
||||
## Develop
|
||||
|
||||
1. Edit %s and its relative imports.
|
||||
2. Run the same go-omni CLI version as the framework dependency:
|
||||
|
||||
`+"```sh"+`
|
||||
go-omni api validate --api %s
|
||||
go-omni api generate --api %s --dir .
|
||||
go-omni api generate --api %s --dir . --check
|
||||
go test ./...
|
||||
`+"```"+`
|
||||
|
||||
The --check command is read-only and suitable for CI. It fails on stale or missing
|
||||
generated files and missing business scaffolds. go.mod, configuration, main,
|
||||
Service, Repository, servicecontext, middleware hook bodies and API files are
|
||||
created once and then preserved byte for byte. Gen.go files and OpenAPI are owned
|
||||
by the generator; manual edits cause a conflict instead of being overwritten.
|
||||
Commit .goOmni/apiManifest.json alongside the generated files. Do not edit it.
|
||||
Avoid editing files while generation is running; api.lock serializes generators.
|
||||
|
||||
Routes and types that disappear are removed from generated output. Handwritten
|
||||
business files remain: remove/adapt obsolete implementations explicitly if an
|
||||
API change makes their old types or signatures fail to compile. Every regeneration
|
||||
must be followed by go test ./... . Do not delete service files to change contracts.
|
||||
|
||||
## Layout
|
||||
|
||||
`+"```text"+`
|
||||
api/ # editable contract and imported types
|
||||
cmd/server/main.go # startup, lifecycle and route registration
|
||||
etc/config.yaml # lowerCamelCase configuration
|
||||
internal/settings/ # defaults -> file -> environment
|
||||
internal/servicecontext/ # dependency initialization and cleanup
|
||||
internal/types/typesGen.go # generated request/response types
|
||||
internal/handler/<group>/ # generated binding and service adapters
|
||||
internal/service/<group>/ # developer-owned business operations
|
||||
internal/repository/<group>/ # developer-owned persistence contracts
|
||||
internal/middleware/ # named hooks and generated registry
|
||||
docs/openapi.json # generated OpenAPI 3.0.3
|
||||
.goOmni/apiManifest.json # ownership and content hashes
|
||||
`+"```"+`
|
||||
|
||||
The API response describes data only. Omni adds code, message and requestId.
|
||||
Do not define the envelope again in each response. Handlers reject nil successful
|
||||
responses. Body DTOs use application/json and have a 1 MiB limit by default.
|
||||
Use contract.Binder.Bind's optional limit in a custom adapter for larger payloads.
|
||||
|
||||
Fields require exactly one source tag: json, path, form (URL query only), header.
|
||||
All are required by presence unless optional, omitempty or default is specified.
|
||||
Zero and false are valid present values; binding:"required" rejects scalar zero.
|
||||
Use *T with optional when business logic needs to distinguish missing from zero.
|
||||
Pointers may carry null; non-pointer null is rejected. Unknown/duplicate JSON
|
||||
fields and duplicate scalar query/header values are rejected. Arrays in the query
|
||||
use repeated keys. Enum/range/default and nested fields share the documentation
|
||||
and binding compiler. Responses are expected to honor their declared constraints;
|
||||
they are not validated on every response. Do not return nil slices/maps for a
|
||||
non-nullable collection (initialize empty collections instead).
|
||||
|
||||
Named middleware is composed in declaration order (first is outermost). Implement
|
||||
the generated hooks or replace entries in servicecontext; never bypass an auth
|
||||
hook merely to make the server start. Middleware names do not imply a JWT policy.
|
||||
|
||||
This CLI supports a documented go-zero-style subset, not all goctl features.
|
||||
Database tables and repositories cannot be inferred from an HTTP contract. Add
|
||||
business repositories through servicecontext. Database and storage options are
|
||||
composed at creation; the original profile is retained on regeneration.
|
||||
`, module, entry, entry, entry, entry)
|
||||
}
|
||||
|
||||
func infrastructureREADME(o Options) string {
|
||||
return fmt.Sprintf(`
|
||||
## Infrastructure
|
||||
|
||||
Selected database: %s; storage: %s; logging: %s/%s.
|
||||
MySQL uses database/sql through deps.DB, including InTx and a readiness probe.
|
||||
Set GO_OMNI_DB_DSN; pool settings live in etc/config.yaml and GO_OMNI_DB_*.
|
||||
No tables are invented or migrated automatically from request/response DTOs.
|
||||
The none/memory selection opens no database; implement repositories per business.
|
||||
|
||||
With storage enabled, services receive deps.Files, Uploads, Downloads and
|
||||
UploadLinks capability interfaces. Local signed GET/HEAD/PUT uses /storage/object;
|
||||
set GO_OMNI_STORAGE_SIGNING_KEY (32+ bytes) and GO_OMNI_STORAGE_PUBLIC_URL.
|
||||
Cloud credentials use GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY; set bucket and endpoint
|
||||
or R2 accountId as appropriate. Signed URLs grant temporary access: authenticate
|
||||
and authorize before issuing them. No public upload-link route is added.
|
||||
Storage failures during startup close an already-open database; Close is idempotent.
|
||||
|
||||
Database/storage/logging selection is recorded in .goOmni/apiManifest.json.
|
||||
Changing selection through generation flags is rejected on an existing project;
|
||||
edit your owned configuration and initialization explicitly, or generate a fresh
|
||||
project for comparison. New routes still regenerate without repeating these flags.
|
||||
`, o.Database, o.Storage, o.LogOutput, o.LogFormat)
|
||||
}
|
||||
295
cmd/go-omni/apigen/write.go
Normal file
295
cmd/go-omni/apigen/write.go
Normal file
@@ -0,0 +1,295 @@
|
||||
package apigen
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const manifestPath = ".goOmni/apiManifest.json"
|
||||
|
||||
type manifest struct {
|
||||
Profile *projectProfile `json:"profile,omitempty"`
|
||||
Version int `json:"version"`
|
||||
Module string `json:"module"`
|
||||
Entry string `json:"entry"`
|
||||
Generated map[string]string `json:"generated"`
|
||||
}
|
||||
type change struct {
|
||||
name string
|
||||
before, after []byte
|
||||
existed, remove bool
|
||||
mode os.FileMode
|
||||
}
|
||||
|
||||
func apply(root, module, entry string, generated, seeds map[string][]byte, check bool, profile projectProfile) (Output, error) {
|
||||
var result Output
|
||||
if info, err := os.Lstat(root); err == nil {
|
||||
if info.Mode()&os.ModeSymlink != 0 || !info.IsDir() {
|
||||
return result, errors.New("api: output root must be a real directory")
|
||||
}
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return result, err
|
||||
}
|
||||
root = canonical(root)
|
||||
names := map[string]bool{}
|
||||
caseNames := map[string]string{}
|
||||
for _, files := range []map[string][]byte{generated, seeds} {
|
||||
for name := range files {
|
||||
if names[name] {
|
||||
return result, fmt.Errorf("duplicate output %s", name)
|
||||
}
|
||||
names[name] = true
|
||||
folded := strings.ToLower(name)
|
||||
if old, ok := caseNames[folded]; ok {
|
||||
return result, fmt.Errorf("case-insensitive path collision: %s and %s", old, name)
|
||||
}
|
||||
caseNames[folded] = name
|
||||
if err := safePath(root, name); err != nil {
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := safePath(root, manifestPath); err != nil {
|
||||
return result, err
|
||||
}
|
||||
if !check {
|
||||
if err := os.MkdirAll(filepath.Join(root, ".goOmni"), 0755); err != nil {
|
||||
return result, err
|
||||
}
|
||||
lock := filepath.Join(root, ".goOmni", "api.lock")
|
||||
f, err := os.OpenFile(lock, os.O_WRONLY|os.O_CREATE|os.O_EXCL, 0600)
|
||||
if err != nil {
|
||||
return result, fmt.Errorf("api: acquire generation lock (remove stale api.lock only after checking no generator is running): %w", err)
|
||||
}
|
||||
_, _ = fmt.Fprintf(f, "pid=%d\n", os.Getpid())
|
||||
_ = f.Close()
|
||||
defer os.Remove(lock)
|
||||
}
|
||||
old := manifest{Generated: map[string]string{}}
|
||||
oldData, err := os.ReadFile(filepath.Join(root, manifestPath))
|
||||
if err == nil {
|
||||
if err = json.Unmarshal(oldData, &old); err != nil {
|
||||
return result, fmt.Errorf("invalid ownership manifest: %w", err)
|
||||
}
|
||||
if old.Profile != nil && *old.Profile != profile {
|
||||
return result, errors.New("api: profile changed during generation; retry")
|
||||
}
|
||||
if old.Version != 1 || old.Module != module || old.Entry != entry {
|
||||
return result, errors.New("api: manifest version/module/entry differs; one API entry owns this project")
|
||||
}
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return result, err
|
||||
}
|
||||
if oldData == nil {
|
||||
if err = filepath.WalkDir(root, func(path string, d os.DirEntry, walkErr error) error {
|
||||
if errors.Is(walkErr, os.ErrNotExist) {
|
||||
return nil
|
||||
}
|
||||
if walkErr != nil {
|
||||
return walkErr
|
||||
}
|
||||
if d.IsDir() && d.Name() == ".git" {
|
||||
return filepath.SkipDir
|
||||
}
|
||||
if !d.IsDir() && strings.HasSuffix(d.Name(), ".go") {
|
||||
return fmt.Errorf("api: %s belongs to an existing Go layout; generate into a new directory", path)
|
||||
}
|
||||
return nil
|
||||
}); err != nil {
|
||||
return result, err
|
||||
}
|
||||
}
|
||||
next := manifest{Profile: &profile, Version: 1, Module: module, Entry: entry, Generated: map[string]string{}}
|
||||
for name, data := range generated {
|
||||
next.Generated[name] = digest(data)
|
||||
}
|
||||
var changes []change
|
||||
// Check ALL previous outputs, even stale ones, before writing any new file.
|
||||
for name, hash := range old.Generated {
|
||||
if !ownedPath(name) {
|
||||
return result, fmt.Errorf("api: invalid owned path %q", name)
|
||||
}
|
||||
if err = safePath(root, name); err != nil {
|
||||
return result, err
|
||||
}
|
||||
data, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(name)))
|
||||
if errors.Is(e, os.ErrNotExist) {
|
||||
continue
|
||||
}
|
||||
if e != nil {
|
||||
return result, e
|
||||
}
|
||||
if digest(data) != hash {
|
||||
return result, fmt.Errorf("api: generated file %s was edited; move custom code into developer-owned files before regenerating", name)
|
||||
}
|
||||
if _, exists := generated[name]; !exists {
|
||||
info, _ := os.Stat(filepath.Join(root, filepath.FromSlash(name)))
|
||||
changes = append(changes, change{name: name, before: data, existed: true, remove: true, mode: info.Mode().Perm()})
|
||||
result.Removed = append(result.Removed, name)
|
||||
}
|
||||
}
|
||||
paths := make([]string, 0, len(names))
|
||||
for name := range names {
|
||||
paths = append(paths, name)
|
||||
}
|
||||
sort.Strings(paths)
|
||||
for _, name := range paths {
|
||||
target := filepath.Join(root, filepath.FromSlash(name))
|
||||
current, err := os.ReadFile(target)
|
||||
exists := err == nil
|
||||
if err != nil && !errors.Is(err, os.ErrNotExist) {
|
||||
return result, err
|
||||
}
|
||||
content, isGenerated := generated[name]
|
||||
if !isGenerated {
|
||||
content = seeds[name]
|
||||
}
|
||||
if exists && !isGenerated {
|
||||
result.Preserved = append(result.Preserved, name)
|
||||
continue
|
||||
}
|
||||
if exists && isGenerated {
|
||||
if _, owned := old.Generated[name]; !owned {
|
||||
return result, fmt.Errorf("api: refusing to adopt existing unowned file %s", name)
|
||||
}
|
||||
}
|
||||
if isGenerated {
|
||||
result.Generated = append(result.Generated, name)
|
||||
} else {
|
||||
result.Created = append(result.Created, name)
|
||||
}
|
||||
if exists && bytes.Equal(current, content) {
|
||||
continue
|
||||
}
|
||||
mode := os.FileMode(0644)
|
||||
if exists {
|
||||
info, e := os.Stat(target)
|
||||
if e != nil {
|
||||
return result, e
|
||||
}
|
||||
mode = info.Mode().Perm()
|
||||
}
|
||||
changes = append(changes, change{name: name, before: current, after: content, existed: exists, mode: mode})
|
||||
}
|
||||
data, err := json.MarshalIndent(next, "", " ")
|
||||
if err != nil {
|
||||
return result, err
|
||||
}
|
||||
data = append(data, '\n')
|
||||
if !bytes.Equal(oldData, data) {
|
||||
changes = append(changes, change{name: manifestPath, before: oldData, after: data, existed: oldData != nil, mode: 0644})
|
||||
}
|
||||
sort.Strings(result.Removed)
|
||||
if check {
|
||||
if len(changes) > 0 {
|
||||
var stale []string
|
||||
for _, c := range changes {
|
||||
stale = append(stale, c.name)
|
||||
}
|
||||
sort.Strings(stale)
|
||||
return result, fmt.Errorf("api: generated project is stale: %s", strings.Join(stale, ", "))
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
// A rollback restores bytes/modes if an ordinary filesystem write fails.
|
||||
// File replacement is atomic; the whole directory is not crash-atomic.
|
||||
var applied []change
|
||||
for _, c := range changes {
|
||||
if err = safePath(root, c.name); err == nil {
|
||||
current, e := os.ReadFile(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||
if c.existed {
|
||||
if e != nil || !bytes.Equal(current, c.before) {
|
||||
err = fmt.Errorf("file changed during generation: %s", c.name)
|
||||
}
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
err = fmt.Errorf("file appeared during generation: %s", c.name)
|
||||
}
|
||||
}
|
||||
if err == nil {
|
||||
if c.remove {
|
||||
err = os.Remove(filepath.Join(root, filepath.FromSlash(c.name)))
|
||||
} else {
|
||||
err = atomicWrite(root, c.name, c.after, c.mode)
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
for i := len(applied) - 1; i >= 0; i-- {
|
||||
previous := applied[i]
|
||||
var rollback error
|
||||
if previous.existed {
|
||||
rollback = atomicWrite(root, previous.name, previous.before, previous.mode)
|
||||
} else {
|
||||
rollback = os.Remove(filepath.Join(root, filepath.FromSlash(previous.name)))
|
||||
}
|
||||
err = errors.Join(err, rollback)
|
||||
}
|
||||
return result, err
|
||||
}
|
||||
applied = append(applied, c)
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func digest(data []byte) string { sum := sha256.Sum256(data); return hex.EncodeToString(sum[:]) }
|
||||
func ownedPath(name string) bool {
|
||||
if name == "docs/openapi.json" || name == "internal/types/typesGen.go" || name == "internal/handler/routesGen.go" || name == "internal/handler/openapiGen.go" || name == "internal/middleware/registryGen.go" {
|
||||
return true
|
||||
}
|
||||
return strings.HasPrefix(name, "internal/handler/") && strings.HasSuffix(name, "/handlersGen.go")
|
||||
}
|
||||
func safePath(root, name string) error {
|
||||
if filepath.IsAbs(name) || strings.Contains(name, "\\") || filepath.ToSlash(filepath.Clean(name)) != name || name == "." || strings.HasPrefix(name, "../") {
|
||||
return fmt.Errorf("api: unsafe output path %q", name)
|
||||
}
|
||||
path := root
|
||||
parts := strings.Split(name, "/")
|
||||
for i, part := range parts {
|
||||
path = filepath.Join(path, part)
|
||||
info, err := os.Lstat(path)
|
||||
if errors.Is(err, os.ErrNotExist) {
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if info.Mode()&os.ModeSymlink != 0 {
|
||||
return fmt.Errorf("api: refusing symlink %s", path)
|
||||
}
|
||||
if i < len(parts)-1 && !info.IsDir() {
|
||||
return fmt.Errorf("api: expected directory %s", path)
|
||||
}
|
||||
if i == len(parts)-1 && !info.Mode().IsRegular() {
|
||||
return fmt.Errorf("api: expected regular file %s", path)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func atomicWrite(root, name string, data []byte, mode os.FileMode) error {
|
||||
target := filepath.Join(root, filepath.FromSlash(name))
|
||||
if err := os.MkdirAll(filepath.Dir(target), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
f, err := os.CreateTemp(filepath.Dir(target), ".apiGen-*")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer os.Remove(f.Name())
|
||||
if err = f.Chmod(mode); err == nil {
|
||||
_, err = f.Write(data)
|
||||
}
|
||||
if err == nil {
|
||||
err = f.Sync()
|
||||
}
|
||||
err = errors.Join(err, f.Close())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return os.Rename(f.Name(), target)
|
||||
}
|
||||
@@ -7,6 +7,7 @@ import (
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/apigen"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cmd/go-omni/scaffold"
|
||||
)
|
||||
|
||||
@@ -17,6 +18,21 @@ func main() {
|
||||
}
|
||||
|
||||
switch os.Args[1] {
|
||||
case "api":
|
||||
if err := runAPI(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni api:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "migrate":
|
||||
if err := runMigrate(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni migrate:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "add":
|
||||
if err := runAdd(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni add:", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
case "new":
|
||||
if err := runNew(os.Args[2:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, "go-omni new:", err)
|
||||
@@ -34,9 +50,43 @@ func main() {
|
||||
func runNew(args []string) error {
|
||||
options := scaffold.Options{}
|
||||
var target string
|
||||
var apiFile string
|
||||
for index := 0; index < len(args); index++ {
|
||||
arg := args[index]
|
||||
switch {
|
||||
case arg == "--api" || strings.HasPrefix(arg, "--api="):
|
||||
_, value, inline := strings.Cut(arg, "=")
|
||||
if !inline {
|
||||
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "-") {
|
||||
return errors.New("--api requires a file")
|
||||
}
|
||||
index++
|
||||
value = args[index]
|
||||
}
|
||||
if value == "" {
|
||||
return errors.New("--api requires a file")
|
||||
}
|
||||
apiFile = value
|
||||
case arg == "--log-output" || arg == "--log-format" || arg == "--storage" || strings.HasPrefix(arg, "--log-output=") || strings.HasPrefix(arg, "--log-format=") || strings.HasPrefix(arg, "--storage="):
|
||||
name, value, hasValue := strings.Cut(arg, "=")
|
||||
if !hasValue {
|
||||
if index+1 >= len(args) || strings.HasPrefix(args[index+1], "--") {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
index++
|
||||
value = args[index]
|
||||
}
|
||||
if strings.TrimSpace(value) == "" {
|
||||
return fmt.Errorf("%s requires a value", name)
|
||||
}
|
||||
switch name {
|
||||
case "--log-output":
|
||||
options.LogOutput = value
|
||||
case "--log-format":
|
||||
options.LogFormat = value
|
||||
case "--storage":
|
||||
options.Storage = value
|
||||
}
|
||||
case arg == "--force":
|
||||
options.Force = true
|
||||
case arg == "--database":
|
||||
@@ -88,10 +138,26 @@ func runNew(args []string) error {
|
||||
if target == "" {
|
||||
return errors.New("project directory is required")
|
||||
}
|
||||
if options.Module == "" {
|
||||
if options.Module == "" && apiFile == "" {
|
||||
options.Module = filepath.Base(filepath.Clean(target))
|
||||
}
|
||||
options.Dir = target
|
||||
if apiFile != "" {
|
||||
if options.Force {
|
||||
return errors.New("--api protects existing business code and does not accept --force")
|
||||
}
|
||||
_, err := apigen.Generate(apigen.Options{API: apiFile, Dir: target, Module: options.Module, FrameworkDir: options.FrameworkDir, FrameworkVersion: options.FrameworkVersion, LogOutput: options.LogOutput, LogFormat: options.LogFormat, Database: options.Database, Storage: options.Storage})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("created API project %s; implement internal/service operations\n", filepath.Clean(target))
|
||||
return nil
|
||||
}
|
||||
if _, err := os.Lstat(filepath.Join(target, ".goOmni", "apiManifest.json")); err == nil {
|
||||
return errors.New("this is an API project; use api generate instead of the CRUD/module starter")
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
if err := scaffold.Generate(options); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -103,8 +169,45 @@ func usage() {
|
||||
fmt.Fprintln(os.Stderr, "GoOmni project tool")
|
||||
fmt.Fprintln(os.Stderr, "")
|
||||
fmt.Fprintln(os.Stderr, "Usage:")
|
||||
fmt.Fprintln(os.Stderr, " go-omni migrate [--dir migrations] [--apply] [--dsn-env GO_OMNI_DB_DSN]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new [--module <module>] [--force] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni new --api <file.api> [--module <module>] <directory>")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api new [--module <module>] <directory> Create a runnable API project")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api generate --api <file.api> --dir <project> [--check]")
|
||||
fmt.Fprintln(os.Stderr, " go-omni api validate --api <file.api> Validate definitions and OpenAPI")
|
||||
fmt.Fprintln(os.Stderr, " go-omni add <name> [--dir <project>] Generate an isolated business module")
|
||||
fmt.Fprintln(os.Stderr, " --framework-version <version> Pin a remote framework version (defaults to CLI version)")
|
||||
fmt.Fprintln(os.Stderr, " --framework-dir <directory> Use a local framework checkout")
|
||||
fmt.Fprintln(os.Stderr, " --database <memory|mysql> Select generated storage (default memory)")
|
||||
fmt.Fprintln(os.Stderr, " --database <none|memory|mysql> API defaults to none; CRUD defaults to memory")
|
||||
fmt.Fprintln(os.Stderr, " --log-output <stdout|stderr|file> Generate configured logging (default stderr)")
|
||||
fmt.Fprintln(os.Stderr, " --log-format <json|text> Select structured log format (default json)")
|
||||
fmt.Fprintln(os.Stderr, " --storage <none|local|s3|r2|oss> Select optional storage adapter (default none)")
|
||||
}
|
||||
|
||||
func runAdd(args []string) error {
|
||||
directory, name := ".", ""
|
||||
for index := 0; index < len(args); index++ {
|
||||
arg := args[index]
|
||||
switch {
|
||||
case arg == "--dir":
|
||||
if index+1 >= len(args) {
|
||||
return errors.New("--dir requires a project directory")
|
||||
}
|
||||
index++
|
||||
directory = args[index]
|
||||
case strings.HasPrefix(arg, "--dir="):
|
||||
directory = strings.TrimPrefix(arg, "--dir=")
|
||||
case strings.HasPrefix(arg, "-"):
|
||||
return fmt.Errorf("unknown option %q", arg)
|
||||
case name == "":
|
||||
name = arg
|
||||
default:
|
||||
return fmt.Errorf("unexpected argument %q", arg)
|
||||
}
|
||||
}
|
||||
if err := scaffold.AddModule(directory, name); err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("created internal/%s; see its README.md for route registration\n", name)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -32,6 +32,30 @@ func TestNewRejectsEmptyFrameworkVersion(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleCLI(t *testing.T) {
|
||||
for _, equals := range []bool{false, true} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
if err := runNew([]string{"--module", "example.com/billing", project}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
args := []string{"orders", "--dir", project}
|
||||
if equals {
|
||||
args = []string{"--dir=" + project, "orders"}
|
||||
}
|
||||
if err := runAdd(args); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(project, "internal/orders/module.go")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, args := range [][]string{nil, {"orders", "--dir"}, {"orders", "--dir="}, {"orders", "extra"}, {"orders", "--unknown"}} {
|
||||
if err := runAdd(args); err == nil {
|
||||
t.Fatalf("accepted invalid arguments %v", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewMySQLDatabaseOptions(t *testing.T) {
|
||||
for _, flag := range [][]string{{"--database", "mysql"}, {"--database=mysql"}} {
|
||||
project := filepath.Join(t.TempDir(), "billing")
|
||||
@@ -49,3 +73,28 @@ func TestNewMySQLDatabaseOptions(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestModuleOptionCLI(t *testing.T) {
|
||||
for _, flags := range [][]string{
|
||||
{"--log-output", "file", "--log-format", "json", "--storage", "local"},
|
||||
{"--log-output=stdout", "--log-format=text", "--storage=none"},
|
||||
} {
|
||||
project := filepath.Join(t.TempDir(), "demo")
|
||||
args := append([]string{"--module", "example.com/demo"}, flags...)
|
||||
if err := runNew(append(args, project)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(project, "etc/config.yaml")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
for _, args := range [][]string{
|
||||
{"--storage"}, {"--storage="}, {"--storage", "--force"},
|
||||
{"--log-output"}, {"--log-output="}, {"--log-format="},
|
||||
{"--storage", "unknown", "demo"}, {"--log-format", "xml", "demo"},
|
||||
} {
|
||||
if err := runNew(args); err == nil {
|
||||
t.Fatal("invalid module option accepted", args)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
60
cmd/go-omni/migrate.go
Normal file
60
cmd/go-omni/migrate.go
Normal file
@@ -0,0 +1,60 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"flag"
|
||||
"fmt"
|
||||
migration "git.yuntuo.hk/k2safe/go-omni/database/migrate"
|
||||
"io"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func runMigrate(args []string) error {
|
||||
flags := flag.NewFlagSet("migrate", flag.ContinueOnError)
|
||||
flags.SetOutput(io.Discard)
|
||||
directory := flags.String("dir", "migrations", "SQL directory")
|
||||
dsnEnv := flags.String("dsn-env", "GO_OMNI_DB_DSN", "DSN environment variable")
|
||||
apply := flags.Bool("apply", false, "apply pending up migrations")
|
||||
if err := flags.Parse(args); err != nil {
|
||||
return err
|
||||
}
|
||||
if flags.NArg() != 0 {
|
||||
return errors.New("migrate: unexpected argument")
|
||||
}
|
||||
_, manifest, err := migration.Snapshot(os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !*apply {
|
||||
data, err := json.MarshalIndent(manifest, "", " ")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Println(string(data))
|
||||
return nil
|
||||
}
|
||||
dsn := os.Getenv(*dsnEnv)
|
||||
if dsn == "" {
|
||||
return errors.New("migrate: DSN environment variable is empty")
|
||||
}
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
runner, err := migration.NewMySQL(ctx, dsn, os.DirFS(*directory), ".")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer runner.Close()
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
return err
|
||||
}
|
||||
version, dirty, err := runner.Version()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Printf("migration version=%d dirty=%t\n", version, dirty)
|
||||
return nil
|
||||
}
|
||||
25
cmd/go-omni/migrate_test.go
Normal file
25
cmd/go-omni/migrate_test.go
Normal file
@@ -0,0 +1,25 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestMigrationPreviewDoesNotConnect(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "0001_createItems.up.sql"), []byte("CREATE TABLE testItems(id BIGINT);"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "not a DSN")
|
||||
if err := runMigrate([]string{"--dir", dir}); err != nil {
|
||||
t.Fatal("preview tried to connect", err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_DB_DSN", "")
|
||||
if err := runMigrate([]string{"--dir", dir, "--apply"}); err == nil {
|
||||
t.Fatal("missing DSN accepted")
|
||||
}
|
||||
if err := runMigrate([]string{"--dir", dir, "--force"}); err == nil {
|
||||
t.Fatal("unsafe unknown flag accepted")
|
||||
}
|
||||
}
|
||||
109
cmd/go-omni/scaffold/addModule.go
Normal file
109
cmd/go-omni/scaffold/addModule.go
Normal file
@@ -0,0 +1,109 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"go/format"
|
||||
"go/token"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strings"
|
||||
"text/template"
|
||||
|
||||
"golang.org/x/mod/modfile"
|
||||
)
|
||||
|
||||
var moduleName = regexp.MustCompile(`^[a-z][a-zA-Z0-9]{0,47}$`)
|
||||
|
||||
// AddModule creates a complete, injectable memory-backed CRUD module. It never
|
||||
// edits existing routes, go.mod, migrations or files in another module.
|
||||
func AddModule(directory, name string) (err error) {
|
||||
if strings.TrimSpace(directory) == "" || !moduleName.MatchString(name) || token.Lookup(name).IsKeyword() || name == "main" || name == "init" || name == "internal" {
|
||||
return errors.New("module name must be a non-keyword lowerCamelCase Go identifier (1..48 characters)")
|
||||
}
|
||||
root, err := filepath.Abs(directory)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
root, err = filepath.EvalSymlinks(root)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data, err := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
if err != nil {
|
||||
return fmt.Errorf("read project go.mod: %w", err)
|
||||
}
|
||||
manifest, err := modfile.Parse("go.mod", data, nil)
|
||||
if err != nil || manifest.Module == nil {
|
||||
return errors.New("project go.mod must declare a valid module")
|
||||
}
|
||||
found := false
|
||||
for _, dependency := range manifest.Require {
|
||||
if dependency.Mod.Path == frameworkModule {
|
||||
found = true
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return errors.New("project must already require go-omni; create it with go-omni new")
|
||||
}
|
||||
module := manifest.Module.Mod.Path
|
||||
if strings.ContainsAny(module, "\"\\\r\n") {
|
||||
return errors.New("invalid project module path")
|
||||
}
|
||||
paths := []string{"module.go", "model/model.go", "repository/repository.go", "service/service.go", "handler/handler.go", "README.md"}
|
||||
rendered := make(map[string][]byte, len(paths))
|
||||
for _, path := range paths {
|
||||
source, readErr := templateFiles.ReadFile("moduleTemplates/" + strings.ReplaceAll(path, "/", "-") + ".tmpl")
|
||||
if readErr != nil {
|
||||
return readErr
|
||||
}
|
||||
tmpl, parseErr := template.New(path).Parse(string(source))
|
||||
if parseErr != nil {
|
||||
return parseErr
|
||||
}
|
||||
var output bytes.Buffer
|
||||
if err := tmpl.Execute(&output, struct{ Module, Name string }{module, name}); err != nil {
|
||||
return err
|
||||
}
|
||||
content := output.Bytes()
|
||||
if strings.HasSuffix(path, ".go") {
|
||||
content, err = format.Source(content)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
rendered[path] = content
|
||||
}
|
||||
parent := filepath.Join(root, "internal")
|
||||
if info, statErr := os.Lstat(parent); statErr == nil {
|
||||
if !info.IsDir() || info.Mode()&os.ModeSymlink != 0 {
|
||||
return errors.New("internal must be a real directory, not a symlink")
|
||||
}
|
||||
} else if !errors.Is(statErr, os.ErrNotExist) {
|
||||
return statErr
|
||||
} else if err := os.Mkdir(parent, 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
target := filepath.Join(parent, name)
|
||||
// Exclusive creation protects existing files, including empty directories.
|
||||
if err := os.Mkdir(target, 0755); err != nil {
|
||||
return fmt.Errorf("create module (existing modules are never overwritten): %w", err)
|
||||
}
|
||||
defer func() {
|
||||
if err != nil {
|
||||
_ = os.RemoveAll(target)
|
||||
}
|
||||
}()
|
||||
for _, path := range paths {
|
||||
file := filepath.Join(target, filepath.FromSlash(path))
|
||||
if err = os.MkdirAll(filepath.Dir(file), 0755); err != nil {
|
||||
return err
|
||||
}
|
||||
if err = os.WriteFile(file, rendered[path], 0644); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
137
cmd/go-omni/scaffold/addModule_test.go
Normal file
@@ -0,0 +1,137 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func moduleProject(t *testing.T) string {
|
||||
t.Helper()
|
||||
root := filepath.Join(t.TempDir(), "billing")
|
||||
if err := Generate(Options{Dir: root, Module: "example.com/billing", FrameworkDir: frameworkRoot(t)}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return root
|
||||
}
|
||||
|
||||
func TestAddModulePreservesProjectAndRejectsOverwrite(t *testing.T) {
|
||||
root := moduleProject(t)
|
||||
manifest, _ := os.ReadFile(filepath.Join(root, "go.mod"))
|
||||
main, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go"))
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, path := range []string{"module.go", "model/model.go", "handler/handler.go", "service/service.go", "repository/repository.go", "README.md"} {
|
||||
if _, err := os.Stat(filepath.Join(root, "internal/orderItems", path)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
marker := filepath.Join(root, "internal/orderItems/keep.txt")
|
||||
if err := os.WriteFile(marker, []byte("keep"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orderItems"); err == nil {
|
||||
t.Fatal("overwrote existing module")
|
||||
}
|
||||
if got, _ := os.ReadFile(marker); string(got) != "keep" {
|
||||
t.Fatal("existing file changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "go.mod")); string(got) != string(manifest) {
|
||||
t.Fatal("go.mod changed")
|
||||
}
|
||||
if got, _ := os.ReadFile(filepath.Join(root, "cmd/server/main.go")); string(got) != string(main) {
|
||||
t.Fatal("main changed")
|
||||
}
|
||||
for _, name := range []string{"../escape", "Order", "order_items", "func", "main", "init", "internal", "", strings.Repeat("a", 49)} {
|
||||
if err := AddModule(root, name); err == nil {
|
||||
t.Fatalf("accepted %q", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAddModuleRejectsForeignProjectAndSymlink(t *testing.T) {
|
||||
root := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(root, "go.mod"), []byte("module example.com/foreign\ngo 1.22\n"), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted foreign project")
|
||||
}
|
||||
if _, err := os.Stat(filepath.Join(root, "internal")); !os.IsNotExist(err) {
|
||||
t.Fatal("created files before validation")
|
||||
}
|
||||
root = moduleProject(t)
|
||||
if err := os.Rename(filepath.Join(root, "internal"), filepath.Join(root, "original")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.Symlink(filepath.Join(root, "original"), filepath.Join(root, "internal")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := AddModule(root, "orders"); err == nil {
|
||||
t.Fatal("accepted symlink parent")
|
||||
}
|
||||
}
|
||||
|
||||
// Opt-in because this compiles a separate consuming module and may need module
|
||||
// downloads. The Linux feature acceptance runner enables it explicitly.
|
||||
func TestAddModuleIntegration(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-project CRUD")
|
||||
}
|
||||
root := moduleProject(t)
|
||||
if err := AddModule(root, "orderItems"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := os.WriteFile(filepath.Join(root, "internal/orderItems/module_test.go"), []byte(generatedModuleTest), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "./...")
|
||||
cmd.Dir = root
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("generated project: %v\n%s", err, output)
|
||||
}
|
||||
}
|
||||
|
||||
const generatedModuleTest = `package orderItems_test
|
||||
import (
|
||||
"context"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"example.com/billing/internal/orderItems"
|
||||
)
|
||||
func TestGeneratedCRUD(t *testing.T) {
|
||||
app:=omni.New()
|
||||
repo:=orderItems.NewMemory()
|
||||
orderItems.Register(app,repo,func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
if c.Header("Authorization")!="Bearer demo" {return omni.Unauthorized("DENIED","denied")};return next(c)
|
||||
}})
|
||||
request:=func(method,path,body string,authorized bool)*httptest.ResponseRecorder{
|
||||
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type","application/json")
|
||||
if authorized {r.Header.Set("Authorization","Bearer demo")}
|
||||
w:=httptest.NewRecorder();app.Engine().ServeHTTP(w,r);return w
|
||||
}
|
||||
if w:=request("POST","/orderItems","{\"name\":\"first\"}",false);w.Code!=401 {t.Fatal(w.Code)}
|
||||
cases:=[]struct{method,path,body string;status int;contains string}{
|
||||
{"POST","/orderItems","{\"name\":\" first \"}",201,"createdAt"},
|
||||
{"GET","/orderItems/1","",200,"first"},
|
||||
{"PUT","/orderItems/1","{\"name\":\"changed\"}",200,"changed"},
|
||||
{"GET","/orderItems","",200,"changed"},
|
||||
{"POST","/orderItems","{\"name\":\" \"}",400,"INVALID_NAME"},
|
||||
{"DELETE","/orderItems/1","",204,""},
|
||||
{"GET","/orderItems/1","",404,"NOT_FOUND"},
|
||||
}
|
||||
for _,tc:=range cases{w:=request(tc.method,tc.path,tc.body,true);if w.Code!=tc.status||!strings.Contains(w.Body.String(),tc.contains){t.Fatalf("%s %s: %d %s",tc.method,tc.path,w.Code,w.Body.String())}}
|
||||
ctx,cancel:=context.WithCancel(context.Background());cancel()
|
||||
if _,err:=repo.List(ctx);err!=context.Canceled {t.Fatal(err)}
|
||||
}
|
||||
`
|
||||
29
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
29
cmd/go-omni/scaffold/apiInfrastructure.go
Normal file
@@ -0,0 +1,29 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// APIInfrastructure renders shared infrastructure without inventing business
|
||||
// tables or routes. The API generator applies these as developer-owned seeds.
|
||||
func APIInfrastructure(module, database, storage, logOutput, logFormat string) (map[string][]byte, error) {
|
||||
data := templateData{Module: module, Database: database, Storage: storage, LogOutput: logOutput, LogFormat: logFormat, API: true, Modules: true}
|
||||
files := map[string]string{
|
||||
"internal/settings/config.go": "templates/modules-settings.go.tmpl",
|
||||
"etc/config.yaml": "templates/modules-config.yaml.tmpl",
|
||||
"internal/servicecontext/context.go": "templates/api-context.go.tmpl",
|
||||
}
|
||||
if storage != "none" {
|
||||
files["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||
}
|
||||
result := make(map[string][]byte, len(files))
|
||||
for name, source := range files {
|
||||
content, err := render(source, data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("render %s: %w", name, err)
|
||||
}
|
||||
result[name] = []byte(strings.TrimRight(content, "\n") + "\n")
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
143
cmd/go-omni/scaffold/moduleProfiles_test.go
Normal file
@@ -0,0 +1,143 @@
|
||||
package scaffold
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
type moduleProfile struct{ database, storage, output, format string }
|
||||
|
||||
var moduleProfiles = []moduleProfile{
|
||||
{"memory", "none", "stdout", "json"},
|
||||
{"memory", "local", "file", "json"},
|
||||
{"memory", "s3", "stdout", "text"},
|
||||
{"memory", "r2", "stderr", "json"},
|
||||
{"memory", "oss", "stderr", "text"},
|
||||
{"mysql", "none", "file", "text"},
|
||||
{"mysql", "local", "stdout", "json"},
|
||||
{"mysql", "s3", "file", "text"},
|
||||
{"mysql", "r2", "stderr", "text"},
|
||||
{"mysql", "oss", "file", "json"},
|
||||
}
|
||||
|
||||
func generateProfile(t *testing.T, p moduleProfile) string {
|
||||
t.Helper()
|
||||
project := filepath.Join(t.TempDir(), "project")
|
||||
if err := Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Database: p.database, Storage: p.storage, LogOutput: p.output, LogFormat: p.format}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return project
|
||||
}
|
||||
func TestGenerateModuleProfiles(t *testing.T) {
|
||||
for _, p := range moduleProfiles {
|
||||
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||
project := generateProfile(t, p)
|
||||
for _, name := range []string{"etc/config.yaml", "internal/settings/config.go", "MODULES.md", ".gitignore"} {
|
||||
if _, err := os.Stat(filepath.Join(project, name)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
_, err := os.Stat(filepath.Join(project, "internal/handler/files.go"))
|
||||
if p.storage == "none" && !os.IsNotExist(err) {
|
||||
t.Fatal("unselected storage was emitted")
|
||||
}
|
||||
if p.storage != "none" && err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
module, err := os.ReadFile(filepath.Join(project, "go.mod"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
pieces := strings.Split(string(module), "=> ")
|
||||
if len(pieces) != 2 {
|
||||
t.Fatal("local replace missing")
|
||||
}
|
||||
relative, err := strconv.Unquote(strings.TrimSpace(pieces[1]))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = os.Stat(filepath.Join(project, relative, "go.mod")); err != nil {
|
||||
t.Fatal("replacement does not resolve", err)
|
||||
}
|
||||
cfg, _ := os.ReadFile(filepath.Join(project, "etc/config.yaml"))
|
||||
if strings.Contains(string(cfg), "secretKey:") || strings.Contains(string(cfg), "filesToken:") {
|
||||
t.Fatal("secret defaults emitted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestModuleOptionsRejectBeforeWriting(t *testing.T) {
|
||||
for _, options := range []Options{
|
||||
{LogOutput: "invalid"}, {LogFormat: "xml"}, {Storage: "ftp"},
|
||||
{Storage: "local", FrameworkVersion: "v0.6.0-preview"},
|
||||
{Module: "example.com/x\nrequire malicious v1.0.0"},
|
||||
} {
|
||||
options.Dir = filepath.Join(t.TempDir(), "project")
|
||||
if options.Module == "" {
|
||||
options.Module = "example.com/profile"
|
||||
}
|
||||
if err := Generate(options); err == nil {
|
||||
t.Fatalf("invalid options accepted: %+v", options)
|
||||
}
|
||||
if _, err := os.Stat(options.Dir); !os.IsNotExist(err) {
|
||||
t.Fatal("invalid options touched target")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestCanonicalPathBeforeDirectoryExists(t *testing.T) {
|
||||
parent := t.TempDir()
|
||||
alias := filepath.Join(t.TempDir(), "alias")
|
||||
if err := os.Symlink(parent, alias); err != nil {
|
||||
t.Skip(err)
|
||||
}
|
||||
path := filepath.Join(alias, "new", "nested")
|
||||
if got, want := canonicalPath(path), filepath.Join(canonicalPath(parent), "new", "nested"); got != want {
|
||||
t.Fatalf("%s != %s", got, want)
|
||||
}
|
||||
}
|
||||
|
||||
// Compile consuming modules, then run their generated tests. No provider account
|
||||
// is used: cloud constructors are checked offline, file workflows use Local.
|
||||
func TestModuleProfilesIntegration(t *testing.T) {
|
||||
if os.Getenv("GO_OMNI_SCAFFOLD_INTEGRATION") != "1" {
|
||||
t.Skip("set GO_OMNI_SCAFFOLD_INTEGRATION=1 for generated-profile tests")
|
||||
}
|
||||
for _, p := range moduleProfiles {
|
||||
t.Run(p.database+"/"+p.storage, func(t *testing.T) {
|
||||
project := generateProfile(t, p)
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
|
||||
defer cancel()
|
||||
cmd := exec.CommandContext(ctx, "go", "test", "-mod=mod", "-race", "-p=2", "-count=1", "-timeout=60s", "./...")
|
||||
cmd.Dir = project
|
||||
cmd.Env = append(os.Environ(), "GOWORK=off", "GOMAXPROCS=2", "GO_OMNI_TEST_MYSQL_DSN=")
|
||||
if output, err := cmd.CombinedOutput(); err != nil {
|
||||
t.Fatalf("generated %s/%s: %v\n%s", p.database, p.storage, err, output)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestForceRejectsRemovedDependenciesWithoutTouchingCode(t *testing.T) {
|
||||
project := generateProfile(t, moduleProfile{"mysql", "local", "file", "json"})
|
||||
path := filepath.Join(project, "cmd/server/main.go")
|
||||
before, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Storage: "none"}); err == nil {
|
||||
t.Fatal("incompatible force overwrite accepted")
|
||||
}
|
||||
after, _ := os.ReadFile(path)
|
||||
if string(after) != string(before) {
|
||||
t.Fatal("failed profile switch changed code")
|
||||
}
|
||||
if err = Generate(Options{Dir: project, Module: "example.com/profile", FrameworkDir: frameworkRoot(t), Force: true, Database: "mysql", Storage: "local", LogOutput: "stdout"}); err != nil {
|
||||
t.Fatal("compatible force overwrite rejected", err)
|
||||
}
|
||||
}
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/README.md.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
# {{.Name}}
|
||||
|
||||
在业务启动文件导入并注册:
|
||||
|
||||
```go
|
||||
import "{{.Module}}/internal/{{.Name}}"
|
||||
|
||||
{{.Name}}.Register(application, {{.Name}}.NewMemory())
|
||||
```
|
||||
|
||||
提供 `POST /{{.Name}}`、`GET /{{.Name}}`、`GET /{{.Name}}/:id`、`PUT /{{.Name}}/:id`、
|
||||
`DELETE /{{.Name}}/:id`。写入 JSON 为 `{"name":"example"}`;输出字段为 id、name、createdAt。
|
||||
代码按 Handler / Service / Repository / Model 分层,`Register` 可注入自己的 Repository 和认证中间件。
|
||||
内存示例不持久化、不提供分页;列表适合小规模演示。接实际业务时定义领域字段、分页、鉴权和数据库实现。
|
||||
生成器不会自动修改主入口、已有模块或数据库结构,也不会覆盖已有文件。
|
||||
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
52
cmd/go-omni/scaffold/moduleTemplates/handler-handler.go.tmpl
Normal file
@@ -0,0 +1,52 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"errors"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Handler struct { service *service.Service }
|
||||
type WriteRequest struct { Name string `json:"name" binding:"required,max=200"` }
|
||||
func New(service *service.Service) *Handler {return &Handler{service:service}}
|
||||
func (h *Handler) Register(routes *omni.RouterGroup) {
|
||||
routes.POST("",h.Create)
|
||||
routes.GET("",h.List)
|
||||
routes.GET("/:id",h.Get)
|
||||
routes.PUT("/:id",h.Update)
|
||||
routes.DELETE("/:id",h.Delete)
|
||||
}
|
||||
func (h *Handler) Create(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Create(c.Context(),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.Created(item)
|
||||
}
|
||||
func (h *Handler) Get(c *omni.Context) error {
|
||||
item,err:=h.service.Get(c.Context(),c.Param("id"))
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) List(c *omni.Context) error {
|
||||
items,err:=h.service.List(c.Context())
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(items)
|
||||
}
|
||||
func (h *Handler) Update(c *omni.Context) error {
|
||||
var request WriteRequest
|
||||
if err:=c.BindJSON(&request); err!=nil {return err}
|
||||
item,err:=h.service.Update(c.Context(),c.Param("id"),request.Name)
|
||||
if err!=nil {return publicError(err)}
|
||||
return c.OK(item)
|
||||
}
|
||||
func (h *Handler) Delete(c *omni.Context) error {
|
||||
if err:=h.service.Delete(c.Context(),c.Param("id")); err!=nil {return publicError(err)}
|
||||
return c.NoContent()
|
||||
}
|
||||
func publicError(err error) error {
|
||||
if errors.Is(err,model.ErrNotFound) {return omni.NotFound("NOT_FOUND","record not found").WithCause(err)}
|
||||
if errors.Is(err,model.ErrInvalidName) {return omni.BadRequest("INVALID_NAME","name must contain 1..200 bytes").WithCause(err)}
|
||||
return err
|
||||
}
|
||||
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
15
cmd/go-omni/scaffold/moduleTemplates/model-model.go.tmpl
Normal file
@@ -0,0 +1,15 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"time"
|
||||
)
|
||||
|
||||
var ErrNotFound = errors.New("{{.Name}}: record not found")
|
||||
var ErrInvalidName = errors.New("{{.Name}}: name must contain 1..200 bytes")
|
||||
|
||||
type Item struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
CreatedAt time.Time `json:"createdAt"`
|
||||
}
|
||||
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
19
cmd/go-omni/scaffold/moduleTemplates/module.go.tmpl
Normal file
@@ -0,0 +1,19 @@
|
||||
package {{.Name}}
|
||||
|
||||
import (
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"{{.Module}}/internal/{{.Name}}/handler"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
"{{.Module}}/internal/{{.Name}}/service"
|
||||
)
|
||||
|
||||
type Repository = repository.Repository
|
||||
|
||||
func NewMemory() Repository { return repository.NewMemory() }
|
||||
|
||||
// Register wires this module with caller-owned storage. Pass auth middleware
|
||||
// here to apply it to every route before module-specific middleware.
|
||||
func Register(app *omni.App, repo Repository, middleware ...omni.Middleware) {
|
||||
if repo == nil { panic("{{.Name}}: repository is required") }
|
||||
handler.New(service.New(repo)).Register(app.Group("/{{.Name}}", middleware...))
|
||||
}
|
||||
@@ -0,0 +1,60 @@
|
||||
package repository
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sort"
|
||||
"strconv"
|
||||
"sync"
|
||||
"time"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
)
|
||||
|
||||
type Repository interface {
|
||||
Create(context.Context,string) (model.Item,error)
|
||||
Get(context.Context,string) (model.Item,error)
|
||||
List(context.Context) ([]model.Item,error)
|
||||
Update(context.Context,string,string) (model.Item,error)
|
||||
Delete(context.Context,string) error
|
||||
}
|
||||
|
||||
// Memory is demonstration storage; data and IDs reset with the process.
|
||||
type Memory struct { mu sync.Mutex; next uint64; items map[string]model.Item }
|
||||
func NewMemory() *Memory { return &Memory{items:make(map[string]model.Item)} }
|
||||
func (m *Memory) Create(ctx context.Context, name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
m.next++
|
||||
item:=model.Item{ID:strconv.FormatUint(m.next,10),Name:name,CreatedAt:time.Now().UTC()}
|
||||
m.items[item.ID]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Get(ctx context.Context,id string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) List(ctx context.Context) ([]model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return nil,err}
|
||||
items:=make([]model.Item,0,len(m.items))
|
||||
for _,item:=range m.items {items=append(items,item)}
|
||||
sort.Slice(items,func(i,j int)bool{return items[i].ID<items[j].ID})
|
||||
return items,nil
|
||||
}
|
||||
func (m *Memory) Update(ctx context.Context,id,name string) (model.Item,error) {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return model.Item{},err}
|
||||
item,ok:=m.items[id]
|
||||
if !ok {return model.Item{},model.ErrNotFound}
|
||||
item.Name=name; m.items[id]=item
|
||||
return item,nil
|
||||
}
|
||||
func (m *Memory) Delete(ctx context.Context,id string) error {
|
||||
m.mu.Lock(); defer m.mu.Unlock()
|
||||
if err:=ctx.Err(); err!=nil {return err}
|
||||
if _,ok:=m.items[id]; !ok {return model.ErrNotFound}
|
||||
delete(m.items,id)
|
||||
return nil
|
||||
}
|
||||
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
27
cmd/go-omni/scaffold/moduleTemplates/service-service.go.tmpl
Normal file
@@ -0,0 +1,27 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"strings"
|
||||
"{{.Module}}/internal/{{.Name}}/model"
|
||||
"{{.Module}}/internal/{{.Name}}/repository"
|
||||
)
|
||||
|
||||
type Service struct { repo repository.Repository }
|
||||
func New(repo repository.Repository) *Service { return &Service{repo:repo} }
|
||||
func name(value string) (string,error) {
|
||||
value=strings.TrimSpace(value)
|
||||
if len(value)==0 || len(value)>200 {return "",model.ErrInvalidName}
|
||||
return value,nil
|
||||
}
|
||||
func (s *Service) Create(ctx context.Context,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Create(ctx,value)
|
||||
}
|
||||
func (s *Service) Get(ctx context.Context,id string) (model.Item,error) { return s.repo.Get(ctx,id) }
|
||||
func (s *Service) List(ctx context.Context) ([]model.Item,error) { return s.repo.List(ctx) }
|
||||
func (s *Service) Update(ctx context.Context,id,value string) (model.Item,error) {
|
||||
value,err:=name(value); if err!=nil {return model.Item{},err}
|
||||
return s.repo.Update(ctx,id,value)
|
||||
}
|
||||
func (s *Service) Delete(ctx context.Context,id string) error {return s.repo.Delete(ctx,id)}
|
||||
@@ -12,9 +12,12 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"text/template"
|
||||
|
||||
"golang.org/x/mod/module"
|
||||
"golang.org/x/mod/semver"
|
||||
)
|
||||
|
||||
//go:embed templates
|
||||
//go:embed templates moduleTemplates
|
||||
var templateFiles embed.FS
|
||||
|
||||
// Options controls project generation.
|
||||
@@ -24,6 +27,9 @@ type Options struct {
|
||||
FrameworkDir string
|
||||
FrameworkVersion string
|
||||
Database string
|
||||
LogOutput string
|
||||
LogFormat string
|
||||
Storage string
|
||||
Force bool
|
||||
}
|
||||
|
||||
@@ -48,6 +54,31 @@ func Generate(options Options) error {
|
||||
if strings.TrimSpace(options.Module) == "" {
|
||||
return errors.New("module path is required")
|
||||
}
|
||||
if err := module.CheckImportPath(options.Module); err != nil {
|
||||
return fmt.Errorf("invalid module path: %w", err)
|
||||
}
|
||||
modules := options.LogOutput != "" || options.LogFormat != "" || options.Storage != ""
|
||||
logOutput, logFormat, storage := options.LogOutput, options.LogFormat, options.Storage
|
||||
if logOutput == "" {
|
||||
logOutput = "stderr"
|
||||
}
|
||||
if logFormat == "" {
|
||||
logFormat = "json"
|
||||
}
|
||||
if storage == "" {
|
||||
storage = "none"
|
||||
}
|
||||
if logOutput != "stdout" && logOutput != "stderr" && logOutput != "file" {
|
||||
return fmt.Errorf("unsupported log output %q: choose stdout, stderr or file", logOutput)
|
||||
}
|
||||
if logFormat != "json" && logFormat != "text" {
|
||||
return fmt.Errorf("unsupported log format %q: choose json or text", logFormat)
|
||||
}
|
||||
switch storage {
|
||||
case "none", "local", "s3", "r2", "oss":
|
||||
default:
|
||||
return fmt.Errorf("unsupported object storage %q: choose none, local, s3, r2 or oss", storage)
|
||||
}
|
||||
database := strings.ToLower(strings.TrimSpace(options.Database))
|
||||
if database == "" {
|
||||
database = "memory"
|
||||
@@ -58,6 +89,9 @@ func Generate(options Options) error {
|
||||
version := strings.TrimSpace(options.FrameworkVersion)
|
||||
if version == "" {
|
||||
version = currentFrameworkVersion()
|
||||
if modules && version == defaultFrameworkVersion {
|
||||
version = modulesFrameworkVersion
|
||||
}
|
||||
}
|
||||
if !validFrameworkVersion(version) {
|
||||
return fmt.Errorf("framework version %q must be a complete v0 or v1 semantic version", version)
|
||||
@@ -70,20 +104,20 @@ func Generate(options Options) error {
|
||||
if err := checkTarget(directory, options.Force); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := os.MkdirAll(directory, 0o755); err != nil {
|
||||
return fmt.Errorf("create project directory: %w", err)
|
||||
}
|
||||
|
||||
// An explicit remote version opts out of automatic local development mode.
|
||||
replace := ""
|
||||
if options.FrameworkDir != "" || options.FrameworkVersion == "" {
|
||||
replace = frameworkReplace(directory, options.FrameworkDir)
|
||||
}
|
||||
if modules && replace == "" && semver.Compare(version, modulesFrameworkVersion) < 0 && version != installedFrameworkVersion() {
|
||||
return fmt.Errorf("module templates require %s or newer; use --framework-dir for unpublished source", modulesFrameworkVersion)
|
||||
}
|
||||
data := templateData{
|
||||
Module: options.Module,
|
||||
FrameworkVersion: version,
|
||||
FrameworkReplace: replace,
|
||||
Database: database,
|
||||
Modules: modules, LogOutput: logOutput, LogFormat: logFormat, Storage: storage,
|
||||
}
|
||||
projectFiles := make(map[string]string, len(files)+3)
|
||||
for path, source := range files {
|
||||
@@ -94,11 +128,42 @@ func Generate(options Options) error {
|
||||
projectFiles["schema.sql"] = "templates/schema.sql.tmpl"
|
||||
projectFiles["internal/servicecontext/databaseIntegration_test.go"] = "templates/databaseIntegration_test.go.tmpl"
|
||||
}
|
||||
if modules {
|
||||
projectFiles["cmd/server/main.go"] = "templates/modules-main.go.tmpl"
|
||||
projectFiles["internal/settings/config.go"] = "templates/modules-settings.go.tmpl"
|
||||
projectFiles["internal/settings/config_test.go"] = "templates/modules-settings_test.go.tmpl"
|
||||
projectFiles["etc/config.yaml"] = "templates/modules-config.yaml.tmpl"
|
||||
projectFiles[".gitignore"] = "templates/modules-gitignore.tmpl"
|
||||
projectFiles["MODULES.md"] = "templates/modules-README.md.tmpl"
|
||||
if storage != "none" {
|
||||
projectFiles["internal/servicecontext/storage.go"] = "templates/modules-storage.go.tmpl"
|
||||
projectFiles["internal/servicecontext/storage_test.go"] = "templates/modules-storage_test.go.tmpl"
|
||||
projectFiles["internal/service/files.go"] = "templates/modules-filesService.go.tmpl"
|
||||
projectFiles["internal/handler/files.go"] = "templates/modules-filesHandler.go.tmpl"
|
||||
projectFiles["internal/handler/files_test.go"] = "templates/modules-filesHandler_test.go.tmpl"
|
||||
}
|
||||
}
|
||||
if options.Force {
|
||||
// Do not leave old generated Go files that reference dependencies omitted
|
||||
// by this profile, and do not silently delete potentially edited code.
|
||||
for _, path := range []string{"internal/repository/mysql.go", "internal/servicecontext/databaseIntegration_test.go", "schema.sql", "internal/settings/config.go", "internal/settings/config_test.go", "etc/config.yaml", "MODULES.md", "internal/servicecontext/storage.go", "internal/servicecontext/storage_test.go", "internal/service/files.go", "internal/handler/files.go", "internal/handler/files_test.go"} {
|
||||
if _, emitted := projectFiles[path]; emitted {
|
||||
continue
|
||||
}
|
||||
if _, err := os.Lstat(filepath.Join(directory, path)); err == nil {
|
||||
return fmt.Errorf("existing %s conflicts with the selected profile; generate into a new directory instead of removing existing code", path)
|
||||
} else if !errors.Is(err, os.ErrNotExist) {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
paths := make([]string, 0, len(projectFiles))
|
||||
for path := range projectFiles {
|
||||
paths = append(paths, path)
|
||||
}
|
||||
sort.Strings(paths)
|
||||
// Render and format the whole project before touching the target directory.
|
||||
contents := make(map[string]string, len(paths))
|
||||
for _, path := range paths {
|
||||
content, err := render(projectFiles[path], data)
|
||||
if err != nil {
|
||||
@@ -111,6 +176,9 @@ func Generate(options Options) error {
|
||||
}
|
||||
content = string(formatted)
|
||||
}
|
||||
contents[path] = content
|
||||
}
|
||||
for _, path := range paths {
|
||||
target := filepath.Join(directory, filepath.FromSlash(path))
|
||||
if err := os.MkdirAll(filepath.Dir(target), 0o755); err != nil {
|
||||
return fmt.Errorf("create %s: %w", filepath.Dir(path), err)
|
||||
@@ -122,7 +190,7 @@ func Generate(options Options) error {
|
||||
return fmt.Errorf("check %s: %w", target, err)
|
||||
}
|
||||
}
|
||||
if err := os.WriteFile(target, []byte(content), 0o644); err != nil {
|
||||
if err := os.WriteFile(target, []byte(contents[path]), 0o644); err != nil {
|
||||
return fmt.Errorf("write %s: %w", target, err)
|
||||
}
|
||||
}
|
||||
@@ -130,10 +198,15 @@ func Generate(options Options) error {
|
||||
}
|
||||
|
||||
type templateData struct {
|
||||
API bool
|
||||
Module string
|
||||
FrameworkVersion string
|
||||
FrameworkReplace string
|
||||
Database string
|
||||
Modules bool
|
||||
LogOutput string
|
||||
LogFormat string
|
||||
Storage string
|
||||
}
|
||||
|
||||
func render(name string, data templateData) (string, error) {
|
||||
@@ -202,11 +275,28 @@ func canonicalPath(path string) string {
|
||||
if err != nil {
|
||||
return path
|
||||
}
|
||||
resolved, err := filepath.EvalSymlinks(abs)
|
||||
if err == nil {
|
||||
return resolved
|
||||
// Resolve the nearest existing ancestor too: generation now renders before
|
||||
// creating directories, and macOS /var commonly resolves through /private.
|
||||
candidate := abs
|
||||
var tail []string
|
||||
for {
|
||||
resolved, err := filepath.EvalSymlinks(candidate)
|
||||
if err == nil {
|
||||
for i := len(tail) - 1; i >= 0; i-- {
|
||||
resolved = filepath.Join(resolved, tail[i])
|
||||
}
|
||||
return resolved
|
||||
}
|
||||
if !errors.Is(err, os.ErrNotExist) {
|
||||
return abs
|
||||
}
|
||||
parent := filepath.Dir(candidate)
|
||||
if parent == candidate {
|
||||
return abs
|
||||
}
|
||||
tail = append(tail, filepath.Base(candidate))
|
||||
candidate = parent
|
||||
}
|
||||
return abs
|
||||
}
|
||||
|
||||
func findFrameworkRoot() string {
|
||||
|
||||
@@ -1,10 +1,13 @@
|
||||
# GoOmni service
|
||||
|
||||
This service was generated with `go-omni new`.
|
||||
{{if .Modules}}
|
||||
**本项目启用了模块化配置,启动步骤和环境变量请先看 [MODULES.md](MODULES.md)。** 配置来自 `etc/config.yaml`,文件模块需要配置环境密钥后启动。
|
||||
{{end}}
|
||||
|
||||
Framework dependency: `git.yuntuo.hk/k2safe/go-omni {{.FrameworkVersion}}`.
|
||||
|
||||
Storage: `{{.Database}}`.
|
||||
Database: `{{.Database}}`.
|
||||
|
||||
{{if eq .Database "mysql"}}
|
||||
先创建业务数据库,再执行项目的建表脚本(替换示例数据库名、账号和地址):
|
||||
@@ -47,6 +50,11 @@ go build -o server ./cmd/server
|
||||
```
|
||||
|
||||
`GO_OMNI_SHUTDOWN_TIMEOUT` 控制等待请求完成的时间,默认 `10s`。
|
||||
|
||||
使用支持统一网络配置的框架版本时,设置 `GO_OMNI_TRANSPORT=native` 即可切换为自研 HTTP/1.1;
|
||||
默认 `nethttp`。`GO_OMNI_TLS_CERT_FILE` 和 `GO_OMNI_TLS_KEY_FILE` 配对启用 HTTPS。
|
||||
实验版 HTTP/2 使用 `GO_OMNI_TRANSPORT=http2`,必须同时配置 TLS 证书和私钥。
|
||||
业务 Handler / Service / Repository 不需要修改;三种适配器共用上述退出等待时间。
|
||||
{{if eq .Database "mysql"}}
|
||||
`/health/ready` 已注册数据库探针:连接失败返回 `503`,数据库恢复后重新返回 `200`。
|
||||
`/health/live` 始终只检查服务存活。`GO_OMNI_DB_PING_TIMEOUT` 限制初始化检查和数据库探针的等待时间。
|
||||
|
||||
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
69
cmd/go-omni/scaffold/templates/api-context.go.tmpl
Normal file
@@ -0,0 +1,69 @@
|
||||
package servicecontext
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sync"
|
||||
{{if ne .Storage "none"}}"net/http"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if eq .Database "mysql"}}"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
"github.com/go-sql-driver/mysql"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/health"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/settings"
|
||||
"{{.Module}}/internal/middleware"
|
||||
)
|
||||
// Context owns infrastructure, while Logger is borrowed from main.
|
||||
// Add business repositories here; HTTP contracts do not define database tables.
|
||||
type Context struct {
|
||||
Logger *logx.Logger
|
||||
Health *health.Checker
|
||||
Middlewares map[string]httpx.Middleware
|
||||
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||
{{if ne .Storage "none"}}
|
||||
Files storage.Store
|
||||
Uploads *storage.Uploader
|
||||
Downloads storage.Presigner
|
||||
UploadLinks storage.UploadPresigner
|
||||
FileHTTP http.Handler
|
||||
storageClose func()error
|
||||
{{end}}
|
||||
closeMu sync.Mutex
|
||||
closed bool
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
}
|
||||
// Pass the settings loaded by main. The optional form supports lightweight
|
||||
// tests without infrastructure; selected adapters still require valid settings.
|
||||
func New(ctx context.Context,logger *logx.Logger,configs ...settings.Config)(result *Context,err error){
|
||||
if ctx==nil{ctx=context.Background()}
|
||||
if err=ctx.Err();err!=nil{return nil,err}
|
||||
if len(configs)>1{return nil,errors.New("expected at most one configuration")}
|
||||
cfg:=settings.Default();if len(configs)==1{cfg=configs[0]}
|
||||
_=cfg
|
||||
checker,err:=health.New();if err!=nil{return nil,err}
|
||||
c:=&Context{Logger:logger,Health:checker,Middlewares:middleware.Defaults()}
|
||||
defer func(){if err!=nil{err=errors.Join(err,c.Close())}}()
|
||||
{{if eq .Database "mysql"}}
|
||||
if err=cfg.Validate();err!=nil{return nil,err}
|
||||
dsn,parseErr:=mysql.ParseDSN(cfg.Database.DSN)
|
||||
if parseErr!=nil || dsn.DBName=="" {return nil,errors.New("GO_OMNI_DB_DSN must contain a valid MySQL database name")}
|
||||
dsn.ParseTime=true;dsn.Loc=time.UTC
|
||||
cfg.Database.DSN=dsn.FormatDSN()
|
||||
c.DB,err=database.Open(ctx,cfg.Database);if err!=nil{return nil,err}
|
||||
if err=checker.Add(health.Probe{Name:"database",Timeout:cfg.Database.PingTimeout,Check:c.DB.Health});err!=nil{return nil,err}
|
||||
{{end}}
|
||||
{{if ne .Storage "none"}}if err=c.InitStorage(ctx,cfg.Storage);err!=nil{return nil,err}{{end}}
|
||||
return c,nil
|
||||
}
|
||||
// Close is idempotent, including startup failure and graceful shutdown.
|
||||
func(c *Context)Close()error{
|
||||
if c==nil{return nil}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock();c.closed=true
|
||||
c.closeOnce.Do(func(){
|
||||
{{if ne .Storage "none"}}if c.storageClose!=nil{c.closeErr=errors.Join(c.closeErr,c.storageClose())}{{end}}
|
||||
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||
})
|
||||
return c.closeErr
|
||||
}
|
||||
@@ -23,7 +23,10 @@ func (s *Users) Create(ctx context.Context, input model.UserInput) (model.User,
|
||||
if err != nil {
|
||||
return model.User{}, err
|
||||
}
|
||||
return s.services.Users.Create(ctx, input)
|
||||
user, err := s.services.Users.Create(ctx, input)
|
||||
if err != nil { s.services.Logger.ErrorContext(ctx,"user.create.failed","error",err); return model.User{},err }
|
||||
s.services.Logger.InfoContext(ctx,"user.created","userId",user.ID)
|
||||
return user,nil
|
||||
}
|
||||
|
||||
func (s *Users) Get(ctx context.Context, id int64) (model.User, error) {
|
||||
|
||||
@@ -2,6 +2,10 @@ package servicecontext
|
||||
|
||||
import (
|
||||
"context"
|
||||
{{if and .Modules (ne .Storage "none")}}"errors"
|
||||
"net/http"
|
||||
"sync"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if eq .Database "mysql"}}"fmt"
|
||||
"time"
|
||||
"git.yuntuo.hk/k2safe/go-omni/database"
|
||||
@@ -17,6 +21,18 @@ type Context struct {
|
||||
Logger *logx.Logger
|
||||
Users repository.UserRepository
|
||||
Health *health.Checker
|
||||
{{if and .Modules (ne .Storage "none")}}
|
||||
Files storage.Store
|
||||
Uploads *storage.Uploader
|
||||
Downloads storage.Presigner
|
||||
UploadLinks storage.UploadPresigner
|
||||
FileHTTP http.Handler
|
||||
storageClose func() error
|
||||
closeMu sync.Mutex
|
||||
closed bool
|
||||
closeOnce sync.Once
|
||||
closeErr error
|
||||
{{end}}
|
||||
{{if eq .Database "mysql"}}DB *database.DB{{end}}
|
||||
}
|
||||
|
||||
@@ -61,6 +77,15 @@ func New(ctx context.Context, logger *logx.Logger) (*Context, error) {
|
||||
|
||||
// Close is safe to call on startup failure and again during application exit.
|
||||
func (c *Context) Close() error {
|
||||
{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}
|
||||
return nil
|
||||
{{if and .Modules (ne .Storage "none")}}
|
||||
if c==nil{return nil}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
c.closed=true
|
||||
c.closeOnce.Do(func(){
|
||||
if c.storageClose!=nil {c.closeErr=errors.Join(c.closeErr,c.storageClose())}
|
||||
{{if eq .Database "mysql"}}if c.DB!=nil{c.closeErr=errors.Join(c.closeErr,c.DB.Close())}{{end}}
|
||||
})
|
||||
return c.closeErr
|
||||
{{else}}{{if eq .Database "mysql"}}if c != nil && c.DB != nil { return c.DB.Close() }{{end}}{{end}}
|
||||
{{if not (and .Modules (ne .Storage "none"))}}return nil{{end}}
|
||||
}
|
||||
|
||||
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
97
cmd/go-omni/scaffold/templates/modules-README.md.tmpl
Normal file
@@ -0,0 +1,97 @@
|
||||
# 按需装配的日志与文件模块
|
||||
|
||||
本项目选用日志输出 `{{.LogOutput}}`、格式 `{{.LogFormat}}`、对象存储 `{{.Storage}}`。数据库选择 `{{.Database}}` 独立于对象存储。
|
||||
|
||||
配置入口为 `etc/config.yaml`,通过 `go run ./cmd/server -config /path/to/config.yaml` 指定其他 JSON/YAML 文件;`-config ''` 只用代码默认值和环境变量。优先级:默认值 → 文件 → 非空环境变量。未知字段、无效参数会在监听前拒绝。
|
||||
|
||||
```sh
|
||||
go mod tidy
|
||||
{{if ne .Storage "none"}}# 示例文件接口使用单一共享 Token,不是用户/租户权限系统。
|
||||
export GO_OMNI_FILES_TOKEN="$(openssl rand -hex 32)"
|
||||
{{if eq .Storage "local"}}# 本地签名密钥需要稳定保存;换密钥后旧链接失效。
|
||||
export GO_OMNI_STORAGE_SIGNING_KEY="$(openssl rand -hex 32)"
|
||||
{{else}}# 配置当前存储平台已有的桶和凭据,不会自动创建桶。
|
||||
export GO_OMNI_STORAGE_BUCKET='your-bucket'
|
||||
export GO_OMNI_STORAGE_ACCESS_KEY='your-access-key'
|
||||
export GO_OMNI_STORAGE_SECRET_KEY='your-secret-key'
|
||||
{{if eq .Storage "s3"}}export GO_OMNI_STORAGE_ENDPOINT='s3.us-east-1.amazonaws.com'
|
||||
export GO_OMNI_STORAGE_REGION='us-east-1'
|
||||
{{else if eq .Storage "r2"}}export GO_OMNI_STORAGE_ACCOUNT_ID='your-32-character-account-id'
|
||||
{{else}}export GO_OMNI_STORAGE_REGION='cn-hangzhou'
|
||||
{{end}}{{end}}{{end}}
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
{{if eq .Database "mysql"}}还需按 README 配置独立 MySQL 数据库并应用 schema.sql;应用不自动迁移。{{end}}
|
||||
|
||||
默认仅监听 `127.0.0.1:8080`。`GO_OMNI_HTTP_ADDR` 覆盖地址;本地签名 URL 的外部地址通过 `GO_OMNI_STORAGE_PUBLIC_URL` 单独配置,始终以 `/storage/object` 结尾,不使用请求 Host 自动拼接可信链接。
|
||||
|
||||
## 日志
|
||||
|
||||
业务示例在创建用户/上传文件成功时通过 logx 写结构化日志。Context 原样传递 requestId,框架不自动增加用户、租户等业务字段。文件模式已配置轮转:100 MiB、10 份、7 天、压缩。调整 `server.log` 即可更改;`redactKeys` 可增加业务敏感键。
|
||||
|
||||
```sh
|
||||
curl -H 'Content-Type: application/json' -H 'X-Request-ID: example-request' \
|
||||
-d '{"name":"Example","email":"example@example.com"}' http://127.0.0.1:8080/users
|
||||
```
|
||||
|
||||
`GO_OMNI_LOG_LEVEL`、`GO_OMNI_LOG_FORMAT`、`GO_OMNI_LOG_FILE` 覆盖对应项;显式设置 `GO_OMNI_LOG_OUTPUT=stdout` 且未设置 LOG_FILE 时,切换到控制台并关闭文件轮转。慢请求阈值为 500ms,可通过 `GO_OMNI_SLOW_REQUEST_THRESHOLD` 调整。字段脱敏不扫描 msg/error 中的自由文本;不要把凭据或签名 URL 拼到日志中。
|
||||
|
||||
{{if ne .Storage "none"}}## 文件示例
|
||||
|
||||
Handler → Service → storage 接口;底层构造与连接池集中在 `internal/servicecontext/storage.go`。示例接口要求 `Authorization: Bearer $GO_OMNI_FILES_TOKEN`,它授权这个示例的全部文件,实际业务应替换认证和对象归属检查。存储适配器不承担业务授权。
|
||||
|
||||
```sh
|
||||
printf 'hello go-omni' > /tmp/goOmniExample.txt
|
||||
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" \
|
||||
-H 'Content-Type: text/plain' -H 'X-Request-ID: file-example' \
|
||||
--data-binary @/tmp/goOmniExample.txt http://127.0.0.1:8080/files
|
||||
```
|
||||
|
||||
保存响应中 `data.key`,再请求下载链接(把示例 key 替换为返回值):
|
||||
|
||||
```sh
|
||||
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||
-d '{"key":"uploads/returned-key"}' http://127.0.0.1:8080/files/downloadLink
|
||||
```
|
||||
|
||||
返回 `data.url`,在有效期内 GET 即可下载,无需再携带 Token。签发前会检查对象存在;URL 默认 5 分钟有效,不写日志,响应设置 no-store。
|
||||
|
||||
需要直传时先申请链接:
|
||||
|
||||
```sh
|
||||
curl -H "Authorization: Bearer $GO_OMNI_FILES_TOKEN" -H 'Content-Type: application/json' \
|
||||
-d '{"size":13,"contentType":"text/plain"}' http://127.0.0.1:8080/files/uploadLink
|
||||
```
|
||||
|
||||
响应包括随机 `data.key` 和 `data.request`(method/url/headers/expiresAt)。按返回的 method 和 headers 用原始文件字节上传到 URL;不能额外包 FormData。浏览器自动填写 Content-Length,其余签名 Header 原样发送;云桶需配置允许业务域名的 CORS。云端直传只绑定声明类型,实际内容需上传后验证;本地和 API 中转上传会检查允许类型及前缀嗅探结果。未完成内容验证前不要把暂存文件当作可信文件公开。
|
||||
|
||||
| 参数 | 默认值 / 环境变量 |
|
||||
| --- | --- |
|
||||
| 大小 | 32 MiB / GO_OMNI_STORAGE_MAX_BYTES |
|
||||
| 类型 | text/plain、image/png、image/jpeg、application/pdf / GO_OMNI_STORAGE_CONTENT_TYPES(逗号分隔) |
|
||||
| 链接有效期 | 5m / GO_OMNI_STORAGE_LINK_TTL(1s..1h) |
|
||||
| 请求超时 | 30s / GO_OMNI_STORAGE_REQUEST_TIMEOUT(>0 且 <=5m) |
|
||||
| 本地目录 | var/files / GO_OMNI_STORAGE_ROOT |
|
||||
| 本地外部 URL | http://127.0.0.1:8080/storage/object / GO_OMNI_STORAGE_PUBLIC_URL |
|
||||
| 云配置 | GO_OMNI_STORAGE_ENDPOINT、REGION、BUCKET、ACCOUNT_ID、SECURE |
|
||||
| 环境密钥 | GO_OMNI_FILES_TOKEN、GO_OMNI_STORAGE_SIGNING_KEY、GO_OMNI_STORAGE_ACCESS_KEY、SECRET_KEY、SESSION_TOKEN |
|
||||
|
||||
全部 JSON/YAML 字段使用 lowerCamelCase。密钥字段仅允许环境变量,配置文件中写入会报未知字段;不要把运行环境文件提交到仓库。S3 的 endpoint 为 host[:port],OSS 的可选 endpoint 为 HTTPS origin;R2 从 accountId 派生官方 endpoint,region 为 auto。仅 S3/MinIO 的本地测试可设置 SECURE=false。
|
||||
|
||||
错误响应:未授权 401、无 Content-Length 411、超限 413、类型不允许 415、长度不符/非法 key 400、对象不存在 404、存储超时 504,其余内部错误统一隐藏细节。底层签名入口遵循存储协议返回 201/403 等状态,流式传输且访问日志保留实际状态。
|
||||
|
||||
云客户端只在组合处创建,使用专用有界连接池;配置正确不代表桶和网络已经可用,首次真实调用仍可能失败。不创建桶、不修改桶权限、不初始化任何业务数据库。更换平台时替换组合处适配器,业务 Service 接口保持不变。
|
||||
{{end}}
|
||||
|
||||
## 生命周期与验证
|
||||
|
||||
启动失败会清理已创建资源;SIGINT/SIGTERM 后先等待 HTTP 请求退出,再关闭服务依赖,最后关闭日志。只关闭项目自己创建的资源,不关闭全局默认 HTTP Transport。
|
||||
|
||||
```sh
|
||||
go test -race ./...
|
||||
go build -o server ./cmd/server
|
||||
./server
|
||||
```
|
||||
|
||||
生成的测试覆盖配置优先级/校验{{if ne .Storage "none"}}、资源关闭幂等性和借用日志所有权、上传与临时链接、签名篡改、限额/类型错误。业务测试使用临时 Local,不访问云账号;云端配置需另做实网验收{{end}}。基础用户 CRUD 示例仍未启用认证,部署前按业务需要替换。
|
||||
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
46
cmd/go-omni/scaffold/templates/modules-config.yaml.tmpl
Normal file
@@ -0,0 +1,46 @@
|
||||
# Non-secret settings only. Environment variables override these values.
|
||||
server:
|
||||
address: '127.0.0.1:8080'
|
||||
enableLogger: true
|
||||
slowRequestThreshold: 500ms
|
||||
shutdownTimeout: 10s
|
||||
log:
|
||||
service: {{printf "%q" .Module}}
|
||||
level: info
|
||||
format: {{.LogFormat}}
|
||||
{{if eq .LogOutput "file"}} file: './var/log/server.log'
|
||||
rotation:
|
||||
enabled: true
|
||||
maxSizeMb: 100
|
||||
maxBackups: 10
|
||||
maxAgeDays: 7
|
||||
compress: true
|
||||
{{else}} output: {{.LogOutput}}
|
||||
{{end}} redactKeys: []
|
||||
{{if ne .Storage "none"}}storage:
|
||||
provider: {{.Storage}}
|
||||
maxBytes: 33554432
|
||||
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||
linkTtl: 5m
|
||||
requestTimeout: 30s
|
||||
{{if eq .Storage "local"}} root: './var/files'
|
||||
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||
{{else}} bucket: '' # set GO_OMNI_STORAGE_BUCKET
|
||||
secure: true
|
||||
{{if eq .Storage "r2"}} accountId: '' # set GO_OMNI_STORAGE_ACCOUNT_ID
|
||||
region: auto
|
||||
{{else if eq .Storage "s3"}} endpoint: '' # set GO_OMNI_STORAGE_ENDPOINT (host[:port])
|
||||
region: us-east-1
|
||||
{{else}} region: cn-hangzhou
|
||||
endpoint: '' # empty uses the official regional endpoint
|
||||
{{end}}{{end}}{{end}}
|
||||
|
||||
{{if and .API (eq .Database "mysql")}}database:
|
||||
driver: mysql
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||
{{end}}
|
||||
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
86
cmd/go-omni/scaffold/templates/modules-filesHandler.go.tmpl
Normal file
@@ -0,0 +1,86 @@
|
||||
package handler
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/subtle"
|
||||
"errors"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"{{.Module}}/internal/service"
|
||||
)
|
||||
|
||||
type Files struct {service *service.Files; maxBytes int64; timeout time.Duration}
|
||||
func NewFiles(service *service.Files,maxBytes int64,timeout time.Duration)*Files{return &Files{service:service,maxBytes:maxBytes,timeout:timeout}}
|
||||
|
||||
// RegisterRoutes protects example control endpoints with one shared demo token.
|
||||
// Replace this gate with project authentication and object ownership checks.
|
||||
// Signed object URLs carry their own short-lived authorization.
|
||||
func (h *Files) RegisterRoutes(application *omni.App,token string,signed http.Handler)error {
|
||||
if len(token)<32{return errors.New("file example requires a token of at least 32 bytes")}
|
||||
bounded:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
ctx,cancel:=context.WithTimeout(c.Context(),h.timeout);defer cancel();c.WithContext(ctx);return next(c)
|
||||
}}
|
||||
authorized:=func(next omni.Handler)omni.Handler{return func(c *omni.Context)error{
|
||||
if len(c.Request().Header.Values("Authorization"))!=1 || subtle.ConstantTimeCompare([]byte(c.Header("Authorization")),[]byte("Bearer "+token))!=1{return omni.Unauthorized("UNAUTHORIZED","invalid file access token")}
|
||||
return next(c)
|
||||
}}
|
||||
group:=application.Group("/files",authorized,bounded)
|
||||
group.POST("",h.Upload)
|
||||
group.POST("/uploadLink",h.UploadLink)
|
||||
group.POST("/downloadLink",h.DownloadLink)
|
||||
if signed!=nil {application.Any("/storage/object",func(c *omni.Context)error{
|
||||
signed.ServeHTTP(contextWriter{c},c.Request());return nil
|
||||
},bounded)}
|
||||
return nil
|
||||
}
|
||||
func (h *Files) Upload(c *omni.Context)error {
|
||||
size:=c.Request().ContentLength
|
||||
if size<0{return omni.NewError(http.StatusLengthRequired,"LENGTH_REQUIRED","Content-Length is required")}
|
||||
if size>h.maxBytes{return omni.NewError(http.StatusRequestEntityTooLarge,"FILE_TOO_LARGE","file exceeds upload limit")}
|
||||
body:=http.MaxBytesReader(c.Raw(),c.Request().Body,h.maxBytes)
|
||||
defer body.Close()
|
||||
info,err:=h.service.Upload(c.Context(),body,size,c.Header("Content-Type"))
|
||||
if err!=nil{return fileError(err)}
|
||||
return c.Created(info)
|
||||
}
|
||||
func (h *Files) UploadLink(c *omni.Context)error {
|
||||
var input struct{Size int64 `json:"size" binding:"gte=0"`;ContentType string `json:"contentType" binding:"required,max=256"`}
|
||||
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||
link,err:=h.service.UploadLink(c.Context(),input.Size,input.ContentType)
|
||||
if err!=nil{return fileError(err)}
|
||||
c.SetResponseHeader("Cache-Control","no-store")
|
||||
return c.OK(link)
|
||||
}
|
||||
func (h *Files) DownloadLink(c *omni.Context)error {
|
||||
var input struct{Key string `json:"key" binding:"required,max=1024"`}
|
||||
if err:=c.BindJSON(&input);err!=nil{return err}
|
||||
if err:=storage.Check(c.Context(),input.Key);err!=nil{return omni.BadRequest("INVALID_KEY","invalid object key")}
|
||||
link,err:=h.service.DownloadLink(c.Context(),input.Key)
|
||||
if err!=nil{return fileError(err)}
|
||||
c.SetResponseHeader("Cache-Control","no-store")
|
||||
return c.OK(map[string]string{"url":link})
|
||||
}
|
||||
func fileError(err error)error {
|
||||
var limit *http.MaxBytesError
|
||||
switch {
|
||||
case errors.Is(err,storage.ErrTooLarge),errors.As(err,&limit):return omni.NewError(413,"FILE_TOO_LARGE","file exceeds upload limit").WithCause(err)
|
||||
case errors.Is(err,storage.ErrContentType):return omni.NewError(415,"CONTENT_TYPE_NOT_ALLOWED","file content type is not allowed").WithCause(err)
|
||||
case errors.Is(err,storage.ErrSizeMismatch):return omni.BadRequest("SIZE_MISMATCH","file size does not match Content-Length").WithCause(err)
|
||||
case errors.Is(err,storage.ErrNotFound):return omni.NotFound("FILE_NOT_FOUND","file was not found").WithCause(err)
|
||||
case errors.Is(err,context.DeadlineExceeded):return omni.NewError(504,"STORAGE_TIMEOUT","storage operation timed out").WithCause(err)
|
||||
default:return err
|
||||
}
|
||||
}
|
||||
|
||||
// Track raw streaming handler responses through Omni so access logs retain the
|
||||
// actual status (including failed signatures), without buffering object bodies.
|
||||
type contextWriter struct{ctx *omni.Context}
|
||||
func (w contextWriter) Header()http.Header{return w.ctx.Raw().Header()}
|
||||
func (w contextWriter) WriteHeader(status int){_ = w.ctx.WriteResponse(status,nil,nil)}
|
||||
func (w contextWriter) Write(data []byte)(int,error){
|
||||
if !w.ctx.Responded(){if err:=w.ctx.WriteResponse(http.StatusOK,nil,nil);err!=nil{return 0,err}}
|
||||
return w.ctx.Raw().Write(data)
|
||||
}
|
||||
@@ -0,0 +1,75 @@
|
||||
package handler_test
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"net/url"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"{{.Module}}/internal/handler"
|
||||
"{{.Module}}/internal/service"
|
||||
)
|
||||
|
||||
// Exercise business code using Local even when production selects a cloud
|
||||
// adapter. These tests do not use cloud accounts or network credentials.
|
||||
func TestFileUploadAndLinks(t *testing.T){
|
||||
files,err:=local.New(t.TempDir(),64);if err!=nil{t.Fatal(err)}
|
||||
policy:=storage.UploadPolicy{MaxBytes:64,ContentTypes:[]string{"text/plain"}}
|
||||
signer,err:=storage.NewUploadSigner(files,"http://localhost/storage/object",bytes.Repeat([]byte{1},32),policy);if err!=nil{t.Fatal(err)}
|
||||
uploader,err:=storage.NewUploader(files,policy);if err!=nil{t.Fatal(err)}
|
||||
var logs bytes.Buffer
|
||||
logger,err:=logx.New(logx.Config{Writer:&logs});if err!=nil{t.Fatal(err)}
|
||||
logx.SetDefault(logger);defer logx.SetDefault(nil)
|
||||
app:=omni.New(omni.WithLogger(true))
|
||||
token:=strings.Repeat("t",32)
|
||||
svc:=service.NewFiles(files,uploader,signer,signer,logger,time.Minute)
|
||||
if err=handler.NewFiles(svc,64,time.Second).RegisterRoutes(app,token,signer);err!=nil{t.Fatal(err)}
|
||||
request:=func(method,path,body,contentType string,authorized bool)*httptest.ResponseRecorder{
|
||||
r:=httptest.NewRequest(method,path,strings.NewReader(body));r.Header.Set("Content-Type",contentType);r.Header.Set("X-Request-ID","upload-test")
|
||||
if authorized{r.Header.Set("Authorization","Bearer "+token)}
|
||||
w:=httptest.NewRecorder();app.ServeHTTP(w,r);return w
|
||||
}
|
||||
if w:=request("POST","/files","hello","text/plain",false);w.Code!=401{t.Fatal(w.Code)}
|
||||
w:=request("POST","/files","hello","text/plain",true)
|
||||
var saved struct{Data storage.Info}
|
||||
if w.Code!=201 || json.Unmarshal(w.Body.Bytes(),&saved)!=nil || saved.Data.Size!=5{t.Fatal(w.Code,w.Body.String())}
|
||||
raw,_:=json.Marshal(map[string]string{"key":saved.Data.Key})
|
||||
w=request("POST","/files/downloadLink",string(raw),"application/json",true)
|
||||
var download struct{Data struct{URL string}}
|
||||
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&download)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||
w=request("GET",download.Data.URL,"","",false)
|
||||
if w.Code!=200 || w.Body.String()!="hello"{t.Fatal(w.Code,w.Body.String())}
|
||||
u,_:=url.Parse(download.Data.URL);q:=u.Query();q.Set("signature","tampered");u.RawQuery=q.Encode()
|
||||
if w=request("GET",u.String(),"","",false);w.Code!=403{t.Fatal(w.Code)}
|
||||
for _,tc:=range []struct{path,body,contentType string;status int}{
|
||||
{"/files",strings.Repeat("x",65),"text/plain",413},
|
||||
{"/files","hello","image/png",415},
|
||||
{"/files","<html>hello</html>","text/plain",415},
|
||||
{"/files/downloadLink",`{"key":"../bad"}`,"application/json",400},
|
||||
{"/files/downloadLink",`{"key":"missing"}`,"application/json",404},
|
||||
{"/files/uploadLink",`{"size":65,"contentType":"text/plain"}`,"application/json",413},
|
||||
{"/files/uploadLink",`{"size":5,"contentType":"image/png"}`,"application/json",415},
|
||||
}{if w=request("POST",tc.path,tc.body,tc.contentType,true);w.Code!=tc.status{t.Fatal(tc.path,w.Code,w.Body.String())}}
|
||||
w=request("POST","/files/uploadLink",`{"size":5,"contentType":"text/plain"}`,"application/json",true)
|
||||
var direct struct{Data service.FileUploadLink}
|
||||
if w.Code!=200 || json.Unmarshal(w.Body.Bytes(),&direct)!=nil{t.Fatal(w.Code,w.Body.String())}
|
||||
if w=request("PUT",direct.Data.Request.URL,"world","text/plain",false);w.Code!=201{t.Fatal(w.Code,w.Body.String())}
|
||||
object,err:=files.Get(context.Background(),direct.Data.Key);if err!=nil{t.Fatal(err)};body,err:=io.ReadAll(object.Body);object.Body.Close();if err!=nil || string(body)!="world"{t.Fatal(err,string(body))}
|
||||
decoder:=json.NewDecoder(&logs);sawBusiness,sawDenied:=false,false
|
||||
for {var entry map[string]any;err:=decoder.Decode(&entry);if err==io.EOF{break};if err!=nil{t.Fatal(err)}
|
||||
if entry["requestId"]!="upload-test"{t.Fatal("lost request correlation")}
|
||||
if entry["msg"]=="file.uploaded"{sawBusiness=true}
|
||||
if entry["msg"]=="http.request" && entry["path"]=="/storage/object" && entry["status"]==float64(http.StatusForbidden){sawDenied=true}
|
||||
}
|
||||
if !sawBusiness || !sawDenied{t.Fatal("business log or signed-handler status missing")}
|
||||
}
|
||||
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
61
cmd/go-omni/scaffold/templates/modules-filesService.go.tmpl
Normal file
@@ -0,0 +1,61 @@
|
||||
package service
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"io"
|
||||
"time"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
)
|
||||
|
||||
// Files borrows provider-neutral capabilities; it never creates or closes clients.
|
||||
// Authentication/ownership rules belong to this project, outside storage adapters.
|
||||
type Files struct {
|
||||
store storage.Store
|
||||
uploader *storage.Uploader
|
||||
downloads storage.Presigner
|
||||
uploadLinks storage.UploadPresigner
|
||||
logger *logx.Logger
|
||||
ttl time.Duration
|
||||
}
|
||||
func NewFiles(store storage.Store,uploader *storage.Uploader,downloads storage.Presigner,uploadLinks storage.UploadPresigner,logger *logx.Logger,ttl time.Duration)*Files{
|
||||
return &Files{store:store,uploader:uploader,downloads:downloads,uploadLinks:uploadLinks,logger:logger.With("component","files"),ttl:ttl}
|
||||
}
|
||||
func newFileKey()(string,error){
|
||||
var id [16]byte
|
||||
if _,err:=rand.Read(id[:]);err!=nil{return "",err}
|
||||
return "uploads/"+hex.EncodeToString(id[:]),nil
|
||||
}
|
||||
func (s *Files) Upload(ctx context.Context,body io.Reader,size int64,contentType string)(storage.Info,error){
|
||||
key,err:=newFileKey();if err!=nil{return storage.Info{},err}
|
||||
info,err:=s.uploader.Put(ctx,key,body,size,contentType)
|
||||
if err!=nil{s.logger.WarnContext(ctx,"file.upload.failed","error",err);return storage.Info{},err}
|
||||
s.logger.InfoContext(ctx,"file.uploaded","key",info.Key,"size",info.Size,"contentType",info.ContentType)
|
||||
return info,nil
|
||||
}
|
||||
type FileUploadLink struct {
|
||||
Key string `json:"key"`
|
||||
Request storage.UploadRequest `json:"request"`
|
||||
}
|
||||
func (s *Files) UploadLink(ctx context.Context,size int64,contentType string)(FileUploadLink,error){
|
||||
key,err:=newFileKey();if err!=nil{return FileUploadLink{},err}
|
||||
// Check the same policy before issuing a local or cloud capability.
|
||||
// Cloud byte-level checks must happen in a post-upload validation workflow.
|
||||
request,err:=s.uploadLinks.PresignPut(ctx,key,size,contentType,s.ttl)
|
||||
if err!=nil{return FileUploadLink{},err}
|
||||
s.logger.InfoContext(ctx,"file.uploadLink.issued","key",key)
|
||||
return FileUploadLink{Key:key,Request:request},nil
|
||||
}
|
||||
func (s *Files) DownloadLink(ctx context.Context,key string)(string,error){
|
||||
if err:=storage.Check(ctx,key);err!=nil{return "",err}
|
||||
// Confirm the object exists without buffering its contents.
|
||||
object,err:=s.store.Get(ctx,key);if err!=nil{return "",err}
|
||||
if err=object.Body.Close();err!=nil{return "",err}
|
||||
link,err:=s.downloads.PresignGet(ctx,key,s.ttl)
|
||||
if err!=nil{return "",err}
|
||||
s.logger.InfoContext(ctx,"file.downloadLink.issued","key",key)
|
||||
return link.String(),nil
|
||||
}
|
||||
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
6
cmd/go-omni/scaffold/templates/modules-gitignore.tmpl
Normal file
@@ -0,0 +1,6 @@
|
||||
/var/
|
||||
/server
|
||||
.env
|
||||
.env.*
|
||||
*.local.yaml
|
||||
*.local.json
|
||||
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
50
cmd/go-omni/scaffold/templates/modules-main.go.tmpl
Normal file
@@ -0,0 +1,50 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/handler"
|
||||
"{{.Module}}/internal/service"
|
||||
"{{.Module}}/internal/servicecontext"
|
||||
"{{.Module}}/internal/settings"
|
||||
)
|
||||
|
||||
func main(){
|
||||
path:=flag.String("config","etc/config.yaml","JSON/YAML settings; empty uses defaults and environment")
|
||||
flag.Parse()
|
||||
if err:=run(*path);err!=nil{log.Fatal(err)}
|
||||
}
|
||||
func run(path string)(runErr error){
|
||||
ctx,stop:=signal.NotifyContext(context.Background(),os.Interrupt,syscall.SIGTERM)
|
||||
defer stop()
|
||||
cfg,err:=settings.Load(path)
|
||||
if err!=nil{return err}
|
||||
logger,err:=logx.New(cfg.Server.Log)
|
||||
if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,logger.Close())}()
|
||||
logx.SetDefault(logger)
|
||||
services,err:=servicecontext.New(ctx,logger)
|
||||
if err!=nil{return err}
|
||||
defer func(){runErr=errors.Join(runErr,services.Close())}()
|
||||
{{if ne .Storage "none"}}if err=services.InitStorage(ctx,cfg.Storage);err!=nil{return err}{{end}}
|
||||
application:=omni.New(omni.WithConfig(cfg.Server))
|
||||
application.OnShutdown(logger.Close)
|
||||
application.OnShutdown(services.Close)
|
||||
application.GET("/health/live",services.Health.LivenessHandler())
|
||||
application.GET("/health/ready",services.Health.ReadinessHandler())
|
||||
handler.NewUsers(service.NewUsers(services)).RegisterRoutes(application)
|
||||
{{if ne .Storage "none"}}
|
||||
files:=service.NewFiles(services.Files,services.Uploads,services.Downloads,services.UploadLinks,logger,cfg.Storage.LinkTTL)
|
||||
if err=handler.NewFiles(files,cfg.Storage.MaxBytes,cfg.Storage.RequestTimeout).RegisterRoutes(application,cfg.Storage.FilesToken,services.FileHTTP);err!=nil{return err}
|
||||
{{end}}
|
||||
logger.InfoContext(ctx,"service.starting","address",cfg.Server.Address)
|
||||
return application.RunContext(ctx,cfg.Server.Address)
|
||||
}
|
||||
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
140
cmd/go-omni/scaffold/templates/modules-settings.go.tmpl
Normal file
@@ -0,0 +1,140 @@
|
||||
// Package settings owns this project's configuration; it adds no business identity model.
|
||||
package settings
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"time"
|
||||
{{if ne .Storage "none"}}"fmt"
|
||||
{{if ne .Storage "r2"}}"net/url"{{else}}"regexp"{{end}}
|
||||
"strconv"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}"git.yuntuo.hk/k2safe/go-omni/database"{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Server config.Config `json:"server"`
|
||||
{{if and .API (eq .Database "mysql")}}Database database.Config `json:"database"`{{end}}
|
||||
{{if ne .Storage "none"}}Storage StorageConfig `json:"storage"`{{end}}
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
server:=config.Default()
|
||||
server.Address="127.0.0.1:8080"
|
||||
server.EnableLogger=true
|
||||
server.SlowRequestThreshold=500*time.Millisecond
|
||||
server.Log.Service={{printf "%q" .Module}}
|
||||
server.Log.Format={{printf "%q" .LogFormat}}
|
||||
{{if eq .LogOutput "file"}}server.Log.File="./var/log/server.log"
|
||||
server.Log.Rotation=logx.RotationConfig{Enabled:true,MaxSizeMB:100,MaxBackups:10,MaxAgeDays:7,Compress:true}
|
||||
{{else}}server.Log.Output={{printf "%q" .LogOutput}}{{end}}
|
||||
return Config{Server:server{{if and .API (eq .Database "mysql")}},Database:database.DefaultConfig(){{end}}{{if ne .Storage "none"}},Storage:StorageConfig{
|
||||
Provider:{{printf "%q" .Storage}}, Root:"./var/files", PublicURL:"http://127.0.0.1:8080/storage/object",
|
||||
Secure:true, MaxBytes:32<<20, ContentTypes:[]string{"text/plain","image/png","image/jpeg","application/pdf"},
|
||||
LinkTTL:5*time.Minute, RequestTimeout:30*time.Second,
|
||||
{{if eq .Storage "s3"}}Region:"us-east-1",{{end}}
|
||||
{{if eq .Storage "r2"}}Region:"auto",{{end}}
|
||||
{{if eq .Storage "oss"}}Region:"cn-hangzhou",{{end}}
|
||||
}{{end}}}
|
||||
}
|
||||
|
||||
// Load uses defaults -> strict YAML/JSON -> nonempty environment variables.
|
||||
// Pass an empty path to use only defaults and environment.
|
||||
func Load(path string)(Config,error) {
|
||||
cfg:=Default()
|
||||
var err error
|
||||
if path!="" { cfg,err=config.DecodeFile(path,cfg);if err!=nil{return Config{},err} }
|
||||
cfg.Server,err=config.ApplyEnv(cfg.Server,"")
|
||||
if err!=nil{return Config{},err}
|
||||
// A stream explicitly selected by environment overrides a file profile.
|
||||
if output:=strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT"));output!="" && strings.TrimSpace(os.Getenv("GO_OMNI_LOG_FILE"))=="" {
|
||||
cfg.Server.Log.File="";cfg.Server.Log.Rotation=logx.RotationConfig{}
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
if err=cfg.Storage.applyEnv();err!=nil{return Config{},err}
|
||||
{{end}}
|
||||
{{if and .API (eq .Database "mysql")}}cfg.Database,err=database.ApplyEnv(cfg.Database,"");if err!=nil{return Config{},err}{{end}}
|
||||
if err=cfg.Validate();err!=nil{return Config{},err}
|
||||
return cfg,nil
|
||||
}
|
||||
func (c Config) Validate()error {
|
||||
{{if and .API (eq .Database "mysql")}}
|
||||
if err:=c.Database.Validate();err!=nil{return err}
|
||||
if c.Database.Driver!="mysql" || c.Database.MaxOpenConns<1 {return errors.New("database requires driver mysql and a positive maxOpenConns")}
|
||||
{{end}}
|
||||
if err:=c.Server.Validate();err!=nil{return err}
|
||||
if c.Server.Log.Format!="json" && c.Server.Log.Format!="text" {return errors.New("log.format must be json or text")}
|
||||
switch strings.ToLower(c.Server.Log.Level) {case "debug","info","warn","warning","error":default:return errors.New("invalid log.level")}
|
||||
{{if ne .Storage "none"}}return c.Storage.Validate(){{else}}return nil{{end}}
|
||||
}
|
||||
|
||||
{{if ne .Storage "none"}}
|
||||
type StorageConfig struct {
|
||||
Provider string `json:"provider"`
|
||||
Root string `json:"root"`
|
||||
PublicURL string `json:"publicUrl"`
|
||||
Endpoint string `json:"endpoint"`
|
||||
Region string `json:"region"`
|
||||
Bucket string `json:"bucket"`
|
||||
AccountID string `json:"accountId"`
|
||||
Secure bool `json:"secure"`
|
||||
MaxBytes int64 `json:"maxBytes"`
|
||||
ContentTypes []string `json:"contentTypes"`
|
||||
LinkTTL time.Duration `json:"linkTtl"`
|
||||
RequestTimeout time.Duration `json:"requestTimeout"`
|
||||
// Secrets are environment-only and cannot be populated by config files.
|
||||
FilesToken string `json:"-"`
|
||||
SigningKey string `json:"-"`
|
||||
AccessKey string `json:"-"`
|
||||
SecretKey string `json:"-"`
|
||||
SessionToken string `json:"-"`
|
||||
}
|
||||
func (s StorageConfig) Policy()storage.UploadPolicy{return storage.UploadPolicy{MaxBytes:s.MaxBytes,ContentTypes:s.ContentTypes}}
|
||||
func (s *StorageConfig) applyEnv()error {
|
||||
values:=map[string]*string{
|
||||
"GO_OMNI_STORAGE_ROOT":&s.Root,"GO_OMNI_STORAGE_PUBLIC_URL":&s.PublicURL,
|
||||
"GO_OMNI_STORAGE_ENDPOINT":&s.Endpoint,"GO_OMNI_STORAGE_REGION":&s.Region,
|
||||
"GO_OMNI_STORAGE_BUCKET":&s.Bucket,"GO_OMNI_STORAGE_ACCOUNT_ID":&s.AccountID,
|
||||
"GO_OMNI_STORAGE_SIGNING_KEY":&s.SigningKey,"GO_OMNI_FILES_TOKEN":&s.FilesToken,
|
||||
"GO_OMNI_STORAGE_ACCESS_KEY":&s.AccessKey,"GO_OMNI_STORAGE_SECRET_KEY":&s.SecretKey,
|
||||
"GO_OMNI_STORAGE_SESSION_TOKEN":&s.SessionToken,
|
||||
}
|
||||
for name,target:=range values {if value:=strings.TrimSpace(os.Getenv(name));value!=""{*target=value}}
|
||||
for name,target:=range map[string]*time.Duration{"GO_OMNI_STORAGE_LINK_TTL":&s.LinkTTL,"GO_OMNI_STORAGE_REQUEST_TIMEOUT":&s.RequestTimeout} {
|
||||
if value:=strings.TrimSpace(os.Getenv(name));value!="" {v,err:=time.ParseDuration(value);if err!=nil{return fmt.Errorf("%s must be a duration",name)};*target=v}
|
||||
}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_MAX_BYTES");value!=""{v,err:=strconv.ParseInt(value,10,64);if err!=nil{return errors.New("GO_OMNI_STORAGE_MAX_BYTES must be an integer")};s.MaxBytes=v}
|
||||
if value:=os.Getenv("GO_OMNI_STORAGE_SECURE");value!=""{v,err:=strconv.ParseBool(value);if err!=nil{return errors.New("GO_OMNI_STORAGE_SECURE must be a boolean")};s.Secure=v}
|
||||
if value:=strings.TrimSpace(os.Getenv("GO_OMNI_STORAGE_CONTENT_TYPES"));value!="" {s.ContentTypes=strings.Split(value,",");for i:=range s.ContentTypes{s.ContentTypes[i]=strings.TrimSpace(s.ContentTypes[i])}}
|
||||
return nil
|
||||
}
|
||||
func (s StorageConfig) Validate()error {
|
||||
if s.Provider!={{printf "%q" .Storage}} {return errors.New("storage.provider must match the adapter selected at generation")}
|
||||
if err:=s.Policy().Validate();err!=nil{return err}
|
||||
if s.LinkTTL<time.Second || s.LinkTTL>time.Hour{return errors.New("storage.linkTtl must be 1s..1h")}
|
||||
if s.RequestTimeout<=0 || s.RequestTimeout>5*time.Minute{return errors.New("storage.requestTimeout must be >0 and <=5m")}
|
||||
{{if not .API}}if len(s.FilesToken)<32 || strings.ContainsAny(s.FilesToken," \t\r\n") {return errors.New("set GO_OMNI_FILES_TOKEN to a token of at least 32 bytes without whitespace")}{{end}}
|
||||
{{if eq .Storage "local"}}
|
||||
if s.Root=="" {return errors.New("storage.root is required")}
|
||||
u,err:=url.Parse(s.PublicURL)
|
||||
if err!=nil || u.Host=="" || u.Hostname()=="" || (u.Scheme!="http" && u.Scheme!="https") || u.User!=nil || u.RawQuery!="" || u.Fragment!="" || u.Path!="/storage/object" {return errors.New("storage.publicUrl must be an absolute HTTP(S) URL ending in /storage/object")}
|
||||
if len(s.SigningKey)<32 {return errors.New("set GO_OMNI_STORAGE_SIGNING_KEY to a stable secret of at least 32 bytes")}
|
||||
{{else}}
|
||||
if s.Bucket=="" || s.Region=="" || s.AccessKey=="" || s.SecretKey=="" {return errors.New("storage bucket, region and GO_OMNI_STORAGE_ACCESS_KEY/SECRET_KEY are required")}
|
||||
{{if eq .Storage "r2"}}
|
||||
if !regexp.MustCompile(`^[a-fA-F0-9]{32}$`).MatchString(s.AccountID) || s.Region!="auto" || !s.Secure {return errors.New("R2 requires a 32-hex accountId, region auto and secure=true")}
|
||||
if s.Endpoint!=""{return errors.New("this R2 profile derives endpoint from accountId; customize servicecontext for a jurisdiction-specific endpoint")}
|
||||
{{else if eq .Storage "s3"}}
|
||||
u,err:=url.Parse("https://"+s.Endpoint)
|
||||
if err!=nil || s.Endpoint=="" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("S3 endpoint must be host[:port] without scheme or path")}
|
||||
{{else if eq .Storage "oss"}}
|
||||
if s.Endpoint!="" {u,err:=url.Parse(s.Endpoint);if err!=nil || u.Scheme!="https" || u.Hostname()=="" || u.User!=nil || u.Path!="" || u.RawQuery!="" || u.Fragment!="" {return errors.New("OSS endpoint must be an HTTPS origin")}}
|
||||
if !s.Secure{return errors.New("OSS requires secure=true")}
|
||||
{{end}}
|
||||
{{end}}
|
||||
return nil
|
||||
}
|
||||
{{end}}
|
||||
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-settings_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
||||
package settings
|
||||
|
||||
import (
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
func environment(t *testing.T){
|
||||
t.Helper()
|
||||
for _,entry:=range os.Environ(){key,_,_:=strings.Cut(entry,"=");if strings.HasPrefix(key,"GO_OMNI_"){t.Setenv(key,"")}}
|
||||
{{if ne .Storage "none"}}
|
||||
t.Setenv("GO_OMNI_FILES_TOKEN",strings.Repeat("t",32))
|
||||
t.Setenv("GO_OMNI_STORAGE_SIGNING_KEY",strings.Repeat("s",32))
|
||||
t.Setenv("GO_OMNI_STORAGE_ACCESS_KEY","fixtureAccess")
|
||||
t.Setenv("GO_OMNI_STORAGE_SECRET_KEY","fixtureSecret")
|
||||
t.Setenv("GO_OMNI_STORAGE_BUCKET","testbucket")
|
||||
{{if eq .Storage "s3"}}t.Setenv("GO_OMNI_STORAGE_ENDPOINT","127.0.0.1:9000"){{end}}
|
||||
{{if eq .Storage "r2"}}t.Setenv("GO_OMNI_STORAGE_ACCOUNT_ID",strings.Repeat("a",32)){{end}}
|
||||
{{end}}
|
||||
}
|
||||
func TestSettingsPrecedenceAndStrictness(t *testing.T){
|
||||
environment(t)
|
||||
path:=filepath.Join(t.TempDir(),"config.yaml")
|
||||
if err:=os.WriteFile(path,[]byte("server:\n address: '127.0.0.1:8123'\n log:\n level: debug\n"),0600);err!=nil{t.Fatal(err)}
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR","127.0.0.1:9123")
|
||||
t.Setenv("GO_OMNI_LOG_OUTPUT","stdout")
|
||||
cfg,err:=Load(path);if err!=nil{t.Fatal(err)}
|
||||
if cfg.Server.Address!="127.0.0.1:9123" || cfg.Server.Log.Level!="debug" || cfg.Server.Log.Output!="stdout" || cfg.Server.Log.File!="" || cfg.Server.Log.Rotation.Enabled {t.Fatal("configuration precedence failed")}
|
||||
for _,body:=range []string{"unexpected: true","server:\n slowRequestThreshold: -1s","server:\n log:\n format: invalid"{{if ne .Storage "none"}},"storage:\n filesToken: forbidden","storage:\n maxBytes: -1","storage:\n provider: unsupported"{{end}}}{
|
||||
if err:=os.WriteFile(path,[]byte(body),0600);err!=nil{t.Fatal(err)}
|
||||
if _,err:=Load(path);err==nil{t.Fatal("invalid configuration accepted")}
|
||||
}
|
||||
{{if ne .Storage "none"}}
|
||||
t.Setenv("GO_OMNI_FILES_TOKEN","")
|
||||
if _,err:=Load("");err==nil{t.Fatal("missing file token accepted")}
|
||||
{{end}}
|
||||
}
|
||||
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
79
cmd/go-omni/scaffold/templates/modules-storage.go.tmpl
Normal file
@@ -0,0 +1,79 @@
|
||||
package servicecontext
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
{{if eq .Storage "local"}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
{{else}}
|
||||
"net"
|
||||
"net/http"
|
||||
"time"
|
||||
{{if eq .Storage "oss"}}
|
||||
ossStore "git.yuntuo.hk/k2safe/go-omni/storage/oss"
|
||||
ossSDK "github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss"
|
||||
"github.com/aliyun/alibabacloud-oss-go-sdk-v2/oss/credentials"
|
||||
{{else}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/s3"
|
||||
"github.com/minio/minio-go/v7/pkg/credentials"
|
||||
{{end}}{{end}}
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"{{.Module}}/internal/settings"
|
||||
)
|
||||
|
||||
// InitStorage is called once, before serving requests. The composition root
|
||||
// owns provider clients; business services receive only capability interfaces.
|
||||
func (c *Context) InitStorage(ctx context.Context,cfg settings.StorageConfig)error {
|
||||
if c==nil || ctx==nil{return errors.New("storage initialization requires context")}
|
||||
c.closeMu.Lock();defer c.closeMu.Unlock()
|
||||
if c.closed{return errors.New("service dependencies are closed")}
|
||||
if err:=ctx.Err();err!=nil{return err}
|
||||
if c.Files!=nil{return errors.New("storage already initialized")}
|
||||
if err:=cfg.Validate();err!=nil{return err}
|
||||
{{if eq .Storage "local"}}
|
||||
files,err:=local.New(cfg.Root,cfg.MaxBytes)
|
||||
if err!=nil{return err}
|
||||
signer,err:=storage.NewUploadSigner(files,cfg.PublicURL,[]byte(cfg.SigningKey),cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
c.Files,c.Uploads,c.Downloads,c.UploadLinks,c.FileHTTP=files,uploader,signer,signer,signer
|
||||
{{else}}
|
||||
// Dedicated bounded pool, never the process-wide default transport.
|
||||
transport:=&http.Transport{
|
||||
Proxy:http.ProxyFromEnvironment,
|
||||
DialContext:(&net.Dialer{Timeout:5*time.Second,KeepAlive:30*time.Second}).DialContext,
|
||||
ForceAttemptHTTP2:true, MaxIdleConns:32,MaxIdleConnsPerHost:8,MaxConnsPerHost:32,
|
||||
IdleConnTimeout:90*time.Second,TLSHandshakeTimeout:5*time.Second,
|
||||
ResponseHeaderTimeout:cfg.RequestTimeout,
|
||||
}
|
||||
initialized:=false
|
||||
defer func(){if !initialized{transport.CloseIdleConnections()}}()
|
||||
{{if eq .Storage "oss"}}
|
||||
sdkConfig:=ossSDK.LoadDefaultConfig().WithRegion(cfg.Region).
|
||||
WithSignatureVersion(ossSDK.SignatureVersionV4).
|
||||
WithAdditionalHeaders([]string{"content-length"}).
|
||||
WithCredentialsProvider(credentials.NewStaticCredentialsProvider(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)).
|
||||
WithHttpClient(&http.Client{Transport:transport,Timeout:cfg.RequestTimeout})
|
||||
if cfg.Endpoint!=""{sdkConfig.WithEndpoint(cfg.Endpoint)}
|
||||
files,err:=ossStore.New(ossSDK.NewClient(sdkConfig),cfg.Bucket,cfg.MaxBytes)
|
||||
{{else}}
|
||||
creds:=credentials.NewStaticV4(cfg.AccessKey,cfg.SecretKey,cfg.SessionToken)
|
||||
{{if eq .Storage "r2"}}
|
||||
options,err:=s3.R2Config(cfg.AccountID,cfg.Bucket,creds)
|
||||
if err!=nil{return err}
|
||||
options.MaxBytes,options.Transport=cfg.MaxBytes,transport
|
||||
{{else}}
|
||||
options:=s3.Config{Endpoint:cfg.Endpoint,Region:cfg.Region,Bucket:cfg.Bucket,Secure:cfg.Secure,MaxBytes:cfg.MaxBytes,Credentials:creds,Transport:transport}
|
||||
{{end}}
|
||||
files,err:=s3.Open(options)
|
||||
{{end}}
|
||||
if err!=nil{return err}
|
||||
uploader,err:=storage.NewUploader(files,cfg.Policy())
|
||||
if err!=nil{return err}
|
||||
c.Files,c.Uploads,c.Downloads,c.UploadLinks=files,uploader,files,uploader
|
||||
c.storageClose=func()error{transport.CloseIdleConnections();return nil}
|
||||
initialized=true
|
||||
{{end}}
|
||||
return nil
|
||||
}
|
||||
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
38
cmd/go-omni/scaffold/templates/modules-storage_test.go.tmpl
Normal file
@@ -0,0 +1,38 @@
|
||||
package servicecontext
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"{{.Module}}/internal/settings"
|
||||
)
|
||||
func TestStorageInitializationAndOwnership(t *testing.T){
|
||||
cfg:=settings.Default().Storage
|
||||
cfg.FilesToken=strings.Repeat("t",32)
|
||||
cfg.SigningKey=strings.Repeat("s",32)
|
||||
cfg.Root=t.TempDir()
|
||||
cfg.AccessKey,cfg.SecretKey,cfg.Bucket="fixtureAccess","fixtureSecret","testbucket"
|
||||
{{if eq .Storage "s3"}}cfg.Endpoint="127.0.0.1:9000"{{end}}
|
||||
{{if eq .Storage "r2"}}cfg.AccountID=strings.Repeat("a",32){{end}}
|
||||
var output bytes.Buffer
|
||||
logger,err:=logx.New(logx.Config{Writer:&output});if err!=nil{t.Fatal(err)};defer logger.Close()
|
||||
c:=&Context{Logger:logger}
|
||||
canceled,cancel:=context.WithCancel(context.Background());cancel()
|
||||
if err=c.InitStorage(canceled,cfg);err==nil || c.Files!=nil{t.Fatal("canceled startup initialized storage")}
|
||||
invalid:=cfg;invalid.FilesToken=""
|
||||
if err=c.InitStorage(context.Background(),invalid);err==nil || c.Files!=nil{t.Fatal("invalid settings initialized storage")}
|
||||
if err=c.InitStorage(context.Background(),cfg);err!=nil{t.Fatal(err)}
|
||||
if c.Files==nil || c.Uploads==nil || c.Downloads==nil || c.UploadLinks==nil{t.Fatal("dependencies not assembled")}
|
||||
if err=c.InitStorage(context.Background(),cfg);err==nil{t.Fatal("duplicate initialization accepted")}
|
||||
cleanup:=c.storageClose;count:=0
|
||||
c.storageClose=func()error{count++;if cleanup!=nil{return cleanup()};return nil}
|
||||
if err=c.Close();err!=nil{t.Fatal(err)};if err=c.Close();err!=nil{t.Fatal(err)}
|
||||
logger.Info("still usable")
|
||||
if count!=1 || !strings.Contains(output.String(),"still usable"){t.Fatal("resource ownership violated")}
|
||||
closed:=&Context{Logger:logger}
|
||||
if err=closed.Close();err!=nil{t.Fatal(err)}
|
||||
if err=closed.InitStorage(context.Background(),cfg);err==nil || closed.Files!=nil{t.Fatal("closed context reopened resources")}
|
||||
}
|
||||
@@ -10,7 +10,19 @@ const frameworkModule = "git.yuntuo.hk/k2safe/go-omni"
|
||||
// Source builds can carry an unpublished VCS version (including +dirty).
|
||||
// Keep this fallback in sync with the next release; installed CLIs use their
|
||||
// downloaded module version, which also has a module checksum.
|
||||
const defaultFrameworkVersion = "v0.6.0-preview"
|
||||
const defaultFrameworkVersion = "v0.7.0-preview"
|
||||
|
||||
// This target is not a claim that the release already exists. Source users
|
||||
// select a local framework checkout until the module templates are published.
|
||||
const modulesFrameworkVersion = "v0.7.0-preview"
|
||||
|
||||
func installedFrameworkVersion() string {
|
||||
info, _ := debug.ReadBuildInfo()
|
||||
if info != nil && info.Main.Path == frameworkModule && info.Main.Sum != "" && validFrameworkVersion(info.Main.Version) {
|
||||
return info.Main.Version
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
var frameworkVersionPattern = regexp.MustCompile(`^v[01]\.(0|[1-9][0-9]*)\.(0|[1-9][0-9]*)(-((0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*)(\.(0|[1-9][0-9]*|[0-9]*[A-Za-z-][0-9A-Za-z-]*))*))?(\+[0-9A-Za-z-]+(\.[0-9A-Za-z-]+)*)?$`)
|
||||
|
||||
|
||||
@@ -10,19 +10,35 @@ import (
|
||||
|
||||
const defaultRequestIDHeader = "X-Request-ID"
|
||||
|
||||
const (
|
||||
TransportNetHTTP = "nethttp"
|
||||
TransportNative = "native"
|
||||
TransportHTTP2 = "http2"
|
||||
)
|
||||
|
||||
// TLSConfig enables HTTPS when both certificate paths are provided.
|
||||
type TLSConfig struct {
|
||||
CertFile string `json:"certFile"`
|
||||
KeyFile string `json:"keyFile"`
|
||||
}
|
||||
|
||||
// Config controls the default runtime behavior of an App.
|
||||
type Config struct {
|
||||
Address string
|
||||
RequestIDHeader string
|
||||
EnableLogger bool
|
||||
TrustedProxies []string
|
||||
ShutdownTimeout time.Duration
|
||||
Log logx.Config
|
||||
Address string `json:"address"`
|
||||
Transport string `json:"transport"`
|
||||
TLS TLSConfig `json:"tls"`
|
||||
RequestIDHeader string `json:"requestIdHeader"`
|
||||
EnableLogger bool `json:"enableLogger"`
|
||||
SlowRequestThreshold time.Duration `json:"slowRequestThreshold"`
|
||||
TrustedProxies []string `json:"trustedProxies"`
|
||||
ShutdownTimeout time.Duration `json:"shutdownTimeout"`
|
||||
Log logx.Config `json:"log"`
|
||||
}
|
||||
|
||||
func Default() Config {
|
||||
return Config{
|
||||
Address: ":8080",
|
||||
Transport: TransportNetHTTP,
|
||||
RequestIDHeader: defaultRequestIDHeader,
|
||||
ShutdownTimeout: 10 * time.Second,
|
||||
Log: logx.Default(),
|
||||
@@ -31,6 +47,9 @@ func Default() Config {
|
||||
|
||||
// Validate checks configuration values before an application starts.
|
||||
func (c Config) Validate() error {
|
||||
if c.SlowRequestThreshold < 0 {
|
||||
return &ConfigError{Field: "slowRequestThreshold", Message: "must be nonnegative"}
|
||||
}
|
||||
if strings.TrimSpace(c.Address) == "" {
|
||||
return &ConfigError{Field: "address", Message: "is required"}
|
||||
}
|
||||
@@ -43,6 +62,24 @@ func (c Config) Validate() error {
|
||||
if _, err := httpx.NewClientIPResolver(c.TrustedProxies...); err != nil {
|
||||
return &ConfigError{Field: "trusted_proxies", Message: err.Error()}
|
||||
}
|
||||
return c.ValidateTransport()
|
||||
}
|
||||
|
||||
// ValidateTransport checks built-in transport selection without opening files
|
||||
// or sockets. Empty Transport preserves the standard net/http default.
|
||||
func (c Config) ValidateTransport() error {
|
||||
switch c.Transport {
|
||||
case "", TransportNetHTTP, TransportNative, TransportHTTP2:
|
||||
default:
|
||||
return &ConfigError{Field: "transport", Message: "must be nethttp, native or http2"}
|
||||
}
|
||||
cert, key := strings.TrimSpace(c.TLS.CertFile), strings.TrimSpace(c.TLS.KeyFile)
|
||||
if (cert == "") != (key == "") {
|
||||
return &ConfigError{Field: "tls", Message: "requires both certFile and keyFile"}
|
||||
}
|
||||
if c.Transport == TransportHTTP2 && cert == "" {
|
||||
return &ConfigError{Field: "tls", Message: "is required for the experimental http2 transport"}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -57,6 +94,16 @@ func (e *ConfigError) Error() string { return "config " + e.Field + " " + e.Mess
|
||||
// Option changes an App's configuration.
|
||||
type Option func(*Config)
|
||||
|
||||
// WithTransport selects a built-in server. HTTP/2 is experimental and requires TLS.
|
||||
func WithTransport(name string) Option {
|
||||
return func(cfg *Config) { cfg.Transport = name }
|
||||
}
|
||||
|
||||
// WithTLS enables HTTPS. Certificates are loaded before Run opens its listener.
|
||||
func WithTLS(certFile, keyFile string) Option {
|
||||
return func(cfg *Config) { cfg.TLS = TLSConfig{CertFile: certFile, KeyFile: keyFile} }
|
||||
}
|
||||
|
||||
// WithRequestIDHeader changes the header used to read and return request IDs.
|
||||
func WithRequestIDHeader(header string) Option {
|
||||
return func(cfg *Config) {
|
||||
@@ -71,6 +118,11 @@ func WithLogger(enabled bool) Option {
|
||||
return func(cfg *Config) { cfg.EnableLogger = enabled }
|
||||
}
|
||||
|
||||
// WithSlowRequestThreshold enables access logs and warns after this duration.
|
||||
func WithSlowRequestThreshold(threshold time.Duration) Option {
|
||||
return func(cfg *Config) { cfg.SlowRequestThreshold = threshold; cfg.EnableLogger = true }
|
||||
}
|
||||
|
||||
// WithTrustedProxies allows only these IPs/CIDRs to supply X-Forwarded-For.
|
||||
// An empty list disables forwarding-header trust. Invalid entries are rejected
|
||||
// by Config.Validate and panic when passed directly to App.New.
|
||||
@@ -83,15 +135,21 @@ func WithTrustedProxies(proxies ...string) Option {
|
||||
// environment. Later options can still override individual fields.
|
||||
func WithConfig(settings Config) Option {
|
||||
settings.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
settings.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
return func(cfg *Config) {
|
||||
*cfg = settings
|
||||
cfg.TrustedProxies = append([]string(nil), settings.TrustedProxies...)
|
||||
cfg.Log.RedactKeys = append([]string(nil), settings.Log.RedactKeys...)
|
||||
}
|
||||
}
|
||||
|
||||
// WithLogConfig configures the logger used by the application runtime.
|
||||
func WithLogConfig(settings logx.Config) Option {
|
||||
return func(cfg *Config) { cfg.Log = settings }
|
||||
settings.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||
return func(cfg *Config) {
|
||||
cfg.Log = settings
|
||||
cfg.Log.RedactKeys = append([]string(nil), settings.RedactKeys...)
|
||||
}
|
||||
}
|
||||
|
||||
// WithShutdownTimeout controls how long RunContext waits for active requests.
|
||||
|
||||
@@ -10,9 +10,11 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
for _, key := range []string{
|
||||
"GO_OMNI_HTTP_ADDR", "GO_OMNI_REQUEST_ID_HEADER", "GO_OMNI_ENABLE_LOGGER",
|
||||
"GO_OMNI_TRUSTED_PROXIES",
|
||||
"GO_OMNI_TRANSPORT", "GO_OMNI_TLS_CERT_FILE", "GO_OMNI_TLS_KEY_FILE",
|
||||
"GO_OMNI_LOG_LEVEL", "GO_OMNI_LOG_FORMAT", "GO_OMNI_LOG_OUTPUT",
|
||||
"GO_OMNI_LOG_FILE", "GO_OMNI_SERVICE", "GO_OMNI_ENVIRONMENT",
|
||||
"GO_OMNI_LOG_ADD_SOURCE", "GO_OMNI_SHUTDOWN_TIMEOUT",
|
||||
"GO_OMNI_SLOW_REQUEST_THRESHOLD",
|
||||
} {
|
||||
t.Setenv(key, "")
|
||||
}
|
||||
@@ -21,7 +23,7 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.Address != ":8080" || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
if cfg.Address != ":8080" || cfg.Transport != TransportNetHTTP || cfg.TLS != (TLSConfig{}) || cfg.RequestIDHeader != "X-Request-ID" || cfg.ShutdownTimeout != 10*time.Second {
|
||||
t.Fatalf("unexpected defaults: %+v", cfg)
|
||||
}
|
||||
if cfg.EnableLogger || cfg.Log.Level != "info" || cfg.Log.Format != "json" || cfg.Log.Output != "stderr" {
|
||||
@@ -29,6 +31,44 @@ func TestLoadFromEnvUsesDefaults(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTransportEnvironment(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport, cert, key, errorField string
|
||||
}{
|
||||
{"default", "", "", "", ""},
|
||||
{"native", "native", "", "", ""},
|
||||
{"nativeTLS", "native", "server.crt", "server.key", ""},
|
||||
{"http2", "http2", "server.crt", "server.key", ""},
|
||||
{"unknown", "typo", "", "", "transport"},
|
||||
{"certOnly", "nethttp", "server.crt", "", "tls"},
|
||||
{"keyOnly", "native", "", "server.key", "tls"},
|
||||
{"http2WithoutTLS", "http2", "", "", "tls"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
t.Setenv("TRANSPORT_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_CERT_FILE", test.cert)
|
||||
t.Setenv("TRANSPORT_TEST_TLS_KEY_FILE", test.key)
|
||||
cfg, err := LoadFromEnv("TRANSPORT_TEST_")
|
||||
if test.errorField != "" {
|
||||
if err == nil || !strings.Contains(err.Error(), test.errorField) {
|
||||
t.Fatalf("unexpected error: %v", err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := test.transport
|
||||
if want == "" {
|
||||
want = TransportNetHTTP
|
||||
}
|
||||
if cfg.Transport != want || cfg.TLS.CertFile != test.cert || cfg.TLS.KeyFile != test.key {
|
||||
t.Fatalf("settings: %+v", cfg)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:9090")
|
||||
t.Setenv("GO_OMNI_REQUEST_ID_HEADER", "X-Correlation-ID")
|
||||
@@ -41,6 +81,7 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_ENVIRONMENT", "test")
|
||||
t.Setenv("GO_OMNI_LOG_ADD_SOURCE", "true")
|
||||
t.Setenv("GO_OMNI_SHUTDOWN_TIMEOUT", "3s")
|
||||
t.Setenv("GO_OMNI_SLOW_REQUEST_THRESHOLD", "500ms")
|
||||
|
||||
cfg, err := LoadFromEnv("")
|
||||
if err != nil {
|
||||
@@ -52,6 +93,9 @@ func TestLoadFromEnvOverridesSettings(t *testing.T) {
|
||||
if cfg.ShutdownTimeout != 3*time.Second || cfg.Log.Level != "debug" || cfg.Log.Format != "text" || cfg.Log.Output != "stdout" || cfg.Log.Service != "billing" || cfg.Log.Environment != "test" || !cfg.Log.AddSource {
|
||||
t.Fatalf("unexpected logger settings: %+v", cfg.Log)
|
||||
}
|
||||
if cfg.SlowRequestThreshold != 500*time.Millisecond {
|
||||
t.Fatal(cfg.SlowRequestThreshold)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadFromEnvRejectsInvalidValues(t *testing.T) {
|
||||
@@ -103,3 +147,23 @@ func TestWithConfigAndWithLogConfig(t *testing.T) {
|
||||
t.Fatalf("log config was not replaced: %+v", cfg.Log)
|
||||
}
|
||||
}
|
||||
|
||||
func TestApplyEnvPreservesProjectDefaults(t *testing.T) {
|
||||
cfg := Default()
|
||||
cfg.Address = "127.0.0.1:8123"
|
||||
cfg.Log.RedactKeys = []string{"bankAccount"}
|
||||
cfg.TrustedProxies = []string{"127.0.0.1"}
|
||||
t.Setenv("OVERLAY_TEST_LOG_LEVEL", "debug")
|
||||
got, err := ApplyEnv(cfg, "OVERLAY_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.Address != cfg.Address || got.Log.Level != "debug" {
|
||||
t.Fatal(got)
|
||||
}
|
||||
got.Log.RedactKeys[0] = "changed"
|
||||
got.TrustedProxies[0] = "192.0.2.1"
|
||||
if cfg.Log.RedactKeys[0] != "bankAccount" || cfg.TrustedProxies[0] != "127.0.0.1" {
|
||||
t.Fatal("caller-owned defaults mutated")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,12 +13,26 @@ const defaultEnvPrefix = "GO_OMNI_"
|
||||
// LoadFromEnv loads application settings from GO_OMNI_* variables. The
|
||||
// database settings remain owned by database.ConfigFromEnv.
|
||||
func LoadFromEnv(prefix string) (Config, error) {
|
||||
return applyEnv(Default(), prefix)
|
||||
}
|
||||
|
||||
// ApplyEnv overlays nonempty environment values on a caller's defaults or file
|
||||
// settings, then validates the result. Caller-owned slices are not retained.
|
||||
func ApplyEnv(defaults Config, prefix string) (Config, error) {
|
||||
defaults.TrustedProxies = append([]string(nil), defaults.TrustedProxies...)
|
||||
defaults.Log.RedactKeys = append([]string(nil), defaults.Log.RedactKeys...)
|
||||
return applyEnv(defaults, prefix)
|
||||
}
|
||||
|
||||
func applyEnv(cfg Config, prefix string) (Config, error) {
|
||||
prefix = strings.TrimSpace(prefix)
|
||||
if prefix == "" {
|
||||
prefix = defaultEnvPrefix
|
||||
}
|
||||
cfg := Default()
|
||||
cfg.Address = envString(prefix+"HTTP_ADDR", cfg.Address)
|
||||
cfg.Transport = envString(prefix+"TRANSPORT", cfg.Transport)
|
||||
cfg.TLS.CertFile = envString(prefix+"TLS_CERT_FILE", cfg.TLS.CertFile)
|
||||
cfg.TLS.KeyFile = envString(prefix+"TLS_KEY_FILE", cfg.TLS.KeyFile)
|
||||
cfg.RequestIDHeader = envString(prefix+"REQUEST_ID_HEADER", cfg.RequestIDHeader)
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "TRUSTED_PROXIES")); value != "" {
|
||||
cfg.TrustedProxies = strings.Split(value, ",")
|
||||
@@ -40,6 +54,12 @@ func LoadFromEnv(prefix string) (Config, error) {
|
||||
if cfg.Log.AddSource, err = envBool(prefix+"LOG_ADD_SOURCE", cfg.Log.AddSource); err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "SLOW_REQUEST_THRESHOLD")); value != "" {
|
||||
cfg.SlowRequestThreshold, err = time.ParseDuration(value)
|
||||
if err != nil {
|
||||
return Config{}, fmt.Errorf("%sSLOW_REQUEST_THRESHOLD must be a duration: %w", prefix, err)
|
||||
}
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "SHUTDOWN_TIMEOUT")); value != "" {
|
||||
cfg.ShutdownTimeout, err = time.ParseDuration(value)
|
||||
if err != nil {
|
||||
|
||||
204
config/file.go
Normal file
204
config/file.go
Normal file
@@ -0,0 +1,204 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"gopkg.in/yaml.v3"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// DecodeFile merges a bounded JSON/YAML document into defaults. It uses JSON
|
||||
// field names for both formats, rejects unknown fields and accepts durations
|
||||
// such as "3s". Application-specific validation remains with the caller.
|
||||
func DecodeFile[T any](path string, defaults T) (T, error) {
|
||||
var zero T
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
defer f.Close()
|
||||
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
if len(data) > 1<<20 {
|
||||
return zero, errors.New("config: file exceeds 1 MiB")
|
||||
}
|
||||
var value any
|
||||
switch strings.ToLower(filepath.Ext(path)) {
|
||||
case ".yaml", ".yml":
|
||||
dec := yaml.NewDecoder(bytes.NewReader(data))
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one YAML document required")
|
||||
}
|
||||
case ".json":
|
||||
dec := json.NewDecoder(bytes.NewReader(data))
|
||||
dec.UseNumber()
|
||||
if err := dec.Decode(&value); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var extra any
|
||||
if err := dec.Decode(&extra); err != io.EOF {
|
||||
return zero, errors.New("config: one JSON document required")
|
||||
}
|
||||
default:
|
||||
return zero, errors.New("config: expected .json, .yaml or .yml")
|
||||
}
|
||||
if _, ok := value.(map[string]any); !ok {
|
||||
return zero, errors.New("config: root must be an object")
|
||||
}
|
||||
value, err = durations(value, reflect.TypeOf(defaults))
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
encoded, err := json.Marshal(value)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
// Deep copy defaults so errors cannot partially mutate caller-owned maps.
|
||||
original, err := json.Marshal(defaults)
|
||||
if err != nil {
|
||||
return zero, err
|
||||
}
|
||||
var result T
|
||||
if err = json.Unmarshal(original, &result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
dec := json.NewDecoder(bytes.NewReader(encoded))
|
||||
dec.DisallowUnknownFields()
|
||||
if err = dec.Decode(&result); err != nil {
|
||||
return zero, err
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
func durations(v any, t reflect.Type) (any, error) {
|
||||
if t == nil {
|
||||
return v, nil
|
||||
}
|
||||
for t.Kind() == reflect.Pointer {
|
||||
t = t.Elem()
|
||||
}
|
||||
if t == reflect.TypeOf(time.Duration(0)) {
|
||||
if s, ok := v.(string); ok {
|
||||
d, err := time.ParseDuration(s)
|
||||
return int64(d), err
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
if t.Kind() == reflect.Struct {
|
||||
if m, ok := v.(map[string]any); ok {
|
||||
for n := 0; n < t.NumField(); n++ {
|
||||
field := t.Field(n)
|
||||
name := strings.Split(field.Tag.Get("json"), ",")[0]
|
||||
if name == "-" {
|
||||
continue
|
||||
}
|
||||
if name == "" {
|
||||
name = field.Name
|
||||
}
|
||||
if item, exists := m[name]; exists {
|
||||
converted, err := durations(item, field.Type)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
m[name] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if t.Kind() == reflect.Slice {
|
||||
if items, ok := v.([]any); ok {
|
||||
for n, item := range items {
|
||||
converted, err := durations(item, t.Elem())
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items[n] = converted
|
||||
}
|
||||
}
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
func LoadFile(path, prefix string) (Config, error) {
|
||||
cfg, err := DecodeFile(path, Default())
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
return applyEnv(cfg, prefix)
|
||||
}
|
||||
|
||||
// Reloader only accepts log.level changes. Static changes and invalid files
|
||||
// leave the current snapshot and logger untouched. Watch does not own a goroutine.
|
||||
type Reloader struct {
|
||||
mu sync.Mutex
|
||||
path, prefix string
|
||||
current Config
|
||||
logger *logx.Logger
|
||||
}
|
||||
|
||||
func NewReloader(path, prefix string, logger *logx.Logger) (*Reloader, error) {
|
||||
cfg, err := LoadFile(path, prefix)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err = logger.SetLevel(cfg.Log.Level); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Reloader{path: path, prefix: prefix, current: cfg, logger: logger}, nil
|
||||
}
|
||||
func (r *Reloader) Reload() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
next, err := LoadFile(r.path, r.prefix)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
comparable := next
|
||||
comparable.Log.Level = r.current.Log.Level
|
||||
if !reflect.DeepEqual(comparable, r.current) {
|
||||
return errors.New("config: static fields changed; restart required")
|
||||
}
|
||||
if err = r.logger.SetLevel(next.Log.Level); err != nil {
|
||||
return err
|
||||
}
|
||||
r.current = next
|
||||
return nil
|
||||
}
|
||||
func (r *Reloader) Snapshot() Config {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
cfg := r.current
|
||||
cfg.TrustedProxies = append([]string(nil), cfg.TrustedProxies...)
|
||||
cfg.Log.RedactKeys = append([]string(nil), cfg.Log.RedactKeys...)
|
||||
return cfg
|
||||
}
|
||||
func (r *Reloader) Watch(ctx context.Context, interval time.Duration, onError func(error)) error {
|
||||
if ctx == nil || interval < 100*time.Millisecond {
|
||||
return errors.New("config: context and interval >=100ms required")
|
||||
}
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
for {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return ctx.Err()
|
||||
case <-ticker.C:
|
||||
if err := r.Reload(); err != nil && onError != nil {
|
||||
onError(err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
59
config/file_test.go
Normal file
59
config/file_test.go
Normal file
@@ -0,0 +1,59 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestFileMergeAndSafeReload(t *testing.T) {
|
||||
path := filepath.Join(t.TempDir(), "service.yaml")
|
||||
write := func(raw string) {
|
||||
t.Helper()
|
||||
if err := os.WriteFile(path, []byte(raw), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: info\n")
|
||||
t.Setenv("TEST_MODULE_HTTP_ADDR", ":9123")
|
||||
cfg, err := LoadFile(path, "TEST_MODULE_")
|
||||
if err != nil || cfg.Address != ":9123" || cfg.ShutdownTimeout != 7*time.Second {
|
||||
t.Fatal(cfg, err)
|
||||
}
|
||||
var output bytes.Buffer
|
||||
lc := cfg.Log
|
||||
lc.Writer = &output
|
||||
l, err := logx.New(lc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer l.Close()
|
||||
child := l.With("scope", "test")
|
||||
r, err := NewReloader(path, "TEST_MODULE_", l)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("hidden")
|
||||
write("address: ':8123'\nshutdownTimeout: 7s\nlog:\n level: debug\n")
|
||||
if err = r.Reload(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
child.Debug("visible")
|
||||
if strings.Contains(output.String(), "hidden") || !strings.Contains(output.String(), "visible") {
|
||||
t.Fatal(output.String())
|
||||
}
|
||||
write("shutdownTimeout: 8s\nlog:\n level: error\n")
|
||||
if err = r.Reload(); err == nil || r.Snapshot().Log.Level != "debug" {
|
||||
t.Fatal("static reload changed live config", err)
|
||||
}
|
||||
for _, raw := range []string{"unknownOption: true", "log:\n level: bad", "shutdownTimeout: invalid", "address: x\naddress: y", "---\n{}\n---\n{}"} {
|
||||
write(raw)
|
||||
if err = r.Reload(); err == nil {
|
||||
t.Fatal(raw)
|
||||
}
|
||||
}
|
||||
}
|
||||
30
contract/README.md
Normal file
30
contract/README.md
Normal file
@@ -0,0 +1,30 @@
|
||||
# contract
|
||||
|
||||
Generated API projects use this package to compile Go field tags once and bind
|
||||
each request without depending on a specific HTTP server. `New[T]` validates a
|
||||
struct and returns an immutable Binder. `ForType` provides the same compiler to
|
||||
the CLI. `BodySchema`, `Parameters`, and `ResponseSchema` return detached OpenAPI
|
||||
snapshots. `MustNew[T]` is intended for generated, already checked declarations.
|
||||
|
||||
```go
|
||||
type Request struct {
|
||||
ID int64 `path:"id,range=[1:100]"`
|
||||
Name string `json:"name" binding:"required,max=50"`
|
||||
}
|
||||
|
||||
var request = contract.MustNew[Request]()
|
||||
|
||||
func Handle(c *httpx.Context) error {
|
||||
var req Request
|
||||
if err := request.Bind(c.Request(), httpx.RouteParams(c.Context()), &req); err != nil {
|
||||
return err
|
||||
}
|
||||
return c.OK(req.Name)
|
||||
}
|
||||
```
|
||||
|
||||
The default body limit is 1 MiB. An optional positive final argument to Bind
|
||||
overrides it. Application transport read timeouts still control slow clients.
|
||||
See [API.md](../API.md) for source tags, validation rules, null/presence behavior,
|
||||
strict decoding and supported type limits. This API leaves the existing
|
||||
`Context.BindJSON/BindQuery/BindURI` behavior unchanged.
|
||||
310
contract/bind.go
Normal file
310
contract/bind.go
Normal file
@@ -0,0 +1,310 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"math"
|
||||
"mime"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"reflect"
|
||||
"strconv"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
)
|
||||
|
||||
// Bind fills a fresh DTO transactionally. A JSON body, if declared, is required,
|
||||
// strictly decoded, and limited to 1 MiB unless maxBytes is supplied. Query
|
||||
// parameters never overwrite path/body/header values. form means URL query only.
|
||||
func (b *Binder) Bind(r *http.Request, params map[string]string, target any, maxBytes ...int64) error {
|
||||
if b == nil || r == nil {
|
||||
return apierrors.Internal(errors.New("contract: nil binder or request"))
|
||||
}
|
||||
v := reflect.ValueOf(target)
|
||||
if v.Kind() != reflect.Pointer || v.IsNil() || v.Elem().Type() != b.typ {
|
||||
return apierrors.Internal(errors.New("contract: target type mismatch"))
|
||||
}
|
||||
limit := DefaultMaxBodyBytes
|
||||
if len(maxBytes) > 1 || (len(maxBytes) == 1 && (maxBytes[0] <= 0 || maxBytes[0] == math.MaxInt64)) {
|
||||
return apierrors.Internal(errors.New("contract: invalid body limit"))
|
||||
}
|
||||
if len(maxBytes) == 1 {
|
||||
limit = maxBytes[0]
|
||||
}
|
||||
fresh := reflect.New(b.typ).Elem()
|
||||
body := map[string]any{}
|
||||
if len(b.root.schema.Properties) > 0 {
|
||||
ct, _, err := mime.ParseMediaType(r.Header.Get("Content-Type"))
|
||||
if err != nil || ct != "application/json" {
|
||||
return apierrors.New(415, "UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json")
|
||||
}
|
||||
if r.Body == nil {
|
||||
return invalid(errors.New("body required"))
|
||||
}
|
||||
if r.ContentLength > limit {
|
||||
return tooLarge()
|
||||
}
|
||||
reader := &io.LimitedReader{R: r.Body, N: limit + 1}
|
||||
dec := json.NewDecoder(reader)
|
||||
dec.UseNumber()
|
||||
value, err := decodeValue(dec, 0)
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != nil {
|
||||
var maxErr *http.MaxBytesError
|
||||
if errors.As(err, &maxErr) {
|
||||
return tooLarge()
|
||||
}
|
||||
return invalid(err)
|
||||
}
|
||||
_, err = dec.Token()
|
||||
if reader.N == 0 {
|
||||
return tooLarge()
|
||||
}
|
||||
if err != io.EOF {
|
||||
return invalid(errors.New("exactly one JSON value required"))
|
||||
}
|
||||
var ok bool
|
||||
body, ok = value.(map[string]any)
|
||||
if !ok {
|
||||
return invalid(errors.New("body must be an object"))
|
||||
}
|
||||
for key := range body {
|
||||
if _, ok = b.root.schema.Properties[key]; !ok {
|
||||
return invalid(fmt.Errorf("unknown body field %s", key))
|
||||
}
|
||||
}
|
||||
}
|
||||
query := url.Values{}
|
||||
if r.URL != nil {
|
||||
var err error
|
||||
query, err = url.ParseQuery(r.URL.RawQuery)
|
||||
if err != nil {
|
||||
return invalid(errors.New("malformed query"))
|
||||
}
|
||||
}
|
||||
for _, f := range b.root.fields {
|
||||
var value any
|
||||
var present bool
|
||||
if f.source == "json" {
|
||||
value, present = body[f.name]
|
||||
} else {
|
||||
var raw []string
|
||||
switch f.source {
|
||||
case "path":
|
||||
if s, ok := params[f.name]; ok {
|
||||
raw = []string{s}
|
||||
}
|
||||
case "form":
|
||||
raw = query[f.name]
|
||||
case "header":
|
||||
raw = r.Header.Values(f.name)
|
||||
}
|
||||
present = len(raw) > 0
|
||||
if present {
|
||||
var err error
|
||||
value, err = fromStrings(f.value, raw)
|
||||
if err != nil {
|
||||
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := assignField(f, fresh.Field(f.index), value, present); err != nil {
|
||||
return invalid(fmt.Errorf("%s: %w", f.name, err))
|
||||
}
|
||||
}
|
||||
v.Elem().Set(fresh)
|
||||
return nil
|
||||
}
|
||||
func invalid(err error) error {
|
||||
return apierrors.BadRequest("INVALID_REQUEST", "request parameters are invalid").WithCause(err)
|
||||
}
|
||||
func tooLarge() error { return apierrors.New(413, "BODY_TOO_LARGE", "request body exceeds limit") }
|
||||
|
||||
func fromStrings(n *node, values []string) (any, error) {
|
||||
n = base(n)
|
||||
if n.typ.Kind() == reflect.Slice {
|
||||
out := make([]any, 0, len(values))
|
||||
for _, s := range values {
|
||||
v, err := parseScalar(s, base(n.elem).typ)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
out = append(out, v)
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
if len(values) != 1 {
|
||||
return nil, errors.New("duplicate scalar parameter")
|
||||
}
|
||||
return parseScalar(values[0], n.typ)
|
||||
}
|
||||
func assignField(f field, dst reflect.Value, value any, present bool) error {
|
||||
if !present {
|
||||
if f.hasDefault {
|
||||
value = f.fallback
|
||||
} else if f.optional {
|
||||
return nil
|
||||
} else {
|
||||
return errors.New("required field missing")
|
||||
}
|
||||
}
|
||||
if err := setValue(f.value, dst, value); err != nil {
|
||||
return err
|
||||
}
|
||||
return checkField(f, dst)
|
||||
}
|
||||
func setValue(n *node, dst reflect.Value, value any) error {
|
||||
if value == nil {
|
||||
if n.typ.Kind() == reflect.Pointer {
|
||||
dst.SetZero()
|
||||
return nil
|
||||
}
|
||||
return errors.New("null requires a pointer type")
|
||||
}
|
||||
if n.typ.Kind() == reflect.Pointer {
|
||||
dst.Set(reflect.New(n.typ.Elem()))
|
||||
return setValue(n.elem, dst.Elem(), value)
|
||||
}
|
||||
switch n.typ.Kind() {
|
||||
case reflect.Struct:
|
||||
object, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return errors.New("expected object")
|
||||
}
|
||||
for key := range object {
|
||||
if _, ok = n.schema.Properties[key]; !ok {
|
||||
return fmt.Errorf("unknown field %s", key)
|
||||
}
|
||||
}
|
||||
for _, f := range n.fields {
|
||||
item, exists := object[f.name]
|
||||
if err := assignField(f, dst.Field(f.index), item, exists); err != nil {
|
||||
return fmt.Errorf("%s: %w", f.name, err)
|
||||
}
|
||||
}
|
||||
case reflect.Slice:
|
||||
items, ok := value.([]any)
|
||||
if !ok {
|
||||
return errors.New("expected array")
|
||||
}
|
||||
dst.Set(reflect.MakeSlice(n.typ, len(items), len(items)))
|
||||
for i, item := range items {
|
||||
if err := setValue(n.elem, dst.Index(i), item); err != nil {
|
||||
return fmt.Errorf("item %d: %w", i, err)
|
||||
}
|
||||
}
|
||||
case reflect.Map:
|
||||
items, ok := value.(map[string]any)
|
||||
if !ok {
|
||||
return errors.New("expected object")
|
||||
}
|
||||
dst.Set(reflect.MakeMapWithSize(n.typ, len(items)))
|
||||
for key, item := range items {
|
||||
v := reflect.New(n.typ.Elem()).Elem()
|
||||
if err := setValue(n.elem, v, item); err != nil {
|
||||
return fmt.Errorf("map value: %w", err)
|
||||
}
|
||||
dst.SetMapIndex(reflect.ValueOf(key).Convert(n.typ.Key()), v)
|
||||
}
|
||||
case reflect.String:
|
||||
s, ok := value.(string)
|
||||
if !ok {
|
||||
return errors.New("expected string")
|
||||
}
|
||||
dst.SetString(s)
|
||||
case reflect.Bool:
|
||||
v, ok := value.(bool)
|
||||
if !ok {
|
||||
return errors.New("expected boolean")
|
||||
}
|
||||
dst.SetBool(v)
|
||||
default:
|
||||
num, ok := value.(json.Number)
|
||||
if !ok {
|
||||
return errors.New("expected number")
|
||||
}
|
||||
switch n.typ.Kind() {
|
||||
case reflect.Float32, reflect.Float64:
|
||||
v, err := strconv.ParseFloat(string(num), n.typ.Bits())
|
||||
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||
return errors.New("number out of range")
|
||||
}
|
||||
dst.SetFloat(v)
|
||||
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
v, err := strconv.ParseUint(string(num), 10, n.typ.Bits())
|
||||
if err != nil {
|
||||
return errors.New("invalid unsigned integer")
|
||||
}
|
||||
dst.SetUint(v)
|
||||
default:
|
||||
v, err := strconv.ParseInt(string(num), 10, n.typ.Bits())
|
||||
if err != nil {
|
||||
return errors.New("invalid integer")
|
||||
}
|
||||
dst.SetInt(v)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Token decoding rejects duplicate keys at every level (encoding/json normally
|
||||
// keeps the last value). Depth and body limits bound client-controlled work.
|
||||
func decodeValue(dec *json.Decoder, depth int) (any, error) {
|
||||
if depth > 64 {
|
||||
return nil, errors.New("JSON nesting exceeds 64")
|
||||
}
|
||||
token, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
switch token {
|
||||
case json.Delim('{'):
|
||||
object := map[string]any{}
|
||||
for dec.More() {
|
||||
key, err := dec.Token()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
name, ok := key.(string)
|
||||
if !ok {
|
||||
return nil, errors.New("invalid object key")
|
||||
}
|
||||
if _, exists := object[name]; exists {
|
||||
return nil, errors.New("duplicate JSON field")
|
||||
}
|
||||
value, err := decodeValue(dec, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
object[name] = value
|
||||
}
|
||||
end, err := dec.Token()
|
||||
if err != nil || end != json.Delim('}') {
|
||||
return nil, errors.New("unterminated object")
|
||||
}
|
||||
return object, nil
|
||||
case json.Delim('['):
|
||||
items := []any{}
|
||||
for dec.More() {
|
||||
v, err := decodeValue(dec, depth+1)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, v)
|
||||
}
|
||||
end, err := dec.Token()
|
||||
if err != nil || end != json.Delim(']') {
|
||||
return nil, errors.New("unterminated array")
|
||||
}
|
||||
return items, nil
|
||||
default:
|
||||
if _, bad := token.(json.Delim); bad {
|
||||
return nil, errors.New("unexpected JSON delimiter")
|
||||
}
|
||||
return token, nil
|
||||
}
|
||||
}
|
||||
394
contract/contract.go
Normal file
394
contract/contract.go
Normal file
@@ -0,0 +1,394 @@
|
||||
// Package contract compiles API field tags into a reusable request binder and
|
||||
// OpenAPI schemas. Generated services and the CLI use the same compiler.
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"reflect"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"github.com/go-playground/validator/v10"
|
||||
)
|
||||
|
||||
const DefaultMaxBodyBytes int64 = 1 << 20
|
||||
|
||||
var wireName = regexp.MustCompile(`^[a-z][A-Za-z0-9]*$`)
|
||||
var headerName = regexp.MustCompile(`^[A-Za-z][A-Za-z0-9-]*$`)
|
||||
var valuesValidator = validator.New()
|
||||
|
||||
// Binder is immutable after compilation and safe for concurrent requests.
|
||||
type Binder struct {
|
||||
root *node
|
||||
typ reflect.Type
|
||||
}
|
||||
type node struct {
|
||||
typ reflect.Type
|
||||
schema *openapi.Schema
|
||||
fields []field
|
||||
elem *node
|
||||
}
|
||||
type field struct {
|
||||
index int
|
||||
name, source string
|
||||
value *node
|
||||
optional bool
|
||||
fallback any
|
||||
hasDefault bool
|
||||
binding string
|
||||
}
|
||||
|
||||
func New[T any]() (*Binder, error) { return ForType(reflect.TypeOf((*T)(nil)).Elem()) }
|
||||
func MustNew[T any]() *Binder {
|
||||
b, err := New[T]()
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
// ForType also lets generators check the exact runtime contract before writing files.
|
||||
// The root must be a struct; recursive and custom-marshaling types are rejected.
|
||||
func ForType(t reflect.Type) (*Binder, error) {
|
||||
if t == nil || t.Kind() != reflect.Struct {
|
||||
return nil, errors.New("contract: root must be a struct")
|
||||
}
|
||||
budget := 10000
|
||||
n, err := compile(t, true, map[reflect.Type]bool{}, 0, &budget)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return &Binder{root: n, typ: t}, nil
|
||||
}
|
||||
|
||||
func compile(t reflect.Type, root bool, stack map[reflect.Type]bool, depth int, budget *int) (*node, error) {
|
||||
*budget--
|
||||
if *budget < 0 {
|
||||
return nil, errors.New("contract: schema expansion exceeds 10000 nodes")
|
||||
}
|
||||
if depth > 32 {
|
||||
return nil, errors.New("contract: type nesting exceeds 32")
|
||||
}
|
||||
if stack[t] {
|
||||
return nil, errors.New("contract: recursive types are not supported")
|
||||
}
|
||||
stack[t] = true
|
||||
defer delete(stack, t)
|
||||
n := &node{typ: t, schema: &openapi.Schema{}}
|
||||
// Wire behavior must be described completely by these tags.
|
||||
marshaler := reflect.TypeOf((*json.Marshaler)(nil)).Elem()
|
||||
unmarshaler := reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
|
||||
if t.Implements(marshaler) || reflect.PointerTo(t).Implements(marshaler) || t.Implements(unmarshaler) || reflect.PointerTo(t).Implements(unmarshaler) {
|
||||
return nil, errors.New("contract: custom JSON marshalers are unsupported")
|
||||
}
|
||||
switch t.Kind() {
|
||||
case reflect.Pointer:
|
||||
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n.elem = e
|
||||
*n.schema = *e.schema
|
||||
n.schema.Nullable = true
|
||||
case reflect.String:
|
||||
n.schema.Type = "string"
|
||||
case reflect.Bool:
|
||||
n.schema.Type = "boolean"
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64, reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
n.schema.Type = "integer"
|
||||
if t.Bits() <= 32 {
|
||||
n.schema.Format = "int32"
|
||||
} else {
|
||||
n.schema.Format = "int64"
|
||||
}
|
||||
if t.Kind() >= reflect.Uint && t.Kind() <= reflect.Uint64 {
|
||||
v := float64(0)
|
||||
n.schema.Minimum = &v
|
||||
if t.Bits() < 64 {
|
||||
max := float64((uint64(1) << uint(t.Bits())) - 1)
|
||||
n.schema.Maximum = &max
|
||||
if t.Bits() == 32 {
|
||||
n.schema.Format = "int64"
|
||||
}
|
||||
} else {
|
||||
n.schema.Format = ""
|
||||
}
|
||||
} else if t.Bits() < 32 {
|
||||
min, max := -float64(int64(1)<<uint(t.Bits()-1)), float64((int64(1)<<uint(t.Bits()-1))-1)
|
||||
n.schema.Minimum, n.schema.Maximum = &min, &max
|
||||
}
|
||||
case reflect.Float32, reflect.Float64:
|
||||
n.schema.Type = "number"
|
||||
if t.Kind() == reflect.Float32 {
|
||||
n.schema.Format = "float"
|
||||
} else {
|
||||
n.schema.Format = "double"
|
||||
}
|
||||
case reflect.Slice, reflect.Map:
|
||||
if t.Kind() == reflect.Map && t.Key().Kind() != reflect.String {
|
||||
return nil, errors.New("contract: map keys must be strings")
|
||||
}
|
||||
if t.Kind() == reflect.Slice && t.Elem().Kind() == reflect.Uint8 {
|
||||
return nil, errors.New("contract: byte slices use base64 in Go; use string for binary data")
|
||||
}
|
||||
e, err := compile(t.Elem(), false, stack, depth+1, budget)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
n.elem = e
|
||||
if t.Kind() == reflect.Slice {
|
||||
n.schema.Type = "array"
|
||||
n.schema.Items = e.schema
|
||||
} else {
|
||||
n.schema.Type = "object"
|
||||
n.schema.AdditionalProperties = e.schema
|
||||
}
|
||||
case reflect.Struct:
|
||||
n.schema.Type = "object"
|
||||
allowed := false
|
||||
n.schema.AdditionalPropertiesAllowed = &allowed
|
||||
n.schema.Properties = map[string]*openapi.Schema{}
|
||||
seen := map[string]bool{}
|
||||
for i := 0; i < t.NumField(); i++ {
|
||||
sf := t.Field(i)
|
||||
if sf.PkgPath != "" || sf.Anonymous {
|
||||
return nil, fmt.Errorf("contract: %s must be an exported named field", sf.Name)
|
||||
}
|
||||
f, err := compileField(sf, root, stack, depth+1, budget)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("contract: %s: %w", sf.Name, err)
|
||||
}
|
||||
f.index = i
|
||||
key := f.source + ":" + strings.ToLower(f.name)
|
||||
if seen[key] {
|
||||
return nil, fmt.Errorf("contract: duplicate field %s", f.name)
|
||||
}
|
||||
seen[key] = true
|
||||
n.fields = append(n.fields, f)
|
||||
if f.source == "json" {
|
||||
n.schema.Properties[f.name] = f.value.schema
|
||||
if !f.optional && !f.hasDefault {
|
||||
n.schema.Required = append(n.schema.Required, f.name)
|
||||
}
|
||||
}
|
||||
}
|
||||
default:
|
||||
return nil, fmt.Errorf("contract: unsupported type %s", t)
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
func compileField(sf reflect.StructField, root bool, stack map[reflect.Type]bool, depth int, budget *int) (field, error) {
|
||||
f := field{}
|
||||
// Reject typos and unsupported tags instead of silently losing validation.
|
||||
tags, err := ParseTags(string(sf.Tag))
|
||||
if err != nil {
|
||||
return f, err
|
||||
}
|
||||
var sourceTag string
|
||||
for key, value := range tags {
|
||||
switch key {
|
||||
case "json", "path", "form", "header":
|
||||
if value == "-" {
|
||||
return f, errors.New("ignored fields are not part of an API contract")
|
||||
}
|
||||
if f.source != "" {
|
||||
return f, errors.New("exactly one source tag is required")
|
||||
}
|
||||
f.source = key
|
||||
sourceTag = value
|
||||
case "binding":
|
||||
f.binding = value
|
||||
default:
|
||||
return f, fmt.Errorf("unsupported tag %q", key)
|
||||
}
|
||||
}
|
||||
if f.source == "" {
|
||||
return f, errors.New("json, path, form or header tag required")
|
||||
}
|
||||
if !root && f.source != "json" {
|
||||
return f, errors.New("nested fields must use json tags")
|
||||
}
|
||||
parts := strings.Split(sourceTag, ",")
|
||||
f.name = parts[0]
|
||||
if (f.source == "header" && !headerName.MatchString(f.name)) || (f.source != "header" && !wireName.MatchString(f.name)) {
|
||||
return f, errors.New("wire names must be lowerCamelCase (headers use HTTP names)")
|
||||
}
|
||||
n, err := compile(sf.Type, false, stack, depth, budget)
|
||||
if err != nil {
|
||||
return f, err
|
||||
}
|
||||
f.value = n
|
||||
scalar := base(n).typ.Kind()
|
||||
if f.source != "json" && !isScalar(scalar) && !(scalar == reflect.Slice && isScalar(base(base(n).elem).typ.Kind())) {
|
||||
return f, errors.New("path/query/header values must be scalars or scalar slices")
|
||||
}
|
||||
if f.source == "path" && !isScalar(scalar) {
|
||||
return f, errors.New("path values must be scalars")
|
||||
}
|
||||
if f.source == "header" && !isScalar(scalar) {
|
||||
return f, errors.New("header values must be scalars")
|
||||
}
|
||||
options := map[string]bool{}
|
||||
for _, option := range parts[1:] {
|
||||
key, value, _ := strings.Cut(option, "=")
|
||||
if options[key] {
|
||||
return f, fmt.Errorf("duplicate option %q", key)
|
||||
}
|
||||
options[key] = true
|
||||
switch key {
|
||||
case "optional", "omitempty":
|
||||
if option != key {
|
||||
return f, errors.New("optional takes no value")
|
||||
}
|
||||
f.optional = true
|
||||
case "default":
|
||||
if !isScalar(scalar) {
|
||||
return f, errors.New("default requires a scalar")
|
||||
}
|
||||
v, e := parseScalar(value, base(n).typ)
|
||||
if e != nil {
|
||||
return f, fmt.Errorf("invalid default: %w", e)
|
||||
}
|
||||
f.fallback = v
|
||||
f.hasDefault = true
|
||||
n.schema.Default = v
|
||||
case "options":
|
||||
if !isScalar(scalar) || value == "" {
|
||||
return f, errors.New("options requires scalar values")
|
||||
}
|
||||
for _, s := range strings.Split(value, "|") {
|
||||
v, e := parseScalar(s, base(n).typ)
|
||||
if e != nil {
|
||||
return f, e
|
||||
}
|
||||
n.schema.Enum = append(n.schema.Enum, v)
|
||||
}
|
||||
case "range":
|
||||
if !isNumber(scalar) {
|
||||
return f, errors.New("range requires numeric type")
|
||||
}
|
||||
if err = applyRange(n.schema, value); err != nil {
|
||||
return f, err
|
||||
}
|
||||
default:
|
||||
return f, fmt.Errorf("unsupported field option %q", option)
|
||||
}
|
||||
}
|
||||
if err = applyBinding(&f); err != nil {
|
||||
return f, err
|
||||
}
|
||||
if n.schema.Nullable && len(n.schema.Enum) > 0 {
|
||||
n.schema.Enum = append(n.schema.Enum, nil)
|
||||
}
|
||||
if f.source == "path" && (f.optional || f.hasDefault || sf.Type.Kind() == reflect.Pointer) {
|
||||
return f, errors.New("path fields must be required non-pointer values")
|
||||
}
|
||||
if f.hasDefault {
|
||||
v := reflect.New(sf.Type).Elem()
|
||||
if err = setValue(n, v, f.fallback); err != nil {
|
||||
return f, err
|
||||
}
|
||||
if err = checkField(f, v); err != nil {
|
||||
return f, fmt.Errorf("default violates validation: %w", err)
|
||||
}
|
||||
}
|
||||
return f, nil
|
||||
}
|
||||
|
||||
// ParseTags strictly parses Go struct tags, including duplicate/unknown key checks
|
||||
// performed by the compiler. It is also used to normalize generated Go JSON tags.
|
||||
func ParseTags(raw string) (map[string]string, error) {
|
||||
out := map[string]string{}
|
||||
for strings.TrimSpace(raw) != "" {
|
||||
raw = strings.TrimLeft(raw, " ")
|
||||
i := strings.Index(raw, ":")
|
||||
if i < 1 {
|
||||
return nil, errors.New("invalid struct tag")
|
||||
}
|
||||
key := raw[:i]
|
||||
raw = raw[i+1:]
|
||||
if strings.ContainsAny(key, " \t\r\n\"`") || len(raw) == 0 || raw[0] != '"' {
|
||||
return nil, errors.New("invalid struct tag")
|
||||
}
|
||||
end := 1
|
||||
for end < len(raw) {
|
||||
if raw[end] == '\\' {
|
||||
end += 2
|
||||
continue
|
||||
}
|
||||
if raw[end] == '"' {
|
||||
break
|
||||
}
|
||||
end++
|
||||
}
|
||||
if end >= len(raw) {
|
||||
return nil, errors.New("unterminated struct tag")
|
||||
}
|
||||
v, err := strconv.Unquote(raw[:end+1])
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if _, ok := out[key]; ok {
|
||||
return nil, fmt.Errorf("duplicate tag %q", key)
|
||||
}
|
||||
out[key] = v
|
||||
raw = raw[end+1:]
|
||||
if raw != "" && raw[0] != ' ' {
|
||||
return nil, errors.New("struct tags require spaces")
|
||||
}
|
||||
}
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func base(n *node) *node {
|
||||
for n.typ.Kind() == reflect.Pointer {
|
||||
n = n.elem
|
||||
}
|
||||
return n
|
||||
}
|
||||
func isNumber(k reflect.Kind) bool {
|
||||
return k >= reflect.Int && k <= reflect.Float64 && k != reflect.Uintptr
|
||||
}
|
||||
func isScalar(k reflect.Kind) bool { return k == reflect.String || k == reflect.Bool || isNumber(k) }
|
||||
|
||||
func (b *Binder) BodySchema() *openapi.Schema {
|
||||
if len(b.root.schema.Properties) == 0 {
|
||||
return nil
|
||||
}
|
||||
return copySchema(b.root.schema)
|
||||
}
|
||||
|
||||
// ResponseSchema rejects request-only fields to keep response docs honest.
|
||||
func (b *Binder) ResponseSchema() (*openapi.Schema, error) {
|
||||
for _, f := range b.root.fields {
|
||||
if f.source != "json" {
|
||||
return nil, errors.New("contract: response fields must use json tags")
|
||||
}
|
||||
}
|
||||
return copySchema(b.root.schema), nil
|
||||
}
|
||||
func (b *Binder) Parameters() []openapi.Parameter {
|
||||
var out []openapi.Parameter
|
||||
for _, f := range b.root.fields {
|
||||
if f.source == "json" {
|
||||
continue
|
||||
}
|
||||
in := f.source
|
||||
if in == "form" {
|
||||
in = "query"
|
||||
}
|
||||
out = append(out, openapi.Parameter{Name: f.name, In: in, Required: !f.optional && !f.hasDefault, Schema: copySchema(f.value.schema)})
|
||||
}
|
||||
return out
|
||||
}
|
||||
func copySchema(s *openapi.Schema) *openapi.Schema {
|
||||
data, _ := json.Marshal(s)
|
||||
var out openapi.Schema
|
||||
_ = json.Unmarshal(data, &out)
|
||||
return &out
|
||||
}
|
||||
244
contract/contract_test.go
Normal file
244
contract/contract_test.go
Normal file
@@ -0,0 +1,244 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"net/http/httptest"
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
apierrors "git.yuntuo.hk/k2safe/go-omni/errors"
|
||||
)
|
||||
|
||||
type address struct {
|
||||
City string `json:"city" binding:"required,max=30"`
|
||||
}
|
||||
type mixed struct {
|
||||
ID int64 `path:"userId,range=[1:100]"`
|
||||
Page int `form:"page,default=1,range=[1:50]"`
|
||||
Tags []string `form:"tag,optional" binding:"max=3"`
|
||||
Client string `header:"X-Client"`
|
||||
Name string `json:"name" binding:"required,max=20"`
|
||||
Active bool `json:"active"`
|
||||
Count *int `json:"count,optional"`
|
||||
Contact address `json:"contact"`
|
||||
Items []address `json:"items,optional"`
|
||||
Labels map[string]address `json:"labels,optional"`
|
||||
}
|
||||
|
||||
const validBody = `{"name":"小明","active":false,"count":0,"contact":{"city":"杭州"}}`
|
||||
|
||||
func bindMixed(body, query string) (mixed, error) {
|
||||
b := MustNew[mixed]()
|
||||
r := httptest.NewRequest("POST", "/users/7"+query, strings.NewReader(body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("X-Client", "web")
|
||||
var v mixed
|
||||
return v, b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||
}
|
||||
func TestMixedSourcesAndPresence(t *testing.T) {
|
||||
b := MustNew[mixed]()
|
||||
r := httptest.NewRequest("POST", "/users/7?tag=a&tag=b&userId=99&name=query", strings.NewReader(validBody))
|
||||
r.Header.Set("Content-Type", "application/json; charset=utf-8")
|
||||
r.Header.Set("X-Client", "web")
|
||||
v := mixed{Name: "previous"}
|
||||
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if v.ID != 7 || v.Page != 1 || v.Name != "小明" || v.Active || v.Count == nil || *v.Count != 0 || len(v.Tags) != 2 || v.Client != "web" || v.Contact.City != "杭州" {
|
||||
t.Fatalf("bad binding: %+v", v)
|
||||
}
|
||||
r = httptest.NewRequest("POST", "/", strings.NewReader(strings.Replace(validBody, `,"count":0`, "", 1)))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("X-Client", "web")
|
||||
if err := b.Bind(r, map[string]string{"userId": "7"}, &v); err != nil || v.Count != nil {
|
||||
t.Fatalf("missing != zero: %+v %v", v, err)
|
||||
}
|
||||
}
|
||||
func TestRejectInvalidRequests(t *testing.T) {
|
||||
cases := []struct{ name, body, query string }{
|
||||
{"missing", `{"name":"a","contact":{"city":"杭州"}}`, ""},
|
||||
{"empty", strings.Replace(validBody, `"小明"`, `""`, 1), ""},
|
||||
{"caseSensitive", strings.Replace(validBody, `"name"`, `"Name"`, 1), ""},
|
||||
{"unknown", strings.Replace(validBody, `"active":false`, `"active":false,"extra":1`, 1), ""},
|
||||
{"duplicate", strings.Replace(validBody, `"active":false`, `"active":false,"active":true`, 1), ""},
|
||||
{"nestedDuplicate", strings.Replace(validBody, `"city":"杭州"`, `"city":"杭州","city":"北京"`, 1), ""},
|
||||
{"nestedRequired", strings.Replace(validBody, `{"city":"杭州"}`, `{}`, 1), ""},
|
||||
{"nestedUnknown", strings.Replace(validBody, `{"city":"杭州"}`, `{"city":"杭州","other":0}`, 1), ""},
|
||||
{"trailing", validBody + ` {}`, ""},
|
||||
{"null", strings.Replace(validBody, `false`, `null`, 1), ""},
|
||||
{"numberOverflow", strings.Replace(validBody, `"count":0`, `"count":99999999999999999999`, 1), ""},
|
||||
{"arrayValidation", strings.TrimSuffix(validBody, "}") + `,"items":[{"city":""}]}`, ""},
|
||||
{"mapValidation", strings.TrimSuffix(validBody, "}") + `,"labels":{"home":{"city":""}}}`, ""},
|
||||
{"queryDuplicate", validBody, "?page=1&page=2"},
|
||||
{"queryDefaultNotOnEmpty", validBody, "?page="},
|
||||
{"range", validBody, "?page=51"},
|
||||
{"malformedQuery", validBody, "?page=%zz"},
|
||||
{"arrayLimit", validBody, "?tag=a&tag=b&tag=c&tag=d"},
|
||||
}
|
||||
b := MustNew[mixed]()
|
||||
for _, tc := range cases {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
r := httptest.NewRequest("POST", "/"+tc.query, strings.NewReader(tc.body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("X-Client", "web")
|
||||
v := mixed{Name: "unchanged"}
|
||||
err := b.Bind(r, map[string]string{"userId": "7"}, &v)
|
||||
if err == nil || apierrors.From(err).Status != 400 {
|
||||
t.Fatalf("wanted 400: %v", err)
|
||||
}
|
||||
if v.Name != "unchanged" || v.Page != 0 {
|
||||
t.Fatal("failed binding modified target")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
func TestBodyLimitsAndMediaType(t *testing.T) {
|
||||
type request struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
b := MustNew[request]()
|
||||
for _, tc := range []struct {
|
||||
body, ct string
|
||||
limit int64
|
||||
status int
|
||||
}{
|
||||
{`{"name":"x"}`, "text/plain", 100, 415},
|
||||
{`{"name":"long"}`, "application/json", 10, 413},
|
||||
{`{"name":"x"}` + strings.Repeat(" ", 2048), "application/json", 1024, 413},
|
||||
{`{"name":"x"}`, "application/json", 12, 0},
|
||||
} {
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||
r.ContentLength = -1
|
||||
r.Header.Set("Content-Type", tc.ct)
|
||||
var req request
|
||||
err := b.Bind(r, nil, &req, tc.limit)
|
||||
if tc.status == 0 {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
} else if err == nil || apierrors.From(err).Status != tc.status {
|
||||
t.Fatalf("status %d wanted: %v", tc.status, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestSchemaUsesSameRules(t *testing.T) {
|
||||
b := MustNew[mixed]()
|
||||
s := b.BodySchema()
|
||||
if len(s.Required) != 3 || s.Properties["count"].Nullable != true || s.Properties["name"].MaxLength == nil || *s.Properties["name"].MaxLength != 20 {
|
||||
t.Fatalf("wrong schema: %+v", s)
|
||||
}
|
||||
encoded, err := json.Marshal(s)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.Contains(string(encoded), `"additionalProperties":false`) {
|
||||
t.Fatal(string(encoded))
|
||||
}
|
||||
params := b.Parameters()
|
||||
if len(params) != 4 || params[1].Required || params[1].Schema.Default != json.Number("1") {
|
||||
t.Fatalf("wrong parameters: %+v", params)
|
||||
}
|
||||
// Returned documentation snapshots do not mutate concurrent binders.
|
||||
s.Properties["name"].MaxLength = nil
|
||||
if b.BodySchema().Properties["name"].MaxLength == nil {
|
||||
t.Fatal("shared schema")
|
||||
}
|
||||
if _, err = b.ResponseSchema(); err == nil {
|
||||
t.Fatal("request sources in response accepted")
|
||||
}
|
||||
}
|
||||
func TestRulesAndConcurrentReuse(t *testing.T) {
|
||||
type req struct {
|
||||
Score float64 `json:"score,options=1.0|2,range=(0:2]"`
|
||||
Email string `json:"email" binding:"email"`
|
||||
Value *int `json:"value" binding:"required"`
|
||||
}
|
||||
b := MustNew[req]()
|
||||
var wg sync.WaitGroup
|
||||
for i := 0; i < 24; i++ {
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(`{"score":1,"email":"a@example.com","value":3}`))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
var v req
|
||||
if err := b.Bind(r, nil, &v); err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
}()
|
||||
}
|
||||
wg.Wait()
|
||||
for _, body := range []string{`{"score":0,"email":"a@example.com","value":1}`, `{"score":2,"email":"bad","value":1}`, `{"score":2,"email":"a@example.com","value":null}`, `{"score":2,"email":"a@example.com","value":0}`} {
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
var v req
|
||||
if err := b.Bind(r, nil, &v); err == nil {
|
||||
t.Fatal(body)
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestInvalidContracts(t *testing.T) {
|
||||
for _, tag := range []string{`json:"bad_name"`, `json:"a" form:"a"`, `json:"a,optionl"`, `json:"a,range=[5:2]"`, `json:"a,default=20,range=[1:10]"`, `json:"a,optional" binding:"required"`, `json:"a" binding:"unknown"`, `path:"a,optional"`, `json:"a" json:"b"`, `jsoon:"a"`, `json:"a,options=x"`} {
|
||||
typ := reflect.StructOf([]reflect.StructField{{Name: "Value", Type: reflect.TypeOf(0), Tag: reflect.StructTag(tag)}})
|
||||
if _, err := ForType(typ); err == nil {
|
||||
t.Fatalf("accepted %s", tag)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestExactLargeIntegerConstraints(t *testing.T) {
|
||||
type req struct {
|
||||
Value int64 `json:"value,range=[:9007199254740992]"`
|
||||
}
|
||||
b := MustNew[req]()
|
||||
for _, tc := range []struct {
|
||||
body string
|
||||
valid bool
|
||||
}{{`{"value":9007199254740992}`, true}, {`{"value":9007199254740993}`, false}} {
|
||||
r := httptest.NewRequest("POST", "/", strings.NewReader(tc.body))
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
var v req
|
||||
if err := b.Bind(r, nil, &v); (err == nil) != tc.valid {
|
||||
t.Fatalf("%s: %v", tc.body, err)
|
||||
}
|
||||
}
|
||||
type defaults struct {
|
||||
Value uint64 `form:"value,default=18446744073709551615,options=18446744073709551615"`
|
||||
}
|
||||
d := MustNew[defaults]()
|
||||
data, err := json.Marshal(d.Parameters())
|
||||
if err != nil || !strings.Contains(string(data), "18446744073709551615") {
|
||||
t.Fatalf("lost precision: %s %v", data, err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIntrinsicIntegerBounds(t *testing.T) {
|
||||
type req struct {
|
||||
Small uint8 `json:"small,range=[-1:500]"`
|
||||
Wide uint64 `json:"wide"`
|
||||
}
|
||||
b := MustNew[req]()
|
||||
s := b.BodySchema()
|
||||
if *s.Properties["small"].Minimum != 0 || *s.Properties["small"].Maximum != 255 || s.Properties["wide"].Format != "" {
|
||||
t.Fatalf("wrong integer bounds: %+v", s.Properties)
|
||||
}
|
||||
}
|
||||
|
||||
func FuzzStrictBody(f *testing.F) {
|
||||
for _, seed := range []string{validBody, `{}`, `{"name":"a","name":"b"}`, `[null]`, `{"x":[]}`} {
|
||||
f.Add(seed)
|
||||
}
|
||||
binder := MustNew[mixed]()
|
||||
f.Fuzz(func(t *testing.T, body string) {
|
||||
if len(body) > 16384 {
|
||||
t.Skip()
|
||||
}
|
||||
req := httptest.NewRequest("POST", "/", strings.NewReader(body))
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
req.Header.Set("X-Client", "fuzz")
|
||||
var value mixed
|
||||
_ = binder.Bind(req, map[string]string{"userId": "1"}, &value, 16384)
|
||||
})
|
||||
}
|
||||
259
contract/rules.go
Normal file
259
contract/rules.go
Normal file
@@ -0,0 +1,259 @@
|
||||
package contract
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"math"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"unicode/utf8"
|
||||
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
)
|
||||
|
||||
func applyRange(s *openapi.Schema, value string) error {
|
||||
if len(value) < 3 || (value[0] != '[' && value[0] != '(') || (value[len(value)-1] != ']' && value[len(value)-1] != ')') {
|
||||
return errors.New("invalid range: use [min:max], (min:max), or mixed bounds")
|
||||
}
|
||||
lo, hi, ok := strings.Cut(value[1:len(value)-1], ":")
|
||||
if !ok || (lo == "" && hi == "") {
|
||||
return errors.New("invalid numeric range")
|
||||
}
|
||||
if lo != "" {
|
||||
v, err := bound(lo)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if s.Minimum == nil || v > *s.Minimum || (v == *s.Minimum && value[0] == '(') {
|
||||
s.Minimum = &v
|
||||
s.ExclusiveMinimum = value[0] == '('
|
||||
}
|
||||
}
|
||||
if hi != "" {
|
||||
v, err := bound(hi)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if s.Maximum == nil || v < *s.Maximum || (v == *s.Maximum && value[len(value)-1] == ')') {
|
||||
s.Maximum = &v
|
||||
s.ExclusiveMaximum = value[len(value)-1] == ')'
|
||||
}
|
||||
}
|
||||
return checkBounds(s)
|
||||
}
|
||||
func bound(s string) (float64, error) {
|
||||
v, err := strconv.ParseFloat(s, 64)
|
||||
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) || math.Abs(v) > 1<<53 {
|
||||
return 0, errors.New("constraint bound must be finite and within ±2^53")
|
||||
}
|
||||
return v, nil
|
||||
}
|
||||
func checkBounds(s *openapi.Schema) error {
|
||||
if s.Minimum != nil && s.Maximum != nil && (*s.Minimum > *s.Maximum || (*s.Minimum == *s.Maximum && (s.ExclusiveMinimum || s.ExclusiveMaximum))) {
|
||||
return errors.New("empty numeric range")
|
||||
}
|
||||
if s.MinLength != nil && s.MaxLength != nil && *s.MinLength > *s.MaxLength {
|
||||
return errors.New("empty string length range")
|
||||
}
|
||||
if s.MinItems != nil && s.MaxItems != nil && *s.MinItems > *s.MaxItems {
|
||||
return errors.New("empty array length range")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func applyBinding(f *field) error {
|
||||
if f.binding == "" {
|
||||
return nil
|
||||
}
|
||||
n := base(f.value)
|
||||
k := n.typ.Kind()
|
||||
s := f.value.schema
|
||||
seen := map[string]bool{}
|
||||
for _, rule := range strings.Split(f.binding, ",") {
|
||||
name, value, _ := strings.Cut(rule, "=")
|
||||
if seen[name] {
|
||||
return fmt.Errorf("duplicate binding rule %s", name)
|
||||
}
|
||||
seen[name] = true
|
||||
switch name {
|
||||
case "required":
|
||||
if rule != name || f.optional || f.hasDefault {
|
||||
return errors.New("required conflicts with optional/default")
|
||||
}
|
||||
if !isScalar(k) {
|
||||
return errors.New("binding required supports scalars; other fields are already required by presence")
|
||||
}
|
||||
s.Nullable = false
|
||||
switch k {
|
||||
case reflect.String:
|
||||
one := 1
|
||||
s.MinLength = &one
|
||||
case reflect.Bool:
|
||||
s.Not = &openapi.Schema{Enum: []any{false}}
|
||||
default:
|
||||
s.Not = &openapi.Schema{Enum: []any{0}}
|
||||
}
|
||||
case "email":
|
||||
if rule != name || k != reflect.String {
|
||||
return errors.New("email requires a string")
|
||||
}
|
||||
s.Format = "email"
|
||||
case "min", "max", "len":
|
||||
if k == reflect.String || k == reflect.Slice {
|
||||
v, err := strconv.Atoi(value)
|
||||
if err != nil || v < 0 {
|
||||
return errors.New("length constraints must be nonnegative integers")
|
||||
}
|
||||
if k == reflect.String {
|
||||
if name == "min" || name == "len" {
|
||||
if s.MinLength == nil || v > *s.MinLength {
|
||||
s.MinLength = &v
|
||||
}
|
||||
}
|
||||
if name == "max" || name == "len" {
|
||||
s.MaxLength = &v
|
||||
}
|
||||
} else {
|
||||
if name == "min" || name == "len" {
|
||||
s.MinItems = &v
|
||||
}
|
||||
if name == "max" || name == "len" {
|
||||
s.MaxItems = &v
|
||||
}
|
||||
}
|
||||
} else if isNumber(k) && name != "len" {
|
||||
v, err := bound(value)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if name == "min" {
|
||||
if s.Minimum == nil || v > *s.Minimum {
|
||||
s.Minimum = &v
|
||||
s.ExclusiveMinimum = false
|
||||
}
|
||||
} else {
|
||||
if s.Maximum == nil || v < *s.Maximum {
|
||||
s.Maximum = &v
|
||||
s.ExclusiveMaximum = false
|
||||
}
|
||||
}
|
||||
} else {
|
||||
return fmt.Errorf("%s requires a number, string or slice", name)
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("unsupported binding rule %q; use required,min,max,len,email or source-tag options/range", name)
|
||||
}
|
||||
}
|
||||
return checkBounds(s)
|
||||
}
|
||||
|
||||
func parseScalar(raw string, t reflect.Type) (any, error) {
|
||||
switch t.Kind() {
|
||||
case reflect.String:
|
||||
return raw, nil
|
||||
case reflect.Bool:
|
||||
if raw == "true" {
|
||||
return true, nil
|
||||
}
|
||||
if raw == "false" {
|
||||
return false, nil
|
||||
}
|
||||
return nil, errors.New("expected true or false")
|
||||
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
|
||||
v, err := strconv.ParseInt(raw, 10, t.Bits())
|
||||
if err != nil {
|
||||
return nil, errors.New("invalid integer")
|
||||
}
|
||||
return json.Number(strconv.FormatInt(v, 10)), nil
|
||||
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
v, err := strconv.ParseUint(raw, 10, t.Bits())
|
||||
if err != nil {
|
||||
return nil, errors.New("invalid unsigned integer")
|
||||
}
|
||||
return json.Number(strconv.FormatUint(v, 10)), nil
|
||||
case reflect.Float32, reflect.Float64:
|
||||
v, err := strconv.ParseFloat(raw, t.Bits())
|
||||
if err != nil || math.IsInf(v, 0) || math.IsNaN(v) {
|
||||
return nil, errors.New("invalid finite number")
|
||||
}
|
||||
return json.Number(strconv.FormatFloat(v, 'g', -1, t.Bits())), nil
|
||||
}
|
||||
return nil, errors.New("expected scalar")
|
||||
}
|
||||
|
||||
func checkField(f field, v reflect.Value) error {
|
||||
if v.Kind() == reflect.Pointer {
|
||||
if v.IsNil() {
|
||||
if strings.Contains(","+f.binding+",", ",required,") {
|
||||
return errors.New("must not be null")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
v = v.Elem()
|
||||
}
|
||||
if strings.Contains(","+f.binding+",", ",required,") && v.IsZero() {
|
||||
return errors.New("must not be zero or empty")
|
||||
}
|
||||
s := f.value.schema
|
||||
if s.Format == "email" && valuesValidator.Var(v.Interface(), "email") != nil {
|
||||
return errors.New("invalid email")
|
||||
}
|
||||
if len(s.Enum) > 0 {
|
||||
found := false
|
||||
for _, item := range s.Enum {
|
||||
option := reflect.New(v.Type()).Elem()
|
||||
if setValue(base(f.value), option, item) == nil && reflect.DeepEqual(option.Interface(), v.Interface()) {
|
||||
found = true
|
||||
break
|
||||
}
|
||||
}
|
||||
if !found {
|
||||
return errors.New("not an allowed option")
|
||||
}
|
||||
}
|
||||
if isNumber(v.Kind()) {
|
||||
var number float64
|
||||
switch v.Kind() {
|
||||
case reflect.Float32, reflect.Float64:
|
||||
number = v.Float()
|
||||
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
|
||||
number = float64(v.Uint())
|
||||
if v.Uint() > 1<<53 {
|
||||
number = math.Inf(1)
|
||||
}
|
||||
default:
|
||||
number = float64(v.Int())
|
||||
if v.Int() > 1<<53 {
|
||||
number = math.Inf(1)
|
||||
} else if v.Int() < -(1 << 53) {
|
||||
number = math.Inf(-1)
|
||||
}
|
||||
}
|
||||
if s.Minimum != nil && (number < *s.Minimum || (s.ExclusiveMinimum && number == *s.Minimum)) {
|
||||
return errors.New("below minimum")
|
||||
}
|
||||
if s.Maximum != nil && (number > *s.Maximum || (s.ExclusiveMaximum && number == *s.Maximum)) {
|
||||
return errors.New("above maximum")
|
||||
}
|
||||
}
|
||||
if v.Kind() == reflect.String {
|
||||
size := utf8.RuneCountInString(v.String())
|
||||
if s.MinLength != nil && size < *s.MinLength {
|
||||
return errors.New("string too short")
|
||||
}
|
||||
if s.MaxLength != nil && size > *s.MaxLength {
|
||||
return errors.New("string too long")
|
||||
}
|
||||
}
|
||||
if v.Kind() == reflect.Slice {
|
||||
size := v.Len()
|
||||
if s.MinItems != nil && size < *s.MinItems {
|
||||
return errors.New("too few items")
|
||||
}
|
||||
if s.MaxItems != nil && size > *s.MaxItems {
|
||||
return errors.New("too many items")
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
35
database/configOverlay_test.go
Normal file
35
database/configOverlay_test.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package database
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestApplyEnvPreservesFilePoolAndHidesDSN(t *testing.T) {
|
||||
cfg := DefaultConfig()
|
||||
cfg.MaxOpenConns = 3
|
||||
cfg.MaxIdleConns = 1
|
||||
cfg.PingTimeout = time.Second
|
||||
t.Setenv("OMNI_TEST_DSN", "user:secret@tcp(localhost:3306)/test")
|
||||
t.Setenv("OMNI_TEST_MAX_IDLE_CONNS", "0")
|
||||
got, err := ApplyEnv(cfg, "OMNI_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if got.MaxOpenConns != 3 || got.MaxIdleConns != 0 || got.PingTimeout != time.Second {
|
||||
t.Fatalf("lost file defaults: %+v", got)
|
||||
}
|
||||
data, err := json.Marshal(got)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(data), "secret") || !strings.Contains(string(data), "maxOpenConns") {
|
||||
t.Fatalf("unsafe config encoding: %s", data)
|
||||
}
|
||||
t.Setenv("OMNI_TEST_MAX_OPEN_CONNS", "bad")
|
||||
if _, err = ApplyEnv(cfg, "OMNI_TEST_"); err == nil {
|
||||
t.Fatal("invalid override accepted")
|
||||
}
|
||||
}
|
||||
@@ -14,13 +14,13 @@ import (
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
Driver string
|
||||
DSN string
|
||||
MaxOpenConns int
|
||||
MaxIdleConns int
|
||||
ConnMaxLifetime time.Duration
|
||||
ConnMaxIdleTime time.Duration
|
||||
PingTimeout time.Duration
|
||||
Driver string `json:"driver"`
|
||||
DSN string `json:"-"`
|
||||
MaxOpenConns int `json:"maxOpenConns"`
|
||||
MaxIdleConns int `json:"maxIdleConns"`
|
||||
ConnMaxLifetime time.Duration `json:"connMaxLifetime"`
|
||||
ConnMaxIdleTime time.Duration `json:"connMaxIdleTime"`
|
||||
PingTimeout time.Duration `json:"pingTimeout"`
|
||||
}
|
||||
|
||||
func DefaultConfig() Config {
|
||||
@@ -35,16 +35,25 @@ func DefaultConfig() Config {
|
||||
|
||||
// ConfigFromEnv reads GO_OMNI_DB_* style settings. DSN is required.
|
||||
func ConfigFromEnv(prefix string) (Config, error) {
|
||||
return ApplyEnv(DefaultConfig(), prefix)
|
||||
}
|
||||
|
||||
// ApplyEnv overlays nonempty environment settings on file/default configuration.
|
||||
// DSN is environment-only when Config is decoded from JSON or YAML.
|
||||
func ApplyEnv(cfg Config, prefix string) (Config, error) {
|
||||
prefix = strings.TrimSpace(prefix)
|
||||
if prefix == "" {
|
||||
prefix = "GO_OMNI_DB_"
|
||||
}
|
||||
cfg := DefaultConfig()
|
||||
cfg.Driver = strings.TrimSpace(os.Getenv(prefix + "DRIVER"))
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "DRIVER")); value != "" {
|
||||
cfg.Driver = value
|
||||
}
|
||||
if cfg.Driver == "" {
|
||||
cfg.Driver = "mysql"
|
||||
}
|
||||
cfg.DSN = strings.TrimSpace(os.Getenv(prefix + "DSN"))
|
||||
if value := strings.TrimSpace(os.Getenv(prefix + "DSN")); value != "" {
|
||||
cfg.DSN = value
|
||||
}
|
||||
if cfg.DSN == "" {
|
||||
return Config{}, fmt.Errorf("%sDSN is required", prefix)
|
||||
}
|
||||
|
||||
245
database/migrate/migrate.go
Normal file
245
database/migrate/migrate.go
Normal file
@@ -0,0 +1,245 @@
|
||||
// Package migrate provides explicit MySQL schema migration jobs. Application
|
||||
// startup never runs migrations automatically; use a separate deployment step.
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"archive/zip"
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"database/sql"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
mysqlDriver "github.com/go-sql-driver/mysql"
|
||||
migration "github.com/golang-migrate/migrate/v4"
|
||||
migrateMySQL "github.com/golang-migrate/migrate/v4/database/mysql"
|
||||
"github.com/golang-migrate/migrate/v4/source/iofs"
|
||||
"io"
|
||||
"io/fs"
|
||||
"path"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type File struct {
|
||||
Version uint64 `json:"version"`
|
||||
Name string `json:"name"`
|
||||
Direction string `json:"direction"`
|
||||
SHA256 string `json:"sha256"`
|
||||
}
|
||||
|
||||
var filename = regexp.MustCompile(`^([0-9]+)_([A-Za-z][A-Za-z0-9]*)\.(up|down)\.sql$`)
|
||||
|
||||
// Snapshot returns immutable migration bytes and a reviewable digest manifest.
|
||||
// Follow lowerCamelCase names, e.g. 0001_createUsers.up.sql.
|
||||
func Snapshot(source fs.FS, directory string) (fs.FS, []File, error) {
|
||||
if source == nil || !fs.ValidPath(directory) {
|
||||
return nil, nil, errors.New("migrate: invalid source directory")
|
||||
}
|
||||
entries, err := fs.ReadDir(source, directory)
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
var archive bytes.Buffer
|
||||
writer := zip.NewWriter(&archive)
|
||||
manifest := []File{}
|
||||
seen := map[string]bool{}
|
||||
total := 0
|
||||
for _, entry := range entries {
|
||||
if entry.IsDir() {
|
||||
continue
|
||||
}
|
||||
parts := filename.FindStringSubmatch(entry.Name())
|
||||
if parts == nil {
|
||||
if strings.HasSuffix(entry.Name(), ".sql") {
|
||||
return nil, nil, fmt.Errorf("migrate: invalid migration filename %s", entry.Name())
|
||||
}
|
||||
continue
|
||||
}
|
||||
if len(manifest) >= 1000 {
|
||||
return nil, nil, errors.New("migrate: too many files")
|
||||
}
|
||||
version, err := strconv.ParseUint(parts[1], 10, 32)
|
||||
if err != nil || version == 0 {
|
||||
return nil, nil, errors.New("migrate: version must be 1..4294967295")
|
||||
}
|
||||
identity := fmt.Sprint(version) + parts[3]
|
||||
if seen[identity] {
|
||||
return nil, nil, errors.New("migrate: duplicate migration version/direction")
|
||||
}
|
||||
seen[identity] = true
|
||||
file, err := source.Open(path.Join(directory, entry.Name()))
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
data, err := io.ReadAll(io.LimitReader(file, (1<<20)+1))
|
||||
closeErr := file.Close()
|
||||
if err == nil {
|
||||
err = closeErr
|
||||
}
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
total += len(data)
|
||||
if len(data) == 0 || len(data) > 1<<20 || total > 16<<20 {
|
||||
return nil, nil, errors.New("migrate: files must be 1..1MiB and <=16MiB total")
|
||||
}
|
||||
sum := sha256.Sum256(data)
|
||||
manifest = append(manifest, File{Version: version, Name: parts[2], Direction: parts[3], SHA256: hex.EncodeToString(sum[:])})
|
||||
fileWriter, err := writer.CreateHeader(&zip.FileHeader{Name: entry.Name(), Method: zip.Store})
|
||||
if err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
if _, err = fileWriter.Write(data); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
}
|
||||
if len(manifest) == 0 {
|
||||
return nil, nil, errors.New("migrate: no SQL migrations")
|
||||
}
|
||||
for _, file := range manifest {
|
||||
if file.Direction == "down" && !seen[fmt.Sprint(file.Version)+"up"] {
|
||||
return nil, nil, errors.New("migrate: down file lacks up migration")
|
||||
}
|
||||
}
|
||||
sort.Slice(manifest, func(i, j int) bool {
|
||||
if manifest[i].Version == manifest[j].Version {
|
||||
return manifest[i].Direction < manifest[j].Direction
|
||||
}
|
||||
return manifest[i].Version < manifest[j].Version
|
||||
})
|
||||
if err = writer.Close(); err != nil {
|
||||
return nil, nil, err
|
||||
}
|
||||
files, err := zip.NewReader(bytes.NewReader(archive.Bytes()), int64(archive.Len()))
|
||||
return files, manifest, err
|
||||
}
|
||||
|
||||
type Runner struct {
|
||||
mu sync.Mutex
|
||||
engine *migration.Migrate
|
||||
db *sql.DB
|
||||
closed bool
|
||||
stopped bool
|
||||
}
|
||||
|
||||
func NewMySQL(ctx context.Context, dsn string, source fs.FS, directory string) (runner *Runner, err error) {
|
||||
if ctx == nil {
|
||||
return nil, errors.New("migrate: context required")
|
||||
}
|
||||
snapshot, _, err := Snapshot(source, directory)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
cfg, err := mysqlDriver.ParseDSN(dsn)
|
||||
if err != nil || cfg.DBName == "" {
|
||||
return nil, errors.New("migrate: valid MySQL DSN with database required")
|
||||
}
|
||||
cfg.MultiStatements = true
|
||||
if cfg.Timeout == 0 || cfg.Timeout > 5*time.Second {
|
||||
cfg.Timeout = 5 * time.Second
|
||||
}
|
||||
if cfg.ReadTimeout == 0 || cfg.ReadTimeout > 30*time.Second {
|
||||
cfg.ReadTimeout = 30 * time.Second
|
||||
}
|
||||
if cfg.WriteTimeout == 0 || cfg.WriteTimeout > 30*time.Second {
|
||||
cfg.WriteTimeout = 30 * time.Second
|
||||
}
|
||||
connector, err := mysqlDriver.NewConnector(cfg)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
db := sql.OpenDB(connector)
|
||||
db.SetMaxOpenConns(1)
|
||||
db.SetMaxIdleConns(1)
|
||||
defer func() {
|
||||
if err != nil {
|
||||
db.Close()
|
||||
}
|
||||
}()
|
||||
conn, err := db.Conn(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
driver, err := migrateMySQL.WithConnection(ctx, conn, &migrateMySQL.Config{DatabaseName: cfg.DBName, StatementTimeout: 30 * time.Second})
|
||||
if err != nil {
|
||||
conn.Close()
|
||||
return nil, err
|
||||
}
|
||||
src, err := iofs.New(snapshot, ".")
|
||||
if err != nil {
|
||||
driver.Close()
|
||||
return nil, err
|
||||
}
|
||||
engine, err := migration.NewWithInstance("iofs", src, cfg.DBName, driver)
|
||||
if err != nil {
|
||||
src.Close()
|
||||
driver.Close()
|
||||
return nil, err
|
||||
}
|
||||
engine.LockTimeout = 10 * time.Second
|
||||
return &Runner{engine: engine, db: db}, nil
|
||||
}
|
||||
|
||||
// Up applies pending up scripts. MySQL DDL can partially commit; a failed
|
||||
// migration remains dirty and requires operator investigation. No automatic
|
||||
// down/force/dirty reset is provided. Cancellation stops after the current SQL
|
||||
// operation; statements/read I/O are bounded to 30 seconds.
|
||||
func (r *Runner) Up(ctx context.Context) error {
|
||||
if ctx == nil {
|
||||
return errors.New("migrate: context required")
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed || r.stopped {
|
||||
return errors.New("migrate: closed")
|
||||
}
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- r.engine.Up() }()
|
||||
var err error
|
||||
select {
|
||||
case err = <-done:
|
||||
case <-ctx.Done():
|
||||
r.stopped = true // golang-migrate retains its graceful-stop state; create a fresh runner to retry.
|
||||
select {
|
||||
case r.engine.GracefulStop <- true:
|
||||
default:
|
||||
}
|
||||
err = <-done
|
||||
return errors.Join(ctx.Err(), err)
|
||||
}
|
||||
if errors.Is(err, migration.ErrNoChange) {
|
||||
return nil
|
||||
}
|
||||
return err
|
||||
}
|
||||
func (r *Runner) Version() (uint, bool, error) {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed {
|
||||
return 0, false, errors.New("migrate: closed")
|
||||
}
|
||||
version, dirty, err := r.engine.Version()
|
||||
if errors.Is(err, migration.ErrNilVersion) {
|
||||
return 0, false, nil
|
||||
}
|
||||
return version, dirty, err
|
||||
}
|
||||
func (r *Runner) Close() error {
|
||||
r.mu.Lock()
|
||||
defer r.mu.Unlock()
|
||||
if r.closed {
|
||||
return nil
|
||||
}
|
||||
r.closed = true
|
||||
a, b := r.engine.Close()
|
||||
return errors.Join(a, b, r.db.Close())
|
||||
}
|
||||
99
database/migrate/migrate_test.go
Normal file
99
database/migrate/migrate_test.go
Normal file
@@ -0,0 +1,99 @@
|
||||
package migrate
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
"go.opentelemetry.io/otel/sdk/trace/tracetest"
|
||||
"io/fs"
|
||||
"os"
|
||||
"testing"
|
||||
"testing/fstest"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestSnapshotBoundsAndIdentity(t *testing.T) {
|
||||
source := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("SELECT 1;")}}
|
||||
f, manifest, err := Snapshot(source, ".")
|
||||
if err != nil || len(manifest) != 1 {
|
||||
t.Fatal(manifest, err)
|
||||
}
|
||||
source["0001_createItems.up.sql"].Data[0] = 'x'
|
||||
raw, err := fs.ReadFile(f, "0001_createItems.up.sql")
|
||||
if err != nil || string(raw) != "SELECT 1;" {
|
||||
t.Fatal(string(raw), err)
|
||||
}
|
||||
for _, files := range []fstest.MapFS{{"0001_bad_name.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: []byte("x")}, "1_b.up.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.down.sql": &fstest.MapFile{Data: []byte("x")}}, {"0001_a.up.sql": &fstest.MapFile{Data: make([]byte, (1<<20)+1)}}} {
|
||||
if _, _, err = Snapshot(files, "."); err == nil {
|
||||
t.Fatal("invalid migration accepted")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestMySQLIntegration(t *testing.T) {
|
||||
dsn := os.Getenv("GO_OMNI_MIGRATE_TEST_DSN")
|
||||
if dsn == "" {
|
||||
t.Skip("requires disposable MySQL database")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
files := fstest.MapFS{"0001_createItems.up.sql": &fstest.MapFile{Data: []byte("CREATE TABLE testItems (id BIGINT NOT NULL PRIMARY KEY, value VARCHAR(64) NOT NULL);")}, "0002_seedItems.up.sql": &fstest.MapFile{Data: []byte("INSERT INTO testItems (id,value) VALUES (1,'original');")}}
|
||||
runner, err := NewMySQL(ctx, dsn, files, ".")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = runner.Up(ctx); err != nil {
|
||||
t.Fatal("repeat", err)
|
||||
}
|
||||
v, dirty, err := runner.Version()
|
||||
if err != nil || v != 2 || dirty {
|
||||
t.Fatal(v, dirty, err)
|
||||
}
|
||||
runner.Close()
|
||||
db, err := sql.Open("mysql", dsn)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer db.Close()
|
||||
exporter := tracetest.NewInMemoryExporter()
|
||||
tr, _ := telemetry.New(telemetry.Config{Service: "dbTest", SampleRatio: 1, Exporter: exporter})
|
||||
defer tr.Close()
|
||||
instrumented := telemetry.SQL{DB: db, Runtime: tr}
|
||||
if _, err = instrumented.ExecContext(ctx, "UPDATE testItems SET value=? WHERE id=1", "secret parameter"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rows, err := instrumented.QueryContext(ctx, "SELECT value FROM testItems WHERE id=1")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !rows.Next() {
|
||||
t.Fatal("missing row")
|
||||
}
|
||||
var value string
|
||||
if err = rows.Scan(&value); err != nil || value != "secret parameter" {
|
||||
t.Fatal(value, err)
|
||||
}
|
||||
rows.Close()
|
||||
tr.Flush(ctx)
|
||||
if len(exporter.GetSpans()) != 2 {
|
||||
t.Fatal("SQL spans missing")
|
||||
}
|
||||
files["0003_failMigration.up.sql"] = &fstest.MapFile{Data: []byte("INSERT INTO missingTable VALUES (1);")}
|
||||
runner, err = NewMySQL(ctx, dsn, files, ".")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer runner.Close()
|
||||
if err = runner.Up(ctx); err == nil {
|
||||
t.Fatal("bad migration passed")
|
||||
}
|
||||
v, dirty, err = runner.Version()
|
||||
if err != nil || v != 3 || !dirty {
|
||||
t.Fatal(v, dirty, err)
|
||||
}
|
||||
if err = runner.Up(ctx); err == nil {
|
||||
t.Fatal("dirty migration auto-cleared")
|
||||
}
|
||||
}
|
||||
15
examples/contracts/README.md
Normal file
15
examples/contracts/README.md
Normal file
@@ -0,0 +1,15 @@
|
||||
# 接口定义示例
|
||||
|
||||
`user.api` 是入口,导入 `shared.api`。包含 JSON/path/query/header 混合请求、分页与重复 query 参数、嵌套对象、optional 指针、user 与 admin/user 分组以及 Auth/Audit 命名中间件。
|
||||
|
||||
在 go-omni 根目录执行:
|
||||
|
||||
```sh
|
||||
go run ./cmd/go-omni api validate --api examples/contracts/user.api
|
||||
go run ./cmd/go-omni api generate --api examples/contracts/user.api \
|
||||
--dir ../userService --module example.com/userService --framework-dir .
|
||||
```
|
||||
|
||||
项目中补充 `internal/service` 的业务实现和 `internal/middleware` 的 Auth/Audit hook,再启动服务。初始占位实现返回 501。
|
||||
|
||||
需要立即能返回成功结果的最小项目,用 `go-omni api new` 创建问候示例。完整语法、目录、更新策略见 [API.md](../../API.md)。
|
||||
11
examples/contracts/shared.api
Normal file
11
examples/contracts/shared.api
Normal file
@@ -0,0 +1,11 @@
|
||||
syntax = "v1"
|
||||
|
||||
type Address {
|
||||
City string `json:"city" binding:"required,max=50"`
|
||||
}
|
||||
|
||||
type UserResp {
|
||||
UserID int64 `json:"userId"`
|
||||
Name string `json:"name"`
|
||||
Active bool `json:"active"`
|
||||
}
|
||||
58
examples/contracts/user.api
Normal file
58
examples/contracts/user.api
Normal file
@@ -0,0 +1,58 @@
|
||||
syntax = "v1"
|
||||
|
||||
info (
|
||||
title: "User API"
|
||||
version: "1.0.0"
|
||||
)
|
||||
|
||||
import "shared.api"
|
||||
|
||||
type (
|
||||
UpdateUserReq {
|
||||
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||
Notify bool `form:"notify,default=false"`
|
||||
Client string `header:"X-Client,optional"`
|
||||
Name string `json:"name" binding:"required,max=50"`
|
||||
Active bool `json:"active"`
|
||||
Age *int `json:"age,optional,range=[0:150]"`
|
||||
Address *Address `json:"address,optional"`
|
||||
}
|
||||
ListUsersReq {
|
||||
Page int `form:"page,default=1,range=[1:1000]"`
|
||||
PageSize int `form:"pageSize,default=20,range=[1:100]"`
|
||||
Tags []string `form:"tag,optional" binding:"max=10"`
|
||||
}
|
||||
ListUsersResp {
|
||||
Items []UserResp `json:"items"`
|
||||
Page int `json:"page"`
|
||||
Total int64 `json:"total"`
|
||||
}
|
||||
DeleteUserReq {
|
||||
UserID int64 `path:"userId,range=[1:1000000]"`
|
||||
}
|
||||
)
|
||||
|
||||
@server (
|
||||
group: user
|
||||
prefix: /api/v1
|
||||
)
|
||||
service user-api {
|
||||
@doc "Update a user"
|
||||
@handler updateUser
|
||||
put /users/:userId (UpdateUserReq) returns (UserResp)
|
||||
|
||||
@doc "List users"
|
||||
@handler listUsers
|
||||
get /users (ListUsersReq) returns (ListUsersResp)
|
||||
}
|
||||
|
||||
@server (
|
||||
group: admin/user
|
||||
prefix: /admin/v1
|
||||
middleware: Auth,Audit
|
||||
)
|
||||
service user-api {
|
||||
@doc "Delete a user; business-owned authentication and auditing required"
|
||||
@handler deleteUser
|
||||
delete /users/:userId (DeleteUserReq)
|
||||
}
|
||||
33
examples/modules/README.md
Normal file
33
examples/modules/README.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# 基础模块组合示例
|
||||
|
||||
在仓库根目录执行:
|
||||
|
||||
```sh
|
||||
go run ./examples/modules
|
||||
```
|
||||
|
||||
默认监听 `127.0.0.1:8080`,使用仅限开发的 `Bearer dev-token`。
|
||||
配置 `GO_OMNI_TRANSPORT=native` 可直接切换自研网络层,TLS 配置与主框架一致。
|
||||
|
||||
```sh
|
||||
curl -H 'Authorization: Bearer dev-token' http://127.0.0.1:8080/api/catalog
|
||||
curl -H 'Authorization: Bearer dev-token' -H 'Content-Type: application/json' \
|
||||
-H 'Idempotency-Key: report-1' -d '{"name":"report"}' http://127.0.0.1:8080/api/jobs
|
||||
curl -H 'Authorization: Bearer dev-token' http://127.0.0.1:8080/api/jobs
|
||||
```
|
||||
|
||||
catalog 缓存 30 秒;可通过启动环境变量 `GO_OMNI_DEMO_UPSTREAM` 指定可信 JSON 上游,
|
||||
演示 HTTP 连接池、请求 ID 传递与合并回源。URL 来自服务配置,不接受请求参数传入。
|
||||
|
||||
jobs 展示固定 worker、队列、结构化日志与状态查询;相同 Idempotency-Key 和请求内容重放
|
||||
第一次 202 响应,内容不同返回 409。每分钟的统计任务演示间隔调度。退出先排空任务,再关闭
|
||||
HTTP 客户端和缓存。这个示例的任务、缓存和幂等记录均保存在进程内。
|
||||
|
||||
组合测试通过真实 TLS 连接覆盖 nethttp、native 和实验 HTTP/2,检查缓存回源次数、
|
||||
幂等重放、认证不可被重放绕过、任务只提交一次与关闭流程:
|
||||
|
||||
```sh
|
||||
go test -race ./examples/modules
|
||||
```
|
||||
|
||||
完整 API、默认上限和适用范围见 [业务基础模块](../../MODULES.md)。
|
||||
133
examples/modules/main.go
Normal file
133
examples/modules/main.go
Normal file
@@ -0,0 +1,133 @@
|
||||
// modules combines caching, outbound calls, middleware and background jobs.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/cache"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpc"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/middleware"
|
||||
"git.yuntuo.hk/k2safe/go-omni/task"
|
||||
)
|
||||
|
||||
func newApp(settings omni.Config, token, upstream string) (application *omni.App, err error) {
|
||||
application = omni.New(omni.WithConfig(settings))
|
||||
defer func(app *omni.App) {
|
||||
if err != nil {
|
||||
_ = app.Close()
|
||||
}
|
||||
}(application)
|
||||
store, err := cache.NewMemory(cache.MemoryConfig{MaxEntries: 128, MaxBytes: 1 << 20})
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
application.OnShutdown(store.Close)
|
||||
loader, err := cache.NewLoader(store, cache.LoaderConfig{MaxLoads: 8, Timeout: time.Second})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
client, err := httpc.New(httpc.Config{Timeout: time.Second, MaxRetries: 1, MaxConnsPerHost: 8})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
application.OnShutdown(client.Close)
|
||||
jobs, err := task.New(task.Config{Workers: 2, QueueSize: 16, TaskTimeout: 2 * time.Second})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
// Reverse cleanup order: drain jobs before closing their dependencies.
|
||||
application.OnShutdown(jobs.Close)
|
||||
if err = jobs.Every("reportStats", time.Minute, func(ctx context.Context) error {
|
||||
logx.InfoContext(ctx, "jobs.stats", "stats", jobs.Stats())
|
||||
return nil
|
||||
}); err != nil {
|
||||
return application, err
|
||||
}
|
||||
limiter, err := middleware.NewRateLimiter(middleware.RateLimitConfig{Rate: 20, Burst: 40, MaxKeys: 128, Key: func(c *omni.Context) string { p, _ := auth.FromContext(c); return p.UserID }})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
idempotency, err := middleware.NewIdempotency(middleware.IdempotencyConfig{Scope: func(c *omni.Context) (string, error) { p, _ := auth.FromContext(c); return p.UserID, nil }})
|
||||
if err != nil {
|
||||
return application, err
|
||||
}
|
||||
api := application.Group("/api", middleware.Timeout(2*time.Second), auth.RequireAuth(auth.StaticAuthenticator{token: {UserID: "demo-user"}}), limiter.Middleware())
|
||||
api.GET("/catalog", func(c *omni.Context) error {
|
||||
value, err := loader.GetOrLoad(c.Context(), "publicCatalog:v1", 30*time.Second, func(ctx context.Context) ([]byte, error) {
|
||||
data := map[string]any{"name": "demo catalog"}
|
||||
if upstream != "" {
|
||||
if err := client.JSON(ctx, "GET", upstream, nil, &data); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
return json.Marshal(data)
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.OK(json.RawMessage(value))
|
||||
})
|
||||
var nextID atomic.Uint64
|
||||
api.POST("/jobs", func(c *omni.Context) error {
|
||||
var input struct {
|
||||
Name string `json:"name" binding:"required,max=100"`
|
||||
}
|
||||
if err := c.BindJSON(&input); err != nil {
|
||||
return err
|
||||
}
|
||||
id := nextID.Add(1)
|
||||
// This job deliberately outlives the HTTP request. Runner still applies
|
||||
// its own deadline and shutdown cancellation; log correlation survives.
|
||||
_, err := jobs.Submit(context.WithoutCancel(c.Context()), "demoJob", func(ctx context.Context) error {
|
||||
logx.InfoContext(ctx, "job.finished", "jobId", id, "name", input.Name)
|
||||
return nil
|
||||
})
|
||||
if errors.Is(err, task.ErrQueueFull) {
|
||||
return omni.NewError(503, "QUEUE_FULL", "job queue is full")
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Accepted(map[string]uint64{"jobId": id})
|
||||
}, idempotency.Middleware())
|
||||
api.GET("/jobs", func(c *omni.Context) error { return c.OK(jobs.Stats()) })
|
||||
return application, nil
|
||||
}
|
||||
|
||||
func run(ctx context.Context) error {
|
||||
settings, err := omni.LoadConfig("")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
|
||||
settings.Address = "127.0.0.1:8080"
|
||||
}
|
||||
token := os.Getenv("GO_OMNI_DEV_TOKEN")
|
||||
if token == "" {
|
||||
token = "dev-token"
|
||||
}
|
||||
application, err := newApp(settings, token, os.Getenv("GO_OMNI_DEMO_UPSTREAM"))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return application.RunContext(ctx)
|
||||
}
|
||||
|
||||
func main() {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := run(ctx); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
126
examples/modules/main_test.go
Normal file
126
examples/modules/main_test.go
Normal file
@@ -0,0 +1,126 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
// Exercise the composed feature chain through real sockets and all runners.
|
||||
func TestModulesConfiguredTransports(t *testing.T) {
|
||||
for _, transport := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||
t.Run(transport, func(t *testing.T) {
|
||||
var loads atomic.Int32
|
||||
upstream := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
loads.Add(1)
|
||||
if r.Header.Get("X-Request-ID") == "" {
|
||||
t.Error("missing outbound correlation")
|
||||
}
|
||||
io.WriteString(w, `{"name":"upstream catalog"}`)
|
||||
}))
|
||||
defer upstream.Close()
|
||||
settings, err := omni.LoadConfig("MODULES_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
settings.Transport = transport
|
||||
settings.TLS.CertFile = cert
|
||||
settings.TLS.KeyFile = key
|
||||
application, err := newApp(settings, "demo", upstream.URL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
listener.Close()
|
||||
t.Error("server failed to stop")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
request := func(method, path, body, key, requestID, token string) (*http.Response, string) {
|
||||
r, err := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
r.Header.Set("Idempotency-Key", key)
|
||||
r.Header.Set("X-Request-ID", requestID)
|
||||
r.Header.Set("Authorization", "Bearer "+token)
|
||||
response, err := client.Do(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
data, err := io.ReadAll(response.Body)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatal("unverified TLS")
|
||||
}
|
||||
if transport == omni.TransportHTTP2 && response.ProtoMajor != 2 {
|
||||
t.Fatal(response.Proto)
|
||||
}
|
||||
return response, string(data)
|
||||
}
|
||||
for n := 0; n < 2; n++ {
|
||||
response, body := request("GET", "/api/catalog", "", "", "catalogRequest", "demo")
|
||||
if response.StatusCode != 200 || !strings.Contains(body, "upstream catalog") {
|
||||
t.Fatal(response.StatusCode, body)
|
||||
}
|
||||
}
|
||||
if loads.Load() != 1 {
|
||||
t.Fatal("cache missed", loads.Load())
|
||||
}
|
||||
first, original := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "firstRequest", "demo")
|
||||
second, replayed := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "secondRequest", "demo")
|
||||
if first.StatusCode != 202 || second.StatusCode != 202 || original != replayed || second.Header.Get("Idempotency-Replayed") != "true" || second.Header.Get("X-Request-ID") != "firstRequest" {
|
||||
t.Fatal(first.StatusCode, second.StatusCode, original, replayed)
|
||||
}
|
||||
if response, _ := request("POST", "/api/jobs", `{"name":"other"}`, "job1", "conflictRequest", "demo"); response.StatusCode != 409 {
|
||||
t.Fatal(response.StatusCode)
|
||||
}
|
||||
if response, _ := request("POST", "/api/jobs", `{"name":"report"}`, "job1", "unauthorized", "wrong"); response.StatusCode != 401 {
|
||||
t.Fatal("replay bypassed authentication", response.StatusCode)
|
||||
}
|
||||
deadline := time.Now().Add(time.Second)
|
||||
for {
|
||||
response, body := request("GET", "/api/jobs", "", "", "statsRequest", "demo")
|
||||
if response.StatusCode != 200 {
|
||||
t.Fatal(response.StatusCode)
|
||||
}
|
||||
if strings.Contains(body, `"completed":1`) {
|
||||
break
|
||||
}
|
||||
if time.Now().After(deadline) {
|
||||
t.Fatal("job did not finish once", body)
|
||||
}
|
||||
time.Sleep(10 * time.Millisecond)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
14
examples/platform/README.md
Normal file
14
examples/platform/README.md
Normal file
@@ -0,0 +1,14 @@
|
||||
# 扩展模块组合示例
|
||||
|
||||
业务通过 `storage.Store` 写文件,身份接口和 Policy 控制权限,OpenAPI 与路由一起注册,链路追踪关联 HTTP 与文件操作。`newApp` 不依赖 Local/S3/OSS 类型;启动处本例选择 Local。可替换成 S3/R2/OSS 构造结果,业务路由不变。
|
||||
|
||||
```sh
|
||||
export GO_OMNI_DEMO_TOKEN='use-a-private-development-token'
|
||||
go run ./examples/platform --root /tmp/goOmniDemoFiles
|
||||
```
|
||||
|
||||
接口:`POST /api/files`,JSON 为 `{"name":"hello.txt","content":"hello"}`,请求加 `Authorization: Bearer <GO_OMNI_DEMO_TOKEN>`;返回 201 和存储信息。`GET /openapi.json` 同样要求身份。
|
||||
|
||||
也可传 `--config service.yaml`,配置规则见 [扩展模块](../../ADVANCED.md)。本例的静态身份只用于演示,实际应用注入 `auth.JWT` 或 `auth.Sessions`;trace provider 默认不外发,接入 collector 时注入 exporter/reader。
|
||||
|
||||
`TestPlatformTLSAdapters` 通过三个 Runner 的真实 TLS 请求验证 401/403/201、文件内容、非法 key、OpenAPI 和关闭。文件根目录必须是应用私有目录。示例不执行自动数据迁移,也不启动 Orbit。
|
||||
132
examples/platform/main.go
Normal file
132
examples/platform/main.go
Normal file
@@ -0,0 +1,132 @@
|
||||
// platform composes storage, authentication, permission policy, tracing and
|
||||
// OpenAPI using provider-neutral business dependencies.
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"flag"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
"syscall"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/openapi"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
)
|
||||
|
||||
type dependencies struct {
|
||||
Files storage.Store
|
||||
Auth auth.Authenticator
|
||||
Tracing *telemetry.Runtime
|
||||
}
|
||||
type upload struct {
|
||||
Name string `json:"name" binding:"required,max=128"`
|
||||
Content string `json:"content" binding:"max=4096"`
|
||||
}
|
||||
|
||||
// Dependencies are borrowed; the composition root owns their shutdown.
|
||||
func newApp(cfg omni.Config, d dependencies) (*omni.App, error) {
|
||||
if d.Files == nil || d.Auth == nil || d.Tracing == nil {
|
||||
return nil, errors.New("platform: dependencies required")
|
||||
}
|
||||
app := omni.New(omni.WithConfig(cfg))
|
||||
app.Use(d.Tracing.Middleware())
|
||||
api, err := openapi.New("File service", "1")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
input, err := openapi.SchemaOf[upload]()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
output, err := openapi.SchemaOf[storage.Info]()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
policy := auth.PolicyFunc(func(_ context.Context, p auth.Principal, action, resource string) error {
|
||||
if p.TenantID != "" && p.HasRole("writer") && action == "create" && resource == "files" {
|
||||
return nil
|
||||
}
|
||||
return errors.New("denied")
|
||||
})
|
||||
err = api.Handle(app, openapi.Operation{Method: "POST", Path: "/api/files", OperationID: "createFile", Summary: "Store a small text file", Request: input, BearerAuth: true, Responses: map[int]openapi.Response{201: {Description: "Created", Schema: openapi.Envelope(output)}}}, func(c *omni.Context) error {
|
||||
var in upload
|
||||
if err := c.BindJSON(&in); err != nil {
|
||||
return err
|
||||
}
|
||||
p, _ := auth.FromContext(c)
|
||||
if strings.ContainsAny(in.Name, "/\\") {
|
||||
return omni.BadRequest("INVALID_NAME", "name must be one path segment")
|
||||
}
|
||||
key := p.TenantID + "/" + in.Name
|
||||
if err := storage.Check(c.Context(), key); err != nil {
|
||||
return omni.BadRequest("INVALID_KEY", "invalid storage key")
|
||||
}
|
||||
var info storage.Info
|
||||
err := d.Tracing.Do(c.Context(), "files.put", func(ctx context.Context) error {
|
||||
var err error
|
||||
info, err = d.Files.Put(ctx, key, strings.NewReader(in.Content), int64(len(in.Content)), "text/plain; charset=utf-8")
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return c.Created(info)
|
||||
}, auth.RequireAuth(d.Auth), auth.RequirePermission(policy, "create", func(*omni.Context) string { return "files" }))
|
||||
if err != nil {
|
||||
app.Close()
|
||||
return nil, err
|
||||
}
|
||||
app.GET("/openapi.json", api.Handler(), auth.RequireAuth(d.Auth))
|
||||
return app, nil
|
||||
}
|
||||
func run(ctx context.Context, configFile, root string) error {
|
||||
var cfg config.Config
|
||||
var err error
|
||||
if configFile != "" {
|
||||
cfg, err = config.LoadFile(configFile, "")
|
||||
} else {
|
||||
cfg, err = config.LoadFromEnv("")
|
||||
if os.Getenv("GO_OMNI_HTTP_ADDR") == "" {
|
||||
cfg.Address = "127.0.0.1:8080"
|
||||
}
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
token := os.Getenv("GO_OMNI_DEMO_TOKEN")
|
||||
if token == "" {
|
||||
return errors.New("set GO_OMNI_DEMO_TOKEN for the demonstration credential")
|
||||
}
|
||||
files, err := local.New(root, 64<<10)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tracing, err := telemetry.New(telemetry.Config{Service: "fileService", SampleRatio: 1})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tracing.Close()
|
||||
app, err := newApp(cfg, dependencies{Files: files, Auth: auth.StaticAuthenticator{token: {UserID: "demoUser", TenantID: "demoTenant", Roles: []string{"writer"}}}, Tracing: tracing})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
return app.RunContext(ctx)
|
||||
}
|
||||
func main() {
|
||||
file := flag.String("config", "", "optional JSON/YAML config")
|
||||
root := flag.String("root", "./fileData", "private local object directory")
|
||||
flag.Parse()
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := run(ctx, *file, *root); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
102
examples/platform/main_test.go
Normal file
102
examples/platform/main_test.go
Normal file
@@ -0,0 +1,102 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/config"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
"git.yuntuo.hk/k2safe/go-omni/storage/local"
|
||||
"git.yuntuo.hk/k2safe/go-omni/telemetry"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestPlatformTLSAdapters(t *testing.T) {
|
||||
for _, name := range []string{omni.TransportNetHTTP, omni.TransportNative, omni.TransportHTTP2} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
cfg := config.Default()
|
||||
cfg.Transport = name
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
cfg.TLS = config.TLSConfig{CertFile: cert, KeyFile: key}
|
||||
store, _ := local.New(t.TempDir(), 64<<10)
|
||||
tr, _ := telemetry.New(telemetry.Config{Service: "platformTest", SampleRatio: 1})
|
||||
defer tr.Close()
|
||||
app, err := newApp(cfg, dependencies{Files: store, Auth: auth.StaticAuthenticator{"writer": {UserID: "u", TenantID: "t", Roles: []string{"writer"}}, "reader": {UserID: "r", TenantID: "t"}}, Tracing: tr})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- app.ServeContext(ctx, listener) }()
|
||||
defer func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
listener.Close()
|
||||
t.Error("shutdown timeout")
|
||||
}
|
||||
}()
|
||||
client := &http.Client{Timeout: 3 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
request := func(method, path, body, token string) (int, string) {
|
||||
t.Helper()
|
||||
r, _ := http.NewRequest(method, "https://"+listener.Addr().String()+path, strings.NewReader(body))
|
||||
r.Header.Set("Authorization", "Bearer "+token)
|
||||
r.Header.Set("Content-Type", "application/json")
|
||||
response, err := client.Do(r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, err := io.ReadAll(response.Body)
|
||||
response.Body.Close()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if response.TLS == nil || len(response.TLS.VerifiedChains) == 0 {
|
||||
t.Fatal("TLS unverified")
|
||||
}
|
||||
if name == omni.TransportHTTP2 && response.ProtoMajor != 2 {
|
||||
t.Fatal("protocol")
|
||||
}
|
||||
return response.StatusCode, string(raw)
|
||||
}
|
||||
for _, test := range []struct {
|
||||
token string
|
||||
status int
|
||||
}{{"", 401}, {"reader", 403}, {"writer", 201}} {
|
||||
status, body := request("POST", "/api/files", `{"name":"hello.txt","content":"hello"}`, test.token)
|
||||
if status != test.status {
|
||||
t.Fatal(status, body)
|
||||
}
|
||||
}
|
||||
object, err := store.Get(context.Background(), "t/hello.txt")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, _ := io.ReadAll(object.Body)
|
||||
object.Body.Close()
|
||||
if string(raw) != "hello" {
|
||||
t.Fatal(string(raw))
|
||||
}
|
||||
if status, _ := request("POST", "/api/files", `{"name":"../escape","content":"x"}`, "writer"); status != 400 {
|
||||
t.Fatal(status)
|
||||
}
|
||||
if status, body := request("GET", "/openapi.json", "", "writer"); status != 200 || !strings.Contains(body, `"/api/files"`) {
|
||||
t.Fatal(status, body)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -2,6 +2,7 @@ package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"strings"
|
||||
@@ -41,28 +42,52 @@ func registerRoutes(application *omni.App, users service.UserRepository, logger
|
||||
}
|
||||
|
||||
func main() {
|
||||
logger, err := logx.New(logx.Config{
|
||||
Service: "reference-api",
|
||||
Environment: "local",
|
||||
Format: "json",
|
||||
Writer: os.Stdout,
|
||||
})
|
||||
if err != nil {
|
||||
os.Exit(1)
|
||||
}
|
||||
defer logger.Close()
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
if err := runReference(ctx); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func referenceConfig() (omni.Config, error) {
|
||||
settings, err := omni.LoadConfig("")
|
||||
if err != nil {
|
||||
return settings, err
|
||||
}
|
||||
// Preserve the example's loopback default and legacy address variable.
|
||||
if strings.TrimSpace(os.Getenv("GO_OMNI_HTTP_ADDR")) == "" {
|
||||
settings.Address = strings.TrimSpace(os.Getenv("GO_OMNI_ADDR"))
|
||||
if settings.Address == "" {
|
||||
settings.Address = "127.0.0.1:8080"
|
||||
}
|
||||
}
|
||||
if settings.Log.Service == "" {
|
||||
settings.Log.Service = "reference-api"
|
||||
}
|
||||
if settings.Log.Environment == "" {
|
||||
settings.Log.Environment = "local"
|
||||
}
|
||||
if strings.TrimSpace(os.Getenv("GO_OMNI_LOG_OUTPUT")) == "" {
|
||||
settings.Log.Output = "stdout"
|
||||
}
|
||||
return settings, nil
|
||||
}
|
||||
|
||||
func runReference(ctx context.Context) error {
|
||||
settings, err := referenceConfig()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
logger, err := logx.New(settings.Log)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer logger.Close()
|
||||
logx.SetDefault(logger)
|
||||
users, closeStorage, storage, err := openUsers(ctx, logger)
|
||||
if err != nil {
|
||||
logger.Error("storage initialization failed", "error", err)
|
||||
os.Exit(1)
|
||||
return err
|
||||
}
|
||||
defer func() {
|
||||
if err := closeStorage(); err != nil {
|
||||
logger.Error("storage close failed", "error", err)
|
||||
}
|
||||
}()
|
||||
devToken := strings.TrimSpace(os.Getenv("GO_OMNI_DEV_TOKEN"))
|
||||
if devToken == "" {
|
||||
devToken = "dev-token"
|
||||
@@ -70,16 +95,11 @@ func main() {
|
||||
authenticator := auth.StaticAuthenticator{
|
||||
devToken: {UserID: "dev-user", Roles: []string{"admin"}},
|
||||
}
|
||||
application := newProtectedApp(users, logger, authenticator)
|
||||
address := os.Getenv("GO_OMNI_ADDR")
|
||||
if address == "" {
|
||||
address = "127.0.0.1:8080"
|
||||
}
|
||||
logger.Info("starting reference API", "address", address, "storage", storage)
|
||||
if err := application.RunContext(ctx, address); err != nil {
|
||||
logger.Error("server stopped", "error", err)
|
||||
os.Exit(1)
|
||||
}
|
||||
application := newProtectedApp(users, logger, authenticator, omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
application.OnShutdown(closeStorage)
|
||||
logger.Info("starting reference API", "address", settings.Address, "transport", settings.Transport, "tls", settings.TLS.CertFile != "", "storage", storage)
|
||||
return application.RunContext(ctx)
|
||||
}
|
||||
|
||||
func openUsers(ctx context.Context, logger *logx.Logger) (service.UserRepository, func() error, string, error) {
|
||||
|
||||
@@ -118,7 +118,7 @@ func TestUserCRUD(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func checkUserCRUD(t *testing.T, app *omni.App) {
|
||||
func checkUserCRUD(t *testing.T, app http.Handler) {
|
||||
t.Helper()
|
||||
health := dataAs[map[string]string](t, request(t, app, "GET", "/health", "", 200, "OK"))
|
||||
if health["status"] != "ok" {
|
||||
|
||||
182
examples/reference/transport_test.go
Normal file
182
examples/reference/transport_test.go
Normal file
@@ -0,0 +1,182 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/auth"
|
||||
"git.yuntuo.hk/k2safe/go-omni/examples/reference/repository"
|
||||
"git.yuntuo.hk/k2safe/go-omni/internal/testutil"
|
||||
)
|
||||
|
||||
// networkHandler runs the same business assertions through a real TCP client.
|
||||
// The application handler is only called by its configured server transport.
|
||||
type networkHandler struct {
|
||||
t *testing.T
|
||||
client *http.Client
|
||||
address, scheme, token, alpn string
|
||||
proto int
|
||||
requests *int
|
||||
}
|
||||
|
||||
func (h networkHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
h.t.Helper()
|
||||
req := r.Clone(r.Context())
|
||||
req.RequestURI = ""
|
||||
req.URL.Scheme, req.URL.Host, req.Host = h.scheme, h.address, h.address
|
||||
if h.token != "" && req.Header.Get("Authorization") == "" {
|
||||
req.Header.Set("Authorization", "Bearer "+h.token)
|
||||
}
|
||||
response, err := h.client.Do(req)
|
||||
if err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if strings.HasPrefix(req.URL.Path, "/api/v1/") {
|
||||
*h.requests++
|
||||
}
|
||||
if response.ProtoMajor != h.proto {
|
||||
h.t.Fatalf("protocol %s, want HTTP/%d", response.Proto, h.proto)
|
||||
}
|
||||
if h.scheme == "https" && (response.TLS == nil || len(response.TLS.VerifiedChains) == 0 || response.TLS.NegotiatedProtocol != h.alpn) {
|
||||
h.t.Fatalf("TLS/ALPN verification failed: %+v", response.TLS)
|
||||
}
|
||||
for key, values := range response.Header {
|
||||
w.Header()[key] = append([]string(nil), values...)
|
||||
}
|
||||
w.WriteHeader(response.StatusCode)
|
||||
if _, err := io.Copy(w, response.Body); err != nil {
|
||||
h.t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
type lockedLog struct {
|
||||
sync.Mutex
|
||||
bytes.Buffer
|
||||
}
|
||||
|
||||
func (b *lockedLog) Write(p []byte) (int, error) {
|
||||
b.Lock()
|
||||
defer b.Unlock()
|
||||
return b.Buffer.Write(p)
|
||||
}
|
||||
|
||||
func (b *lockedLog) snapshot() string {
|
||||
b.Lock()
|
||||
defer b.Unlock()
|
||||
return b.Buffer.String()
|
||||
}
|
||||
|
||||
func TestBusinessConfiguredTransports(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
name, transport string
|
||||
tls bool
|
||||
proto int
|
||||
alpn string
|
||||
}{
|
||||
{"nethttp", omni.TransportNetHTTP, false, 1, ""},
|
||||
{"native", omni.TransportNative, false, 1, ""},
|
||||
{"nethttpTLS", omni.TransportNetHTTP, true, 2, "h2"},
|
||||
{"nativeTLS", omni.TransportNative, true, 1, "http/1.1"},
|
||||
{"http2TLS", omni.TransportHTTP2, true, 2, "h2"},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
cert, key, tlsClient := testutil.CertificateFiles(t)
|
||||
t.Setenv("REFERENCE_TEST_TRANSPORT", test.transport)
|
||||
t.Setenv("REFERENCE_TEST_SHUTDOWN_TIMEOUT", "3s")
|
||||
t.Setenv("REFERENCE_TEST_TLS_CERT_FILE", "")
|
||||
t.Setenv("REFERENCE_TEST_TLS_KEY_FILE", "")
|
||||
if test.tls {
|
||||
t.Setenv("REFERENCE_TEST_TLS_CERT_FILE", cert)
|
||||
t.Setenv("REFERENCE_TEST_TLS_KEY_FILE", key)
|
||||
}
|
||||
settings, err := omni.LoadConfig("REFERENCE_TEST_")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var logs lockedLog
|
||||
logger := testLogger(&logs)
|
||||
application := newProtectedApp(repository.NewMemoryUsers(), logger, auth.StaticAuthenticator{
|
||||
"admin": {UserID: "admin", Roles: []string{"admin"}},
|
||||
"user": {UserID: "user", Roles: []string{"user"}},
|
||||
}, omni.WithConfig(settings))
|
||||
application.OnShutdown(logger.Close)
|
||||
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- application.ServeContext(ctx, listener) }()
|
||||
t.Cleanup(func() {
|
||||
cancel()
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
t.Error(err)
|
||||
}
|
||||
case <-time.After(5 * time.Second):
|
||||
t.Error("business server did not stop")
|
||||
listener.Close()
|
||||
}
|
||||
})
|
||||
client := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{TLSClientConfig: tlsClient, ForceAttemptHTTP2: true}}
|
||||
defer client.CloseIdleConnections()
|
||||
scheme := "http"
|
||||
if test.tls {
|
||||
scheme = "https"
|
||||
}
|
||||
requests := 0
|
||||
network := networkHandler{t: t, client: client, address: listener.Addr().String(), scheme: scheme, proto: test.proto, alpn: test.alpn, requests: &requests}
|
||||
request(t, network, "POST", "/api/v1/users", `{"name":"Alice","email":"alice@example.com"}`, 401, "UNAUTHENTICATED")
|
||||
requestWithAuthorization(t, network, "DELETE", "/api/v1/users/1", "", "Bearer user", 403, "FORBIDDEN")
|
||||
network.token = "admin"
|
||||
checkUserCRUD(t, network)
|
||||
request(t, network, "POST", "/api/v1/users", `{"name":`, 400, "INVALID_BODY")
|
||||
count := 0
|
||||
for _, line := range strings.Split(strings.TrimSpace(logs.snapshot()), "\n") {
|
||||
var record map[string]any
|
||||
if err := json.Unmarshal([]byte(line), &record); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if record["requestId"] == "reference-test" {
|
||||
count++
|
||||
}
|
||||
}
|
||||
if count < requests {
|
||||
t.Fatalf("missing correlated business logs: %d", count)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestReferenceAddressCompatibility(t *testing.T) {
|
||||
t.Setenv("GO_OMNI_TRANSPORT", "native")
|
||||
t.Setenv("GO_OMNI_TLS_CERT_FILE", "")
|
||||
t.Setenv("GO_OMNI_TLS_KEY_FILE", "")
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "")
|
||||
t.Setenv("GO_OMNI_ADDR", "")
|
||||
settings, err := referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8080" || settings.Transport != "native" {
|
||||
t.Fatalf("default: %+v %v", settings, err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_ADDR", "127.0.0.1:8081")
|
||||
settings, err = referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8081" {
|
||||
t.Fatalf("legacy: %+v %v", settings, err)
|
||||
}
|
||||
t.Setenv("GO_OMNI_HTTP_ADDR", "127.0.0.1:8082")
|
||||
settings, err = referenceConfig()
|
||||
if err != nil || settings.Address != "127.0.0.1:8082" {
|
||||
t.Fatalf("standard: %+v %v", settings, err)
|
||||
}
|
||||
}
|
||||
6
examples/userService/.gitignore
vendored
Normal file
6
examples/userService/.gitignore
vendored
Normal file
@@ -0,0 +1,6 @@
|
||||
/server
|
||||
/var/
|
||||
.env
|
||||
.env.*
|
||||
!.env.example
|
||||
/.goOmni/api.lock
|
||||
19
examples/userService/.goOmni/apiManifest.json
Normal file
19
examples/userService/.goOmni/apiManifest.json
Normal file
@@ -0,0 +1,19 @@
|
||||
{
|
||||
"profile": {
|
||||
"database": "mysql",
|
||||
"storage": "local",
|
||||
"logOutput": "file",
|
||||
"logFormat": "json"
|
||||
},
|
||||
"version": 1,
|
||||
"module": "example.com/userService",
|
||||
"entry": "api/user.api",
|
||||
"generated": {
|
||||
"docs/openapi.json": "0675e4eb1278b36828aa31b12587b0d33d2f384112f67a43179eb72b52c3a0ea",
|
||||
"internal/handler/openapiGen.go": "8ddc60c338630db915b9f074ade776f526eabf60fc99b33e425359fd415c4926",
|
||||
"internal/handler/routesGen.go": "7f45030e7eb9f055c4b4d2ff12ad855d7c726aad4aee63c5b3c3f97559a64257",
|
||||
"internal/handler/user/handlersGen.go": "06f3c2798ff323812ff3eb2d8b933a569af3ce1b86a67b66036374add98c0d7e",
|
||||
"internal/middleware/registryGen.go": "a6122a5453dedc6a4415860108814c50e155128cfe5412b39dd33ce2d01b4b00",
|
||||
"internal/types/typesGen.go": "3ab2ce9ffe467c3ea53ac3d1fec283ecebc87cef57fd592d08dca991fd964876"
|
||||
}
|
||||
}
|
||||
79
examples/userService/README.md
Normal file
79
examples/userService/README.md
Normal file
@@ -0,0 +1,79 @@
|
||||
# API 驱动的用户管理示例
|
||||
|
||||
这个独立 Go module 从 `api/user.api` 生成,随后在开发者拥有的文件中实现业务。
|
||||
使用 MySQL 持久化、事务审计、环境变量 Token 鉴权、文件日志和 local 存储;业务层只接收标准 context 和基础模块接口。
|
||||
|
||||
## 启动
|
||||
|
||||
先准备空的业务数据库,再设置环境变量(替换占位值,密钥不写进配置或 Git):
|
||||
|
||||
```sh
|
||||
export GO_OMNI_DB_DSN='user:password@tcp(127.0.0.1:3306)/user_service?parseTime=true'
|
||||
export USER_SERVICE_API_TOKEN='<至少32字节的随机值>'
|
||||
export GO_OMNI_STORAGE_SIGNING_KEY='<另一个至少32字节的随机值>'
|
||||
export GO_OMNI_STORAGE_PUBLIC_URL='http://127.0.0.1:8080/storage/object'
|
||||
go mod tidy
|
||||
# 在本目录运行与框架版本一致的 CLI;迁移必须显式执行。
|
||||
go-omni migrate --dir migrations --apply
|
||||
go run ./cmd/server
|
||||
```
|
||||
|
||||
源码示例的 go.mod 使用 `replace ... => ../..`。复制到独立仓库时,删除这条 replace,
|
||||
保留已发布的固定依赖版本,再执行 go mod tidy。远程版本验证脚本会自动检查没有本地替换。
|
||||
`GO_OMNI_TRANSPORT=native` 可切换为 native,业务代码无需调整。
|
||||
|
||||
配置是默认值 → etc/config.yaml → 环境变量。日志默认写入 var/log/server.log 并轮转,
|
||||
`GO_OMNI_LOG_OUTPUT=stderr` 可以切换为标准错误输出。MySQL 默认最多 10 条连接。
|
||||
应用检查 users 和 user_audit 表,缺失时拒绝启动,不自动改表。
|
||||
|
||||
## 接口
|
||||
|
||||
除健康检查、OpenAPI 和持有效签名的对象下载外,业务接口需要 `Authorization: Bearer <Token>`。
|
||||
这个 Token 代表示例的管理员;可通过同一个命名中间件槽位替换为 JWT/OIDC 和业务授权策略。
|
||||
框架不会推断租户模型或自动给接口放行。
|
||||
|
||||
| 接口 | 行为 |
|
||||
| --- | --- |
|
||||
| POST /api/v1/users | 创建,返回 201;邮箱规范化并唯一 |
|
||||
| GET /api/v1/users/:id | 查询,缺失返回 404 |
|
||||
| GET /api/v1/users?page=1&pageSize=20 | 分页,pageSize 最大 100,空结果返回 [] |
|
||||
| PUT /api/v1/users/:id | 完整更新;邮箱冲突返回 409 |
|
||||
| DELETE /api/v1/users/:id | 删除,返回 204 |
|
||||
| POST /api/v1/users/:id/export | 私有文本导出,返回 5 分钟下载链接 |
|
||||
| GET /health/ready | 数据库不可用时返回 503,恢复后重新就绪 |
|
||||
| GET /openapi.json | 与 docs/openapi.json 一致 |
|
||||
|
||||
参数不合法返回 400,认证失败 401,邮箱重复 409,内部故障只返回通用 500。
|
||||
OpenAPI 的业务数据类型来自契约;401/404/409 的业务错误码见上表。
|
||||
|
||||
创建、更新、删除都会在**同一个 MySQL 事务**写入 user_audit;审计写入失败时业务变更一起回滚。
|
||||
事务提交后才写成功业务日志,requestId 同时出现在 HTTP 响应、业务日志和审计记录。
|
||||
数据库异常日志不记录 SQL、DSN 或用户输入。
|
||||
|
||||
文件导出演示 `deps.Uploads` 和 `deps.Downloads`。对象使用随机 key,签名 URL 不进入日志。
|
||||
签名过期只限制下载,不删除文件;实际项目应给 exports/ 配置保留期限或清理任务。
|
||||
只签发下载链接,不对外开放任意对象上传或签名接口。
|
||||
|
||||
## 开发与验证
|
||||
|
||||
```sh
|
||||
go-omni api generate --api api/user.api --dir .
|
||||
go-omni api generate --api api/user.api --dir . --check
|
||||
go test ./...
|
||||
```
|
||||
|
||||
Gen.go 和 docs/openapi.json 由生成器维护;Service、Repository、迁移、鉴权、配置和 main 保留。
|
||||
修改 .api 的类型/方法签名后,需要同步修改自己的业务实现并运行测试。
|
||||
|
||||
在框架仓库运行:
|
||||
|
||||
```sh
|
||||
python3 scripts/verifyAPIWorkflow.py
|
||||
# 已发布版本:全新 module cache 安装 CLI,无本地 replace 消费框架。
|
||||
python3 scripts/verifyAPIWorkflow.py --version v0.7.0-preview \
|
||||
--report performance/apiWorkflowRemote.json
|
||||
```
|
||||
|
||||
验收创建只绑定 localhost 的临时 MySQL 容器(1 CPU / 768 MiB),使用随机密钥,结束时删除。
|
||||
覆盖 nethttp/native 的 CRUD、校验、鉴权、事务错误/异常/取消回滚、数据库停机恢复、
|
||||
重启持久化、签名文件下载、文件日志与优雅退出。不会连接业务服务器。
|
||||
60
examples/userService/api/user.api
Normal file
60
examples/userService/api/user.api
Normal file
@@ -0,0 +1,60 @@
|
||||
syntax = "v1"
|
||||
info (
|
||||
title: "User service reference"
|
||||
version: "1.0.0"
|
||||
)
|
||||
type (
|
||||
UserResp {
|
||||
ID int64 `json:"id"`
|
||||
Name string `json:"name"`
|
||||
Email string `json:"email"`
|
||||
CreatedAt string `json:"createdAt"`
|
||||
UpdatedAt string `json:"updatedAt"`
|
||||
}
|
||||
CreateUserReq {
|
||||
Name string `json:"name" binding:"required,max=100"`
|
||||
Email string `json:"email" binding:"required,email,max=254"`
|
||||
}
|
||||
UserIDReq {
|
||||
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||
}
|
||||
UpdateUserReq {
|
||||
ID int64 `path:"id,range=[1:9007199254740991]"`
|
||||
Name string `json:"name" binding:"required,max=100"`
|
||||
Email string `json:"email" binding:"required,email,max=254"`
|
||||
}
|
||||
ListUsersReq {
|
||||
Page int `form:"page,default=1,range=[1:1000000]"`
|
||||
PageSize int `form:"pageSize,default=20,range=[1:100]"`
|
||||
}
|
||||
UserPageResp {
|
||||
Items []UserResp `json:"items"`
|
||||
Total int `json:"total"`
|
||||
Page int `json:"page"`
|
||||
PageSize int `json:"pageSize"`
|
||||
}
|
||||
ExportResp {
|
||||
DownloadURL string `json:"downloadUrl"`
|
||||
}
|
||||
)
|
||||
@server (
|
||||
group: user
|
||||
prefix: /api/v1
|
||||
middleware: Auth
|
||||
)
|
||||
service user-api {
|
||||
@handler createUser
|
||||
@status 201
|
||||
post /users (CreateUserReq) returns (UserResp)
|
||||
@handler getUser
|
||||
get /users/:id (UserIDReq) returns (UserResp)
|
||||
@handler listUsers
|
||||
get /users (ListUsersReq) returns (UserPageResp)
|
||||
@handler updateUser
|
||||
put /users/:id (UpdateUserReq) returns (UserResp)
|
||||
@handler deleteUser
|
||||
delete /users/:id (UserIDReq)
|
||||
@doc "Export one user as a private text object with a short-lived download link"
|
||||
@handler exportUser
|
||||
post /users/:id/export (UserIDReq) returns (ExportResp)
|
||||
}
|
||||
58
examples/userService/cmd/server/main.go
Normal file
58
examples/userService/cmd/server/main.go
Normal file
@@ -0,0 +1,58 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"example.com/userService/internal/handler"
|
||||
"example.com/userService/internal/servicecontext"
|
||||
"example.com/userService/internal/settings"
|
||||
"flag"
|
||||
omni "git.yuntuo.hk/k2safe/go-omni"
|
||||
"git.yuntuo.hk/k2safe/go-omni/httpx"
|
||||
"git.yuntuo.hk/k2safe/go-omni/logx"
|
||||
"log"
|
||||
"os"
|
||||
"os/signal"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
func main() {
|
||||
path := flag.String("config", "etc/config.yaml", "configuration file; empty uses defaults and environment")
|
||||
flag.Parse()
|
||||
if err := run(*path); err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
}
|
||||
func run(path string) (runErr error) {
|
||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
cfg, err := settings.Load(path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
logger, err := logx.New(cfg.Server.Log)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { runErr = errors.Join(runErr, logger.Close()) }()
|
||||
logx.SetDefault(logger)
|
||||
deps, err := servicecontext.New(ctx, logger, cfg)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer func() { runErr = errors.Join(runErr, deps.Close()) }()
|
||||
app := omni.New(omni.WithConfig(cfg.Server))
|
||||
app.OnShutdown(logger.Close)
|
||||
app.OnShutdown(deps.Close)
|
||||
for _, method := range []string{"GET", "HEAD", "PUT"} {
|
||||
app.Handle(method, "/storage/object", func(c *httpx.Context) error { deps.FileHTTP.ServeHTTP(c.Raw(), c.Request()); return nil })
|
||||
}
|
||||
if err = handler.RegisterRoutes(app, deps); err != nil {
|
||||
return err
|
||||
}
|
||||
app.GET("/openapi.json", handler.OpenAPI)
|
||||
app.GET("/health/live", deps.Health.LivenessHandler())
|
||||
app.GET("/health/ready", deps.Health.ReadinessHandler())
|
||||
logger.InfoContext(ctx, "service.starting", "address", cfg.Server.Address)
|
||||
return app.RunContext(ctx, cfg.Server.Address)
|
||||
}
|
||||
1249
examples/userService/docs/openapi.json
Normal file
1249
examples/userService/docs/openapi.json
Normal file
File diff suppressed because it is too large
Load Diff
36
examples/userService/etc/config.yaml
Normal file
36
examples/userService/etc/config.yaml
Normal file
@@ -0,0 +1,36 @@
|
||||
# Non-secret settings only. Environment variables override these values.
|
||||
server:
|
||||
address: '127.0.0.1:8080'
|
||||
enableLogger: true
|
||||
slowRequestThreshold: 500ms
|
||||
shutdownTimeout: 10s
|
||||
log:
|
||||
service: "example.com/userService"
|
||||
level: info
|
||||
format: json
|
||||
file: './var/log/server.log'
|
||||
rotation:
|
||||
enabled: true
|
||||
maxSizeMb: 100
|
||||
maxBackups: 10
|
||||
maxAgeDays: 7
|
||||
compress: true
|
||||
redactKeys: []
|
||||
storage:
|
||||
provider: local
|
||||
maxBytes: 33554432
|
||||
contentTypes: ['text/plain', 'image/png', 'image/jpeg', 'application/pdf']
|
||||
linkTtl: 5m
|
||||
requestTimeout: 30s
|
||||
root: './var/files'
|
||||
publicUrl: 'http://127.0.0.1:8080/storage/object'
|
||||
|
||||
|
||||
database:
|
||||
driver: mysql
|
||||
maxOpenConns: 10
|
||||
maxIdleConns: 5
|
||||
connMaxLifetime: 30m
|
||||
connMaxIdleTime: 5m
|
||||
pingTimeout: 5s
|
||||
# GO_OMNI_DB_DSN supplies the secret connection string.
|
||||
26
examples/userService/go.mod
Normal file
26
examples/userService/go.mod
Normal file
@@ -0,0 +1,26 @@
|
||||
module example.com/userService
|
||||
|
||||
go 1.22.0
|
||||
|
||||
require (
|
||||
git.yuntuo.hk/k2safe/go-omni v0.7.0-preview
|
||||
github.com/go-sql-driver/mysql v1.9.3
|
||||
)
|
||||
|
||||
require (
|
||||
filippo.io/edwards25519 v1.1.0 // indirect
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 // indirect
|
||||
github.com/go-playground/locales v0.14.1 // indirect
|
||||
github.com/go-playground/universal-translator v0.18.1 // indirect
|
||||
github.com/go-playground/validator/v10 v10.20.0 // indirect
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 // indirect
|
||||
github.com/leodido/go-urn v1.4.0 // indirect
|
||||
golang.org/x/crypto v0.31.0 // indirect
|
||||
golang.org/x/net v0.33.0 // indirect
|
||||
golang.org/x/sys v0.28.0 // indirect
|
||||
golang.org/x/text v0.21.0 // indirect
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1 // indirect
|
||||
gopkg.in/yaml.v3 v3.0.1 // indirect
|
||||
)
|
||||
|
||||
replace git.yuntuo.hk/k2safe/go-omni => ../..
|
||||
38
examples/userService/go.sum
Normal file
38
examples/userService/go.sum
Normal file
@@ -0,0 +1,38 @@
|
||||
filippo.io/edwards25519 v1.1.0 h1:FNf4tywRC1HmFuKW5xopWpigGjJKiJSV0Cqo0cJWDaA=
|
||||
filippo.io/edwards25519 v1.1.0/go.mod h1:BxyFTGdWcka3PhytdK4V28tE5sGfRvvvRV7EaN4VDT4=
|
||||
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
|
||||
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
|
||||
github.com/gabriel-vasile/mimetype v1.4.3 h1:in2uUcidCuFcDKtdcBxlR0rJ1+fsokWf+uqxgUFjbI0=
|
||||
github.com/gabriel-vasile/mimetype v1.4.3/go.mod h1:d8uq/6HKRL6CGdk+aubisF/M5GcPfT7nKyLpA0lbSSk=
|
||||
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
|
||||
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
|
||||
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
|
||||
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
|
||||
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
|
||||
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
|
||||
github.com/go-playground/validator/v10 v10.20.0 h1:K9ISHbSaI0lyB2eWMPJo+kOS/FBExVwjEviJTixqxL8=
|
||||
github.com/go-playground/validator/v10 v10.20.0/go.mod h1:dbuPbCMFw/DrkbEynArYaCwl3amGuJotoKCe95atGMM=
|
||||
github.com/go-sql-driver/mysql v1.9.3 h1:U/N249h2WzJ3Ukj8SowVFjdtZKfu9vlLZxjPXV1aweo=
|
||||
github.com/go-sql-driver/mysql v1.9.3/go.mod h1:qn46aNg1333BRMNU69Lq93t8du/dwxI64Gl8i5p1WMU=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1 h1:kYf81DTWFe7t+1VvL7eS+jKFVWaUnK9cB1qbwn63YCY=
|
||||
github.com/golang-jwt/jwt/v5 v5.3.1/go.mod h1:fxCRLWMO43lRc8nhHWY6LGqRcf+1gQWArsqaEUEa5bE=
|
||||
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
|
||||
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
|
||||
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
|
||||
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
|
||||
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk=
|
||||
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
|
||||
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
|
||||
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
|
||||
golang.org/x/net v0.33.0 h1:74SYHlV8BIgHIFC/LrYkOGIwL19eTYXQ5wc6TBuO36I=
|
||||
golang.org/x/net v0.33.0/go.mod h1:HXLR5J+9DxmrqMwG9qjGCxZ+zKXxBru04zlTvWlWuN4=
|
||||
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
|
||||
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
||||
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
|
||||
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
|
||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1 h1:bBRl1b0OH9s/DuPhuXpNl+VtCaJXFZ5/uEFST95x9zc=
|
||||
gopkg.in/natefinch/lumberjack.v2 v2.2.1/go.mod h1:YD8tP3GAjkrDg1eZH7EGmyESg/lsYskCTPBJVb9jqSc=
|
||||
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
|
||||
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
|
||||
13
examples/userService/internal/handler/openapiGen.go
Normal file
13
examples/userService/internal/handler/openapiGen.go
Normal file
File diff suppressed because one or more lines are too long
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user