Follow-up to #23716. Moves ConfigPermission.Info from zod-first (with a preprocess hack) to Effect Schema canonical using Schema.StructWithRest + Schema.decodeTo, and deletes the now-unused ZodPreprocess plumbing. Core change: rule precedence in `Permission.fromConfig` now sorts top-level keys so wildcard permissions (e.g. `*`, `mcp_*`) come before specific ones (e.g. `bash`, `edit`). Combined with `findLast` in evaluate(), this gives the intuitive semantic 'specific tool rules override the `*` fallback' regardless of the user's JSON key order. This silently fixes the previously-broken case `{bash: "allow", "*": "deny"}` (which under the old semantics denied bash because `*` came last). Once rule precedence no longer depends on JSON insertion order, the `__originalKeys` + ZodPreprocess hack can go — StructWithRest's natural canonicalisation is fine because fromConfig sorts anyway. - src/config/permission.ts: rewrite. InputObject is StructWithRest with known permission keys (read/edit/bash/... as Rule, todowrite/webfetch/... as Action-only for type narrowing) + Record rest. Schema.decodeTo normalises the Action shorthand into { "*": action }. .zod is derived — walker already carries the decodeTo transform. - src/config/config.ts, src/config/agent.ts: reference ConfigPermission.Info directly instead of via Schema.Any + ZodOverride. The Effect decoder now applies the permission transform at load time. - src/permission/index.ts: fromConfig sorts wildcards-before-specifics at top level. Sub-pattern order inside a tool key is preserved (documented `*` first, specifics after). - src/util/effect-zod.ts: delete ZodPreprocess symbol, its walkUncached branch, and the TODO comment. Zero remaining consumers. - test/permission/next.test.ts: 6 new tests pinning the new semantics — order-independent precedence, wildcard-as-fallback, sub-pattern order preservation, canonical documented-example regression guard. - test/config/config.test.ts: updated the "preserves key order" test to reflect the new canonical output shape (declaration-order known fields, then input-order rest keys). Behavioural guarantees live in the new permission tests. - test/util/effect-zod.test.ts: delete the ZodPreprocess describe block (~115 lines of tests for the now-removed feature). SDK diff vs dev: - Removed `__originalKeys?: Array<string>` (internal leak). - Catchall cleaned up (no unrelated `Array<string>`). - Known-field types preserved (autocomplete + narrowing). - Only shape change: PermissionConfig union order swap (commutative). Safety audit: no config, test, or doc in the repo (including all 16 translations) exercises the pattern where specifics come before wildcards at the top level. The only configs whose behaviour changes are ones that were silently broken.
312 lines
10 KiB
Markdown
312 lines
10 KiB
Markdown
# Schema migration
|
|
|
|
Practical reference for migrating data types in `packages/opencode` from
|
|
Zod-first definitions to Effect Schema with Zod compatibility shims.
|
|
|
|
## Goal
|
|
|
|
Use Effect Schema as the source of truth for domain models, IDs, inputs,
|
|
outputs, and typed errors. Keep Zod available at existing HTTP, tool, and
|
|
compatibility boundaries by exposing a `.zod` static derived from the Effect
|
|
schema via `@/util/effect-zod`.
|
|
|
|
The long-term driver is `specs/effect/http-api.md` — once the HTTP server
|
|
moves to `@effect/platform`, every Schema-first DTO can flow through
|
|
`HttpApi` / `HttpRouter` without a zod translation layer, and the entire
|
|
`effect-zod` walker plus every `.zod` static can be deleted.
|
|
|
|
## Preferred shapes
|
|
|
|
### Data objects
|
|
|
|
Use `Schema.Class` for structured data.
|
|
|
|
```ts
|
|
export class Info extends Schema.Class<Info>("Foo.Info")({
|
|
id: FooID,
|
|
name: Schema.String,
|
|
enabled: Schema.Boolean,
|
|
}) {
|
|
static readonly zod = zod(Info)
|
|
}
|
|
```
|
|
|
|
If the class cannot reference itself cleanly during initialization, use the
|
|
two-step `withStatics` pattern:
|
|
|
|
```ts
|
|
export const Info = Schema.Struct({
|
|
id: FooID,
|
|
name: Schema.String,
|
|
}).pipe(withStatics((s) => ({ zod: zod(s) })))
|
|
```
|
|
|
|
### Errors
|
|
|
|
Use `Schema.TaggedErrorClass` for domain errors.
|
|
|
|
```ts
|
|
export class NotFoundError extends Schema.TaggedErrorClass<NotFoundError>()("FooNotFoundError", {
|
|
id: FooID,
|
|
}) {}
|
|
```
|
|
|
|
### IDs and branded leaf types
|
|
|
|
Keep branded/schema-backed IDs as Effect schemas and expose
|
|
`static readonly zod` for compatibility when callers still expect Zod.
|
|
|
|
### Refinements
|
|
|
|
Reuse named refinements instead of re-spelling `z.number().int().positive()`
|
|
in every schema. The `effect-zod` walker translates the Effect versions into
|
|
the corresponding zod methods, so JSON Schema output (`type: integer`,
|
|
`exclusiveMinimum`, `pattern`, `format: uuid`, …) is preserved.
|
|
|
|
```ts
|
|
const PositiveInt = Schema.Number.check(Schema.isInt()).check(Schema.isGreaterThan(0))
|
|
const NonNegativeInt = Schema.Number.check(Schema.isInt()).check(Schema.isGreaterThanOrEqualTo(0))
|
|
const HexColor = Schema.String.check(Schema.isPattern(/^#[0-9a-fA-F]{6}$/))
|
|
```
|
|
|
|
See `test/util/effect-zod.test.ts` for the full set of translated checks.
|
|
|
|
## Compatibility rule
|
|
|
|
During migration, route validators, tool parameters, and any existing
|
|
Zod-based boundary should consume the derived `.zod` schema instead of
|
|
maintaining a second hand-written Zod schema.
|
|
|
|
The default should be:
|
|
|
|
- Effect Schema owns the type
|
|
- `.zod` exists only as a compatibility surface
|
|
- new domain models should not start Zod-first unless there is a concrete
|
|
boundary-specific need
|
|
|
|
## When Zod can stay
|
|
|
|
It is fine to keep a Zod-native schema temporarily when:
|
|
|
|
- the type is only used at an HTTP or tool boundary and is not reused elsewhere
|
|
- the validator depends on Zod-only transforms or behavior not yet covered by `zod()`
|
|
- the migration would force unrelated churn across a large call graph
|
|
|
|
When this happens, prefer leaving a short note or TODO rather than silently
|
|
creating a parallel schema source of truth.
|
|
|
|
## Escape hatches
|
|
|
|
The walker in `@/util/effect-zod` exposes two explicit escape hatches for
|
|
cases the pure-Schema path cannot express. Each one stays in the codebase
|
|
only as long as its upstream or local dependency requires it — inline
|
|
comments document when each can be deleted.
|
|
|
|
### `ZodOverride` annotation
|
|
|
|
Replaces the entire derivation with a hand-crafted zod schema. Used when:
|
|
|
|
- the target carries external `$ref` metadata (e.g.
|
|
`config/model-id.ts` points at `https://models.dev/...`)
|
|
- the target is a zod-only schema that cannot yet be expressed as Schema
|
|
(e.g. `ConfigAgent.Info`, `Log.Level`)
|
|
|
|
### Local `DeepMutable<T>` in `config/config.ts`
|
|
|
|
`Schema.Struct` produces `readonly` types. Some consumer code (notably the
|
|
`Config` service) mutates `Info` objects directly, so a readonly-stripping
|
|
utility is needed when casting the derived zod schema's output type.
|
|
|
|
`Types.DeepMutable` from effect-smol would be a drop-in, but it widens
|
|
`unknown` to `{}` in the fallback branch — a bug that affects any schema
|
|
using `Schema.Record(String, Schema.Unknown)`.
|
|
|
|
Tracked upstream as `effect:core/x228my`: "Types.DeepMutable widens unknown
|
|
to `{}`." Once that lands, the local `DeepMutable` copy can be deleted and
|
|
`Types.DeepMutable` used directly.
|
|
|
|
## Ordering
|
|
|
|
Migrate in this order:
|
|
|
|
1. Shared leaf models and `schema.ts` files
|
|
2. Exported `Info`, `Input`, `Output`, and DTO types
|
|
3. Tagged domain errors
|
|
4. Service-local internal models
|
|
5. Route and tool boundary validators that can switch to `.zod`
|
|
|
|
This keeps shared types canonical first and makes boundary updates mostly
|
|
mechanical.
|
|
|
|
## Progress tracker
|
|
|
|
### `src/config/` ✅ complete
|
|
|
|
All of `packages/opencode/src/config/` has been migrated. Files that still
|
|
import `z` do so only for local `ZodOverride` bridges or for `z.ZodType`
|
|
type annotations — the `export const <Info|Spec>` values are all Effect
|
|
Schema at source.
|
|
|
|
- [x] skills, formatter, console-state, mcp, lsp, permission (leaves), model-id, command, plugin, provider
|
|
- [x] server, layout
|
|
- [x] keybinds
|
|
- [x] permission#Info
|
|
- [x] agent
|
|
- [x] config.ts root
|
|
|
|
### `src/*/schema.ts` leaf modules
|
|
|
|
These are the highest-priority next targets. Each is a small, self-contained
|
|
schema module with a clear domain.
|
|
|
|
- [ ] `src/control-plane/schema.ts`
|
|
- [ ] `src/permission/schema.ts`
|
|
- [ ] `src/project/schema.ts`
|
|
- [ ] `src/provider/schema.ts`
|
|
- [ ] `src/pty/schema.ts`
|
|
- [ ] `src/question/schema.ts`
|
|
- [ ] `src/session/schema.ts`
|
|
- [ ] `src/sync/schema.ts`
|
|
- [ ] `src/tool/schema.ts`
|
|
|
|
### Session domain
|
|
|
|
Major cluster. Message + event types flow through the SSE API and every SDK
|
|
output, so byte-identical SDK surface is critical.
|
|
|
|
- [ ] `src/session/compaction.ts`
|
|
- [ ] `src/session/message-v2.ts`
|
|
- [ ] `src/session/message.ts`
|
|
- [ ] `src/session/prompt.ts`
|
|
- [ ] `src/session/revert.ts`
|
|
- [ ] `src/session/session.ts`
|
|
- [ ] `src/session/status.ts`
|
|
- [ ] `src/session/summary.ts`
|
|
- [ ] `src/session/todo.ts`
|
|
|
|
### Provider domain
|
|
|
|
- [ ] `src/provider/auth.ts`
|
|
- [ ] `src/provider/models.ts`
|
|
- [ ] `src/provider/provider.ts`
|
|
|
|
### Tool schemas
|
|
|
|
Each tool declares its parameters via a zod schema. Tools are consumed by
|
|
both the in-process runtime and the AI SDK's tool-calling layer, so the
|
|
emitted JSON Schema must stay byte-identical.
|
|
|
|
- [ ] `src/tool/apply_patch.ts`
|
|
- [ ] `src/tool/bash.ts`
|
|
- [ ] `src/tool/codesearch.ts`
|
|
- [ ] `src/tool/edit.ts`
|
|
- [ ] `src/tool/glob.ts`
|
|
- [ ] `src/tool/grep.ts`
|
|
- [ ] `src/tool/invalid.ts`
|
|
- [ ] `src/tool/lsp.ts`
|
|
- [ ] `src/tool/plan.ts`
|
|
- [ ] `src/tool/question.ts`
|
|
- [ ] `src/tool/read.ts`
|
|
- [ ] `src/tool/registry.ts`
|
|
- [ ] `src/tool/skill.ts`
|
|
- [ ] `src/tool/task.ts`
|
|
- [ ] `src/tool/todo.ts`
|
|
- [ ] `src/tool/tool.ts`
|
|
- [ ] `src/tool/webfetch.ts`
|
|
- [ ] `src/tool/websearch.ts`
|
|
- [ ] `src/tool/write.ts`
|
|
|
|
### HTTP route boundaries
|
|
|
|
Every file in `src/server/routes/` uses hono-openapi with zod validators for
|
|
route inputs/outputs. Migrating these individually is the last step; most
|
|
will switch to `.zod` derived from the Schema-migrated domain types above,
|
|
which means touching them is largely mechanical once the domain side is
|
|
done.
|
|
|
|
- [ ] `src/server/error.ts`
|
|
- [ ] `src/server/event.ts`
|
|
- [ ] `src/server/projectors.ts`
|
|
- [ ] `src/server/routes/control/index.ts`
|
|
- [ ] `src/server/routes/control/workspace.ts`
|
|
- [ ] `src/server/routes/global.ts`
|
|
- [ ] `src/server/routes/instance/index.ts`
|
|
- [ ] `src/server/routes/instance/config.ts`
|
|
- [ ] `src/server/routes/instance/event.ts`
|
|
- [ ] `src/server/routes/instance/experimental.ts`
|
|
- [ ] `src/server/routes/instance/file.ts`
|
|
- [ ] `src/server/routes/instance/mcp.ts`
|
|
- [ ] `src/server/routes/instance/permission.ts`
|
|
- [ ] `src/server/routes/instance/project.ts`
|
|
- [ ] `src/server/routes/instance/provider.ts`
|
|
- [ ] `src/server/routes/instance/pty.ts`
|
|
- [ ] `src/server/routes/instance/question.ts`
|
|
- [ ] `src/server/routes/instance/session.ts`
|
|
- [ ] `src/server/routes/instance/sync.ts`
|
|
- [ ] `src/server/routes/instance/tui.ts`
|
|
|
|
The bigger prize for this group is the `@effect/platform` HTTP migration
|
|
described in `specs/effect/http-api.md`. Once that lands, every one of
|
|
these files changes shape entirely (`HttpApi.endpoint(...)` and friends),
|
|
so the Schema-first domain types become a prerequisite rather than a
|
|
sibling task.
|
|
|
|
### Everything else
|
|
|
|
Small / shared / control-plane / CLI. Mostly independent; can be done
|
|
piecewise.
|
|
|
|
- [ ] `src/acp/agent.ts`
|
|
- [ ] `src/agent/agent.ts`
|
|
- [ ] `src/bus/bus-event.ts`
|
|
- [ ] `src/bus/index.ts`
|
|
- [ ] `src/cli/cmd/tui/config/tui-migrate.ts`
|
|
- [ ] `src/cli/cmd/tui/config/tui-schema.ts`
|
|
- [ ] `src/cli/cmd/tui/config/tui.ts`
|
|
- [ ] `src/cli/cmd/tui/event.ts`
|
|
- [ ] `src/cli/ui.ts`
|
|
- [ ] `src/command/index.ts`
|
|
- [ ] `src/control-plane/adaptors/worktree.ts`
|
|
- [ ] `src/control-plane/types.ts`
|
|
- [ ] `src/control-plane/workspace.ts`
|
|
- [ ] `src/file/index.ts`
|
|
- [ ] `src/file/ripgrep.ts`
|
|
- [ ] `src/file/watcher.ts`
|
|
- [ ] `src/format/index.ts`
|
|
- [ ] `src/id/id.ts`
|
|
- [ ] `src/ide/index.ts`
|
|
- [ ] `src/installation/index.ts`
|
|
- [ ] `src/lsp/client.ts`
|
|
- [ ] `src/lsp/lsp.ts`
|
|
- [ ] `src/mcp/auth.ts`
|
|
- [ ] `src/patch/index.ts`
|
|
- [ ] `src/plugin/github-copilot/models.ts`
|
|
- [ ] `src/project/project.ts`
|
|
- [ ] `src/project/vcs.ts`
|
|
- [ ] `src/pty/index.ts`
|
|
- [ ] `src/skill/index.ts`
|
|
- [ ] `src/snapshot/index.ts`
|
|
- [ ] `src/storage/db.ts`
|
|
- [ ] `src/storage/storage.ts`
|
|
- [ ] `src/sync/index.ts`
|
|
- [ ] `src/util/fn.ts`
|
|
- [ ] `src/util/log.ts`
|
|
- [ ] `src/util/update-schema.ts`
|
|
- [ ] `src/worktree/index.ts`
|
|
|
|
### Do-not-migrate
|
|
|
|
- `src/util/effect-zod.ts` — the walker itself. Stays zod-importing forever
|
|
(it's what emits zod from Schema). Goes away only when the `.zod`
|
|
compatibility layer is no longer needed anywhere.
|
|
|
|
## Notes
|
|
|
|
- Use `@/util/effect-zod` for all Schema → Zod conversion.
|
|
- Prefer one canonical schema definition. Avoid maintaining parallel Zod and
|
|
Effect definitions for the same domain type.
|
|
- Keep the migration incremental. Converting the domain model first is more
|
|
valuable than converting every boundary in the same change.
|
|
- Every migrated file should leave the generated SDK output (`packages/sdk/
|
|
openapi.json` and `packages/sdk/js/src/v2/gen/types.gen.ts`) byte-identical
|
|
unless the change is deliberately user-visible.
|