migrate

Recipe card from the charly-build plugin (Commands — runtime CLI verbs).

charly migrate — single-command schema migration

charly migrate is one idempotent command that brings any opencharly config — as far back as the supported schema floor — up to the latest schema CalVer, and only to the latest. There are no sub-verbs to choose between: charly migrate always applies every migration-table step newer than the config, up to HEAD.

charly migrate            # migrate every reachable config to the latest schema CalVer
charly migrate --dry-run  # print every change migrate would make; touch nothing

The project directory is the current working directory; use the top-level -C / --dir / CHARLY_PROJECT_DIR global to point elsewhere (main() chdir’s before dispatch).

CalVer schema versioning

The YAML schema version is a CalVer string — version: YYYY.DDD.HHMM, the same fixed-width scheme as image tags (e.g. a HEAD like version: 2026.174.1100). It is CANONICAL fixed-width: a 4-digit year, a 3-digit zero-padded day-of-year, and a 4-digit zero-padded HHMM, so a plain alphanumeric sort of CalVer strings is chronological. ParseCalVer is EXTREMELY STRICT — it accepts ONLY that exact form (no version: 4, no non-padded 2026.45.830); a non-canonical value is “not a CalVer”, which the load gate treats as older-than-HEAD so it flows into charly migrate. charly migrate re-stamps every versioned file it brings forward to the canonical HEAD (universalStamp); a value that predates the supported floor — or is not a CalVer at all — is unmigratable and refused untouched (see “How it runs”). Every versioned file carries the stamp:

every project charly.yml (splitting into per-kind vm.yml / local.yml / … siblings is an optional import: convenience a project MAY use — never the default; see /charly-image:image)
the per-host ~/.config/charly/charly.yml

The parsed ParsedCalVer type, ParseCalVer(string) (ParsedCalVer, bool), ParsedCalVer.Less, and MustCalVer live in spec/spec/calver_parse.go (the parsed type is named ParsedCalVer because spec already binds CalVer = string, the CUE wire scalar); sdk/kit/calver.go re-exports them for kit consumers. CUE owns the HEAD and floor: spec/schema/version.cue defines #SchemaVersion (the HEAD CalVer every file is stamped to) and #SchemaFloor (the oldest version migratable FROM). The spec repo’s task cue:gen emits spec/spec/version_gen.go with the SchemaVersion / SchemaFloor string consts (the superproject task cue:gen chains it); spec.LatestSchemaCalVer() and spec.SchemaFloorCalVer() parse them, re-exported as kit.LatestSchemaVersion() / kit.SchemaFloor() (what candy/plugin-migrate/engine.go calls) and shimmed in core by charly/version.go’s LatestSchemaVersion(). There is NO hand-maintained Go HEAD literal — the load-time gate requires kit.LatestSchemaVersion(), and charly migrate refuses anything below kit.SchemaFloor().

Per-merge release git tags (decoupled from `version:`)

Every landing of ANY repo carries a fresh annotated release git tag v<YYYY.DDD.HHMM> — ONE per merge, so a repo accumulates MULTIPLE v… tags over time. The tag is generated and applied by the fresh pr-validator at MERGE, from the current UTC merge time — never by the author at push time (author-time stamps collide and mis-order across concurrent out-of-sync PRs; see /charly-internals:git-workflow “CalVer”). This is decoupled from the charly.yml version: field: version: is the SCHEMA version (bumped only when a cutover raises #SchemaVersion), whereas the tag marks the merge moment. Tag EVERY landing — including one at an unchanged version: (a content change, submodule extraction, image drop). Tags are immutable: only ever ADD new ones, never move or force-push an existing tag (so the load-time gate never sees a “newer than supported” CalVer from a re-tag). Every component is fixed-width zero-padded — 4-digit year, 3-digit day-of-year, 4-digit HHMM (v2026.064.1937, v2026.142.1640) — so tags sort chronologically under a plain alphanumeric sort; the evaluator computes v$(date -u +%Y.%j.%H%M) at merge (%j and %H%M are already zero-padded). Each repo — the superproject, every box/<distro> submodule, plugins, and docs — is tagged independently on its merged main HEAD (the tag marks the merge, so a repo needs no charly.yml to be tagged); a producer repo’s PR merges + tags BEFORE its consumer’s pointer bump. The SOLE exception is sdk, which tags under its Go-module v0.<YYYYDDD>.<HHMM leading-zeros-stripped> scheme (a semver requirement, not an exemption). See the project rulebook “Post-Execution Policies” (AGENTS.md / CLAUDE.md) and /charly-internals:git-workflow.

The migration table (declarative data)

The migration steps are declarative DATA, not code: an ordered list embedded from candy/plugin-migrate/migrations.cue (the TABLE lives in candy/plugin-migrate), validated at process start against the #Migration schema (candy/plugin-migrate/schema/migration.cue — a plugin-only validation schema that lives in the plugin per the kernel/plugin boundary law, concatenated at compile with the SDK’s version.cue for #CanonCalVer). Each step is stamped with the CalVer of the date it landed and listed chronologically — the order the cutovers were authored in, which is the only correct replay order for an arbitrarily-old config. charly migrate applies every step whose version is newer than the config’s stamp; each step is idempotent, so applying the whole set is safe (an already-current file is a no-op).

A step is a small record:

{version: "YYYY.DDD.HHMM", name: "<slug>", touches_host?: bool, ops?: [...#Op] | apply?: "<hook>"}

touches_host: true flags a step that mutates per-host state (~/.config/charly, quadlets, .secrets) — those steps are skipped by remote-cache auto-migration (see below). A step carries EITHER an ops: list OR a single apply: hook, never both.

The op vocabulary — each op is DATA, zero Go. Four generic key transforms cover the common cutovers, all applied by ONE comment-preserving yaml.v3 interpreter (the op-walker in candy/plugin-migrate/engine.go):

Op	Shape	Effect
`rename_key`	`{from, to, scope, under_kind?}`	rename a mapping key `from` → `to`
`delete_key`	`{key, scope, under_kind?}`	drop a mapping key
`remap_scalar`	`{key, from, to, under_kind?}`	rewrite a scalar VALUE `from` → `to` at `key`
`move_key`	`{key, from_parent, to_parent, under_kind?}`	relocate a key between parent mappings

scope is root (top-level keys only) or any (every depth); under_kind further scopes an op to entities of a named kind (e.g. only vm: nodes). None of the four ops needs a line of new Go — the walker interprets the data.

The apply: hook — the deliberate, visible EXCEPTION. A structural reshape the four ops can’t express (splitting one node into siblings, folding a list into a map) sets apply: "<hook>" naming a Go function registered in the core goHooks map. This is the ONLY path that runs bespoke Go for a migration, and it is intentionally conspicuous: a table entry carrying apply: is the signal that this step needed real code.

CUE has NO transformation capability. CUE is monotone unification — it can validate, default, require, and close, and nothing more; it CANNOT rewrite a config. So CUE owns exactly two things here: the version pins (#SchemaVersion / #SchemaFloor) and the SHAPE of the declarative table (#Migration validates every entry at startup). Every actual transform is Go — the generic op-walker, or an apply: hook. Do not expect CUE to migrate anything.

The current table. The floor sits at the migration-baseline reset version (2026.174.1100), so any config predating it is unmigratable — the accepted clean-slate consequence. The table carries ONE entry, the schema-compaction cutover:

version	name	touches_host	apply
`2026.186.2323`	`compact-node-form`	`true`	`compactNodeForm` (goHook)

The compactNodeForm hook is a structural reshape the four generic ops can’t express: it folds the former named data/step child-node grammar into the compact node form — collections (env/volume/port/service/…) inline in the kind value, steps as the ordered plan: list (a meaningful former step-node name becomes the step’s id:), deploy data (add_candy/install_opts/vm_state/…) inline in the substrate node, and every authored step plugin:+plugin_input: pair rewritten to the <word>: <input> verb sugar. Member children (sub-entities under a deployable kind) stay named children. Remote layer caches auto-migrate on fetch (see “Remote-cache auto-migration”). The same cutover DELETED the Calamares target: / module: / package-group: kinds (and their plugins) — they had zero core readers; they return as their own cutover when the installer feature is real.

How it runs

The migration engine is a COMPILED-IN command plugin (candy/plugin-migrate, command:migrate) — the engine, table, and CLI live in the plugin, out of charly core; core keeps only the below-floor/behind-head load-gate HINTS (the Run: charly migrate messages). charly migrate dispatches to the compiled-in command:migrate provider; the remote-cache auto-migration (refs.go) resolves the same provider and Invokes it (OpRun --project-only) in-proc.

runMigrations (candy/plugin-migrate/engine.go) is floor-gated, comparing the config’s version: stamp against kit.SchemaFloor() and kit.LatestSchemaVersion():

at HEAD → no-op; prints nothing to migrate (already at schema <HEAD>).
below the floor, or not a CalVer at all → unmigratable: refused with an actionable error (predates the supported floor … re-author against the current schema) and NO filesystem change. Configs from before the baseline reset fall here. A stranded per-host config (~/.config/charly/charly.yml) at a below-floor version is doubly stuck: it can neither migrate NOR be WRITTEN — saveDeployState refuses to overwrite a config it cannot load (refusing to overwrite … the existing per-host config fails to load … fix it (or remove it to regenerate) first), so every pod/vm/local deploy — including a charly check run <bed> at its config/deploy-add step — fails until it is reset. Since it holds only regenerable deploy STATE (no authored intent), back it up and reset it to a bare version: <HEAD> stub (or remove it); the state rebuilds on the next deploy.
in [floor, HEAD) → apply every migration-table step newer than the stamp (in order, via the op-walker / apply: hooks), then re-stamp every versioned file to HEAD (universalStamp).

Per-step backups follow the established <file>.bak.<unix-ts> convention.

Remote-cache auto-migration (project-only)

sdk/loaderkit’s refs seams (the former charly/refs.go is DELETED, K-wave 2) auto-run the remote-cache migration (migrateCacheViaPeer, sdk/loaderkit/refs_seams_executor.go) on a freshly-cloned remote-repo cache so external repos pull through at the latest schema. It skips every touches_host step and leaves the host-deploy path empty, so a remote fetch never mutates the user’s per-host state — even the final re-stamp touches only the cache’s project files. A remote whose config predates the floor fails the fetch with the same predates-floor error (an old remote is unmigratable — the accepted clean-slate consequence).

Load-time gate

The load-time gate GateSchemaVersion (sdk/loaderkit/load_unified.go — the former charly/unified.go is DELETED, K-wave 2) is unchanged: LoadUnified parses the merged version: and rejects anything below HEAD (or absent, or non-CalVer):

charly.yml: schema 2026.186.2323 is required (found "4"). Run: charly migrate

A non-CalVer value (a legacy integer, empty, or garbage) parses as “older than every real CalVer”, so a stale config trips the gate with a uniform Run: charly migrate hint — the forward-compat trigger that makes a future migration fire. Whether that migrate then SUCCEEDS depends on the floor: a config in [floor, HEAD) migrates; one below the floor is refused (see “How it runs”). Residual-key checks (e.g. kind: deployment, target: host, secret_backend: kdbx) remain as defense-in-depth, but every remediation hint points uniformly at bare charly migrate.

Adding a future cutover

From here to the end of “Standing rule” is maintenance the charly project performs on its OWN schema, run from checkouts of the spec repo and the superproject. Nothing in these two sections is a step an charly user runs — the user-facing surface of a schema cutover is the single command charly migrate, unchanged. They are recorded here because this skill owns the migration table; the commands are named as this project’s maintenance, not as instructions for the reader.

The payoff of the declarative engine: the common case needs zero new Go — but since the schema lives in the spec module, a schema-version bump is CROSS-REPO: the spec repo lands + tags FIRST, then the superproject consumes the published module at the bumped require version (no submodule, no local checkout).

Bump #SchemaVersion in spec/schema/version.cue to the new CalVer.
Run the spec repo’s task cue:gen to regenerate spec/spec/version_gen.go (the superproject task cue:gen regenerates only the plugin params).
Land + tag the spec repo (its Go-module tag scheme v0.<YYYYDDD>.<HHMM leading-zeros-stripped> — see /charly-internals:git-workflow).
In the superproject: bump the github.com/opencharly/spec require version in charly/go.mod and every lockstep go.mod (the canonical-go.mod gate in tools/gomod-canonical asserts one shared pin; task mods:tidy re-syncs the go.sum files), append the matching entry to candy/plugin-migrate/migrations.cue (the TABLE lives in candy/plugin-migrate) with the same version: — strictly greater than the previous HEAD, expressing the transform as an ops: list (rename_key / delete_key / remap_scalar / move_key) — and run the superproject’s own gates.

That is the whole common case — no bespoke migrator type, no registry function, no per-migrator Go file. Only a structural reshape the four ops can’t express additionally registers ONE goHooks entry (a Go function named by the step’s apply: field). Update the HEAD-CalVer fixtures + the repo’s own versioned YAML in the same change.

The operator command never changes — it stays charly migrate.

The validated schema is CUE-single-source. The @go()-annotated spec/schema/*.cue defs are the sole source for the Go param structs (generated into spec/spec by task cue:gen); a wire-key change is a CUE edit first, then task cue:gen, then — only if it breaks existing on-disk configs — a migration-table entry here (plus the #SchemaVersion bump above). A pure codegen refactor that leaves every authored wire key untouched needs NO migration entry and NO version: bump. See the /charly-internals:go recipe “How to change the charly.yml schema (CUE is the single source of truth)”.

Standing rule: a schema/format change bumps `version:` AND mints a git tag

Any change to the YAML schema or composition format (a key rename, a deleted key, a new key shape) is a hard-cutover that MUST:

Bump the #SchemaVersion CalVer — edit spec/schema/version.cue, run task cue:gen, and append the matching entry to candy/plugin-migrate/migrations.cue (the cross-repo sequence under “Adding a future cutover”), then land the sdk PR (the fresh pr-validator merges + tags it), and in the superproject adopt the tagged sdk release as the new shared pinned require (the task mods:tidy sweep + canonical-go.mod gate keep every module on the one pin). The load-time gate then rejects any not-yet-migrated config with a Run: charly migrate hint, so every reader sees the new format. Raising #SchemaVersion WITHOUT the migration-table entry (or vice-versa) is forbidden — and version: is NEVER set above LatestSchemaVersion() (newer configs hard-fail at load). The author writes a PLACEHOLDER #SchemaVersion / version: / migrations.cue CalVer; the fresh pr-validator FINALIZES it at MERGE — re-stamping it strictly ABOVE the current main HEAD’s schema version (two concurrent schema PRs would otherwise collide or mis-order; see /charly-internals:git-workflow “CalVer”).
The fresh pr-validator mints the per-merge git tag at merge — v<YYYY.DDD.HHMM> from the merge moment (see “Per-merge release git tags” above). The tag and the version: bump are decoupled: the tag marks the merge, version: marks the schema. A schema cutover happens to do BOTH at once, but a content-only landing (no schema change) still mints a tag at an unchanged version:.

Idempotency

Running charly migrate twice is a no-op: after the first run the config is stamped at HEAD, so the second run hits the floor gate’s “at HEAD → nothing to migrate” branch. Every op is itself idempotent (a rename with no matching key does nothing). The migration table’s own invariants — every entry a valid #Migration, versions strictly ascending and unique, HEAD == #SchemaVersion — are validated against CUE at process start.