go — source-map
Detail page of the go recipe card.
Unified YAML loader (LoadUnified)
Section titled “Unified YAML loader (LoadUnified)”The unified format’s entry point is LoadUnified(dir) (sdk/loaderkit/load_unified.go, reached through the ProjectLoader seam from charly/loader_threaded.go — the former charly/unified.go is DELETED, K-wave 2). It reads <dir>/charly.yml, recursively resolves the import: statement (max depth 8, cycle-safe via visited set), and parses every file as a YAML multi-document stream (so fleet files with --- separators work). Every document is unified node-form (name-first): the loader’s per-document merge (sdk/loaderkit/walk.go’s (*walker).walkFile, the port of the former mergeUnifiedDocs) runs each document through the shared routing core — spec.ClassifyDoc (top-level-key inspection; kit.ClassifyDoc re-exports it) → the closed #NodeDoc CUE gate → normalizeNodeInto (the reserved-word-driven node decomposer, charly/node_parsed.go). #NodeDoc (schema/node.cue) is the SOLE load-time gate for every loaded document, the root charly.yml and discovered manifests alike. spec.ClassifyDoc does NOT route a legacy kind-keyed / root-shape document — it HARD-REJECTS it with a charly migrate hint (the legacy mergeKindDoc / firstKindKey / kindKeyedDoc / VmDoc routing was deleted in the #NodeDoc-sole-gate cutover; the legacy-shape detector rootShapeKeySet is gone too — ClassifyDoc now classifies every non-empty top-level mapping as node-form). F9 BOOTSTRAP PHASE: before the early GateSchemaVersion schema gate, LoadUnified runs runBootstrapPhase(rootData) (charly/bootstrap_phase.go) — it enumerates providerRegistry.providersInPhase(sdk.PhaseBootstrap) and invokes each one’s Invoke(OpBootstrap, {config}), threading the returned (possibly transformed) bytes, so a bootstrap-phase plugin can rewrite the raw root bytes BEFORE validation rejects them. LoadUnified seeds the transformed root into the loader walk (sdk/loaderkit/walk.go’s (*walker).walkFile) via the fileOverrides map (keyed on the root’s abs path), so the rewrite reaches the actual PARSE + the post-merge gate — not just the early version gate. Bootstrap plugins are compiled-in (in-proc), so this never re-enters the validated-config load. Today only the no-op candy/plugin-example-bootstrap registers in this phase — migrate is NOT a bootstrap-phase transform (it is the compiled-in candy/plugin-migrate command:migrate plugin, invoked explicitly by charly migrate and by the refs-seams remote-cache auto-migration Invoke — sdk/loaderkit/refs_seams_executor.go’s migrateCacheViaPeer, the former charly/refs.go is DELETED — never as an in-loader byte-rewrite; the load gate keeps the Run: charly migrate reject for a stale config, because migration is whole-project file-based + host-coupled and cannot run on root bytes inside LoadUnified). Phases are an sdk.Phase* set declared per-capability via ProvidedCapability.phase (proto field 9, lifted in buildUnit/buildUnitInProc onto the phaseCarrier; phaseOfProvider defaults to runtime). A no-op bootstrap plugin (candy/plugin-example-bootstrap) returns the bytes unchanged.
UnifiedFile.ApplyDiscover(rootDir) walks the flat generic discover: list after initial merge. discover: is DiscoverConfig = []ScanSpec ({path, recursive, manifest}) — no kind dimension. For each spec, sdk/loaderkit/discover.go’s RunDiscover (the K1 port of the former charly/unified.go discover walk) finds directories containing the spec’s manifest (default UnifiedFileName — charly.yml, the ONE filename the code knows; a missing discover path is a no-op, not an error), and validates each discovered document through the SAME spec.ClassifyDoc → #NodeDoc gate: a candy: node registers a lazy From: directory reference (charly.ScanCandy parses + validates it later), every other node decomposes + merges via normalizeNodeInto. Explicit map entries always win over discovered entries. ApplyDiscover runs in the loader’s main path (sdk/loaderkit/walk.go’s walkFile depth-0 boundary, for the root AND every namespace), so discovered image nodes (candy: nodes carrying base:/from:, the former box:) reach ProjectConfig — not just the layer-loading path. The authoring kind vocabulary is CUE-derived and registry-driven — spec.KindWords (empty today; every kind is plugin-served) feeds the loader’s Threaded snapshot (spec.Threaded.Kinds) via the provider registry, and the former in-core kindWordSet/stepKeywordSet membership sets were removed with the kind-externalization; the former hand kindKeys/kindKeysSet/entityKind lists were deleted.
Projections to today’s concrete types: ProjectConfig() → *Config, ProjectDistroConfig() → *DistroConfig, etc. Existing LoadConfig / LoadBuildConfigForBox / LoadFleetConfig continue to work unchanged — migration to the unified entry point is incremental.
Binary-embedded default config (charly/embed_defaults.go). The loader has ONE document-interpretation path (sdk/loaderkit/walk.go’s walkFile, the port of the former mergeUnifiedDocs). The binary-embedded default config is plain node-form YAML at charly/charly.yml (//go:embed charly.yml, embed_defaults.go), parsed by the SAME unified loader as any project charly.yml — there is no CUE-source front-end and no compile step: embeddedDefaults feeds the embedded bytes straight through the UNCHANGED loader walk, then applyEmbeddedDefaults merges the vocabulary in as the lowest-priority base (project-wins). The embedded vocabulary is schema-validated against the sdk schema (spec/schema — #Distro/#Builder/#Init/#Resource/#Sidecar) at LOAD time via the #NodeDoc gate (ValidateNodeDocCUE, sdk/loaderkit/cue_validate.go) — guarded by TestEmbeddedDefaults_SchemaConformance, with TestEmbeddedDefaults_SameLoaderPath proving the embed flows through the identical loader core. (Corrected: the former claim that this ran through a shared validateVocabularyCollections helper “also used by charly box validate for project files” was stale — that call site was already cut in c9befd83 when the legacy root-shape collection format it validated became unreachable [HARD-REJECTED at load before validation], and validateVocabularyCollections/its sibling validateEntityCUE were themselves deleted as dead code in the dead-code-radical-removal batch. RDD-verified live: a project’s own vocabulary override — e.g. an unknown key in a builder: entity — is rejected at LOAD time via the per-kind plugin gate, plugin kind:builder: plugin_input fails #BuilderInput, not a separate charly box validate-time pass.)
Schema Driven Design (SDD)
Section titled “Schema Driven Design (SDD)”The operationalization of the project rulebook “Schema Driven Design (SDD)” pillar (AGENTS.md / CLAUDE.md) — the mandate lives there, the how lives here: the configuration schema comes BEFORE the code, and as much code as possible is GENERATED from the schema. The full pipeline map, source → generator → artifact:
| Source (authored) | Generator | Generated artifact |
|---|---|---|
spec/schema/*.cue (the base ingress schema) |
task cue:gen — cue exp gengotypes + spec/internal/schemagen (concat/retag/vocab/version), both over the shared spec/schemaconcat |
spec/spec/cue_types_gen.go, spec/spec/vocab_gen.go, spec/spec/version_gen.go |
each plugin’s own schema/*.cue |
the same pipeline (the superproject task cue:gen per-plugin params loop, -pkg=params) |
candy/plugin-*/params/cue_types_gen.go |
charly/charly.yml compiled_plugins: |
pluginsgen (charly/internal/pluginsgen, run by task build:binary) |
charly/plugins_generated.go + the repo-root go.work |
spec/protocol/schema/*.cue |
task wire:gen (wiregen + pinned protoc plugins) |
spec/proto/plugin.proto, plugin.pb.go, plugin_grpc.pb.go |
Validation at every boundary derives from the SAME schema: ingress — sharedCueSchema (charly/cue_schema.go, the #NodeDoc load gate) + validateKindValueCUE; plugin inputs — registerPluginUnitSchema + validateAuthoredPluginInput (/charly-internals:plugin); migrations — the declarative table candy/plugin-migrate/migrations.cue (/charly-build:migrate); egress — the files charly WRITES (/charly-internals:egress).
Reproducibility gates — regeneration on a clean tree is a NO-OP; drift is an R1 incident: TestGenReproducible (spec/spec/gen_repro_test.go), TestPluginsGenReproducible (charly/internal/pluginsgen/main_test.go), and TestGeneratedProtocolReproducible (spec/internal/wiregen).
A high-risk schema shape is spiked first (RDD — /charly-internals:strict-policy “The spike”): prove the def compiles, the generated type round-trips, and the gate accepts/rejects as intended on a throwaway run BEFORE coding against it. The def-level @go(CharlyName) breakage documented in the next section was caught exactly that way.
Generation coverage — the wire-type mandate + the spike-verified exceptions.
WIRE TYPES ARE CUE-SOURCED WITHOUT EXCEPTION (the project rulebook SDD): every
spec/spec/*_wire.go host↔plugin / render-context data-carrier struct AND every
plugin’s params is a CUE def in spec/schema/*.cue generated by task cue:gen —
hand-writing a wire struct is FORBIDDEN. A wire type is a plain or DISCRIMINATED
struct, which cue exp gengotypes generates faithfully — RDD-proven live (a
{kind!: "a"|"b", a?: …, b?: …} def generates a real Go struct with the
discriminator + per-variant optional fields), so a wire type NEVER needs a
disjunction. No @go(-) / hand-written type is EVER added without a full RCA +
a live cue exp gengotypes spike proving CUE genuinely cannot express it — an
unverified @go(-) is a mandate violation, not an “exception”. The ONLY
spike-proven cases are four NON-wire categories (each @go(-)’d / documented +
kept in lockstep with its def):
spec/spec/union_types.go— faithful union/shorthand types for AUTHORED-CONFIG CUE disjunctions (the user’s either/or authoring surface, e.g.VmSourcecloud_image⊻bootc). RDD spike:#X: {cloud_image!: string} | {bootc!: string}→type X map[string]any—gengotypesgenuinely degrades a disjunction (Go has no sum type). The matching CUE def is@go(-)’d. A wire type that would carry a union is a DISCRIMINATED struct instead (aKind/discriminator field + per-variant optional fields —deploy.cue’s#ReverseOptagged-union pattern, RDD-proven to generate) and REFERENCES a disjunction type by name; it never redefines one.spec/spec/hand_state_types.go— open-tailed struct+map authoring/state shapes (PortSpec, the open-tailedVmDeployState). RDD spike:#X: {known?: string, {[string]: _}}→type X map[string]any— the known fields collapse. Mirrored against@go(-)’d defs. A new wire type uses an EXPLICIT map field instead of an open tail so it generates.spec/spec/charly_names.go— charly-name Go type aliases (def-level@go(CharlyName)is broken in cue v0.16.1; see the next section). Not a wire struct.- The host↔plugin & render-context wire structs are fully CUE-SOURCED — the
spec/spec/*_wire.gohand-written-file class is EXTINCT (the last 13 — deploy/init/gpu/doctor/enc/feature/k8sgen/resource/settings/substrate_template/agent/distro/vm — converted in the SDD-conversion batch;arbiter_wire.golanded earlier, FLOOR-SLIM Unit-8B).buildctx.cue(InstallContext/BuildStageContext),clean.cue(RetentionRequest/RetentionReply),arbiter.cue(+ the handspec/spec/arbiter_consts.gostring-enum residue), and nowagent.cue/distro.cue/init.cue/vm.cue/resource.cue/sidecar.cue(appended) plussettings.cue/feature.cue/doctor.cue/gpu.cue/enc.cue/k8sgen.cue/substrate_template.cue/buildwire.cue(new) anddeploy.cue/seam.cue(appended, the deploy-IR/build-time/lifecycle wire) are ALL LANDED — no hand-written wire-struct file remains anywhere inspec/spec. Any NEW wire type is authored the same way: a CUE def inspec/schema/*.cue, generated bytask cue:gen(a plain/discriminated struct always generates, referencing a disjunction/state type by name where one is genuinely needed) — hand-writing one is a mandate violation from the first line, never a “conversion-in-progress” grace period. The handful of NAMED ENUM TYPES with Go behavior gengotypes cannot generate (Scope+ itsString()/ScopeFromName, mirroringStatusinstatus_result.go) and the per-domain STRING-CONSTANT groups (ReverseOpKind,EncMethod*,GpuMode*/HostDriver*,ArbiterAction*) stay hand-written in smallspec/spec/*_consts.go/spec/spec/*_methods.gofiles beside their CUE-generated struct siblings — never a type CUE could express, always a documented, spike-justified exception per the taxonomy above. spec/protocol/schema/*.cue— the gRPC transport model, including services and streaming flags. The project-ownedspec/internal/wiregengenerator closes the former CUE→proto tooling gap;spec/proto/plugin.protoand all language stubs are generated and must never be edited directly.json:"-"fields (keep-in-Go, drop-from-wire) —gengotypeshas NO construct for a field that exists in memory but is excluded from marshaling (it emits a realjson:"<name>,omitempty"tag instead). RDD spike (cue v0.16.x, the P12 check seam):kit.CheckResult.DeadlineExceeded bool json:"-"(an engine-internal retry signal that must never cross the wire) is genuinely inexpressible — but the spike ALSO confirmed the REST ofCheckResult(Op/Verb/Status/Message/Elapsed/Attempts/TotalElapsed/CapturedValue) generates faithfully. FLOOR-SLIM Unit 4 acted on that:spec/schema/checkresult.cue→spec.CheckResultis CUE-sourced (the base), andkit.CheckResultis nowstruct { spec.CheckResult; DeadlineExceeded bool json:"-" }— an EMBEDDING WRAPPER adding back ONLY the one field, not a hand-duplicated whole type. The legal exception this bullet documents is narrower than it once was: a hand-written field on an otherwise CUE-sourced type is the LEGAL documented shape, not a hand-written type in its entirety — apply the same embed-wrapper pattern to any other single-fieldjson:"-"exception before assuming the whole containing type must stay hand-written.Op.Kind()(spec/spec/charly_methods.go) — the exactly-one-discriminator cross-field rule, kept in Go because CUE cannot express it as a generable annotation. A method, not a type.
Spike-proven CAN/CANNOT quick reference (reuse these results — do NOT re-spike settled shapes; DO spike any new shape class):
| Shape | gengotypes result |
|---|---|
Plain / DISCRIMINATED struct (kind!: + per-variant optionals) |
✅ faithful struct — the wire-type workhorse |
Reference to another generated def (Op?: #Op → *Op) |
✅ typed pointer — a def-having embed crosses the wire TYPED |
time.Duration / custom scalar via @go(,type=…) override |
✅ (T-P14a SubstrateKind, T-P12 Elapsed spikes) |
| Untagged-PascalCase-no-omitempty JSON | ✅ via required (!) fields + PascalCase CUE names |
Self-recursive struct (nested?: [...#Self]) |
✅ pointer slice []*Self — JSON byte-identical to a value slice |
[string]: T map |
✅ map[string]T |
Disjunction {a!:…} | {b!:…} |
❌ map[string]any → hand-written union (authored-config only; a wire type uses a discriminated struct) |
Open tail {known?: string, {[string]: _}} |
❌ known fields collapse → hand_state_types |
Int-keyed map [int]: string |
❌ degrades to an empty struct → re-shape to map[string]string or spike-justified @go(-) |
json:"-" keep-in-Go field |
❌ no construct → documented hand-written exception |
The member-by-member detail of the spec/spec package is the next section; the step-by-step recipe is “How to change the charly.yml schema (CUE is the single source of truth)” under Common Workflows — neither is restated here.
CUE is the single source of truth — the spec/spec package
Section titled “CUE is the single source of truth — the spec/spec package”The charly.yml ingress schema has ONE author-of-record: the CUE definitions in spec/schema/*.cue — in the spec contract module (github.com/opencharly/spec, resolved from the module proxy at the pinned require version in every go.mod — there is NO submodule and NO local checkout). The Go param structs, the reserved-word vocabulary, and the kind/verb wiring are GENERATED or DERIVED from that source — there are no hand-maintained parallel copies. charly/go.mod requires github.com/opencharly/spec at the pinned version (no replace — the require version IS the resolution), and the repo-root go.work carries NO entry for either contract module (generated by pluginsgen; both the sdk and spec contract modules resolve via the module proxy at the pinned require versions). The pieces:
spec/spec— the generated param structs (import pathgithub.com/opencharly/spec/spec; charly core reaches these types viaspec.Xdirectly — the ZERO-ALIASES v2 target is REACHED: nocharly/*_aliases.gore-export file exists, held that way byTestZeroAliases_NoAliasFilesInCharlyCoreincharly/import_purity_test.go; see “The alias surface” below).task cue:gen(the spec repo’s task; the superprojecttask cue:genregenerates only the per-plugin params loop) regenerates this package fromspec/schema/*.cue. Its members:spec/spec/cue_types_gen.go— generated bycue exp gengotypes(then yaml-tag retagged). Carries theCode generated … DO NOT EDITbanner; NEVER hand-edit. Every authored param type lives here (Box,Candy,Vm,Op,Deploy, …).spec/spec/vocab_gen.go— generated by the companionspec/internal/schemagen(-mode=vocab). The CUE-derived reserved-word slices:KindWords,ResourceKinds,DocDirectives,StepKeywords,ContextWords,OpFields,OpVerbs,AuthoringVerbs.spec/spec/union_types.go— hand-written faithful union / shorthand types.cue exp gengotypesdegrades every CUE disjunction toany/map[string]any/an empty struct, so the matching CUE def is annotated@go(-)(suppressing the lossy generated type) and the precise Go type is hand-written here in the SAME package, referenced by the generated structs by name.spec/spec/charly_names.go— hand-written charly-name aliases (type BoxConfig = Box,type VmSpec = Vm, …). Def-level@go(CharlyName)is BROKEN in cue v0.16.1 (it dangles the fields that reference the renamed def, producing uncompilable Go — RDD-verified on a live spike), so the charly NAME is exposed as a Go type alias here instead of via a def-level attribute. The per-FIELD@go(GoName,…)attributes (which DO work) carry the name/pointer/type overrides inspec/schema/*.cue.spec/spec/gen_repro_test.go—TestGenReproducible, the reproducibility gate: re-runs the sametask cue:gentools into a temp dir and diffs against the committedcue_types_gen.go+vocab_gen.go, failing on any drift (skips gracefully when the pinnedcueCLI is absent).spec/spec/scalar_aliases.go,spec/spec/hand_state_types.go,spec/spec/charly_methods.go— hand-written supporting scalar aliases, runtime state types, and the pure methods (Op.Kind(), …) that moved into packagespecalongside the types they operate on.
spec/internal/schemagen(main.go) — the companion generator. Four modes:-mode=concat(concatenateschema/*.cueinto oneschema_spec.cuecompilation unit headedpackage spec),-mode=vocab(compile that and emitspec/spec/vocab_gen.go),-mode=version(emit theSchemaVersion/SchemaFloorconsts →spec/spec/version_gen.go),-mode=retag(the principled Go yaml-tag transform on thegengotypesoutput →spec/spec/cue_types_gen.go). Compiled and documented — neversedon generated Go.spec/schemaconcat(schemaconcat.go) — the shared PUBLIC concat contract (R3,github.com/opencharly/spec/schemaconcat). The ONEConcatSchemaboth the RUNTIME (charly/cue_schema.go’ssharedCueSchema, over thespec/schemaembed FS) and the dev-time generator call to fold every package-lessschema/*.cuefile into one compilation unit, so the schema the runtime validates against and the Go typesgengotypesproduces can never drift. A leaf package depending only on the stdlibio/fsabstraction (runtime passes thespec/schema//go:embedFS, the generator passesos.DirFS).- The alias surface — ZERO in core (the v2 ZERO-ALIASES target, REACHED; see the project rulebook “Core is a PLUGIN HOST”). Authored param types are referenced FULLY-QUALIFIED as
spec.Xthroughoutcharly/*.go(spec.BoxConfig,spec.Op,spec.CandyYAML,spec.ServiceEntry, …) — package main defines NO re-export alias for them, and the hand struct DEFINITIONS are gone. Nocharly/*_aliases.gofile exists at all:TestZeroAliases_NoAliasFilesInCharlyCore(charly/import_purity_test.go) globs*_aliases.go/*_alias.goincharly/and FAILS on any reappearance, and its sibling alias-FORM assertion in the same file policies inlineX = spec.Yre-exports. An alias means the call site is MISLOCATED — the fix is moving that call site into its owning plugin, never re-exporting the type. The alias files this skill once inventoried live in their owning plugin / kit packages now (candy/plugin-vm/vmshared_aliases.go,sdk/deploykit/kit_aliases.go, …); their membership still changes, so grep the actual file before citing which one binds a given symbol — never cite this skill for it. Because every core reference is a directspec.Xcompiling against the real spec types, the Go compiler IS the field-parity check: a renamed or removed spec field (or wire-key/type change) fails the build at this surface. The former package-main name-collision carve-outs are gone with the types themselves — core holds no concreteCandystruct (charly/layers.go), noCandyRef, and no in-coreCalVer(charly/version.gorecords its W0 deletion). The authored-config type isspec.CandyYAML, itself an intra-spec-package alias toCandyatspec/spec/charly_names.go— a STABLE spec-side fact, not a charly-core alias;CandyRefissdk/deploykit’s own alias tospec.CandyRefEntry. charly/reserved_registry.go(package main) — the CUE-derived reserved-word membership sets + the startup VERB bijection gate. Hosts the remaining CUE-derived membership sets:resourceKindSet+authoredOpFieldSet(each a set view of aspec.*slice regenerated bytask cue:gen; no hand-maintained parallel list). The formerkindWordSet/stepKeywordSetwere removed with the kind-externalization (classifyKind reads the registry’s Threaded snapshot), anddocDirectiveSetmoved to loaderkit.VerbCatalogdispatches the generic install verbs;normalizeNodeInto(the reserved-word-driven node decomposer) also lives here. Theinit()runs the VERB bijection check fail-fast at process start (mirrored asTestReservedWordRegistry_*):checkVerbBijection(VerbCatalog, spec.OpVerbs, spec.AuthoringVerbs)— a verb can never be added to the schema without a handler, nor a handler kept after its word is dropped. The KIND bijection ischeckKindProviderBijection(spec.KindWords)incharly/provider_kind.go(panics fromregistry_bootstrap.go): it gates ClassKind PROVIDERS againstspec.KindWords, which is now EMPTY — every authoring kind is plugin-served, so there is no in-corereservedKindHandlersmap (a kind is decoded by its provider viarunPluginKind, not a core handler).charly/uniform_api_gate_test.go(package main) — the F11 uniform-API gate (TestNoSinglePluginAPISurface), the externalization capstone. Asserts the “generic over ad-hoc” invariant STRUCTURALLY: no provider WORD appears in the plugin↔kernel API SURFACE — not as ansdk.Op*selector value, asdk.ProvidedCapability/StepContractfield name, a reverse-channel RPC method name (ExecutorService/CheckContextService/Provider/PluginMeta), or ahostBuilderskey. The forbidden-word universe is the union of the CUE-derived generic config vocabulary (spec.OpVerbs ∪ spec.KindWords ∪ spec.OpFields ∪ spec.AuthoringVerbs) plus every compiled-in non-command provider’sReserved()word — which is where the externalized check verbs cdp/vnc/mcp/kube/libvirt/spice/adb/appium enter, since they are plugin-served and NOT#Op/spec.OpFieldsfields — minusgenericConceptCollisions(venue— the genericExecutorService.VenueRPC coincides with the#Opvenue field). The invariant is structural, NOT a user-count (each capability flag has exactly ONEcandy/plugin-example-*user — “≥1 by construction”). Per-plugin data rides the opaqueSubstrate json.RawMessage; a host MAY call the genericconnectPluginByWord(class, word)with a specific word ARGUMENT (the word is data, never API shape — those call-sites are not scanned). Sibling of the startup bijection gates; has a teeth arm (a re-introduced provider word must trip it) + fixed RPC-method allowlists (a new reverse RPC is a conscious, reviewed addition).
import-namespace loader (UnifiedFile.Import + Namespaces)
Section titled “import-namespace loader (UnifiedFile.Import + Namespaces)”UnifiedFile.Import (type ImportList, YAML tag import) is the single composition statement. Its custom UnmarshalYAML accepts a mixed-shape sequence: a scalar item → ImportEntry{Ref: …} (flat, Namespace == ""); a single-key mapping item → ImportEntry{Namespace: alias, Ref: …} (namespaced). A matching MarshalYAML round-trips both shapes (migrators rely on it). validateNamespaceAlias enforces a bare lowercase-hyphenated alias (no dots).
loadUnifiedInto processes the queue:
- Flat entries are loaded and root-merged into the importing
UnifiedFile(same root-wins merge that drives same-repo file splits + an imported build-vocabulary override). - Namespaced entries call
loadNamespaceCached(ref, base, nsCache, loadingRepos), which loads the target as a fully-resolved, isolatedUnifiedFile(its own flat imports + its own namespaced imports, with a FRESHvisitedset for its file-cycle detection) and mounts it undermerged.Namespaces[alias]. These entries are NOT flat-merged into the root maps — they are referenced qualified.
UnifiedFile.Namespaces (map[string]*UnifiedFile, YAML tag -, never authored directly) holds the mounted children; projectConfigCached projects it to Config.Namespaces (map[string]*Config, pointer-keyed cache → self-references project safely).
Cycle-break by REPO IDENTITY (ns_identity.go), not pinned version. Two maps cooperate in loadNamespaceCached: nsCache is the version-keyed (canonicalRef: repo@version/subpath) diamond memo — it dedups identical refs across a load; loadingRepos is the ancestor/cycle set, keyed by REPO IDENTITY (nsRepoIdentity: a remote ref’s RepoPath, or a local path’s git remote origin). BEFORE any fetch, if the ref’s repo identity is already in loadingRepos (an ancestor still on the load stack), the loader returns that in-progress node — so an import cycle between two projects that import each other (or a transitive back-import of an ancestor still being loaded) terminates even when the loop’s pins diverge: a back-reference to a DIFFERENT pinned version of an in-progress repo resolves to the in-progress node instead of fetching (and recursing into) a divergent — possibly stale-schema — snapshot. LoadUnified seeds loadingRepos[rootIdentity] = merged (the root’s identity comes from its optional repo: field, else git remote origin), so any transitive import of the root’s OWN repo resolves to the local working tree — the importing project’s namespace pins win. loadingRepos entries are pushed before recursing and popped after (stack-scoped — two SIBLING imports of the same repo at different versions still each load); the root seed is never popped. A whole-repo ref with an empty sub-path resolves to that repo’s charly.yml. Covered by TestImportNamespace_DivergentVersionMutualCycle + TestNsRepoIdentity in ns_identity_test.go.
Namespace resolver (charly/namespace.go: DELETED, #55 W3 B3)
Section titled “Namespace resolver (charly/namespace.go: DELETED, #55 W3 B3)”The resolver implements Go-package-member semantics over Config.Namespaces. Most of it was
ALREADY relocated before this doc last caught up (a pre-existing drift this sweep also corrects):
ResolveBoxRef/FindBoxByLeaf (spec-typed) live in spec/spec/config.go;
resolveNamespacedBases/pullNamespacedBox (buildkit-typed) live in
sdk/buildkit/config_resolve.go. The ONE piece that genuinely stayed in charly/namespace.go —
resolveLocalRefFor (registry-coupled via resolveLocalViaPlugin) — is gone too now:
checkLocalDeployScope/runLocalDeployScopePlan, its only caller, relocated to
candy/plugin-fleet/verify_local.go, which resolves a kind: local template PLUGIN-SIDE via
node_resolve.go’s lookupLocalTemplate instead (the resolved-project envelope + a direct
InvokeProvider(kind,"local",OpResolve) call — no LoadUnified, no registry). charly/namespace.go
had zero remaining content once this last function left, and is DELETED.
spec.SplitNamespaceRef(ref)(spec/spec/) — splits a qualified ref on its FIRST.into(ns, rest); a bare ref returnsok=false; the remainder may itself be qualified (a.b.c→"a","b.c").spec.ResolveBoxRef(ref)(spec/spec/config.go) — bare names resolve in the currentConfig;ns.namedescends intoc.Namespaces[ns]recursively, returning the entry plus theConfig(namespace context) it lives in.kind:localtemplate resolution now runs plugin-side (lookupLocalTemplate, above) instead of aresolveLocalRefsibling.resolveNamespacedBases(cfg, out, …)(sdk/buildkit/config_resolve.go, unexported — internal toResolveBox/ResolveAllBox) — after the local image set resolves, pulls every namespace-qualifiedbase:(and qualifiedbuilder:ref, but only for images that actually have layers to build) intoout, keyed by fully-qualified name, iterating to a fixpoint (a pulled-in image may reference a deeper namespaced base).pullNamespacedBox(from, ref, keyPrefix, …)(sdk/buildkit/config_resolve.go, unexported) — descends the namespace chain to the leaf, re-keys the entry’s own internal base to the fully-qualified ancestor so the build graph references it correctly, and recurses to pull that ancestor.
The inheritance rule lives here: distro:/build: are VALUES → inherited across a namespace boundary; builder: is a map of namespace-relative REFS → NOT inherited (the consumer declares its own). See the file header comment for the rationale (avoid leaking a base-namespace-relative ref into a consumer where that namespace doesn’t exist). leafName(ref) strips every namespace prefix to the final member name (arch.arch-builder → arch-builder); paired with resolveBoxRef’s returned namespace Config it keys the resolved entity in that Config.Box map (used by the reachability walk below).
Remote-layer resolver (sdk/loaderkit’s refs seams — the former charly/refs.go is DELETED, K-wave 2; charly/layers.go keeps the ScanAllCandyWithConfigOpts wrapper) — per-entity version + reachability-scoped collection
Section titled “Remote-layer resolver (sdk/loaderkit’s refs seams — the former charly/refs.go is DELETED, K-wave 2; charly/layers.go keeps the ScanAllCandyWithConfigOpts wrapper) — per-entity version + reachability-scoped collection”@github layer refs resolve in TWO phases: the :vTAG git tag is only the FETCH coordinate (which commit to clone); the layer’s own version: field — read AFTER fetch — is the authoritative identity that drives dedup + warn-and-newest-wins.
CandyRef(sdk/deploykit/candy_ref.go—CandyRef = spec.CandyRefEntry; the formercharly/refs.gois DELETED, K-wave 2) — the single representation of arequire:/candy:ref. It stores the ORIGINAL ref string (Raw, with any@repoprefix and:versionsuffix);.Bare()(the map-key form),.Version()(the pinned git tag — the FETCH coordinate, NOT the identity), and.IsRemote()are DERIVED. Aresolvedslot carries the qualified sibling key set byQualifyRemoteSiblingDeps(sdk/loaderkit/scan_candy.go) after a remote layer is fetched, so ONE list serves both the graph (keys on.Bare()) and the transitive fetch (keys on the immutable.Raw).Candy.Require/Candy.IncludedCandyare[]CandyRef— there are no parallel bare/raw arrays.- Two-phase per-entity-version resolution —
CollectRemoteRefsOpts(sdk/loaderkit/refs_collect.go— the formercharly/refs.gois DELETED, K-wave 2) collects EVERY distinct(repo, git-tag)a bare ref is referenced at — it does NOT collapse to one winning tag and does NOT warn (the git tag is just where to clone from). TheScanAllCandyWithConfigOptsfix-point (charly/layers.go) fetches each(repo, git-tag)(tracking scanned(repo,git-tag,ref)triples), reads each materialization’s per-entityversion:, accumulates candidates per bare ref, thenloaderkit.PickCandyVersion(sdk/loaderkit/candy_version.go, overspec.CandyCandidate) arbitrates: same per-entity version across different git tags → NO warning, the newest git tag wins for freshness (CompareSemver); different per-entity versions → warn once (naming both per-entity versions + sources) and the newest per-entity version wins (CompareCalVer). Exactly one materialization per bare ref reaches the layer map, so the graph + intermediates are unchanged. A fetched layer with NOversion:is a HARD ERROR (no fallback — first-party remotes are backfilled by remote-cache auto-migration,EnsureRepoDownloaded→migrateCacheViaPeer).PickCandyVersionis the SOLE arbiter for direct AND transitive refs, so a transitive dep can never silently pull a different version of an already-resolved layer. This is why a repo re-tag of an UNCHANGED layer no longer warns — the old resolver compared the repo git tag, which advances on every landing. - Reachability-scoped collection (
CollectRemoteRefsOpts.collectBox) — collection walks ONLY the enabled root images + the namespaced images reachable via theirbase:/builder:edges (Config.ResolveBoxRef+spec.LeafName), plus local layers’ transitive deps. It does NOT scan every image andkind:localtemplate of every imported namespace (that over-collection pulled unrelated layers pinned at a different tag — e.g. a namespace’scharly-cachyosworkstation template’schrome— and tripped the version policy). Builder edges ARE followed when an image builds (a namespacedfedora.fedora-builderis built as an intermediate and needs itsrpmfusion/yaylayers); dropping them under-collects (“unknown layer”). - One unified populator (
scanFromParsed,sdk/loaderkit/scan_candy.go— the port of the formerpopulateCandyFromYAMLfromcharly/unified.go, DELETED, K-wave 2; the plugin-side serialized adapter issdk/deploykit/spec_candy_adapter.go) — bothScanCandyManifest(discovered-layer-dir path) andScanInlineCandy(charly.yml inline path) call it, so they can’t drift. TheHas*predicates (HasEnv/HasPorts/HasVolumes/…) are derived methods; only the filesystem-probe caches (HasPixiToml/HasSrcDir/…) stay fields.
charly box reconcile (see /charly-build:reconcile) is the operator tool that aligns the on-disk git-tag pins so every reference of a repo fetches one commit, clearing any residual per-entity-version warning.
Capabilities — BoxMetadata alias + label completeness check
Section titled “Capabilities — BoxMetadata alias + label completeness check”spec.BoxMetadata (CUE-generated, spec/schema/boxmetadata.cue) — there is no more Capabilities alias; the former charly/capabilities.go type alias was ZERO-ALIASES residue, deleted (see /charly-internals:capabilities). CapabilityLabelMap (sdk/deploykit/capabilities.go) lists every field with its OCI label home; TestCapabilityLabelCompleteness fails the build if a spec.BoxMetadata field lacks a mapping. This invariant keeps charly fleet from-box reliable: every field deploy code might consult is readable from a pushed image’s labels alone, independent of charly.yml.
Kubernetes substrate (EXTERNAL — deploy:kubernetes, candy/plugin-kube)
Section titled “Kubernetes substrate (EXTERNAL — deploy:kubernetes, candy/plugin-kube)”target: kubernetes is an EXTERNAL deploy substrate (F1): there is no in-proc kubernetes deploy target — it resolves to pluginDeployTarget (charly/unified_targets.go, S3b), dispatched via candy/plugin-fleet’s Invoke(OpDeployDispatch) to the ACTUAL substrate provider, served out-of-process by candy/plugin-kube’s deploy:kubernetes provider (beside its kube: verb). The Kustomize GENERATOR is the COMPILED-IN candy/plugin-k8sgen (M13, verb:k8sgen serving OpEmit; the workload-kind heuristic selectWorkloadKind maps the generic kind: enum to Deployment/StatefulSet/DaemonSet/Job/CronJob/Pod) — kept SEPARATE from the heavy external plugin-kube because it has no client-go dependency and must resolve in the project-less from-box path. charly/k8s_generate.go and the former host_build_k8s_generate.go “k8s-generate-kustomize” HostBuild seam it served are BOTH DELETED (K5-A item 6): candy/plugin-kube/materialize.go’s materializeKustomize now does the whole thing PLUGIN-SIDE — lifts the 3 caps scalars (Port/UID/GID) + spec.Deploy + spec.Kubernetes into a spec.KubernetesGenInput, InvokeProviders verb:k8sgen/OpEmit (→ spec.KubernetesGenReply manifest docs) peer-to-peer, validates each doc via verb:egress/OpValidate (ValidateEgressValue) ALSO peer-to-peer (no host round trip for either leg), then writes the base/+overlays/ tree. The PLUGIN-side deploy:kubernetes preresolver (candy/plugin-kube/preresolve.go, F6/FINAL-K5-unit-6a) + charly fleet from-box (candy/plugin-fleet/deploy_from_box.go) both call materializeKustomize directly. The plugin runs kubectl --context <ctx> apply -k. See /charly-internals:install-plan + /charly-kubernetes:kubernetes.
VM target (external substrate)
Section titled “VM target (external substrate)”target: vm is an EXTERNAL deploy substrate, exactly like local/android/kubernetes: there is no in-proc VM deploy target. It resolves to pluginDeployTarget (S3b), dispatched via candy/plugin-fleet’s Invoke(OpDeployDispatch) to the ACTUAL substrate, served out-of-process by candy/plugin-deploy-vm’s deploy:vm provider. UNLIKE kubernetes, the vm substrate DOES consume the InstallPlan IR — the plugin walks the plan via the SAME shared sdk/kit.WalkPlans the local deploy uses, but the executor the reverse channel serves is the guest SSHExecutor (spec/exec/deploy_executor_ssh.go), so the same walk runs INSIDE the guest (bash bodies via ssh guest 'sudo bash -s'). The DeployExecutor interface (spec/exec/deploy_executor.go) decouples “how shell commands run” from the walk — ShellExecutor + SSHExecutor are the two implementations.
The VM venue lifecycle (boot the domain, build the guest SSH executor, nested pod-in-guest, teardown, the charly vm Start/Stop/Status/Logs/Shell/Rebuild) is IMPLEMENTED IN THE PLUGIN plugin-deploy-vm/candy/plugin-deploy-vm/lifecycle.go (Lifecycle:true) over generic seams — sdk/kit (ssh-config stanza, guest waits, charly delivery), HostBuild("cli") (the charly vm/charly box build family), self-loading the project directly for its OWN spec.LifecyclePrepareInput (sdk/loaderkit.ResolveVmEntityViaExecutor, K-wave W3a A3-phase-2 — the former host-side lifecyclePrepareHook DATA-seam AND the “deploy-entity-resolve” HostBuild seam it later used are BOTH gone), and the served guest executor reverse channel (nested pod-in-guest). Core keeps NO residual cleanup for the vm substrate: the F12 vmAttachResolver (charly/vm_lifecycle_preresolve.go) was DELETED at K-wave 2 cone CONTESTED — its body was strings.Join(cmd, " "), derivable from the raw wire cmd candy/plugin-fleet threads for OpAttach; plugin-deploy-vm/candy/plugin-deploy-vm/lifecycle.go’s vmAttach derives it from lifecycleParams.Cmd and charly/unified_targets.go’s Attach threads the raw cmd for hookless lifecycle substrates. The vm lifecyclePostTeardownHook moved plugin-side too (plugin-deploy-vm/candy/plugin-deploy-vm/lifecycle.go’s vmPostTeardown, F6 vm-lifecycle move). The lifecycle Ops are reached through pluginDeployTarget → candy/plugin-fleet’s Invoke(OpDeployDispatch) → the plugin’s own sdk.Executor.InvokeProvider call into the vm substrate provider (S3b, replacing the former dedicated grpcSubstrateLifecycle proxy + substrateLifecycle interface, both DELETED) — and persists the returned VmDeployState. Both pod and vm own a real venue lifecycle, each externalized to its plugin.
charly fleet add vm:<name> dispatches through fleet_add_cmd.go::dispatchNode → ResolveTarget → pluginDeployTarget (no per-kind dispatch function); fleet_add_cmd_vm.go carries the host-side VM-only helpers that REMAIN (vmNameFromDeployName, sshReverseRunner, resolveVmSshUser / resolveVmSshPort, saveVmDeployState, removeVmDeployEntry). Full architecture + preflight flow lives in /charly-internals:vm-deploy-target.
YAML surface ↔ Go identifier convention
Section titled “YAML surface ↔ Go identifier convention”The codebase keeps wire format (YAML keys) and internal names (Go fields/types) in strict symmetry — plural YAML keys get plural Go identifiers, singular get singular. The singular builder: / distro: / init: top-level keys in the embedded build vocabulary (charly/charly.yml) and project charly.yml carry singular Go identifiers: BuilderMap, BoxConfig.Builder, BuilderConfig.Builder, DistroConfig.Distro, InitConfig.Init. The rule: if you change a YAML tag, also rename the Go identifier. Tests enforce this indirectly — struct literals won’t compile if they disagree. Note: the OCI label key is grouped under platform.* / builder.* sub-namespaces — see LabelPlatformDistro, LabelPlatformFormat, LabelBuilderUse, LabelBuilderProvide in labels.go. Label wire-names are decoupled from YAML/Go identifiers by design.
Kong default:"withargs" for parent+leaf commands
Section titled “Kong default:"withargs" for parent+leaf commands”Kong normally treats a struct as either a branch (has child cmd:"" subcommands) OR a leaf (accepts arg:"" positionals and has a Run() method) — not both. When you want both shapes on the same parent command (e.g., charly config <image> runs setup AND charly config mount|status|… dispatch to subcommands), tag the default child with default:"withargs". Kong then dispatches to that child when the first token doesn’t match a subcommand name, passing positional args/flags through.
One use in the codebase:
candy/plugin-pod/pod_cmd.go—ConfigSetupCmdis the default (default:"withargs");charly config <image>routes through the setup cmd whilecharly config mount|status|…dispatch explicitly. (The formercharly/config_image.gois DELETED, K-wave 2.)
charly check — now the compiled-in command:check plugin (candy/plugin-check) — needs no such pattern: every live-container verb (wl/cdp/vnc/dbus/… and libvirt) is an out-of-process declarative verb, NOT a charly check subcommand, so no subcommand name can shadow the charly check live <image> positional.
Mode purity: LoadConfig must NOT read charly.yml
Section titled “Mode purity: LoadConfig must NOT read charly.yml”OCI labels are written exclusively from charly.yml at charly box build / charly box generate time. charly.yml is deploy-mode state and must never bleed into the baked image. The key guarantee lives in charly/loader_threaded.go:LoadConfig — it reads the PROJECT charly.yml only, never merging the per-host charly.yml overlay (no MergeDeployOverlay; the former charly/config.go is DELETED, K-wave 2).
The rule: every build-mode command (anything under charly box …) calls LoadConfig. If you ever re-introduce MergeDeployOverlay inside LoadConfig, you will silently contaminate OCI labels with whatever is in the user’s local charly.yml — exactly the bug that made images bake ports: ["5900:5900","9250:9222"] from a stale charly.yml entry instead of the charly.yml-declared ["5900:5900","9222:9222","9224:9224"].
Deploy-mode commands (charly config, charly start, charly stop, charly update, charly fleet add, charly fleet del, charly shell, charly cmd, charly service, charly vm create, …) read labels via ExtractMetadata and then apply the deploy overlay explicitly via MergeDeployOntoMetadata(meta, dc, instance). This split is load-bearing — never collapse it.
Host-deploy specifics: charly fleet add host is deploy mode (reads both charly.yml and charly.yml), not build mode — despite looking like “install on host, not into an image”. The compiler (BuildDeployPlan in install_build.go) is pure and shared with build mode, but the invocation path reads charly.yml for add_candy: and install_opts: like every other deploy-mode command.
InstallPlan IR — the shared intermediate representation
Section titled “InstallPlan IR — the shared intermediate representation”The DEPLOY paths that consume the shared IR are pod/vm plus the external local and android substrates (kubernetes is external too but does NOT consume it — see below); build-mode Containerfile emission is a SEPARATE generator (WriteCandySteps → EmitTasks in sdk/deploykit, relocated from charly/generate.go in #67, driven by candy/plugin-build over the envelope + HostBuild("render-seam"); the pod-overlay deploy path reaches deploykit.Generator.EmitTasks directly — charly/tasks.go holds NO shim and cannot (P16 import purity forbids charly/ importing the sdk); that path stays for the pod-overlay), NOT the IR. The kubernetes substrate is EXTERNAL and does NOT consume the IR — it materializes its Kustomize tree PLUGIN-side (candy/plugin-kube/materialize.go’s materializeKustomize, reaching candy/plugin-k8sgen peer-to-peer over verb:k8sgen/OpEmit) (see “Kubernetes substrate” above). Flow:
Layer + ResolvedBox + HostContext → BuildDeployPlan (install_build.go) [pure; deploy-path only, NOT charly box build] → InstallPlan (install_plan.go) → EmitTarget.Emit (NO in-proc EmitTargets remain — the former in-proc overlay walker + the pod overlay target were DELETED in P11c; the pod overlay render now lives in the candy `plugin-deploy-pod/overlay.go`, via `deploykit.OCITarget` + `deploykit.NewRenderGeneratorFromProject`) / UnifiedDeployTarget lifecycle └── pluginDeployTarget (charly/unified_targets.go, S3b) → candy/plugin-fleet's Invoke(OpDeployDispatch) → the ACTUAL substrate provider's own InvokeProvider (S1) over the OpExecute reverse channel (deploy:local — candy/plugin-deploy-local walks the IR via kit.WalkPlans, host-engine steps via RunHostStep; deploy:vm — candy/plugin-deploy-vm runs the SAME walk INSIDE the guest over the guest SSHExecutor, with the vm venue lifecycle plugin-implemented, reached through the SAME OpDeployDispatch generic dispatch; deploy:kubernetes — plugin-side preresolver generates the Kustomize tree, plugin runs kubectl apply -k; deploy:android)deploykit.OCITarget is constructed only by the candy plugin-deploy-pod’s buildOverlay; charly box build/generate emit via the WriteCandySteps → EmitTasks generator in sdk/deploykit (deploykit.Generator, relocated from charly/generate.go in #67; the pod-overlay deploy path reaches deploykit.Generator.EmitTasks directly — charly/tasks.go holds NO shim and cannot (P16 import purity forbids charly/ importing the sdk); that path stays for the pod-overlay), sharing the package-cascade / shell-snippet / localpkg compiler helpers with the IR. Full reference lives in /charly-internals:install-plan — go there before touching any of those files. Supporting Go files (ledger, builder_run, shell_profile, reverse_ops, sdk/deploykit/compile_service_steps.go (service_render, relocated), candy/plugin-fleet/deploy_ref.go (deploy_ref, relocated), hostdistro, migrate_services_tool) are covered in /charly-internals:local-infra.
VM-path architecture
Section titled “VM-path architecture”The VM path spans the following module topology:
| File | Role |
|---|---|
spec/spec/cue_types_gen.go (generated) |
VmSpec (= Vm) + VmSource discriminated union (cloud_image / bootc) + VmChecksum + VmNetwork + VmSsh + VmKeyInjection |
spec/spec/cue_types_gen.go (generated) |
VmCloudInit + VmCloudInitUser/File/Network/Mirrors + VmCharlyInstall (auto/scp/skip state machine) |
sdk/vmshared/libvirt_yaml.go |
stub — the per-device structs moved to #LibvirtDomain in spec/schema/vm.cue (generated LibvirtDomain into spec/spec/cue_types_gen.go); the opencharly YAML-facing shape of the libvirt: stanza is now CUE, its renderers plugin-vm/candy/plugin-vm/libvirt_yaml_bridge.go |
plugin-vm/candy/plugin-vm/libvirt_yaml_bridge.go |
RenderDomainXML/BuildLibvirtDomainXML pure functions (build a libvirtxml.Domain tree, marshal to XML) + buildDomainDevices device emission (passt backend, portForward attribute order, virtio-gpu default, SMBIOS credentials, XMLPassthrough merge) |
sdk/vmshared/qemu_render.go |
RenderQemuArgv for direct-QEMU backend |
sdk/vmshared/cloud_init_render.go + cloud_init_iso.go |
RenderCloudInit + ResolveKeyInjectionChannels + composeUsers (adopt-merge) + WriteSeedISO via xorriso/genisoimage/mkisofs |
candy/plugin-vm/vm_cloud_image.go + sdk/kit/http_fetch.go |
BuildCloudImage pipeline: fetch URL (FetchQcow2, in sdk/kit) + sha256 sidecar + resize + seed ISO render (the former charly/vm_cloud_image.go is DELETED, K-wave 2) |
spec/exec/charly_install.go |
exec.EnsureCharlyInDeployVenue — the GENERIC “copy charly into a running venue” mechanism over spec.DeployExecutor (container podman cp / VM-SSH scp / host install, all via DeployExecutor.PutFile): returns the charly invocation command, copying the host os.Executable() to a non-$PATH /tmp/charly-<hostVer>-<digest8> on absence/older (idempotent, never shadows a packaged charly). Used by nested from-image delegation (charly/plugin_grpc.go) + the --host SSH re-exec (spec/hostenv/reexec.go), so an image need not bake the charly layer. exec.EnsureCharlyInGuest is the VM-deploy PrepareVenue strategy wrapper (auto/scp/skip) layered on top — host-surface ssh/scp against the managed alias, before the reverse channel serves a guest executor. Both re-exported as kit.EnsureCharlyInDeployVenue/kit.EnsureCharlyInGuest in sdk/kit/exec_aliases.go |
sdk/vmshared/ovmf_paths.go |
ResolveOvmfPaths (per-distro OVMF_CODE/VARS paths) + EnsurePerVmNvram + ResolveOvmfForSpec (bios-sentinel returning empty strings) |
spec/schema/vm.cue (registered per-kind via the spec CUE registry) |
#Vm — the closed CUE schema validating VmSpec + the #LibvirtDomain/#VmCloudInit subtrees (the Go VM/libvirt validators were deleted; CUE owns it via the per-kind registry) |
spec/exec/deploy_executor*.go |
DeployExecutor interface + ShellExecutor + SSHExecutor with WaitForSSH + WaitForCloudInit |
charly/unified_targets.go + charly/deploy_target_dispatch.go |
S3b: pluginDeployTarget — the thin, data-only adapter EVERY external substrate (local/vm/pod/kubernetes/android) routes through, dispatching via dispatchDeployTarget to candy/plugin-fleet’s Invoke(OpDeployDispatch); the ONE surviving core arbiter primitive is the op=“remove” release chain folded into charly/host_build_pod_lifecycle_dispatch.go from the deleted preempt.go (K-wave 2 cone CONTESTED — the deploy-dispatch Start/Stop bracket went peer-dispatch at R2 bank E, host_build_arbiter_bracket.go DELETED). Replaces the DELETED charly/deploy_target_external.go (externalDeployTarget), charly/substrate_lifecycle_grpc.go (grpcSubstrateLifecycle), charly/deploy_preresolve.go (wireDeployPreresolver), and charly/deploy_substrate_lifecycle.go (the substrateLifecycle interface) |
plugin-fleet/candy/plugin-fleet/deploy_target.go |
S3b: the ORCHESTRATION bulk ported from the deleted core files above, behind runDeployDispatch (sdk.OpDeployDispatch, discriminated by an Op field: add/update/del/start/stop/status/logs/shell/attach/rebuild); reaches the ACTUAL substrate provider via its own sdk.Executor.InvokeProvider (S1) |
charly/vm_lifecycle_preresolve.go — DELETED (K-wave 2 cone CONTESTED). |
FINAL/K5 unit 6a + CONTESTED: the vm lifecyclePrepareHook DATA-seam is GONE — the plugin resolves its own spec.LifecyclePrepareInput by self-loading the project directly now (sdk/loaderkit.ResolveVmEntityViaExecutor, K-wave W3a A3-phase-2 — the “deploy-entity-resolve” HostBuild seam it used in between is also gone). The last survivor, the F12 vmAttachResolver, was DELETED at K-wave 2 cone CONTESTED (the plugin derives the attach script from the raw wire cmd — vmAttach over lifecycleParams.Cmd; charly/unified_targets.go’s Attach threads it for hookless lifecycle substrates). The vm lifecyclePostTeardownHook moved plugin-side too (plugin-deploy-vm/candy/plugin-deploy-vm/lifecycle.go’s vmPostTeardown); the vm venue lifecycle itself lives in plugin-deploy-vm/candy/plugin-deploy-vm/lifecycle.go |
candy/plugin-deploy-vm/ |
the out-of-process deploy:vm plugin — the plan WALK (kit.WalkPlans over the guest SSHExecutor) + the venue lifecycle (lifecycle.go, over kit + HostBuild("cli") + the served guest executor) |
sdk/deploykit/vm_deploy_state.go + plugin-vm/candy/plugin-vm/vm_util_copies.go |
VM-only deploy helpers (the former charly/fleet_add_cmd_vm.go is DELETED, K-wave 2): vmshared.VmNameFromDeployName (spec/spec/vm_domain.go), resolveVmSshUser / resolveVmSshPort (plugin-vm/candy/plugin-vm/vm_util_copies.go), deploykit.SaveVmDeployState / deploykit.RemoveVmDeployEntry (sdk/deploykit/vm_deploy_state.go); charly fleet add vm:<name> itself dispatches through dispatchNode → ResolveTarget → pluginDeployTarget |
candy/plugin-vm/vm_create_spec.go + candy/plugin-vm/vm_build_resolve.go |
charly vm create CLI wiring (in the command:vm plugin) + the VM-disk build ENGINE (plugin-side — the former charly/vm_build.go HostBuild(“vm-build”) host-builder is DELETED; the resolve moved into the plugin) reading kind: vm entities |
sdk/vmshared/libvirt_helpers.go + libvirt_yaml_listen.go |
helpers shared by the libvirt YAML bridge + qemu_render argv emitter (VmRuntimeParams); structured <listen> support for LibvirtGraphics |
unified.go VM support (C2-substrate): "vm" is NO LONGER a spec.KindWords kind — the substrate kinds (pod/vm/kubernetes/local/android) were externalized to the compiled-in candy/plugin-substrate (kind:pod/vm/kubernetes/local/android, Structural:true), so vm LEFT spec.KindWords + the #Node disjunction (no #VmArm) but STAYS in spec.ResourceKinds (member nesting) and its #VmValue def is KEPT for the host-side value gate. A vm: node resolves via recognizedKind (the compiled-in provider) → runPluginKind → foldSubstrateKind, which host-decodes the CANONICAL node via the relocated sdk/loaderkit tree-builder, reached through the ProjectLoader seam (K1 unit 3b — BuildFleetNode for a deploy shape → uf.Fleet; for a bare template, DecodeStandaloneTemplateJSON — the C2-substrate TEMPLATE fold arm), validates its value against the KEPT #VmValue def (validateKindValueCUE), threads it to plugin-substrate’s OpLoad via op.Env (spec.StructuralKindLoadEnv.Standalone), and folds the plugin’s ECHO into uf.VM via foldStandaloneTemplateReply → foldOpaqueTemplateReply, which stores the reply JSON VERBATIM into uf.VM map[string]json.RawMessage — an OPAQUE map, not a typed map[string]*VmSpec: the kernel keeps the template body opaque and candy/plugin-substrate resolves it on read (the same de-typed pattern as local/android). The type VmSpec is no longer bound in package main at all (charly core names it fully-qualified as spec.VmSpec where it needs it); the VM plugin’s own candy/plugin-vm/vmshared_aliases.go binds it, and it resolves through the stable vmshared.VmSpec (sdk/vmshared/spec_aliases.go) to spec.ResolvedVm — a wire-envelope type mirroring spec.Vm’s fields (CUE-sourced at spec/schema/vm.cue, generated into spec/spec/cue_types_gen.go), NOT spec.Vm itself. #NodeDoc is the sole STRUCTURE gate; a residual legacy vm:-keyed (or vms:-plural) document is hard-rejected by classifyDoc with a charly migrate hint (rootShapeKeySet unions spec.ResourceKinds so the substrate words stay legacy-detectable).
Full subsystem references: /charly-internals:vm-spec, /charly-internals:libvirt-renderer, /charly-internals:cloud-init-renderer, /charly-internals:vm-deploy-target, /charly-internals:ovmf, /charly-internals:cutover-policy.
Self-exec coordination: host → container AND host → host
Section titled “Self-exec coordination: host → container AND host → host”The charly binary self-execs in two distinct directions.
Host → container — the host charly delegates to a container-baked (or copied-in) charly via exec … charly <subcommand>. The surviving site is nested from-image delegation: a from-image plan re-invokes charly inside the venue, and exec.EnsureCharlyInDeployVenue (spec/exec/charly_install.go, re-exported as kit.EnsureCharlyInDeployVenue) copies the host binary in on demand when the venue lacks it. The best-effort desktop notification (candy/plugin-cmd/notify.go’s sendVenueNotification — the former charly/notify.go is DELETED, K-wave 2) is NOT a self-exec site — it drives the venue’s session bus with gdbus directly, no in-container charly.
Host → host (none) — there is NO host→host self-exec for check verbs. Every live-container verb (wl/cdp/vnc/dbus/mcp/record/kube/adb/appium/spice/libvirt) dispatches OUT-OF-PROCESS through the provider registry to its plugin candy (EXEC-based verbs drive the venue over the DeployExecutor reverse channel; endpoint verbs dial a host-pre-resolved address) — never by spawning a charly subprocess. charly’s core carries no in-proc live-verb dispatch machinery.
The rule: whenever you rename a subcommand path crossed by the surviving host→container self-exec site (nested from-image delegation), edit the host-side invocation strings AND plan a coordinated rebuild of every image that bakes the charly layer (affected images: grep charly.yml for - charly$).
Source Code Map
Section titled “Source Code Map”Core Generation
Section titled “Core Generation”| File | Purpose |
|---|---|
main.go |
CLI entry point (Kong framework). CLI struct carries two global path fields: Dir (-C / --dir / env CHARLY_PROJECT_DIR) and Repo (--repo / env CHARLY_PROJECT_REPO). When Repo is set, main() resolves it via ResolveProjectRepo and assigns the cache path back into Dir; when Dir is non-empty (after that resolution), main() calls os.Chdir(Dir) before ctx.Run() — one-line intervention that propagates to every os.Getwd() call site throughout build-mode commands without requiring per-command plumbing. --repo and --dir are mutually exclusive (fast-fail). Covered by TestCharlyDir_FlagChdir, TestCharlyDir_Errors, TestCharlyRepo_FlagChdir, TestCharlyRepo_DirConflict, TestCharlyRepo_DefaultExpansion in main_dir_test.go + main_repo_test.go. Load-bearing for charly mcp serve inside a container where cwd resolves to /workspace (the charly-mcp layer default) — either bind-mounted with the project, or empty in which case the externalized MCP server’s managed --repo default child prefix falls back to the upstream repo (computeProjectPrefix in candy/plugin-mcp/serve.go). |
spec/spec/repo_identity.go + charly/loader_threaded.go |
--repo resolver (the former charly/main_repo.go is DELETED, K-wave 2). spec.DefaultProjectRepo = "github.com/opencharly/charly". spec.NormalizeRepoSpec(spec) handles four spec shapes: "default" literal, bare owner/repo (auto-prefix github.com/ when first segment has no dot), bare owner/repo@ref, host-qualified host.tld/owner/repo[@ref]. ResolveProjectRepo (charly/loader_threaded.go) reuses loaderkit.EnsureRepoDownloaded from the refs seams (sdk/loaderkit/refs_collect.go) so the project-repo cache shares ~/.cache/charly/repos/ (override CHARLY_REPO_CACHE) with the existing remote-layer cache. Empty version triggers GitDefaultBranch resolution. |
sdk/buildkit/config_resolve.go + charly/loader_threaded.go |
charly.yml parsing + box resolution (the former charly/config.go is DELETED, K-wave 2). ResolveBox/ResolveAllBox (buildkit) are free functions over *spec.Config; LoadConfig / LoadBuildConfigForBox (charly/loader_threaded.go) load the project. ResolvedBox.Tags/BuildFormats are wire-clean fields on spec.ResolvedBox |
sdk/buildkit/format_config.go + spec/spec/{distro_config_methods.go,builder_config_methods.go} |
The build-vocabulary config types (the former charly/format_config.go is DELETED, K-wave 2): DistroConfig / BuilderConfig aliases (sdk/buildkit/format_config.go), with the CUE-sourced vocabulary-resolution methods (ResolveDistro / ResolveInherits / AllFormatNames / BuilderNames) living in spec. LoadBuildConfigForBox (charly/loader_threaded.go) reads the project charly.yml (via LoadUnified, with the embedded build vocabulary merged in as the project-wins base) and splits it into DistroConfig / BuilderConfig / InitConfig views |
sdk/buildkit/render.go |
Go text/template rendering engine (the former charly/format_template.go is DELETED, K-wave 2). RenderTemplate + the template helpers (cacheMounts, cacheMountsOwned, quote, default, splitFirst, replace, join, anyRepoHasURL). InstallContext, BuildStageContext types |
layers.go |
Layer scanning, file detection. spec.CandyYAML (an intra-spec-package alias to spec.Candy, the real generated struct — no hand struct; the alias lives at spec/spec/charly_names.go, NOT a charly-core file), CUE-decoded via cue_loader.go; load-time top-level typo-detection via the rejectUnknownCandyTopLevelKeys guard — no custom UnmarshalYAML. Task struct + Kind() method (exactly-one-verb). scanFromParsed (sdk/loaderkit/scan_candy.go:111) is the SOLE package-surface populator — the derivePackageSectionsFromCalamares it was ported from exists in NO module and survives only in comments: every distro: key (bare / versioned / compound) → a per-distro TagSections entry (NOT a shared format section — that collapse caused the non-deterministic deb-repo bug); top-level package: → TopPackages (folded at resolve time); arch aur: keeps its aur format section. CompileSystemPackageSteps (sdk/deploykit/install_build.go) cascades these — see /charly-internals:install-plan. The runtime Candy.ExternalBuilder field (the reserved word of an EXTERNAL builder plugin a candy selects; from the candy manifest external_builder:) is populated by the loader’s candy scan (sdk/loaderkit/scan_candy.go’s scanFromParsed, the port of the former populateCandyFromYAML from the DELETED charly/unified.go, K-wave 2) and resolved at build via OpResolve (sdk/deploykit EmitExternalBuilderStages, relocated in #67). |
tasks.go |
64 lines, ONE function: invokeOpEmitFragmentOpt — the reverse-channel helper that Invoke(OpEmit)s a plugin verb and returns its Containerfile fragment (rejecting an empty one unless allowEmpty). It is NOT a shim to deploykit.Generator.EmitTasks and cannot be: the P16 import-purity gate forbids charly/ importing github.com/opencharly/sdk at all, so core cannot call a deploykit method. All per-verb emission (EmitVarsEnv, EmitMkdirBatch, EmitCopy, EmitWrite, EmitLinkBatch, EmitSetcapBatch, EmitDownload, EmitCmd, BuildStepShellDashC, BuildStepShellHeredoc) and StageInlineContent live in sdk/deploykit (tasks_emit.go, tasks_stage.go); the orchestrator is (g *Generator) EmitTasks (tasks_render.go). Shell quoting is spec.ShellQuote. |
sdk/deploykit (the generator — the former charly/generate.go is DELETED, K-wave 2) |
The build-mode Containerfile render DRIVE (Generate / generateContainerfile / WriteCandySteps / WriteLabels / writeJSONLabel / WriteBootstrap / EmitBuilderStages / EmitBuilderArtifacts / EmitExternalBuilderStages / EmitExternalBuilderArtifacts / GenerateTraefikRoutes / EmitTraefikRouteStage / EmitInitFragmentStages / EmitInitAssembly) moved to sdk/deploykit (deploykit.Generator) in #67, driven by candy/plugin-build over the resolved-project envelope + HostBuild("render-seam"). emitBakedPlugins likewise moved to sdk/deploykit (deploykit.EmitBakedPlugins, wired directly by NewRenderGeneratorFromProject — the former HostBuild("bake-plugins") round-trip + host_build_bake_plugins.go are DELETED). The Generator is constructed via deploykit.NewRenderGeneratorFromProject + the render-prep (Generator.RenderPrepAll/RenderPrepBox, called from candy/plugin-build/resolve.go fills the caches); the shared builder OpResolve helper (renderSeamCaller.resolveBuilderStage, sdk/deploykit/render_generator_from_project.go) drives both builder legs (EmitExternalBuilderStages lives in sdk/deploykit/builders_render.go); generateInitFragments is sdk/deploykit/init.go‘s Generator.GenerateInitFragments (reached peer-to-peer from candy/plugin-deploy-pod/overlay.go); writeContextIgnore is candy/plugin-build/host_prep.go; the host-fs helpers are sdk/deploykit/header_copy_remote.go (materializeBuildConfigAsset/rewriteHeaderCopyForRemote) + candy/plugin-build/host_prep.go (createRemoteCandyCopies) + candy/plugin-deploy-pod/overlay.go (candyByName); resolveStatus is candy/plugin-box/inspect_list.go. WriteCandySteps (deploykit) orchestrates per-layer: packages → EmitTasks (deploykit; the pod-overlay deploy path reaches deploykit.Generator.EmitTasks directly — charly/tasks.go holds NO shim and cannot (P16 import purity forbids charly/ importing the sdk); that path stays for the pod-overlay) → builders → USER reset. Package resolution goes through the SAME ResolveCascadePackages (sdk/deploykit/install_build.go) the deploy compiler uses — ONE distro-specificity cascade for build AND deploy (folds the top-level package: base + unions distro tag sections most-specific-first), then renders the primary format’s install template; non-primary build formats (aur) emit from their own format section. Config-driven format install and bootstrap from the build vocabulary (distro: + builder: sections — the embedded default lives in charly/charly.yml); builder STAGE templates moved out to the plugins’ kit.BuilderResolve (C10 — the builder: section retains detection + cache mounts + the deploy host phase + pixi’s context inputs). WriteLabels (deploykit) is called at the END of the final stage (after the final USER directive) — the volatile LabelDescription value would otherwise invalidate every downstream RUN/COPY on a baked-plan edit; with LABELs-at-end, only the LABEL steps themselves re-emit (cache preserves all install work). writeJSONLabel (deploykit) routes every JSON label value through spec.ShellQuote so embedded ' chars in test commands (awk '{print $1}') don’t break podman’s key=value LABEL parser. |
candy/plugin-box/{validate_config_rules.go, validate_schema_rules.go} |
The host-natural validation checks that need the raw loader — the former charly/validate.go is DELETED, K-wave 2 (they moved to the compiled-in command:box plugin): CUE-conformance (validateCandyCUESchemas / validateProjectCUESchemas), validateBuildAndDistro, validateBuildTunables, validateMergeConfig, validateBuilderRefs, validateBoxBaseFrom, validateRemoteCandies, boxEntityWireYAML, isNodeFormFile (validate_schema_rules.go); the box+config rule set lives in validate_config_rules.go. Format/builder validation against config definitions (not hardcoded maps). The per-kind/op/candy/graph rule ENGINE (validateCandyTasks / validateCandyContents / validateAliases / validateVolume / validateCandyReferences / the box+candy DAGs / validateOps / the candy plugin: block checks) moved to the compiled-in command:box plugin (candy/plugin-box/{validate.go, validate_rules.go, validate_graph.go, validate_check.go}), reading the resolved-project envelope. (validateVocabularyCollections + its sibling validateEntityCUE, and core’s validatePluginCandy, were dead-code-radical-removal-batch deletions — the modern per-kind LOAD-time plugin gate and candy/plugin-box’s own plugin:-block checks supersede them; see the batch’s CHANGELOG entry for the coverage-comparison evidence.) |
version.go |
CalVer computation |
sdk/kit/scaffold.go |
new layer scaffolding (kit.ScaffoldCandy — single-layer dir creation with stub charly.yml; the former charly/scaffold.go is DELETED, K-wave 2) |
box_fetch_reentry.go — DELETED (K-wave 2). |
The hidden charly __box-fetch / __box-refresh core reentry points behind the COMPILED-IN candy/plugin-authoring command:fetch / command:refresh verbs (P14b) moved with the file. The repo resolver (ResolveProjectRepo → EnsureRepoDownloaded: CHARLY_REPO_OVERRIDE + the refs-backend dispatch + the command:migrate auto-migration) is host-coupled, so the authoring plugin re-runs these hidden commands over HostBuild("cli"). The box set/add-candy/rm-candy/write/cat authoring verbs + the AddCandyToBox/RemoveCandyFromBox yaml.Node helpers + resolveProjectFile (the path-traversal guard for box write / box cat) MOVED to the compiled-in command:authoring plugin (candy/plugin-authoring, P14b) — they run PURE on sdk/kit (kit.SetByDotPath / kit.MappingChild / kit.SaveYAMLNodeFile) + stdlib, zero core reentry. The create-side box new project/box/candy ENGINE is kit.ScaffoldProject / kit.AddBox / kit.ScaffoldCandy (shared with candy/plugin-box’s command:new). Tested in sdk/kit/scaffold_test.go (the kit-scaffolder tests) + candy/plugin-authoring/authoring_edit_test.go (the moved helpers). |
sdk/kit/yaml.go (SDK kit, not charly/) |
kit.SetByDotPath(path, dotpath, valueYAML) + kit.MappingChild(m, key) — the generic comment-preserving YAML node utilities used by charly box set and the command:candy plugin. Walk *yaml.Node trees; create intermediate mappings on demand; reject descent into scalars. Tested in sdk/kit/yaml_test.go (TestSetByDotPath_ScalarReplacement + list-value, intermediate-mapping, scalar-descent-error cases). |
Plugins, external deploy & build-time emit
Section titled “Plugins, external deploy & build-time emit”Provider/registry/SDK internals are owned by /charly-internals:plugin; the external-deploy lifecycle + wire types by /charly-internals:install-plan. The file map:
| File | Purpose |
|---|---|
unified_targets.go + deploy_target_dispatch.go + host_build_arbiter_bracket.go |
S3b: pluginDeployTarget — the thin, data-only UnifiedDeployTarget adapter EVERY out-of-process deploy substrate routes through, dispatching via candy/plugin-fleet’s Invoke(OpDeployDispatch); records teardown ops to the ledger keyed on computeDeployID. Replaces the DELETED deploy_target_external.go (externalDeployTarget) + substrate_lifecycle_grpc.go (grpcSubstrateLifecycle) |
plugin_step_external_test.go |
The surviving witness that the in-proc external-plugin step path is GONE. charly/plugin_step_external.go, externalPluginStepProvider and its in-proc EmitOCI were all deleted — charly/plugin_dispatch_reverse.go:65 records it as “is gone” and plugin_step_external_test.go:80 as “the former externalPluginStepProvider/EmitOCI is deleted, R1: zero live”. A run: plugin: <verb> step served by an out-of-process plugin now dispatches through charly/plugin_executor_reverse.go’s invokeExternalStep and candy/plugin-installstep/oci_dispatch.go’s dispatchExternalPluginVerb. The step kind is StepKindExternalPlugin (deploykit.ExternalPluginStep); the kind list is AllStepKinds (sdk/deploykit/steps.go:104, aliasing spec/install_step_vocab.go:902). Note the capital A: the lowercase spelling that older prose used exists in no module. |
plugin_prescan.go |
Byte-gated, additive parse pre-scan: prescanPluginManifest registers an external deploy SUBSTRATE word (ClassDeployTarget, consumed by unified.go’s loader path before the provider connects), an external COMMAND word (ClassCommand, registered via registerDeclaredExternalCommand, snapshot via declaredExternalCommandWords, consumed by prescanProjectCommandWords in main.go before kong.Parse), AND (F4) an external KIND word (ClassKind → registerDeclaredKind; recognizedKind = connected-OR-prescanned). F4 kind connect: unlike a deploy substrate (which defers to the fleet builder) or a command (lazy-connect on invocation), a kind: <plugin-word> entity must DECODE its body during load (runPluginKind), so connectDeclaredKindPlugins (called at the depth-0 loader hook right after the prescan) host-builds + connects the declared kind plugins BEFORE mergeUnifiedDocs. It is re-entrancy-GUARDED (inKindConnectPass): the connect re-loads the project (LoadConfig/ScanAllCandyWithConfigOpts → LoadUnified → the SAME root that contains the kind node), and the nested load skips the pre-pass while normalizeNodeInto DEFERS (skips, no error) the not-yet-connected kind node — so the nested scan succeeds and the OUTER pass then has the provider registered + decodes. A declared kind whose provider never connects is WARN-SKIPPED in normalizeNodeInto — a loud stderr warning + the node dropped (never a silent drop, never a hard load error), so read-only commands (box list, validate) still work when a plugin can’t build/connect in a degraded environment (a minimal container with no Go toolchain); a command that actually USES the kind fails loudly at that point. Example: candy/plugin-example-kind (out-of-process-only). F5 flat-vs-structural decode (runPluginKind, provider_kind_invoke.go): a FLAT kind lands its OpLoad body opaquely in uf.PluginKinds[disc][name] (F4); a STRUCTURAL kind (capability Structural=true, carried by the structuralKindCarrier on the grpc/inproc provider) returns a spec.Deploy (FleetNode) member tree runPluginKind json-unmarshals + folds into uf.Fleet[name] — the SAME map sdk/loaderkit.BuildFleetNodeInto populates for a builtin pod (reached through the ProjectLoader seam since K1 unit 3b, not a core-resident function), so the folded member goes through the SAME validateDeploy. This is the channel that externalizes the structural kind decoders: group is DONE (C2-group — the COMPILED-IN candy/plugin-group serves kind:group, Structural:true) and the deploy-substrate kinds pod/vm/kubernetes/local/android are DONE (C2-substrate — the COMPILED-IN candy/plugin-substrate serves all of them, Structural:true; the shared builtin standaloneKind + cue_kind_*.go-arm removal, all left spec.KindWords + the #Node disjunction but STAY in spec.ResourceKinds so the loader still nests their members). Unlike group (a scalar #GroupInput value decoded in the plugin from op.Params), a substrate value is RICH + core-referencing, so the host uses the F5 Standalone channel: foldSubstrateKind (provider_kind_invoke.go) host-decodes the CANONICAL node via the relocated sdk/loaderkit tree-builder, reached through the ProjectLoader seam (K1 unit 3b — BuildFleetNode deploy → uf.Fleet, DecodeStandaloneTemplateJSON template → uf.Pod/uf.VM/… — the TEMPLATE-map fold arm extending F5’s deploy-only fold), validates the value host-side against the KEPT #<Kind>Value def (validateKindValueCUE, replacing the removed #Node arm’s closedness — a self-contained plugin schema can’t carry the rich value), threads it via op.Env (spec.StandaloneLoad), and folds the plugin’s pure ECHO. candy is DONE too (C2-candy — the LAST structural kind; candy/plugin-candy-kind, COMPILED-IN): foldCandyKind host-decodes via the bootstrap-critical core candyIsImage + buildCandy (which STAY core — the discovered-candy pre-check calls them directly, so the compiled-in plugin has no bootstrap cycle), validates against the KEPT #CandyValue (validateKindValueCUE), threads spec.Box/spec.Candy via the SAME StandaloneLoad channel (candy-image/candy-layer shapes), and folds the echo into uf.Box/uf.Candy. candy is Structural:false (it nests no deploy members) and routes via an explicit gn.disc=="candy" host branch. So the #Node disjunction now has ZERO built-in arms (#Node: {...} — a structural gate only; per-kind value closedness is host-side) and spec.KindWords is EMPTY — every authoring kind is plugin-served. Authored-member INPUT-threading: the node’s AUTHORED resource-member children cannot ride op.Params (closed #<Kind>Input), so runPluginKind PRE-DECODES them host-side via the SAME sdk/loaderkit recursion the builtin path uses (BuildResourceMemberChildren, reached through the ProjectLoader seam since K1 unit 3b — the ONE member-decode source, called by BuildFleetNode too, R3) and threads the decoded subtree to OpLoad via op.Env (spec.StructuralKindLoadEnv); the plugin attaches them to its reply, so the reconstructed Fleet is byte-equivalent to the former builtin group (proven by TestExternalStructKind_StructuralDecode + the check-group / check-structkind runtime beds). A FLAT kind carrying members is a hard error (no silent drop). The parser gate admits sub-entity children under a recognized external STRUCTURAL kind (externalKindMayNestMembers/recognizedStructuralKind); core non-resource kinds stay guarded. Example: candy/plugin-group (compiled-in structural kind); candy/plugin-example-structkind (out-of-process-only witness). |
plugin_command_prescan.go |
The EARLY (pre-kong.Parse) external-COMMAND-word prescan: prescanProjectCommandWords resolves the project dir pre-parse (projectDirPreParse: CHARLY_PROJECT_DIR → scanDirFlag over os.Args → scanRepoFlag/CHARLY_PROJECT_REPO via ResolveProjectRepo → cwd) and registers each declared command word so charly <word> PARSES; connectCommandPlugin is the LAZY connect (LoadConfig → ScanAllCandyWithConfigOpts → loadProjectPlugins scoped to the one word → resolve(ClassCommand, word)), paid only on an actual charly <word> invocation. --repo MUST resolve in the prescan, not only at main()’s post-parse chdir: kong.Parse freezes the grammar first, so a word living only in the --repo target would never be registered and charly --repo <owner/repo> <word> reported an unknown verb — reading charly.yml worked (after the chdir), finding the verb did not. Resolution can clone, so it is attempted ONLY when the flag or env var is present; an unresolvable spec falls through to cwd so the LOCAL grammar survives and main() reports cannot resolve --repo rather than a misleading unknown-verb error |
provider_command_external.go |
OUT-OF-PROCESS command dispatch: collectExternalCommandPlugins builds a Kong grammar holder per prescanned word with the provider UNconnected (prov nil) so the CLI parses; dispatchExternalCommand lazy-connects on invocation (connectCommandPlugin) and forwards the pass-through args via Invoke(OpRun, {"args":[…]}); NestedCommandProvider nests an external command under a parent (e.g. charly check kube). The BUILTIN command path is provider_command.go (CommandProvider.KongCommand() + Go Run; builtinCommandBase.Invoke is in-proc-only). F8 command compile-in: dispatchCommand (the dispatch entry, called from main) routes a parsed dynamic command by PLACEMENT — a COMPILED-IN command candy (registered in-proc as an inprocProvider, not a *grpcProvider) dispatches IN-PROC via dispatchInProcCommand → Invoke(OpRun, {"args":[…]}), so the candy’s handler runs in charly’s own process (native stdio); an out-of-process one keeps dispatchExternalCommand/syscall.Exec. The dynamic Kong grammar (externalCommandHolder) is identical for both placements — only the dispatch transport differs (the command half of placement-invisibility). Example: candy/plugin-example-command (dual-placement, compiled-in) |
check_venue.go |
checkLocalTarget routes an external deploy host-side (the SAME path target: local takes) for charly check live / charly check <verb>, R3 |
spec/schema/deploy.cue + spec/schema/buildwire.cue + spec/schema/seam.cue (appended) |
Deploy IR wire types shared with the plugin SDK, CUE-sourced (SDD conversion; the former hand-written sdk/spec/deploy_wire.go is deleted): #ReverseOp (+ the hand Scope/ReverseOpKind named-enum types + ReverseOpPluginScript const, spec/spec/deploy_consts.go), #InstallPlanView, #DeployVenue, #DeployReply (deploy.cue); the build-time #BuildEnv / #EmitReply for OpEmit, and #BuilderResolveInput + #BuilderResolveReply ({Stage, CopyArtifacts, CopyBinary, InlineFragment}) for the builder OpResolve leg (buildwire.cue); the M4 substrate-lifecycle #LifecycleOpts/#HostEnv/#LifecyclePrepareInput/#PrepareVenueReply/#PostTeardownReply/#CliRequest/#CliReply (seam.cue, appended) |
tasks.go:invokeOpEmitFragmentOpt |
Renders a plugin verb’s BUILD-context Containerfile fragment via Invoke(OpEmit) → spec.EmitReply.Fragment (placement-agnostic above the registry) |
deploykit: EmitBuilderStages / EmitBuilderArtifacts + sdk/deploykit: resolveBuilderStage |
The DETECTION-builder BUILDER leg (C10, relocated to sdk/deploykit in #67): for each candy a builder DETECTS, connects the plugin (ensureBuildersConnected) + Invoke(OpResolve) via the shared resolveBuilderStage (sdk/deploykit — the former charly/generate.go is DELETED, K-wave 2) → spec.BuilderResolveReply (Stage pre-main-FROM, CopyArtifacts+CopyBinary post-main-FROM; cargo’s InlineFragment splices in WriteCandySteps (deploykit)). Renders via the plugins’ kit.BuilderResolve, NOT an in-core vocabulary. Detection stays host-side (candyNeedsBuilder) |
deploykit: EmitExternalBuilderStages / EmitExternalBuilderArtifacts + sdk/deploykit: resolveExternalBuilder |
The external_builder: BUILDER leg (relocated in #67): emit an out-of-tree ClassBuilder candy’s multi-stage via the SAME resolveBuilderStage/Invoke(OpResolve) (minimal input — candy name only); selected by a candy’s external_builder: field, requires a non-empty Stage. resolveExternalBuilder lives in sdk/deploykit (the former charly/generate.go is DELETED, K-wave 2) |
build_emit_test.go |
TestOpActsInBuildDeploy_PlacementAgnosticBuildEmit — the build-time-plugin-execution gate: a connected builtin BuildEmitter and a prescan-declared EXTERNAL verb both act in build/deploy, while an unknown undeclared verb does NOT (no blanket accept). TestKitVerbActAdapter_OpEmitActScript covers the act-script half |
provider_bench_test.go |
The E3 perf go/no-go gate: TestPerfGate_BuiltinVerbsSkipEnvelope, BenchmarkVerbTypedDispatchFork (0-alloc) vs BenchmarkVerbEnvelopeMarshal — builtins skip the JSON Invoke envelope; it is paid ONLY out-of-process |
Dependency & Graph
Section titled “Dependency & Graph”| File | Purpose |
|---|---|
graph_shim.go — DELETED (K-wave 2). |
The thin package-main wrappers (ResolveBoxOrder(), BoxNeedsBuilder()) are gone — the topological sort lives directly in sdk/deploykit/graph.go (the old core graph.go is gone) |
sdk/deploykit/intermediates.go |
Auto-intermediate image computation (trie analysis — the former charly/intermediates.go is DELETED, K-wave 2). ComputeIntermediates inherits Distro and BuildFormats from the parent image first, falling back to cfg.Defaults.* only when the parent is external or empty. Inverting this (defaults winning over the explicit parent) mis-tags every arch-rooted intermediate as build: [rpm], so every layer section keyed on pac: emits an empty RUN step (symptom: arch-ssh-client ships without direnv / gnupg / openssh). Regression guard: TestComputeIntermediates_InheritDistroFromParent uses defaults.Build=[rpm] but expects arch-rooted intermediates to come out [pac]. |
Build & Runtime
Section titled “Build & Runtime”| File | Purpose |
|---|---|
candy/plugin-build/ |
build command (the compiled-in build:box plugin — runBoxBuild + the podman drive; sequential image building, retry logic; the former charly/build.go is DELETED, K-wave 2) |
candy/plugin-box/box.go + candy/plugin-box/merge_cmd.go |
merge command (post-build layer merging — the former charly/merge.go is DELETED, K-wave 2) |
candy/plugin-pod/command.go |
shell command (execs engine run — the former charly/shell.go is DELETED, K-wave 2) |
candy/plugin-pod/command.go |
start/stop commands (the former charly/start.go is DELETED, K-wave 2) |
candy/plugin-status/command.go |
status command (the Kong grammar + dispatch; structured table/detail view, live tool probing, --json) — relocated from core, reached via InvokeProvider(verb:status-fanout) directly (the status-substrate HostBuild seam is DELETED, K-wave 2) |
candy/plugin-substrate/status_flat.go |
the charly status collection ENGINE (K6, whole-file relocated from charly/status_collector.go — the “stays core, registry-boundary blocker” verdict was reopened and reversed): flatCollector.collectFlat (substrate fan-out, a direct in-package call to statusCollect) / flatCollector.collectSingle / enrichOne (deploy enrichment). The former charly/status_substrate_host.go is DELETED (K-wave 2) |
commands.go |
enable/disable/logs/update/remove |
candy/plugin-pod/service_resolve.go |
service command (init system service management inside containers — the former charly/service.go is DELETED, K-wave 2) |
sdk/deploykit/data.go |
Volume data seeding (ProvisionData, SeederHelperImage) for bind-backed + named-volume targets, driven by charly config --seed/--force-seed (the former charly/data.go is DELETED, K-wave 2) |
sdk/deploykit/hooks_collect.go |
Lifecycle hooks (post_enable, pre_remove) collection and execution (CollectHooks — the former charly/hooks.go is DELETED, K-wave 2) |
candy/plugin-build/ensure.go |
Remote image ref resolution, pull-or-build (resolveRemoteImageRef — the former charly/remote_image.go is DELETED, K-wave 2) |
candy/plugin-vm/vm.go |
VM lifecycle: create, start, stop, destroy, list, console, ssh (the compiled-in command:vm plugin) |
candy/plugin-vm/vm_build.go |
VM disk image builds (qcow2, raw via bootc install — the former charly/vm_build.go is DELETED, K-wave 2) |
candy/plugin-vm/vm_libvirt.go |
Libvirt backend: VM operations via session-level libvirt |
candy/plugin-vm/vm_qemu.go |
QEMU backend: direct VM operations via qemu-system |
sdk/vmshared/smbios_credentials.go |
SSH key injection via SMBIOS/systemd credentials at VM boot |
candy/plugin-vm/libvirt.go |
Libvirt XML snippet collection and injection (InjectLibvirtXML — the former charly/libvirt.go is DELETED, K-wave 2) |
check_endpoint_resolve.go |
SHRUNK to the fixed reverse-RPC SERVICE surface (#55 W3 B7): the CheckContext interface methods (ResolveEndpoint/ResolveGraphicsEndpoint/ResolveImageLabel) every out-of-process live-container verb (cdp/wl/vnc/dbus/mcp) dials back into over CheckContextService (the Uniform API Invariant — class-generic, never a per-verb RPC), plus resolveVerbEndpointFor/resolveImageLabelFor — now THIN WIRE FORWARDS to verb:check-resolve’s OpResolveEndpoint/OpResolveImageLabel (the actual venue-classify + checkhost.EndpointForVenue / image-label-read WORK relocated to candy/plugin-check/resolve_endpoint.go, compiled-in-REQUIRED placement class) — and runEndpointCleanups, which now ALSO signals the plugin’s own OpDrainEndpointCleanups leg. resolveVerbGraphics(kind) (a VM’s <graphics type='vnc'|'spice'> via the vm plugin + any qemu+ssh:// tunnel + the vnc socket→TCP bridge + the credential-store ticket) is a DIFFERENT file, check_graphics_endpoint.go — a permanent core STAY (x/crypto/ssh containment), not renamed by B7. resolveClusterContext(cluster) no longer exists here — kube’s cluster: <profile> → kubeconfig-context resolution self-loads in candy/plugin-kube/cluster.go (an earlier W3a cutover; charly core issues no reverse-leg for it). |
candy/plugin-substrate/status_probes.go (plugin, not charly/) |
The live tool probes (cdp/vnc/supervisord/dbus/charly/wl/sway HostProbe + GuestProbe) + the devToolsTab CDP-tab decode struct, P14a — moved from core (charly/status_probes.go + charly/cdp_preresolve.go). The cdp endpoint resolution is the cc.ResolveEndpoint reverse-leg; the cdp: verb (open/list/close/text/html/url/screenshot/click/type/eval/wait/coords/raw + spa-*) + its CDP WebSocket client live out-of-process in candy/plugin-cdp (the core’s former minimal CDP client browser_cdp.go was DELETED when wl externalized, so golang.org/x/net is an INDIRECT dependency). |
vnc_helpers.go (folded into check_endpoint_resolve.go, FLOOR-SLIM Unit 4 — no longer a separate file) |
The host-side VNC support the cc.ResolveGraphicsEndpoint reverse-leg needs but the out-of-process plugin cannot reach: resolveVNCPassword (the VNC credential store), now living in check_endpoint_resolve.go (unmoved by #55 W3 B7 — it is host-fabric, not resolution work). The UNIX-socket→TCP bridge for the TCP-only RFB client is pure host-side networking with zero core state; it moved to sdk/kit/vnc_bridge.go (kit.UnixToTCPBridge, P12a follow-up) — check_graphics_endpoint.go calls it there directly (the file that also owns resolveVerbGraphics, NOT check_endpoint_resolve.go). The RFB verb (screenshot/click/type/key/mouse/status/passwd/rfb) + its RFC 6143 client live out-of-process in candy/plugin-vnc; the venue-aware dual pod/vm resolution (a pod’s 5900, or a VM’s libvirt VNC display bridged/tunneled) is resolveVerbGraphics("vnc") in check_graphics_endpoint.go. The separate charly ssh tunnel vnc relocated out of core to candy/plugin-ssh (command:ssh, an earlier cutover; there is no core ssh.go any more). |
Infrastructure
Section titled “Infrastructure”| File | Purpose |
|---|---|
sdk/enginekit + sdk/kit |
Docker/Podman abstraction — kit.ResolveRuntime / enginekit.EngineClient (the only place that touches podman/docker; the former charly/engine.go is DELETED, K-wave 2) |
candy/plugin-oci + sdk/kit |
Remote image inspection (go-containerregistry; kit.InspectImageLabels — the former charly/registry.go is DELETED, K-wave 2) |
sdk/kit |
Cross-engine image transfer (kit.TransferImage — the former charly/transfer.go is DELETED, K-wave 2) |
candy/plugin-settings/config.go |
~/.config/charly/config.yml, secret_backend key, credential maps (the former charly/runtime_config.go is DELETED, K-wave 2) |
sdk/deploykit/quadlet.go + quadlet_pod.go |
Shared “charly” container network management (the Network=charly quadlet emission — the former charly/network.go is DELETED, K-wave 2) |
candy/plugin-vm/machine.go |
Podman machine management (rootful VM builds; in the command:vm plugin) |
Configuration
Section titled “Configuration”Key types — user_policy + exclude_distros architecture:
| Type / Field | File | Purpose |
|---|---|---|
spec.ResolvedDistro.BaseUser *BaseUser |
spec/spec/cue_types_gen.go (the former charly/format_config.go is DELETED, K-wave 2) |
Pointer to a declared pre-existing uid-1000 account in the upstream base image. Nil when not declared (fedora/arch/debian); set for ubuntu ({ubuntu, 1000, 1000, /home/ubuntu}). Inherited via DistroConfig.ResolveInherits (spec/spec/distro_config_methods.go) so a child distro with no base_user: inherits the parent’s |
spec.BaseUser |
spec/spec/cue_types_gen.go |
Four required fields: Name, UID, GID, Home. Parsed from the embedded build vocabulary’s distro.<name>.base_user: |
BoxConfig.UserPolicy string |
spec/spec/cue_types_gen.go (the former charly/config.go is DELETED, K-wave 2) |
YAML field user_policy. Values: auto (default) / adopt / create. Drives the reconciliation switch in ResolveBox |
ResolvedBox.UserAdopted bool |
spec/spec/cue_types_gen.go + sdk/buildkit/resolved_box.go |
True when the policy reconciliation adopted a distro’s BaseUser (User/UID/GID/Home overwritten). Consumed by WriteBootstrap in sdk/deploykit (relocated in #67) to skip the useradd step |
Op.ExcludeDistros []string |
checkspec.go |
Per-test filter — test runner in checkrun.go:runOne skips the check when any of the image’s distro tags intersects with this list. Reason reported as excluded on distro "<tag>" |
TagPkgConfig.Raw map[string]any |
layers.go |
Captures the full YAML map for a tag section (e.g. debian:13:), not just package:. Enables repos:, keys:, options: inside tag sections. Read by the generator’s install-template emission path |
Policy reconciliation flow (sdk/buildkit/config_resolve.go:ResolveBox, after distroDef loaded):
policy := img.UserPolicyif policy == "" { policy = c.Defaults.UserPolicy }if policy == "" { policy = "auto" }baseUser := (*BaseUserDef)(nil)if resolved.DistroDef != nil { baseUser = resolved.DistroDef.BaseUser }userExplicitlySet := img.User != "" || c.Defaults.User != ""
switch policy {case "adopt": if baseUser == nil { return nil, fmt.Errorf(...) } // overwrite User/UID/GID/Home resolved.UserAdopted = truecase "auto": if baseUser != nil && !userExplicitlySet { // overwrite User/UID/GID/Home resolved.UserAdopted = true }case "create": // no-op}See /charly-image:image “user_policy” for the user-facing decision matrix, /charly-build:build “base_user:” for the declarative side, and /charly-build:generate “writeBootstrap” for the consumer side.
Existing configuration files
Section titled “Existing configuration files”| File | Purpose |
|---|---|
spec/hostenv/envfile.go + sdk/deploykit/deploy_state.go |
ENV merging, path expansion (the former charly/env.go is DELETED, K-wave 2) |
spec/hostenv/envfile.go |
.env file parsing (ParseEnvFile, ParseEnvBytes), runtime env var resolution/merging (the former charly/envfile.go is DELETED, K-wave 2) |
sdk/deploykit/security.go |
Container security config collection, CLI args generation (SecurityArgs, CollectSecurity). Merges Mounts from layer security configs (the former charly/security.go is DELETED, K-wave 2) |
sdk/deploykit/write_labels.go + sdk/deploykit/capabilities.go + sdk/kit/description_merge.go |
OCI label writing + constants (the former charly/labels.go is DELETED, K-wave 2). LabelDescription (ai.opencharly.description) carries the LabelDescriptionSet — each LabeledDescription (a Description string) plus its Plan []Step list; BoxMetadata’s *LabelDescriptionSet field is populated by ExtractMetadata when present |
init_resolve.go |
The HOST side of the init kind’s config-resolve leg — resolveInitConfigViaPlugin/invokeInitResolve (M kind-dispatch callback wrappers, colocated from the deleted service_render.go, K-wave 2 cone R2). The egress shim (ValidateEgress/ValidateEgressValue + the spec.ValidateRecord init binding) that used to share that file moved to candy/plugin-fleet/egress.go; the validation logic + CUE schemas live in the compiled-in candy/plugin-egress (M16). See /charly-internals:egress. |
sdk/deploykit/deploy_volume.go |
Named volume collection/mounting (VolumeMount, ResolveVolumeBacking — the former charly/volumes.go is DELETED, K-wave 2) |
candy/plugin-alias/ |
Command aliases (wrapper scripts) — the command:alias plugin (the former charly/alias.go is DELETED, K-wave 2) |
sdk/deploykit/deploy_state.go + deploy_volume.go |
Per-deployment config overlay, DeployVolumeConfig, ResolveVolumeBacking(), SaveDeployState(), cleanDeployEntry() (instance-aware provides cleanup; the former charly/deploy.go is DELETED, K-wave 2) |
candy/plugin-deploy-pod/provides_inject.go + sdk/deploykit/deploy_state.go |
Env/MCP provides injection, RemoveBySource(), RemoveByExactSource() (instance-specific cleanup), podAwareMCPProvides() (the former charly/provides.go is DELETED, K-wave 2) |
enc.go |
DELETED (K-wave 2) — the enc SHIM + deploy-model (C16a) relocated to candy/plugin-pod/enc_cmd.go (pluginEncMount/pluginEncUnmount/encPasswd/ensureEncryptedMounts — thin shims that HOST-PRELIFT the per-volume plan + passphrase, then encExecViaPlugin resolves verb:enc and Invokes OpExecute). Keeps ResolvedBindMount, the config loader (loadEncryptedVolume), the path/probe helpers (encryptedPlainDir/isEncryptedMounted/isEncryptedInitialized/cipherPopulatedPlainEmpty — consumed synchronously by the mandatorily-core ResolveVolumeBacking + verifyBindMounts), encStatus (pure probe+print), and the credential passphrase resolution (resolveEncPassphrase*/awaitKeyringUnlockViaPlugin). The gocryptfs / systemd-run --scope --unit=charly-enc-<dir>-<volume> / fusermount3 / extpass SHELLING lives in candy/plugin-enc, NOT core (-allow_other for rootless keep-id, stale-scope retry, all there). The encMount all-mounted fast-path (skip passphrase when every volume is already mounted) stays in the shim |
devices.go |
DELETED (K-wave 2 cone R3 — the B6 per-leg death list). LogDetectedDevices (the pure stderr formatter) relocated to candy/plugin-deploy-pod/detect_devices.go; the earlier partial dissolution had already removed appendEnvUnique/appendAutoDetectedEnv/appendGroupsForAMDGPU (dead — the plugin carries its own copies), the 3 embedded data tables (to candy/plugin-gpu’s own embed), and deviceDescriptions (to candy/plugin-doctor’s own embed). GPU detection now runs wholly plugin-side: charly config/start/shell/update-all reach verb:gpu peer-to-peer from candy/plugin-deploy-pod (detect_devices.go), the same dispatch charly doctor/charly vm gpu/the arbiter already use. |
gpu_shim.go |
THINNED to the operator-dropped GPU-host-DETECTION exception leg (K-wave 2 cone R3, 80→30 LOC — the same family as the gpu_allocate.go EXCEPTION-GPU row): only gpuProbeReply + DetectVFIO survive, serving gpu_allocate.go’s bedGPUPrereqMissing via the host_build_check_bed_gpu_prereq.go seam. The DetectHostDevices/EnsureCDI shims + LogDetectedDevices relocated to candy/plugin-deploy-pod/detect_devices.go as peer InvokeProvider(verb:gpu) dispatches (the deleted pod-config-detect-devices seam). The earlier C11/K5 dissolution had already moved the detection primitives + data tables into candy/plugin-gpu and deleted the alias re-exports. The DRIVER-SWITCH has NO in-core shim — every consumer dispatches verb:gpu directly. |
preempt.go — DELETED (K-wave 2 cone CONTESTED). |
The HOST side of the resource arbiter after cutover C9: the arbiter LOGIC (ResourceArbiter) moved into the COMPILED-IN candy/plugin-preempt (verb:arbiter). K-wave 2 cone CONTESTED then deleted the last core survivors: the in-core PROXY + acquire shims (newResourceArbiter/arbiterProxy/arbiterInvoke/Lease/Release/ReleaseFailed/acquireResourceForClaimant/acquireExclusiveForClaimant/acquireSharedForClaimant/isPodMember/acquireDispatch/envPreemptLeaseHeld) had ZERO production callers — every former consumer went peer-dispatch (candy/plugin-check’s bed_session, candy/plugin-vm’s vm_arbiter_shim, candy/plugin-fleet’s handleLifecycleSimple Invoke verb:arbiter directly). The two live functions folded into their sole callers: gatherResources → charly/gpu_allocate.go (the plan’s EXCEPTION-GPU fold, for bedGPUPrereqMissing) and releaseResourceClaim + its proxy chain → charly/host_build_pod_lifecycle_dispatch.go (the op=“remove” arbiter-release bracket — host-process CHARLY_PREEMPT_LEASE state a placement-agnostic plugin cannot own; candy/plugin-pod ships out-of-process). Persisted + seam wire types are CUE-sourced at spec/schema/arbiter.cue; the ledger I/O + poison + liveness + mode-math live IN the plugin (arbiter.go/arbiter_support.go) |
arbiter_host.go — DELETED (FLOOR-SLIM-proper Unit-8, the host-seam impls moved directly into candy/plugin-preempt’s holder_dispatch.go; the C9 ExecutorService.HostArbiter reverse channel + arbiterHostServer are gone). |
(row kept for history; see candy/plugin-preempt/holder_dispatch.go for the successor) |
sdk/deploykit/tunnel_resolve.go |
The RESOLUTION half of the tunnel subsystem (the former charly/tunnel.go is DELETED, K-wave 2): the wire types TunnelConfig/TunnelPort, the pure helpers schemeTarget/tailscaleFlag/isTCPFamily/ValidPublicPorts (shared with the quadlet emitter), the config-path helpers tunnelConfigDir/tunnelConfigPath, and the resolution TunnelConfigFromMetadata/parseHostPorts/buildPortMapping/resolveProto. The EXECUTION leg (tailscale serve/funnel + the cloudflared lifecycle) externalized to candy/plugin-tunnel |
candy/plugin-tunnel/ |
The EXTERNALIZED tunnel execution leg (C16b — the former charly/tunnel_plugin.go core adapter is DELETED, K-wave 2): verb:tunnel’s {method, config} envelope, driven directly by the pod-lifecycle plugins (candy/plugin-deploy-pod for start/stop, candy/plugin-pod for remove) via InvokeProvider; tunnel_exec.go runs the actual tailscale serve/funnel + cloudflared lifecycle, stopping at the exec/auth boundary. verb:tunnel also serves a creds-free plan dry-run returning the argv it WOULD run (box/fedora’s check-tunnel-pod bed R10) |
sdk/deploykit/quadlet.go + quadlet_pod.go |
Quadlet .container file generation, Secret= directives (the former charly/quadlet.go is DELETED, K-wave 2) |
credential_plugin.go |
The CORE adapter for the EXTERNALIZED credential store (C2 dep-shed removed go-keyring; the godbus dep-shed removed godbus too — charly/go.mod links neither): CredServiceVNC, ResolveCredential(), DefaultCredentialStore() (→ the credentialResolver interface), and pluginCredentialStore.call/callCtx/resolve — the VNC-password resolve path check_endpoint_resolve.go’s resolveVNCPassword needs. K-wave 2 cone CONTESTED THINned 213→109: the CredentialStore interface, Get/Set/Delete/List/Name, awaitUnlock + the credentialAwaiter seam (the keyring-unlock wait lives plugin-side in candy/plugin-pod’s pluginAwaitKeyringUnlock), resolveSecretBackend, and credentialHealth are ALL DELETED (zero production callers; credentialHealth was already gone at K5 — candy/plugin-doctor peer-InvokeProviders verb:credential’s health). Every method forwards to verb:credential (served out-of-process by candy/plugin-secrets, or the baked /usr/lib/charly/plugins binary). The store backends + the keyring-unlock waiter + the charly secrets CLI + GPG .secrets surface live in candy/plugin-secrets/ now, NOT core. Generic host-adapter seam (F7/C7): callCtx connects via connectPluginByWord(ClassVerb, "credential") — the ONE on-demand connect for a verb word that appears in NO plan step. vm_plugin_client.go (invokeVmPluginEnv → verb:libvirt) and k8s_plugin.go (invokeKubePluginWithBroker → verb:kube) now route through the SAME seam (connectPluginByWordRef adds an optional canonical-ref fallback for a project whose closure references the plugin candy nowhere, e.g. a box/<distro> VM bed) — the bespoke ensureVmPluginConnected sync.Once + kube’s bare ResolveVerb were deleted (R3) |
sdk/deploykit/secret_provision.go |
Container secret collection from labels, Podman secret provisioning, SecretArgs(), GenerateAndStoreSecret, the interactive promptPassword (a deploy-time operator prompt) — the former charly/secrets.go is DELETED, K-wave 2 |
The encrypted-volume surface is no longer part of this package: its probe/plan half is
sdk/deploykit/enc_probe.go, its passphrase-resolution half sdk/deploykit/enc_passphrase.go,
the charly config status|mount|unmount|passwd leaf bodies candy/plugin-pod/enc_cmd.go, and
the gocryptfs shelling candy/plugin-enc/enc.go. See /charly-automation:enc “Source files”.
Remote Layer Refs
Section titled “Remote Layer Refs”| File | Purpose |
|---|---|
sdk/loaderkit’s refs seams (refs_collect.go, refs_seams_executor.go, canonical_ref.go) |
Remote ref types, parsing, cache management (the former charly/refs.go is DELETED, K-wave 2). CHARLY_REPO_OVERRIDE (proc.RepoOverrideEnv) Go-replace-style local-tree override (repoOverrideDir). proc.SelfSuperprojectOverridePair(dir) / proc.MergeRepoOverrides relocated to spec/proc (#55 W3 B2-full — pure git-shelling + string manipulation, needed by BOTH plugin_loader.go’s deployNodePluginContext AND candy/plugin-check/bed_session.go’s bedSetup, which auto-applies it plugin-side so a box/<distro> bed tests LOCAL parent-repo candies, never the pinned remote — the candy-ref analogue of auto --dev-local-pkg). Tests: spec/proc/repo_override_test.go. |
spec/refs/git.go + sdk/kit/refs_git.go |
Git operations: clone, resolve ref, tag resolution (GitLatestTag/RepoGitURL/CompareSemver/GitDefaultBranch/DownloadRepo — the former charly/refs_git.go is DELETED, K-wave 2) |
Declarative Testing
Section titled “Declarative Testing”Implements the check gathering, validation, and host seams behind charly check live / charly check box (whose CLI now lives in the command:check plugin
candy/plugin-check) and the ai.opencharly.description OCI label. User-facing
authoring, verb catalog,
runtime variables, and charly.yml overlay rules live in /charly-check:check — this
section is the Go-implementation map.
| File | Purpose |
|---|---|
checkspec.go |
opActsInBuildDeploy — the registry-coupled act-capability predicate: consults providerRegistry.ResolveVerb/ResolveStep + the ProvisionActor/TypedStepProvider/BuildEmitter capability interfaces (and the prescan-declared external tables) to answer whether a plugin: op’s act form has a real build/deploy install path. Sole production caller: validate_project_host.go’s “validate-word-sets” leg (the validate plugin’s envelope-derived word inventory). K-wave 2 cone R4 thinned 107 → 49: the do-mode/context grammar (opEffectiveContexts/opInContext) moved to planrun_adapter.go, the vocabulary DATA (VerbCatalog/VerbSpec/DoMode) lives in spec/spec/verb_context.go, and the ${NAME[:arg]} variable grammar lives ONCE in sdk/kit/checkvars_expand.go (no charly-side re-export — ZERO-ALIASES; no bash-style defaults, ${VAR:-fallback} unsupported). The Op type is spec.Op (Op.Kind() enforces exactly-one verb); Matcher/MatcherList are spec/spec/union_types.go defs. |
sdk/kit/checkvars.go (sdk, not charly/, P12a) |
ContainerInspection + siblings, CheckVarResolver, ResolveCheckVarsBuild / ResolveCheckVarsRuntime — moved from core (charly/checkvars.go): every dependency was already portable (spec.BoxMetadata, vmshared.FleetNode, a plain {engine} inspect shell-out). IsHostNetworked() is a method on kit.ContainerInspection; ContainerInspection (renamed from the former package-level swappable InspectContainer var) maps podman inspect output into HOST_PORT:<N>, VOLUME_PATH:<name>, VOLUME_CONTAINER_PATH:<name>, CONTAINER_IP, CONTAINER_NAME, ENV_<NAME>. Package main reaches none of these through a charly-side re-export — the ZERO-ALIASES gate forbids one. |
checkrun.go |
The check-verdict result constructors (passf/failf/skipf over spec.CheckResult) + the committed-APK anchoring data (candyDirsFromScan / the checkRunnerContext carrier, relocated here from check_cmd.go in K-wave 2 cone R4 — read by checkrun_charly_verbs.go’s resolveCheckApk via h.cc.CandyDirs()/CandyScanErr()). Spec-only glue, zero sdk/kit import. The check-engine driver itself is kit.Runner (sdk/kit); the host-coupled dispatch surface (hostVerbResolver/hostCheckCarrier/hostPlanGrammar) stays core in planrun_adapter.go, produced for the check reverse channel by plugin_dispatch_reverse.go from the wire spec.CheckEnv snapshot. The IN-PROC plan-drive construction (the former newCheckRunner/carrierFromRunner/resolverEnv) is GONE from production: the deploy-scope check DRIVE moved PLUGIN-SIDE (command:check OpVerifyChecks, #55 CHECK-ENGINE cone Unit 2 — candy/plugin-check’s newPluginCheckRunner), so core no longer builds a kit.Runner itself; the former constructor survives only as a test helper (checkrun_helpers_test.go). |
checkrun_verbs.go |
Dispatch for the remaining verbs: package (rpm/dpkg/pacman), service (supervisorctl + systemctl), process (pgrep), dns (host-side net.LookupIP or in-container getent), user/group (getent passwd/group), interface (ip -o addr show + MTU), kernel-param (sysctl -n), mount (findmnt), addr (host-side net.DialTimeout or in-container nc -z), matching (pure in-process value matching). resolvePackageName(c, distros) implements the distro-aware package-map: when Check.PackageMap is non-empty, the first entry in Runner.Distros that matches a key wins; otherwise Check.Package is used as-is. Covered by TestResolvePackageName (6 sub-cases including empty-map fallback, first-matching-tag-wins priority, and empty-string-map-value fall-through). Runner.Distros is populated from meta.Distro wherever a Runner is built — the check_cmd.go live-gather engine and the command:check plugin’s box/feature/harness runners. |
fleet_members.go |
DELETED (#55 W3 B2-full) — sibling-member bring-up/tear-down relocated entirely to sdk/deploykit.BringUpMembers/TearDownMembers (#55 W3 A4’s R3 3-way audit: the venue-classification predicate, not the whole engine, was the actual duplication risk — resolved via the promoted spec.IsVmVenue/IsContainerVenue, mirroring spec.HostRooted). candy/plugin-fleet/walk.go (operator path) and candy/plugin-check/bed_run.go (bed path) both call deploykit.BringUpMembers/TearDownMembers directly now — no HostBuild seam either way. foldMembers/validateMembers (the LOAD-half) already lived in sdk/loaderkit. Members are excluded from bedCheckLiveRefs (instruments, never check-live’d). |
checkrun_charly_verbs.go |
Shared host-side helpers for the EXTERNAL live-container verbs: resolveCheckApk (anchors an adb/appium apk: fixture to its authoring candy’s source tree, host-side, before the marshalled Op crosses to the plugin) + the noVmDisplayDeviceErr skip sentinel (the spice/vnc VM-display N/A skip). Every live-container verb (wl/cdp/vnc/dbus/kube/adb/appium/spice/mcp/record/libvirt) is an EXTERNAL out-of-process plugin dispatched via invokeVerbProvider with the full Op (wl/record/dbus are EXEC-based: the provider drives the venue over the DeployExecutor reverse channel; cdp/vnc dial a host-pre-resolved endpoint). The EXEC-based verbs’ (wl/record/dbus) shared boilerplate lives in the SDK/kit, ONE copy each (R3): sdk.RunArtifactValidators (post-run artifact validators) + sdk.MatchAll (the matcher pipeline) + sdk.ResultJSON (the {status,message} reply) + sdk.CheckRequiredModifiers (the required-modifier check) + the *sdk.Executor venue methods VenueCapture/VenueHasTool/VenueRunSilent, with kit.ShellQuote/kit.TrimPreview the pure quoter/preview; each plugin keeps only its per-verb requiredModifiers map + modifierZero. The nested-CLI argv contract a plugin imports (kit.MethodSpec + the kit.Pos* builders) lives in sdk/kit/methodspec.go. |
candy/plugin-mcp (mcp resolution) |
The mcp: verb resolves its OWN context via the reverse-legs (resolve.go): cc.ResolveImageLabel("ai.opencharly.mcp_provide") for the declared servers, then {{.ContainerName}} substitution + spec.PodAwareMCPProvides localhost rewrite + pick + cc.ResolveEndpoint to map the container port → host address. The MCP CLIENT (the go-sdk dial + the MCP methods ping/servers/list-tools/list-resources/list-prompts/call/read) lives here too, beside the SERVER (serve.go, command:mcp); charly’s core links NO MCP SDK — its host half is the charly __cli-model seam (cli_model_cmd.go). |
cli_model_cmd.go |
charly __cli-model (hidden machinery) — emits charly’s ASSEMBLED Kong command tree (the core CLI struct + the builtin command-provider grammar) as an sdk.CLIModel JSON document on stdout: the host half of the EXTERNALIZED MCP server. candy/plugin-mcp (command:mcp — the externalized charly mcp … CLI; serve.go) fork/execs it at startup (fetchCLIModel, deliberately with NO project prefix — the model needs no project), registers one MCP tool per model leaf (cliLeafToTool/argToSchema, additionalProperties: false), annotates/filters mutating tools via its mcpDestructivePaths allowlist (--read-only skips registering them), and executes each tool call as a charly <path> <args…> SUBPROCESS (makeToolHandler → argvFromJSON → forkCharly) carrying a managed project prefix (computeProjectPrefix: charly.yml in cwd → none; --no-default-repo → none, project tools error at call time; else --repo default) with childCharlyEnv stripping CHARLY_PROJECT_DIR/CHARLY_PROJECT_REPO so the prefix stays authoritative. Core links NO MCP SDK — the fork/exec design replaced the former in-process server wholesale. Test coverage: cli_model_cmd_test.go. Full reference: /charly-build:charly-mcp-cmd. |
main_dir_test.go |
Integration tests for the -C / --dir / CHARLY_PROJECT_DIR global: spawns a freshly-compiled charly binary from /tmp with a scratch project, verifies all three flag forms make charly box list boxes resolve the scratch charly.yml. Error cases: missing dir, file-not-dir. |
sdk/kit/local_image.go (sdk, not charly/, P12a) |
ResolveLocalImageRef(engine, input) (renamed from resolveLocalImageRef) — moved from core (charly/local_image.go); test-mode-only image resolution that never reads charly.yml. Full refs pass through with a LocalImageExists check; short names match against ListLocalImages() output using label-preferred matching (ai.opencharly.box=<name>) with a repo-name trailing-component fallback. Returns ErrImageNotLocal on no-match so FormatCLIError renders the “charly box pull / charly box build” recommendation. Used to keep charly check box purely OCI-labels-driven — short names resolve against local podman storage, never charly.yml. Also carries ListLocalImages, LocalImageInfo, ParseLocalImagesJSON / ExtractCalVerTag (exported — candy/plugin-clean/retention.go calls them now, since the retention engine relocated there from core in the K1-alpha core-minimization wave), ResolveNewestLocalCalVer, LooksLikeFullRef (4 other core call sites repointed to kit.LooksLikeFullRef, R3 single-source). |
description_collect.go |
CollectDescriptions(cfg, layers, imageName) *LabelDescriptionSet walks the base-image chain — mirror of CollectHooks in hooks.go:18-68 — with a visited-image guard so pathological cycles reported by validateBoxDAG can’t hang the collector. Bucketizes plan steps into candy/box/deploy by source + context, stamps Origin for reporting. MergeDeployDescriptions(baked, local) implements id-based replace, append, and {id: X, skip: true} disable semantics. |
check_cmd.go |
resolveCheckRunnerContext — the check-scoped plugin-context helper (feeds the “check-load-plugins” seam, scans candies + calls loadProjectPlugins into this process’s registry) — the SOLE remaining content (108 LOC after K-wave 2 cone R4 moved candyDirsFromScan + the checkRunnerContext carrier to checkrun.go). checkLocalDeployScope / runLocalDeployScopePlan (the external target: local deploy’s own --verify path) RELOCATED to candy/plugin-fleet/verify_local.go (#55 W3 B3, verifyLocalDeployScope/localDeployScopePlan — a peer plugin calling command:check via direct InvokeProvider now, no host round-trip). resolveMergedDeployTree / deployNodePluginContext / resolveDeployNodeByPath relocated to plugin_loader.go (#55 W3 B3) — deployNodePluginContext‘s real significance is as loadDeployPlugins’ direct input (plugin-loader infrastructure, clause M), not a check-only concern; check_cmd.go’s resolveCheckRunnerContext still calls it (same package, different file). The former CLI-free live-check GATHER engine (checkLiveGather / checkLiveVM / checkLivePod / checkLiveLocal / checkLiveGroup + the CheckLiveCmd struct) RELOCATED to the compiled-in command:check plugin candy/plugin-check (live_gather.go / run_box.go / score*.go), which drives its OWN kit.Runner via newPluginCheckRunner; the “check-run” HostBuild kind is DELETED entirely (K-wave 2 cone R4 — every mode, incl. the last one “preflight”, now dispatches plugin-side). The charly check CLI — its command tree, the box/live/run/feature verbs, the check-run management subcommands, and the charly check box disposable-container flow — lives in that same candy/plugin-check plugin. |
plugin_loader.go |
Also carries resolveMergedDeployTree / deployNodePluginContext / resolveDeployNodeByPath (#55 W3 B3, relocated from check_cmd.go) beside loadDeployPlugins — the deploy-node tree read + plugin-word collection feeding the plugin-loading M-mechanism (loadProjectPlugins/providerRegistry), consumed by check_venue_resolve.go and deploy_tree.go’s header too. |
check_runner_cmd.go |
DELETED — the check-run management Cmds + orchestrator preflight (runWithPhaseResync) moved to the command:check plugin candy/plugin-check; the former scorePodTargetEntry is gone (zero production callers). |
check_runner_live.go |
DELETED — RunCheckLive (the “score” mode body) is now candy/plugin-check/score_live.go’s pluginCheckRunScore/pluginRunCheckLive family. |
check_image_preflight.go |
DELETED — the “preflight” mode body is now candy/plugin-check/preflight_images.go (preflightImageCandidates) + command.go’s pluginCheckRunPreflight. |
host_build_check_run.go |
DELETED (K-wave 2 cone R4) — the “check-run” HostBuild kind is gone; every spec.CheckRunRequest{mode} (box / live / feature-box / feature-live / score / preflight) now dispatches to candy/plugin-check’s OWN bodies (hostCheckRunCtx in command.go). |
host_build_check_bed.go |
DELETED (#55 W3 B2-full) — the “check-bed” host-session seam dissolved entirely. Every piece it held turned out reachable without a host round-trip: flock via spec/lock (plugin-importable), the preempt lease via a direct InvokeProvider(verb,"arbiter") call (the vm_arbiter_shim.go precedent), the repo-override/deploy-config env vars via plain os.Setenv (candy/plugin-check is COMPILED-IN, so this lands in the same process hostBuildCli’s cli-reentry children fork from — the ONE genuine placement constraint, documented as a compiled-in-REQUIRED class in the new home’s header). See candy/plugin-check/bed_session.go (bedSetup/bedTeardown, constructing spec.CheckBedReply directly as a plain in-process value, never a wire reply anymore). |
host_build_check_bed_gpu_prereq.go |
The ONE narrow seam surviving check-bed’s dissolution: GPU host-DETECTION (gpu_allocate.go’s bedGPUPrereqMissing/DetectVFIO) is the project’s explicitly operator-dropped exception (no hardware to verify a relocation against). B6 LANDED (K-wave 2 cone R3): the OTHER gpu_shim.go legs (DetectHostDevices/EnsureCDI) relocated to candy/plugin-deploy-pod, but this seam + DetectVFIO stay untouched — it threads the claimant’s resource tokens out and the GPU-unsatisfiable verdict back, so the fenced core logic runs UNCHANGED. |
host_build_check_config.go |
The “check-config” projection seam — the check-project reads the harness makes (bed-vs-iterate classify, sandbox class, pod-target disposability, resolved iterate config, include-expanded plan, kind:agent catalog) that a plugin (a separate module) cannot LoadUnified for. Transitional (K1). |
check_bed_run.go |
DELETED (#55 W3 B2-full) — bedCheckLevel/bedExternalInPlace (its last two functions) relocated to candy/plugin-check/bed_session.go, ported unchanged (bedCheckLevel, a plain *spec.UnifiedFile method call, needed no core coupling) or replaced by the promoted spec.ExternalInPlaceVenue predicate (bedExternalInPlace’s registry query was refuted as an incomplete-seam trap — every bed-root node is already Descent-stamped). bedVmDomains/acquireVmDomainLock had already dissolved into spec/lock earlier. |
Related skill: /charly-check:check is the authoring-facing reference.