ollama-layer

Recipe card from the charly-ollama plugin (Images — the deployable catalog).

ollama – Local LLM inference server

Candy Properties

Property	Value
Dependencies	supervisord
Ports	11434
Volumes	models -> ~/.ollama
Aliases	ollama -> ollama
Service	ollama (supervisord)
Install	the packaged ollama via distro: where the distro ships one (Arch and its derivatives), else a download: + extract: plan step gated on unless_exists:

Environment Variables

Variable	Value
OLLAMA_HOST	0.0.0.0
OLLAMA_MODELS	~/.ollama/models

Build-time Variables (var:)

Variable	Default	Purpose
OLLAMA_VERSION	v0.32.14	Upstream release for the TARBALL path only, and it must be a release TAG — the download cache is content-addressed by the sha256 of the URL, so a latest URL would hash to one key forever and keep serving the first tarball it ever fetched. Ignored on Arch, where the packaged ollama is installed and pacman owns the version

GPU backends (opt-in, box level)

On a distro that PACKAGES ollama the candy itself is CPU-only, and a box adds a backend by composing one of the sibling candies — which is what keeps a CPU image small. On a TARBALL distro the single upstream archive already contains the CUDA backend, so there is nothing to opt out of there; ROCm is never in it:

Candy	Adds	Installed size
ollama-cuda	NVIDIA CUDA backend (libggml-cuda.so)	~988 MiB
ollama-rocm	AMD ROCm/HIP backend (libggml-hip.so)	~2.9 GiB

On Arch these are the split ollama-cuda / ollama-rocm packages, which depend on ollama and drop their backend into the same /usr/lib/ollama/ runner directory rather than replacing anything.

Service Environment (injected into other containers)

Variable	Template Value	Resolved Example
OLLAMA_HOST	http://{{.ContainerName}}:11434	http://charly-ollama:11434

Pod-aware: same-container consumers receive http://localhost:11434, cross-container consumers receive http://charly-ollama:11434. When charly config ollama runs, OLLAMA_HOST is automatically injected into the global charly.yml env. Use charly config ollama --update-all to propagate to already-deployed services immediately.

See /charly-image:layer for env_provide field docs and /charly-core:charly-config for --update-all.

Usage

charly.yml

ollama:
  candy:
    - ollama

charly alias install ollama    # install host 'ollama' command
ollama run llama3          # uses the alias

Used In Boxes

• /charly-ollama:ollama

Cross-Container Integration

The env_provide mechanism makes OLLAMA_HOST available to all containers. The hermes candy auto-detects this variable and configures itself to use local Ollama as its LLM provider (highest priority in the auto-detection chain: OLLAMA_HOST > OLLAMA_API_KEY > OPENROUTER_API_KEY). See /charly-hermes:hermes for details on the auto-provider-configuration.

Tests

The candy ships its acceptance steps in its plan:, baked into the ai.opencharly.description OCI label (see /charly-check:check for the full schema). Each step is one inline Op — a probe is a check: step — and its context: list gates where it runs:

• ungated (no context:, so it runs wherever the plan runs):

  • /usr/bin/ollama exists

• context: [runtime] (run against a live service; uses 127.0.0.1:${HOST_PORT:11434} — the host-side form, not ${CONTAINER_IP}, so port remapping works unchanged):

  • GET http://127.0.0.1:${HOST_PORT:11434}/api/tags returns 200
  • ollama --version stdout matches ^ollama version
  • ollama list exits 0 against the live service
  • three HOST-side steps (in_container: false) driving ${CHARLY_BIN}, which prove the companion command:ollama plugin end to end: charly ollama help dispatches, charly ollama version round-trips the live API, and charly ollama list reads the model table. These three DO carry id: keys (ollama-cli-help, ollama-cli-version, ollama-cli-list); the steps above them do not, and are named here by what they assert rather than by an identifier the source does not define. They live on the candy rather than on any one bed, so every box composing ollama inherits them.

  The plan also ends with an agent-check: step covering model persistence across a service restart.

A composed backend candy adds its own build-scope probe — ollama-cuda asserts libggml-cuda.so and ollama-rocm asserts libggml-hip.so, each globbed under /usr/lib/ollama/*/ so a CUDA/ROCm version bump does not break the check.

Related Candies

• /charly-distros:cuda – CUDA toolkit; composed OUTSIDE this candy (today by the GPU boxes, at the image level) and never declared as its dependency
• /charly-infrastructure:supervisord – process manager dependency
• /charly-ollama:ollama-cli – the compiled-in charly ollama management CLI (candy/plugin-ollama)
• /charly-openclaw:openclaw – AI gateway that can use Ollama as backend
• /charly-hermes:hermes – AI agent that auto-detects OLLAMA_HOST for local Ollama provider

Related Commands

• /charly-core:charly-config — Deploy with quadlet (secrets, volumes, env_provide injection)
• /charly-core:start — Start the Ollama service
• /charly-core:service — Manage Ollama service inside container

When to Use This Skill

Use when the user asks about:

• Ollama setup or model management
• LLM serving or inference
• Port 11434 configuration
• Ollama model storage volume
• The ollama host alias