Skip to content
OpenCharly

notebook-llm-on-supercomputers

Recipe card from the charly-jupyter plugin (Images — the deployable catalog).

notebook-llm-on-supercomputers – LLM course data candy

Section titled “notebook-llm-on-supercomputers – LLM course data candy”

Candy Properties

Section titled “Candy Properties”
Property Value
Dependencies (none)
Packages (none)
Services (none)
Volumes workspace -> /workspace (from jupyter)
Data data/llms_on_supercomputers -> workspace volume, dest: llms_on_supercomputers
Install files (none)

How It Works

Section titled “How It Works”

This is a data candy — it uses the data: field in charly.yml to map a directory of notebooks to a named volume with a subdirectory destination:

info: "LLMs on Supercomputers course notebooks (TU Wien AI Factory Austria)"


data:
  - src: data/llms_on_supercomputers
    volume: workspace
    dest: llms_on_supercomputers

At build time, the contents are staged into /data/workspace/llms_on_supercomputers/ inside the box. At deploy time, charly config or charly update provisions them into the workspace volume.

Included Notebooks

Section titled “Included Notebooks”

15 Jupyter notebooks organized in 4 categories:

D0 — Setup (1 notebook)

Section titled “D0 — Setup (1 notebook)”
Notebook Topic
D0_00_Bazzite_Setup.ipynb Environment setup, GPU verification, Ollama connectivity

D1 — Prompt Engineering (6 notebooks)

Section titled “D1 — Prompt Engineering (6 notebooks)”
Notebook Topic Libraries
D1_01_Prompting_with_LangChain.ipynb LangChain basics, local + Ollama models LangChain, OpenAI, HuggingFace
D1_02_Prompt_templates_and_parsing.ipynb Prompt templates, few-shot, structured output LangChain, OpenAI
D1_05_Chaining.ipynb Chain composition and routing LangChain
D1_08_LLM_Evaluation.ipynb LLM evaluation with evidently.ai ollama, evidently
D1_09_LLM_as_a_Judge.ipynb LLM-as-a-Judge evaluation pattern OpenAI, evidently
D1_10_Prompt_Optimization.ipynb Prompt optimization techniques OpenAI, evidently

D2 — Retricheck Augmented Generation (2 notebooks)

Section titled “D2 — Retricheck Augmented Generation (2 notebooks)”
Notebook Topic Libraries
D2_01_rag_with_basic_tools.ipynb RAG with pandas DataFrame as vector store OpenAI, numpy, pandas
D2_02_rag_with_langchain_and_chromadb.ipynb RAG pipeline with ChromaDB LangChain, ChromaDB, OllamaEmbeddings

D3 — Fine-tuning on One GPU (6 notebooks)

Section titled “D3 — Fine-tuning on One GPU (6 notebooks)”
Notebook Topic Libraries
D3_01_Transformer_Architecture.ipynb Transformer theory (no LLM calls) PyTorch
D3_02_Finetuning_LLM_with_PyTorch.ipynb Manual fine-tuning loop PyTorch, transformers
D3_03_Finetuning_LLM_with_Huggingface.ipynb Fine-tuning with HF Trainer transformers, datasets
D3_04_Quantization.ipynb Model quantization techniques bitsandbytes, transformers
D3_05_PEFT.ipynb Parameter-Efficient Fine-Tuning (LoRA) peft, transformers
D3_06_Unsloth.ipynb Fine-tuning with Unsloth unsloth

Supporting Files

Section titled “Supporting Files”
File Purpose
notebooks.yaml Structured catalog with subcategories
LICENSE Apache 2.0 (TU Wien AI Factory Austria)
simplified_output.json Sample data for D1_02 prompt templates
datasets/booking_queries_dataset.csv Booking classification data
datasets/code_review_dataset.csv Code review data
datasets/health_and_fitness_qna.csv Health Q&A data

Network Connectivity

Section titled “Network Connectivity”

9 notebooks (D0, D1, D2 series) connect to Ollama via the OLLAMA_HOST environment variable (default: http://localhost:11434). When Ollama is deployed via charly config ollama --update-all, the OLLAMA_HOST env var is auto-injected via env_provide to http://charly-ollama:11434 — both containers must be on the same charly Podman network. The D3 fine-tuning notebooks work locally with GPU — no Ollama needed.

Notebook Compatibility Notes

Section titled “Notebook Compatibility Notes”

ollama Python library

Section titled “ollama Python library”

The same importlib.reload(ollama) pattern from /charly-jupyter:notebook-ollama is applied to all cleanup cells that use import ollama for model unloading.

External services

Section titled “External services”
  • D1_08, D1_09, D1_10 reference https://app.evidently.cloud for LLM evaluation dashboards (optional, notebooks work without it)

Default models

Section titled “Default models”
  • D1 series: hf.co/NousResearch/Nous-Hermes-2-Mistral-7B-DPO-GGUF:Q4_K_M
  • D2 series: llama3.2:latest

Used In Boxes

Section titled “Used In Boxes”
Section titled “Related Skills”

When to Use This Skill

Section titled “When to Use This Skill”

Use when the user asks about:

  • The notebook-llm-on-supercomputers candy or its contents
  • LangChain prompt engineering tutorials
  • RAG examples with ChromaDB
  • The TU Wien LLM course notebooks
  • Fine-tuning notebooks (PyTorch, HuggingFace, PEFT, Unsloth)
Section titled “Related”
  • /charly-check:check — declarative testing (check: block, charly check box, charly check live)