hermes.llm
Models
Systems receive an LLM client, so experiments choose the model. Completions can be cached on disk, so a rerun replays the same outputs.
Clients
from hermes import llm
client = llm.create(provider="ollama", model="llama3.1", cache=".cache/llm")
text = client.complete(prompt, system="...")Every client has complete(prompt, system), model, and provider. llm.create builds one from a config's llm section; llm.from_env() builds the default from environment variables. Temperature is 0. Source: llm.py.
Ollama
| Variable | Default |
|---|---|
HERMES_OLLAMA_BASE_URL | http://127.0.0.1:11434 |
HERMES_OLLAMA_MODEL | chosen from installed models: llama3.1, llama3, mistral, phi3, qwen2.5, qwen, then the first installed |
The timeout is 180 seconds. Connection failures, timeouts, and missing models raise LookupError with the command to fix them.
OpenAI-compatible
| Variable | Default |
|---|---|
HERMES_LLM_PROVIDER | set to openai to select this client |
HERMES_LLM_API_KEY | required |
HERMES_LLM_BASE_URL | https://api.together.xyz/v1 |
HERMES_LLM_MODEL | meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo |
The timeout is 60 seconds.
Cache and retries
CachedLLM stores each completion as a JSON file keyed by a hash of the provider, model, system prompt, and prompt, and counts calls and cache hits for run.json. Both clients retry up to three attempts on read errors and on HTTP 429, 502, 503, and 504, with backoff.