How to Set Up a Hermes Agent Server in Proxmox

Self-hosted AI agents are quickly becoming a common addition to homelabs, and Hermes Agent has emerged as one of the more popular options for running your own private, model-agnostic AI gateway. Deploying a Hermes Agent server in Proxmox gives you an isolated, easily backed-up environment for running the agent, separate from your other services, with full control over which AI model actually powers it.

What Is Hermes Agent?

Hermes Agent is a self-hosted AI agent platform built around a CLI and terminal UI, offering persistent memory, skill-based extensibility, and an OpenAI-compatible API gateway. That last part matters most for homelab integration: because Hermes exposes an endpoint that mimics OpenAI’s API format, existing tools and scripts built to talk to OpenAI can be pointed at your Hermes instance instead, with no client-side changes required.

Why Run Hermes in Proxmox Rather Than Directly on Hardware

Running Hermes inside a dedicated VM or LXC container on Proxmox keeps the agent’s environment cleanly isolated from your host and other services. This makes it easy to snapshot before testing changes, restore quickly if something goes wrong, and back it up through your existing Proxmox or Proxmox Backup Server routine, the same way you would any other VM.

Choosing Between an LXC Container and a Full VM

Hermes can run in either a Proxmox LXC container or a full virtual machine. LXC is the lighter-weight option and works well for most homelab deployments, though it requires enabling nesting in the container’s options, since Hermes manages its own subprocesses internally. A full VM avoids this consideration entirely at the cost of slightly higher resource overhead.

Installing Hermes Agent

Community-maintained installation scripts are commonly used to streamline setup on a fresh Proxmox LXC or VM, handling the underlying runtime dependencies (Python and Node.js environments) automatically before installing Hermes itself. After installation, running the built-in setup walks you through configuring your model provider and gateway settings.

Choosing a Model Provider

During setup, Hermes asks which model provider to route requests through. Options typically include:

• A dedicated API key from a single provider (Anthropic, OpenAI, OpenRouter, etc.)
• A local model endpoint – pointing Hermes at a self-hosted inference server like Ollama running elsewhere on your network
• A managed multi-model subscription service – for accessing many different models through one account without juggling separate API keys

For homelab testing, pointing Hermes at a local Ollama instance running on a separate GPU-equipped machine is a common, cost-free starting point before committing to a paid cloud model.

Where Configuration Lives

Hermes stores its configuration, including whichever API key or endpoint you’ve set, in a dotfile under the service account’s home directory. Keeping track of this path matters, since the same file is also read by Hermes’s built-in API server — useful to know if you ever need to manually adjust settings or troubleshoot connectivity to your chosen model provider.

Securing Remote Access

Rather than exposing Hermes’s dashboard or API directly to the internet, homelab users typically choose one of two approaches:

• Tailscale – installed inside the same LXC or VM running Hermes, giving you private access over your own tailnet from anywhere with nothing exposed publicly (the safer, more common choice)
• A tunnel service like Cloudflare Tunnel – for cases where a persistent public URL is genuinely needed

Given Hermes’s access to your chosen AI model and any connected tools, keeping it off the public internet entirely is the more cautious default.

Backing Up Your Hermes Deployment

Since Hermes accumulates configuration, memory, and skill data over time, treat its VM or LXC like any other stateful service in your homelab: include it in your regular Proxmox backup schedule (or Proxmox Backup Server, if you’re running one), and snapshot before making any significant configuration changes so you can roll back quickly if something breaks.

Final Thoughts

Setting up a Hermes Agent server in Proxmox gives you a private, self-hosted AI gateway with the same isolation, backup, and snapshot benefits you already rely on for every other service in your homelab. Whether you’re pointing it at a local model for cost-free experimentation or a cloud provider for more capable results, running it inside its own VM or LXC keeps the whole setup clean, recoverable, and separate from the rest of your infrastructure.