That’s 2 red flags right there.

I recommend you tu use either a dedicated computer or a VPS.

The way I see it, Hermes is like any new employee. It needs a computer, a GitHub account, its email, and so on…

This way, you can manage what Hermes can access.

I chose to run Hermes on a VPS, but that doesn’t mean it’s entirely safe yet.

Let’s find our attack surfaces and fix them1. The VPS — Virtual Private Server.A VPS has a public URL — a door anyone on the internet can walk up to. So if someone decided to break in, they could access everything. Obviously, I don't want that.

To fix it, I removed the URL: no public door to find, no break-in.

So how do I get in? Using Tailscale — a private tunnel that only my devices can enter. I go through; strangers can't, because the tunnel only exists between things I own.

2. The agentYou need to think about what your agent is allowed to do, not just what it can see.

It can push code, delete repos, post on Slack, burn API credits — all without ever seeing the actual API key. A malicious or sloppy prompt can convince it to do something destructive, and if you didn’t set limitations, there’s nothing you can do about it.

I don’t give it permissions it doesn’t need. I am careful each step of the way to avoid accidents.

My Hermes doesn't use my personal GitHub. It has its own bot account, with access to the repos it needs only. On my critical GitHub repos: Hermes can’t merge anything; it can just create PRs.

Read-only where possible. Does the agent really need to push code, or just read issues and open pull requests? Most of the time, read + write issues are enough.

Spending caps. OpenRouter, DeepSeek — hard limits at the provider level. The agent literally cannot drain more than the cap, no matter what it tries.

Also, keeping backups is always a good idea.

3. External skillsThere are some places where you can find interesting skills on the internet and add them to your Hermes agent.

I recommend that you use them for inspiration only. Copying the skill idea and asking your Hermes to create its own version, this will prevent potential malware threats.

4. Multiple usersWhen you have multiple users, you shouldn’t have a single agent. If you do, then everybody is going to have access to the same big pile of secrets. All the passwords, API keys… from everyone. THAT IS NOT SAFE!

The fix is obvious: multiple users = multiple agents. To do that, you can go two ways.

Either you set up a profile for each person (Louise, Denis…), or you set up a profile for each job description (dev agent, design agent…)

Both are valid.

I use the person model for personal use (my wife has a profile, and I have another). I just made sure each agent is using different API keys to gate everything.

In a company context, I would use job descriptions. It would allow people with the same job to use the same agent, so everybody has access to the same skills.

5. The chatAnother risk is the chat — Slack, Telegram, wherever you talk to the agent.

Let’s say your API keys and credentials live in a file called .env, and your agent can read it. The right prompt could convince Hermes to hand them over to anyone (provided they can access the chat).

The easiest fix would be for the agent NOT to have the keys. It can’t hand something it doesn’t have.

To manage this, I decided to use Infisical.

Buckle up, this is where it gets tricky.

I spent two days setting up Infisical on my VPS. It's meant for DevOps teams — engineers manually provisioning credentials for known services, approving workflows… while keeping humans in the loop.

I was excited to get a place where I could manage all the secrets — especially since I’m not a fan of using their web UI or the terminal.

I went all in thinking it was going to be pretty, secure, and clean.

Except it didn't.

Infisical injected the keys at the gateway startup, but Hermes loaded the .env file right after and overwrote them with whatever was still sitting in there.

In the end, Hermes could still see the key values. Exactly the problem I'd brought Infisical in to fix.

So I dug deeper and found Infisical's other project: Agent Vault.

Big difference: the credentials are saved as a placeholder in Hermes. When the key is called, the vault injects it at run time, and Hermes never sees the actual key. Keys live outside the agent's context entirely.

[Bonus is you can create one vault per user, or per project, each with its own set of credentials. The agent vault address and agent config are baked into the different Hermes profiles. Two birds, one stone!

[[After 3 days, it was finally done: everything was clean and separated. No door in, no leaking keys, different profiles per person.

Last check, to make it extra

When deploying for a client, I go one step further and put the Agent Vault on its own dedicated VPS, connected to Hermes over a private network.

It would make sure an attacker trying to compromise the Hermes box hits a dead end.

Having the real credentials live on a separate machine, they exist independently. It's an extra €5–10/month and maybe an hour of setup. Cheap and easy, to make sure you can sleep on your two ears. For a company, that's the difference between "hard to breach" and "even if you breach it, you get nothing."

Our data is now safe!

This is how I manage security, but If you have better practices, I would love to hear them :)

If you are curious about setting Hermes up for yourself or your team, hit me up!

Built with good vibes,

Benoît

Share