I could use some advice on a situation at work.
A few colleagues in ops have, almost overnight, started to fully trust their LLM agent because it has gotten so good lately and "doesn't make mistakes anymore". They built their own harness around it and log what it does (using Grok and Google models), but the agent still runs with root access to our entire production infrastructure. We have no clear team rules or policy yet, and they don't see the danger themselves.
A typical example: whenever the agent needs to check something on a server, they just hand it full access. But looking at info only needs read-only access. The permission should match the task, not the most convenient option.
My concerns:
Logging is detection, not prevention. It shows what broke afterwards, but doesn't stop anything.
Root bypasses the harness. An agent with root can in theory edit the harness, the logs and the backups, so those logs are not a reliable audit trail.
No least privilege. One wrong command or misread context can touch everything.
Prompt injection via logs, tickets or web pages the agent reads.
Model updates. Providers change models without us deciding when, so "no mistakes in recent months" says little about the next version.
Data going to third parties. With root, the agent can read .env files, keys and customer data, which then end up with the model provider (a GDPR question too).
"It has gone fine so far" only means the failure hasn't happened yet.
I don't want to ban anything. I want them to see the risk themselves and set up proper guardrails together with me: a read-only user by default, scoped credentials, just-in-time access, human approval for write actions, staging first, and no access to backups and logs.
Questions for you:
How do I explain the difference between "we log it" and "it can't do it in the first place"?
How have you convinced overconfident colleagues about AI tooling risks without a confrontation?
Do you have horror stories or post-mortems I can share? Concrete incidents tend to work better than theory.
Which guardrails or policy examples have worked well for you?
How do you raise this with management without coming across as "anti-AI"?