Excessive Agency
Damaging actions performed in response to unexpected or ambiguous LLM output, stemming from excessive functionality, permissions, or autonomy.
What it is
An LLM-based system is often granted a degree of agency by its developer — the ability to interface with other systems and undertake actions in response to a prompt. The decision over which functions to invoke may be delegated to an LLM “agent” that determines them dynamically.
Excessive Agency is the vulnerability that enables damaging actions to be performed in response to unexpected or ambiguous LLM output — regardless of the cause (hallucination, direct/indirect prompt injection, a malicious plugin, poorly-engineered prompts, or just a poorly-performing model).
Its root cause is typically one or more of excessive functionality, excessive permissions, or excessive autonomy. This differs from Insecure Output Handling, which concerns insufficient scrutiny of outputs. Impacts span confidentiality, integrity, and availability, depending on which systems the app can interact with.
Common examples
- Excessive functionality: an agent has access to plugins with functions not needed for the intended operation — e.g. a plugin meant to read documents that can also modify and delete them.
- Excessive functionality: a plugin trialed during development is dropped but remains available to the agent.
- Excessive functionality: an open-ended plugin fails to filter instructions — e.g. a plugin meant to run one specific shell command allows others to be executed.
- Excessive permissions: a plugin has permissions on other systems beyond what's needed — e.g. a read plugin whose database identity also has UPDATE, INSERT, and DELETE.
- Excessive permissions: a plugin acting for a user connects to downstream systems with a generic high-privileged identity that can access all users' files.
- Excessive autonomy: a plugin performs high-impact actions (e.g. deleting documents) without any user confirmation.
Prevention & mitigation
- Limit the plugins/tools the agent can call to the minimum necessary.
- Limit the functions implemented within plugins to the minimum necessary — e.g. an email-summarizer should only read, not delete or send.
- Avoid open-ended functions (run a shell command, fetch a URL); prefer granular, purpose-built tools.
- Limit plugin permissions to other systems to the minimum, enforced via appropriate database and identity permissions.
- Track user authorization and scope so actions run in the context of that specific user with minimum privileges (e.g. OAuth with minimum scope).
- Require human-in-the-loop approval for all high-impact actions.
- Implement authorization in downstream systems (complete mediation) rather than relying on the LLM to decide what is allowed.
- Damage-limiting (not prevention): log and monitor plugin and downstream activity, and apply rate-limiting.
Example attack scenarios
- A personal-assistant app's email plugin can both read and send. An indirect prompt injection in an incoming email tricks the LLM into commanding the plugin to send spam from the user's mailbox. It could be avoided by (a) a read-only plugin, (b) OAuth with read-only scope, and/or (c) requiring the user to review and send every drafted mail — or mitigated by rate-limiting the send interface.
References
- Embrace the Red: Confused Deputy Problem: Embrace The Red
- NeMo-Guardrails: NVIDIA GitHub
- LangChain: Human-approval for tools: LangChain Documentation
- Simon Willison: The Dual LLM Pattern
How Neural Industries helps. We test, exploit, and remediate Excessive Agency as part of a full OWASP LLM Top 10 assessment — then hand you the evidence to prove it.
Request an assessment →