Whose privileges run the tool the model picked — excessive agency

In one sentence

An agent runs with the service account's privileges, and the model follows the user's words and the words in documents without telling them apart. So the server decides what to execute with the requesting user's privileges, not the model's choice, validates the arguments, and has a person confirm anything that cannot be undone.

Why this was needed

Say you build a file assistant. You give the model tools to list, read, write, and delete, and the app server executes the tools. The app server runs as an account that can touch every user's files. What happens when a viewer user says "delete this file"? The model picks the tool, and the server executes it as is. A privileged agent granting the request of someone without privileges is a problem with an old name in security, the confused deputy. Layer prompt injection on top, and the one making the request is not even the user but a sentence in a document.

The OWASP Top 10 for LLM Applications 2025 lists this as Excessive Agency (LLM06) and splits its roots into three: unnecessary functionality (too many tools), unnecessary permissions (tools reach too widely), and unnecessary autonomy (doing irreversible things without a person's confirmation).

How it works

Root	Where to fix it	Function in this lab
Functionality	Trim the tools shown to the model to fit the role. Remove development tools (the shell) entirely, and do not execute names that are not on the list	offered_tools, allowed
Permissions	Right before execution, check permission again against the requesting user's role, regardless of what the model picked	allowed
Permissions (arguments)	Resolve path arguments to real paths (.., absolute paths, symbolic links) and check that they are inside that user's home	resolve
Autonomy	Always hold deletions, and execute them only when the requesting user confirms once with a separate request	run_call, handle_confirm

The last row is subtle. "The delete tool deletes only when it has a confirm: true argument" looks like a confirmation, but the one who fills in that argument is the model. If an instruction in a document puts in confirm, the confirmation disappears. Confirmation must come through a channel the model cannot write to: a second request from the person who received the hold id.

Receiving confirmation through a separate channel gives you one more thing. You can show the held action on screen as "what, where, and with whose privileges". What a person confirms must be the call the server will actually execute, not the model's words ("I'll tidy that up for you"). And the confirmation id is bound to the requesting user, disappears once used, and must pass the role and path checks again even after confirmation. Otherwise the hold becomes a back door that skips the checks.

Path validation leaks too if done on strings. Reject .. and absolute paths get through; block both and a symbolic link placed inside the home can still point outside. It is safe to resolve to the real path (os.path.realpath) and then compare the common ancestor with the base directory.

The fake model in this course always calls the tool when there is an instruction in a document. A real model mostly does what the user means and occasionally does not. Because tests cannot find every "occasionally", permissions are applied outside the model. Where the mcp-server course covers the tool server's side of the contract (allowlists, read-only, confirmation), here we deal with the app carrying the end user's identity and role all the way to tool execution.

What it looks like in the field

Common shapes are an internal document agent tidying up a shared folder at the request of an employee who had only read permission, an agent marking "confirmed" by itself and deleting files because of a sentence in shared meeting notes, and a shell tool attached during development remaining in production. When investigating, for each missing file look in the execution records at who (role), with what sentence and context, made which tool call. And at the same time, separate out that what the owner confirmed and deleted directly is not an incident.

Recovery is a judgment too. Restoring everything from backup is easy, but then even what users deliberately deleted comes back, going against their intent, and if it was the handling of a request to delete personal information, that itself becomes a new incident. So first work out from the records "whose intent was it", and restore only what was deleted by accident. You can make this judgment only if the request sentence and context remain in the execution records, which is how the investigation of excessive agency connects with the record design of the previous module.

The role table itself is also something to design. If you set it broadly, like "admin can do anything", one line of a document mixed into an admin's request uses the greatest privileges. If you narrow it so each tool requires only the roles it really needs, and irreversible tools require both a role and confirmation, the reach of the damage shrinks when an injection succeeds.

What you will do in the next lab

You count out-of-role executions and out-of-home paths in the file assistant's past execution records, then add in turn a role check, a trimmed tool list, path validation, and confirmation through a separate channel. In the final incident, you work out the different reasons three people's files disappeared and restore only what was wrongly deleted.