Shadow AI self-assessment

Visibility

01Do you know which AI coding tools and assistants are installed on developer machines?

No. We would find out from an incident or an expense report.Partly. We know the tools we licensed, not what people installed themselves.Mostly. We run periodic surveys or endpoint scans for AI tools.Yes. A continuous inventory covers enrolled machines and flags new tools.

02Can you list the MCP servers, agent skills and plugins your agents can reach?

No. They are configured per machine and nobody holds a list.Some. A few teams document theirs; most do not.Mostly. We have an inventory, but it is updated by hand.Yes. They are discovered automatically and unregistered ones are flagged for review.

03When a SaaS vendor or a browser extension adds an AI feature, how do you find out?

We do not, unless someone mentions it.From the vendor’s release notes, when someone reads them.Through a vendor review process that includes AI features.Through the same inventory and review that covers every other AI surface.

Policy

04Is there a written policy on which AI tools are approved and how they may be used?

No written policy.A general acceptable-use statement that does not name AI tools.A written AI policy with an approved-tool list, reviewed at least yearly.A written policy with an approved and tested catalogue, plus rules for specific actions.

05What happens when an AI agent tries a high-risk action, such as reading a credential file or running a destructive command?

Nothing. The agent has whatever access the developer has.It depends on the tool’s own settings, which each developer configures.A central rule warns or blocks for some tools and some actions.Central policy is evaluated before the action runs, with block, review and allow outcomes per rule.

06How is policy on AI tool use enforced?

It is not; it relies on people remembering it.Through onboarding and occasional reminders.Through tool configuration pushed to managed machines.Through enforcement at the point of action, with the policy version recorded on each decision.

Identity & access

07When an agent calls a tool or an MCP server, whose credentials does it use?

Shared or long-lived credentials pasted into config files.The developer’s own credentials, with the developer’s full access.Per-team credentials, scoped to what the team needs.Per-user, short-lived credentials issued for that call and tied to the person’s directory identity.

08Can you tie AI activity back to a person, a team and a machine from your directory?

No. Activity is anonymous or tied only to a local username.Sometimes, by cross-referencing logs by hand.Usually. Tools authenticate through SSO, but tool calls are not linked to it.Yes. Every recorded action carries the directory identity, role and machine.

Accountability

09If an agent made a change in production last week, could you reconstruct what it did and why it was allowed?

No. There is no record beyond the change itself.Partly, from the tool’s own logs, if they were kept.Yes, from central logs, but not the policy decision that allowed it.Yes. A decision record holds the actor, action, matched policy, verdict and reason.

10Who reviews AI tool decisions and exceptions, and how often?

Nobody. Exceptions are granted ad hoc and never expire.Security, when something goes wrong.Security on a fixed cadence; exceptions are tracked but open-ended.IT, security, legal and the teams involved, on a cadence; exceptions are approved by a person and time-boxed.

Where does your organisation stand?

Start the conversation. One call to establish fit and how we would work together.