More Intelligence Won’t Solve It.

As AI moves from generating information to taking action, enterprise adoption becomes a question of digital trust. Organisations need to know what an AI system is allowed to do, who authorised it, what it can access and who remains accountable for the decisions it makes.

For most of the generative AI era, progress has been relatively easy to describe. Models became better at reasoning, coding, analysing information and working across different forms of data. Each new generation promised greater capability, and the competitive argument largely followed the same trajectory: make artificial intelligence more intelligent and more useful things become possible.

That logic becomes less comfortable when AI stops simply producing an answer and begins taking action. An AI agent that can access corporate systems, call applications, retrieve information, initiate workflows and make decisions presents an organisation with a different problem. The question is no longer simply whether its answer can be trusted, but whether the organisation can trust it to act.

A September EY survey of 202 senior AI decision-makers at large US companies found that 91% were already using agentic AI through pilots or enterprise deployments. Yet 49% of those organisations had not updated their existing governance frameworks specifically for agentic AI. More strikingly, 85% said at least some of their agentic systems were already executing actions without real-time human involvement.

The technology is therefore beginning to cross a boundary that enterprise governance was largely designed around.

From information to agency

Software has always performed actions, but conventional enterprise software normally operates within predetermined processes. A payment system processes a payment because something explicitly triggered that process, while an application accesses a database according to permissions and instructions established in advance.

Agentic AI introduces more discretion into that relationship. An agent may determine which tool it needs, decide which information to retrieve, choose a sequence of actions and interact with several systems to complete an objective. More capable agents can potentially delegate tasks to other agents or operate over longer periods without continuous human supervision.

That capability is precisely why businesses are interested in them, but it is also why simply making the underlying model more intelligent does not solve the trust problem. A more capable system may make fewer mistakes, understand context better and make better decisions, but capability does not establish authority. An organisation still needs to determine what the system is permitted to access, which actions it may perform, under whose authority it is acting and where human approval remains necessary.

Those are governance and control questions rather than intelligence problems, and they become more consequential as the actions delegated to AI become more significant.

The controls are following the deployment

There is growing evidence that organisations recognise the problem but have not yet caught up with it. The Cloud Security Alliance reported in April that 53% of organisations surveyed had experienced AI agents exceeding their intended permissions, while 47% reported an AI-agent-related security incident during the previous year. The same research found significant gaps around ownership and visibility of agents operating inside organisations.

Deloitte, meanwhile, surveyed more than 3,200 business and IT leaders across 24 countries and found that only 21% believed their organisations had mature governance models in place for agentic AI.

Taken together, those findings suggest organisations are discovering what AI agents can do faster than they are defining what those agents should be allowed to do. That gap becomes more consequential as agents gain access to valuable systems and information. A chatbot generating an inaccurate paragraph creates one category of risk; an autonomous system with permission to alter a customer record, execute code, change a configuration or initiate a transaction creates another.

The trust requirement changes with the consequence of the action.

Policy is not the same as control

Most large organisations are not ignoring AI governance. The EY research found that 98% of respondents had formal AI governance policies, yet 47% acknowledged that their organisations had previously bypassed their own governance processes for urgent AI deployments. More than a third reported an AI incident or failure during the previous year that had caused materially negative consequences including data loss, financial damage, operational disruption or reputational harm.

This exposes an important distinction as AI becomes more autonomous. Writing down what an AI system should be allowed to do is not the same as technically enforcing what it can do.

Enterprise trust has traditionally relied upon mechanisms that translate policy into control. Employees have identities, access can be granted according to role, permissions can be limited, transactions can require additional approval, credentials can be revoked and activity can be logged and investigated. AI agents need equivalent mechanisms, although they will not necessarily be identical.

An agent might be authorised to read a customer account but not change it. Another might be allowed to prepare a payment but not release it. A security agent might isolate a compromised endpoint automatically but require human approval before shutting down a production system.

The useful question for an organisation is therefore not simply whether it trusts AI. It is which AI system it trusts to perform which action, using which information, under whose authority and within which limits.

The visibility problem

Before organisations can answer those questions, they face an even more basic challenge: knowing which AI systems are operating.

Research published by Okta this year found a substantial difference between management confidence and employee behaviour. Ninety percent of executives surveyed believed their organisations had visibility into AI tool usage, while 52% of employees acknowledged using unapproved AI tools. Only 34% of organisations said they consistently applied the same security controls to their agentic digital workforce as they did to human workers.

EY found a similar problem from another direction, with 26% of respondents using agentic AI saying their organisations could not detect unauthorised AI agents operating internally.

This matters because invisible agency is difficult to govern. An organisation cannot meaningfully control what an autonomous system is permitted to do if it cannot reliably identify that system, establish who owns it or understand which resources it can reach.

The problem starts to resemble identity management, but with an important difference. AI agents can potentially be created, modified and multiplied much faster than human identities. They can also operate continuously and interact with systems at machine speed, meaning the mechanisms used to control them will increasingly need to operate at comparable speed.

Trust needs an operating model

This is where the enterprise AI conversation needs to move beyond the familiar argument about whether models themselves are trustworthy. Model reliability remains important, and hallucination, bias, security vulnerabilities, data provenance and explainability are not disappearing. Autonomous AI, however, adds another layer because organisations also need an operating model for agency.

That means being able to identify an agent, establish its owner, define its permissions, restrict its access, understand which data it uses, record what it has done and withdraw its authority when necessary. It also means deciding where autonomy should stop.

The answer will differ according to context. Allowing an AI agent to reschedule an internal meeting carries very different consequences from allowing one to move money, modify industrial equipment or respond autonomously to a cyberattack. Human oversight therefore cannot simply mean placing a person somewhere vaguely inside the process. Organisations need to decide which actions require human approval, which can safely be delegated and what happens when a system behaves outside its expected boundaries.

None of this should be interpreted as an argument against greater AI autonomy. In many cases, stronger controls are precisely what will allow organisations to delegate more responsibility to AI systems with confidence. The ability to define and enforce the limits of autonomy becomes an enabler of adoption rather than simply another compliance requirement.

Intelligence is only half the equation

The technology industry will continue making AI more capable. Models will reason better, operate for longer, use more tools and perform increasingly complicated tasks. Enterprise adoption, however, will depend on organisations developing confidence that autonomous systems can operate inside defined boundaries and that those boundaries can be demonstrated rather than merely promised.

That changes what progress in enterprise AI looks like. Intelligence remains important, but so do identity, authority, permissions, accountability and the ability to intervene when something goes wrong. The organisations able to establish those controls may ultimately be the ones able to give AI systems the greatest degree of useful autonomy.

There is a paradox in that. The more autonomous artificial intelligence becomes, the more precisely organisations will have to engineer the limits around it. More intelligence will undoubtedly make AI more useful, but intelligence alone will not make it trustworthy.