The attack surface has moved

Catherine Ives-YimPublished 27 September 2026 · 6 min read

For most of my career, securing a system meant understanding its boundaries. Where does data come in? Who can reach which interface? What runs with which privileges? The connected vehicle platform I designed was security tested throughout its development: thousands of automated tests with industry-standard security tools, run by different AI models from the ones that wrote the code, across its database, web interfaces, APIs and Bluetooth connections. Every one of those tests was ultimately about those questions.

Connecting AI to an organisation's data and systems does not remove any of those questions. It adds new ones, and it moves some of the boundaries to places most security teams are not yet looking.

This is the first article in a series on securing AI in engineering companies. It follows two earlier series: one on the five levels of AI value, from finding information through to closing the loop from field data, and one on AI agents. Everything described in those series creates value. This series is about making sure it does not also create a way in.

What is actually different

Three things change when AI is connected to your systems.

Text becomes a way to give orders. In conventional software, data and instructions are separate. A customer's support ticket is data; the code that processes it is instructions. A language model reads both as text, and it can be influenced by instructions hidden in the data it is processing. That single fact underlies the most important new class of attack, which I cover in When the data gives the orders.

AI systems concentrate access. A search system over company documents, as described in Your company already knows the answer, holds an organised copy of everything it has indexed. An agent connected to several systems, as described in Agents at work in engineering, can reach all of them. That concentration is exactly what makes these systems useful, and exactly what makes them attractive to an attacker.

Behaviour is shaped, not programmed. A conventional system does what its code says. An AI system's behaviour depends on its model, its instructions and everything it reads. It cannot be fully specified in advance or fully tested, and it may change when the model changes. Security has to be designed around behaviour that is probabilistic rather than guaranteed.

A simple threat model

When I review an AI system, I work through five areas. None requires specialist AI knowledge. Each requires the same discipline as any other security review.

1. Data. What does the system hold, read and produce? Where does it go? Who can see it through the AI that could not see it directly? Data leaving through prompts, outputs, logs and indexes is the subject of How data leaks out of AI systems.

2. Models. Where did the model come from, can it be trusted, and where does it run? A hosted model means your data goes to a provider. A downloaded model means trusting the file you downloaded. The choice of where AI runs, discussed in Where should your AI live?, is a security decision as well as a cost one.

3. Tools and permissions. What can the AI reach and change? An assistant that only answers questions has a small blast radius. An agent with access to production systems has a large one. This is the subject of Agents with keys.

4. People. Who can instruct the AI, who approves what it does, and who can write content it will read? Customers, suppliers and anyone who can put text in front of your AI are, in effect, part of its input.

5. Supply chain. What components, libraries, plugins and models does the system depend on, and what code has AI written into your products? That is covered in The AI supply chain.

The same principles, applied in new places

The encouraging part is that the core principles of good security still hold. They simply need applying to places they have not been applied before.

Least privilege. Give each AI system and agent the minimum access it needs. The guardrails in Guardrails: autonomy is earned are security controls as much as safety controls.

Defence in depth. Assume any single control will fail. An AI that can be tricked into wanting to do something harmful should still be prevented from doing it by permissions, approvals and monitoring.

Separation of trust. Keep instructions from the organisation clearly separate from content from outside, and never let outside content grant new capabilities.

No exposed surfaces. In the systems I have built, including the connected vehicle platform and a dual-node environmental monitoring system, remote access runs through authenticated tunnels with no open inbound ports, with one-time passcodes and rate limiting on user access. The same thinking applies to AI services: they should not be reachable by anyone who happens to find them.

Record everything. Every AI action should be logged well enough to reconstruct what happened. Without that, an incident cannot be understood, let alone prevented from recurring.

Why this matters now

Two things make this urgent for engineering companies in particular.

First, the regulatory ground is moving. The EU Cyber Resilience Act now requires manufacturers of connected products to report actively exploited vulnerabilities, with full obligations from December 2027. The UK already requires basic security measures for consumer connectable products. The EU AI Act is bringing obligations of its own. Engineering companies are increasingly expected to show that the systems they build and use are secure.

Second, attackers are using AI too. Work that once took skilled people weeks, such as analysing firmware or probing interfaces, can now be done faster and more cheaply. I discuss what that means for connected products in Connected products versus AI-assisted attackers.

A note on how this series is written

These articles explain how AI systems are attacked in enough depth for engineering and technology leaders to recognise the risks and close them. They are deliberately not step-by-step instructions. The aim is to help organisations defend themselves, and that needs understanding, not recipes.

Four things worth taking seriously

For engineering leaders: every AI system you connect to your data is a new system to secure. Ask for it to be threat-modelled like any other.

For security teams: the new boundary is text. Anything the AI reads is a potential input from an attacker.

For anyone building AI systems: design security in from the start. Permissions, approvals and records are very hard to add afterwards.

For everyone: good security principles have not changed. The places they need to be applied have.

I would be interested to hear whether your security reviews have started to include the AI systems your organisation uses, and what they found.