AI in engineering companies · Security
Connected products versus AI-assisted attackers
Most of this security series has been about protecting the AI systems an engineering company uses. This article turns to the products it makes. Connected products, from vehicles and wearables to building controls and industrial equipment, have always been targets. What has changed is that the people attacking them now have AI too.
Throughout the earlier series, I followed a single example: a fault that keeps appearing in a connected product in the field. In Agents at work in engineering, an investigation agent traced it to units sharing a component batch and a recent firmware update. Here, the story takes a turn that engineering companies should be prepared for. The fault turns out to have a security dimension.
What AI changes for attackers
Attacking a connected product has traditionally taken skill and time. Extracting and understanding firmware, working out an undocumented wireless protocol, finding a weakness in an update mechanism: these were jobs for specialists, and that scarcity was part of the defence.
AI reduces that barrier. It can help explain disassembled firmware, summarise how an unfamiliar protocol works, suggest where weaknesses might lie, and automate the tedious parts of probing an interface. It does not turn a novice into an expert, but it makes a competent attacker considerably faster.
The practical consequence is simple: the time between a weakness existing in your product and someone finding it is shrinking. Security that relied on attackers not bothering, or not having the time, is weaker than it was.
Our fault, revisited
In our running example, engineering has traced the recurring fault to units with a particular component batch running the latest firmware. On closer investigation, the firmware update changed how the product handles certain messages over its wireless connection, and one malformed message causes the fault.
That raises a new question. If an accidental condition in the field can cause it, can someone cause it deliberately? A fault that looked like a quality problem may be a vulnerability. If there is evidence it is being exploited, a manufacturer selling into the EU now has a legal obligation to report it quickly, with an early warning within twenty-four hours of becoming aware.
This is where the earlier series come together. Knowing exactly which products run which firmware, from From documents to data; being able to investigate quickly, from Agents at work in engineering; being able to draft a notification for an engineer to approve, from AI drafts, engineers decide; and being able to verify the fix in the field, from Closing the loop: each turns a potential crisis into a managed response.
The defences that matter most
None of the following is new. What is new is that each has become more important, because the attacker is faster.
Signed, verifiable updates with rollback. Every firmware update should be cryptographically verified before it is installed, and a device should be able to recover to a known good version if an update fails. In the connected vehicle platform I designed, over-the-air updates are checked for integrity and can be rolled back. This is the single most important control for a connected product, because it is how every other vulnerability gets fixed.
No default credentials, no unnecessary interfaces. Every open port, debug interface, test mode and shared default password is an opportunity. In the UK, consumer connectable products are already legally required to avoid universal default passwords, to publish how vulnerabilities can be reported, and to state how long security updates will be provided.
Secure wireless design. Bluetooth and other short-range connections are a frequent weak point, partly because they are often designed for convenience first. Pairing, authentication and message handling all need deliberate design and specific testing. The security testing of the connected vehicle platform included tests focused on its Bluetooth connections for exactly this reason.
Robust message handling. The weakness in our example is a classic one: a device that does not handle unexpected input safely. Every message a device receives, over any interface, should be treated as potentially hostile. AI makes this kind of weakness faster to find, which makes defensive testing for it more important.
Knowing what is in the product. A software bill of materials, as described in The AI supply chain, is what lets you answer, when a vulnerability is announced in a component, which of your products and units are affected.
Independent testing, repeatedly. Products change, firmware changes and attack techniques change. Testing once at launch is not enough. The connected vehicle platform was security tested repeatedly as it developed, each time against the system as it then was.
Using AI on your own side
The same capabilities that help attackers help defenders, and engineering companies should use them.
AI can review firmware and code for common weaknesses, generate unusual and malformed inputs to test message handling, analyse field telemetry for signs of attack as well as faults, and speed up vulnerability triage when a new issue is announced. It can take much of the effort out of the documentation that regulation requires.
Two cautions apply. Firmware and source code are among an organisation's most sensitive assets, so where that analysis runs matters, as discussed in Where should your AI live?. And AI-assisted security review supplements expert review and independent testing. It does not replace them. I discuss this further in Defending with AI, and testing your AI.
Regulation is catching up
For manufacturers of connected products, security is no longer only good practice. The EU Cyber Resilience Act now requires actively exploited vulnerabilities to be reported promptly, with the full set of obligations, including secure design, vulnerability handling and updates throughout a product's support period, applying from December 2027. UK law already sets baseline requirements for consumer connectable products.
The companies that will find this easiest are those that already know what is in their products, can update them securely, and can investigate and respond quickly. Those are exactly the capabilities described across these three series.
Four things worth taking seriously
For engineering leaders: assume that weaknesses in your connected products will be found faster than they used to be, and plan your response capability accordingly.
For product teams: secure, verifiable, recoverable updates are the foundation of everything else. If you cannot fix your products in the field, you cannot secure them.
For quality and service teams: some field faults are security issues in disguise. Build a path for suspicious faults to reach someone who will ask that question.
For everyone: attackers have AI. Use it on your own side, carefully, and keep testing.
I would be interested to hear how your organisation would find out if a fault in the field was actually an attack, and how quickly.
Further reading in this series
- From documents to data: building an engineering model you own (the five levels series)
- AI drafts, engineers decide (the five levels series)
- Closing the loop: letting field data change the product (the five levels series)
- Agents at work in engineering (agents series)
- The AI supply chain
- Defending with AI, and testing your AI