AI in engineering companies · The five levels of AI value

Beyond search: the five levels of AI value in an engineering company

Catherine Ives-YimPublished 27 September 2026 · 7 min read

Most of the conversations I have about AI in engineering companies start and end in the same place: "Could we use it to search our documents?" It is a good question, and the answer is yes. I have written about it separately. But it is the first step on a much longer ladder, and in my experience most organisations stop there, not because the higher steps lack value, but because nobody has described them.

This article describes the whole ladder. There are five levels, and each builds on the one below. The value rises at each step, and so does the effort and the need for care. Understanding where you are, and what the next level would actually take, is more useful than any individual tool choice.

To keep it concrete, I will follow a single problem up the ladder: a fault that keeps appearing in a connected product in the field. It could equally be a machine on a production line, a component in a building system, or a sensor in an environmental monitoring network. The pattern is the same.

Level 1: Find

At the first level, AI helps people find what the organisation already knows. Manuals, procedures, service history, standards and design notes become searchable in plain language, and every answer points back to its source. The technique is usually retrieval-augmented generation, or RAG.

Our fault at level 1: a support engineer describes the symptom and immediately finds the relevant diagnostic procedure, the service bulletin issued last year, and three past cases that looked similar.

The value is time. Less searching, fewer interruptions of experienced staff, faster onboarding, better first answers.

What it takes is well-kept source documents, careful handling of how they are split and indexed, and a system that says plainly when it cannot find the answer.

It is worthwhile. It is also the level most easily copied, and the one where the value is hardest to show on a balance sheet.

Level 2: Structure

At the second level, AI reads the documents and records and extracts structured, linked information from them: which product, which part, which firmware version, which fault, which fix, which procedure, which requirement, which test. A pile of text becomes an engineering model that can be queried.

Our fault at level 2: the question is no longer "what do we know about this symptom?" but "which products in the field share the component and firmware version involved, and how many have reported it?" Search cannot answer that. A structured model can.

The value is traceability and impact analysis. It underpins compliance work that is becoming mandatory for connected products: knowing exactly which software components are in which products, which vulnerabilities affect them, and where the evidence is. It also creates something the company owns outright. The structured model is a durable asset, independent of whichever AI model built it.

What it takes is a clear definition of the things that matter and how they relate, careful validation of what the AI extracts, and ownership of the result by someone in engineering. This is the level most organisations skip, and it is the one everything above depends on.

Level 3: Connect

At the third level, AI works with live systems as well as documents: the telemetry platform, the service ticketing system, product lifecycle and manufacturing records, test rig outputs. It can combine what the manual says with what a specific unit is doing now.

Our fault at level 3: "Why is this particular unit failing?" can be answered by bringing together its build record, its firmware history, its recent telemetry and the known fault patterns, in one place, in minutes.

The value is faster, more accurate diagnosis, higher first-time fix rates, and fewer units returned that had nothing wrong with them.

What it takes is integration work, careful control over what the AI can see and do in each system, and attention to where the data is processed, because operational data often includes customer and location information. This is where the choice between cloud, UK-hosted and local AI becomes a real engineering decision rather than a policy statement.

Level 4: Act, with a person signing off

At the fourth level, AI produces the work that engineering has to produce: draft root-cause reports, service bulletins, updates to failure analyses, change impact assessments, compliance files, test plans. An engineer reviews, corrects and approves. Nothing leaves without a person's signature.

Our fault at level 4: the investigation report is drafted from the data gathered at level 3. The affected products are identified from level 2. A service bulletin and a proposed test to confirm the fix are drafted for review.

The value is skilled engineering time. In most engineering teams I have worked with, a large share of senior time goes on necessary but repetitive documentation. Handing the first draft to AI, and keeping the judgement with the engineer, frees that time for the problems that need it.

What it takes is levels 1 to 3 working properly, clear rules about what the AI may draft and who must approve it, and a record of what was generated and what was changed. An AI that drafts from messy data simply produces confident mistakes faster.

Level 5: Learn

At the top level, the loop closes. AI continuously reads the service logs and field data, spots an emerging fault before it becomes a trend, links it to a batch, supplier or software release, starts the investigation, and, after a fix is released, checks the field data to confirm the fault has actually stopped.

Our fault at level 5: it is noticed after the fifth report rather than the fiftieth, traced to a cause within days rather than months, and the fix is verified by evidence rather than assumed.

The value is measured in warranty cost, recalls avoided, product reputation and the speed at which the product improves. This is the level a board cares about, because it shows up in the accounts.

What it takes is everything below it, plus a genuine connection between service, quality and engineering, which is an organisational change as much as a technical one.

You cannot skip levels

The most common mistake I see is ambition aimed at the wrong level. An organisation hears about AI agents that take action, and wants level 4 or 5, while its documents are inconsistent, its service records are free text in three systems, and nobody can say reliably which firmware is on which product.

Every level depends on the ones below. Search is only as good as the documents. Structure is only as good as the extraction and its validation. Connection is only as safe as the controls around it. Action is only as sound as the data it acts on. Learning is only as useful as the organisation's willingness to respond.

That is not a reason to be timid. It is a reason to sequence the work. Each level delivers value in its own right, and each makes the next one possible.

Where most companies are

In my experience, most engineering organisations that have done anything are experimenting at level 1, often with a general-purpose assistant rather than a system built around their own documents. A few are building level 1 properly. Very few have started level 2, which is a pity, because it is where the lasting asset is created and where regulatory pressure is about to make the case for them.

The encouraging part is that levels 1 and 2 are affordable, can be run on modest infrastructure, and can be kept entirely in-house if the material is sensitive. The higher levels need more integration and more organisational commitment, but by the time you get there, the lower levels will already have paid for themselves.

Four things worth taking seriously

For engineering leaders: decide which level you are aiming for and be honest about which level you are at. The gap between them is your plan.

For anyone commissioning AI work: ask what the supplier's work leaves you owning. At level 2 and above, the answer should be a structured model of your products and their history, not just access to a tool.

For quality and service teams: you hold the data that makes levels 3 to 5 possible. Treat it as engineering data, and insist it is captured in a form that can be used.

For engineers: the most valuable AI work is not replacing your judgement. It is getting everything your judgement needs in front of you, and taking the paperwork off your desk afterwards.

I will be looking at each of these levels in more depth in the coming weeks: what each involves in practice, what goes wrong, and how to get started. I would be interested to hear which level your organisation is at, and which one you would most like to reach.