The fastest way to destroy trust in AI is to give it the wrong job | Daily Reports Online
You will almost certainly have heard many business and technology leaders extolling the virtue of AI. They will enthusiastically tell you that the technology will have a transformative impact on businesses across the globe.
I think they are right. However, with two important caveats: AI must be given the right jobs, and the people using it must trust what it does.
Too often, AI is deployed across a business without much thought, as overconfident bosses assign agents tasks it isn’t designed for. The problem this creates is that many employees using AI tools every day are not yet prepared for their new role in managing these agents. Trust only comes from experience and that builds over time.
Latest Videos FromTechRadar
Chief Product & Technology Officer at OutSystems.
When an AI system provides you with results that make sense and actually helps with your work, people start to feel more confident in using it. But if it makes a mistake, that confidence can be lost in an instant. If things go wrong, employees start finding ways around AI or stop using it altogether, and when leaders ignore employee concerns about making AI work better, its benefits are undermined.
The best way to ensure that AI delivers the efficiencies it promises is to ensure that it is doing the right job, and this isn’t as straightforward as it sounds.
The areas where AI excels
AI works on probabilities, producing answers that are likely to be right. The key word here is ‘likely’, which is why it occasionally hallucinates.
Being probabilistic makes AI very effective when dealing with unclear or complicated information. But it’s not the best fit for tasks where a wrong answer could have serious consequences.
There are three core areas where AI really can make a difference. Firstly, when processing documents; secondly, decision support; and finally, personalization.
AI is especially effective at tasks involving many documents. It can extract, classify and summarize information that would normally take humans many hours to review. For example, an AI system could compare contracts against a standard set of clauses, or organize a large collection of customer emails by the issues they concern. This way, a person only has to look at the important findings instead of reviewing entire documents.
AI can also play a key role in supporting company executives to make decisions, as it’s very good at taking lots of different pieces of information, finding patterns and delivering options as to what to do next.
It is, however, important to remember that AI is not always the right technology to make that decision. For example, imagine you’re buying something and have to choose among many different suppliers.
AI can look at all the information, like how well they’ve done in the past, what they can deliver, how quickly it is likely to arrive and whether the supplier offers discounts for repeat purchases. It can even articulate why one option might be better than another. In most scenarios, though, it is the human who makes the final choice.
For some businesses, personalization is a promising tool, as AI can be used to make products more relevant to the individual using them. This is an area where working with probabilities can become a key driver for AI. The content only has to be good enough to make that connection so the recipient feels they are being addressed in a bespoke way.
The cost of assigning AI the wrong job
One of the key concerns companies should have about managing AI is that while AI-generated answers can sound assured, there is still a possibility that the technology has got something wrong. If that answer is allowed to trigger action without validation, a small error can travel rapidly through a workflow.
Think about what happens when someone makes a small mistake with a rule or detail in a contract. By the time someone catches the error, it may have already been added to records, sent to people outside the company, or even used to create new software.
To remind myself of AI’s limitations, I find it useful to think of the technology as the equivalent of a super-intelligent, hard-working intern. Interns need guidance and oversight, which should be provided by experienced, knowledgeable colleagues. Crucially, their access to sensitive information needs to be limited and granted only as they demonstrate they can make sound decisions.
AI needs to be managed in the same way. It should only be given greater levels of control when it has proved itself, and then its work must still be monitored by humans.
A good example of the importance of managing AI is its role in software development. AI can deliver a lot of code quickly, but its fast pace means the quality isn’t always the same as that of human developers.
In fact, managing large amounts of AI-generated code can be a real headache for IT professionals. For example, surging token consumption and a shift to consumption-based pricing is ballooning AI coding costs, forcing developers to be selective on when and where they apply AI models, as reported by Gartner.
Let AI help design the model, not control the machinery
The most important design decision is where AI is allowed to reason. The riskiest place is deep in the implementation layer, where a small mistake can cause big problems that are hard to fix.
I believe a sound approach is to use AI to work with business ideas and rules. This way, it’s easier for people to check and understand what the AI is suggesting.
Domain-specific languages and other abstractions used in model-driven development can provide that separation. AI helps create or refine a high-level application model, while a developer then validates the model before a deterministic platform transforms it into executable software.
AI can suggest how something should be done, but it doesn’t have to be responsible for executing every step.
Abstraction helps developers, and in agent-driven systems it becomes a control tool. It simplifies complex systems and prevents small errors from becoming big problems, making AI output easier to test and fix.
Human expertise is becoming more valuable, not less
As AI becomes more prevalent, developers’ roles will evolve. They will still write code, but a larger part of their role will be overseeing systems, checking outputs and catching mistakes, a change that brings with it the opportunity to learn new skills.
Developers will increasingly work at the level of systems and architecture, understanding the business rationale, recognizing patterns and judging whether AI-generated code fits the wider application and holds together.
One company that has innovated in this area and is already reaping the rewards for their experience is Ford. They recently hired 350 experienced engineers to help train younger colleagues and improve their AI and automation tools.
The shift was driven by the fact that the AI tools weren’t producing the results they wanted without the expertise brought by seasoned professionals. Now, these experienced engineers act as internal auditors, reviewing designs and finding potential problems before they become major issues on the factory floor.
When you combine AI’s ability to recognize patterns and process information with human knowledge and experience, it becomes a much more powerful tool. This combination is what makes it truly effective, not just relying on one or the other.
Building trust
Enterprises should also resist the pressure to deploy agents everywhere simply because competitors appear to be doing so. When it comes to using AI, leaders need to think carefully about each situation.
They should ask themselves three important questions. First, are they asking AI to only suggest what to do or to actually do it? Next, can someone check and fix what the AI says before an error spreads? Finally, what would happen if the system is certain about something but is actually wrong?
The answers play a big role in deciding how much freedom to give the technology. For example, a tool that summarizes documents might only need a quick check, but a system that suggests business decisions needs someone’s logged approval.
When companies start using AI for specific, practical tasks, it shows employees that it can really make a difference in their work. This approach gives the company a chance to get its data, rules and processes in order. If people can see that the system is working well, and they understand what it can and can’t do, they start to trust it more. Over time, this trust grows.
The key is to create a work setup where humans and machines do what they’re good at. AI is great at processing large volumes of information and surfacing patterns. People are good at understanding the bigger picture, pointing out mistakes, and taking responsibility for their actions.
The wrong job to give to AI is any task where a mistake carries real consequences, no one checks the work before it causes harm, or the system sounds certain while being wrong. Companies that respect this division will build the confidence to give agentic AI greater autonomy over time. Those who ignore it may discover that a single poorly chosen task can undermine their entire AI strategy.
We’ve featured the best AI website builder.
This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here:
