What happens when AI is confidently wrong in the workplace? | Daily Reports Online
AI tools are now a normal part of the working day. Employees lean on AI for tasks ranging from drafting emails and summarizing content, to preparing legal documents and creating financial reports but, while there’s no doubt that AI and automation are making our workdays more efficient, the more we rely on these tools, the greater the risk of them being exploited, misused or simply trusted too much.
In 2023, US lawyers were sanctioned after filing legal documents built on false citations that were invented by ChatGPT. If experienced professionals are trusting AI outputs without verifying primary sources, even in high-stakes legal proceedings, it raises concerns about our growing dependence on AI and the risks of unchecked and hallucinated data.
Chief Technology Officer, MetaCompliance.
But hallucinations are only part of the problem. As employees become more comfortable using AI at work, organizations also need to understand what information employees are feeding into AI engines and whether they have the right controls in place to manage the security, privacy and compliance risks that come with unsanctioned AI use.
Latest Videos FromTechRadar
Ungoverned AI is no longer a theoretical compliance risk. It’s a real issue that CISOs are having to deal with, from protecting consumer and employee data to defending against AI-enabled attacks. As AI regulation moves from guidance to enforceable requirements, organizations are facing more pressure to understand where AI is being used and who’s responsible for its outputs.
The cost of AI without governance
Ungoverned AI use poses significant risks, particularly when employees share sensitive data with AI tools without clear guidance on how it should be used. That information could then influence business decisions based on unverified AI outputs or fall into the hands of a bad actor looking to exploit the business.
Once sensitive information leaves an organization’s control, there’s no telling where it could end up. If an AI platform is breached, data entered into the tool could be exposed, potentially giving attackers exactly what they need to target a company’s systems. What starts as a quick shortcut to save time can just as easily become an entry point for credential theft or serious financial loss.
The EU AI Act’s transparency requirements, applicable from 2 August 2026, expects organizations within scope to be transparent about their business’ AI interactions and AI-generated or manipulated content. It also requires staff to be equipped with an appropriate level of AI literacy, alongside existing regulations such as GDPR and relevant FCA requirements that involve the effective governance of AI processing of personal data. The cost of breaching these transparency requirements can carry fines of up to €15 million or 3% of an organization’s total annual turnover.
This means that now more than ever, organizations need to understand exactly how AI is being used, what information is being entered into AI tools and what controls are needed around higher-risk use cases, especially where personal data and financial decisions are involved. This is where AI verification protocols become critical.
Why verification isn’t a one-size-fits-all
A generic AI policy that isn’t tailored to specific use cases will tend to get ignored. Employers telling staff to simply “check everything” they receive from these tools isn’t realistic or clear enough guidance, since not every AI-assisted task carries the same risk.
Scrutiny needs to scale with what’s at stake if the output turns out to be wrong. By treating every task the same, you either take up valuable employee time on unnecessary verification processes or risk a potential security incident because an AI-generated output is moved forward unchecked.
For low-risk tasks like internal drafting, brainstorming and research support, light spot-checking can be enough. However, anything feeding into decisions about employment, finance or customer-facing content needs strict approval processes, ongoing monitoring, and a genuine human in the loop.
Practical steps to reduce the AI hallucination risk
Reducing AI cyber risk starts with proper governance. Organizations need clear rules on which AI tools are approved, what information employees can share within the platforms, and who’s accountable when something goes wrong. Waiting until a hallucinated output causes damage is the most expensive way to learn this lesson.
It’s becoming increasingly common for employees to bring their own AI tools into work. Our latest research found that 40% of CISOs fear their staff are sharing sensitive information with generative AI platforms, while 68% identify employees as their organizations biggest cyber risk. Ungoverned AI is becoming a growing governance and security blind spot for organizations .
But the risk goes far beyond what employees put into AI tools. When organizations don’t know which tools are being used or to what extent employees are relying on their outputs, they have limited visibility into whether AI-generated information is being factchecked before it’s acted upon.
Organizations can’t manage risks they don’t know exist, which means mapping where ungoverned AI is genuinely being used and then offering secure, approved alternatives, rather than banning it completely. This gives organizations greater visibility over how AI is being used and where hallucinations or inaccurate outputs could enter business processes, before they become a liability.
Generic training won’t fix the AI problem
Employees need to be equipped with the right processes, tools and guardrails to use AI effectively, without putting businesses at risk. This includes training to recognize automation bias (our tendency to trust outputs blindly), paired with explicit verification routines that turn “double-check the AI output” from a vague instruction into an actionable process for employees to feel in control of how they can use AI responsibly.
Role-based security awareness targeting and behavior-triggered interventions are also critical for putting the right guidance in front of the right people at the moment they need it, so that sensitive information is protected, and the right human judgement is used.
Ultimately, addressing AI hallucination and the risk of ungoverned AI comes down to training employees properly. Most CISOs (81%) say that security awareness training fails because it’s too generic to feel personally relevant. Without context that reflects the decisions employees face day to day, training gets completed but rarely absorbed and actioned on.
Checkout our ratings for the best payroll software in the UK.
This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.
The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here:
