The risky world of malicious AI

AI is changing how businesses work. It enhances productivity, accelerates decision-making, and automates repetitive tasks. But it also introduces new risks that most organisations are not fully prepared for.

This blog breaks down how AI is being used maliciously, the real-world threats that result, and what you can do to stay prepared.

The business benefits of AI

AI adoption is no longer experimental. It is a fundamental part of how many businesses operate today. From frontline support to strategic planning, AI is helping organisations solve problems faster and with fewer resources.

Here is where it is making a real difference:

• 40% productivity gains are being reported by companies applying AI across operations, customer service, and marketing

• Faster analysis of complex data sets helps teams make better decisions

• Automated processes reduce manual effort and improve accuracy

The benefits are clear. But there is a catch. These same capabilities are now in the hands of attackers.

Figure 1: AI is already integrated into daily operations, revolutionising how businesses automate processes and make decisions.

Problem: Attackers are using AI to bypass traditional defences

While businesses are using AI to increase efficiency, attackers are using the same tools to increase the success of their scams. As these capabilities evolve, so does the risk profile for every organisation.

Here's how.

How AI attacks work

• Phishing at scale: Language models mimic tone and phrasing to craft personalised emails that bypass spam filters

• Deepfakes in real-world use: Audio and video impersonations are being used in fraud and social engineering

• Malicious LLMs on the dark web: Tools like WormGPT and FraudGPT are now available to help criminals generate malicious code, phishing templates and fake identities

• AI-enhanced malware: Earlier proofs like IBM’s DeepLocker paved the way for adaptive, target-specific malware

These are no longer theoretical. They are active tools being used to exploit trust, confusion and speed.

Why AI-powered attacks are harder to stop

AI enables threat actors to:

• Act quickly: Thousands of messages or scripts can be generated and launched in minutes

• Customise content: Emails or attacks look credible because they are tailored to your business and your people

• Change tactics: AI lets attackers adapt their approach in real time to avoid detection

Cybercrime is expected to cost over US$10 trillion globally.

Case in point: The Deepfake risk

One of the most disruptive uses of malicious AI is the creation of synthetic audio and video. Deepfakes are already being used to manipulate real-world decisions, making them a serious concern for business leaders.

They are being used to:

• Replicate an executive’s voice in phone calls

• Deliver fraudulent instructions in video format

• Create false confirmations that lead to fund transfers or data access

The visual and audio realism make them difficult to detect and easy to believe.

Solution: A Practical response to AI-enabled threats

These aren’t hypothetical risks. Organisations need to treat AI-enabled attacks as a present and growing challenge. That means shifting from passive defences to proactive, layered responses.

1. Raise awareness across the business

Attackers are counting on ignorance. Counter that with training:

• • • Run simulated phishing and vishing exercises

    • Include AI-specific threat examples in your training programs

2. Lock down the basics

Strong access controls are still essential. Many breaches happen because of preventable gaps:

• • • Enforce multi-factor authentication (MFA)

    • Adopt the ASD Essential Eight controls

    • Use anomaly detection tools to flag impersonation and odd behaviour

3. Monitor for unusual behaviour

AI threats are subtle. That is why detection must go beyond basic rules:

• • Deploy Managed Detection and Response (MDR) to catch outliers

    • Look for shifts in data usage, login timing, and access locations

4. Validate your readiness

Do not assume your current defences will hold up. Test them:

• • • Commission penetration tests and red team simulations that include AI-enabled techniques

    • Explore Deepfake detection tools and threat emulation exercises

Proof: AI is both the risk and the solution

Despite the risks, AI is also improving cyber defence. The same technologies that help attackers adapt can also help defenders anticipate and respond faster.

Many detection systems now use AI to:

• Flag anomalies that humans would miss

• Correlate subtle patterns across systems

• Predict emerging attack paths before they are used

Organisations that treat AI as part of both the problem and the solution will be better positioned to respond.

Next steps

Knowing the risks is one thing. Acting on them is another. Here are practical next steps to help you build AI-aware defences.

• Understand how AI changes the threat landscape

• Align your controls with today’s risks, not yesterday’s

• Test, refine and improve continuously

Not sure where your blind spots are? Talk to our team about AI-aware threat assessments.

If you liked this article, you may also like:

The top 3 cloud security challenges

The best practices of Administrative Privilege Management

Incremental vs differential backup: which one is right for your company?