• Verify by process, not by senses. Voices, faces, and polish can be faked. Anything involving money, passwords, MFA codes, or “keep this quiet” gets confirmed through a separate channel you choose yourself, never one the request supplies.
• Apply the Rule of Two. Don’t let one AI assistant read untrusted content, see private data, and take outside actions all at once.
• Report weirdness fast. Odd requests, possible impersonation, confidential data in an AI output, or an assistant doing something you didn’t ask: tell your security team early. Fast reporting is how Fideuram got €53 million back.

Previously, Daniel’s AI assistant nearly took orders from invisible text hidden in a vendor proposal, a reminder that outside content is information, not authority. This time, we kick off a three-part story on how AI is upgrading old cons, from cloned voices to tricked assistants, and the habits that keep you a step ahead. Picture a busy Tuesday afternoon. A message arrives that appears to come from a senior executive. The wording is polished, the signature looks right, and an attached voice note sounds exactly like the person you work with. They need an urgent payment made before a deadline.

Nothing about it is obviously ridiculous. That’s the problem.

Now a second scenario. Your organization’s AI assistant is allowed to read certain documents and perform simple tasks. An attacker tucks instructions inside a routine-looking file, hoping the assistant will obey them and hand over information it was never meant to share.

The first scenario is AI-assisted social engineering: manipulating people (rather than breaking software) into handing over money or access, now with better scripts and cloned voices. The second is prompt injection: hiding hostile instructions inside material an AI system is supposed to treat as something to read, not something to obey. Neither requires an evil supercomputer plotting humanity’s downfall. Both have already happened to real organizations.

Take Fideuram, the private-banking arm of Intesa Sanpaolo, Italy’s largest bank. In February 2026, its then-chairman received a WhatsApp message that appeared to come from the parent company’s CEO, asking for urgent help with a confidential overseas deal. Then came a phone call from a senior partner at a prominent law firm, confirming the plan. The lawyer’s voice had been cloned with AI. Roughly €95 million left the bank, mostly for accounts in China and Hong Kong. The story only became public in late September.

The clever part wasn’t the voice. It was the paperwork. The scammers supplied a stack of official-looking documents, including a confidentiality agreement, and the people approving the transfers didn’t check with their own leadership because the agreement told them not to. The criminals didn’t just fake a boss. They faked a reason not to call one.

Now the part worth cheering. Fideuram spotted the irregularities quickly and raised the alarm with banks and authorities in several countries, and about €53 million came back. One story, two habits: secrecy sent €95 million out the door, and fast reporting hauled more than half of it back.

The same old con, in a better wardrobe

Humans tend to absorb big technological change in a loop: encounter → understand → experiment → integrate → reinvent. Academics have given pieces of this loop impressive names (Diffusion of Innovations, the Technology Acceptance Model, Domestication Theory), but anyone who has watched an office adopt video calls already knows the pattern.

With AI, everyone is in a different phase at once. Some people are still poking at a chatbot. Others use it daily. Software teams are building entire systems around it. And criminals have skipped straight to experimenting at scale, because they have no procurement process, no legal review, and no committee that meets every other Thursday. Innovation is not a members-only club for nice people.

Security professionals call this dual-use: the same capability can help or harm, depending on whose hands it’s in. A defender can use AI to summarize thousands of technical alerts and spot the one that matters. An attacker can use the same language skills to write a message tailored to an employee’s role, recent projects, and fear of missing a deadline. The technology doesn’t pick a side. People and systems do.

The numbers have started to show up. In 2025, for the first time in the nearly 25-year history of its annual Internet Crime Report, the FBI broke out a dedicated section on AI: 22,364 complaints that referenced it, with about $893 million in reported losses. That counts only cases where victims noticed and said AI was involved, so read it as a floor, not a ceiling. IBM’s 2026 Cost of a Data Breach Report, which surveyed 602 organizations that suffered incidents, found AI-driven attacks up 56% in a year and adding roughly $1 million to the average breach, which reached a record $4.99 million.

Is it all doom? The International AI Safety Report, a February 2026 assessment by more than 100 experts chaired by Turing Award winner Yoshua Bengio, offered a calibrated answer. Criminals are actively using AI in attacks, but at the time the report judged fully autonomous attacks still limited, because AI systems could not reliably carry out long, multi-stage attack sequences without humans. Keep that sentence in your back pocket. Part Two puts it to the test.

Three changes worth your attention

1. The message can be flawless and still be fake

For years, the cheapest phishing detector was the typo. Phishing means fraudulent messages designed to trick you into clicking, paying, or sharing something. The classics announced themselves with mangled grammar and a prince with a liquidity problem. AI sands off those tells. Correct grammar, a believable corporate tone, translation into any language, and dozens of tailored variations now cost an attacker almost nothing.

Then there’s the media. A deepfake is synthetic or manipulated audio, video, or imagery that convincingly imitates a real person. Pair one with business email compromise (BEC), a fraud in which criminals impersonate a colleague or vendor to redirect a payment, and you get a category that drew about $3 billion in reported losses to the FBI in 2025. (Not every BEC case involved AI, but AI is quietly improving the pitch.)

Notice what made the Fideuram scheme work. Nobody fell for a voice alone. The “confirmation” came from a second trusted person, and that second person was part of the script. A callback only protects you if you pick the number. And don’t bet on your ears: a 2024 review of 56 studies put human accuracy at spotting deepfakes at about 55%, barely better than a coin toss. Confidence doesn’t help, either. In an October survey of Singapore office workers by security-training firm KnowBe4, senior managers were the group most sure they could spot a fake and the group most likely to obey a cloned CEO voice before checking.

So skip the guessing game. Confirm anything involving money or secrecy through a separate, trusted channel that you choose yourself: the number in the company directory, not the one in the message, and not the “lawyer” the message introduces. Agree in advance on a code phrase for sensitive requests such as payment changes. The tools are catching up, too. Starting in mid-November, Microsoft Teams will let organizations plug in certified services that flag suspected deepfake audio and video during meetings. Treat that as a smoke detector, not the fire department.

Field test: When a request combines money, secrecy, and urgency, treat the combination itself as the red flag. A confidentiality agreement never outranks a phone call to someone you already know.

2. An assistant can be tricked by what it reads

Modern chatbots and assistants run on large language models (LLMs): software trained on enormous amounts of text to generate responses by predicting which words should come next. They’re remarkable at following instructions written in plain English. They’re also not great at telling instructions from you apart from instructions buried in something they’re merely reading. Imagine a receptionist who obeys any sticky note found in the mail. That’s prompt injection.

The best-known early proof was EchoLeak, a flaw researchers found in Microsoft 365 Copilot in 2025. A single crafted email was enough to trigger data theft with no action required from the victim, earning a severity score of 9.3 out of 10 and an official vulnerability ID, CVE-2025-32711. (A zero-click attack needs nothing from the victim. A CVE is a public catalog number for a known security flaw.)

It has also moved from the lab to the wild, sometimes through a door nobody thinks about. In October, security firm Forever Security traced an alert on a customer’s computer to a planted text file sitting in the Downloads folder. The file was written for AI coding assistants, which are designed to read and obey instruction files in whatever folder they’re working in, and its instructions said to install a malicious software package. The firm’s software stopped it, and its researchers then showed the same trick could push assistants from both OpenAI and Anthropic into installing malware or handing over a login token. Why Downloads? Because it’s one of the few places a website can drop a file onto your computer without asking. The AI wasn’t hacked. It did exactly what it was built to do: read the note and follow it.

So what do you do about a weakness nobody has fully fixed? Design around it. Meta’s security team, building on a concept developer Simon Willison calls the “lethal trifecta,” proposed the Agents Rule of Two: until researchers can reliably detect and refuse prompt injection, an AI agent should have no more than two of three powers in a single session. (A) It can read untrusted content such as emails, web pages, and uploads. (B) It can access private data or sensitive systems. (C) It can take actions in the world, like sending messages or changing files. Hold all three and a poisoned web page can walk off with your data. Hold two and the attacker is missing a leg of the stool.

Think of it as a household rule. Your assistant can open mail from strangers, or read your bank statements, or send wires on your behalf. Pick any two.

Field test: Before connecting an AI tool to your inbox, ask which two of the three powers it already has. If the answer is “all of them,” that’s not a setup step. That’s a conversation with your security team.

3. The AI invents things, and criminals read the inventory

Most of us know AI sometimes hallucinates: it produces confident, fluent, wrong answers, including citations to papers that don’t exist. In cybersecurity there’s a nastier version. Software developers build programs from prefabricated components called packages, installed with a single command from public libraries. When researchers had 16 popular AI coding models generate 576,000 code samples, about 19.7% of the packages the models recommended did not exist, yielding more than 205,000 unique invented names. Open source models invented them far more often than commercial ones (roughly 22% versus 5%), and many of the fake names came back whenever the same question was repeated.

Repeatable mistakes are exploitable. An attacker can register one of those invented names, fill it with malware, and wait for the next AI-assisted developer (or automated build system) to install it. The attack is called slopsquatting. The AI invents the address; the criminal builds the house.

You don’t have to write code to take the lesson: confident and correct are different things, and in security the gap between them is where attackers live.

Field test: If someone on your team ships AI-assisted code, “does this package actually exist, and who maintains it?” is a perfectly reasonable thing to ask.

Back to that Tuesday afternoon

You ignore the flawless delivery of the voice note, look up the executive’s number in the company directory, and call. It takes ninety seconds. The real executive has no idea what you’re talking about, which is exactly the right answer. Meanwhile, that routine-looking file with the hidden instructions gets read by an assistant that can’t act on them, because nobody gave it the power to. Nothing about either outcome required spotting the fake. It required a habit.

So far, we’ve mostly met AI that talks and reads: cloned voices, polished messages, assistants that believe whatever they’re handed.

In Part Two, AI starts doing things. We’ll meet an agent that wouldn’t take no for an answer, a state-backed campaign that borrowed a trusted vendor’s keys, the truth behind the AI that “blackmailed” an engineer, and where all of this is heading, including the genuinely good news.

The takeaway: Use AI where it helps. Question it where it matters. Don’t outsource your judgment, your passwords, or your responsibility to a system simply because it can finish your sentences.