For years, security training taught employees to notice awkward grammar, strange greetings and obvious spelling mistakes. Those signals were never reliable, and generative AI makes them weaker still.
The FBI has warned that criminals are using generative AI to create convincing text and other synthetic content in fraud and social-engineering schemes. Business email compromise (BEC) remains dangerous because the target is not the inbox itself. The target is a trusted business process: a payment, bank-detail change, credential reset or disclosure of sensitive information.
The durable defence is therefore verification, not literary analysis.
Treat BEC as a transaction-control problem
An employee can receive a perfectly written email from a real compromised account. No amount of grammar awareness will identify that reliably.
Instead, define which requests require independent verification because of what they could cause. Typical triggers include:
- new or changed bank details;
- urgent or unusual payments;
- requests to bypass normal approval;
- payroll or beneficiary changes;
- release of confidential customer or employee data;
- password or MFA resets; and
- executive requests involving secrecy or exceptional speed.
The Certified AI Email Threat Hunter develops specialist capability for analysing AI-enabled email threats, but the business process must also make a successful deception hard to monetise.
Use the “leave the channel” rule
If an email asks for a high-risk change, do not verify it by replying to the same email thread.
Use a known phone number, approved supplier portal, previously recorded contact or separate authenticated channel. Do not use a phone number supplied in the suspicious message.
This simple rule breaks an attacker’s control of the conversation. It also works whether the fraudulent message was written by a person, generated by AI or sent from a compromised mailbox.
Build a five-step verification playbook
Step 1: classify the requested action
Ask what would change if the request were genuine: money, credentials, access, personal data, supplier master data or contractual information. The consequence determines the verification strength.
Step 2: verify identity independently
Contact the requester through a trusted route already on record. For supplier bank changes, verify with an authorised supplier contact rather than the contact details in the change request.
Step 3: verify the transaction, not only the person
Even a real executive can have a compromised account. Confirm the amount, beneficiary, reason and exception being requested. A familiar voice or name should not override the control.
Step 4: enforce dual approval for material changes
Separate requester, verifier and approver where risk warrants it. Configure finance and identity systems so an email cannot silently bypass the control.
Step 5: preserve evidence and escalate anomalies
If the request fails verification, retain the message and relevant headers, alert the security team and protect affected accounts quickly. Do not continue engaging with the attacker merely to “see what happens”.
Add AI-era signals without relying on them
AI-assisted fraud may still leave contextual clues: an unusual request, a change in payment pattern, new contact route, false urgency or a story designed to defeat normal controls.
The FBI has also highlighted malicious use of AI-generated content in impersonation schemes. Voice or video therefore should not be treated as infallible proof of identity. For highly sensitive transactions, use pre-agreed verification processes rather than improvised questions during a suspicious call.
Strengthen the mail environment
Process controls should sit alongside technical controls. CISA phishing guidance emphasises layered defences across email security, identity and user reporting.
Practical measures include strong MFA, secure email gateways, domain protections such as SPF/DKIM/DMARC, conditional access, rapid reporting, restrictions on auto-forwarding and monitoring for suspicious mailbox rules.
These foundations are relevant to teams developing broader capability through the Certified AI-Powered Cybersecurity Foundations course.
Test the process with realistic scenarios
Run exercises that measure behaviour rather than clicks.
Example one: a long-standing supplier sends a professional request to change bank details two days before a large invoice is due. Does the employee use the approved supplier contact and record the verification?
Example two: a senior executive appears to request a confidential acquisition document while travelling. Does hierarchy defeat the data-release control?
Example three: a helpdesk receives an urgent identity-reset request supported by a convincing voice call. Does the agent follow the identity-proofing process?
Score the control on whether the employee noticed the trigger, used the independent channel, completed the correct approval and escalated the anomaly.
Measure what predicts resilience
Click rate alone is a weak measure. Track controls that influence loss prevention:
- percentage of high-risk changes independently verified;
- attempted exceptions to dual approval;
- time from suspicious request to reporting;
- percentage of supplier bank changes using the approved verification path;
- time to disable a compromised account; and
- recurrence of the same process weakness.
A Certified AI Cyber Risk Assessor perspective can help connect these operational measures to wider enterprise risk rather than treating BEC as an isolated awareness problem.
A two-minute rule for employees
When a message asks you to move money, change access or disclose sensitive information, pause and ask:
- Is this action high impact or unusual?
- Am I being asked to bypass a normal control?
- Can I verify the requester through a trusted route I already have?
- Does a second person need to approve the change?
- If it is fraudulent, who needs to know immediately?
That rule remains useful even as AI-generated messages become harder to distinguish from genuine ones.
Final takeaway
AI changes the polish and scale of social engineering, but it does not change the core objective of BEC: make a trusted employee perform an attacker-controlled action.
Design the business process so a persuasive message is insufficient. Independent verification, controlled approvals, strong identity security and fast escalation are more durable than trying to guess whether an email “sounds like AI”.
Responses