The greatest financial risk created by generative AI may not be that machines can make decisions. It is that machines can make people believe they are dealing with someone they trust.
Executive summary
A convincing voice note, a familiar face on a video call, or a message that appears to come from a chief executive used to provide useful clues about identity. Those clues are becoming easier to manufacture. Generative AI can produce realistic synthetic voices, images, video and identity documents at a scale that changes the economics of impersonation.
The issue is no longer simply whether a deepfake can fool an individual. It is whether organisations have designed payment, onboarding and authorisation systems that remain safe when visual and audio evidence can be forged.
The evidence is already material. The FBI's 2025 Internet Crime Report (opens in a new tab) recorded 22,364 complaints reporting AI-related information and adjusted losses exceeding US$893 million. These figures describe complaints with an AI nexus, not losses proven to have been caused exclusively by deepfakes. Separately, Hong Kong authorities described (opens in a new tab) a 2024 case in which an employee authorised transfers totalling about HK$200 million after attending a fabricated video conference. Malaysia's Securities Commission (opens in a new tab) and Bank Negara Malaysia (opens in a new tab) have also warned about deepfake investment promotions and impersonation of senior officials.
The response must therefore be systemic: independent identity verification, transaction controls, trusted communication channels, provenance signals, human accountability and rapid incident response. No single detector, biometric check or blockchain system can solve the problem alone.
1. The day seeing is no longer believing
Imagine receiving a video call from your chief financial officer. The face is familiar. The voice sounds right. The person explains that a confidential acquisition payment must be completed before a deadline. A colleague appears to confirm the instruction. The meeting ends. The transfer is made.
The question is not whether the video looked convincing. The question is whether the payment process required anything beyond a convincing video.
That distinction matters. Deepfake fraud is best understood as an attack on trust and authorisation, not merely as an image-manipulation problem. Fraudsters exploit the social meaning attached to a face, a voice, a title, a logo or an apparently official channel. They then use urgency, secrecy and authority to pressure a victim into bypassing normal controls.
The technology lowers the cost of producing convincing impersonations. The organisation's own processes determine whether that impersonation can become a financial loss.
View the image full size
2. What makes a deepfake possible?
A deepfake is synthetic or manipulated audio, image or video that resembles a real person or event closely enough to create a false impression of authenticity. Generative models learn patterns from examples and produce new material that imitates those patterns.
Three capabilities are particularly relevant:
- Voice cloning: a model imitates vocal characteristics, cadence and accent from audio samples. The result may be used in a phone call, voice note or live interaction.
- Face and video synthesis: a system can alter a face, generate a talking-head video or manipulate lip movement and expressions.
- Synthetic identity material: image-generation and editing tools can assist in creating fabricated profile pictures, identity documents or supporting material.
These capabilities are not equally easy to deploy. A short generated clip, a pre-recorded video played during a meeting and a convincing real-time interactive impersonation are different technical challenges. A report should not treat every suspicious video as proof of a real-time deepfake.
The FBI has warned (opens in a new tab) that generative AI can produce convincing profiles, fraudulent identification documents, cloned voices and videos used in impersonation and investment fraud. It also notes that AI can make scams more scalable and remove errors that might otherwise alert a target.
3. The economics of impersonation have changed
Traditional impersonation often required time, language skills, access to a target's information and repeated human effort. Generative AI can reduce the effort required to produce tailored messages and synthetic media. It can also help a fraudster adapt tone and content to different targets.
The FBI's 2025 Internet Crime Report (opens in a new tab) provides a useful indicator of the scale of AI-associated cybercrime reported in the United States:
| 2025 measure | Reported figure | How to interpret it |
|---|---|---|
| Complaints reporting AI-related information | 22,364 | Complaint count, not confirmed deepfake cases |
| Adjusted losses associated with those complaints | US$893.3 million | Broad AI nexus; not exclusively deepfake losses |
| AI-nexus investment complaint losses | More than US$632 million | Investment complaints where an AI connection was reported |
| AI-involved employment scam losses | Almost US$13 million | A distinct fraud category described in the report |
These figures should not be added together: the categories may overlap or use different classification methods. They are US complaint data, not a global estimate and not a measure of all AI-enabled fraud. Under-reporting, uncertainty about whether AI was involved and changes in reporting practices all affect the totals.
View the image full size
The key lesson is not that AI explains every scam. It is that AI can increase the speed, personalisation and apparent credibility of fraud across multiple categories.
4. When the fake meeting becomes a real loss
One of the clearest examples came from Hong Kong. In a government response published on 26 June 2024 (opens in a new tab), authorities described a case in which an employee received a phishing email impersonating the UK-based chief financial officer of the employee's head office. The employee joined a video conference involving apparent colleagues and authorised transfers to five local bank accounts. The reported loss was about HK$200 million.
Investigators believed the conference had been pre-recorded using publicly available video clips and the impersonated officer's voice. The official account said there was no actual interaction between the employee and the fraudster during that conference. The case was under investigation when the government response was published.
The detail is important: the attack did not need to defeat a bank's cryptography. It needed to persuade a human with payment authority to approve transfers. A second Hong Kong case described in the same government response involved an employee being instructed to transfer nearly HK$4 million after a video conference with a purported senior executive.
These cases demonstrate why organisations must not treat a video call as a payment credential. A meeting can support a conversation; it cannot independently authenticate a high-value instruction.
5. Malaysia: the threat is already local
Malaysia's regulators have publicly addressed impersonation and deepfake investment promotions. On 22 July 2024, the Securities Commission Malaysia warned (opens in a new tab) that scammers were using AI-generated videos to mimic prominent personalities and reputable companies, then circulating the material through social media and messaging applications to promote purported investment opportunities.
The warning is not theoretical. On 13 October 2026, Bank Negara Malaysia issued a scam alert (opens in a new tab) about deepfake videos impersonating senior BNM officers, covering investment content purportedly from senior officers of BNM or the SC. It reminded the public that BNM and the SC do not promote or recommend private investment schemes.
Bank Negara Malaysia's dedicated verification resource (opens in a new tab) also warns about fake advertisements, profiles and messages misusing the central bank's identity. The SC's Investor Alert List (opens in a new tab) identifies unauthorised entities and potential clone entities. These tools are important, but a list can only reflect information available to the regulator at a particular time; absence from a list is not proof that an offer is legitimate.
For Malaysian readers, a practical rule is straightforward: never treat a familiar face, a government logo, a regulator's name or a WhatsApp group as proof that an investment is authorised. Verify the entity and the offer independently through official channels.
6. Why traditional KYC and biometrics are not enough on their own
Know-your-customer (KYC) systems are designed to establish who a customer is and assess relevant risks. Document checks, facial comparison, liveness tests, device intelligence and sanctions screening can all contribute. But each control answers a different question.
A document may be genuine while the person presenting it is not its rightful holder. A face may match a reference image while the interaction is being manipulated. A liveness check may resist known presentation attacks but still face new forms of injection, replay or synthetic-media attacks. A successful onboarding check also does not prove that a later payment instruction is legitimate.
A stronger design separates at least four questions:
- Identity: Who is the person, and how was that identity established?
- Presence: Is the person genuinely present in this interaction, rather than replayed or injected media?
- Authority: Is this person permitted to request this action?
- Intent: Has the person knowingly approved this exact transaction, to this beneficiary, for this amount?
The final two questions are often neglected. A real executive can be impersonated, but a real executive can also be socially engineered. Authentication is not the same as authorisation, and authorisation is not the same as informed intent.
7. The next generation of financial controls
A resilient system assumes that one signal can be compromised. It combines independent controls so that a convincing voice or video cannot override the entire process.
For corporate payments
- Require dual approval for material transfers and changes to beneficiary details.
- Verify new or changed instructions using a pre-registered number or secure workflow, not the contact details supplied in the request.
- Apply cooling-off periods, amount limits and enhanced review for unusual payments.
- Separate the person requesting a payment from the person releasing it.
- Record approvals, device context and changes to payment instructions.
View the image full size
For banks, exchanges and fintechs
- Combine document verification with risk-based liveness, device and behavioural signals.
- Detect repeated identities, reused documents, unusual device clusters and suspicious onboarding patterns.
- Apply step-up verification to account recovery, withdrawal-address changes and high-risk transactions.
- Use transaction monitoring and beneficiary-risk controls after onboarding.
- Provide clear escalation paths for suspected impersonation.
For individuals
- Call the person back using a number already saved or independently verified.
- Agree on a family or team passphrase for urgent requests, while recognising that passphrases themselves can be exposed.
- Never approve a transfer solely because a familiar person appears on video.
- Be suspicious of secrecy, urgency, unusual payment routes and requests to bypass normal process.
- Verify investment offers with the relevant regulator and the named institution.
8. Can blockchain and cryptography restore trust?
Blockchain can help establish that a record has not been altered after it was written, and cryptographic signatures can prove that a particular key signed a particular message. These are valuable properties. They do not automatically prove that the human behind the key was genuine, that the key was not compromised, or that the signer understood the transaction.
A signed fraudulent instruction is still a fraudulent instruction.
The useful role for cryptography is narrower and more precise:
- Digital signatures can bind an approval to the exact transaction details.
- Hardware-backed keys can reduce exposure of signing credentials.
- Verifiable credentials can allow an issuer to make a cryptographically verifiable claim about a credential holder.
- Tamper-evident audit trails can preserve the sequence of approvals and changes.
- Content provenance standards can carry information about the origin and editing history of media, where supported.
These technologies improve evidence and control. They do not eliminate deception, coercion, compromised devices or dishonest issuers. Provenance metadata can also be stripped during distribution, and an authentic recording can still be used out of context.
The goal should not be to find a technology that makes trust automatic. It should be to make trust claims verifiable, scoped and revocable.
9. Detection is useful, but it is not a security boundary
Deepfake detectors can identify patterns associated with generated or manipulated media. Yet detection performance can vary across generators, compression levels, languages, recording conditions and attack methods. A detector may produce false positives and false negatives. An attacker can also change the media after a detector has been trained.
For that reason, detection should be treated as one risk signal, not as a final verdict. A “not detected” result does not establish authenticity. A “likely synthetic” result should trigger review, not automatically determine guilt or liability.
Organisations should test systems against their own threat models, monitor performance over time and preserve a human escalation route. High-impact decisions should not depend on a single opaque model score.
10. Regulation and accountability
The European Union's AI Act includes transparency duties for certain AI-generated or manipulated content. Article 50 (opens in a new tab) addresses machine-readable marking by providers and disclosure by deployers in specified circumstances, including deepfakes. The European Commission's implementation guidance (opens in a new tab) states that Article 50 applies from 2 August 2026. Providers of systems placed on the market before that date have until 2 December 2026 to meet the machine-readable marking obligation.
This is a transparency framework, not a guarantee that every deepfake will be detected or prevented. It also does not replace existing laws on fraud, privacy, consumer protection, financial services or data protection.
In Malaysia, regulators are already warning about deepfake impersonation and unauthorised investment promotions. The challenge for businesses is to translate those warnings into operational controls: clear payment authority, reliable identity proofing, incident reporting, evidence retention and staff training.
Accountability should follow control. AI providers should build appropriate safeguards and transparency into products; platforms should respond to impersonation and fraudulent promotion; financial institutions should maintain proportionate controls; and organisations should ensure that employees are not forced to make high-risk decisions based on a video call alone.
11. A practical trust framework for the AI era
A board-level deepfake response should cover five layers:
| Layer | Core question | Example control |
|---|---|---|
| Identity | Who is this? | Verified identity and credential checks |
| Channel | Is this the approved channel? | Pre-registered contact and secure portal |
| Authority | Can this person approve it? | Role-based permissions and dual control |
| Transaction | What exactly is being approved? | Signed transaction details and beneficiary checks |
| Evidence | Can the decision be reconstructed? | Tamper-evident logs and incident records |
The layers must be independent enough that compromising one does not collapse the others. A voice call should not be able to change a beneficiary, approve a transfer and bypass the audit trail in one step.
View the image full size
12. What to do when a deepfake incident is suspected
Speed matters. Contact the bank or relevant platform immediately, request that pending transactions be stopped where possible, preserve messages and original media, record account details and timestamps, and report the incident to law enforcement and the relevant regulator.
In Malaysia, victims should contact their bank immediately and call the National Scam Response Centre at 997 (opens in a new tab) as soon as possible. The government guidance says the centre handles online financial scam cases and advises reporting within 24 hours to support action on bank accounts and transactions. A police report should also be made. Do not pay a so-called recovery agent who promises to retrieve funds for an upfront fee.
Conclusion: trust must become a system, not a feeling
For generations, people have used faces, voices, signatures and familiar institutions as shortcuts for trust. AI does not make those signals worthless, but it makes them insufficient on their own.
The answer is not to stop using video calls, biometrics or AI. It is to stop allowing any single signal to carry more authority than it deserves. Financial trust must be supported by independent verification, clear permissions, transaction-specific approval, traceable evidence and a culture that rewards employees for pausing when something feels wrong.
The next generation of financial infrastructure will not be defined only by faster payments or smarter automation. It will be defined by whether it can distinguish identity from appearance, authority from persuasion, and proof from performance.
When seeing is no longer believing, trust has to be engineered.
Frequently asked questions
Can AI deepfakes fool banks? They can be used to attack customers, employees and identity-verification processes. The risk depends on the bank's controls. A video or voice match should not be the sole basis for a high-risk transaction.
Are deepfake detectors reliable? They can help identify suspicious media, but performance varies and no detector should be treated as definitive proof of authenticity or fraud.
Can blockchain prevent deepfake scams? No. Blockchain can support tamper-evident records and cryptographic approvals, but it cannot by itself prove that a person is genuine or that a transaction was authorised freely and knowingly.
How can a company prevent fake CEO payment instructions? Use dual approval, pre-registered out-of-band verification, beneficiary-change controls, transaction limits and audit logs. Never rely solely on a video call or email.
What should I do if I receive a suspected deepfake investment video in Malaysia? Do not transfer money or share personal information. Verify the offer through BNM or the Securities Commission's official channels, check the SC Investor Alert List, and report suspected fraud. If money has been transferred, contact your bank and NSRC 997 immediately.
Sources and Further Reading
Federal Bureau of Investigation, Internet Crime Complaint Center, "2025 Internet Crime Report", 2026. The 22,364 complaints reporting AI-related information, adjusted losses above US$893 million, AI-nexus investment losses above US$632 million and almost US$13 million in AI-involved employment scam losses. Read the report (opens in a new tab).
Federal Bureau of Investigation, "Criminals Use Generative Artificial Intelligence to Facilitate Financial Fraud", public service announcement I-120324-PSA, 3 December 2024. How criminals use generative AI for fake profiles, identity documents, cloned voices and video impersonation. Read the announcement (opens in a new tab).
Hong Kong Government, "LCQ9: Combating frauds involving deepfake", 26 June 2024. The HK$200 million and nearly HK$4 million video-conference cases, as described in the Secretary for Security's written reply. Read the reply (opens in a new tab).
Securities Commission Malaysia, "SC Warns Public on Deepfake Investment Scams Impersonating Prominent Personalities and Using Names of Reputable Companies", 22 July 2024. Deepfake investment videos spread on social media and messaging apps. Read the release (opens in a new tab).
Bank Negara Malaysia, "Scam Alert: Beware of Deepfake Videos Impersonating BNM Senior Officers", 13 October 2026. Deepfake videos impersonating senior officers, and the reminder that BNM and the SC do not promote or recommend private investment schemes. Read the alert (opens in a new tab).
Bank Negara Malaysia, "Deepfakes and Impersonations". How scammers misuse BNM's identity in ads, profiles and messages, and how to verify. Read the guidance (opens in a new tab).
Securities Commission Malaysia, "Investor Alert List". Unauthorised entities and potential clones; the list is not exhaustive. See the list (opens in a new tab).
European Commission, AI Act Service Desk, "Article 50: Transparency obligations for providers and deployers of certain AI systems". Machine-readable marking of synthetic content and disclosure of deepfakes. Read the article (opens in a new tab).
European Commission, "Transparency obligations under Article 50 of the AI Act", last updated 24 July 2026. Application from 2 August 2026, and the 2 December 2026 date for the marking obligation for systems already on the market. Read the FAQ (opens in a new tab).
Government of Malaysia, "Hotline NSRC 997". The National Scam Response Centre, its round-the-clock hotline and the advice to call within 24 hours of discovering a scam. Read the page (opens in a new tab).
Editorial note
This article is for general information and education, not legal, investment, cybersecurity or financial advice. Statistics reflect the scope and definitions of the cited sources and are not a complete measure of global deepfake fraud. Regulatory requirements may change; consult the relevant authorities and qualified professionals.
The FBI figures are US complaints in which AI-related information was reported, not confirmed deepfake cases. The Hong Kong cases are described as the Hong Kong Government reported them while they were under investigation.
Sources last checked on 6 November 2026.
